Intelligent water quality prediction method based on rank correlation and XGBoost model

Abstract

The invention relates to the technical field of intelligent industrial park management, and discloses an intelligent water quality prediction method based on rank correlation and an XGBoost model, and the method comprises the steps: a data collection step, collecting the multi-dimensional data of an industrial park, and carrying out the preprocessing of the data; in the correlation analysis step, a Spearman rank correlation coefficient is adopted to analyze the correlation between the multi-dimensional data and the inflow water quality; the key characteristic variable screening step is used for screening multi-dimensional data of which the correlation meets a preset standard as key characteristic variables based on the analysis result of the correlation; an intelligent prediction model construction step: carrying out normalization processing on the key characteristic variables, and constructing an inflow water quality intelligent prediction model based on an XGBoost method; in the prediction step, real-time prediction is carried out based on the inflow water quality intelligent prediction model. By analyzing the multi-dimensional correlation data of the industrial park sewage treatment plant and establishing the intelligent prediction model, accurate and timely prediction of the inlet water quality of the industrial park sewage treatment plant is realized, the sewage treatment efficiency is improved, and the operation cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of smart industrial park management, and in particular to a water quality intelligent prediction method based on rank correlation and XGBoost model. Background Art

[0002] With the deep integration of industrialization and informatization, smart industrial parks have become an important carrier for promoting industrial transformation and upgrading. Smart industrial parks optimize resource allocation, improve operational efficiency, reduce environmental pollution, and promote sustainable development by integrating advanced information technology, the Internet of Things, big data analysis, and artificial intelligence. As a key facility to ensure the environmental quality of the industrial park, the intelligent prediction and management of the influent water quality of the industrial park sewage treatment plant is particularly important.

[0003] There are many enterprises in the industrial park, covering multiple industries such as chemical, electronics, and textiles. The sewage pollution and discharge patterns are complex and changeable, which brings great challenges to the efficient and stable operation of sewage treatment plants. Sewage treatment is a dynamic system. Traditional water quality monitoring methods often have lags and are difficult to effectively respond to sudden changes in water quality in real time, affecting the effluent water quality compliance rate and the ecological environment quality of the park. The influent water quality of sewage treatment plants is affected by many factors, including weather, industrial production, population, and changes in economic activities. There are complex multi-dimensional correlations between these influencing factors and water quality, and traditional methods are difficult to accurately capture these relationships and make effective predictions. With the rapid development of big data technology and the widespread application of artificial intelligence algorithms, it is possible to establish an accurate influent water quality prediction model by deeply mining multi-dimensional correlation data, providing strong support for the stable operation and efficient management of sewage treatment plants. Summary of the invention

[0004] The present invention aims to provide a method for intelligent water quality prediction based on rank correlation and XGBoost model. By collecting and analyzing multi-dimensional correlation data of industrial park sewage treatment plants, an intelligent prediction model is established according to the correlation between multi-dimensional data and influent pollutants, so as to achieve accurate and timely prediction of influent water quality of industrial park sewage treatment plants, improve sewage treatment efficiency and reduce operating costs.

[0005] In order to achieve the above object, the present invention adopts the following technical scheme:

[0006] The water quality intelligent prediction method based on rank correlation and XGBoost model includes:

[0007] Data collection step: collect multi-dimensional data of industrial parks and pre-process the data;

[0008] In the correlation analysis step, the Spearman rank correlation coefficient was used to analyze the correlation between the multi-dimensional data and the influent water quality;

[0009] The key feature variable screening step is to screen the multi-dimensional data whose correlation meets the preset standards as the key feature variables based on the correlation analysis results;

[0010] The intelligent prediction model construction step is to normalize the key characteristic variables and build an intelligent prediction model for influent water quality based on the XGBoost method;

[0011] Prediction step: real-time prediction based on the intelligent prediction model of influent water quality.

[0012] The principles and advantages of this solution are: in actual application, multi-dimensional data of the industrial park are collected, and the correlation between the correlation data and the influent water quality is judged by the Spearman rank correlation coefficient. Based on this, high-correlation data is selected as input features to construct a distributed gradient enhancement reservoir influent water quality prediction model, thereby deeply mining multi-dimensional correlation data, improving prediction accuracy, and realizing accurate forward-looking prediction of the influent water quality of the sewage treatment plant, providing a scientific basis for environmental management and sewage treatment optimization of smart industrial parks.

[0013] Beneficial effects:

[0014] Improve prediction accuracy: Through multi-dimensional correlation data analysis and comprehensive consideration of multiple influencing factors, the accuracy of influent water quality prediction has been significantly improved.

[0015] Enhanced decision support: Provide forward-looking water quality prediction information for smart industrial parks, supporting managers to make more scientific and reasonable sewage treatment decisions.

[0016] Optimize resource allocation: Dynamically adjust the sewage treatment process according to the prediction results to effectively avoid resource waste and reduce operating costs.

[0017] Promote green development: It helps to achieve precise management of sewage treatment, reduce pollutant emissions, and promote the smart industrial park to move towards green, low-carbon and sustainable development.

[0018] Preferably, as an improvement, the correlation analysis step includes:

[0019] In the sorting sub-step, the multi-dimensional data and the influent water quality are aligned and expanded according to the time sequence, and the ranks are obtained respectively;

[0020] A rank difference obtaining sub-step, obtaining the rank difference between each pair of multi-dimensional data and the influent water quality in the observation value;

[0021] The rank correlation coefficient calculation sub-step runs a preset rank correlation coefficient calculation model based on the rank difference to calculate the rank correlation coefficient.

[0022] Technical effect: It is easy to accurately obtain the correlation between multi-dimensional data and influent water quality.

[0023] Preferably, as an improvement, the rank correlation coefficient calculation model is:

[0024]

[0025] in, is the rank correlation coefficient, d ij is the rank difference, and m is the number of observations.

[0026] Technical effect: It is easy to quantify rank correlation.

[0027] Preferably, as an improvement, in the sorting sub-step, when there are equal observation values, the rank is the average value of the corresponding sorting positions.

[0028] Technical effect: Avoid the situation where the rank cannot be obtained when the observations are equal.

[0029] Preferably, as an improvement, in the sorting sub-step, when there are equal observations, the rank correlation coefficient calculation model is:

[0030]

[0031] in, is the rank correlation coefficient, T x and T y are the number of observations of the stalemate level in the variable, m is the number of observations, d ij is the rank difference.

[0032] Technical effect: It is convenient to quantify the rank correlation when the observation values ​​are equal.

[0033] Preferably, as an improvement, the correlation includes positive correlation, negative correlation and no correlation. When the rank correlation coefficient is close to 1, it indicates a strong positive correlation. When the rank correlation coefficient is close to -1, it indicates a strong negative correlation. When the rank correlation coefficient is close to 0, it indicates a weak correlation.

[0034] Technical effect: It is easy to accurately obtain key characteristic variables.

[0035] Preferably, as an improvement, in the intelligent prediction model construction step, the objective function of the influent water quality intelligent prediction model based on the XGBoost method is:

[0036]

[0037] Among them, y i is the true value of a certain influent water quality, For the model based on the input (x1, x2, ... x M)(M≤m), T represents the number of leaf nodes in a tree, λ is the adjustment parameter of this item, ω represents the vector composed of the output values ​​of the leaf nodes, and β is the regularization parameter that controls the complexity of the output values ​​of the leaf nodes.

[0038] Technical effect: It is easy to improve the accuracy, computational efficiency, robustness and interpretability of influent water quality prediction.

[0039] Preferably, as an improvement, it also includes a model verification and training step, based on the real-time prediction results of the influent water quality intelligent prediction model, verify the model prediction accuracy and adjust the sewage treatment process parameters.

[0040] Technical effect: It is easy to ensure that the effluent water quality is stable and meets the standards, while reducing energy consumption and operating costs.

[0041] Preferably, as an improvement, an integration and optimization step is also included to integrate the intelligent prediction model of influent water quality into the management system of the industrial park sewage treatment plant.

[0042] Technical effect: It is easy to realize online optimization of model parameters to meet the needs of high-accuracy real-time monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flowchart of the water quality intelligent prediction method based on rank correlation and XGBoost model;

[0044] Figure 2 Schematic diagram for multidimensional correlation data analysis and model building. DETAILED DESCRIPTION

[0045] The following is further described in detail through specific implementation methods:

[0046] The embodiment is basically as shown in the attached Figure 1 , Attachment Figure 2 As shown:

[0047] The water quality intelligent prediction method based on rank correlation and XGBoost model includes:

[0048] Data collection steps: collect multi-dimensional data of industrial parks and pre-process the data; multi-dimensional data of industrial parks include but are not limited to historical influent water quality monitoring data of sewage treatment plants (such as COD, BOD, ammonia nitrogen, total phosphorus, total nitrogen, etc.), economic data (such as industrial production index, park GDP), enterprise production activity data (such as production volume, raw material usage), park population data, meteorological data (such as temperature, humidity, rainfall, etc.). Pre-processing technologies such as data cleaning, missing value filling, and outlier detection are used to pre-process the multi-dimensional data of industrial parks to ensure data quality.

[0049] Correlation analysis steps: Spearman rank correlation coefficient is used to analyze multi-dimensional data (x1, x2, ... x m ) and the inlet water quality (y1,y2,…,y n ). Specifically, the sorting sub-step aligns the multi-dimensional data and the influent water quality according to the time sequence, and obtains the ranks respectively; and They are the rankings of multi-dimensional data and influent water quality, i.e., ranks. ij is the rank difference of a pair of observations, that is, and The difference, is the rank of multi-dimensional data, is the rank of the influent water quality. The rank and rank difference are shown in Table 1.

[0050] Table 1

[0051]

[0052]

[0053] The rank correlation coefficient calculation model is:

[0054]

[0055] in, is the rank correlation coefficient, d ij is the rank difference, and m is the number of observations.

[0056] When there are equal observations, as shown in Table 2, the observations The rank of two corresponding observations is the average of the corresponding sorts.

[0057] Table 2

[0058]

[0059] When there are equal observations, the rank correlation coefficient calculation model is:

[0060]

[0061] in, is the rank correlation coefficient, T x and T y are the number of observations of the stalemate level in the variable, m is, d ij is the rank difference.

[0062] The value range of is between -1 and 1. When it is close to 1, it indicates that there is a strong positive correlation between the two variables. When it is close to -1, it indicates that there is a strong negative correlation between the two variables. When it is close to 0, it means that there is almost no correlation between the two variables. The values ​​are divided into different categories, as shown in Table 3.

[0063] Table 3

[0064]

[0065] In the key feature variable screening step, based on the correlation analysis results, multi-dimensional data with correlations that meet preset standards are screened as key feature variables. Specifically, according to the multi-dimensional data x i (i=1,2,…,m) and the water quality y j The absolute value of the rank correlation coefficient between (j=1,2,…,n) Selection and inlet water quality j (j=1,2,…,n) Multi-dimensional data with moderate correlation or above, that is, key characteristic variables that have a significant impact on the changes in influent water quality.

[0066] The intelligent prediction model construction step is to normalize the key feature variables and build an intelligent prediction model for influent water quality based on the XGBoost method; the intelligent prediction model for influent water quality created by the XGBoost method has the characteristics of accuracy, high computational efficiency, robustness, and strong interpretability. Specifically,

[0067] Assume that the multi-dimensional correlation data combination is (x1, x2, ... x M )(M≤m), establish a certain influent water quality y j (j=1,2,…,n) intelligent prediction model XGBoost, XGBoost is a multi-learner simultaneous learning model, which is:

[0068]

[0069] Among them, y i is the true value of a certain influent water quality, For the model based on the input (x1, x2, ... x M )(M≤m) gives the predicted value, is the loss function of all learners, and the mean square error is used for regression prediction problems. is the complexity of K learners, T represents the number of leaf nodes in a tree, λ is the adjustment parameter of this item, ω represents the vector composed of leaf node output values, and β is the regularization parameter that controls the complexity of leaf node output values.

[0070] The data set is divided into training set, validation set and test set in a ratio of 7:2:1. The model is trained with the training set data, and the model parameters are adjusted with the validation set to ensure the prediction accuracy of the model.

[0071] It also includes model verification and training steps, applying the constructed intelligent prediction model for influent water quality to actual data, solving the optimal output value of each leaf node and the optimal tree structure by minimizing the objective function, and verifying its prediction accuracy based on the prediction results, adjusting the sewage treatment process parameters in time, optimizing resource allocation, ensuring that the effluent water quality is stable and meets the standards, while reducing energy consumption and operating costs.

[0072] It also includes integration and optimization steps, integrating the intelligent prediction model of influent water quality into the management system of the industrial park sewage treatment plant, realizing automatic data collection, intelligent prediction, result display and early warning functions, and providing managers with an intuitive and convenient operation interface. According to the difference between the measured data at the inlet end of the sewage treatment plant and the predicted data of the intelligent prediction model, the model parameters are optimized online to achieve online optimization of the model parameters to meet the needs of high-accuracy real-time monitoring.

[0073] The above is only an embodiment of the present invention, and the common knowledge such as the known specific technical solutions and / or characteristics in the solution is not described in detail here. It should be pointed out that for those skilled in the art, without departing from the technical solution of the present invention, several modifications and improvements can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. The water quality intelligent prediction method based on rank correlation and XGBoost model is characterized by: include: Data collection step: collect multi-dimensional data of industrial parks and pre-process the data; In the correlation analysis step, the Spearman rank correlation coefficient was used to analyze the correlation between the multi-dimensional data and the influent water quality; The key feature variable screening step is to screen the multi-dimensional data whose correlation meets the preset standards as the key feature variables based on the correlation analysis results; The intelligent prediction model construction step is to normalize the key characteristic variables and build an intelligent prediction model for influent water quality based on the XGBoost method; Prediction step: real-time prediction based on the intelligent prediction model of influent water quality.

2. The water quality intelligent prediction method based on rank correlation and XGBoost model according to claim 1 is characterized in that: The correlation analysis step comprises: In the sorting sub-step, the multi-dimensional data and the influent water quality are aligned and expanded according to the time sequence, and the ranks are obtained respectively; A rank difference obtaining sub-step, obtaining the rank difference between each pair of multi-dimensional data and the influent water quality in the observation value; The rank correlation coefficient calculation sub-step runs a preset rank correlation coefficient calculation model based on the rank difference to calculate the rank correlation coefficient.

3. The water quality intelligent prediction method based on rank correlation and XGBoost model according to claim 2 is characterized in that: The rank correlation coefficient calculation model is: in, is the rank correlation coefficient, d ij is the rank difference, and m is the number of observations.

4. The water quality intelligent prediction method based on rank correlation and XGBoost model according to claim 2 is characterized in that: In the sorting sub-step, when there are equal observation values, the rank is the average value of the corresponding sorting positions.

5. The water quality intelligent prediction method based on rank correlation and XGBoost model according to claim 2 is characterized in that: In the sorting sub-step, when there are equal observations, the rank correlation coefficient calculation model is: in, is the rank correlation coefficient, T x and T y are the number of observations of the stalemate level in the variable, m is the number of observations, d ij is the rank difference.

6. The water quality intelligent prediction method based on rank correlation and XGBoost model according to claim 1 is characterized in that: The correlation includes positive correlation, negative correlation and no correlation. When the rank correlation coefficient is close to 1, it indicates a strong positive correlation. When the rank correlation coefficient is close to -1, it indicates a strong negative correlation. When the rank correlation coefficient is close to 0, it indicates a weak correlation.

7. The water quality intelligent prediction method based on rank correlation and XGBoost model according to claim 1 is characterized in that: In the intelligent prediction model construction step, the objective function of the influent water quality intelligent prediction model based on the XGBoost method is: Among them, y i is the true value of a certain influent water quality, For the model based on the input (x1, x2, ... x M )(M≤m), T represents the number of leaf nodes in a tree, λ is the adjustment parameter of this item, ω represents the vector composed of the output values ​​of the leaf nodes, and β is the regularization parameter that controls the complexity of the output values ​​of the leaf nodes.

8. The water quality intelligent prediction method based on rank correlation and XGBoost model according to claim 1 is characterized in that: It also includes model verification and training steps. Based on the real-time prediction results of the influent water quality intelligent prediction model, the model prediction accuracy is verified and the sewage treatment process parameters are adjusted.

9. The water quality intelligent prediction method based on rank correlation and XGBoost model according to claim 1 is characterized in that: It also includes integration and optimization steps to integrate the intelligent prediction model of influent water quality into the management system of the industrial park sewage treatment plant.

Images