Groundwater pollution source tracking method, equipment and storage medium based on machine learning

Abstract

The present invention relates to the field of pollution source tracking, and discloses a method, device and storage medium for tracking groundwater pollution sources based on machine learning. The method includes the following steps: obtaining groundwater historical data and environmental data; extracting environmental features from environmental data, and extracting pollutant features from historical data; configuring a fusion model for groundwater pollution tracking; the fusion model is obtained by fusing multiple machine learning models; obtaining the correlation between environmental features and pollutant features, and obtaining weight factors based on the correlation, and using the weight factors as additional features input to the fusion model; deploying the fusion model, and completing pollution source tracking using the fusion model. The present invention introduces the mutual fusion of machine learning algorithms and supervised learning algorithms, and substitutes the improved multi-dimensional empirical formula, realizes efficient identification and tracking of pollution sources through autonomous model learning, introduces edge computing, performs real-time data processing and analysis, reduces data transmission delays, and improves the real-time nature of identification.

Description

Technical Field

[0001] The present invention relates to the field of pollution source tracking, and in particular to a method, device and storage medium for tracking groundwater pollution sources based on machine learning. Background Art

[0002] Groundwater pollution sources include agricultural pollution sources, industrial pollution sources, domestic pollution sources, natural pollution sources and other pollution sources; agricultural pollution sources refer to the fertilizers and pesticides used in large quantities in agricultural production, which will enter the groundwater through infiltration and runoff, causing nitrate, phosphate and pesticide residue pollution; industrial pollution sources refer to harmful chemicals such as heavy metals and volatile organic compounds produced in the production process of chemical plants, which will seep into the groundwater; domestic pollution sources refer to untreated or improperly treated domestic sewage entering the groundwater through infiltration or leakage, which will cause pollution by bacteria, viruses and organic pollutants; natural pollution sources refer to groundwater in some areas that contains high concentrations of natural pollutants, such as arsenic and fluorine, which come from the natural dissolution of underground rocks; other pollution sources include but are not limited to traffic accidents or leakage of harmful substances during transportation, which may seep into the groundwater;

[0003] Conventional methods for tracing groundwater pollution sources mainly rely on the calculation of empirical formulas, which are based on historical data under specific conditions and the knowledge of field experts. However, the expressive power of empirical formulas is limited and it is difficult to adapt to complex groundwater pollution environments. When actually tracing and identifying pollution sources, empirical formulas have obvious deficiencies in dealing with nonlinear relationships, multivariate coupling effects and dynamic changes. When actually put into use, it is difficult to efficiently and accurately identify and track pollution sources. In addition, empirical formulas are highly dependent on data. When data quality is not high or data is missing, the accuracy of their calculation results will also be relatively affected. Summary of the invention

[0004] The purpose of the present invention is to propose a method, device and storage medium for tracing groundwater pollution sources based on machine learning, so as to solve the technical problem that traditional groundwater pollution source tracing relies on empirical formulas, resulting in inaccurate final results.

[0005] The present invention provides a method for tracing groundwater pollution sources based on machine learning, comprising the following steps:

[0006] S1. Data collection: Obtaining historical groundwater data and environmental data;

[0007] S2. Feature extraction: extract environmental features from environmental data and pollutant features from historical data;

[0008] S3. Model construction: configuring a fusion model for groundwater pollution tracking; the fusion model is obtained by fusing multiple machine learning models;

[0009] S4, weight calculation: obtain the correlation between environmental characteristics and pollutant characteristics, and obtain weight factors based on the correlation, and use the weight factors as additional features input into the fusion model;

[0010] S5. Pollutant tracking: Deploy the fusion model and use it to track pollution sources.

[0011] A storage medium stores instructions and data for implementing a method for tracing groundwater pollution sources based on machine learning.

[0012] A groundwater pollution source tracking device based on machine learning includes: a processor and the storage medium; the processor loads and executes instructions and data in the storage medium to implement a groundwater pollution source tracking method based on machine learning.

[0013] The beneficial effects provided by the present invention are: overcoming the limitations of conventional methods based on empirical formula judgment, introducing the integration of machine learning algorithms and supervised learning algorithms, and substituting the improved multi-dimensional empirical formula as an additional input feature into the integrated model, and realizing efficient judgment and tracking of pollution sources through autonomous learning of the model. In addition, edge computing technology is introduced in the data acquisition terminal to perform real-time data processing and preliminary analysis, reduce data transmission delays, and improve the real-time performance of judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic flow chart of the method of the present invention;

[0015] Figure 2 It is a working schematic diagram of the hardware device of the present invention. DETAILED DESCRIPTION

[0016] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0017] Before formally describing the present invention, the scheme of the present invention is first generally described for easy understanding.

[0018] Please refer to Figure 1 , Figure 1 It is a schematic flow diagram of the method of the present invention;

[0019] The present invention provides a method for tracing groundwater pollution sources based on machine learning, comprising the following steps:

[0020] S1. Data collection: Obtaining historical groundwater data and environmental data;

[0021] It should be noted that step S1 is specifically as follows:

[0022] S11. Collect historical groundwater data and environmental data;

[0023] As an embodiment, historical data is collected, wherein the historical data includes pollutant types, pollutant concentrations, geographical locations, occurrence times and durations of pollution events, pollution sources, and information on affected groundwater wells or water sources.

[0024] Collect environmental data, including groundwater flow rate and direction, soil type and permeability, geological structure, interaction between surface water and groundwater, precipitation and seasonal changes, and distribution of industrial areas.

[0025] S12, preprocessing the collected data; the preprocessing includes: removing missing values, processing outliers and standardizing.

[0026] As an embodiment, removing missing values ​​specifically means directly deleting records containing missing values ​​for data with a missing value ratio of less than 5%;

[0027] Specifically, processing outliers means that for numerical outlier data, the original outliers are removed and filled with the mean of the feature; for categorical data, the original outliers are removed and filled with the mode of the feature;

[0028] Standardization specifically refers to standardizing the data using the Min-Max standardization method and scaling the data to the [0, 1] interval.

[0029] S2. Feature extraction: extract environmental features from environmental data and pollutant features from historical data;

[0030] As an embodiment, the types and processes of environmental feature extraction in the present invention are as follows:

[0031] Groundwater velocity and direction: obtain groundwater velocity and direction data through hydrological monitoring stations and field monitoring equipment; feature calculation: calculate the average velocity, flow direction vector and velocity change rate as features;

[0032] Soil type and permeability, soil type and permeability data were obtained through soil sampling and experimental analysis, and soil type was converted into numerical features using one-hot encoding;

[0033] Geological structure: geological structure data is obtained through geological surveys and geophysical exploration, and rock layer types and fracture density are extracted as features;

[0034] Precipitation and seasonal changes, precipitation data is obtained based on the meteorological public platform, and the monthly average precipitation and seasonal changes are calculated as features.

[0035] The types and contents of pollutant feature extraction in the present invention are as follows:

[0036] Pollutant concentration: pollutant concentration data at different locations and times are obtained from water quality monitoring data, and the average concentration and concentration change are calculated as features;

[0037] Pollutant types, by recording the types of pollutants involved in the pollution incident, using one-hot encoding to convert the pollutant types into numerical features;

[0038] Pollutant diffusion coefficient: The diffusion coefficient of each pollutant is obtained from the public experimental data platform and added to the data set as a feature;

[0039] Pollutant reactivity, the reactivity of pollutants is obtained through an open chemical experiment data platform, and the half-life and degradation rate in the reactivity are used as characteristics.

[0040] S3. Model construction: configuring a fusion model for groundwater pollution tracking; the fusion model is obtained by fusing multiple machine learning models;

[0041] It should be noted that step S3 is specifically as follows:

[0042] S31, using random forest as a machine learning algorithm and integrating it with support vector machine;

[0043] Among them, random forest makes predictions through multiple decision trees, specifically:

[0044] ,

[0045] In the formula, is the prediction result of random forest, is the number of decision trees, For the i The prediction results of a decision tree.

[0046] Among them, the core formula of the support vector machine is: f ( x )= sign ( w · x + b ).

[0047] In the formula, w is the weight vector, x is the input feature vector, b For bias.

[0048] S32, load the data set, separate the features and labels, and prepare for model training;

[0049] S33, dividing the data into a training set and a test set; in the present invention, the ratio of the training set to the test set is 78:22.

[0050] S34, using the training set to train the random forest model and simultaneously train the support vector machine;

[0051] The specific parameters of the random forest model include the number of decision trees, which is set to 138, and the support vector machine uses probability estimation. Both the random forest model and the support vector machine use random seeds to ensure repeatable results.

[0052] S35, the result of using weighted average fusion random forest model and support vector machine model, specifically:

[0053] P final = w rf · P rf + w svm · P svm ,

[0054] In the formula, P final is the final prediction probability after fusion, P rf is the predicted probability of the random forest, P svm is the predicted probability of the support vector machine, w rf and w svm is the weight of the corresponding model, and w rf + w svm =1;

[0055] After the optimization process and evaluation results are obtained, the optimal weight state is found based on the cross-validation method.

[0056] The cross validation is as follows:

[0057] ,

[0058] In the formula, yes k The average error of the cross-validation, k It is the discount number. It is i Folding error.

[0059] S36. Use the test set to make predictions and evaluate model performance.

[0060] S4, weight calculation: obtain the correlation between environmental characteristics and pollutant characteristics, and obtain weight factors based on the correlation, and use the weight factors as additional features input into the fusion model;

[0061] It should be noted that step S4 is specifically as follows:

[0062] S41. Obtain standardized environmental and pollutant characteristics, conduct correlation analysis, and determine preliminary weights;

[0063] Among them, the correlation analysis was performed by combining the Pearson correlation coefficient and the Spearman rank correlation coefficient, specifically:

[0064] Calculate the Pearson correlation coefficients between environmental characteristics and pollutant characteristics and pollutant concentrations to determine preliminary weights;

[0065] Among them, the Pearson correlation coefficient is calculated by the following formula:

[0066] ,

[0067] In the formula, r is the Pearson correlation coefficient, and The two variables are i Observations, and are the means of the two variables, n is the sample size;

[0068] The Spearman rank correlation coefficients between environmental characteristics and pollutant characteristics and the target variables were calculated as a supplementary analysis;

[0069] Among them, the Spearman rank correlation coefficient is calculated by the following formula:

[0070] ,

[0071] In the formula, is the Spearman rank correlation coefficient, d i For the i The level difference of samples, n is the sample size.

[0072] Combining the results of Pearson correlation coefficient and Spearman rank correlation coefficient, the preliminary weight of each feature was determined by average weighting method;

[0073] Among them, let the feature matrix be The target variable is , here, Represents the pollutant concentration, first calculate each feature and The Pearson correlation coefficient is:

[0074] ,

[0075] In the formula, For the j The feature of i Observations, For the j The mean of the features.

[0076] The Spearman rank correlation coefficients between environmental characteristics and pollutant characteristics and the target variable were calculated as follows:

[0077] ,

[0078] In the formula, and Respectively i The characteristics of the observations X j and the target variable y level.

[0079] The results of the Pearson correlation coefficient and the Spearman rank correlation coefficient were combined to determine preliminary weights;

[0080] By combining the results of the Pearson correlation coefficient and the Spearman rank correlation coefficient, the average weighting method was used to determine the preliminary weight of each feature, specifically:

[0081] ,

[0082] In the formula, For the j The initial weights of the features;

[0083] S42, calculating the importance score of each feature based on the random forest model configured in S3, and using it as a reference value of the weight factor;

[0084] S43, substituting each parameter, setting the formula and calculating, specifically:

[0085] ,

[0086] In the formula, Indicates location and time t The pollutant concentration, Indicates i The weight of the pollutants, C i0 Indicates i The initial concentration of the pollutant, It is iThe attenuation coefficient of the pollutant is is the influence function of geographical location, taking into account the influence of geological structure, is the groundwater flow velocity x and y The influence function in the direction, n is the number of pollutant types;

[0087] Among them, the pollutant concentration is calculated and used as an additional feature and input into the fusion model for training and prediction.

[0088] As an example, there are two pollutants, namely A and B , then the specific calculation content is:

[0089] Initial concentration ;

[0090] Attenuation coefficient ;

[0091] Geographical location influence function ;

[0092] Groundwater flow velocity influence function ;

[0093] Weight Factor ;

[0094] time ;

[0095] Calculated at location and time The pollutant concentrations are:

[0096]

[0097] The calculation results are as follows:

[0098] .

[0099] In the formula, the final specific value is 10.2546.

[0100] Among them, the time variable in the above formula is t Dynamics reflects the change of pollutant concentration over time through an exponential decay function To simulate the natural attenuation process of pollutants over time, more accurately predict the concentration changes of pollutants at different time points.

[0101] For the spatial location feature level, the geographical location affects the function and groundwater flow velocity influence function , the model can simulate the diffusion of pollutants in different geographical locations, reflect spatial heterogeneity, take into account the influence of geological structure and groundwater flow, and accurately predict the diffusion path and concentration distribution of pollutants in groundwater.

[0102] The formula helps realize a highly accurate and widely applicable method for tracing groundwater pollution sources by integrating multi-dimensional features and combining dynamic changes in time and space. The model's multi-factor combination and weight adjustment mechanism enable it to cope with complex pollution scenarios and accurately predict pollutant concentrations and track pollution sources.

[0103] S5. Pollutant tracking: Deploy the fusion model and use it to track pollution sources.

[0104] Deploy the model and obtain tracking results, including:

[0105] S51, saving the model, based on the model trained in S300, saving it as a file state;

[0106] S52. Introduce edge computing technology, deploy lightweight models based on edge computing devices, and realize real-time data collection, preprocessing and preliminary analysis at the edge devices. The edge computing device is specifically Raspberry Pi 4.

[0107] In the embodiment of the present application, historical data and environmental data are collected, and environmental features and pollutants are extracted based on the obtained data. Random forest and support vector machine are integrated to build a fusion model, and weight factors are calculated synchronously. The calculated results are input into the fusion model as additional features, and finally the model is deployed and the tracking results are obtained. This method overcomes the limitations of conventional methods based on empirical formula discrimination, introduces the integration of machine learning algorithms and supervised learning algorithms, and substitutes the improved multi-dimensional empirical formula as an additional input feature into the fused model, and realizes efficient discrimination and tracking of pollution sources through model autonomous learning. In addition, edge computing technology is introduced in the data acquisition terminal to perform real-time data processing and preliminary analysis, reduce data transmission delays, and improve the real-time nature of discrimination; similarly, edge devices collect and process data in real time on site, and transmit preliminary analysis results to the central server for in-depth analysis and result integration, thereby realizing dynamic tracking and real-time monitoring of pollution sources.

[0108] See also Figure 2 , Figure 2 It is a working diagram of the hardware device of an embodiment of the present invention, and the hardware device specifically includes: a groundwater pollution source tracking device 401 based on machine learning, a processor 402 and a storage medium 403.

[0109] A groundwater pollution source tracking device 401 based on machine learning: The groundwater pollution source tracking device 401 based on machine learning implements the groundwater pollution source tracking method based on machine learning.

[0110] Processor 402: The processor 402 loads and executes the instructions and data in the storage medium 403 to implement the groundwater pollution source tracking method based on machine learning.

[0111] Storage medium 403: The storage medium 403 stores instructions and data; the storage medium 403 is used to implement the method for tracing groundwater pollution sources based on machine learning.

[0112] The beneficial effects of the present invention are: overcoming the limitations of conventional methods based on empirical formula judgment, introducing the integration of machine learning algorithms and supervised learning algorithms, and substituting the improved multi-dimensional empirical formula as an additional input feature into the integrated model, and realizing efficient judgment and tracking of pollution sources through autonomous learning of the model. In addition, edge computing technology is introduced in the data acquisition terminal to perform real-time data processing and preliminary analysis, reduce data transmission delays, and improve the real-time performance of judgment.

[0113] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for tracing groundwater pollution sources based on machine learning, characterized in that: The method comprises the following steps: S1. Data collection: Obtaining historical groundwater data and environmental data; S2. Feature extraction: extract environmental features from environmental data and pollutant features from historical data; Step S2 is specifically as follows: S21. Obtain groundwater flow rate and direction, soil type and permeability, geological structure, precipitation and seasonal changes from environmental data as environmental characteristics; S22. Obtain pollutant concentration, pollutant type, pollutant diffusion coefficient, and pollutant reactivity from historical groundwater data as pollutant characteristics; S23, integrating the environmental characteristics and pollutant characteristics into a data set; S3. Model construction: configuring a fusion model for groundwater pollution tracking; the fusion model is obtained by fusing multiple machine learning models; Step S3 is as follows: S31, using random forest as a machine learning algorithm and integrating it with support vector machine; S32, load the data set, separate the features and labels, and prepare for model training; S33, dividing the data into a training set and a test set; S34, using the training set to train the random forest model and simultaneously train the support vector machine; S35, the result of using weighted average fusion random forest model and support vector machine model, specifically: Pfinal = wfr · Prf + wsvm · Psvm ， In the formula, Pfinal is the final prediction probability after fusion, Prf is the predicted probability of the random forest, Psvm is the predicted probability of the support vector machine, wfr and wsvm is the weight of the corresponding model, and wfr + wsvm =1; S36, use the test set to make predictions and evaluate the model performance; S4, weight calculation: obtain the correlation between environmental characteristics and pollutant characteristics, and obtain weight factors based on the correlation, and use the weight factors as additional features input into the fusion model; Step S4 specifically includes: S41. Obtain standardized environmental and pollutant characteristics, conduct correlation analysis, and determine preliminary weights; Among them, the correlation analysis was performed by combining the Pearson correlation coefficient and the Spearman rank correlation coefficient, specifically: Calculate the Pearson correlation coefficients between environmental characteristics and pollutant characteristics and pollutant concentrations to determine preliminary weights; Among them, the Pearson correlation coefficient is calculated by the following formula: ， In the formula, r is the Pearson correlation coefficient, and The two variables are i Observations, and are the means of the two variables, n is the sample size; The Spearman rank correlation coefficients between environmental characteristics and pollutant characteristics and the target variables were calculated as a supplementary analysis; Among them, the Spearman rank correlation coefficient is calculated by the following formula: ， In the formula, is the Spearman rank correlation coefficient, d i For the i The level difference of samples, n is the sample size; Combining the results of Pearson correlation coefficient and Spearman rank correlation coefficient, the preliminary weight of each feature was determined by average weighting method; Among them, let the feature matrix be X The target variable is y , here, y Represents the pollutant concentration, first calculate each feature and y The Pearson correlation coefficient is: ， In the formula, For the j The feature of i Observations, For the j The mean of the features; The Spearman rank correlation coefficients between environmental characteristics and pollutant characteristics and the target variable were calculated as follows: ， In the formula, and Respectively i The characteristics of the observations X j and the target variable y level; The results of the Pearson correlation coefficient and the Spearman rank correlation coefficient were combined to determine preliminary weights; By combining the results of the Pearson correlation coefficient and the Spearman rank correlation coefficient, the average weighting method was used to determine the preliminary weight of each feature, specifically: ， In the formula, For the j The initial weights of the features; S42, calculating the importance score of each feature based on the random forest model configured in S3, and using it as a reference value of the weight factor; S43, substituting each parameter, setting the formula and calculating, specifically: ， In the formula, Indicates location and time t The pollutant concentration, Indicates i The weight of the pollutants, Ci 0 means i The initial concentration of the pollutant, It is i The attenuation coefficient of the pollutant is is the influence function of geographical location, taking into account the influence of geological structure, is the groundwater flow velocity x and y The influence function in the direction, n is the number of pollutant types; Among them, the pollutant concentration is calculated and used as an additional feature to be input into the fusion model for training and prediction; S5. Pollutant tracking: deploy the fusion model and use it to track pollution sources; Step S5 is specifically as follows: save the trained model; introduce edge computing technology, deploy lightweight models based on edge computing devices, realize real-time data collection, preprocessing and preliminary analysis at the edge devices, and complete pollution source tracking.

2. The method for tracing groundwater pollution sources based on machine learning according to claim 1, characterized in that: Step S1 is specifically as follows: S11. Collect historical groundwater data and environmental data; S12, preprocessing the collected data; the preprocessing includes: removing missing values, processing outliers and standardizing.

3. A storage medium, characterized in that: The storage medium stores instructions and data for implementing a method for tracing groundwater pollution sources based on machine learning as described in any one of claims 1 to 2.

4. A groundwater pollution source tracking device based on machine learning, characterized in that: include: A processor and a storage medium; the processor loads and executes instructions and data in the storage medium to implement a method for tracing groundwater pollution sources based on machine learning as described in any one of claims 1 to 2.

Images