A method for predicting new neonicotinoid pesticide pollution in an inland water body in a watershed based on sub-basins and machine learning models

Abstract

The application discloses a kind of based on sub-basin and machine learning model's prediction method of new neonicotinoid insecticide pollution in watershed surface water body, belong to surface water pollution prediction field;Including the following steps, step one, determine the scope of sub-basin, the coordinate range of each sub-basin is regarded as its identity mark ID;Step two, data acquisition and pretreatment;Step three, establish data set, construct machine learning model and model tuning;Step four, realize the prediction of sub-basin surface water neonicotinoid insecticide pollution;Step five, the visualization of result.The method of the application realizes the synchronous fast and accurate prediction of multiple sub-basins under national scale, provides a new method reference for the forecast monitoring of the field of surface water neonicotinoid insecticide pollution;Meanwhile, integrate six common neonicotinoid insecticides, realize the synchronous prediction of a model to multiple pollutants;Supplement social and economic factors, fully capture pollution influencing factors.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of water environment monitoring and pollution prediction, and particularly relates to a prediction method for neonicotinoid pesticide pollution in surface water in a basin based on sub-basins and a machine learning model. BACKGROUND

[0002] Neonicotinoids have high efficiency and broad-spectrum insecticidal properties, but these compounds have high water solubility and environmental persistence, and are easily transported into the surface water system through surface runoff, leaching and other pathways, posing a potential threat to aquatic organisms and the ecological environment.

[0003] At present, the prediction of pesticide pollution at the basin scale mainly relies on traditional monitoring methods and empirical models:

[0004] 1. Fixed-point monitoring method: by setting limited monitoring points in the basin for sampling and analysis, but it is difficult to cover the entire basin space, and there are monitoring blind spots;

[0005] 2. Physical model method: such as SWAT, HSPF, etc. Hydrological model, which needs to input a large number of parameters (such as terrain, soil, weather, etc.), the modeling process is complex and requires high data, and cannot be used for prediction at the same time in multiple sub-basins and other large-scale perspectives;

[0006] 3. Statistical regression model: ignores the complexity of the basin hydrological process, and is difficult to accurately reflect the pollution migration and transformation rule.

[0007] The main defects of the prior art are that it cannot achieve high-precision spatial distribution prediction, the model has insufficient generalization ability and is difficult to adapt to different basin characteristics, and lacks a dynamic updating mechanism, which cannot reflect the impact of environmental changes on pollution in a timely manner. For example, Chinese patent CN202210338319.X discloses a "method for analyzing and predicting large-scale water quality space-time rules in a basin", but it only considers conventional water quality indicators (such as COD, ammonia nitrogen), does not model the characteristics of neonicotinoid pesticides, and the model lacks generalization ability and cannot be used for prediction in multiple sub-basins at the national scale. SUMMARY

[0008] In view of the deficiencies of the prior art, the present application aims to provide a prediction method for neonicotinoid pesticide pollution in surface water in a basin based on sub-basins and a machine learning model, which realizes the prediction of pollution spatial distribution in multiple sub-basins at a large scale through multi-source data fusion modeling.

[0009] The core technical solution of the present application includes the following steps:

[0010] S1, sub-basin boundary determination: based on the sub-basin division data (resolution ≥ 12 levels) of the HydroBASINS database, the boundary of each sub-basin in the study area is determined.

[0011] S2, obtain concentration data samples in each sub-basin; collect lake or river water samples, and record: sampling time (accurate to the hour); latitude and longitude coordinates (WGS84 coordinate system, accuracy to 6 decimal places); water type (river, lake, reservoir, etc.); record the detection limit of the sample.

[0012] S3, construct a spatial coordinate system, take the coordinate range of each sub-basin as its identity ID, and obtain the modeling basis data of each sub-basin in the study area, and perform multi-source data fusion modeling. Specifically, three types of basic data need to be obtained:

[0013] S31, soil and terrain data: quantitative characteristic data and qualitative classification data of surface (0-30cm) and deep (30-100cm) soil.

[0014] The quantitative characteristic data of the soil includes: sand content, clay content, organic carbon content, silt content, soil bulk density, cation exchange capacity, soil pH, basic saturation, exchangeable salt ion content, electrical conductivity, exchangeable sodium salt content, and average sub-basin elevation.

[0015] The qualitative classification of the soil includes soil texture classification and hydrological group.

[0016] S32, pollution monitoring data: concentration values (μg / L) of 6 neonicotinoid insecticides (imidacloprid, thiamethoxam, acetamiprid, dinotefuran, clothianidin, and thiacloprid), including the time nodes of concentration monitoring of these types of insecticides.

[0017] S33, quantitative characteristic data of the environment and human activities: water flow intensity index, convergence accumulation, pesticide use in sub-basins, cultivated land area, drought index, population density, total irradiance at the optimal tilt angle, horizontal total irradiance, normal total irradiance, horizontal scattered irradiance, potential photovoltaic power generation, precipitation, and air temperature.

[0018] S4, subsequent preprocessing of the collected data:

[0019] S41, convert discrete data such as soil texture classification and hydrological group into numerical data through one-hot encoding;

[0020] The soil texture classification is classified according to the soil texture classification standard developed by the USDA Natural Resources Conservation Service, and a total of 13 textures are classified, which are assigned as discrete numerical data through one-hot encoding, for example, the soil texture classification of 13 textures is assigned as individual codes 1-13.

[0021] Hydrological group is divided into four grades according to the permeability of water in soil, and the four grades of texture are high permeability, medium permeability, slow permeability and extremely low permeability. Through one-hot encoding, they are respectively assigned as discrete numerical data, for example, the hydrological group of the above four textures is respectively assigned as single codes of 1, 2, 3 and 4.

[0022] S42, convert the multiple insecticide concentrations into imidacloprid equivalent concentration according to the relative efficacy factor method, and the calculation formula is: IMI RPF Imidacloprid equivalent concentration, C i The concentration of a certain neonicotinoid insecticide, RPF i The relative efficacy factor of a certain neonicotinoid insecticide (the relative efficacy factors are respectively: imidacloprid = 1, thiamethoxam = 9.5, acetamiprid = 0.803, dinotefuran = 2.850, clothianidin = 5.816, thiacloprid = 14.25);

[0023] S43, in the data preprocessing process of the quantitative characteristic data and qualitative classification data of soil and terrain data, and the quantitative characteristic data of environment and human activities, the average value of continuous quantitative factors is calculated in the sub-basin unit, and the mode of discrete classification factors is counted. The specific way is:

[0024] ① If the data obtained for model operation is continuous quantitative data, calculate the average value of the quantitative data in the minimum time span of the sub-basin as the smallest statistical unit, and use the calculated average value as the characteristic value of the quantitative data for constructing the data set. Among them, the quantitative characteristic data of soil and the quantitative characteristic data of environment and human activities are all continuous quantitative factor data.

[0025] ② If the data obtained for model operation is discrete qualitative classification data, count the mode of the qualitative classification parameter in the sub-basin as the smallest statistical unit, and use the counted mode as the characteristic value of the qualitative classification data for constructing the data set. Among them, the qualitative classification of soil is a discrete classification factor.

[0026] S44, the characteristics of skew distribution are logarithmically transformed, and the missing values are interpolated by multiple imputation method.

[0027] For example, when pre-processing multiple types of data of neonicotinoid insecticides, if the converted imidacloprid equivalent concentration presents a skew distribution, the imidacloprid equivalent concentration data is logarithmically transformed.

[0028] ​A data set containing feature values X and target values Y is constructed using the preprocessed data described above, the target values Y represent the imidacloprid equivalent concentration of a sub-basin of a certain identity ID at a certain time, and the feature values X represent a set of multi-dimensional factor data that affect Y, including feature values obtained after data preprocessing of each feature data in soil and terrain data, environmental and human activity data;And randomly divide the data set into training set and test set.

[0029] S5, after completing all data collection and preprocessing, start model training, including the following steps:

[0030] S51, use at least linear regression, support vector machine, gradient boosting tree, XGBoost, random forest five kinds of algorithm to train regression model;

[0031] S52, the data set is randomly divided into training sample Xtrain, Ytrain, test sample Xtest, Ytest according to the proportion of 8:2;

[0032] S53, optimize model hyperparameters, with the determination coefficient (R²) and root mean square error (RMSE) on the test set as evaluation indexes, select the optimal model combination.

[0033] S6, unknown area pollution prediction, in addition to the prediction object pesticide concentration, the influence factor statistical method of unknown area is the same as S4.

[0034] S7, the results are visualized and analyzed, and the pollution spatial distribution map (to represent the concentration difference by color) can be generated;Statistical overall concentration level of the study area is drawn to draw the time series trend chart;Support monthly, quarterly, annual three scale spatial distribution query function.

[0035] Compared with the prior art, the present application has the beneficial effects that it fills the blank in the field of time and space distribution prediction of surface water neonicotinoid pesticide pollution, realizes the synchronous, rapid and accurate prediction of multiple sub-basins at the national scale, provides a new method for the prediction and monitoring of surface water neonicotinoid pesticide pollution, integrates six common neonicotinoid pesticides, realizes the synchronous prediction of multiple pollutants by one model, and supplements social and economic factors to comprehensively capture pollution influencing factors. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 For the flowchart of the prediction method of the surface water neonicotinoid pesticide pollution in the sub-basin and machine learning model in the specific embodiment.

[0037] Figure 2 For the prediction and observation sample distribution map of the training set of the data trained by the random forest model;

[0038] Figure 3 are prediction and observed sample distribution maps of a test set of data trained using a random forest model. DETAILED DESCRIPTION

[0039] The implementation process of the present application is described in detail below in combination with specific cases and drawings.

[0040] Please refer to Figure 1 The present application provides a prediction method for neonicotinoid pesticide pollution in a watershed based on sub-basins and research area selection and sub-basin machine learning model. In view of the deficiencies of the present prediction method for neonicotinoid pesticide pollution in surface water at a large scale in terms of precision, efficiency and robustness, the present application combines a representative method of artificial intelligence machine learning to realize prediction of neonicotinoid pesticide pollution in surface water in a sub-basin, and has practical significance of accurately quantifying the pollution degree of neonicotinoid pesticides in the whole surface water of each sub-basin.

[0041] The United States nationwide is a case area where neonicotinoid pesticides are used early and has a wide publicly available water body pesticide pollutant monitoring system. As the largest agricultural producer in the world, the United States has a long history of neonicotinoid pesticide use, a large amount of use, and a relatively serious degree of water pollution. Therefore, the United States is selected as the main case area for research, which mainly includes the following steps:

[0042] S1, determination of sub-basins: download the sub-basin boundary data of the research area from the HydroBASINS database, which can be divided into 12 sub-basin levels according to the size of the watershed. The higher the level, the higher the prediction accuracy. In this case, the highest level sub-basin, the 12th level, is selected.

[0043] S2, after determining the research area and assigning the boundaries of each sub-basin, collect concentration data samples. Because the United States Geological Survey has fixed monitoring sites in major rivers and lakes throughout the country, and periodically selects some sites for sampling investigation, we select the monitoring data of neonicotinoid pesticides in such investigation samples, record the sampling time, latitude and longitude coordinates, water type, and record the detection limit of the sample.

[0044] S3, multi-source data fusion modeling:

[0045] Use Arcgis Pro software to read soil, terrain, climate, and socio-economic data: quantitative feature data and qualitative classification data of surface soil (0-30 cm) and deep soil (30-100 cm).

[0046] The quantitative characteristic data of the soil includes sand content, clay content, organic carbon content, silt content, soil bulk density, cation exchange capacity, soil pH, basic saturation, exchangeable base ion content, electrical conductivity, exchangeable sodium content, and sub-basin average elevation.

[0047] The qualitative classification of the soil includes soil texture classification and hydrological group classification. The soil texture classification is classified according to the soil texture classification standard formulated by the USDA, and a total of 13 textures are classified, and they are respectively assigned as discrete numerical data by one-hot encoding, for example, the soil texture classification of the 13 textures is respectively assigned as the individual codes of 1-13. The hydrological group classification is divided into four levels according to the permeability of water in the soil, and the four levels of texture are high permeability, medium permeability, slow permeability and very low permeability, and they are respectively assigned as discrete numerical data by one-hot encoding, for example, the hydrological group classification of the four textures is respectively assigned as the individual codes of 1, 2, 3 and 4.

[0048] The quantitative characteristic data of the environment and human activities is read by using Arcgis Pro software, including stream intensity index, flow accumulation, pesticide use amount in sub-basin, cultivated land area, drought index, population density, total irradiance under the optimal tilt angle, horizontal total irradiance, normal total irradiance, horizontal scattered irradiance, potential photovoltaic power generation, precipitation and air temperature. In the embodiment of the present application, the data of pesticide use amount, climate (precipitation and temperature) and cultivated land area are read from the data of the U.S. government public website, and the data of other stream intensity index, flow accumulation, horizontal total irradiance and the like are extracted from the data published by relevant research institutions in the United States.

[0049] S4, preprocessing the prediction object and environmental and social factors:

[0050] The environmental factors are matched by using Microsoft Office Excel. The discrete data such as soil texture classification and hydrological group classification are converted into numerical data by one-hot encoding. The soil texture classification of the 13 textures is respectively assigned as the individual codes of 1-13. The hydrological group classification of the four textures is respectively assigned as the individual codes of 1, 2, 3 and 4.

[0051] The various insecticide concentrations are converted into imidacloprid equivalent concentration according to the relative efficacy factor method, and the calculation formula is: , IMI RPF Imidacloprid equivalent concentration, C i The concentration of a new neonicotinoid insecticide, RPF iThe relative efficacy factor of a certain neonicotinoid insecticide (the relative efficacy factors are: imidacloprid = 1, thiamethoxam = 9.5, acetamiprid = 0.803, dinotefuran = 2.850, clothianidin = 5.816, and thiacloprid = 14.25, respectively).

[0052] The features of the skewed distribution are logarithmically transformed, and the missing values are imputed using multiple imputation. For example:

[0053] When pre-processing various types of data of neonicotinoid insecticides, the converted imidacloprid equivalent concentration presents a skewed distribution, so the imidacloprid equivalent concentration data is logarithmically transformed. For samples with a concentration lower than the detection limit, the detection limit is processed according to the detection limit of the corresponding pesticide.

[0054] The data used for model operation obtained is continuous quantitative data, and the average value of a sub-basin in the smallest time span is calculated with the sub-basin as the smallest statistical unit; the data used for model operation obtained is a discrete classification parameter, and the mode of the parameter in the sub-basin is counted with the sub-basin as the smallest statistical unit.

[0055] The data set containing feature values X and target values Y is constructed using the pre-processed data described above, the target values Y represent the imidacloprid equivalent concentration of a sub-basin of a certain identity ID at a certain time, and the feature values X represent a set of multi-dimensional factor data affecting Y, including feature values obtained after data pre-processing of each feature data in soil and terrain data, environmental and human activity data; and the data set is randomly divided into a training set and a test set.

[0056] S5, model training is programmed, hyperparameter optimization and data division training using Matlab software.

[0057] S51, before model training, the entire training data set is divided into training samples Xtrain, Ytrain and test samples Xtest, Ytest in the training: test 8:2 manner using the method of randomly dividing the data set, and iterative training is performed, the model hyperparameters are optimized, the determination coefficient (R²) and the root mean square error (RMSE) on the test set are used as evaluation indexes, and the optimal model combination is selected.

[0058] The determination coefficient of the test set is used as an important evaluation index for stopping iteration. The data set will be re-divided each time to avoid the problem of too high determination coefficient caused by data leakage.

[0059] 5 model evaluation results:

[0060] .

[0061] According to the evaluation results of the above five models, the comprehensive error of the random forest is the smallest, and the model performance is the best.

[0062] S52, the decision coefficient level root mean square error of the training set and the test set is evaluated, and the random forest is selected as the best model, and the super parameter is 200 trees, the bag-out inspection is opened, and the importance analysis is opened. The sample passes The importance of the variable is calculated, wherein M is the number of OOB samples (i.e. out-of-bag samples), 、 The prediction value and the true value of the Mth OOB sample are respectively. This evaluation method helps us to identify the features that have the greatest impact on the regression task, thereby providing a basis for subsequent feature selection and model optimization.

[0063] The parallel computing function of MATLAB is enabled during model training, and multiple decision trees are trained in parallel using multi-core processors, thereby significantly improving the training efficiency. During the training process, the random forest algorithm completes the regression task by constructing N decision trees. The prediction and observation sample distribution diagrams of the training set and the test set of the random forest trained data are shown in Figure 2 and Figure 3 .

[0064] The best-performing random forest model in Matlab is saved for subsequent data prediction using the model.

[0065] S6, in addition to the prediction object pesticide concentration, the influence factor statistics and data processing method of unknown area in previous years is the same as S4.

[0066] Then use the Matlab software to call the saved S5 trained model, bring in the parameters processed in S6, and export the model prediction data under the corresponding parameters.

[0067] S7, visualization analysis and application:

[0068] S71, use Origin software to perform mathematical statistics (calculate the mean, median, and upper and lower quartiles) on all sub-basins participating in prediction, draw box plots and line graphs. Visualize the concentration-time change in the prediction period.

[0069] S72, use Arcgis Pro software to link the prediction concentration of each sub-basin in a certain time period through the sub-basin code of HydroBASIN, divide the concentration into 10 levels, and mark the concentration with 3 colors (green, yellow, and red) to clearly show the spatial distribution map. It can show the hot spots of neonicotinoid pesticide surface water pollution.

[0070] S73, by statistical analysis of the prediction data with the minimum prediction time, different prediction time lengths can be selected, and the pollutant distribution at different times under different time lengths can be output.

[0071] S74, according to the population density map of the past years, key parameters such as affected population and affected area can be calculated.

[0072] S75, according to the environmental factor importance ranking chart generated by the random forest model, the most important influencing factors can be identified, which has guiding significance for risk management and control.

[0073] In addition, it should be understood that the above specific embodiments of the present application are only used for illustrative or explanatory purposes of the principles of the present application, and do not constitute a limitation on the present application. Those skilled in the art can make various modifications, replacements and improvements to the present application according to the above content without departing from the spirit and scope of the present application. The modified, equivalent replaced and improved technical solutions, as long as they meet the technical features defined in the claims of the present application and within the equivalent range, should be covered within the protection scope of the present application.

Claims

1. A method for predicting neonicotinoid pesticide contamination of in- stream surface waters based on sub-basin and machine learning models, characterized in that, Comprise: Step one, determine the partition boundary of each sub-basin, the sub-basin boundary provides the grid cell division basis for step two and step three; Step two, obtain the latitude and longitude coordinates of each sub-basin, construct a spatial coordinate system, and take the coordinate range of each sub-basin as its identity ID; Step three, obtain the modeling basis data of each sub-basin in the study area, the required basis data is divided into the following three categories: Soil and terrain data of sub-basin: including quantitative characteristic data and qualitative classification data of surface and deep soil soil; Surface water pollution monitoring data: that is, various data of neonicotinoid insecticides, including the specific type, concentration, time node and sampling river information of the monitored neonicotinoid insecticides; Quantitative characteristic data of environment and human activities based on sub-basin and station information: Step four, data preprocessing: convert the qualitative classification of soil into numerical data through coding; the various data of neonicotinoid insecticides are converted into the equivalent concentration of typical neonicotinoid insecticide imidacloprid according to the relative efficacy factor method; Step five, establish a data set: use the preprocessed data in step four to construct a data set containing feature values X and target values Y, the target value Y represents the equivalent concentration of imidacloprid of a sub-basin with a certain identity ID at a certain time, and the feature value X represents a set of multi-dimensional factor data that affect Y, including soil and terrain data, and various feature data in environmental and human activity data; and randomly divide the data set into a training set and a test set; Step six, select a machine learning model: select a regression model suitable for multi-factor prediction of pollutants from a machine learning library, train the regression model on the training set, verify the accuracy of the model using test set data, and use root mean square error RMSE and determination coefficient R 2 Evaluate the performance of the model and select the model with the smallest error. Step seven, use the model selected in step six to realize the prediction of the concentration of neonicotinoid insecticides in surface water in the target area.

2. The method of predicting neonicotinoid pesticide contamination in an inland water body within a watershed based on sub-basin and machine learning model according to claim 1, wherein, Step one, based on the sub-basin division data of HydroBASINS database, determine the partition boundary of each sub-basin, and the area of the sub-basin can be divided by the smallest unit sub-basin of the HydroBASINS database.

3. The method of predicting neonicotinoid pesticide contamination in an inland water body within a watershed based on sub-basin and machine learning model according to claim 1, wherein, In step three, the quantitative data of soil includes sand content, clay content, organic carbon content, silt content, soil bulk density, cation exchange capacity, soil pH, basic saturation, exchangeable salt ion content, electrical conductivity, exchangeable sodium salt content, and sub-basin average elevation; the qualitative classification of soil includes soil texture classification and hydrological group classification; The soil texture classification is classified according to the soil texture classification standard formulated by the USDA, the Natural Resources Conservation Service of the United States; The hydrological group is divided into four levels according to the permeability of water in the soil.

4. The method of predicting neonicotinoid pesticide contamination in an inland water body within a watershed based on sub-basin and machine learning model according to claim 1, wherein, In step four, the data preprocessing steps of the quantitative characteristic data and qualitative classification data of soil and terrain data, and the quantitative characteristic data of environment and human activities, further include the following processes: If the data used for model operation is continuous quantitative data, calculate the average value of the quantitative data in the minimum time span of the sub-basin as the feature value of the quantitative data for constructing the data set; If the data used in the participation model operation is qualitative classification data with discrete properties, the mode of the qualitative classification parameter in the sub-basin is calculated as the minimum statistical unit, and the statistical mode is used as the characteristic value of the qualitative classification data to construct the data set.

5. The method of predicting neonicotinoid pesticide contamination in an inland water body within a watershed based on sub-watershed and machine learning model as claimed in claim 1, wherein, In step four, when pre-processing the various types of data of neonicotinoid insecticides, if the converted imidacloprid equivalent concentration presents a skew distribution, the pre-processing step further includes a step of logarithmic transformation of the concentration data.

6. The method of predicting neonicotinoid pesticide contamination in an inland water body within a watershed based on sub-watershed and machine learning model according to claim 1, wherein, The quantitative characteristic data of the environment and human activities include: water flow intensity index, confluence cumulative amount, pesticide use amount in sub-basin, cultivated land area, drought index, population density, total irradiance at the optimal inclination angle, horizontal total irradiance, normal total irradiance, horizontal scattering irradiance, photovoltaic potential power generation, precipitation, and air temperature.

7. The method of predicting neonicotinoid pesticide contamination in an inland water body within a watershed based on sub-watershed and machine learning model according to claim 1, wherein, In step four, the imidacloprid equivalent concentration is calculated using the six commonly used neonicotinoid insecticides using the following formula , IMI RPF the imidacloprid equivalent concentration, C i the concentration of a neonicotinoid insecticide, RPF i the relative efficacy factor of a neonicotinoid insecticide.

8. The method of predicting neonicotinoid pesticide contamination in an inland water body within a watershed based on sub-basin and machine learning model as claimed in claim 7, wherein, The relative efficacy factor of imidacloprid is 1, thiamethoxam is 9.5, acetamiprid is 0.803, dinotefuran is 2.850, clothianidin is 5.816, and thiacloprid is 14.

25.

9. The method of predicting neonicotinoid pesticide contamination in an inland water body within a watershed based on sub-watershed and machine learning model according to claim 1, wherein, The regression model of step six includes five algorithms: linear regression, support vector machine, gradient boosting tree, XGBoost, and random forest.

10. The method of predicting neonicotinoid pesticide contamination in an inland water body within a watershed based on sub-watershed and machine learning model as claimed in claim 1, wherein, When predicting the concentration of neonicotinoid insecticides in surface water in the target area using the model in step seven, the following processes are further included: The pollution prediction result spatial distribution map at the same time and the pollutant change trend graph of the same sub-basin are visualized; for the prediction results of different sub-basins at the same time, the differences in imidacloprid equivalent concentration are distinguished by color; and for the prediction results of the same sub-basin at different times, the change trend of the imidacloprid equivalent concentration of the sub-basin is presented through a line chart, thereby visualizing the pollutant prediction results. ​ ​

Images