Method and system for evaluating influence of soil on underground water based on big data analysis

Abstract

The invention discloses a method and system for evaluating the influence of soil on underground water based on big data analysis, and relates to the technical field of hydrological resources. Comprising the following steps: S1, dividing a to-be-analyzed region, obtaining a plurality of sub-regions and sample data in the sub-regions, and carrying out preprocessing and sample balance processing on the sample data; s2, combining the sample data after preprocessing and sample balance processing with external data, and constructing a prediction model according to a combined data set; and S3, taking the sample sampling point data obtained in real time as the input of the prediction model, and outputting the obtained real-time evaluation data. According to the method, the number of the samples is taken as a processing standard, the number of the samples in different areas is increased or deleted, so that the number of the samples in different areas is kept consistent, the model bias caused by too many or too few samples in some areas is reduced, and the representativeness and fairness of data are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of hydrological resources, and in particular to a method and system for evaluating the impact of soil on groundwater based on big data analysis. Background Art

[0002] With the acceleration of urbanization and industrialization, soil pollution is becoming increasingly serious, posing a potential threat to groundwater. Traditional soil and groundwater environmental monitoring methods often rely on fixed-point sampling and laboratory testing, which is not only time-consuming and labor-intensive, but also difficult to capture the true situation of pollutant migration and diffusion.

[0003] Big data analysis can reveal trends and patterns hidden behind a large number of observational records by processing massive and diverse data sets. For the study of soil-groundwater interaction, it can obtain a more comprehensive understanding from different time scales (such as seasonal changes) and spatial resolutions (from local sites to regional levels). For example, with the help of big data technology, the soil properties that are most likely to cause groundwater contamination can be identified, and targeted prevention and control measures can be formulated accordingly.

[0004] The Chinese invention patent with the publication number CN117494042A discloses a soil water fusion method for data-deficient areas based on climate similarity transfer learning. This method collects data on soil water (including ground station data, remote sensing and reanalysis data) and its key influencing factors in the source domain and target domain. After preprocessing and normalization, a data fusion framework is established. The CNN-LSTM model can effectively extract the spatial and temporal characteristics of the data, and based on climate similarity, the source domain is screened and the target domain is refined and classified. From the perspective of climate similarity, different fusion sub-models are selected for different climate types, which effectively solves the problem of poor migration effect caused by large differences in data distribution between the source domain and the target domain in transfer learning. This method covers the steps of source domain screening, model pre-training, target domain classification, sub-model migration and fusion model integration. The effectiveness of the method is verified by evaluating the fusion results on the test set. This method uses deep learning and transfer learning technology to alleviate the data shortage in data-deficient areas and the negative migration problem in transfer learning, and improve the accuracy of soil water data in data-deficient areas.

[0005] Most of the above and similar evaluation methods use machine learning algorithms to evaluate the impact of soil on groundwater, but in the process of prediction, they often lead to unfair results due to biases in the training data set. For example, when the number of soil samples in some areas far exceeds that in other areas, in the process of prediction, it tends to more accurately reflect the situation in that area and ignore the actual conditions in other areas. Although this bias can be reduced by introducing regularization terms or adjusting weights, it will become an obstacle to practical application as performance decreases in the process of improving algorithm transparency. Summary of the invention

[0006] The purpose of the present invention is to provide a method and system for evaluating the impact of soil on groundwater based on big data analysis, so as to solve the problems raised in the above-mentioned background technology.

[0007] To achieve the above object, the present invention provides the following technical solution: a method for evaluating the impact of soil on groundwater based on big data analysis, comprising:

[0008] S1: Adjusting the sample quantity ratio: dividing the area to be analyzed, obtaining multiple sub-areas and sample data in the sub-areas, and performing pre-processing and sample balancing processing on the sample data;

[0009] S2: Establishing a prediction model: combining the sample data after the preprocessing and sample balancing processing with external data, and building a prediction model based on the combined data set;

[0010] S3: Obtaining real-time evaluation data: using the sample sampling point data obtained in real time as the input of the prediction model, and outputting the real-time evaluation data;

[0011] Performing sample balancing processing on the sample data includes:

[0012] M1: Determine the standard number of sample data: According to the total amount of sample data corresponding to all sub-areas, obtain the average sample number and determine the standard number of sample data;

[0013] M2: Classify the sub-areas: According to the standard quantity, divide the sub-areas into sub-areas that exceed the standard quantity and sub-areas that do not exceed the standard quantity, specifically:

[0014] When the total amount of sample data corresponding to the sub-region is greater than the standard amount, the sub-region is a sub-region that exceeds the standard amount; otherwise, the sub-region is a sub-region that does not exceed the standard amount;

[0015] M3: Sample quantity adjustment: increase or decrease the total amount of sample data corresponding to the sub-region according to the classification category corresponding to the sub-region.

[0016] Furthermore, the total amount of sample data corresponding to the sub-region is increased or decreased, including:

[0017] M3.1: Sample data deletion: Obtain the local reachable density and abnormality level of each sample sampling point in the sub-area that exceeds the standard number, and delete the total amount of sample data according to the local reachable density and abnormality level;

[0018] M3.2: Sample data increase: Obtain the local density of each sample sampling point in the sub-area that does not exceed the standard number, and divide the area where the sample sampling point is located into a high-density area and a low-density area according to the local density, and at the same time obtain the newly added sample sampling point data through the sample sampling point data in the low-density area.

[0019] Furthermore, the total amount of sample data is reduced, including:

[0020] N1: Obtaining local reachable density: According to the position of the sample sampling point, obtain the distance between the sample sampling point and the adjacent sample sampling point, determine the reachable distance between the sample sampling point and the adjacent sample sampling point, and obtain the local reachable density corresponding to the sample sampling point according to the reachable distance, specifically:

[0021]

[0022] in: is the local reachable density corresponding to the i-th sample sampling point, is the k nearest neighbor set of the i-th sample point, is the i-th sample sampling point, is the jth sample point, is the accessible distance between the i-th sample sampling point and the j-th sample sampling point;

[0023] N2: Obtaining the degree of abnormality: Obtain the LOF value corresponding to each sample sampling point through the local reachable density corresponding to the sample sampling point and the LOF calculation formula, specifically:

[0024]

[0025] in: is the LOF value corresponding to the i-th sample sampling point, is the local reachable density corresponding to the i-th sample sampling point, is the local reachable density corresponding to the jth sample point, is the k nearest neighbor set of the i-th sample point, is the i-th sample sampling point, is the jth sample sampling point;

[0026] N3: Determine the deleted sampling points: According to the local reachable density and LOF value corresponding to the sample sampling points, obtain the initial deleted sample sampling points and the secondary deleted sample sampling points, and determine the final deleted sample points according to the initial deleted sample sampling points and the secondary deleted sample sampling points.

[0027] Furthermore, the final deleted sampling points are determined, including:

[0028] N3.1: Obtain the quantity to be deleted: obtain the quantity difference between the total amount of sample data of each sub-region and the standard quantity, and the quantity difference is the quantity to be deleted;

[0029] N3.2: Determine the sample sampling points to be deleted: sort all the sample sampling points in descending order according to the LOF values ​​corresponding to the sample sampling points, and obtain the same number of sample sampling points as the number to be deleted according to the sorting result, and the sample sampling points are the initial sample sampling points to be deleted. At the same time, sort all the sample sampling points in ascending order according to the local reachable density corresponding to the sample sampling points, and obtain the same number of sample sampling points as the number to be deleted according to the sorting result, and the sample sampling points are the secondary sample sampling points to be deleted;

[0030] N3.3: Determine preliminary deleted data: match the initial deleted sample sampling points with the secondary deleted sample sampling points to determine the overlapping deleted sample sampling points, which are the initial deleted sample sampling points.

[0031] Furthermore, the final deletion sampling points are determined, including:

[0032] N3.4: Determine the actual number of deletions: Compare the number of samples corresponding to the sample sampling points of the preliminary deletion with the number to be deleted, and determine the final deleted sampling points based on the comparison result, specifically:

[0033] When the number of samples corresponding to the sample sampling points initially deleted is the same as the number to be deleted, the sample sampling points initially deleted are the final deleted sampling points, otherwise, the next step is executed;

[0034] N3.5: Determine the reorganized deleted data: Determine the remaining number to be deleted according to the number of samples corresponding to the sample sampling points of the preliminary deletion and the number to be deleted, and at the same time, obtain the sub-deleted sample sampling points of the remaining number to be deleted from the remaining sample sampling points sorted according to the local reachable density, and combine the sub-deleted sample sampling points of the remaining number to be deleted with the unmatched sub-deleted sample sampling points to obtain the reorganized sub-deleted sample sampling points;

[0035] From the remaining sample sampling points sorted according to the LOF values, obtain the remaining number of the initially deleted sample sampling points to be deleted, and combine the remaining number of the initially deleted sample sampling points to be deleted with the unmatched initially deleted sample sampling points to obtain recombined initially deleted sample sampling points;

[0036] N3.6: Determine the final deleted sample points: match the reorganized initial deleted sample sampling points with the reorganized secondary deleted sample sampling points, determine the overlapping reorganized deleted sample sampling points, and compare the number of samples at the reorganized deleted sample sampling points with the remaining number to be deleted, and determine the final deleted sample sampling points based on the comparison results, specifically:

[0037] When the number of samples of the reorganized and deleted sample sampling points is the same as the remaining number to be deleted, the reorganized and deleted sample sampling points and the initially deleted sample sampling points are both final deleted sample sampling points; otherwise, the reorganized and deleted sample sampling points are reordered according to the local reachable density, and the same number of re-deleted sample sampling points as the remaining number to be deleted are determined, and the re-deleted sample sampling points and the initially deleted sample sampling points are both final deleted sample sampling points.

[0038] Furthermore, the newly added sample sampling point data is obtained, including:

[0039] W1: Obtaining data to be added: Obtain the difference between the total amount of sample data and the standard amount in each sub-area, and the difference is the data to be added;

[0040] W2: Divide density area: Obtain the local density corresponding to each sample sampling point through a clustering algorithm, and compare the local density with a preset density threshold. According to the comparison result, determine the density area corresponding to each sample sampling point, specifically:

[0041] When the local density corresponding to the sample sampling point is less than the preset density threshold, the density area corresponding to the sample sampling point is a low-density area; otherwise, the density area corresponding to the sample sampling point is a high-density area;

[0042] W3: Adding sample sampling point data: According to the sample sampling point data in the low-density area and the data to be added, obtaining the newly added sample sampling point data, specifically:

[0043]

[0044] in: For the newly added sample sampling point data, is a random number, is the i-th sample sampling point data in the low-density area, is the data of the i-th sample point in the low-density area.

[0045] Furthermore, a prediction model is constructed, including:

[0046] S2.1: constructing a comprehensive data set: combining the sample data after the preprocessing and sample balancing processing with external data to obtain a comprehensive data set;

[0047] S2.2: Constructing a prediction model: Dividing the comprehensive data set into a training set and a test set, and constructing a prediction model through the training set and the test set, specifically:

[0048]

[0049] in: is the pollutant concentration corresponding to time t+1, is the pollutant concentration corresponding to time t, is the precipitation corresponding to time t, is the soil permeability, To optimize the parameters, is a nonlinear mapping function.

[0050] Furthermore, the optimization parameter acquisition formula is specifically:

[0051]

[0052] in: is the gradient operator, is the comprehensive loss function, To optimize the parameters, is a task-specific loss function, is a hyperparameter of the relative importance of the density penalty term, To control the hyperparameters of the domain adversarial loss, is the domain adversarial loss function, is the density penalty term.

[0053] A soil impact on groundwater assessment system based on big data analysis uses the above-mentioned soil impact on groundwater assessment method based on big data analysis.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] The present invention collects a sufficient number of high-quality samples from more diverse geographical areas through sensor networks and remote sensing technology, thereby increasing the breadth and depth of the data. At the same time, the number of samples is used as a processing standard to increase or decrease the number of samples in different regions, so that the number of samples in different regions remains consistent, thereby reducing the model bias caused by too many or too few samples in certain regions, thereby improving the representativeness and fairness of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a schematic diagram of the process of the evaluation method of the present invention;

[0057] Figure 2 A schematic diagram of the process of obtaining the final deleted sample sampling points in the present invention;

[0058] Figure 3 A schematic diagram of the process of obtaining the sampling points of the recombinant deletion samples in the present invention;

[0059] Figure 4 The figure is a schematic diagram of the process of obtaining new sample sampling points in the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] Most existing evaluation methods evaluate the impact of soil on groundwater through machine learning algorithms, but in the process of prediction, they often lead to unfair results due to biases in the training data set. For example, when the number of soil samples in some areas far exceeds that in other areas, in the process of prediction, it will tend to more accurately reflect the situation in the area and ignore the actual conditions in other areas. Although this bias can be reduced by introducing regularization terms or adjusting weights, in the process of improving the transparency of the algorithm, it will become an obstacle to practical application as performance decreases. The technical solution of the present application obtains the standard number of sample data, increases or deletes the number of samples in different areas, so that the number of samples in different areas remains consistent, and combines the processed sample data with external data to obtain a comprehensive data set, and builds a prediction model based on the comprehensive data set, thereby reducing the model bias caused by too many or too few samples in certain areas, thereby improving the representativeness and fairness of the data.

[0062] refer to Figure 1-Figure 4 This embodiment provides an assessment method for the impact of soil on groundwater based on big data analysis, and the assessment method includes the following steps:

[0063] Step S1: Adjust the sample quantity ratio. That is, divide the area to be analyzed, obtain multiple sub-areas and soil samples and groundwater samples in the sub-areas, and perform pre-processing and sample balance processing on the obtained soil samples and groundwater samples. The details are as follows:

[0064] Step S1.1: Obtain preprocessed sample data. The area to be analyzed is divided into multiple sub-areas. Specifically, the area is divided according to administrative regions (such as counties and cities) or natural geographical features (such as mountains and rivers), so that each sub-area is homogeneous and representative.

[0065] Furthermore, in the process of regional division, for sub-regions with uniform geographical distribution, a preset number (specifically set according to actual needs, not specifically explained in this embodiment) of sampling locations are randomly selected in each sub-region for sample collection. For sub-regions with uneven geographical distribution, the sub-regions are divided into multiple levels according to relevant characteristics (such as soil type, land use method), and the relevant number of samples is extracted in each level according to a preset ratio (specifically set according to actual needs, not specifically explained in this embodiment).

[0066] It is worth noting that in the process of sample collection, soil samples need to be sampled in layers at selected locations, and groundwater samples need to be extracted by drilling, etc. At the same time, relevant environmental information of each sampling point needs to be recorded, such as climate conditions, vegetation coverage, and intensity of human activities.

[0067] In this embodiment, the obtained samples are preprocessed to obtain preprocessed sample data. Specifically, through data cleaning, obviously erroneous data in the sample data, such as values ​​beyond a reasonable range or repeated records, are obtained. At the same time, through data standardization and normalization, the sample data after data cleaning is unified. Furthermore, the unified sample data is further filled with missing values, that is, the missing data values ​​are filled by the mean / median filling and K nearest neighbor interpolation method to ensure the integrity and accuracy of the sample data.

[0068] Step S1.2: Perform sample data balance. That is, determine the standard number of sample data based on the total amount of sample data obtained, and divide the sub-regions into sub-regions that exceed the standard number and sub-regions that do not exceed the standard number based on the determined standard number. According to the division result, the number of samples corresponding to the sub-regions that exceed the standard number is deleted, and the number of samples corresponding to the sub-regions that do not exceed the standard number is increased, so that the number of samples in each sub-region is consistent. The details are as follows:

[0069] Step M1: Determine the standard number of sample data. That is, according to the total amount of sample data corresponding to all sub-regions, obtain the average number of samples corresponding to the sample data, and the average number of samples is the standard number of sample data.

[0070] In the specific implementation process, 500 samples are obtained in sub-area A, 100 samples are obtained in sub-area B, and 200 samples are obtained in sub-area C. Therefore, the standard number corresponding to the three sub-areas A, B and C is 267 samples.

[0071] Step M2: Classify the sub-areas. That is, according to the standard quantity obtained in step M1, all sub-areas are divided into two categories: sub-areas that exceed the standard quantity and sub-areas that do not exceed the standard quantity. At the same time, according to the two divided categories, the classification category corresponding to each sub-area is determined, specifically:

[0072] When the total amount of sample data corresponding to a sub-region is greater than the standard quantity in step M1, the classification category corresponding to the sub-region is a sub-region that exceeds the standard quantity; otherwise, the classification category corresponding to the sub-region is a sub-region that does not exceed the standard quantity.

[0073] During the specific implementation process, based on the determined standard number of 267 samples, the classification category of sub-area A was the sub-area that exceeded the standard number, and the classification categories of sub-areas B and C were the sub-areas that did not exceed the standard number.

[0074] Step M3: Sample quantity adjustment. That is, according to the classification category corresponding to each sub-region determined in step M2, the total amount of sample data of the sub-region that exceeds the standard quantity is deleted, and the total amount of sample data of the sub-region that does not exceed the standard quantity is increased, so that the total amount of sample data of each sub-region is consistent with the standard quantity in step M1, so as to facilitate the centralized processing of sample data of all sub-regions. The details are as follows:

[0075] Step M3.1: Sample data deletion. That is, obtain the local reachability density and abnormality of each sample sampling point, and delete the total amount of sample data of sub-areas that exceed the standard number according to the local reachability density and abnormality. The details are as follows:

[0076] Step N1: Obtain local reachable density. That is, according to the location of the sample sampling point, obtain the distance between it and the adjacent sample sampling points, and determine the reachable distance between the sample sampling point and its adjacent sample sampling points. At the same time, the local reachable density corresponding to the sample sampling point is obtained through the reachable distances corresponding to all sample sampling points, which is specifically:

[0077]

[0078] in: is the local reachable density corresponding to the i-th sample sampling point, is the k nearest neighbor set of the i-th sample point, is the i-th sample sampling point, is the jth sample point, is the reachable distance between the i-th sample sampling point and the j-th sample sampling point.

[0079] In this embodiment, the reachable distance between the i-th sample sampling point and the j-th sample sampling point is specifically:

[0080]

[0081] in: is the reachable distance between the i-th sample sampling point and the j-th sample sampling point, is the direct distance between the i-th sample sampling point and the j-th sample sampling point, is the k distance of the jth sample point.

[0082] Step N2: Obtain the degree of abnormality. That is, the LOF value corresponding to each sample sampling point is obtained through the local reachable density and LOF calculation formula corresponding to the sample sampling point obtained in step N1. In other words, the degree of abnormality of the sample sampling point is determined by the LOF value corresponding to the sample sampling point. Specifically, when the LOF value corresponding to the sample sampling point is close to 1, the corresponding sample sampling point is a normal sampling point, otherwise, the sample sampling point is an abnormal sampling point. In this embodiment, the LOF calculation formula is specifically:

[0083]

[0084] in: is the LOF value corresponding to the i-th sample sampling point, is the local reachable density corresponding to the i-th sample sampling point, is the local reachable density corresponding to the jth sample point, is the k nearest neighbor set of the i-th sample point, is the i-th sample sampling point, is the jth sample point.

[0085] Step N3: Determine the deleted sampling points. That is, according to the LOF values ​​corresponding to the sample sampling points obtained in step N2 and the local reachable density corresponding to the sample sampling points obtained in step N1, sort all the sample sampling points in the sub-area respectively, and according to the difference between the total amount of sample data in each sub-area and the standard quantity, determine the initial deleted sample sampling points and the secondary deleted sample sampling points from all the sorted sample sampling points, and at the same time, determine the final deleted sample sampling points according to the initial deleted sample sampling points and the secondary deleted sample sampling points. The details are as follows:

[0086] Step N3.1: Obtain the number to be deleted. That is, according to the total amount of sample data in each sub-region and the standard amount of sample data obtained in step M1, determine the difference between the total amount of sample data and the standard amount, and the difference is the number of sample data to be deleted.

[0087] Step N3.2: Determine the sample sampling points to be deleted. That is, according to the LOF values ​​corresponding to the sample sampling points obtained in step N2, all sample sampling points are sorted in descending order. At the same time, according to all the sample sampling points sorted from large to small by the LOF values, the initial sample sampling points that are consistent with the number of sample data to be deleted are obtained one by one. Similarly, according to the local reachable density corresponding to the sample sampling points obtained in step N1, all sample sampling points are sorted in ascending order. And according to all the sample sampling points sorted from small to large by the local reachable density, the secondary sample sampling points that are consistent with the number of sample data to be deleted are obtained one by one.

[0088] Step N3.3: Determine preliminary deleted data. Match the initial deleted sample sampling points and the secondary deleted sample sampling points determined in step N3.2, and determine the overlapping deleted sample sampling points, which are the preliminary deleted sample sampling points.

[0089] Step N3.4: Determine the actual number of deletions. That is, compare the number of sample points initially deleted obtained in step N3.3 with the number to be deleted obtained in step N3.1, and determine whether the total amount of sample data in the sub-region is the same as the standard amount of sample data based on the comparison result. Specifically:

[0090] When the number of sample sampling points initially pruned is the same as the number to be pruned, the sample sampling points initially pruned are the final pruned sampling points; otherwise, the next step is executed until the number of sample sampling points pruned is the same as the number to be pruned, that is, all pruned sample sampling points are the final pruned sampling points.

[0091] Step N3.5: Determine the reorganized and deleted data. That is, when the number of preliminary deleted data obtained in step N3.3 is less than the number of sample data to be deleted, obtain the remaining number of secondary deleted sample sampling points to be deleted from the remaining sample sampling points sorted from small to large according to the local reachable density. Similarly, obtain the remaining number of initial deleted sample sampling points to be deleted from the remaining sample sampling points sorted from large to small according to the LOF value. And combine the remaining number of secondary deleted sample sampling points to be deleted with the secondary deleted sample sampling points that were not matched in step N3.3 to obtain the reorganized secondary deleted sample sampling points. Similarly, combine the remaining number of initial deleted sample sampling points to be deleted with the initial deleted sample sampling points that were not matched in step N3.3 to obtain the reorganized initial deleted sample sampling points.

[0092] Step N3.6: Determine the final deleted sample points. That is, match the reorganized initial deleted sample sampling points and the reorganized secondary deleted sample sampling points obtained in step N3.5, and determine the overlapping reorganized deleted sample sampling points. Compare the number of samples at the reorganized deleted sample sampling points with the remaining number to be deleted, and determine the final deleted sample sampling points based on the comparison results. Specifically:

[0093] When the number of samples of the reorganized deleted sample sampling points is the same as the remaining number to be deleted, the reorganized deleted sample sampling points and the initially deleted sample sampling points are both the final deleted sample sampling points. On the contrary, when the number of samples of the reorganized deleted sample sampling points is greater than the remaining number to be deleted, the reorganized deleted sample sampling points are reordered in ascending order according to the local reachable density, and according to the reordered order, the sample sampling points with the same number as the remaining number to be deleted are determined, that is, the sample sampling points are the re-deleted sample sampling points, and the re-deleted sample sampling points and the initially deleted sample sampling points are both the final deleted sample sampling points.

[0094] In the process of specific implementation, a data table as shown in Table 1 below is set, specifically:

[0095] Table 1: Data table

[0096] Further, in the above Table 1, 10 sample sampling points are set. In this embodiment, the standard number of sample data is set to 7, so the number to be deleted is 3. Specifically, according to the LOF value, the sample is sorted as: E, C, I, G, A, D, F, B, H, J. According to the local reachable density, the sample is sorted as: H, D, B, J, G, A, F, I, C, E. In other words, the initial deletion sample sampling points are: E, C and I, and the secondary deletion sample sampling points are: H, D and B. Since there are no overlapping sample sampling points between the initial deletion sample sampling points and the secondary deletion sample sampling points, the reorganized deleted data is continued to be obtained, that is, the reorganized initial deletion sample sampling points are: E, C, I, G, A and D, and the reorganized secondary deletion sample sampling points are: H, D, B, J, G and A. Further, the sample sampling points that overlap in the reorganized initial deletion sample sampling points and the reorganized secondary deletion sample sampling points are: A, D and G. In other words, the number of deleted samples determined is the same as the number to be deleted, so the final deleted sample sampling points are: A, D and G.

[0097] Step M3.2: Sample data increase. That is, obtain the local density of each sample sampling point in the sub-area that does not exceed the standard number, and divide the area where the sample sampling point is located according to the local density, that is, into a high-density area and a low-density area, and synthesize new sample sampling point data based on the data of the sample sampling point in the low-density area. The details are as follows:

[0098] Step W1: Obtain the data to be added. That is, obtain the total amount of sample data in each sub-area that does not exceed the standard amount and the standard amount of sample data obtained in step M1, and determine the difference between the total amount of sample data and the standard amount, which is the amount of sample data to be added.

[0099] Step W2: Divide the density area. That is, through the clustering algorithm, obtain the local density around each sample sampling point, specifically:

[0100]

[0101] in: is the local density corresponding to the sample sampling point P, is the distance between the sample sampling point P and the sample sampling point Q, is the kernel function, is the sample point Q, is the sample sampling point P, is the set of all other sample sampling points within the area of ​​the sample sampling point P.

[0102] Furthermore, the local density corresponding to each sample sampling point is compared with a preset density threshold (specifically set according to the actual scenario, which is not specifically explained in this embodiment), and the density area corresponding to each sample sampling point is determined based on the comparison result. Specifically:

[0103] When the local density corresponding to the sample sampling point is less than the preset density threshold, the density area corresponding to the sample sampling point is a low-density area; otherwise, the density area corresponding to the sample sampling point is a high-density area.

[0104] Step W3: Adding sample sampling point data. That is, according to the low-density area determined in step W2, all sample sampling points in the low-density area are collected, and new sample sampling point data are added according to the sample sampling point data in the low-density area, specifically:

[0105]

[0106] in: For the newly added sample sampling point data, is a random number, is the i-th sample sampling point data in the low-density area, is the data of the i-th sample point in the low-density area.

[0107] Furthermore, according to the data to be added obtained in step W1, newly added sample sampling point data with the same number as the data to be added is obtained.

[0108] In the specific implementation process, the data of the two sample points in the low-density area are: X (x1=0.5, x2=0.7) and Y (x1=0.4, x2=0.6), and the random number It is set to 0.3, so the new sample sampling point data is: (x1=0.47, x2=0.67).

[0109] Step S2: Establish a prediction model. That is, the total amount of sample data after increase or decrease in step M3 is combined with external data (such as soil chemical composition, climate conditions and geological structure) to obtain a comprehensive data set. At the same time, a prediction model is constructed based on the comprehensive data set. The details are as follows:

[0110] Step S2.1: Construct a comprehensive data set. That is, the total amount of sample data after addition and subtraction in step M3 is combined with external data to obtain a comprehensive data set. Specifically, soil chemical composition includes pH value and organic matter content, climate conditions include annual average precipitation and seasonal temperature changes, and geological structure includes rock layer distribution and fault line location.

[0111] Step S2.2: Construct a prediction model. The comprehensive data set is divided into a training set and a test set according to a ratio. Specifically, the comprehensive data set is divided into a 6:4 ratio, one part of which is used as a training set for training and establishing a prediction model, and the other part is used as a test set for testing the established prediction model. In this embodiment, the prediction model is specifically:

[0112]

[0113] in: is the pollutant concentration corresponding to time t+1, is the pollutant concentration corresponding to time t, is the precipitation corresponding to time t, is the soil permeability, To optimize the parameters, is a nonlinear mapping function.

[0114] Furthermore, the optimization parameters The specific formula for obtaining is:

[0115]

[0116] in: is the gradient operator, is the comprehensive loss function, To optimize the parameters, is a task-specific loss function, is a hyperparameter of the relative importance of the density penalty term, To control the hyperparameters of the domain adversarial loss, is the domain adversarial loss function, is the density penalty term.

[0117] Step S3: Obtaining impact assessment. That is, according to the prediction model established in step S2.2, the data of the sample sampling points obtained in real time is used as input, and the corresponding assessment data is obtained as output.

[0118] In the specific implementation process, the soil pollutant concentration corresponding to the current time is 5.2mg / L, the precipitation is 80mm, and the soil permeability is 0.001cm / s. At the same time, it is known that the precipitation at the adjacent future time is 60mm, and the soil permeability is 0.001cm / s, so the soil pollutant concentration at the adjacent future time is 5.8mg / L.

[0119] This embodiment also provides an assessment system for the impact of soil on groundwater based on big data analysis. The adaptive control system uses an assessment method for the impact of soil on groundwater based on big data analysis described in the above embodiment.

[0120] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is limited by the attached embodiments and their equivalents.

Claims

1. A method for evaluating the impact of soil on groundwater based on big data analysis, characterized in that: Included are: S1: Adjusting the sample quantity ratio: dividing the area to be analyzed, obtaining multiple sub-areas and sample data in the sub-areas, and performing pre-processing and sample balancing processing on the sample data; S2: Establishing a prediction model: combining the sample data after the preprocessing and sample balancing processing with external data, and building a prediction model based on the combined data set; S3: Obtaining real-time evaluation data: using the sample sampling point data obtained in real time as the input of the prediction model, and outputting the real-time evaluation data; Performing sample balancing processing on the sample data includes: M1: Determine the standard number of sample data: According to the total amount of sample data corresponding to all sub-areas, obtain the average sample number and determine the standard number of sample data; M2: Classify the sub-areas: According to the standard quantity, divide the sub-areas into sub-areas that exceed the standard quantity and sub-areas that do not exceed the standard quantity, specifically: When the total amount of sample data corresponding to the sub-region is greater than the standard amount, the sub-region is a sub-region that exceeds the standard amount; otherwise, the sub-region is a sub-region that does not exceed the standard amount; M3: Sample quantity adjustment: increase or decrease the total amount of sample data corresponding to the sub-region according to the classification category corresponding to the sub-region.

2. The method for evaluating the impact of soil on groundwater based on big data analysis according to claim 1 is characterized in that: The total amount of sample data corresponding to the sub-region is increased or decreased, including: M3.1: Sample data deletion: Obtain the local reachable density and abnormality level of each sample sampling point in the sub-area that exceeds the standard number, and delete the total amount of sample data according to the local reachable density and abnormality level; M3.2: Sample data increase: Obtain the local density of each sample sampling point in the sub-area that does not exceed the standard number, and divide the area where the sample sampling point is located into a high-density area and a low-density area according to the local density, and at the same time obtain the newly added sample sampling point data through the sample sampling point data in the low-density area.

3. The method for evaluating the impact of soil on groundwater based on big data analysis according to claim 2 is characterized in that: The total amount of sample data is deleted, including: N1: Obtaining local reachable density: According to the position of the sample sampling point, obtain the distance between the sample sampling point and the adjacent sample sampling point, determine the reachable distance between the sample sampling point and the adjacent sample sampling point, and obtain the local reachable density corresponding to the sample sampling point according to the reachable distance, specifically: ,in: is the local reachable density corresponding to the i-th sample sampling point, is the k nearest neighbor set of the i-th sample point, is the i-th sample sampling point, is the jth sample point, is the accessible distance between the i-th sample sampling point and the j-th sample sampling point; N2: Obtaining the degree of abnormality: Obtain the LOF value corresponding to each sample sampling point through the local reachable density corresponding to the sample sampling point and the LOF calculation formula, specifically: ,in: is the LOF value corresponding to the i-th sample sampling point, is the local reachable density corresponding to the i-th sample sampling point, is the local reachable density corresponding to the jth sample point, is the k nearest neighbor set of the i-th sample point, is the i-th sample sampling point, is the jth sample sampling point; N3: Determine the deleted sampling points: According to the local reachable density and LOF value corresponding to the sample sampling points, obtain the initial deleted sample sampling points and the secondary deleted sample sampling points, and determine the final deleted sample points according to the initial deleted sample sampling points and the secondary deleted sample sampling points.

4. The method for evaluating the impact of soil on groundwater based on big data analysis according to claim 3 is characterized in that: Determine the final deleted sampling points, including: N3.1: Obtain the quantity to be deleted: obtain the quantity difference between the total amount of sample data of each sub-region and the standard quantity, and the quantity difference is the quantity to be deleted; N3.2: Determine the sample sampling points to be deleted: sort all the sample sampling points in descending order according to the LOF values ​​corresponding to the sample sampling points, and obtain the same number of sample sampling points as the number to be deleted according to the sorting result, and the sample sampling points are the initial sample sampling points to be deleted. At the same time, sort all the sample sampling points in ascending order according to the local reachable density corresponding to the sample sampling points, and obtain the same number of sample sampling points as the number to be deleted according to the sorting result, and the sample sampling points are the secondary sample sampling points to be deleted; N3.3: Determine preliminary deleted data: match the initial deleted sample sampling points with the secondary deleted sample sampling points to determine the overlapping deleted sample sampling points, which are the initial deleted sample sampling points.

5. The method for evaluating the impact of soil on groundwater based on big data analysis according to claim 4 is characterized in that: Determine the final deletion sampling points, including: N3.4: Determine the actual number of deletions: Compare the number of samples corresponding to the sample sampling points of the preliminary deletion with the number to be deleted, and determine the final deleted sampling points based on the comparison result, specifically: When the number of samples corresponding to the sample sampling points initially deleted is the same as the number to be deleted, the sample sampling points initially deleted are the final deleted sampling points, otherwise, the next step is executed; N3.5: Determine the reorganized deleted data: Determine the remaining number to be deleted according to the number of samples corresponding to the sample sampling points of the preliminary deletion and the number to be deleted, and at the same time, obtain the sub-deleted sample sampling points of the remaining number to be deleted from the remaining sample sampling points sorted according to the local reachable density, and combine the sub-deleted sample sampling points of the remaining number to be deleted with the unmatched sub-deleted sample sampling points to obtain the reorganized sub-deleted sample sampling points; From the remaining sample sampling points sorted according to the LOF values, obtain the remaining number of the initially deleted sample sampling points to be deleted, and combine the remaining number of the initially deleted sample sampling points to be deleted with the unmatched initially deleted sample sampling points to obtain recombined initially deleted sample sampling points; N3.6: Determine the final deleted sample points: match the reorganized initial deleted sample sampling points with the reorganized secondary deleted sample sampling points, determine the overlapping reorganized deleted sample sampling points, and compare the number of samples at the reorganized deleted sample sampling points with the remaining number to be deleted, and determine the final deleted sample sampling points based on the comparison results, specifically: When the number of samples of the reorganized and deleted sample sampling points is the same as the remaining number to be deleted, the reorganized and deleted sample sampling points and the initially deleted sample sampling points are both final deleted sample sampling points; otherwise, the reorganized and deleted sample sampling points are reordered according to the local reachable density, and the same number of re-deleted sample sampling points as the remaining number to be deleted are determined, and the re-deleted sample sampling points and the initially deleted sample sampling points are both final deleted sample sampling points.

6. The method for evaluating the impact of soil on groundwater based on big data analysis according to claim 2 is characterized in that: Get the newly added sample sampling point data, including: W1: Obtaining data to be added: Obtain the difference between the total amount of sample data and the standard amount in each sub-area, and the difference is the data to be added; W2: Divide density area: Obtain the local density corresponding to each sample sampling point through a clustering algorithm, and compare the local density with a preset density threshold. According to the comparison result, determine the density area corresponding to each sample sampling point, specifically: When the local density corresponding to the sample sampling point is less than the preset density threshold, the density area corresponding to the sample sampling point is a low-density area; otherwise, the density area corresponding to the sample sampling point is a high-density area; W3: Adding sample sampling point data: According to the sample sampling point data in the low-density area and the data to be added, obtaining the newly added sample sampling point data, specifically: ,in: For the newly added sample sampling point data, is a random number, is the i-th sample sampling point data in the low-density area, is the data of the i-th sample point in the low-density area.

7. The method for evaluating the impact of soil on groundwater based on big data analysis according to claim 1, characterized in that: Build a predictive model, including: S2.1: constructing a comprehensive data set: combining the sample data after the preprocessing and sample balancing processing with external data to obtain a comprehensive data set; S2.2: Constructing a prediction model: Dividing the comprehensive data set into a training set and a test set, and constructing a prediction model through the training set and the test set, specifically: ,in: is the pollutant concentration corresponding to time t+1, is the pollutant concentration corresponding to time t, is the precipitation corresponding to time t, is the soil permeability, To optimize the parameters, is a nonlinear mapping function.

8. The method for assessing the impact of soil on groundwater based on big data analysis according to claim 7 is characterized in that: The specific formula for obtaining the optimization parameters is: ,in: is the gradient operator, is the comprehensive loss function, To optimize the parameters, is a task-specific loss function, is a hyperparameter of the relative importance of the density penalty term, To control the hyperparameters of the domain adversarial loss, is the domain adversarial loss function, is the density penalty term.

9. A soil impact assessment system based on big data analysis, characterized in that: A method for assessing the impact of soil on groundwater based on big data analysis as described in any one of claims 1 to 8 is used.

Images