Heavy metal pollution inversion method and system based on hyperspectral imaging technology

Abstract

The invention provides a heavy metal pollution inversion method and system based on a hyperspectral imaging technology, and relates to the technical field of environmental monitoring and remote sensing, and the method comprises the following steps: 1, employing a hyperspectral imaging device to collect hyperspectral data of soil in a target area; step 2, preprocessing the acquired hyperspectral data; step 3, carrying out characteristic wave band screening on the spectral data, and extracting key spectral wave bands related to soil heavy metal pollution; step 4, in combination with actually measured soil heavy metal concentration data, establishing a soil heavy metal pollution prediction model by adopting a machine learning algorithm, and optimizing hyper-parameters; and 5, evaluating the performance of the model by using a cross validation method so as to ensure the prediction precision. The method has the characteristics of high efficiency, accuracy and high adaptability, can be widely applied to the fields of agricultural environment monitoring, industrial pollution site evaluation, mining area treatment and the like, and provides an accurate and convenient technical means for soil pollution monitoring.

Description

Technical Field

[0001] The present invention relates to the field of environmental monitoring and remote sensing technology, and more specifically, to a method and system for inverting heavy metal pollution based on hyperspectral imaging technology. Background Art

[0002] With the rapid development of industrialization and urbanization, the problem of heavy metal pollution has become increasingly serious. Heavy metals such as lead, cadmium, mercury, arsenic, etc. pose great hazards to the environment and human health. Therefore, the monitoring and treatment of heavy metal pollution have become an important task in environmental protection. Traditional heavy metal detection methods usually rely on laboratory chemical analysis. Although they have high precision, they have disadvantages such as long time consumption, high cost, and sample destruction, and it is difficult to meet the needs of large-scale and real-time monitoring. As an emerging remote sensing technology, hyperspectral imaging technology can simultaneously obtain the spatial information and spectral information of the target object, and has advantages such as high resolution, multi-band, and non-contact. In recent years, hyperspectral technology has gradually been applied to the field of environmental monitoring, especially showing great potential in the detection of soil and water pollution. However, existing hyperspectral inversion heavy metal technologies still have problems such as insufficient spectral feature extraction and insufficient model generalization ability, and do not have a complete inversion system.

[0003] Therefore, in order to overcome the above problems, a method and system for inverting heavy metal pollution based on hyperspectral imaging technology are provided, aiming to quickly and non-destructively detect the heavy metal content and its distribution in environmental media such as soil and water. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and system for inverting heavy metal pollution based on hyperspectral imaging technology, including the following steps: 1. Use a hyperspectral imaging device to collect hyperspectral data of the soil in the target area, with a spectral range of 400nm - 2500nm; 2. Preprocess the obtained hyperspectral data, including noise removal, normalization, multiplicative scatter correction (MSC), standard normal variate transformation (SNV), etc.; 3. Use competitive adaptive reweighted sampling (CARS), successive projections algorithm (SPA) or principal component analysis (PCA) to screen the characteristic bands of the spectral data, and extract the key spectral bands related to soil heavy metal pollution; 4. Combine the measured heavy metal concentration data of the soil, and use machine learning algorithms (including but not limited to partial least squares regression PLSR, random forest RF, support vector regression SVR, extreme gradient boosting XGBoost) to establish a soil heavy metal pollution prediction model and optimize the hyperparameters; 5. Use the cross-validation method to evaluate the model performance, and calculate indicators such as the coefficient of determination (R²), root mean square error (RMSE), etc. to ensure the prediction accuracy. The present invention can quickly and non-destructively detect the heavy metal content in environmental media, and solve the problems of insufficient spectral feature extraction, insufficient model generalization ability, and low data processing efficiency in the prior art.

[0005] To achieve the above object, the present invention adopts the following technical solutions: A rapid inversion method for soil heavy metal pollution based on hyperspectral and machine learning, comprising the following specific steps: Step 1: Use a hyperspectral imaging device to collect hyperspectral data of the soil in the target area, with a spectral range of 400nm - 2500nm; Step 2: Preprocess the obtained hyperspectral data; Step 3: Screen the characteristic bands of the spectral data to extract the key spectral bands related to soil heavy metal pollution; Step 4: Combine the measured heavy metal concentration data of the soil, and use a machine learning algorithm to establish a soil heavy metal pollution prediction model and optimize the hyperparameters; Step 5: Use the cross - validation method to evaluate the model performance to ensure the prediction accuracy.

[0006] Preferably, in step 2, the obtained hyperspectral data is preprocessed using methods such as noise removal, normalization, multiplicative scatter correction (MSC), and standard normal variate transformation (SNV).

[0007] Preferably, in step 3, competitive adaptive reweighted sampling (CARS), successive projections algorithm (SPA), or principal component analysis (PCA) is used to screen the characteristic bands of the spectral data.

[0008] Preferably, in step 4, combined with the measured heavy metal concentration data of the soil, machine learning algorithms such as partial least squares regression PLSR, random forest RF, and support vector regression SVR are used to establish a soil heavy metal pollution prediction model.

[0009] In actual research, the random forest adopted is constructed by selecting n data from the training data as the training data input. After selecting the input training data, a decision tree is constructed, and the splitting attribute is determined according to the strategy of decreasing Gini index until it cannot be split or reaches the set threshold. At this time, 1 decision tree is established, and each decision tree grows as much as possible without pruning.

[0010] is the number of samples, is the number of classes in the sample, is the frequency of the class, is the sample appears in.

[0011] Repeat the above steps until the predetermined number of trees is reached. All the generated decision trees form a random forest, which is then used to predict new input data.

[0012] Preferably, in step 5, in order to evaluate the prediction ability and stability of the model, two determining factors were selected: the coefficient of determination (R²) and the root mean square error (RMSE).

[0013] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a method for inverting soil heavy metals based on hyperspectral technology, which breaks through the problems of traditional methods such as relying on laboratory testing, limited selection of characteristic bands, and low model accuracy. By introducing CARS characteristic band screening, multi-model fusion algorithm and data denoising optimization, the accuracy and stability of heavy metal inversion are improved, and the interference of external environmental factors is reduced. At the same time, by combining aerial hyperspectral images with ground measured data, the adaptability and generalization ability of the model are improved. The invention can be widely applied to environmental monitoring, agricultural management and land pollution control, and the effects that can be produced are as follows: (1) Traditional soil heavy metal detection mainly relies on laboratory analysis, which requires a large amount of sampling, chemical treatment and spectral determination, with a long time-consuming and high cost. The present invention uses hyperspectral technology to realize large-scale and rapid inversion monitoring of soil heavy metals. Only remote sensing images or on-site spectral data are required to identify polluted areas, greatly improving the monitoring efficiency. At the same time, this method can dynamically track the spatio-temporal changes of soil pollution, support long-term environmental monitoring and pollution warning, and provide real-time data support for the government and environmental protection departments.

[0014] (2) Traditional spectral analysis methods are often limited by fixed band selection and single models, and it is difficult to adapt to different soil types and environmental conditions. The present invention screens key characteristic bands through CARS (competitive adaptive reweighted sampling), and combines various machine learning algorithms such as PLS and SVM to construct a high-precision and multi-adaptive heavy metal inversion model. This method not only reduces redundant information and improves the prediction accuracy, but also can adapt to the soil characteristics of different regions, making the inversion model have stronger generalization ability. In addition, the present invention adopts hyperspectral data denoising and vegetation and soil moisture correction technologies to reduce the influence of environmental factors on the inversion accuracy, and further improves the stability and application range of the model.

[0015] (3) The present invention provides an efficient and accurate means for soil pollution monitoring for the government and environmental protection departments, enabling the transformation of environmental management from the traditional passive response mode to the active monitoring and prediction and early warning mode. With the aid of this technology, the government can formulate more scientific soil pollution control policies and improve the efficiency of environmental supervision. At the same time, this method can be used to evaluate the environmental impacts of key polluted areas such as mining areas, industrial emission areas, landfills, etc., optimize the pollution remediation plan, and reduce the risk of secondary pollution. In addition, the present invention can also be used to formulate the red line for land pollution, support the improvement and implementation of environmental regulations, and provide technical support for ecological environmental protection and sustainable development. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart for hyperspectral inversion of heavy metals in the present invention.

[0017] Figure 2 It is a spectral preprocessing diagram in the present invention.

[0018] Figure 3 It is a diagram for screening characteristic wavelengths in the present invention.

[0019] Figure 4 It is a flowchart for the random forest model in the present invention.

[0020] Figure 5 It is the model prediction result in the present invention.

[0021] Figure 6 It is a scatter plot of the measured and predicted heavy metal contents in the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0022] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0023] The following will describe the specific implementation manners of the present invention in detail with reference to the accompanying drawings. The purpose of the present invention is to provide a heavy metal pollution inversion method and system based on hyperspectral imaging technology (such as Figure 1), including the following steps: Step 1, collect the hyperspectral data of the soil in the target area using a hyperspectral imaging device; Step 2, preprocess the obtained hyperspectral data; Step 3, screen the characteristic bands of the spectral data to extract the key spectral bands related to soil heavy metal pollution; Step 4, combine the measured heavy metal concentration data of the soil, and use a machine learning algorithm to establish a soil heavy metal pollution prediction model and optimize the hyperparameters; Step 5, use the cross-validation method to evaluate the model performance to ensure the prediction accuracy. This method has the characteristics of high efficiency, accuracy, and strong adaptability, and can be widely applied to fields such as agricultural environmental monitoring, industrial pollution site assessment, and mining area governance, providing a precise and convenient technical means for soil pollution monitoring.

[0024] To achieve the above object, the present invention adopts the following technical solutions: A rapid inversion method for soil heavy metal pollution based on hyperspectral and machine learning, including the following specific steps: Step 1: Collect the hyperspectral data of the soil in the target area using a hyperspectral imaging device, and the spectral range is 400nm - 2500nm; Step 2: Preprocess the obtained hyperspectral data; Step 3: Screen the characteristic bands of the spectral data to extract the key spectral bands related to soil heavy metal pollution; Step 4: Combine the measured heavy metal concentration data of the soil, and use a machine learning algorithm to establish a soil heavy metal pollution prediction model and optimize the hyperparameters; Step 5: Use the cross-validation method to evaluate the model performance to ensure the prediction accuracy.

[0025] Preferably, in the said Step 1, the heavy metals in the soil of S area are selected as the actual research area to verify the effectiveness of the present invention, and the heavy metal content values of the soil in S City are shown in Table 1.

[0026] Table 1 Element Average value Standard deviation Maximum value Minimum value Coefficient of variation Pollution coefficient Zn 157.11 184.85 579 17 1.18 2.34 Cu 31.87 14.02 63.5 7.75 0.44 1.69 Cd 0.25 0.15 0.91 0.05 0.59 2.27 Preferably, in the said Step 2, the obtained hyperspectral data is preprocessed using methods such as noise removal, normalization, multiplicative scatter correction (MSC), standard normal variate transformation (SNV), etc. (such as Figure 2 ).

[0027] Preferably, in the said Step 3, competitive adaptive reweighted sampling (CARS), successive projections algorithm (SPA), or principal component analysis (PCA) is used to screen the characteristic bands of the spectral data (such as Figure 3 ).

[0028] Preferably, in step 4, combined with the measured heavy metal concentration data of the soil, machine learning algorithms such as partial least squares regression (PLSR), random forest (RF), and support vector regression (SVR) are used to establish a soil heavy metal pollution prediction model.

[0029] In actual research, the random forest adopted is constructed by selecting n data from the training data as the training data input. After selecting the input training data, a decision tree is constructed, and the splitting attribute is determined according to the strategy of decreasing Gini index until it cannot be split or reaches the set threshold. At this time, 1 decision tree is established, and each decision tree grows as much as possible without pruning (such as Figure 4 ).

[0030] is the number of samples, is the number of categories in the sample, is the frequency of the category, is the one that appears in the sample in.

[0031] Repeat the above steps until the predetermined number of trees is reached. All the generated decision trees form a random forest, which is then used to predict new input data.

[0032] Preferably, in step 5, in order to evaluate the prediction ability and stability of the model, two determining factors are selected: the coefficient of determination (R²) and the root mean square error (RMSE). The specific evaluation results are shown in Table 2.

[0033] n is the number of samples, is the actual observed value, is the predicted value obtained from the regression model.

[0034] Table 2 RF PLS SVM Training set Test set Training set Test set Training set Test set Cu R² 0.96 0.87 R² 0.25 0.36 R² 0.07 0.01 RMSE 6.08 6.88 RMSE 9.71 9.13 RMSE 13.78 15.63 Zn R² 0.93 0.90 R² 0.06 -1.22 R² 0.04 0.09 RMSE 53.44 57.92 RMSE 270.84 424.91 RMSE 194.78 261.09 Cd R² 0.91 0.87 R² -0.29 0.017 R² 0.71 0.03 RMSE 0.07 0.05 RMSE 29.71 35.40 RMSE 0.09 0.14 By performing Step 1 (data acquisition and preprocessing), Step 2 (feature band extraction), and Step 3 (machine learning modeling), feasible countermeasures and suggestions for soil heavy metal pollution monitoring and control can be obtained: (1) In terms of pollution monitoring and risk assessment, an inversion model for soil heavy metal content is constructed based on hyperspectral data. The prediction accuracy is improved through feature band screening and machine learning methods. The pollution distribution map is drawn using spatial interpolation technology to achieve regional assessment of pollution risks. According to the monitoring results, the pollution areas can be classified into risk levels, divided into slightly, moderately, and severely polluted areas, and differential control strategies can be formulated. (2) In terms of pollution prevention and control and remediation, combined with hyperspectral remote sensing monitoring data, a precise pollution control strategy is implemented. For highly polluted areas, pollution source control measures can be taken, such as strengthening the supervision of industrial enterprises, promoting clean production technologies, and reducing heavy metal emissions. At the same time, by adjusting the land use pattern, the scale of crop cultivation can be reduced in severely polluted areas, and ecological restoration measures such as returning farmland to forest and grassland can be implemented. In addition, a pollution emergency plan can be formulated, and emergency drills can be carried out regularly to improve the response ability to pollution incidents. (3) In terms of monitoring and intelligent management, by building a long-term soil heavy metal monitoring system, monitoring points are set up in polluted areas, and combined with big data, artificial intelligence, and machine learning algorithms, dynamic monitoring and prediction of pollution change trends are achieved. Based on hyperspectral data and GIS systems, an automatic early warning model is constructed, and an intelligent early warning trigger device is installed to achieve real-time monitoring and early warning of pollution risks, ensuring the intelligence and high efficiency of soil safety management.

[0035] In Step 4 (result analysis and application), the effectiveness of Steps 1 to 3 is verified, further verifying the feasibility of the present invention. Based on the above steps, the present invention integrates pollution assessment, risk early warning, and precise control, constructs a full-process system for soil heavy metal pollution monitoring and control, can not only comprehensively evaluate the soil heavy metal pollution status, but also effectively prevent and control pollution risks, providing scientific support for the sustainable development of the ecological environment. The present invention has wide application value in aspects such as soil pollution monitoring, environmental governance, and agricultural sustainable management.

[0036] The various embodiments in this specification are described in a progressive manner. Each embodiment mainly highlights its unique features compared to other embodiments. For the same or similar parts, the embodiments can refer to each other and will not be repeated. For the device part disclosed in the embodiments, since it corresponds to the method embodiments, the description is relatively brief, and the specific details can be referred to the relevant descriptions in the method part.

[0037] Through the detailed description of various embodiments of the present invention, those skilled in the art can understand and implement the present invention. At the same time, without departing from the core idea of the present invention, relevant technicians can appropriately adjust or optimize the implementation manner to meet different application requirements. The general principles defined in this specification can be applied in other embodiments without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention is not limited to the specific embodiments listed in this specification, but should be based on the innovative ideas and technical features it discloses, covering all technical variants and improvement solutions that conform to the core principles of the present invention.

Claims

1. A rapid inversion method for soil heavy metal pollution based on hyperspectral and machine learning, characterized in that: The method includes the following steps: Step 1: Use hyperspectral imaging equipment to collect hyperspectral data of the soil in the target area, with a spectral range of 400nm-2500nm; Step 2: Preprocess the acquired hyperspectral data; Step 3: Screen the characteristic bands of the spectral data and extract the key spectral bands related to soil heavy metal pollution; Step 4: Combined with the measured heavy metal concentration data in the soil, use a machine learning algorithm to establish a soil heavy metal pollution prediction model and optimize the hyperparameters; Step 5: Use the cross-validation method to evaluate the model performance to ensure the prediction accuracy.

2. The method according to claim 1, characterized in that Hyperspectral data collection methods can be ground-based hyperspectral imaging or drone-mounted hyperspectral sensors to obtain remote sensing data.

3. The method according to claim 1, characterized in that The spectral preprocessing includes spectral denoising methods, such as Savitzky-Golay (SG) smoothing and detrending.

4. The method according to claim 1, characterized in that: The feature band screening step can be performed by combining a variety of methods to improve the stability of feature selection and prediction accuracy.

5. The method according to claim 1, characterized in that The machine learning model can be optimized for hyperparameters through grid search or Bayesian optimization.

6. A rapid inversion system for soil heavy metal pollution based on hyperspectral and machine learning, characterized in that: The system includes: Step 1: Hyperspectral data acquisition module, used to collect hyperspectral data of soil in the target area; Step 2: Data preprocessing module, which performs noise reduction, normalization and characteristic band screening on spectral data; Step 3: Machine learning prediction module calculates the soil heavy metal pollution concentration based on the trained prediction model.

7. The system according to claim 6, characterized in that The hyperspectral data acquisition module can be applied to a variety of acquisition devices, including handheld spectrometers, unmanned aerial vehicle hyperspectral sensors, satellite remote sensing, etc.

8. The system according to claim 6, characterized in that The machine learning prediction module can automatically update the training data set to adapt to the heavy metal pollution prediction needs of different regions and soil types.

Images