Soil heavy metal pollution detection method, equipment and system

Abstract

The invention relates to the technical field of soil heavy metal detection, in particular to a soil heavy metal pollution detection method, equipment and system.The method comprises the steps that the abnormal degree of the difference between each spectrum value and the adjacent spectrum value in each window is analyzed, and the first disturbance degree is determined; analyzing the similarity of the spectral value fitting results between each feature window and the other feature windows and the goodness of fit of the spectral value fitting results in each feature window to obtain a second disturbed degree, and obtaining a second disturbed degree on the basis of the dispersion degree of all spectral values under the same wavelength in each spectral signal and all similar spectral signals thereof; and de-noising the spectral signal in combination with the first disturbed degree and the second disturbed degree so as to perform heavy metal detection on the to-be-detected soil sample. According to the method, the problem of reasonability of filtering window setting during denoising of the spectral signal of the to-be-detected soil sample is solved, and the accuracy of soil heavy metal pollution condition detection is improved.

Description

Technical Field

[0001] This application relates to the technical field of soil heavy metal detection, and particularly relates to a method, device and system for detecting soil heavy metal pollution. Background Art

[0002] With the rapid development of industrialization and urbanization, soil heavy metal pollution has become a global environmental problem. Once heavy metals such as lead (Pb), mercury (Hg), cadmium (Cd), chromium (Cr), etc. enter the soil, they are not only difficult to degrade, but also pose a serious threat to human health through the bioamplification effect of the food chain. Therefore, it is particularly important to develop efficient and accurate methods and technologies for detecting soil heavy metal pollution.

[0003] The XRF instrument (X-ray fluorescence spectrometer) is an instrument used to analyze the chemical composition of substances. It excites the sample with X-rays and detects the generated X-ray fluorescence to determine the types and contents of elements in the sample. This type of instrument has the characteristics of being portable, fast in detection speed, and simple in operation, so it is widely used in the detection of soil heavy metal pollution. However, during the analysis process of the XRF instrument, the spectral data is affected not only by the inherent noise of electronic components in the hardware system and the statistical fluctuations of the detector, but also by unstable factors in the external environment and experimental conditions, resulting in large deviations or even inaccuracies in the measurement results. Especially in the measurement of samples with low contents, it may even lead to incorrect results. Therefore, it is necessary to denoise the spectral data. However, the existing filtering algorithms use a fixed window size to denoise the spectral data. If the window is set too large, the soil spectral information may be masked. If the window is set too small, the purpose of denoising cannot be achieved. The setting of the window size in the denoising algorithm will affect the accuracy of soil heavy metal pollution detection. Summary of the Invention

[0004] In a first aspect, an embodiment of this application provides a method for detecting soil heavy metal pollution, which includes the following steps: Obtain multiple spectral signals of the soil sample to be detected; Cluster all spectral values in each spectral signal, divide the results of arranging all spectral values in each clustering cluster according to wavelength into the same window, analyze the abnormal degree of the difference between each spectral value and its adjacent spectral values within each window, and determine the first perturbation degree of each spectral value within each window; Use the lengths of all windows of each spectral value as the input of the threshold segmentation algorithm. Denote the windows with lengths greater than the segmentation threshold as characteristic windows. Fit all spectral values within each characteristic window, and determine the confidence levels of the characteristic windows of each spectral signal by analyzing the similarity of the fitting curves between each characteristic window and the other characteristic windows, as well as the goodness of fit of the fitting curves within each characteristic window, so as to obtain the second perturbation degree of each spectral value in each spectral signal; By comparing the differences in the number of characteristic windows and the differences in characteristic peaks between each spectral signal and the remaining spectral signals, and combining the similarities between each spectral signal and the remaining spectral signals, the similarity coefficient between each spectral signal and the remaining spectral signals is determined to obtain the similar spectral signals of each spectral signal; Based on the dispersion degree of all spectral values at the same wavelength in each spectral signal and its all similar spectral signals, and combining the first perturbation degree and the second perturbation degree, the perturbation degree of each spectral value in each spectral signal is determined to improve the filtering window length during denoising of the spectral signal, and heavy metal detection is performed on the soil sample to be detected based on the denoised spectral signal.

[0005] Preferably, the determination method of the first perturbation degree of each spectral value in each window is as follows: Calculate the mean value of the differences between each spectral value in each window and its adjacent spectral values, use the mean values of all spectral values as the input of the anomaly detection algorithm, output the anomaly scores of the mean values of all spectral values in each window respectively, and take the normalized value of the anomaly scores of the mean values of each spectral value in each window as the first perturbation degree of each spectral value in each window.

[0006] Preferably, the expression of the confidence degree of each characteristic window of each spectral signal is: ; In the formula, represents the confidence degree of the s-th characteristic window of spectral signal A; represents the goodness of fit of the fitting curve in the s-th characteristic window of spectral signal A; represents the similarity of the fitting curves between the s-th characteristic window and the j-th characteristic window of spectral signal A; represents the number of all characteristic windows of spectral signal A; norm( ) represents the normalization function.

[0007] Preferably, the second perturbation degree of each spectral value in each spectral signal is the normalized value of the reciprocal of the confidence degree of the characteristic window where each spectral value is located.

[0008] Preferably, the determination method of the similarity coefficient between each spectral signal and the remaining spectral signals is as follows: Calculate the difference in the central wavelengths of all characteristic windows between each spectral signal and the remaining spectral signals, and use the two characteristic windows corresponding to the central wavelength difference less than the preset threshold as the matching windows between each spectral signal and its respective spectral signals; The similarity coefficient between spectral signal A and spectral signal B is expressed as: ; In the formula, represents the difference in the number of characteristic windows between spectral signal A and spectral signal B; Represents the difference in characteristic peaks in the h-th pair of matching windows between spectral signal A and spectral signal B; Represents the number of all pairs of matching windows between spectral signal A and spectral signal B; norm[ ] represents the normalization function.

[0009] Preferably, the method for obtaining the similar spectral signal of each spectral signal is as follows: Taking the similarity coefficient between each spectral signal and all the other spectral signals as the input of the threshold segmentation algorithm, and outputting the segmentation threshold, denoted as the similarity threshold. Among all the other spectral signals, the spectral signals with a similarity coefficient greater than the similarity threshold are used as the similar spectral signals of each spectral signal.

[0010] Preferably, the method for determining the degree of perturbation of each spectral value in each spectral signal is as follows: Among all the similar spectral signals of each spectral signal, obtaining all the spectral values at the same wavelength as each spectral value in each spectral signal, and calculating the variance of each spectral value and all the spectral values at the same wavelength as it as the third degree of perturbation of each spectral value in each spectral signal; For all the spectral values in all the characteristic windows of each spectral signal, taking the normalized value of the average of the first degree of perturbation, the second degree of perturbation, and the third degree of perturbation of each spectral value as the degree of perturbation of each spectral value in each spectral signal.

[0011] Preferably, improving the filter window length for denoising the spectral signal and detecting soil heavy metal pollution based on the denoised spectral signal includes: The filter window length of the spectral value a in spectral signal A The expression is: ; In the formula, Represents the degree of perturbation of the spectral value a in spectral signal A; n, respectively represent a preset first value and a preset second value, where the preset first value is less than the preset second value; Taking all the spectral values in each spectral signal as the input of the filtering algorithm, where the size of the filter window is set to the filter window length of each spectral value, and outputting the denoised spectral signal; Using the multi-feature concatenation strategy to obtain the characteristic variables in all the denoised spectral signals, and adopting the multivariate statistical data analysis method to build a model based on the characteristic variables to obtain the quantitative analysis model of the heavy metal content of the soil sample, and using the quantitative analysis model of the heavy metal content to detect the soil heavy metal pollution.

[0012] In a second aspect, an embodiment of the present application further provides a soil heavy metal pollution detection device, in which a computer program is stored, and when the computer program is executed by a processor, it implements the soil heavy metal pollution detection method described in any one of the above.

[0013] In a third aspect, an embodiment of the present application provides a soil heavy metal pollution detection system, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of the soil heavy metal pollution detection method described in any one of the above.

[0014] As can be seen from the above embodiments, the soil heavy metal pollution detection method provided by the embodiments of the present application has at least the following beneficial effects: By calculating the abnormality degree of the difference between adjacent spectral values in each window of the spectral signal, the present application constructs a first perturbation degree, which can effectively evaluate the degree of noise interference, and thus accurately denoise the spectral value according to the degree of noise interference received by the spectral value, and further improve the spectral value to improve the accuracy of soil heavy metal pollution detection; further, by analyzing the possibility of including real characteristic peaks in the characteristic window, the present application constructs a second perturbation degree, which helps to determine the true reliability of the spectral value, thereby improving the accuracy of the soil heavy metal pollution detection result; further, by comprehensively considering the first perturbation degree and the second perturbation degree, and combining the dispersion degree of each spectral signal and all spectral values at the same wavelength in all its similar spectral signals, the present application constructs a perturbation degree, which can more comprehensively evaluate the degree of noise interference received by each spectral value, and thus dynamically adjust the filtering window of the spectral value, perform more effective denoising processing on the spectral signal, retain more real spectral information, and thus improve the accuracy of detecting the soil heavy metal content. By comprehensively analyzing the abnormality degree of the difference between adjacent spectral values, the difference between the characteristic peak and the noise, and the similarity of multiple measurement results, the present application can more accurately evaluate and remove the noise interference in the spectral data, and improve the accuracy of detecting the soil heavy metal pollution situation. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for description in the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0016] Figure 1 It is a flowchart of the steps of a soil heavy metal pollution detection method provided by an embodiment of the present application; Figure 2Schematic diagram of the process for obtaining the second disturbance degree provided by an embodiment of the present application. Detailed implementation manners

[0017] In order to further elaborate on the technical means and effects adopted by the present application to achieve the intended invention purpose, the following specifically describes, in conjunction with the accompanying drawings and preferred embodiments, a method, device, and system for detecting soil heavy metal pollution proposed according to the present application, including its specific implementation manners, structures, features, and effects. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments may be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which this application belongs.

[0019] The following specifically describes, in conjunction with the accompanying drawings, the specific solutions of a method, device, and system for detecting soil heavy metal pollution provided by the present application.

[0020] Please refer to Figure 1 , which shows a flowchart of the steps of a method for detecting soil heavy metal pollution provided by an embodiment of the present application. The method includes the following steps: S1: Obtain multiple spectral signals of the soil sample to be detected.

[0021] In environmental monitoring and soil pollution assessment, spectral data analysis is a very effective means. To accurately identify and quantify the heavy metal content in the soil, an X-ray fluorescence (XRF) analyzer is used to detect the soil sample to be detected.

[0022] The reason for using an XRF analyzer to detect the soil sample is that XRF technology can non-destructively determine the elemental composition of the soil. By emitting X-rays to irradiate the soil sample, different elements will emit characteristic fluorescent X-rays, which are received by the detector and converted into spectral signals. These spectral signals contain the characteristic information of the elements and are the key to analyzing the heavy metal content in the soil.

[0023] Soil samples are usually collected from contaminated or suspected contaminated areas. Due to the heterogeneity of the soil, even samples collected at the same location may have different degrees of contamination. Therefore, in order to obtain more representative data, multiple spectral signal collections of the same soil are required. Specifically: use an existing XRF analyzer to detect the soil and obtain multiple groups of spectral signals of the same soil sample.

[0024] S2: Cluster all spectral values in each spectral signal, divide the results of arranging all spectral values in each cluster by wavelength into the same window, analyze the degree of abnormality of the difference between each spectral value and its adjacent spectral values within each window, and determine the first degree of perturbation of each spectral value within each window.

[0025] Spectral signals identify elements based on characteristic peaks, and characteristic peaks are prominent in spectral data. However, characteristic peaks change continuously. Therefore, the influence of noise on spectral signals can be analyzed based on the differences in the changes of adjacent spectral data of characteristic peaks. Since the change speeds of spectral data in the characteristic peak region and other regions are different, for better analysis, this example first classifies spectral signals, and the classification method is as follows: Take all spectral values in each spectral signal as the input of the clustering algorithm, and output multiple clusters. In this embodiment, the results of arranging all spectral values in each cluster in ascending order of wavelength are divided into the same window.

[0026] It should be noted that there are many commonly used clustering algorithms. In this embodiment, the DBSCAN density clustering algorithm is used to cluster all spectral values in each spectral signal. In actual application processes, as other implementation manners, implementers can also use DPC density peak clustering or k-means clustering algorithms to cluster spectral values. Regarding the selection of clustering algorithms, this embodiment does not make special restrictions.

[0027] Among them, the DBSACN density clustering algorithm is a well-known technology, and its specific clustering principle will not be elaborated here.

[0028] Supplementary note: The method for determining the metric distance in the clustering algorithm is as follows: form a binary group with the spectral value and its corresponding wavelength, and use the Euclidean distance between binary groups as the metric distance.

[0029] Among them, the calculation method of the Euclidean distance is a well-known technology, and its specific calculation process will not be elaborated here.

[0030] Furthermore, analyze the degree of abnormality of the difference between each spectral value and its adjacent spectral values within each window, and determine the first degree of perturbation of each spectral value within each window. Specifically: Calculate the mean of the differences between each spectral value and its adjacent spectral values within each window, take the means of all spectral values as the input of the anomaly detection algorithm, output the anomaly scores of the means of all spectral values within each window respectively, and take the normalized value of the anomaly scores of the means of each spectral value within each window as the first degree of perturbation of each spectral value within each window.

[0031] Specifically, for the first and last spectral values within each window, the difference between the first spectral value and the subsequent adjacent spectral value is used as the input to the anomaly detection algorithm, and the difference between the last spectral value and the previous adjacent spectral value is used as the input to the anomaly detection algorithm.

[0032] It should be noted that there are many methods to measure the difference between two data. In this embodiment, the absolute value of the difference between each spectral value within each window and its two adjacent spectral values is used as the difference between each spectral value within each window and its two adjacent spectral values. In actual application processes, as other implementation manners, implementers can also adopt other methods to measure the difference between data, such as the square or ratio of the difference, according to specific situations. Regarding the selection of the method to measure the difference between data, this embodiment does not make special restrictions.

[0033] In addition, it should be understood that there are many commonly used anomaly detection algorithms. In this embodiment, the LOF anomaly detection algorithm is used to obtain the anomaly scores of the means of all spectral values. In actual application processes, as other implementation manners, implementers can also adopt other anomaly detection algorithms, such as the Isolation Forest algorithm. Regarding the selection of the anomaly detection algorithm, this embodiment does not make special restrictions.

[0034] Among them, the LOF anomaly detection algorithm is a well-known technology, and the specific process of using it to evaluate the anomaly score of data will not be elaborated here.

[0035] From the first perturbation degree of each spectral value within each window, it can be understood that the first perturbation degree reflects whether the difference between adjacent spectral values in the spectral signal is abnormal. If the first perturbation degree of the current spectral value is larger, it indicates that the difference between the current spectral value and its adjacent spectral value in the spectral signal is larger. This difference is very likely caused by noise interference, indicating that the spectral value is severely affected by noise interference and its spectral information may have been masked by the noise; On the contrary, if the first perturbation degree of the current spectral value is smaller, it means that the difference between the current spectral value and its adjacent spectral value is smaller, and the current spectral value is less affected by noise, indicating that the current spectral value retains more original signal characteristics. During the denoising process, only slight filtering or no filtering may be required to avoid losing important spectral information.

[0036] Thus far, by analyzing the abnormal degree of the difference between each spectral value within each window and its adjacent spectral value, the first perturbation degree of each spectral value is obtained.

[0037] Step S3: Use all window lengths of each spectral value as the input of the threshold segmentation algorithm. Denote the windows with lengths greater than the segmentation threshold as characteristic windows. Fit all spectral values within each characteristic window. By analyzing the similarity of the fitting curves between each characteristic window and the other characteristic windows, as well as the goodness of fit of the fitting curves within each characteristic window, determine the confidence level of each characteristic window of each spectral signal, so as to obtain the second degree of perturbation of each spectral value in each spectral signal.

[0038] During the analysis process of the XRF instrument, in addition to being affected by the inherent noise of electronic components in the hardware system and the statistical fluctuations of the detector, the spectral data is also affected by unstable factors in the external environment and experimental conditions, resulting in large deviations or even inaccuracies in the measurement results, especially in the measurement of samples with low contents.

[0039] Among them, the noise data will make the signal protrude abnormally. Since the window where it is located has a small length due to the clustering in step S2, and the signal that is more prominent and has a smaller window may also be the characteristic peak of the spectral data, it is necessary to analyze the information within each window. By performing clustering analysis on the window length, the abnormal degree of the signal within the window can be confirmed. Since the characteristic peak conforms to the characteristics of the peak, when the adjacent signals change, they will also conform to the characteristics of the "peak", while for the signals with a large degree of noise interference, their signal changes do not conform to the characteristics of the characteristic peak.

[0040] Therefore, based on the above analysis, first, use all window lengths of each spectral value as the input of the threshold segmentation algorithm. Denote the windows with lengths greater than the segmentation threshold as characteristic windows, which are used to represent the windows where the characteristic peaks are suspected to be located.

[0041] It should be noted that there are many commonly used threshold segmentation algorithms. In this embodiment, the maximum inter-class variance algorithm is used to obtain the segmentation threshold. In the actual application process, as other implementation manners, the implementer can also use other threshold segmentation methods. Regarding the selection of the threshold segmentation method, this embodiment does not make special restrictions.

[0042] Among them, the maximum inter-class variance algorithm is a well-known technology, and its specific principle will not be elaborated here.

[0043] It is further explained that in this embodiment, whenever threshold segmentation content is involved, the maximum inter-class variance algorithm is used.

[0044] Secondly, by analyzing the similarity of the fitting results of spectral values between each characteristic window and the other characteristic windows, as well as the goodness of fit of the spectral value fitting process within each characteristic window, determine the confidence level of each characteristic window of each spectral signal, so as to obtain the second degree of perturbation of each spectral value in each spectral signal. The specific process is as follows: (1)Fit all the spectral values within each feature window to obtain the fitting curve of each feature window. In this embodiment, the polynomial function fitting method is used to fit all the spectral values. In actual application processes, as other implementation manners, the implementer can also use other fitting methods such as the least squares method according to specific situations. Regarding the selection of the fitting method, this embodiment does not make special restrictions.

[0045] Among them, the polynomial function fitting is a well-known technology, and the specific process of using it to fit the spectral values will not be elaborated here.

[0046] (2)Furthermore, by analyzing the similarity between the fitting curves of each feature window and the fitting curves of the remaining feature windows, as well as the goodness of fit of the fitting curves within each feature window, determine the confidence level of each feature window of each spectral signal, specifically as follows: In this embodiment, as an implementation manner, the confidence level of the s-th feature window of spectral signal A is expressed as: ; in the formula, represents the goodness of fit of the fitting curve in the s-th feature window of spectral signal A; represents the similarity between the fitting curves of the s-th feature window and the j-th feature window of spectral signal A;

[0047]

[0048]

[0049]

[0050] It should be noted that there are many methods to measure the similarity of fitting curves. In this embodiment, the reciprocal of the DTW distance between the fitting curves of the s-th feature window and the j-th feature window of spectral signal A is used as the similarity between the fitting curves of the s-th feature window and the j-th feature window of spectral signal A. In actual application processes, the implementer can also use the cosine similarity or the reciprocal of the Euclidean distance to measure the similarity between the fitting curves. Regarding the selection of the method for measuring the similarity, this embodiment does not make special restrictions.

[0050] Among them, the calculation method of the DTW distance and the calculation method of the goodness of fit are both well-known technologies, and their specific calculation processes will not be elaborated here. It should be supplemented that in this embodiment, for all contents related to calculating the similarity, the reciprocal of the DTW distance is used.It can be understood from the confidence levels of the respective characteristic windows of each spectral signal that the confidence level reflects the credibility of the characteristic window being a window containing a characteristic peak. If the similarity of the fitting curves between the s-th characteristic window and the j-th characteristic window is greater, it indicates that the spectral values between the s-th characteristic window and the j-th characteristic window may all be generated by the spectral emission of the same specific element. Moreover, the greater the goodness of fit of the fitting curve in the s-th characteristic window, the greater the likelihood that the s-th characteristic window contains a characteristic peak, and the greater the finally obtained confidence level, indicating that the s-th characteristic window is more likely to contain a characteristic peak, rather than being an illusion caused by noise or other interference; Conversely, if the similarity of the fitting curves between the s-th characteristic window and the j-th characteristic window is smaller, it indicates that the spectral values between the s-th characteristic window and the j-th characteristic window may be generated by the spectral emissions of different specific elements. Also, the smaller the goodness of fit of the fitting curve in the s-th characteristic window, the smaller the likelihood that the s-th characteristic window contains a characteristic peak, and the smaller the finally obtained confidence level, indicating that the s-th characteristic window is less likely to contain a characteristic peak, and the peak value of the suspected characteristic peak in the characteristic window is more likely to be caused by noise or other interference.

[0051] (3) Further, based on the confidence levels of the respective characteristic windows of each spectral signal, to obtain the second perturbation degree of each spectral value in each spectral signal, specifically: Take the normalized value of the reciprocal of the confidence level of the characteristic window where each spectral value is located as the second perturbation degree of each spectral value in each spectral signal, which reflects the likelihood that the characteristic window where the spectral value is located contains a true characteristic peak, and quantifies the likelihood that the spectral value is affected by noise interference. If the second perturbation degree is greater, it means that the likelihood that the characteristic window where the spectral value is located contains a true characteristic peak is smaller, and the likelihood that the signal in the characteristic window where the spectral value is located is affected by noise or other interference is greater, indicating that the spectral value is more severely affected by noise interference and the authenticity of its signal is lower; Conversely, if the second perturbation degree is smaller, it means that the likelihood that the characteristic window where the spectral value is located contains a true characteristic peak is greater, and the likelihood that the signal in the characteristic window where the spectral value is located is affected by noise or other interference is smaller, indicating that the spectral value is less affected by noise interference and the authenticity of its signal is higher.

[0052] Preferably, the schematic diagram of the process for obtaining the second perturbation degree provided in this embodiment is as Figure 2 shown.

[0053] Thus far, by analyzing the fitting differences of the spectral values between each characteristic window and the remaining characteristic windows, the second perturbation degree of each spectral value has been obtained, quantifying the degree of noise interference suffered by each spectral value in the spectral signal.

[0054] Step S4: Determine the similarity coefficient between each spectral signal and the rest of the spectral signals by comparing the differences in the number of characteristic windows and the differences in characteristic peaks between each spectral signal and the rest of the spectral signals, and combining the similarity between each spectral signal and the rest of the spectral signals, so as to obtain the similar spectral signals of each spectral signal.

[0055] Since the soil sample is fixed, when it is measured and analyzed multiple times, the spectral data obtained by the measurement may be similar, while the interference of noise is random. Therefore, when performing denoising analysis, it is possible to analyze by combining the spectral signals similar to the current spectral signal in the multiple measurement results, so as to determine the degree of interference of the current spectral signal by noise. Therefore, by comparing the differences in the number of characteristic windows and the differences in characteristic peaks between each spectral signal and the rest of the spectral signals, and combining the similarity between each spectral signal and the rest of the spectral signals, determine the similarity coefficient between each spectral signal and the rest of the spectral signals, so as to obtain the similar spectral signals of each spectral signal. Specifically: Calculate the difference in the central wavelength of all characteristic windows between each spectral signal and the rest of the spectral signals, and use the two characteristic windows corresponding to the central wavelength difference less than the preset threshold as the matching windows between each spectral signal and its spectral signals. It should be noted that the central wavelength is the median of all wavelengths in the characteristic window.

[0056] Furthermore, based on the differences in the number of characteristic windows and the differences in characteristic peaks between each spectral signal and the rest of the spectral signals, and combining the similarity between each spectral signal and the rest of the spectral signals, determine the similarity coefficient between each spectral signal and the rest of the spectral signals. Specifically: In this embodiment, as an implementation manner, the similarity coefficient between spectral signal A and spectral signal B The expression is: ; In the formula, represents the difference in the number of characteristic windows between spectral signal A and spectral signal B; represents the difference in characteristic peaks in the h-th pair of matching windows between spectral signal A and spectral signal B; represents the number of all pairs of matching windows between spectral signal A and spectral signal B; norm[ ] represents the normalization function.

[0057] Among them, take the reciprocal of the DTW distance between spectral signal A and spectral signal B as the similarity between spectral signal A and spectral signal B.

[0058] It should be noted that the characteristic peak is a well-known technology, and its specific principle and concept will not be elaborated here.

[0059] According to the similarity coefficient between each spectral signal and the other spectral signals, it can be understood that the similarity coefficient reflects the similarity between two spectral signals. If the similarity coefficient is larger, it means that the similarity between the two spectral signals is higher, that is, they are closer in the number of characteristic windows, the position and shape of characteristic peaks, which means that the possibility of similar noise interference patterns of the two spectral signals is greater, therefore, they show higher consistency in the spectrum; On the contrary, if the similarity coefficient is smaller, it means that the similarity between the two spectral signals is lower, that is, they are more different in the number of characteristic windows, the position and shape of characteristic peaks, which means that the possibility of the two spectral signals being subject to similar noise interference patterns is smaller. Therefore, they show lower similarity in the spectrum and are not suitable for mutual verification.

[0060] Furthermore, based on the similarity coefficient obtained above, similar spectral signals of the spectral signals are obtained, specifically: the similarity coefficient between each spectral signal and all other spectral signals is used as the input of the threshold segmentation algorithm, and the segmentation threshold is output, which is recorded as the similarity threshold. Among all other spectral signals, the spectral signal with a similarity coefficient greater than the similarity threshold is used as the similar spectral signal of each spectral signal.

[0061] So far, by analyzing the similarity of spectral information between spectral signals, similar spectral signals of each spectral signal are obtained.

[0062] Step S5: Based on the degree of dispersion of all spectral values ​​at the same wavelength in each spectral signal and all similar spectral signals thereof, and in combination with the first disturbance degree and the second disturbance degree, the disturbance degree of each spectral value in each spectral signal is determined to improve the filter window length when denoising the spectral signal, and heavy metal detection is performed on the soil sample to be detected based on the denoised spectral signal.

[0063] Based on the degree of dispersion of all spectral values ​​at the same wavelength in each spectral signal and all similar spectral signals thereof, and in combination with the first disturbance degree obtained in step S2 and the second disturbance degree obtained in step S3, the disturbance degree of each spectral value in each spectral signal is determined, specifically: in all similar spectral signals of each spectral signal, all spectral values ​​at the same wavelength as each spectral value in each spectral signal are obtained, and the variance of each spectral value and all spectral values ​​at the same wavelength is calculated as the third disturbance degree of each spectral value in each spectral signal; For all spectral values ​​in all characteristic windows of each spectral signal, a normalized value of the average of the first disturbance degree, the second disturbance degree and the third disturbance degree of each spectral value is used as the disturbance degree of each spectral value in each spectral signal.

[0064] It can be understood from the degree of perturbation of each spectral value in each spectral signal that the degree of perturbation reflects the comprehensive situation of the spectral value being disturbed by noise. If the degree of perturbation of the current spectral value is larger, it indicates that the current spectral value is more severely disturbed by noise, which may lead to incorrect judgment or underestimation of soil components; conversely, if the degree of perturbation of the current spectral value is smaller, it indicates that the spectral value is less disturbed by noise, the current spectral value is closer to the true spectral signal, has higher reliability, and helps to more accurately evaluate the components and pollutant content in the soil sample.

[0065] Thus, by comprehensively considering the first degree of perturbation and the second degree of perturbation, and combining the dispersion degree of all spectral values at the same wavelength in each spectral signal and all its similar spectral signals, the degree of perturbation of the spectrum is obtained, which quantifies the degree of noise interference for each spectral value in the spectral signal.

[0066] Furthermore, based on the degree of perturbation of each spectral value in each spectral signal, the filter window length for denoising the spectral signal is improved, and heavy metal detection is performed on the soil sample to be detected based on the denoised spectral signal. Specifically: The filter window length of spectral value a in spectral signal A The expression is: ; In the formula, represents the degree of perturbation of spectral value a in spectral signal A; n, respectively represent a preset first value and a preset second value, where the preset first value is less than the preset second value.

[0067] It should be noted that in this embodiment, the preset first value is taken as 5 and the preset second value is taken as 99. This is because the preset first value is usually set as a smaller number to ensure that when the degree of perturbation of the spectral value is low, the length of the filter window will not be too large. A smaller filter window helps to retain more spectral details, especially for spectral values less affected by noise, so as to avoid over-smoothing the signal and thus retain the fine structure of the characteristic peak; the preset second value is usually set as a larger number to increase the length of the filter window when the degree of perturbation of the spectral value is high. A larger filter window helps to better smooth those spectral values more affected by noise and reduce the influence of noise on the signal; the implementer can also set it according to specific circumstances, and this embodiment does not make special restrictions.

[0068] Furthermore, all spectral values in each spectral signal are used as the input of the filtering algorithm. Among them, the size of the filter window is set to the filter window length of each spectral value, and the denoised spectral signal is output; It should be noted that in this embodiment, the Gaussian filtering algorithm is used to denoise the spectral signal. The implementer can also use other filtering methods such as median filtering. Regarding the selection of the filtering algorithm, this embodiment does not make special restrictions.

[0069] Among them, the Gaussian filtering algorithm is a well-known technology, and its specific filtering principle will not be elaborated here.

[0070] Furthermore, a multi-feature concatenation strategy is used to obtain the characteristic variables in all denoised spectral signals, and a multivariate statistical data analysis method is adopted to build a model based on the characteristic variables, so as to obtain a quantitative analysis model for the heavy metal content of soil samples, which is used for detecting the heavy metal content of soil samples.

[0071] It should be added that the multi-feature concatenation strategy in this embodiment adopts the interval combination optimization algorithm (ICO). In the actual application process, the implementer can also adopt other methods such as the competitive adaptive reweighted sampling method (CARS) or the successive projections algorithm (SPA) according to the specific situation. In addition, regarding the multivariate statistical data analysis method, the partial least squares method (PLS) is adopted in this embodiment. As other implementation manners, the implementer can also adopt other multivariate statistical data analysis methods such as the multiple regression method according to the specific situation. Regarding the selection of the multi-feature concatenation strategy and the multivariate statistical data analysis method, no special limitation is made in this embodiment.

[0072] Among them, both the interval combination optimization algorithm (ICO) and the partial least squares method (PLS) are well-known technologies, and the specific processes of obtaining characteristic variables by using the interval combination optimization algorithm (ICO) and building a model by using the partial least squares method (PLS) will not be elaborated here.

[0073] So far, in this embodiment, by comprehensively analyzing the differences between adjacent spectral values, the abnormality degree of the signals within the window, and the similarity of multiple measurement results, the noise interference in the spectral data is accurately evaluated and removed, and the reliability of the spectral values is improved. Furthermore, during the denoising process, special attention is paid to the retention of spectral characteristic peaks to ensure that the key information for element identification and quantitative analysis is not mis-denoised, thereby improving the usability of the data and the accuracy of the analysis. Further, through the similarity analysis and comprehensive denoising of multiple groups of measurement data, the consistency and stability between different measurement results are enhanced, and the fluctuations of the analysis results caused by the contingency of single measurement or noise influence are avoided, ensuring the stability and reliability of the analysis results. Furthermore, the noise interference in the spectral data is accurately evaluated and removed, and the accuracy of the detection results of the heavy metal content in the soil is improved.

[0074] Based on the same inventive concept as the above method, an embodiment of the present application also provides a soil heavy metal pollution detection device, in which a computer program is stored, and when the computer program is executed by a processor, it implements the soil heavy metal pollution detection method described in any one of the above.

[0075] Based on the same inventive concept as the above method, an embodiment of the present application further provides a soil heavy metal pollution detection system, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of any one of the above-mentioned soil heavy metal pollution detection methods are implemented.

[0076] It should be noted that: the above sequence of embodiments of the present application is only for description and does not represent the superiority or inferiority of the embodiments. And the above description of specific embodiments of this specification is given. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0077] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments.

[0078] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the principles of the present application shall be included within the protection scope of the present application.

Claims

1. A method for detecting heavy metal pollution in soil, characterized in that, The method includes the following steps: Obtain multiple spectral signals of the soil sample to be detected; Cluster all spectral values in each spectral signal, divide the results of arranging all spectral values in each clustering cluster according to wavelength into the same window, analyze the abnormal degree of the difference between each spectral value and its adjacent spectral value in each window, and determine the first perturbation degree of each spectral value in each window; Take the window lengths of all spectral values of each spectral value as the input of the threshold segmentation algorithm, mark the windows with lengths greater than the segmentation threshold as characteristic windows, fit all spectral values in each characteristic window, and determine the confidence of each characteristic window of each spectral signal by analyzing the similarity of the fitting curves between each characteristic window and the other characteristic windows, as well as the goodness of fit of the fitting curves in each characteristic window, so as to obtain the second perturbation degree of each spectral value in each spectral signal; Determine the similarity coefficient between each spectral signal and the other spectral signals by comparing the differences in the number of characteristic windows and the differences in characteristic peaks between each spectral signal and the other spectral signals, and combining the similarity between each spectral signal and the other spectral signals, so as to obtain the similar spectral signals of each spectral signal; Based on the dispersion degree of all spectral values at the same wavelength in each spectral signal and its all similar spectral signals, and combining the first perturbation degree and the second perturbation degree, determine the perturbation degree of each spectral value in each spectral signal, so as to improve the filtering window length during denoising of the spectral signal, and perform heavy metal detection on the soil sample to be detected based on the denoised spectral signal.

2. The soil heavy metal pollution detection method according to claim 1, wherein The method for determining the first perturbation degree of each spectral value in each window is as follows: Calculate the mean value of the difference between each spectral value and its adjacent spectral value in each window, take the mean values of all spectral values as the input of the anomaly detection algorithm, output the anomaly scores of the mean values of all spectral values in each window respectively, and take the normalized value of the anomaly score of the mean value of each spectral value in each window as the first perturbation degree of each spectral value in each window.

3. The soil heavy metal pollution detection method according to claim 1, characterized in that The expression for the confidence level of each characteristic window of each spectral signal is as follows: ; where represents the confidence level of the s-th characteristic window of spectral signal A; represents the goodness of fit of the fitted curve in the s-th characteristic window of spectral signal A; represents the similarity of the fitted curves between the s-th characteristic window and the j-th characteristic window of spectral signal A; represents the number of all characteristic windows of spectral signal A; norm( ) represents the normalization function.

4. The soil heavy metal pollution detection method according to claim 1, wherein The second perturbation degree of each spectral value in each spectral signal is the normalized value of the reciprocal of the confidence of the characteristic window where the spectral value is located.

5. The soil heavy metal pollution detection method according to claim 1, characterized in that, The method for determining the similarity coefficient between each spectral signal and the other spectral signals is as follows: Calculate the difference in the central wavelengths of all characteristic windows between each spectral signal and the other spectral signals, and take the two characteristic windows with the difference in the central wavelengths less than the preset threshold as the matching windows between each spectral signal and the other spectral signals; The similarity coefficient between spectral signal A and spectral signal B is expressed as: ; where represents the number difference of characteristic windows between spectral signal A and spectral signal B; represents the difference of characteristic peaks in the h-th pair of matching windows between spectral signal A and spectral signal B; represents the number of all pairs of matching windows between spectral signal A and spectral signal B; norm[ ] represents the normalization function.

6. The soil heavy metal pollution detection method according to claim 1, characterized in that, The method for obtaining the similar spectral signals of each spectral signal is as follows: Take the similarity coefficients between each spectral signal and all the other spectral signals as the input of the threshold segmentation algorithm, output the segmentation threshold, denoted as the similarity threshold, and in all the other spectral signals, take the spectral signals with similarity coefficients greater than the similarity threshold as the similar spectral signals of each spectral signal.

7. The soil heavy metal pollution detection method according to claim 1, characterized in that, The method for determining the perturbation degree of each spectral value in each spectral signal is as follows: Among all the similar spectral signals of each spectral signal, obtain all the spectral values at the same wavelength as each spectral value in each spectral signal, and calculate the variance of each spectral value and all the spectral values at the same wavelength as it, as the third perturbation degree of each spectral value in each spectral signal; For all the spectral values within all the characteristic windows of each spectral signal, take the normalized value of the average of the first perturbation degree, the second perturbation degree, and the third perturbation degree of each spectral value as the perturbation degree of each spectral value in each spectral signal.

8. The soil heavy metal pollution detection method according to claim 1, characterized in that The method for improving the filtering window length when denoising the spectral signal and detecting soil heavy metal pollution based on the denoised spectral signal includes: The filtering window length of the spectral value a in the spectral signal A The expression is as follows: ; In the formula, represents the degree of perturbation of the spectral value a in the spectral signal A; n, respectively represent a preset first value and a preset second value, where the preset first value is less than the preset second value; Take all the spectral values in each spectral signal as the input of the filtering algorithm, where the size of the filtering window is set to the filtering window length of each spectral value, and output the denoised spectral signal; Use the multi-feature concatenation strategy to obtain the characteristic variables in all the denoised spectral signals, and adopt the multivariate statistical data analysis method to build a model based on the characteristic variables to obtain a quantitative analysis model of the heavy metal content of the soil sample, and use the quantitative analysis model of the heavy metal content to detect soil heavy metal pollution.

9. A soil heavy metal pollution detection device, in which a computer program is stored, characterized in that, When the computer program is executed by the processor, it implements a method for detecting soil heavy metal pollution according to any one of claims 1-8.

10. A soil heavy metal pollution detection system, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements a method for detecting soil heavy metal pollution according to any one of claims 1-8.

Images