Surface water quality detection method and device based on spectrum method

Abstract

The invention discloses an earth surface water quality detection method and device based on a spectrum method, and relates to the technical field of water quality detection. Acquiring an ultraviolet-visible absorption spectrum corresponding to the target detection area within a period of time, preprocessing the ultraviolet-visible absorption spectrum to obtain initial spectrums, acquiring all the initial spectrums to perform spectrum type recognition, and if the types of the initial spectrums are recognized as normal spectrums, removing abnormal spectrums in the initial spectrum set to obtain a second spectrum set; substituting the target spectrum into a preset model to obtain a target reference value; and calculating an average value of all the target reference values to obtain a target comparison value, and substituting the target comparison value into a preset database to obtain a detection result. The collected spectrum is preprocessed and then subjected to type identification, so that noise interference is effectively removed, normal spectrum data is quickly screened out, and the overall detection efficiency is improved; the target spectrum is substituted into the preset model, the target reference value is calculated, the detection result is further obtained according to database comparison, and the reliability of the detection result is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of water quality detection, and specifically relates to a method and device for detecting surface water quality based on spectroscopy. Background Art

[0002] Surface water is an important source of water for human life and production activities. However, with the acceleration of industrialization and the increase in population, water pollution is becoming increasingly serious. The discharge of pollutants, especially organic pollutants, has become an important factor affecting water quality. Chemical oxygen demand (COD), as one of the important indicators to measure the degree of water pollution, is widely used in surface water pollution monitoring.

[0003] Ultraviolet-visible (UV-Vis) spectroscopy has become an effective technology for detecting water pollution due to its high sensitivity, rapidity, non-destructiveness and environmental friendliness. This method can obtain pollution information in water bodies by analyzing the absorption characteristics of light of different wavelengths in water samples, especially in the detection of organic pollutants, showing good application prospects. However, the quality of surface water is affected by many factors, including hydrological fluctuations caused by rainfall and flow changes. These fluctuations can cause the spectral characteristics of water bodies to change over time, thus posing a challenge to the accurate identification of polluted water spectra.

[0004] Patent CN109459399B discloses a spectral water quality COD and turbidity detection method, which obtains a number of spectral characteristic curves affected only by turbidity, and fits them after pretreatment to obtain a turbidity correlation model F1; obtains a number of spectral characteristic curves affected only by turbidity and COD, and divides them into spectral band data sets when turbidity is different values ​​after pretreatment, and fits each set separately, so as to obtain a COD correlation model when turbidity is different values, and then fits each COD correlation model and the corresponding turbidity, so as to obtain an associated turbidity and COD correlation model F2; according to the spectral characteristic curve of the water sample to be detected, the turbidity value is calculated using the turbidity correlation model F1, and the COD value is calculated based on the turbidity using the correlation model F2. However, this scheme does not take into account that the changes in hydrological fluctuations and the confusion of polluted water spectra will interfere with the accurate detection of COD, thereby causing false alarms or omissions, and reducing the reliability of the detection results. Summary of the invention

[0005] The purpose of the present invention is to solve the above-mentioned problem and to propose a method and device for detecting surface water quality based on spectroscopy.

[0006] In the first aspect of the present invention, a surface water quality detection method based on spectroscopy is first proposed, the method comprising: Acquire the ultraviolet-visible absorption spectrum corresponding to the target detection area within a period of time to obtain a first spectrum set; For each ultraviolet-visible absorption spectrum in the first spectrum set, preprocess the ultraviolet-visible absorption spectrum to obtain an initial spectrum, and obtain all the initial spectra to obtain an initial spectrum set; Performing spectrum type identification according to the initial spectrum set, and if the type is identified as a normal spectrum, removing abnormal spectra from the initial spectrum set to obtain a second spectrum set; Substituting the target spectrum into the preset model to obtain a target reference value; the target spectrum is any one of the second spectrum set; The average value of all target reference values ​​is calculated to obtain a target comparison value, and the target comparison value is substituted into a preset database to obtain a detection result.

[0007] Optionally, preprocessing the UV-visible absorption spectrum to obtain an initial spectrum includes: Performing discrete wavelet transform on the ultraviolet-visible absorption spectrum to obtain coefficients corresponding to different frequencies to obtain a target coefficient set; the target coefficient set includes a low-frequency coefficient set and a high-frequency coefficient set; For each high frequency coefficient in the high frequency coefficient set, the high frequency coefficient is processed by a threshold function to obtain a target high frequency coefficient; All target high-frequency coefficients are obtained to obtain a target high-frequency coefficient set, and an ultraviolet-visible absorption spectrum is reconstructed according to the target high-frequency coefficient set and the low-frequency coefficient set to obtain an initial spectrum corresponding to the ultraviolet-visible absorption spectrum.

[0008] Optionally, performing spectrum type identification according to the initial spectrum set includes: For each initial spectrum in the initial spectrum set, obtaining a detection parameter vector corresponding to the initial spectrum; Constructing a detection matrix according to the detection parameter vectors corresponding to all the initial spectra, and marking each parameter value corresponding to each row in the detection matrix with a serial number from large to small to obtain a serial number matrix; For each column in the sequence number matrix, reorder according to the sequence number to obtain a target ordering set corresponding to each column; Calculate the ranking difference corresponding to each detection parameter in the target ranking set and the preset ranking set to obtain a target difference value; The target value is obtained according to the sum of all target difference values ​​corresponding to the column. If the target value is less than the preset threshold, the target value is recorded as a valid value; The ratio of the number of effective values ​​to the number of target values ​​is calculated to obtain a target ratio. If the target ratio is greater than a preset ratio, the type of the initial spectrum set is identified as a normal spectrum.

[0009] Optionally, the preset model includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, a preset attention module, a preset fusion module, a global average pooling layer, a flattening layer and a linear layer, and substituting the initial spectrum into the preset model to obtain a target reference value includes: Acquire the initial spectrum, substitute the initial spectrum into a first convolution block to obtain a first convolution image, and substitute the first convolution image into a second convolution block to obtain a second convolution image; Substituting the second convolution image into a preset attention module to obtain a first attention image, and substituting the first attention image into a preset fusion module to obtain a first fused image; Substituting the first fused image into the third convolution block to obtain a third convolution image, and substituting the third convolution image into the preset fusion module to obtain a second fused image; Substituting the second fused image into the fourth convolution block to obtain a fourth convolution image, and substituting the fourth convolution image into the preset fusion module to obtain a third fused image; Substituting the third fused image into the fifth convolution block to obtain a fifth convolution image, and substituting the fifth convolution image into the preset attention module to obtain a second attention image; The second attention image is substituted into the global average pooling layer and then passed through the flattening layer and the linear layer in sequence to obtain the reference value.

[0010] Optionally, the preset attention module includes a first attention module and a second attention module; The first attention module receives the convolution image and performs attention extraction on the convolution image to obtain a first attention extraction image; The first attention extraction image and the convolution image are weightedly fused to obtain a first attention fusion image; The second attention module receives the first attention fusion image and performs attention extraction on the first attention fusion image to obtain a second attention extraction image; The second attention extraction image and the first attention extraction image are weightedly fused to obtain an attention image.

[0011] In a second aspect of the present invention, a surface water quality detection device based on spectroscopy is provided, comprising: A spectrum acquisition module, used to obtain the ultraviolet-visible absorption spectrum corresponding to the target detection area within a period of time to obtain a first spectrum set; A spectrum preprocessing module, for preprocessing each ultraviolet-visible absorption spectrum in the first spectrum set to obtain an initial spectrum, and obtaining all the initial spectra to obtain an initial spectrum set; a spectrum type identification module, configured to identify the spectrum type according to the initial spectrum set, and if the type is identified as a normal spectrum, remove the abnormal spectrum from the initial spectrum set to obtain a second spectrum set; A target reference value determination module, used for substituting a target spectrum into a preset model to obtain a target reference value; the target spectrum is any one of the second spectrum set; The detection result determination module is used to calculate the average value of all target reference values ​​to obtain a target comparison value, and substitute the target comparison value into a preset database to obtain a detection result.

[0012] Optionally, the spectrum preprocessing module includes: A spectrum decomposition module, used for performing discrete wavelet transform decomposition on the ultraviolet visible absorption spectrum to obtain coefficients corresponding to different frequencies to obtain a target coefficient set; the target coefficient set includes a low-frequency coefficient set and a high-frequency coefficient set; A high frequency coefficient determination module, for processing each high frequency coefficient in the high frequency coefficient set by a threshold function to obtain a target high frequency coefficient; The spectrum reconstruction module is used to obtain all target high-frequency coefficients to obtain a target high-frequency coefficient set, and reconstruct the ultraviolet-visible absorption spectrum according to the target high-frequency coefficient set and the low-frequency coefficient set to obtain an initial spectrum corresponding to the ultraviolet-visible absorption spectrum.

[0013] Optionally, the spectrum type recognition module includes: A detection parameter vector determination module, used for obtaining, for each initial spectrum in the initial spectrum set, a detection parameter vector corresponding to the initial spectrum; A serial number matrix generation module is used to construct a detection matrix according to the detection parameter vectors corresponding to all the initial spectra, and to mark each parameter value corresponding to each row in the detection matrix with a serial number from large to small to obtain a serial number matrix; A sorting module, used for re-sorting each column in the sequence number matrix according to the sequence number to obtain a target sorting set corresponding to each column; A target difference determination module is used to calculate the ranking difference corresponding to each detection parameter in the target ranking set and the preset ranking set to obtain a target difference; A difference summing module is used to obtain a target value according to the sum of all target difference values ​​corresponding to the column. If the target value is less than a preset threshold, the target value is recorded as a valid value; The type determination module is used to calculate the ratio of the number of effective values ​​to the number of target values ​​to obtain a target ratio. If the target ratio is greater than a preset ratio, the type of the initial spectrum set is identified as a normal spectrum.

[0014] Optionally, the preset model includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, a preset attention module, a preset fusion module, a global average pooling layer, a flattening layer and a linear layer, and substituting the initial spectrum into the preset model to obtain a target reference value includes: Acquire the initial spectrum, substitute the initial spectrum into a first convolution block to obtain a first convolution image, and substitute the first convolution image into a second convolution block to obtain a second convolution image; Substituting the second convolution image into a preset attention module to obtain a first attention image, and substituting the first attention image into a preset fusion module to obtain a first fused image; Substituting the first fused image into the third convolution block to obtain a third convolution image, and substituting the third convolution image into the preset fusion module to obtain a second fused image; Substituting the second fused image into the fourth convolution block to obtain a fourth convolution image, and substituting the fourth convolution image into the preset fusion module to obtain a third fused image; Substituting the third fused image into the fifth convolution block to obtain a fifth convolution image, and substituting the fifth convolution image into the preset attention module to obtain a second attention image; The second attention image is substituted into the global average pooling layer and then passed through the flattening layer and the linear layer in sequence to obtain the reference value.

[0015] Optionally, the preset attention module includes a first attention module and a second attention module; The first attention module receives the convolution image and performs attention extraction on the convolution image to obtain a first attention extraction image; The first attention extraction image and the convolution image are weightedly fused to obtain a first attention fusion image; The second attention module receives the first attention fusion image and performs attention extraction on the first attention fusion image to obtain a second attention extraction image; The second attention extraction image and the first attention extraction image are weightedly fused to obtain an attention image.

[0016] Beneficial effects of the present invention: The present invention proposes a surface water quality detection method based on spectroscopy, which obtains the ultraviolet-visible absorption spectrum corresponding to the target detection area within a period of time to obtain a first spectrum set; for each ultraviolet-visible absorption spectrum in the first spectrum set, the ultraviolet-visible absorption spectrum is preprocessed to obtain an initial spectrum, and all initial spectra are obtained to obtain an initial spectrum set; spectrum type identification is performed according to the initial spectrum set, and if the type identification is a normal spectrum, the abnormal spectrum in the initial spectrum set is removed to obtain a second spectrum set; the target spectrum is substituted into a preset model to obtain a target reference value; the target spectrum is any one of the second spectrum set; the average value of all target reference values ​​is calculated to obtain a target comparison value, and the target comparison value is substituted into a preset database to obtain a detection result. By preprocessing the collected spectrum and then performing type identification, noise interference is effectively removed, thereby quickly screening out normal spectral data, and improving the overall detection efficiency; by substituting the target spectrum into a preset model, calculating the target reference value, and further obtaining the detection result according to the database comparison, the reliability of the detection result is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present invention will be further described below in conjunction with the accompanying drawings.

[0018] Figure 1 A flow chart of a method for detecting surface water quality based on spectroscopy is provided for an embodiment of the present invention; Figure 2 A schematic structural diagram of a surface water quality detection device based on spectroscopy is provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the present invention, the description of "first", "second", etc. is only used for descriptive purposes, and cannot be understood as indicating or implying its relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" can explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0020] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.

[0021] The embodiment of the present invention provides a method for detecting surface water quality based on spectroscopy. Figure 1 , Figure 1 A flow chart of a surface water quality detection method based on spectroscopy provided in an embodiment of the present invention. The method comprises the following steps: S101, obtaining the ultraviolet-visible absorption spectrum corresponding to the target detection area within a period of time to obtain a first spectrum set; S102, for each ultraviolet-visible absorption spectrum in the first spectrum set, preprocess the ultraviolet-visible absorption spectrum to obtain an initial spectrum, and obtain all the initial spectra to obtain an initial spectrum set; S103, performing spectrum type recognition based on the initial spectrum set, and if the type is recognized as a normal spectrum, removing abnormal spectra from the initial spectrum set to obtain a second spectrum set; S104, substituting the target spectrum into a preset model to obtain a target reference value; S105, calculating the average value of all target reference values ​​to obtain a target comparison value, and substituting the target comparison value into a preset database to obtain a detection result.

[0022] The target spectrum is any one of the second spectrum set.

[0023] A surface water quality detection method based on spectroscopy provided in an embodiment of the present invention effectively removes noise interference by preprocessing the collected spectrum and then performing type recognition, thereby quickly screening out normal spectral data and improving the overall detection efficiency; by substituting the target spectrum into a preset model, calculating the target reference value, and further obtaining the detection result based on database comparison, the reliability of the detection result is improved.

[0024] In one implementation, by preprocessing the UV-visible absorption spectra and removing abnormal spectra, the quality of the data can be greatly improved, noise interference can be reduced, and subsequent analysis results can be made more accurate. Removing abnormal spectra helps avoid the influence of erroneous data, thereby improving the overall detection accuracy.

[0025] In one implementation, the final target comparison value can be compared with the standard value through the database to obtain an accurate test result, which makes the decision-making process more scientific and data-based, reduces the errors caused by human judgment, and improves the accuracy and reliability of the decision. The preset database is used to save the water quality results corresponding to different target comparison values.

[0026] In one implementation, through spectral type recognition, abnormal spectra can be effectively identified and these abnormal data can be eliminated to ensure the quality of the remaining data. This optimization process can improve the reliability of the overall detection and avoid the influence of abnormal data on the judgment results; and if the type is identified as an abnormal spectrum, it means that the currently collected UV-visible absorption spectrum is affected by multiple factors, including hydrological fluctuations caused by rainfall, flow changes, etc., and it is necessary to re-collect the UV-visible absorption spectrum under the control of technical personnel.

[0027] In one embodiment, preprocessing the UV-visible absorption spectrum to obtain an initial spectrum includes: The UV-visible absorption spectrum is decomposed by discrete wavelet transform to obtain coefficients corresponding to different frequencies to obtain a target coefficient set; the target coefficient set includes a low-frequency coefficient set and a high-frequency coefficient set; For each high-frequency coefficient in the high-frequency coefficient set, the high-frequency coefficient is processed by a threshold function to obtain a target high-frequency coefficient; All target high-frequency coefficients are obtained to obtain a target high-frequency coefficient set, and the ultraviolet-visible absorption spectrum is reconstructed according to the target high-frequency coefficient set and the low-frequency coefficient set to obtain an initial spectrum corresponding to the ultraviolet-visible absorption spectrum.

[0028] In one implementation, the discrete wavelet transform (DWT) decomposes the signal into low-frequency approximate coefficients and high-frequency detail coefficients through a pair of low-pass filters and high-pass filters. The low-pass filter retains the low-frequency components in the signal and outputs the approximate coefficients. The low-frequency part represents the smooth part and overall trend of the signal; the high-pass filter extracts the high-frequency components in the signal and outputs the detail coefficients. The high-frequency part represents the local rapid changes, noise, etc. in the signal.

[0029] In one implementation, the wavelet transform can decompose the spectrum into components of different frequencies, thereby separating the high-frequency part where noise usually appears from the low-frequency part of the signal. By performing threshold processing on the high-frequency coefficients, the noise can be effectively removed, retaining only the useful signal components, thereby improving the quality of the data and reducing interference in spectral analysis.

[0030] In one implementation, through wavelet transform decomposition and processing, important characteristic components in the spectrum can be extracted and optimized for reconstruction, which helps to retain key information during the spectrum reconstruction process while removing irrelevant or redundant parts, thereby improving the accuracy of the final spectrum; the reconstructed initial spectrum can significantly improve the quality and reliability of the data by combining the processed high-frequency coefficients and low-frequency coefficients. This supports the subsequent target recognition, classification or other analysis processes, making the final detection results more stable and accurate.

[0031] In one implementation, the threshold function can be a soft threshold function, a hard threshold function, etc.; the target high-frequency coefficient set and the low-frequency coefficient set are input into the inverse wavelet transform algorithm to reconstruct the ultraviolet-visible absorption spectrum to obtain the initial spectrum corresponding to the ultraviolet-visible absorption spectrum.

[0032] In one embodiment, performing spectrum type recognition according to the initial spectrum set includes: For each initial spectrum in the initial spectrum set, obtaining a detection parameter vector corresponding to the initial spectrum; A detection matrix is ​​constructed according to the detection parameter vectors corresponding to all the initial spectra, and for the parameter values ​​corresponding to each row in the detection matrix, each parameter value is marked with a serial number from large to small to obtain a serial number matrix; For each column in the sequence number matrix, reorder according to the sequence number to obtain the target sorting set corresponding to each column; Calculate the ranking difference corresponding to each detection parameter in the target ranking set and the preset ranking set to obtain a target difference value; The target value is obtained according to the sum of all target difference values ​​corresponding to the column. If the target value is less than the preset threshold, the target value is recorded as a valid value; The ratio of the number of effective values ​​to the number of target values ​​is calculated to obtain a target ratio. If the target ratio is greater than a preset ratio, the type of the initial spectrum set is identified as a normal spectrum.

[0033] In one implementation, by constructing a detection matrix and sorting and marking each row, the detection parameters of each spectrum can be carefully evaluated, and the actual detection results can be compared with the preset standard sorting, which can effectively distinguish normal spectra from abnormal spectra and enhance the classification accuracy.

[0034] In one implementation, the detection parameter vector is a combination of any several of the following: Mahalanobis distance, Q-residual, spectral divergence variable, Euclidean distance, Pearson correlation coefficient, and Jacard similarity; the maximum serial number mark is 1, and the preset sorting set, preset threshold, and preset ratio are determined by technical personnel; the parameters sorted in the preset sorting set are the same as the parameters in the detection parameter vector.

[0035] In one implementation, if the length of the detection parameter vector is 5, the preset sorting set is {1,2,3,4,5}, and the sorting is (Mahalanobis distance, Q-residual, spectral divergence variable, Euclidean distance, Pearson correlation coefficient), the detection matrix has 5 rows, the first row is the corresponding value of the Mahalanobis distance, the second row is the corresponding value of the Q-residual, and similarly the fifth row is the corresponding value of the Pearson correlation coefficient. The columns of the detection matrix are the detection parameter vectors corresponding to each spectrum, and each row will be mapped from the maximum value to the minimum value. The maximum value corresponds to 1 and the minimum value is the number of spectra. If the serial numbers in a column are the same, then according to the following example, the column corresponds to {2,2,6,4,1} in the serial number matrix, then the column is mapped. The row order is {2,2,5,4,1}, and the repeated serial numbers remain consistent, but when there is a repetition, a number will jump, for example, 3 in the example is not there. At this time, the target difference is calculated between the target sorting set and the preset sorting set, and the target difference is |1-2|=1, |2-2|=0, |3-5|=2, |4-4|=0, |5-1|=4. At this time, the target value is 7. If the preset threshold is 8 at this time, it means that the current target value is a valid value. After calculating all the target values, determine the proportion of valid values ​​in the target values. If the proportion is greater than the preset ratio, the type of the initial spectrum set is identified as a normal spectrum, otherwise it is abnormal. The preset threshold and preset ratio are determined by technical personnel.

[0036] The length and sorting method of the detection parameter vector are determined by technical personnel.

[0037] In one implementation, by calculating the sorting difference between the target sorting set and the preset sorting set and summing them up, spectra that differ greatly from the standard can be identified. If the difference is too large, it means that the spectrum is likely to be an abnormal spectrum. This method can quickly screen out abnormal data and effectively improve detection efficiency.

[0038] In one implementation, by setting a threshold of a target value to determine whether a spectrum is valid, spectra that do not meet the standards can be effectively excluded. Threshold control provides flexibility and allows adjustment according to the needs of specific applications to meet different detection accuracy requirements.

[0039] In one implementation, the ratio of the effective value to the target value is calculated and compared with a preset ratio threshold to quantify the test results. The standardized evaluation method not only helps to determine the type of spectrum, but also adjusts the preset ratio according to different samples and test conditions, further improving the consistency of the test results.

[0040] In one embodiment, the preset model includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, a preset attention module, a preset fusion module, a global average pooling layer, a flattening layer, and a linear layer, and substituting the initial spectrum into the preset model to obtain a target reference value includes: Obtain an initial spectrum, substitute the initial spectrum into the first convolution block to obtain a first convolution image, and substitute the first convolution image into the second convolution block to obtain a second convolution image; Substituting the second convolution image into a preset attention module to obtain a first attention image, and substituting the first attention image into a preset fusion module to obtain a first fused image; Substituting the first fused image into the third convolution block to obtain a third convolution image, and substituting the third convolution image into the preset fusion module to obtain a second fused image; Substituting the second fused image into the fourth convolution block to obtain a fourth convolution image, and substituting the fourth convolution image into the preset fusion module to obtain a third fused image; Substituting the third fused image into the fifth convolution block to obtain a fifth convolution image, and substituting the fifth convolution image into the preset attention module to obtain a second attention image; The second attention image is substituted into the global average pooling layer and then passed through the flattening layer and the linear layer to obtain the reference value.

[0041] In one implementation, each convolution block gradually extracts higher-level feature information. Through layer-by-layer convolution, the model's expressiveness can be gradually enhanced from local features to global features, thereby improving the ability to recognize input data. A preset attention module is applied to the second convolution image and the fifth convolution image, which can dynamically adjust the weight of the input information, thereby helping the model to pay more attention to important features. This is particularly important for spectral data, because spectral data often contains a lot of redundant information. The attention mechanism can help filter out irrelevant noise and improve the performance of the model.

[0042] In one implementation, features of different convolution blocks are fused by presetting a fusion module, which can make full use of features at different levels and improve the effects of information transmission and feature extraction. The role of the fusion module is to integrate information from different convolution levels, thereby improving the feature representation ability of the overall model. At the end of the model, the global average pooling layer converts the high-dimensional feature map into a low-dimensional vector, which helps to reduce the number of model parameters, reduce the risk of overfitting, and retain global information. The subsequent flattening layer and linear layer further convert it into the final reference value.

[0043] In one implementation, the first convolution block has a convolution kernel size of 3*1, a stride of 1, and a number of 32; the second convolution block has a convolution kernel size of 3*1, a stride of 2, and a number of 64; the third convolution block has a convolution kernel size of 3*1, a stride of 2, and a number of 128; the fourth convolution block has a convolution kernel size of 3*1, a stride of 2, and a number of 256; the first convolution block has a convolution kernel size of 3*1, a stride of 2, and a number of 256; the preset attention module includes a first attention module and a second attention module, and the preset fusion module operation is to obtain an image to be processed, perform a 1*1 convolution on the image to be processed, halve the channels to obtain a first image to be processed, perform a 3*1 convolution on the first image to be processed, multiply the channels by 2 to obtain a second image to be processed, and perform weighted fusion on the image to be processed and the second image to be processed to obtain a fused image; the flattening layer contains 256 nodes.

[0044] In one embodiment, the preset attention module includes a first attention module and a second attention module; The first attention module receives the convolution image and extracts attention from the convolution image to obtain a first attention extraction image; The first attention extraction image and the convolution image are weighted fused to obtain the first attention fusion image; The second attention module receives the first attention fusion image and performs attention extraction on the first attention fusion image to obtain a second attention extraction image; The attention image is obtained by weighted fusion of the second attention extraction image and the first attention extraction image.

[0045] In one implementation, the first attention module first extracts attention from the convolution image, and by adaptively adjusting the weights, emphasizes the important feature areas in the image and suppresses the unimportant parts. This enables the model to focus more on signals that are helpful to the final task; the image after attention extraction is weighted fused with the original convolution image, which not only retains all the information in the convolution image, but also strengthens the key parts of attention. Weighted fusion not only enhances the recognition ability of the model, but also avoids information loss.

[0046] In one implementation, the second attention module receives the image after the first fusion and extracts attention from it again, which can further refine the focus on image features. Especially when processing complex or high-dimensional data, the second attention mechanism can further extract more discriminative features from the fused feature map.

[0047] In one implementation, in the first attention module, the input feature map undergoes global maximum pooling and global average pooling to obtain the maximum value and average value of each channel, respectively. The pooled results are input into two multi-layer perceptrons, respectively, and the importance of each channel is obtained through learning. The pooled results are processed by MLP and summed to obtain the attention weight of each channel. Finally, the obtained channel attention tensor will also generate the channel attention weight through the Sigmoid activation function. The generated channel attention weight is multiplied with the original feature map in the channel dimension to obtain the first attention extraction image.

[0048] In one implementation, in the second attention module, the input feature map undergoes maximum pooling and average pooling, the results of maximum pooling and average pooling are concatenated in the channel dimension, and then convolution operation is performed through a 7×1 convolution kernel. The convolution result generates a spatial attention weight through a Sigmoid activation function. This weight is used to adjust the importance of each spatial position in the input feature map. The generated spatial attention weight is multiplied with the first attention fusion image in the channel dimension to obtain the second attention extraction image.

[0049] In one implementation, all weighted fusions in the entire text are equally weighted. For example, if two images are weightedly fused, the weights corresponding to the two images are both 0.5.

[0050] Based on the same inventive concept, the present invention also provides a surface water quality detection device based on spectroscopy. Figure 2 , Figure 2 A schematic structural diagram of a surface water quality detection device based on spectroscopy provided in an embodiment of the present invention includes: A spectrum acquisition module, used to obtain the ultraviolet-visible absorption spectrum corresponding to the target detection area within a period of time to obtain a first spectrum set; A spectrum preprocessing module, for preprocessing each ultraviolet-visible absorption spectrum in the first spectrum set to obtain an initial spectrum, and obtaining all the initial spectra to obtain an initial spectrum set; A spectrum type recognition module is used to perform spectrum type recognition based on the initial spectrum set. If the type is recognized as a normal spectrum, the abnormal spectrum in the initial spectrum set is removed to obtain a second spectrum set. A target reference value determination module, used for substituting a target spectrum into a preset model to obtain a target reference value; the target spectrum is any one of the second spectrum set; The test result determination module is used to calculate the average value of all target reference values ​​to obtain the target comparison value, and substitute the target comparison value into a preset database to obtain the test result.

[0051] A surface water quality detection device based on spectroscopy provided in an embodiment of the present invention effectively removes noise interference by preprocessing the collected spectrum and then performing type recognition, thereby quickly screening out normal spectral data and improving the overall detection efficiency; by substituting the target spectrum into a preset model, calculating the target reference value, and further obtaining the detection result based on database comparison, the reliability of the detection result is improved.

[0052] In one embodiment, the spectrum preprocessing module includes: A spectrum decomposition module is used to perform discrete wavelet transform decomposition on the ultraviolet-visible absorption spectrum to obtain coefficients corresponding to different frequencies to obtain a target coefficient set; the target coefficient set includes a low-frequency coefficient set and a high-frequency coefficient set; A high frequency coefficient determination module, for processing each high frequency coefficient in the high frequency coefficient set by a threshold function to obtain a target high frequency coefficient; The spectrum reconstruction module is used to obtain all target high-frequency coefficients to obtain a target high-frequency coefficient set, and reconstruct the ultraviolet-visible absorption spectrum according to the target high-frequency coefficient set and the low-frequency coefficient set to obtain an initial spectrum corresponding to the ultraviolet-visible absorption spectrum.

[0053] In one embodiment, the spectrum type recognition module includes: A detection parameter vector determination module, for obtaining, for each initial spectrum in the initial spectrum set, a detection parameter vector corresponding to the initial spectrum; A serial number matrix generation module is used to construct a detection matrix according to the detection parameter vectors corresponding to all the initial spectra, and to mark each parameter value corresponding to each row in the detection matrix with a serial number from large to small to obtain a serial number matrix; A sorting module is used to re-sort each column in the sequence matrix according to the sequence number to obtain a target sorting set corresponding to each column; A target difference determination module is used to calculate the ranking difference corresponding to each detection parameter in the target ranking set and the preset ranking set to obtain a target difference; A difference summing module is used to obtain a target value according to the sum of all target difference values ​​corresponding to the column. If the target value is less than a preset threshold, the target value is recorded as a valid value; The type determination module is used to calculate the ratio of the number of effective values ​​to the number of target values ​​to obtain a target ratio. If the target ratio is greater than a preset ratio, the type of the initial spectrum set is identified as a normal spectrum.

[0054] In one embodiment, the preset model includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, a preset attention module, a preset fusion module, a global average pooling layer, a flattening layer, and a linear layer, and substituting the initial spectrum into the preset model to obtain a target reference value includes: Obtain an initial spectrum, substitute the initial spectrum into the first convolution block to obtain a first convolution image, and substitute the first convolution image into the second convolution block to obtain a second convolution image; Substituting the second convolution image into a preset attention module to obtain a first attention image, and substituting the first attention image into a preset fusion module to obtain a first fused image; Substituting the first fused image into the third convolution block to obtain a third convolution image, and substituting the third convolution image into the preset fusion module to obtain a second fused image; Substituting the second fused image into the fourth convolution block to obtain a fourth convolution image, and substituting the fourth convolution image into the preset fusion module to obtain a third fused image; Substituting the third fused image into the fifth convolution block to obtain a fifth convolution image, and substituting the fifth convolution image into the preset attention module to obtain a second attention image; The second attention image is substituted into the global average pooling layer and then passed through the flattening layer and the linear layer to obtain the reference value.

[0055] In one embodiment, the preset attention module includes a first attention module and a second attention module; The first attention module receives the convolution image and extracts attention from the convolution image to obtain a first attention extraction image; The first attention extraction image and the convolution image are weighted fused to obtain the first attention fusion image; The second attention module receives the first attention fusion image and performs attention extraction on the first attention fusion image to obtain a second attention extraction image; The attention image is obtained by weighted fusion of the second attention extraction image and the first attention extraction image.

[0056] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A surface water quality detection method based on spectroscopy, characterized in that: The method comprises: Acquire the ultraviolet-visible absorption spectrum corresponding to the target detection area within a period of time to obtain a first spectrum set; For each ultraviolet-visible absorption spectrum in the first spectrum set, preprocess the ultraviolet-visible absorption spectrum to obtain an initial spectrum, and obtain all the initial spectra to obtain an initial spectrum set; Performing spectrum type identification according to the initial spectrum set, and if the type is identified as a normal spectrum, removing abnormal spectra from the initial spectrum set to obtain a second spectrum set; Substituting the target spectrum into the preset model to obtain the target reference value; The target spectrum is any one of the second spectrum set; The average value of all target reference values ​​is calculated to obtain a target comparison value, and the target comparison value is substituted into a preset database to obtain a detection result.

2. A surface water quality detection method based on spectroscopy according to claim 1, characterized in that: The UV-visible absorption spectrum is preprocessed to obtain an initial spectrum including: Performing discrete wavelet transform on the ultraviolet-visible absorption spectrum to obtain coefficients corresponding to different frequencies to obtain a target coefficient set; the target coefficient set includes a low-frequency coefficient set and a high-frequency coefficient set; For each high frequency coefficient in the high frequency coefficient set, the high frequency coefficient is processed by a threshold function to obtain a target high frequency coefficient; All target high-frequency coefficients are obtained to obtain a target high-frequency coefficient set, and an ultraviolet-visible absorption spectrum is reconstructed according to the target high-frequency coefficient set and the low-frequency coefficient set to obtain an initial spectrum corresponding to the ultraviolet-visible absorption spectrum.

3. The surface water quality detection method based on spectroscopy according to claim 1 is characterized in that: Identifying the spectrum type according to the initial spectrum set includes: For each initial spectrum in the initial spectrum set, obtaining a detection parameter vector corresponding to the initial spectrum; Constructing a detection matrix according to the detection parameter vectors corresponding to all the initial spectra, and marking each parameter value corresponding to each row in the detection matrix with a serial number from large to small to obtain a serial number matrix; For each column in the sequence number matrix, reorder according to the sequence number to obtain a target ordering set corresponding to each column; Calculate the ranking difference corresponding to each detection parameter in the target ranking set and the preset ranking set to obtain a target difference value; The target value is obtained according to the sum of all target difference values ​​corresponding to the column. If the target value is less than the preset threshold, the target value is recorded as a valid value; The ratio of the number of effective values ​​to the number of target values ​​is calculated to obtain a target ratio. If the target ratio is greater than a preset ratio, the type of the initial spectrum set is identified as a normal spectrum.

4. The surface water quality detection method based on spectroscopy according to claim 1 is characterized in that: The preset model includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, a preset attention module, a preset fusion module, a global average pooling layer, a flattening layer and a linear layer, and the initial spectrum is substituted into the preset model to obtain a target reference value including: Acquire the initial spectrum, substitute the initial spectrum into a first convolution block to obtain a first convolution image, and substitute the first convolution image into a second convolution block to obtain a second convolution image; Substituting the second convolution image into a preset attention module to obtain a first attention image, and substituting the first attention image into a preset fusion module to obtain a first fused image; Substituting the first fused image into the third convolution block to obtain a third convolution image, and substituting the third convolution image into the preset fusion module to obtain a second fused image; Substituting the second fused image into the fourth convolution block to obtain a fourth convolution image, and substituting the fourth convolution image into the preset fusion module to obtain a third fused image; Substituting the third fused image into the fifth convolution block to obtain a fifth convolution image, and substituting the fifth convolution image into the preset attention module to obtain a second attention image; The second attention image is substituted into the global average pooling layer and then passed through the flattening layer and the linear layer in sequence to obtain the reference value.

5. The surface water quality detection method based on spectroscopy according to claim 4 is characterized in that: The preset attention module includes a first attention module and a second attention module; The first attention module receives the convolution image and performs attention extraction on the convolution image to obtain a first attention extraction image; The first attention extraction image and the convolution image are weightedly fused to obtain a first attention fusion image; The second attention module receives the first attention fusion image and performs attention extraction on the first attention fusion image to obtain a second attention extraction image; The second attention extraction image and the first attention extraction image are weightedly fused to obtain an attention image.

6. A surface water quality detection device based on spectroscopy, characterized in that: The device comprises: A spectrum acquisition module, used to obtain the ultraviolet-visible absorption spectrum corresponding to the target detection area within a period of time to obtain a first spectrum set; A spectrum preprocessing module, for preprocessing each ultraviolet-visible absorption spectrum in the first spectrum set to obtain an initial spectrum, and obtaining all the initial spectra to obtain an initial spectrum set; a spectrum type identification module, configured to identify the spectrum type according to the initial spectrum set, and if the type is identified as a normal spectrum, remove the abnormal spectrum from the initial spectrum set to obtain a second spectrum set; A target reference value determination module, used for substituting a target spectrum into a preset model to obtain a target reference value; the target spectrum is any one of the second spectrum set; The detection result determination module is used to calculate the average value of all target reference values ​​to obtain a target comparison value, and substitute the target comparison value into a preset database to obtain a detection result.

7. A surface water quality detection device based on spectroscopy according to claim 6, characterized in that: The spectrum preprocessing module comprises: A spectrum decomposition module, used for performing discrete wavelet transform decomposition on the ultraviolet visible absorption spectrum to obtain coefficients corresponding to different frequencies to obtain a target coefficient set; the target coefficient set includes a low-frequency coefficient set and a high-frequency coefficient set; A high frequency coefficient determination module, for processing each high frequency coefficient in the high frequency coefficient set by a threshold function to obtain a target high frequency coefficient; The spectrum reconstruction module is used to obtain all target high-frequency coefficients to obtain a target high-frequency coefficient set, and reconstruct the ultraviolet-visible absorption spectrum according to the target high-frequency coefficient set and the low-frequency coefficient set to obtain an initial spectrum corresponding to the ultraviolet-visible absorption spectrum.

8. The surface water quality detection device based on spectroscopy according to claim 6 is characterized in that: The spectrum type recognition module includes: A detection parameter vector determination module, used for obtaining, for each initial spectrum in the initial spectrum set, a detection parameter vector corresponding to the initial spectrum; A serial number matrix generation module is used to construct a detection matrix according to the detection parameter vectors corresponding to all the initial spectra, and to mark each parameter value corresponding to each row in the detection matrix with a serial number from large to small to obtain a serial number matrix; A sorting module, used for re-sorting each column in the sequence number matrix according to the sequence number to obtain a target sorting set corresponding to each column; A target difference determination module is used to calculate the ranking difference corresponding to each detection parameter in the target ranking set and the preset ranking set to obtain a target difference; A difference summing module is used to obtain a target value according to the sum of all target difference values ​​corresponding to the column. If the target value is less than a preset threshold, the target value is recorded as a valid value; The type determination module is used to calculate the ratio of the number of effective values ​​to the number of target values ​​to obtain a target ratio. If the target ratio is greater than a preset ratio, the type of the initial spectrum set is identified as a normal spectrum.

9. The surface water quality detection device based on spectroscopy according to claim 6, characterized in that: The preset model includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, a preset attention module, a preset fusion module, a global average pooling layer, a flattening layer and a linear layer, and the initial spectrum is substituted into the preset model to obtain a target reference value including: Acquire the initial spectrum, substitute the initial spectrum into a first convolution block to obtain a first convolution image, and substitute the first convolution image into a second convolution block to obtain a second convolution image; Substituting the second convolution image into a preset attention module to obtain a first attention image, and substituting the first attention image into a preset fusion module to obtain a first fused image; Substituting the first fused image into the third convolution block to obtain a third convolution image, and substituting the third convolution image into the preset fusion module to obtain a second fused image; Substituting the second fused image into the fourth convolution block to obtain a fourth convolution image, and substituting the fourth convolution image into the preset fusion module to obtain a third fused image; Substituting the third fused image into the fifth convolution block to obtain a fifth convolution image, and substituting the fifth convolution image into the preset attention module to obtain a second attention image; The second attention image is substituted into the global average pooling layer and then passed through the flattening layer and the linear layer in sequence to obtain the reference value.

10. A surface water quality detection device based on spectroscopy according to claim 9, characterized in that: The preset attention module includes a first attention module and a second attention module; The first attention module receives the convolution image and performs attention extraction on the convolution image to obtain a first attention extraction image; The first attention extraction image and the convolution image are weightedly fused to obtain a first attention fusion image; The second attention module receives the first attention fusion image and performs attention extraction on the first attention fusion image to obtain a second attention extraction image; The second attention extraction image and the first attention extraction image are weightedly fused to obtain an attention image.

Images