A surface water quality detection method and device based on spectroscopy

Abstract

The present invention discloses a surface water quality detection method and device based on spectroscopy, which relates to the field of water quality detection technology. The method comprises the following steps: obtaining an ultraviolet-visible absorption spectrum corresponding to a target detection area within a period of time and pre-processing the ultraviolet-visible absorption spectrum to obtain an initial spectrum; obtaining all initial spectra to identify the spectrum type; if the type is identified as a normal spectrum, removing abnormal spectra from the initial spectrum set to obtain a second spectrum set; substituting the target spectrum into a preset model to obtain a target reference value; calculating the average value of all target reference values ​​to obtain a target comparison value; substituting the target comparison value into a preset database to obtain a detection result. After pre-processing the collected spectrum, the type is identified, effectively removing noise interference, thereby quickly screening out normal spectral data and improving overall detection efficiency; substituting the target spectrum into a preset model, calculating the target reference value, and further obtaining the detection result based on database comparison, thereby improving the reliability of the detection result.

Description

Technical Field

[0001] The present invention belongs to the technical field of water quality detection, and in particular relates to a surface water quality detection method and device based on spectroscopy. Background Art

[0002] Surface water is a vital source of water for human life and production. However, with the acceleration of industrialization and population growth, water pollution is becoming increasingly serious. Pollutant emissions, particularly organic pollutants, have become a significant factor affecting water quality. Chemical oxygen demand (COD), a key indicator of water pollution, is widely used in surface water pollution monitoring.

[0003] Ultraviolet-visible (UV-Vis) spectroscopy has become an effective technique for detecting water pollution due to its high sensitivity, rapidity, non-destructiveness, and environmental friendliness. By analyzing the absorption characteristics of light at different wavelengths in water samples, this method can provide information on water pollution, particularly in the detection of organic pollutants. However, surface water quality is affected by a variety of factors, including hydrological fluctuations caused by rainfall and flow changes. These fluctuations can cause the spectral characteristics of water to change over time, posing a challenge to accurately identifying contaminated water spectra.

[0004] Patent CN109459399B discloses a spectral water quality COD and turbidity detection method, which obtains several spectral characteristic curves affected only by turbidity, and fits them after preprocessing to obtain a turbidity correlation model F1; obtains several spectral characteristic curves affected only by turbidity and COD, and after preprocessing, divides them into spectral band data sets when turbidity is different, and fits each set separately to obtain a COD correlation model when turbidity is different, and then fits each COD correlation model to the corresponding turbidity to obtain a correlated turbidity and COD correlation model F2; based on the spectral characteristic curve of the water sample to be tested, the turbidity value is calculated using the turbidity correlation model F1, and the COD value is calculated based on the turbidity using the correlation model F2. However, this scheme does not take into account that changes in hydrological fluctuations and confusion in the polluted water spectrum will interfere with the accurate detection of COD, thereby leading to false positives or false negatives, and reducing the reliability of the test results. Summary of the Invention

[0005] The purpose of the present invention is to solve the above problems and to propose a surface water quality detection method and device based on spectroscopy.

[0006] In a first aspect of the present invention, a surface water quality detection method based on spectroscopy is first proposed, the method comprising:

[0007] Obtaining the ultraviolet-visible absorption spectrum corresponding to the target detection area within a period of time to obtain a first spectrum set;

[0008] For each ultraviolet-visible absorption spectrum in the first spectrum set, preprocess the ultraviolet-visible absorption spectrum to obtain an initial spectrum, and obtain all the initial spectra to obtain an initial spectrum set;

[0009] performing spectrum type identification based on the initial spectrum set, and if the type is identified as a normal spectrum, removing abnormal spectra from the initial spectrum set to obtain a second spectrum set;

[0010] Substituting a target spectrum into a preset model to obtain a target reference value; the target spectrum is any one of the second spectrum set;

[0011] The average value of all target reference values ​​is calculated to obtain a target comparison value, and the target comparison value is substituted into a preset database to obtain a test result.

[0012] Optionally, preprocessing the UV-visible absorption spectrum to obtain an initial spectrum includes:

[0013] Performing discrete wavelet transform on the ultraviolet-visible absorption spectrum to obtain coefficients corresponding to different frequencies to obtain a target coefficient set; the target coefficient set includes a low-frequency coefficient set and a high-frequency coefficient set;

[0014] For each high frequency coefficient in the high frequency coefficient set, processing the high frequency coefficient by a threshold function to obtain a target high frequency coefficient;

[0015] All target high-frequency coefficients are obtained to obtain a target high-frequency coefficient set, and an ultraviolet-visible absorption spectrum is reconstructed according to the target high-frequency coefficient set and the low-frequency coefficient set to obtain an initial spectrum corresponding to the ultraviolet-visible absorption spectrum.

[0016] Optionally, performing spectrum type identification according to the initial spectrum set includes:

[0017] For each initial spectrum in the initial spectrum set, obtaining a detection parameter vector corresponding to the initial spectrum;

[0018] Constructing a detection matrix based on the detection parameter vectors corresponding to all initial spectra, and marking the parameter values ​​corresponding to each row in the detection matrix with serial numbers from large to small to obtain a serial number matrix;

[0019] For each column in the sequence number matrix, reorder according to the sequence number to obtain a target sorting set corresponding to each column;

[0020] Calculate the ranking difference between the target ranking set and the preset ranking set corresponding to each detection parameter to obtain a target difference value;

[0021] The target value is obtained by summing up all target differences corresponding to the column. If the target value is less than the preset threshold, the target value is recorded as a valid value.

[0022] A ratio of the number of effective values ​​to the number of target values ​​is calculated to obtain a target ratio. If the target ratio is greater than a preset ratio, the type of the initial spectrum set is identified as a normal spectrum.

[0023] Optionally, the preset model includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, a preset attention module, a preset fusion module, a global average pooling layer, a flattening layer, and a linear layer, and substituting the initial spectrum into the preset model to obtain a target reference value includes:

[0024] Obtaining the initial spectrum, substituting the initial spectrum into a first convolution block to obtain a first convolution image, and substituting the first convolution image into a second convolution block to obtain a second convolution image;

[0025] Substituting the second convolution image into a preset attention module to obtain a first attention image, and substituting the first attention image into a preset fusion module to obtain a first fused image;

[0026] Substituting the first fused image into a third convolution block to obtain a third convolution image, and substituting the third convolution image into the preset fusion module to obtain a second fused image;

[0027] Substituting the second fused image into the fourth convolution block to obtain a fourth convolution image, and substituting the fourth convolution image into the preset fusion module to obtain a third fused image;

[0028] Substituting the third fused image into the fifth convolution block to obtain a fifth convolution image, and substituting the fifth convolution image into the preset attention module to obtain a second attention image;

[0029] The second attention image is substituted into the global average pooling layer and then passed through the flattening layer and the linear layer to obtain the reference value.

[0030] Optionally, the preset attention module includes a first attention module and a second attention module;

[0031] The first attention module receives the convolution image and performs attention extraction on the convolution image to obtain a first attention extraction image;

[0032] The first attention extraction image and the convolution image are weightedly fused to obtain a first attention fusion image;

[0033] The second attention module receives the first attention fusion image and performs attention extraction on the first attention fusion image to obtain a second attention extraction image;

[0034] The second attention extraction image and the first attention extraction image are weightedly fused to obtain an attention image.

[0035] In a second aspect of the present invention, a surface water quality detection device based on spectroscopy is provided, comprising:

[0036] A spectrum acquisition module is used to obtain the ultraviolet-visible absorption spectrum corresponding to the target detection area within a period of time to obtain a first spectrum set;

[0037] a spectrum preprocessing module, configured to preprocess each ultraviolet-visible absorption spectrum in the first spectrum set to obtain an initial spectrum, and obtain all the initial spectra to obtain an initial spectrum set;

[0038] a spectrum type identification module, configured to identify the spectrum type according to the initial spectrum set, and if the type is identified as a normal spectrum, remove abnormal spectra from the initial spectrum set to obtain a second spectrum set;

[0039] a target reference value determination module, configured to substitute a target spectrum into a preset model to obtain a target reference value; the target spectrum being any one of the second spectrum set;

[0040] The detection result determination module is used to calculate the average value of all target reference values ​​to obtain a target comparison value, and substitute the target comparison value into a preset database to obtain a detection result.

[0041] Optionally, the spectrum preprocessing module includes:

[0042] a spectrum decomposition module, configured to perform discrete wavelet transform on the ultraviolet-visible absorption spectrum to obtain coefficients corresponding to different frequencies to obtain a target coefficient set; the target coefficient set includes a low-frequency coefficient set and a high-frequency coefficient set;

[0043] a high-frequency coefficient determination module, configured to process each high-frequency coefficient in the high-frequency coefficient set by a threshold function to obtain a target high-frequency coefficient;

[0044] The spectrum reconstruction module is used to obtain all target high-frequency coefficients to obtain a target high-frequency coefficient set, and reconstruct the ultraviolet-visible absorption spectrum according to the target high-frequency coefficient set and the low-frequency coefficient set to obtain an initial spectrum corresponding to the ultraviolet-visible absorption spectrum.

[0045] Optionally, the spectrum type recognition module includes:

[0046] a detection parameter vector determination module, configured to obtain, for each initial spectrum in the initial spectrum set, a detection parameter vector corresponding to the initial spectrum;

[0047] A serial number matrix generation module is used to construct a detection matrix based on the detection parameter vectors corresponding to all initial spectra, and to mark the parameter values ​​corresponding to each row in the detection matrix with serial numbers from large to small to obtain a serial number matrix;

[0048] A sorting module, configured to re-sort each column in the sequence number matrix according to the sequence number to obtain a target sorting set corresponding to each column;

[0049] A target difference determination module is used to calculate the ranking difference corresponding to each detection parameter in the target ranking set and the preset ranking set to obtain a target difference;

[0050] The difference summation module is used to obtain the target value according to the sum of all target differences corresponding to the column. If the target value is less than the preset threshold, the target value is recorded as a valid value;

[0051] The type determination module is configured to calculate a ratio of the number of valid values ​​to the number of target values ​​to obtain a target ratio. If the target ratio is greater than a preset ratio, the type of the initial spectrum set is identified as a normal spectrum.

[0052] Optionally, the preset model includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, a preset attention module, a preset fusion module, a global average pooling layer, a flattening layer, and a linear layer, and substituting the initial spectrum into the preset model to obtain a target reference value includes:

[0053] Obtaining the initial spectrum, substituting the initial spectrum into a first convolution block to obtain a first convolution image, and substituting the first convolution image into a second convolution block to obtain a second convolution image;

[0054] Substituting the second convolution image into a preset attention module to obtain a first attention image, and substituting the first attention image into a preset fusion module to obtain a first fused image;

[0055] Substituting the first fused image into a third convolution block to obtain a third convolution image, and substituting the third convolution image into the preset fusion module to obtain a second fused image;

[0056] Substituting the second fused image into the fourth convolution block to obtain a fourth convolution image, and substituting the fourth convolution image into the preset fusion module to obtain a third fused image;

[0057] Substituting the third fused image into the fifth convolution block to obtain a fifth convolution image, and substituting the fifth convolution image into the preset attention module to obtain a second attention image;

[0058] The second attention image is substituted into the global average pooling layer and then passed through the flattening layer and the linear layer to obtain the reference value.

[0059] Optionally, the preset attention module includes a first attention module and a second attention module;

[0060] The first attention module receives the convolution image and performs attention extraction on the convolution image to obtain a first attention extraction image;

[0061] The first attention extraction image and the convolution image are weightedly fused to obtain a first attention fusion image;

[0062] The second attention module receives the first attention fusion image and performs attention extraction on the first attention fusion image to obtain a second attention extraction image;

[0063] The second attention extraction image and the first attention extraction image are weightedly fused to obtain an attention image.

[0064] Beneficial effects of the present invention:

[0065] The present invention proposes a surface water quality detection method based on spectroscopy, which obtains the ultraviolet-visible absorption spectrum corresponding to the target detection area within a period of time to obtain a first spectrum set; for each ultraviolet-visible absorption spectrum in the first spectrum set, the ultraviolet-visible absorption spectrum is preprocessed to obtain an initial spectrum, and all initial spectra are obtained to obtain an initial spectrum set; spectrum type identification is performed based on the initial spectrum set, and if the type identification is a normal spectrum, the abnormal spectrum in the initial spectrum set is removed to obtain a second spectrum set; the target spectrum is substituted into a preset model to obtain a target reference value; the target spectrum is any one of the second spectrum set; the average value of all target reference values ​​is calculated to obtain a target comparison value, and the target comparison value is substituted into a preset database to obtain a detection result. By preprocessing the collected spectrum and then performing type identification, noise interference is effectively removed, thereby quickly screening out normal spectral data, thereby improving the overall detection efficiency; by substituting the target spectrum into a preset model, calculating the target reference value, and further obtaining the detection result based on the database comparison, the reliability of the detection result is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The present invention will be further described below with reference to the accompanying drawings.

[0067] Figure 1 A flow chart of a surface water quality detection method based on spectroscopy is provided for an embodiment of the present invention;

[0068] Figure 2 The present invention provides a schematic structural diagram of a surface water quality detection device based on spectroscopy. DETAILED DESCRIPTION

[0069] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention, in conjunction with the accompanying drawings. Obviously, the described embodiments represent only a portion of the embodiments of the present invention, not all of them. The term "and / or" herein simply describes an association relationship between associated objects, indicating that three possible relationships exist. For example, "A" and "B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, references to "first," "second," and so on in the present invention are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined as "first" or "second" may explicitly or implicitly include at least one of these features. Furthermore, the technical solutions of the various embodiments may be combined, but only if they are achievable by a person of ordinary skill in the art. If a combination of technical solutions contradicts or is unachievable, such combination shall be deemed non-existent and outside the scope of protection claimed by the present invention.

[0070] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.

[0071] The embodiment of the present invention provides a method for detecting surface water quality based on spectroscopy. Figure 1 , Figure 1 A flow chart of a method for detecting surface water quality based on spectroscopy is provided in an embodiment of the present invention. The method comprises the following steps:

[0072] S101, obtaining ultraviolet-visible absorption spectra corresponding to a target detection area within a period of time to obtain a first spectrum set;

[0073] S102, for each ultraviolet-visible absorption spectrum in the first spectrum set, preprocessing the ultraviolet-visible absorption spectrum to obtain an initial spectrum, and obtaining all the initial spectra to obtain an initial spectrum set;

[0074] S103, performing spectrum type identification based on the initial spectrum set, and if the type is identified as a normal spectrum, removing abnormal spectra from the initial spectrum set to obtain a second spectrum set;

[0075] S104, substituting the target spectrum into a preset model to obtain a target reference value;

[0076] S105, calculating the average value of all target reference values ​​to obtain a target comparison value, and substituting the target comparison value into a preset database to obtain a detection result.

[0077] The target spectrum is any one in the second spectrum set.

[0078] A surface water quality detection method based on spectroscopy provided by an embodiment of the present invention effectively removes noise interference by preprocessing the collected spectra and then performing type identification, thereby quickly screening out normal spectral data and improving overall detection efficiency. The target spectrum is substituted into a preset model, the target reference value is calculated, and the detection result is further obtained by comparison with the database, thereby improving the reliability of the detection result.

[0079] In one implementation, preprocessing the UV-visible absorption spectra and removing abnormal spectra can significantly improve data quality, reduce noise interference, and make subsequent analysis results more accurate. Removing abnormal spectra helps avoid the influence of erroneous data, thereby improving overall detection accuracy.

[0080] In one implementation method, the final target comparison value can be compared with the standard value through the database to obtain an accurate test result, which makes the decision-making process more scientific and data-based, reduces the errors caused by human judgment, and improves the accuracy and reliability of the decision. The preset database is used to save the water quality results corresponding to different target comparison values.

[0081] In one implementation, through spectral type recognition, abnormal spectra can be effectively identified and these abnormal data can be eliminated to ensure the quality of the remaining data. This optimization process can improve the reliability of the overall detection and avoid the influence of abnormal data on the judgment results. If the type is identified as an abnormal spectrum, it means that the currently collected UV-visible absorption spectrum is affected by multiple factors, including hydrological fluctuations caused by rainfall, flow changes, etc., and the UV-visible absorption spectrum needs to be re-collected under the control of technical personnel.

[0082] In one embodiment, preprocessing the UV-visible absorption spectrum to obtain an initial spectrum includes:

[0083] The UV-visible absorption spectrum is decomposed by discrete wavelet transform to obtain coefficients corresponding to different frequencies to obtain a target coefficient set; the target coefficient set includes a low-frequency coefficient set and a high-frequency coefficient set;

[0084] For each high-frequency coefficient in the high-frequency coefficient set, the high-frequency coefficient is processed by a threshold function to obtain a target high-frequency coefficient;

[0085] All target high-frequency coefficients are obtained to obtain a target high-frequency coefficient set, and an ultraviolet-visible absorption spectrum is reconstructed according to the target high-frequency coefficient set and the low-frequency coefficient set to obtain an initial spectrum corresponding to the ultraviolet-visible absorption spectrum.

[0086] In one implementation, the discrete wavelet transform (DWT) decomposes a signal into low-frequency approximate coefficients and high-frequency detail coefficients using a pair of low-pass and high-pass filters. The low-pass filter retains the low-frequency components of the signal and outputs approximate coefficients. The low-frequency components represent smooth parts and overall trends of the signal. The high-pass filter extracts the high-frequency components of the signal and outputs detail coefficients. The high-frequency components represent local rapid changes and noise in the signal.

[0087] In one implementation, the wavelet transform can decompose the spectrum into components of different frequencies, thereby separating the noise that usually appears in the high-frequency part from the low-frequency part of the signal. By thresholding the high-frequency coefficients, the noise can be effectively removed, retaining only the useful signal components, which can improve the quality of the data and reduce interference in spectral analysis.

[0088] In one implementation, wavelet transform decomposition and processing can extract important characteristic components from the spectrum and optimize their reconstruction. This helps retain key information during spectral reconstruction while removing irrelevant or redundant components, thereby improving the accuracy of the final spectrum. The reconstructed initial spectrum is then combined with processed high-frequency and low-frequency coefficients to significantly improve data quality and reliability. This supports subsequent target recognition, classification, or other analysis processes, making the final detection results more stable and accurate.

[0089] In one implementation, the threshold function can be a soft threshold function, a hard threshold function, etc.; the target high-frequency coefficient set and the low-frequency coefficient set are input into the inverse wavelet transform algorithm to reconstruct the ultraviolet-visible absorption spectrum to obtain the initial spectrum corresponding to the ultraviolet-visible absorption spectrum.

[0090] In one embodiment, performing spectrum type identification based on the initial spectrum set includes:

[0091] For each initial spectrum in the initial spectrum set, obtaining a detection parameter vector corresponding to the initial spectrum;

[0092] A detection matrix is ​​constructed based on the detection parameter vectors corresponding to all initial spectra. For the parameter values ​​corresponding to each row in the detection matrix, each parameter value is serially marked from large to small to obtain a serial number matrix.

[0093] For each column in the sequence matrix, reorder according to the sequence number to obtain the target sorting set corresponding to each column;

[0094] Calculate the ranking difference between the target ranking set and the preset ranking set corresponding to each detection parameter to obtain a target difference value;

[0095] The target value is obtained by summing up all target differences corresponding to the column. If the target value is less than the preset threshold, the target value is recorded as a valid value.

[0096] The ratio of the number of effective values ​​to the number of target values ​​is calculated to obtain a target ratio. If the target ratio is greater than a preset ratio, the type of the initial spectrum set is identified as a normal spectrum.

[0097] In one implementation, by constructing a detection matrix and sorting and labeling each row, the detection parameters of each spectrum can be carefully evaluated. By comparing the actual detection results with the preset standard sorting, normal and abnormal spectra can be efficiently distinguished, thereby enhancing the classification accuracy.

[0098] In one implementation, the detection parameter vector is a combination of any several of the following: Mahalanobis distance, Q-residual, spectral divergence variable, Euclidean distance, Pearson correlation coefficient, and Jacard similarity; the maximum serial number mark is 1, and the preset sorting set, preset threshold, and preset ratio are determined by technical personnel; the parameters sorted in the preset sorting set are the same as the parameters in the detection parameter vector.

[0099] In one implementation, if the length of the detection parameter vector is 5, the preset sorting set is {1, 2, 3, 4, 5}, and the sorting is (Mahalanobis distance, Q-residual, spectral divergence variable, Euclidean distance, Pearson correlation coefficient), the detection matrix has 5 rows, the first row is the corresponding value of the Mahalanobis distance, the second row is the corresponding value of the Q-residual, and similarly the fifth row is the corresponding value of the Pearson correlation coefficient. The columns of the detection matrix are the detection parameter vectors corresponding to each spectrum, and each row will be mapped from the maximum value to the minimum value. The maximum value corresponds to 1 and the minimum value is the number of spectra. If the serial numbers in a column are the same, then according to the following example, the column corresponds to {2, 2, 6, 4, 1} in the serial number matrix, then the column is mapped. The row order is {2,2,5,4,1}, and the repeated serial numbers remain consistent, but when there is a repetition, a number will jump. For example, 3 in the example is not there. At this time, the target difference is calculated between the target sorting set and the preset sorting set, and the target difference is |1-2|=1, |2-2|=0, |3-5|=2, |4-4|=0, |5-1|=4. At this time, the target value is 7. If the preset threshold is 8 at this time, it means that the current target value is a valid value. After calculating all the target values, the proportion of the valid value in the target value is determined. If the proportion is greater than the preset ratio, the type of the initial spectrum set is identified as a normal spectrum, otherwise it is abnormal. The preset threshold and preset ratio are determined by technical personnel.

[0100] The length and sorting method of the detection parameter vector are determined by technical personnel.

[0101] In one implementation, by calculating and summing the order differences between the target order set and the preset order set, spectra that differ significantly from the standard can be identified. If the difference is too large, it indicates that the spectrum is likely an anomaly. This method can quickly filter out abnormal data and effectively improve detection efficiency.

[0102] In one implementation, a threshold value of a target value is set to determine whether a spectrum is valid, which can effectively exclude spectra that do not meet the standard. Threshold control provides flexibility and allows adjustment according to the needs of specific applications, thereby adapting to different detection accuracy requirements.

[0103] In one implementation, the ratio of the effective value to the target value is calculated and compared with a preset ratio threshold to quantify the test results. This standardized evaluation method not only helps to determine the type of spectrum, but also adjusts the preset ratio according to different samples and test conditions, further improving the consistency of the test results.

[0104] In one embodiment, the preset model includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, a preset attention module, a preset fusion module, a global average pooling layer, a flattening layer, and a linear layer. Substituting the initial spectrum into the preset model to obtain a target reference value includes:

[0105] Obtain an initial spectrum, substitute the initial spectrum into the first convolution block to obtain a first convolution image, and substitute the first convolution image into the second convolution block to obtain a second convolution image;

[0106] Substituting the second convolution image into a preset attention module to obtain a first attention image, and substituting the first attention image into a preset fusion module to obtain a first fused image;

[0107] Substituting the first fused image into the third convolution block to obtain a third convolution image, and substituting the third convolution image into the preset fusion module to obtain a second fused image;

[0108] Substituting the second fused image into the fourth convolution block to obtain a fourth convolution image, and substituting the fourth convolution image into the preset fusion module to obtain a third fused image;

[0109] Substituting the third fused image into the fifth convolution block to obtain a fifth convolution image, and substituting the fifth convolution image into the preset attention module to obtain a second attention image;

[0110] The second attention image is substituted into the global average pooling layer and then passed through the flattening layer and the linear layer to obtain the reference value.

[0111] In one implementation, each convolution block gradually extracts higher-level feature information. Through layer-by-layer convolution, the model's expressiveness can be gradually enhanced from local features to global features, thereby improving the ability to recognize input data. A preset attention module is applied to the second and fifth convolution images, which can dynamically adjust the weight of the input information, thereby helping the model pay more attention to important features. This is particularly important for spectral data because spectral data often contains a large amount of redundant information. The attention mechanism can help filter out irrelevant noise and improve the performance of the model.

[0112] In one implementation, features from different convolutional blocks are fused through a preset fusion module, which can fully utilize features from different levels and improve the effects of information transmission and feature extraction. The fusion module integrates information from different convolutional levels, thereby enhancing the feature representation capability of the overall model. At the end of the model, the global average pooling layer converts the high-dimensional feature map into a low-dimensional vector, which helps reduce the number of model parameters, reduces the risk of overfitting, and retains global information. The subsequent flattening layer and linear layer further convert it into the final reference value.

[0113] In one implementation, the first convolution block has a convolution kernel size of 3*1, a stride of 1, and a number of 32; the second convolution block has a convolution kernel size of 3*1, a stride of 2, and a number of 64; the third convolution block has a convolution kernel size of 3*1, a stride of 2, and a number of 128; the fourth convolution block has a convolution kernel size of 3*1, a stride of 2, and a number of 256; the first convolution block has a convolution kernel size of 3*1, a stride of 2, and a number of 256; the preset attention module includes a first attention module and a second attention module, and the preset fusion module operation is to obtain the image to be processed, perform 1*1 convolution on the image to be processed, halve the channels to obtain the first image to be processed, perform 3*1 convolution on the first image to be processed, multiply the channels by 2 to obtain the second image to be processed, and perform weighted fusion on the image to be processed and the second image to be processed to obtain a fused image; the flattening layer contains 256 nodes.

[0114] In one embodiment, the preset attention module includes a first attention module and a second attention module;

[0115] The first attention module receives the convolution image and performs attention extraction on the convolution image to obtain a first attention extraction image;

[0116] The first attention extracted image and the convolution image are weighted fused to obtain the first attention fused image;

[0117] The second attention module receives the first attention fusion image and performs attention extraction on the first attention fusion image to obtain a second attention extraction image;

[0118] The attention image is obtained by weighted fusion of the second attention extraction image and the first attention extraction image.

[0119] In one implementation, the first attention module extracts attention from the convolved image. By adaptively adjusting weights, it emphasizes important feature areas in the image and suppresses unimportant ones. This allows the model to focus more on signals that are helpful for the final task. The attention-extracted image is then weighted-fused with the original convolved image. This method preserves all information in the convolved image while strengthening the key areas of focus. This weighted fusion not only enhances the model's recognition capabilities but also prevents information loss.

[0120] In one implementation, the second attention module receives the image after the first fusion and performs attention extraction on it again, which can further refine the focus on image features. Especially when processing complex or high-dimensional data, the second attention mechanism can further extract more discriminative features from the fused feature map.

[0121] In one implementation, in the first attention module, the input feature map undergoes global maximum pooling and global average pooling to obtain the maximum value and average value of each channel respectively. The pooled results are input into two multi-layer perceptrons respectively. The importance of each channel is obtained through learning. The pooling results are processed by MLP and summed to obtain the attention weight of each channel. Finally, the obtained channel attention tensor will also generate the channel attention weight through the Sigmoid activation function. The generated channel attention weight is multiplied with the original feature map in the channel dimension to obtain the first attention extraction image.

[0122] In one implementation, in the second attention module, the input feature map undergoes maximum pooling and average pooling, and the results of maximum pooling and average pooling are spliced ​​in the channel dimension. Then, a convolution operation is performed through a 7×1 convolution kernel. The convolution result is converted into a spatial attention weight through a Sigmoid activation function. This weight is used to adjust the importance of each spatial position in the input feature map. The generated spatial attention weight is multiplied with the first attention fusion image in the channel dimension to obtain the second attention extraction image.

[0123] In one implementation, all weighted fusions in the entire text are equally weighted. For example, if two images are weightedly fused, the weights corresponding to the two images are both 0.5.

[0124] Based on the same inventive concept, the present invention also provides a surface water quality detection device based on spectroscopy. Figure 2 , Figure 2 A schematic structural diagram of a surface water quality detection device based on spectroscopy provided in an embodiment of the present invention includes:

[0125] A spectrum acquisition module is used to obtain the ultraviolet-visible absorption spectrum corresponding to the target detection area within a period of time to obtain a first spectrum set;

[0126] a spectrum preprocessing module, configured to preprocess each ultraviolet-visible absorption spectrum in the first spectrum set to obtain an initial spectrum, and obtain all the initial spectra to obtain an initial spectrum set;

[0127] a spectrum type recognition module, configured to perform spectrum type recognition based on the initial spectrum set, and if the type is recognized as a normal spectrum, remove abnormal spectra from the initial spectrum set to obtain a second spectrum set;

[0128] a target reference value determination module, configured to substitute a target spectrum into a preset model to obtain a target reference value; the target spectrum being any one of the second spectrum set;

[0129] The test result determination module is used to calculate the average value of all target reference values ​​to obtain a target comparison value, and substitute the target comparison value into a preset database to obtain a test result.

[0130] A surface water quality detection device based on spectroscopy provided by an embodiment of the present invention effectively removes noise interference by preprocessing the collected spectra and then performing type identification, thereby quickly screening out normal spectral data and improving overall detection efficiency. The target spectrum is substituted into a preset model, the target reference value is calculated, and the detection result is further obtained based on database comparison, thereby improving the reliability of the detection result.

[0131] In one embodiment, the spectrum preprocessing module includes:

[0132] A spectrum decomposition module is used to perform discrete wavelet transform on the ultraviolet-visible absorption spectrum to obtain coefficients corresponding to different frequencies to obtain a target coefficient set; the target coefficient set includes a low-frequency coefficient set and a high-frequency coefficient set;

[0133] A high-frequency coefficient determination module is used to process each high-frequency coefficient in the high-frequency coefficient set through a threshold function to obtain a target high-frequency coefficient;

[0134] The spectrum reconstruction module is used to obtain all target high-frequency coefficients to obtain a target high-frequency coefficient set, and reconstruct the ultraviolet-visible absorption spectrum according to the target high-frequency coefficient set and the low-frequency coefficient set to obtain an initial spectrum corresponding to the ultraviolet-visible absorption spectrum.

[0135] In one embodiment, the spectrum type recognition module includes:

[0136] a detection parameter vector determination module, configured to obtain, for each initial spectrum in the initial spectrum set, a detection parameter vector corresponding to the initial spectrum;

[0137] A serial number matrix generation module is used to construct a detection matrix based on the detection parameter vectors corresponding to all initial spectra, and to mark the parameter values ​​corresponding to each row in the detection matrix with serial numbers from large to small to obtain a serial number matrix;

[0138] The sorting module is used to re-sort each column in the sequence matrix according to the sequence number to obtain the target sorting set corresponding to each column;

[0139] A target difference determination module is used to calculate the ranking difference corresponding to each detection parameter in the target ranking set and the preset ranking set to obtain a target difference;

[0140] The difference summation module is used to obtain the target value according to the sum of all target differences corresponding to the column. If the target value is less than the preset threshold, the target value is recorded as a valid value;

[0141] The type determination module is used to calculate the ratio of the number of valid values ​​to the number of target values ​​to obtain a target ratio. If the target ratio is greater than a preset ratio, the type of the initial spectrum set is identified as a normal spectrum.

[0142] In one embodiment, the preset model includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, a preset attention module, a preset fusion module, a global average pooling layer, a flattening layer, and a linear layer. Substituting the initial spectrum into the preset model to obtain a target reference value includes:

[0143] Obtain an initial spectrum, substitute the initial spectrum into the first convolution block to obtain a first convolution image, and substitute the first convolution image into the second convolution block to obtain a second convolution image;

[0144] Substituting the second convolution image into a preset attention module to obtain a first attention image, and substituting the first attention image into a preset fusion module to obtain a first fused image;

[0145] Substituting the first fused image into the third convolution block to obtain a third convolution image, and substituting the third convolution image into the preset fusion module to obtain a second fused image;

[0146] Substituting the second fused image into the fourth convolution block to obtain a fourth convolution image, and substituting the fourth convolution image into the preset fusion module to obtain a third fused image;

[0147] Substituting the third fused image into the fifth convolution block to obtain a fifth convolution image, and substituting the fifth convolution image into the preset attention module to obtain a second attention image;

[0148] The second attention image is substituted into the global average pooling layer and then passed through the flattening layer and the linear layer to obtain the reference value.

[0149] In one embodiment, the preset attention module includes a first attention module and a second attention module;

[0150] The first attention module receives the convolution image and performs attention extraction on the convolution image to obtain a first attention extraction image;

[0151] The first attention extracted image and the convolution image are weighted fused to obtain the first attention fused image;

[0152] The second attention module receives the first attention fusion image and performs attention extraction on the first attention fusion image to obtain a second attention extraction image;

[0153] The attention image is obtained by weighted fusion of the second attention extraction image and the first attention extraction image.

[0154] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and should not be considered to limit the scope of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A surface water quality detection method based on spectroscopy, characterized in that: The method comprises: Obtaining the ultraviolet-visible absorption spectrum corresponding to the target detection area within a period of time to obtain a first spectrum set; For each ultraviolet-visible absorption spectrum in the first spectrum set, preprocess the ultraviolet-visible absorption spectrum to obtain an initial spectrum, and obtain all the initial spectra to obtain an initial spectrum set; Performing spectrum type identification based on the initial spectrum set, and if the type identification is a normal spectrum, removing abnormal spectra from the initial spectrum set to obtain a second spectrum set; if the type identification is an abnormal spectrum, re-collecting ultraviolet-visible absorption spectra as the first spectrum set; Substitute the target spectrum into the preset model to obtain the target reference value; The target spectrum is any one of the second spectrum set; Calculate the average of all target reference values ​​to obtain a target comparison value, and substitute the target comparison value into a preset database to obtain a test result; Identifying the spectrum type according to the initial spectrum set includes: For each initial spectrum in the initial spectrum set, obtaining a detection parameter vector corresponding to the initial spectrum; the detection parameter vector is a combination of any of the following: Mahalanobis distance, Q-residual, spectral divergence variable, Euclidean distance, Pearson correlation coefficient, and Jacard similarity; Constructing a detection matrix based on the detection parameter vectors corresponding to all initial spectra, and marking the parameter values ​​corresponding to each row in the detection matrix with serial numbers from large to small to obtain a serial number matrix; For each column in the sequence number matrix, reorder according to the sequence number to obtain a target sorting set corresponding to each column; Calculate the ranking difference between the target ranking set and the preset ranking set corresponding to each detection parameter to obtain a target difference value; The target value is obtained by summing up all target differences corresponding to the column. If the target value is less than the preset threshold, the target value is recorded as a valid value. A target ratio is obtained by calculating a ratio of the number of valid values ​​to the number of target values. If the target ratio is greater than a preset ratio, the type of the initial spectrum set is identified as a normal spectrum; if the target ratio is not greater than the preset ratio, the type of the initial spectrum set is identified as an abnormal spectrum.

2. The surface water quality detection method based on spectroscopy according to claim 1, characterized in that: The UV-visible absorption spectrum is preprocessed to obtain an initial spectrum including: Performing discrete wavelet transform on the ultraviolet-visible absorption spectrum to obtain coefficients corresponding to different frequencies to obtain a target coefficient set; the target coefficient set includes a low-frequency coefficient set and a high-frequency coefficient set; For each high frequency coefficient in the high frequency coefficient set, processing the high frequency coefficient by a threshold function to obtain a target high frequency coefficient; All target high-frequency coefficients are obtained to obtain a target high-frequency coefficient set, and an ultraviolet-visible absorption spectrum is reconstructed according to the target high-frequency coefficient set and the low-frequency coefficient set to obtain an initial spectrum corresponding to the ultraviolet-visible absorption spectrum.

3. The surface water quality detection method based on spectroscopy according to claim 1, characterized in that: The preset model includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, a preset attention module, a preset fusion module, a global average pooling layer, a flattening layer, and a linear layer. Substituting the initial spectrum into the preset model to obtain a target reference value includes: Obtaining the initial spectrum, substituting the initial spectrum into a first convolution block to obtain a first convolution image, and substituting the first convolution image into a second convolution block to obtain a second convolution image; Substituting the second convolution image into a preset attention module to obtain a first attention image, and substituting the first attention image into a preset fusion module to obtain a first fused image; Substituting the first fused image into a third convolution block to obtain a third convolution image, and substituting the third convolution image into the preset fusion module to obtain a second fused image; Substituting the second fused image into the fourth convolution block to obtain a fourth convolution image, and substituting the fourth convolution image into the preset fusion module to obtain a third fused image; Substituting the third fused image into the fifth convolution block to obtain a fifth convolution image, and substituting the fifth convolution image into the preset attention module to obtain a second attention image; The second attention image is substituted into the global average pooling layer and then passed through the flattening layer and the linear layer to obtain the reference value.

4. The surface water quality detection method based on spectroscopy according to claim 3, characterized in that: The preset attention module includes a first attention module and a second attention module; The first attention module receives the convolution image and performs attention extraction on the convolution image to obtain a first attention extraction image; The first attention extraction image and the convolution image are weightedly fused to obtain a first attention fusion image; The second attention module receives the first attention fusion image and performs attention extraction on the first attention fusion image to obtain a second attention extraction image; The second attention extraction image and the first attention extraction image are weightedly fused to obtain an attention image.

5. A surface water quality detection device based on spectroscopy, characterized in that: The device comprises: A spectrum acquisition module is used to obtain the ultraviolet-visible absorption spectrum corresponding to the target detection area within a period of time to obtain a first spectrum set; a spectrum preprocessing module, configured to preprocess each ultraviolet-visible absorption spectrum in the first spectrum set to obtain an initial spectrum, and obtain all the initial spectra to obtain an initial spectrum set; a spectrum type recognition module, configured to perform spectrum type recognition based on the initial spectrum set, and if the type recognition is normal, remove abnormal spectra from the initial spectrum set to obtain a second spectrum set; if the type recognition is abnormal, recollect ultraviolet-visible absorption spectra as the first spectrum set; a target reference value determination module, configured to substitute a target spectrum into a preset model to obtain a target reference value; the target spectrum being any one of the second spectrum set; A test result determination module is used to calculate the average value of all target reference values ​​to obtain a target comparison value, and substitute the target comparison value into a preset database to obtain a test result; The spectrum type recognition module includes: a detection parameter vector determination module, configured to obtain, for each initial spectrum in the initial spectrum set, a detection parameter vector corresponding to the initial spectrum; the detection parameter vector being a combination of any of the following: Mahalanobis distance, Q-residual, spectral divergence variable, Euclidean distance, Pearson correlation coefficient, and Jacard similarity; A serial number matrix generation module is used to construct a detection matrix based on the detection parameter vectors corresponding to all initial spectra, and to mark the parameter values ​​corresponding to each row in the detection matrix with serial numbers from large to small to obtain a serial number matrix; A sorting module, configured to re-sort each column in the sequence number matrix according to the sequence number to obtain a target sorting set corresponding to each column; A target difference determination module is used to calculate the ranking difference corresponding to each detection parameter in the target ranking set and the preset ranking set to obtain a target difference; The difference summation module is used to obtain the target value according to the sum of all target differences corresponding to the column. If the target value is less than the preset threshold, the target value is recorded as a valid value; The type determination module is configured to calculate a ratio of the number of valid values ​​to the number of target values ​​to obtain a target ratio. If the target ratio is greater than a preset ratio, the type of the initial spectrum set is identified as a normal spectrum; if the target ratio is not greater than the preset ratio, the type of the initial spectrum set is identified as an abnormal spectrum.

6. The surface water quality detection device based on spectroscopy according to claim 5, characterized in that: The spectrum preprocessing module includes: a spectrum decomposition module, configured to perform discrete wavelet transform on the ultraviolet-visible absorption spectrum to obtain coefficients corresponding to different frequencies to obtain a target coefficient set; the target coefficient set includes a low-frequency coefficient set and a high-frequency coefficient set; a high-frequency coefficient determination module, configured to process each high-frequency coefficient in the high-frequency coefficient set by a threshold function to obtain a target high-frequency coefficient; The spectrum reconstruction module is used to obtain all target high-frequency coefficients to obtain a target high-frequency coefficient set, and reconstruct the ultraviolet-visible absorption spectrum according to the target high-frequency coefficient set and the low-frequency coefficient set to obtain an initial spectrum corresponding to the ultraviolet-visible absorption spectrum.

7. The surface water quality detection device based on spectroscopy according to claim 5, characterized in that: The preset model includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, a preset attention module, a preset fusion module, a global average pooling layer, a flattening layer, and a linear layer. Substituting the initial spectrum into the preset model to obtain a target reference value includes: Obtaining the initial spectrum, substituting the initial spectrum into a first convolution block to obtain a first convolution image, and substituting the first convolution image into a second convolution block to obtain a second convolution image; Substituting the second convolution image into a preset attention module to obtain a first attention image, and substituting the first attention image into a preset fusion module to obtain a first fused image; Substituting the first fused image into a third convolution block to obtain a third convolution image, and substituting the third convolution image into the preset fusion module to obtain a second fused image; Substituting the second fused image into the fourth convolution block to obtain a fourth convolution image, and substituting the fourth convolution image into the preset fusion module to obtain a third fused image; Substituting the third fused image into the fifth convolution block to obtain a fifth convolution image, and substituting the fifth convolution image into the preset attention module to obtain a second attention image; The second attention image is substituted into the global average pooling layer and then passed through the flattening layer and the linear layer to obtain the reference value.

8. The surface water quality detection device based on spectroscopy according to claim 7, characterized in that: The preset attention module includes a first attention module and a second attention module; The first attention module receives the convolution image and performs attention extraction on the convolution image to obtain a first attention extraction image; The first attention extraction image and the convolution image are weightedly fused to obtain a first attention fusion image; The second attention module receives the first attention fusion image and performs attention extraction on the first attention fusion image to obtain a second attention extraction image; The second attention extraction image and the first attention extraction image are weightedly fused to obtain an attention image.

Images