A method for extracting full-spectrum characteristic maps of water bodies

By integrating the methods of spectral signal acquisition, processing and feature extraction, and using multi-scale sparse autoencoder and variational modal decomposition technology, a full spectrum feature map was generated, which solved the problems of baseline drift, noise processing and feature extraction in the spectral data analysis of contaminated water bodies, significantly improving the analysis accuracy and efficiency.

CN119415932BActive Publication Date: 2025-04-15CHENGDU YIQINGYUAN TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510015573.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-01-06
Publication Date
2025-04-15
Estimated Expiration
2045-01-06

AI Technical Summary

Technical Problem

When processing spectral data of contaminated water bodies, the prior art has problems such as limited baseline drift correction accuracy, insufficient noise processing capability, and insufficient feature extraction sparsity and visual details.

Method used

By integrating the acquisition, processing and feature extraction of spectral signals, a multi-scale sparse autoencoder is used for deep feature extraction, and a full spectrum feature map is generated by combining variational modal decomposition and color mapping technology.

Benefits of technology

It significantly improves the analysis accuracy and efficiency of spectral data of polluted water bodies, accurately deals with complex background drifts, takes into account high-frequency details retention and low-frequency fluctuation suppression, and enhances feature sparsity and analytical efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119415932B_ABST
    Figure CN119415932B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for extracting a full-spectrum characteristic map of a water body, and relates to the field of water body spectrum extraction. The present invention significantly improves the accuracy and efficiency of polluted water analysis by integrating spectral signal acquisition, processing and feature extraction in the whole process. In baseline correction, the local minimum method and polynomial fitting are combined to accurately deal with complex background drift; in noise filtering, the variational mode decomposition method is adopted to take into account both high-frequency detail retention and low-frequency fluctuation suppression; in feature extraction, a sparse autoencoder model is used to enhance feature sparsity and parsing efficiency. Finally, the interpolation objective function and gradient compensation term are defined to generate a two-dimensional feature matrix and a full-spectrum characteristic map, which accurately reflects the spectral characteristics of pollutants and their changes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the field of water body spectrum extraction, and in particular to a method for extracting a full-spectrum characteristic spectrum of a water body. Background Art

[0002] At present, spectral analysis technology is widely used in water quality testing, especially in analyzing the content of organic and inorganic substances in polluted water bodies. Spectroscopic technology uses the absorption characteristics of substances to light of different wavelengths to quantitatively analyze target substances in water bodies by measuring absorption or scattering spectra. Ultraviolet-visible-near infrared (UV-VIS-NIR) spectral data acquisition equipment has been highly developed and can cover spectral signals in the range of 200-2500 nm. Generally, the ultraviolet band is used to analyze organic matter, especially molecular structures containing conjugated double bonds; the visible band can provide basic information on chromaticity and optical absorption; and the near-infrared band is more suitable for analyzing water content, molecular vibrations, and chemical bond characteristics of molecules.

[0003] There are several key issues that need to be addressed when processing polluted water spectral data using existing technologies. First, the accuracy of baseline drift correction is limited. Most existing baseline correction methods rely on polynomial fitting or local correction methods. Although these techniques can adapt to simple background changes, they are unable to cope with the nonlinear drift and multiple background interferences present in complex polluted water spectral signals. In this case, the corrected baseline may not truly reflect the background distribution of the spectral signal, thus affecting the accuracy of subsequent analysis. Secondly, noise processing methods are not sufficiently adaptable to high-frequency noise and low-frequency fluctuations. Existing noise filtering techniques are usually designed for a specific frequency band and cannot fully balance the retention of high-frequency details and the removal of low-frequency background. In addition, traditional filtering techniques may cause key features in the signal to be over-smoothed, resulting in the loss of important information.

[0004] In terms of deep feature extraction, although existing technologies have achieved dimensionality reduction and feature compression of spectral data through deep learning algorithms, existing models have limitations in extracting sparse features and retaining key details of spectral signals. For example, the sparsity control of deep learning models is insufficient, and too many activated units in the hidden layer may cause the extracted features to be too redundant, reducing the parsing efficiency. More importantly, when the existing technology generates a visual map after feature extraction, it usually projects the signal into a two-dimensional feature map through simple interpolation or color mapping. However, this method lacks the ability of gradient compensation and local feature optimization, and it is difficult to accurately describe the slight changes in spectral data, especially the complex change patterns in the wavelength and frequency directions.

[0005] In summary, existing technologies lack the ability to comprehensively analyze the spectral data of organic and inorganic substances in complex polluted water bodies, especially in the integration of multi-step signal processing, sparse optimization of feature extraction, and detailed expression of visualization results. Summary of the invention

[0006] The present invention significantly improves the analysis capability of polluted water spectral data by systematically integrating the collection, processing and feature extraction of spectral signals.

[0007] A method for extracting a full-spectrum characteristic map of a water body comprises the following steps:

[0008] S1. Collect spectral data of organic and inorganic substances in polluted water in the ultraviolet 200-400nm, visible light 400-800nm, and near infrared 800-2500nm bands respectively;

[0009] S2. Process the low-frequency background signal in the spectral data and perform baseline correction on the baseline drift by low-order polynomial fitting;

[0010] S3. Process the high-frequency noise and low-frequency fluctuations of the baseline-corrected signal and filter out the noise through variational mode decomposition;

[0011] S4. Perform deep feature extraction on the processed signal through a multi-scale sparse autoencoder;

[0012] S5. Establish a two-dimensional feature matrix based on the features extracted from the deep features, and generate the two-dimensional feature matrix into a full-spectrum feature map through color mapping;

[0013] Wherein, the step S4 specifically includes the following sub-steps:

[0014] S401. Constructing a multi-scale sparse autoencoder including an input layer, a hidden layer and an output layer, wherein the input layer of the multi-scale sparse autoencoder simultaneously receives the original signal and its transformed signals of different scales, constructs a multi-branch structure, and the processed signal is compressed to the hidden layer by a linear mapping and an activation function through the encoder, and the hidden layer representation is then restored to the input through the decoder;

[0015] S402. Construct a loss function with the goal of minimizing the reconstruction error, introduce a sparsity constraint, limit the number of hidden layer activation units, and set different sparsity weights for different branch features;

[0016] S403. Output the sparse features of the spectral data through the hidden layer.

[0017] Furthermore, the step S2 specifically includes the following sub-steps:

[0018] S201. Perform preliminary smoothing by a Gaussian smoothing filter;

[0019] S202. Using the local minimum method, points in the low light signal area in the spectral data are selected as a baseline point set;

[0020] S203. Based on the selected baseline point, a preliminary baseline function is generated using polynomial fitting;

[0021] S204. Subtract the fitted baseline from the original spectrum to obtain a corrected spectrum signal.

[0022] Furthermore, in step S203, the baseline function is expressed as:

[0023] ;

[0024] The objective function of the baseline function is:

[0025] ;

[0026] Among them, the represents the baseline function of the ith band, represents the fitting coefficient, represents the wavelength, represents the baseline fitting function of the ith band, represents the wavelength of the i-th band, represents the order of the polynomial, and the optimization goal of the objective function is to minimize the deviation between the actual spectral signal and the fitted baseline.

[0027] Furthermore, in step S204, the corrected spectral signal is expressed as:

[0028] ;

[0029] Among them, the represents the corrected spectral signal, Represents the original acquired spectral signal.

[0030] Furthermore, the step S3 specifically includes the following sub-steps:

[0031] S301. Decomposing the spectral signal into a plurality of eigenmodes, each mode representing a signal component within a specific frequency band;

[0032] S302. Solve the optimization problem to obtain a signal for each frequency band;

[0033] S303. Filter out noise and retain the main components of the signal by adjusting the number of decomposition modes and the center frequency.

[0034] Furthermore, the multi-scale sparse multi-branch structure includes:

[0035] Main branch: the branch that inputs the original spectral signal;

[0036] High-frequency branch: a branch that inputs high-frequency signals after high-pass filtering;

[0037] Low-frequency branch: The branch that inputs the low-frequency signal after passing through the low-pass filter.

[0038] Furthermore, in step S402, the loss function is expressed as:

[0039] ;

[0040] Among them, the represents the loss function, represents the original input signal, the represents the signal reconstructed by the autoencoder, and Respectively represent the sparsity weights corresponding to the main signal features, high-frequency features and low-frequency features, , and Respectively represent the target mean of the sparsity of the main signal feature, high-frequency feature and low-frequency feature, represents the number of hidden layers, Represents the Kullback-Leibler divergence, which is a measure of the difference between two distributions.

[0041] Furthermore, in step S5, establishing a two-dimensional feature matrix according to the features extracted from the deep features specifically includes the following sub-steps:

[0042] S501. According to the sparse features of the spectral data, the features are mapped to a two-dimensional feature matrix through a basis function;

[0043] S502. Define an interpolation objective function, wherein the objective function includes a known value fitting term and a gradient compensation term;

[0044] S503. Estimate the local changes of the characteristic matrix in the wavelength direction and the frequency direction by using a gradient operator;

[0045] S504. Iteratively update by gradient descent method;

[0046] S505. After the interpolation is completed, a complete spectral feature matrix is ​​obtained.

[0047] Furthermore, in step S503, the interpolation objective function is expressed as:

[0048] ;

[0049] Among them, the represents the objective function of interpolation, Respectively represent the indexes in the wavelength direction and the frequency direction, represents the weight of the data point, In the matrix The element at position represents the known sparse feature data points, Represents the smoothing coefficient of the gradient compensation term, which controls the influence of the gradient penalty. represents a two-dimensional feature matrix, represents the wavelength, Indicates the frequency, Represents a two-dimensional feature matrix For wavelength and frequency The gradient of Represents the square of the Euclidean norm.

[0050] Furthermore, in step S504, the iterative update of the gradient descent method is expressed as:

[0051] ;

[0052] Among them, the represents the number of iterations, Represents the learning rate.

[0053] The beneficial effects of the invention are:

[0054] The present invention significantly improves the accuracy and efficiency of polluted water analysis by integrating spectral signal acquisition, processing and feature extraction throughout the entire process. The local minimum method and polynomial fitting are combined in baseline correction to accurately deal with complex background drift; the variational mode decomposition method is used in noise filtering to balance high-frequency detail retention and low-frequency fluctuation suppression; the sparse autoencoder model is used in feature extraction to enhance feature sparsity and parsing efficiency. Finally, the interpolation objective function and gradient compensation terms are defined to generate a two-dimensional feature matrix and a full-spectrum feature map to accurately reflect the spectral characteristics of pollutants and their changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A method flow chart of a method for extracting a full-spectrum characteristic map of a water body provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings, but the protection scope of the present invention is not limited to the following.

[0057] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention, that is, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0058] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative work are within the scope of protection of the present invention. It should be noted that relational terms such as the terms "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0059] Moreover, the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a list of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or apparatus. In the absence of more restrictions, an element defined by the phrase "comprises a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0060] The features and performance of the present invention are further described in detail below in conjunction with the embodiments.

[0061] Among them, Figure 1 , a method for extracting a full-spectrum characteristic map of a water body, comprising the following steps:

[0062] S1. Collect spectral data of organic and inorganic substances in polluted water in the ultraviolet 200-400nm, visible light 400-800nm, and near infrared 800-2500nm bands respectively;

[0063] S2. Process the low-frequency background signal in the spectral data and perform baseline correction on the baseline drift by low-order polynomial fitting;

[0064] S3. Process the high-frequency noise and low-frequency fluctuations of the baseline-corrected signal and filter out the noise through variational mode decomposition;

[0065] S4. Perform deep feature extraction on the processed signal through a multi-scale sparse autoencoder;

[0066] S5. Establish a two-dimensional feature matrix based on the features extracted from the deep features, and generate the two-dimensional feature matrix into a full-spectrum feature map through color mapping;

[0067] Wherein, the step S4 specifically includes the following sub-steps:

[0068] S401. Constructing a multi-scale sparse autoencoder including an input layer, a hidden layer and an output layer, wherein the input layer of the multi-scale sparse autoencoder simultaneously receives the original signal and its transformed signals of different scales, constructs a multi-branch structure, and the processed signal is compressed to the hidden layer by a linear mapping and an activation function through the encoder, and the hidden layer representation is then restored to the input through the decoder;

[0069] Specifically, the process principle of the above embodiment is as follows: a three-layer autoencoder network model is constructed, and the structure includes an input layer, a hidden layer and an output layer. The input layer receives the preprocessed spectral signal, and each input node corresponds to a wavelength or frequency point in the spectral signal. The encoder part maps the signal from the high-dimensional space to the low-dimensional space of the hidden layer, compresses the input data by setting appropriate weights and bias parameters, and enhances the nonlinear expression ability of the signal characteristics through the activation function. The hidden layer is used to capture the core features of the data; the decoder part remaps the hidden layer back to the original data dimension to generate a signal similar to the input layer.

[0070] S402. Construct a loss function with the goal of minimizing the reconstruction error, introduce a sparsity constraint, limit the number of hidden layer activation units, and set different sparsity weights for different branch features;

[0071] Specifically, the process principle of the above embodiment is: design a loss function and minimize the difference between the input signal and the output signal as the optimization goal. At the same time, in order to enhance the sparse characteristics of the hidden layer, a sparsity constraint is introduced, and by limiting the activation of fewer nodes in the hidden layer, the network only focuses on the most significant characteristic signal in the spectrum and ignores redundant information. This method of limiting the activation of the hidden layer can prevent overfitting and highlight the characteristic differences of different pollutants in the spectrum, thereby improving the accuracy of feature extraction.

[0072] S403. Outputting sparse features of spectral data through a hidden layer;

[0073] Specifically, the process principle of the above embodiment is as follows: after completing the autoencoder training, the output of the hidden layer is the sparse features of the spectral data, which is expressed as a signal representation after dimensionality reduction and feature extraction. The sparse feature set contains the key information of the spectral signal and eliminates the noise and redundant interference parts. The sparse features retain the main absorption characteristics of the spectral signal and can be directly used for the subsequent construction of the two-dimensional feature matrix and interpolation optimization processing.

[0074] Furthermore, the step S2 specifically includes the following sub-steps:

[0075] S201. Perform preliminary smoothing by a Gaussian smoothing filter;

[0076] Specifically, the process principle of the above embodiment is as follows: the Gaussian smoothing filter performs weighted averaging of adjacent data points through a weight distribution function to smooth the local fluctuations of the spectral signal while retaining the overall trend. By adjusting the standard deviation of the Gaussian function, the degree of smoothing can be controlled to ensure that the key features in the signal are not excessively weakened, thereby providing a more reliable data basis for the subsequent selection of baseline points.

[0077] S202. Using the local minimum method, points in the low light signal area in the spectral data are selected as a baseline point set;

[0078] Specifically, the process principle of the above embodiment is: the baseline points are usually distributed in the area with low signal intensity in the spectrum, and can represent the main direction of the spectral baseline. The specific operation process is: for each wavelength point, check the value of its adjacent area, identify the local minimum and mark it as a candidate baseline point. In addition, the selected baseline point set needs to meet the requirements of covering the entire spectral range and being evenly distributed. In addition, the low light signal area refers to the area with relatively low absorbance in spectral measurement, which is usually an area without absorption in the sample. In this area, the instrument response is relatively stable and is not affected by the light absorption characteristics of the sample. Therefore, the data in these areas can be used as reference points for estimating baseline drift. In these areas, the signal is mainly composed of interference from instrument background noise and environmental factors, reflecting the low-frequency drift of the instrument.

[0079] S203. Based on the selected baseline point, a preliminary baseline function is generated using polynomial fitting;

[0080] Specifically, the process principle of the above embodiment is as follows: polynomial fitting constructs a polynomial that best matches the baseline point set to fit the global baseline trend of the spectrum. The order of the polynomial needs to be selected according to the complexity of the spectral data: when the order is low, it is suitable for simple and smooth baselines; when the order is high, it is suitable for processing complex spectra with large baseline fluctuations. The baseline function finally generated is used as the basis for subsequent baseline correction to eliminate the influence of baseline drift in the spectrum.

[0081] S204. Subtracting the fitted baseline from the original spectrum to obtain a corrected spectral signal;

[0082] Specifically, the process principle of the above embodiment is: subtract the baseline function value at the corresponding wavelength from the original absorption value at each wavelength point to obtain the corrected spectral signal. This process effectively eliminates the baseline drift component in the spectral signal, making the characteristic absorption peak of the signal clearer and more accurate.

[0083] Furthermore, in step S203, the baseline function is expressed as:

[0084] ;

[0085] The objective function of the baseline function is:

[0086] ;

[0087] Among them, the represents the baseline function of the ith band, represents the fitting coefficient, represents the wavelength, represents the baseline fitting function of the ith band, represents the wavelength of the i-th band, represents the order of the polynomial, and the optimization goal of the objective function is to minimize the deviation between the actual spectral signal and the fitted baseline.

[0088] Furthermore, in step S204, the corrected spectral signal is expressed as:

[0089] ;

[0090] Among them, the represents the corrected spectral signal, Represents the original acquired spectral signal.

[0091] Furthermore, the step S3 specifically includes the following sub-steps:

[0092] S301. Decomposing the spectral signal into a plurality of eigenmodes, each mode representing a signal component within a specific frequency band;

[0093] Specifically, the process principle of the above embodiment is as follows: each eigenmode represents the signal component of the spectral signal in a specific frequency band, covering high-frequency noise, low-frequency background and the main characteristic signal of the intermediate frequency. During the decomposition process, the complexity of the signal is decomposed into each frequency band by optimizing the energy distribution, so that each mode can be analyzed and processed independently, thereby effectively separating the noise from the main signal.

[0094] S302. Solve the optimization problem to obtain a signal for each frequency band;

[0095] Specifically, the process principle of the above embodiment is: by iteratively solving the optimization problem, the center frequency and signal components of each mode are gradually updated. The optimization result directly outputs multiple eigenmodes, which correspond to different frequency bands of the spectral signal. The signal components in these modes can clearly separate high-frequency noise and low-frequency background, and retain important information in the spectral signal.

[0096] S303. By adjusting the number of decomposition modes and the center frequency, the noise is filtered out and the main components of the signal are retained;

[0097] Specifically, the process principle of the above embodiment is: suppress the mode containing high-frequency noise, and retain the main signal components of the intermediate frequency mode. By accurately adjusting these parameters, the noise can be effectively removed while maintaining the integrity and key features of the spectral signal. The final output spectral signal has low noise interference, providing input data for deep feature extraction of the sparse autoencoder.

[0098] Furthermore, the multi-scale sparse multi-branch structure includes:

[0099] Main branch: the branch that inputs the original spectral signal;

[0100] High-frequency branch: a branch that inputs high-frequency signals after high-pass filtering;

[0101] Low-frequency branch: The branch that inputs the low-frequency signal after passing through the low-pass filter.

[0102] Furthermore, in step S402, the loss function is expressed as:

[0103] ;

[0104] Among them, the represents the loss function, represents the original input signal, the represents the signal reconstructed by the autoencoder, and Respectively represent the sparsity weights corresponding to the main signal features, high-frequency features and low-frequency features, , and Respectively represent the target mean of the sparsity of the main signal feature, high-frequency feature and low-frequency feature, represents the number of hidden layers, Represents the Kullback-Leibler divergence, which is a measure of the difference between two distributions.

[0105] Furthermore, in step S5, establishing a two-dimensional feature matrix according to the features extracted from the deep features specifically includes the following sub-steps:

[0106] S501. According to the sparse features of the spectral data, the features are mapped to a two-dimensional feature matrix through a basis function;

[0107] Specifically, the specific principle flow of the above embodiment is: using the feature data output from the hidden layer of the sparse autoencoder, selecting an appropriate basis function, converting the sparse features into a two-dimensional feature matrix, the rows and columns of the matrix correspond to the spectral wavelength and frequency respectively, and generating a preliminary two-dimensional matrix by reorganizing the feature data and projecting the basis function. Each unit of the feature matrix represents the absorbance feature under a certain wavelength and frequency combination, ensuring that the entire matrix can fully express the overall change trend of the spectrum and provide an initial template for interpolation compensation.

[0108] S502. Define an interpolation objective function, wherein the objective function includes a known value fitting term and a gradient compensation term;

[0109] Specifically, the specific principle process of the above embodiment is as follows: after constructing the preliminary feature matrix, in order to fill in the missing values ​​therein, the goal of interpolation optimization is clarified. The goal of interpolation comprehensively considers both data fitting and gradient smoothing, and makes the interpolation result both accurate and stable by optimizing the fitting degree of known points and limiting the variation range of the matrix in the wavelength and frequency directions.

[0110] S503. Estimate the local changes of the characteristic matrix in the wavelength direction and the frequency direction by using a gradient operator;

[0111] Specifically, the specific principle flow of the above embodiment is as follows: during the interpolation process, the gradient of the feature matrix is ​​calculated to evaluate its change trend in different directions, and the gradient operator is used to analyze the change amplitude of the surrounding values ​​of each unit in the matrix to determine the rationality of the interpolation point. After the preliminary interpolation is completed, the feature matrix is ​​smoothly adjusted according to the gradient distribution to reduce unreasonable mutation points.

[0112] S504. Iterative update by gradient descent method: After completing the initial interpolation result, the feature matrix is ​​optimized and updated multiple times by gradient descent method, and the value in the matrix is ​​adjusted according to the error feedback of the objective function in each iteration. Through multiple cycles of calculation, the error is gradually reduced, and finally an interpolation result with high fitting degree and smooth local gradient is obtained.

[0113] S505. After the interpolation is completed, a complete spectral feature matrix is ​​obtained;

[0114] Specifically, the specific principle flow of the above embodiment is: interpolation compensation is completed through multiple optimization iterations to generate a complete spectral feature matrix. In this matrix, each unit contains the absorption characteristics of the spectrum at the corresponding wavelength and frequency, and compensates for the missing points in the original data. This complete feature matrix can not only reflect the spectral absorption characteristics of organic and inorganic substances in polluted water, but also provide a complete and high-quality input basis for the generation of full-spectrum feature maps.

[0115] Furthermore, in step S503, the interpolation objective function is expressed as:

[0116] ;

[0117] Among them, the represents the objective function of interpolation, Respectively represent the indexes in the wavelength direction and the frequency direction, represents the weight of the data point, In the matrix The element at position represents the known sparse feature data points, Represents the smoothing coefficient of the gradient compensation term, which controls the influence of the gradient penalty. represents a two-dimensional feature matrix, represents the wavelength, Indicates the frequency, Represents a two-dimensional feature matrix For wavelength and frequency The gradient of Represents the square of the Euclidean norm.

[0118] Furthermore, in step S504, the iterative update of the gradient descent method is expressed as:

[0119] ;

[0120] Among them, the represents the number of iterations, Represents the learning rate.

[0121] Specifically, the objective function is the feature matrix Gradient , which is calculated in two parts:

[0122] Gradient of the fitted term:

[0123] ;

[0124] Gradient of the gradient regularization term:

[0125] ;

[0126] Among them, the gradient of the regularization term is specifically expanded as: ; ;

[0127] Combine the two parts:

[0128] .

[0129] In addition, for the initialization feature matrix, the feature matrix is ​​initialized to the initial value of the sparse feature points, then:

[0130] .

[0131] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not deviate from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A method for extracting a full-spectrum characteristic map of a water body, characterized in that: The following steps are involved: S1. Collect spectral data of organic and inorganic substances in polluted water in the ultraviolet 200-400nm, visible light 400-800nm, and near infrared 800-2500nm bands respectively; S2. Process the low-frequency background signal in the spectral data and perform baseline correction on the baseline drift by low-order polynomial fitting; S3. Process the high-frequency noise and low-frequency fluctuations of the baseline-corrected signal and filter out the noise through variational mode decomposition; S4. Perform deep feature extraction on the processed signal through a multi-scale sparse autoencoder; S5. Establish a two-dimensional feature matrix based on the features extracted from the deep features, and generate the two-dimensional feature matrix into a full-spectrum feature map through color mapping; Wherein, the step S4 specifically includes the following sub-steps: S401. Construct a multi-scale sparse autoencoder including an input layer, a hidden layer and an output layer, wherein the input layer of the multi-scale sparse autoencoder simultaneously receives the original signal and its transformed signals of different scales, constructs a multi-branch structure, and the processed signal is compressed to the hidden layer by a linear mapping and an activation function through the encoder, and the hidden layer representation is then restored to the input through the decoder, wherein the multi-branch structure comprises: Main branch: the branch that inputs the original spectral signal; High-frequency branch: a branch that inputs high-frequency signals after high-pass filtering; Low-frequency branch: a branch that inputs the low-frequency signal after low-pass filtering; S402. Construct a loss function with the goal of minimizing the reconstruction error, introduce a sparsity constraint, limit the number of hidden layer activation units, and set different sparsity weights for different branch features. The loss function is expressed as: ; Among them, the represents the loss function, represents the original input signal, the represents the signal reconstructed by the autoencoder, and Respectively represent the sparsity weights corresponding to the main signal features, high-frequency features and low-frequency features, , and Respectively represent the target mean of the sparsity of the main signal feature, high-frequency feature and low-frequency feature, represents the number of hidden layers, represents the Kullback-Leibler divergence, which is used to measure the difference between two distributions; S403. Output the sparse features of the spectral data through the hidden layer.

2. The method for extracting a water body full spectrum characteristic spectrum according to claim 1, characterized in that: The step S2 specifically includes the following sub-steps: S201. Perform preliminary smoothing by a Gaussian smoothing filter; S202. Using the local minimum method, points in the low light signal area in the spectral data are selected as a baseline point set; S203. Based on the selected baseline point, a preliminary baseline function is generated using polynomial fitting; S204. Subtract the fitted baseline from the original spectrum to obtain a corrected spectrum signal.

3. The method for extracting a water body full spectrum characteristic spectrum according to claim 2, characterized in that: In step S203, the baseline function is expressed as: ; The objective function of the baseline function is: ; Among them, the represents the baseline function of the ith band, represents the fitting coefficient, represents the wavelength, represents the baseline fitting function of the ith band, represents the wavelength of the i-th band, represents the order of the polynomial, and the optimization goal of the objective function is to minimize the deviation between the actual spectral signal and the fitted baseline.

4. The method for extracting a water body full spectrum characteristic spectrum according to claim 3, characterized in that: In step S204, the corrected spectral signal is expressed as: ; Among them, the represents the corrected spectral signal, Represents the original acquired spectral signal.

5. A method for extracting a full-spectrum characteristic map of a water body as claimed in claim 1, characterized in that: The step S3 specifically includes the following sub-steps: S301. Decomposing the spectral signal into a plurality of eigenmodes, each mode representing a signal component within a specific frequency band; S302. Solve the optimization problem to obtain a signal for each frequency band; S303. By adjusting the number of decomposition modes and the center frequency, the noise is filtered out and the main components of the signal are retained.

6. The method for extracting a water body full spectrum characteristic spectrum according to claim 1, characterized in that: In step S5, establishing a two-dimensional feature matrix based on the features extracted from the deep features specifically includes the following sub-steps: S501. According to the sparse features of the spectral data, the features are mapped to a two-dimensional feature matrix through a basis function; S502. Define an interpolation objective function, wherein the objective function includes a known value fitting term and a gradient compensation term; S503. Estimate the local changes of the characteristic matrix in the wavelength direction and the frequency direction by using a gradient operator; S504. Iteratively update by gradient descent method; S505. After the interpolation is completed, a complete spectral feature matrix is ​​obtained.

7. The method for extracting a full-spectrum characteristic map of a water body according to claim 6, characterized in that: In step S503, the interpolation objective function is expressed as: ; Among them, the represents the objective function of interpolation, Represents the index in wavelength direction and frequency direction, represents the weight of the data point, In the matrix The element at position represents the known sparse feature data points, Represents the smoothing coefficient of the gradient compensation term, which controls the influence of the gradient penalty. represents a two-dimensional feature matrix, represents the wavelength, Indicates the frequency, Represents a two-dimensional feature matrix For wavelength and frequency The gradient of Represents the square of the Euclidean norm.

8. The method for extracting a full-spectrum characteristic map of a water body according to claim 7, characterized in that: In step S504, the iterative update of the gradient descent method is expressed as: ; Among them, the represents the number of iterations, Represents the learning rate.

Citation Information

Patent Citations

  • Multispectral image fusion method and system based on frequency domain enhancement in degraded environment

    CN116452930A

  • Deep learning techniques for fast anomaly detection in experimental data

    US20240280522A1