Underwater sensor signal adaptive denoising method based on attention mechanism and CNN-LSTM

Abstract

The invention discloses an underwater sensor signal adaptive denoising method based on an attention mechanism and CNN-LSTM, and the method comprises the steps: S1, obtaining an original underwater signal collected by an underwater sensor, carrying out the preprocessing of the original underwater signal, carrying out the preliminary filtering of the signal through a band-pass filter, and removing an overhigh or overlow frequency component; s2, carrying out feature extraction on the preprocessed signal by adopting a convolutional neural network; and S3, performing time sequence information processing by adopting a long short-term memory network based on an attention mechanism. The invention discloses an underwater sensor signal adaptive denoising method based on an attention mechanism and CNN-LSTM, and aims to solve the problem of interference of complex noise in an underwater environment on sensor signals. In combination with an attention mechanism in deep learning, a key part in a signal is focused through self-adaptive weight distribution, so that a more accurate denoising effect is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of signal denoising, and in particular relates to an underwater sensor signal adaptive denoising method based on an attention mechanism and CNN-LSTM. Background Art

[0002] With the continuous advancement of science and technology, sensor technology has achieved rapid development and plays a vital role in many fields such as aviation, aerospace, navigation, Internet of Things, and medical care. As the front-end unit of the information acquisition system, the accuracy of the sensor output signal has long been a key issue in the scientific research field. The accuracy of the sensor signal directly affects the reliability of subsequent data analysis and processing. Accurate sensor signals can not only provide high-quality data support for the analysis system, but also improve the overall performance of the system. However, in the actual measurement process, interference from the external environment is inevitable, especially in complex underwater environments. Underwater noise will mix into the output signal of the sensor, resulting in a decrease in signal quality and increasing the difficulty of subsequent processing. Therefore, how to effectively remove the influence of noise in the underwater environment on the sensor signal has become a major challenge in the current research on sensor technology.

[0003] In recent years, with the vigorous development of machine learning technology, researchers have gradually applied it to the field of signal processing, especially in image denoising and audio denoising, and have achieved remarkable results. However, the denoising technology for underwater sensor output signals still faces some shortcomings and challenges. Although classic denoising methods, such as Fourier transform, wavelet transform, median filtering and Kalman filtering, have been widely used in the field of signal processing and have achieved certain results, with the continuous evolution of noise characteristics in complex environments and the improvement of signal requirements, these traditional methods have exposed some limitations in practical applications. As a basic tool for frequency domain analysis, Fourier transform can effectively separate signal components of different frequencies by converting signals from time domain to frequency domain. However, its shortcomings in processing actual complex signals are also obvious. Fourier transform can only globally describe the frequency components of the signal and cannot obtain time information at the same time, which makes it unable to effectively process non-stationary signals whose frequency changes with time. In underwater noisy environments, the frequency characteristics of the signal may change dynamically with the changes in water flow and environmental noise. The Fourier transform is difficult to identify the local change characteristics of the signal due to its lack of time resolution. In addition, the Fourier transform is based on the spectrum analysis of the entire signal and cannot cope with instantaneous changes or short-term noise interference, especially when dealing with sudden noise. At the same time, the Fourier transform assumes that the signal and noise are linearly superimposed, but in complex actual environments, noise often exhibits nonlinear characteristics, which makes the Fourier transform unable to accurately remove these noise components.

[0004] Adaptive filters are widely used in signal processing, especially in variable noise environments, to suppress noise by adjusting filter coefficients in real time. However, adaptive filtering methods also have defects. They rely on complex algorithms, such as the least mean square error (LMS) algorithm or the recursive least squares (RLS) algorithm, which have high computational complexity. Especially in large-scale signal processing or real-time processing tasks, it is often difficult to meet the requirements of speed and efficiency. In addition, the coefficient adjustment of the adaptive filter depends on the setting of parameters such as the learning rate. If the learning rate is not set properly, it may cause the filter to diverge or converge slowly, thereby affecting the noise suppression effect. In dynamic environments, the adaptability of parameter selection further becomes its main drawback. Although the above methods have improved the quality of sensor signals to a certain extent, most of them require human intervention for parameter adjustment, resulting in limited noise reduction performance. For example, in practical applications, these methods usually require manual selection of the best threshold or basis function, which increases the complexity of the operation. In addition, manual participation may also lead to the uncontrollability of the noise reduction effect. Especially in complex environments, the noise reduction performance of traditional methods is difficult to further improve.

[0005] In summary, although the existing signal processing technology has played a certain role in the field of sensor noise reduction, there is still much room for improvement in the processing of non-stationary signals, the degree of automation, and the accuracy of noise removal. Therefore, combining deep learning technology to develop more automated, efficient, and adaptable noise reduction methods has become an important research direction for improving the signal processing capabilities of underwater sensors. Summary of the invention

[0006] The main purpose of the present invention is to provide an adaptive denoising method for underwater sensor signals based on attention mechanism and CNN-LSTM, aiming to solve the problem of interference of complex noise on sensor signals in underwater environment. The noise in underwater environment is complex and diverse, and the signal often shows non-stationarity and high dynamic changes. Traditional denoising methods often perform poorly in such environments. The present invention combines the attention mechanism in deep learning, and focuses on the key parts of the signal by adaptively allocating weights, thereby achieving a more accurate denoising effect.

[0007] To achieve the above objectives, the present invention provides an underwater sensor signal adaptive denoising method based on attention mechanism and CNN-LSTM, comprising the following steps:

[0008] Step S1: obtaining the original underwater signal collected by the underwater sensor, and preprocessing the original underwater signal, performing preliminary filtering on the signal through a bandpass filter, removing excessively high or low frequency components, and obtaining a relatively stable signal, thereby constructing a sample library containing pure signals and noise signals;

[0009] Step S2: Use a convolutional neural network to extract features from the preprocessed signal. Through multi-layer convolution operations, the time and frequency features in the signal are extracted and a corresponding feature map is generated. The feature map reflects the local details and global information in the signal, including the different characteristics of the signal and noise.

[0010] Step S3: A long short-term memory network based on the attention mechanism is used to process time series information. On the basis of feature extraction, the attention mechanism is introduced to enhance the ability to focus on key features. By calculating the weights of each position in the feature map, the attention allocation of the model is adaptively adjusted, so that the model pays more attention to the key features in the signal and ignores the features corresponding to irrelevant or noise signals.

[0011] As a further preferred technical solution of the above technical solution, in step S3, the long short-term memory network based on the attention mechanism is specifically implemented as follows:

[0012] Step S3.1: Input the extracted feature map into the self-attention module to calculate the correlation between each position and generate a weight matrix;

[0013] Step S3.2: Perform weight assignment. According to the weight matrix, enhance the features with high correlation and suppress the related features with low correlation or noise signals, so as to optimize the feature representation. The weight matrix calculation formula is:

[0014]

[0015] Where Q represents the query matrix; K represents the key matrix (K T represents matrix transpose); d k Represents the dimension of the key vector (usually used to scale the dot product to prevent the gradient from being too large);

[0016] The dot product of K is scaled by the dimension of the key, and then the weight matrix W is calculated through the softmax function to ensure that the sum of all weights is 1;

[0017] The weights assigned by the weight matrix W are used to weight the features and sum them, thereby enhancing the features with high correlation and suppressing the features with low correlation or noise signals. The calculation formula is:

[0018] Output = W·V;

[0019] Among them, W is the weight matrix (calculated by the above formula), V is represented by the value matrix (eigenvector), and the final output features are generated by multiplying it with the weight matrix W.

[0020] As a further preferred technical solution of the above technical solution, in step S3, the timing dependency of the signal is captured, wherein:

[0021] The first step is to use the forget gate to determine the information to be discarded in the current state. The forget gate reads the hidden state h at the previous moment. t-1 and x t , after a sigmoid layer output f t For values ​​between 0 and 1, the calculation formula of the forget gate is:

[0022] f t =σ(W f [h t-1 x t ]+b f );

[0023] Among them, W f and b f is a trainable parameter; [h t-1 x t ] means concatenating the hidden state at the current moment with the input at the current moment; σ() means the sigmoid function;

[0024] In the second step, the new information added to the state of the cell is determined by the input gate, i t Represents the output of the input gate, which is a value between 0 and 1, indicating the amount of information to be retained. Its calculation formula is:

[0025] i t =σ(W i [h t-1 x t ]+b i );

[0026]

[0027] Among them, W i , W C represents the weight matrix; b i 、b C represents the corresponding bias; tanh() is the hyperbolic tangent function; C t Represents the new candidate unit state, each of its elements is between -1 and 1;

[0028] In the third step, the output gate weights the information stored in the memory cell with the input of the current time step to generate the output. t The calculation method is similar to the input gate. First, a sigmoid function is used to determine the degree of opening of the gate, and then a tanh function is used to generate the output vector. The calculation formula is:

[0029] O t =σ(W O [h t-1 x t]+b O );

[0030] h t =O t ×tanh(C t );

[0031] Where: W O represents the weight matrix; b O represents the corresponding bias; O t Represents the output of the output gate, and finally obtains the hidden state h at the current moment t .

[0032] As a further preferred technical solution of the above technical solution, in step S3, the model is trained using the constructed sample library to optimize the model weights.

[0033] As a further preferred technical solution of the above technical solution, step S3 further includes:

[0034] Step S4: Use the trained model to perform noise reduction processing on the original underwater signal actually collected in real time, so as to output a denoised signal.

[0035] The beneficial effects of the present invention are:

[0036] By combining the attention mechanism with the deep learning model, non-stationary signals in complex underwater environments can be quickly denoised, which greatly saves the time of manual intervention and the complexity of parameter adjustment, improves the accuracy and efficiency of signal processing, and performs particularly well in complex underwater environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is the overall architecture diagram of the present invention based on the attention mechanism and the CNN-LSTM model.

[0038] Figure 2 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0039] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

[0040] In the preferred embodiments of the present invention, those skilled in the art should note that the underwater sensors and the like involved in the present invention may be regarded as prior art.

[0041] Preferred embodiments.

[0042] like Figure 1-2 As shown, the present invention discloses an underwater sensor signal adaptive denoising method based on attention mechanism and CNN-LSTM, comprising the following steps:

[0043] Step S1: obtaining the original underwater signal collected by the underwater sensor, and preprocessing the original underwater signal, performing preliminary filtering on the signal through a bandpass filter, removing excessively high or low frequency components, and obtaining a relatively stable signal, thereby constructing a sample library containing pure signals and noise signals;

[0044] Step S2: Using a convolutional neural network (CNN) to extract features from the preprocessed signal, extract the time and frequency features in the signal through multi-layer convolution operations, and generate a corresponding feature map, which reflects the local details and global information in the signal, including the different characteristics of the signal and noise;

[0045] Step S3: A long short-term memory network (LSTM) based on an attention mechanism is used to process time series information. On the basis of feature extraction, an attention mechanism is introduced to enhance the ability to focus on key features (such as the features of pure signals). By calculating the weights of each position in the feature map, the attention allocation of the model (referring to the entire network architecture including the attention mechanism and LSTM) is adaptively adjusted, so that the model pays more attention to the key features in the signal and ignores the features corresponding to irrelevant or noise signals.

[0046] Specifically, in step S3, the long short-term memory network based on the attention mechanism is specifically implemented as follows:

[0047] Step S3.1: Input the extracted feature map into the self-attention module to calculate the correlation between each position (referring to each spatial position in the feature map) and generate a weight matrix;

[0048] Step S3.2: Perform weight assignment. According to the weight matrix, enhance the features with high correlation and suppress the related features with low correlation or noise signals, so as to optimize the feature representation. The weight matrix calculation formula is:

[0049]

[0050] Where Q represents the query matrix; K represents the key matrix (K T represents matrix transpose); d k Represents the dimension of the key vector (usually used to scale the dot product to prevent the gradient from being too large);

[0051] The dot product of K is scaled by the dimension of the key, and then the weight matrix W is calculated through the softmax function to ensure that the sum of all weights is 1;

[0052] The weights assigned by the weight matrix W are used to weight the features and sum them, thereby enhancing the features with high correlation and suppressing the features with low correlation or noise signals. The calculation formula is:

[0053] Output = W·V;

[0054] Among them, W is the weight matrix (calculated by the above formula), V is represented by the value matrix (eigenvector), and the final output features are generated by multiplying it with the weight matrix W.

[0055] More specifically, in step S3, (in the denoising task, the time series characteristics of the signal are crucial. The present invention further introduces a long short-term memory network (LSTM) to) capture the temporal dependency of the signal (LSTM can process the long-term dependency in the signal, ensuring that the model can effectively use the historical information of the signal to improve the denoising effect during the denoising process), wherein:

[0056] The first step is to use the forget gate to determine the information to be discarded in the current state. The forget gate reads the hidden state h at the previous moment. t-1 and x t , after a sigmoid layer output f t For values ​​between 0 and 1, the calculation formula of the forget gate is:

[0057] f t =σ(W f [h t-1 x t ]+b f );

[0058] Among them, W f and b f is a trainable parameter; [h t-1 x t ] means concatenating the hidden state at the current moment with the input at the current moment; σ() means the sigmoid function;

[0059] In the second step, the new information added to the state of the memory cell is determined by the input gate, i t Represents the output of the input gate, which is a value between 0 and 1, indicating the amount of information to be retained. Its calculation formula is:

[0060] i t =σ(W i [h t-1 x t ]+b i );

[0061]

[0062] Among them, W i , W C represents the weight matrix; b i 、b C represents the corresponding bias; tanh() is the hyperbolic tangent function; C t Represents the new candidate unit state, each of its elements is between -1 and 1;

[0063] In the third step, the output gate weights the information stored in the memory cell with the input of the current time step to generate the output. t The calculation method is similar to the input gate. First, a sigmoid function is used to determine the degree of opening of the gate, and then a tanh function is used to generate the output vector. The calculation formula is:

[0064] O t =σ(W O [h t-1 x t ]+b O );

[0065] h t =O t ×tanh(C t );

[0066] Where: W O represents the weight matrix; b O represents the corresponding bias; O t Represents the output of the output gate, and finally obtains the hidden state h at the current moment t .

[0067] Furthermore, in step S3, the model is trained using the constructed sample library to optimize the model weights (so that it has a good noise reduction effect).

[0068] Furthermore, after step S3, the following steps are further included:

[0069] Step S4: Use the trained model to perform noise reduction processing on the original underwater signal actually collected in real time, so as to output a denoised signal.

[0070] It is worth mentioning that the technical features such as underwater sensors involved in the patent application of this invention should be regarded as prior art. The specific structure, working principle and possible control method and spatial layout method of these technical features can be selected according to the conventional methods in the field, and should not be regarded as the inventive point of this patent. This patent will not be further elaborated.

[0071] For those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An adaptive denoising method for underwater sensor signals based on attention mechanism and CNN-LSTM, characterized in that: The following steps are involved: Step S1: obtaining the original underwater signal collected by the underwater sensor, and preprocessing the original underwater signal, performing preliminary filtering on the signal through a bandpass filter, removing excessively high or low frequency components, and obtaining a relatively stable signal, thereby constructing a sample library containing pure signals and noise signals; Step S2: Use a convolutional neural network to extract features from the preprocessed signal. Through multi-layer convolution operations, the time and frequency features in the signal are extracted and a corresponding feature map is generated. The feature map reflects the local details and global information in the signal, including the different characteristics of the signal and noise. Step S3: A long short-term memory network based on the attention mechanism is used to process time series information. On the basis of feature extraction, the attention mechanism is introduced to enhance the ability to focus on key features. By calculating the weights of each position in the feature map, the attention allocation of the model is adaptively adjusted, so that the model pays more attention to the key features in the signal and ignores the features corresponding to irrelevant or noise signals.

2. According to claim 1, the method for adaptive denoising of underwater sensor signals based on attention mechanism and CNN-LSTM is characterized in that: In step S3, the long short-term memory network based on the attention mechanism is specifically implemented as follows: Step S3.1: Input the extracted feature map into the self-attention module to calculate the correlation between each position and generate a weight matrix; Step S3.2: weight allocation is performed, and according to the weight matrix, features with high correlation are enhanced, and features with low correlation or noise signals are suppressed, thereby optimizing feature representation; The weight matrix calculation formula is: Where Q represents the query matrix; K represents the key matrix; d k represents the dimension of the key vector; The dot product of K is scaled by the dimension of the key, and then the weight matrix W is calculated through the softmax function to ensure that the sum of all weights is 1; The weights assigned by the weight matrix W are used to weight the features and sum them, thereby enhancing the features with high correlation and suppressing the features with low correlation or noise signals. The calculation formula is: Output = W·V; Among them, W is the weight matrix, V is represented as the value matrix, and the final output features are generated by multiplying with the weight matrix W.

3. According to claim 2, the method for adaptive denoising of underwater sensor signals based on attention mechanism and CNN-LSTM is characterized in that: In step S3, the timing dependency of the captured signal is performed, wherein: The first step is to use the forget gate to determine the information to be discarded in the current state. The forget gate reads the hidden state h at the previous moment. t-1 and x t , after a sigmoid layer output f t For values ​​between 0 and 1, the calculation formula of the forget gate is: f t =σ(W f [h t-1 x t ]+b f ); Among them, W f and b f is a trainable parameter; [h t-1 x t ] means concatenating the hidden state at the current moment and the input at the current moment; σ( ) means the sigmoid function; In the second step, the new information added to the state of the memory cell is determined by the input gate, i t Represents the output of the input gate, which is a value between 0 and 1, indicating the amount of information to be retained. Its calculation formula is: i t =σ(W i [h t-1 x t ]+b i ); Among them, W i , W C represents the weight matrix; b i , b C represents the corresponding bias; tanh( ) is the hyperbolic tangent function; C t Represents the new candidate unit state, each of its elements is between -1 and 1; In the third step, the output gate weights the information stored in the memory cell with the input of the current time step to generate the output. t The calculation method is similar to the input gate. First, a sigmoid function is used to determine the degree of opening of the gate, and then a tanh function is used to generate the output vector. The calculation formula is: The t =σ(W O [h t-1 x t ]+b O ); h t =O t ×tanh(C t ); Where: W O represents the weight matrix; b O represents the corresponding bias; O t Represents the output of the output gate, and finally obtains the hidden state h at the current moment t .

4. The method for adaptive denoising of underwater sensor signals based on attention mechanism and CNN-LSTM according to claim 3 is characterized in that: In step S3, the model is trained using the constructed sample library to optimize the model weights.

5. The method for adaptive denoising of underwater sensor signals based on attention mechanism and CNN-LSTM according to claim 4, characterized in that: After step S3, the following steps are also included: Step S4: Use the trained model to perform noise reduction processing on the original underwater signal actually collected in real time, so as to output a denoised signal.

Images