Underwater acoustic signal enhancement method based on dual-path recurrent neural network structure
By employing an underwater acoustic signal enhancement method based on a dual-path recurrent neural network structure, three-dimensional tensor features are obtained and local and global features are identified, solving the problem of signal detail loss in existing technologies and achieving effective enhancement of underwater acoustic signals.
Patent Information
- Application Number
- CN202410894776.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-06-27
- Filing Date
- 2024-07-04
- Publication Date
- 2024-11-22
AI Technical Summary
Existing underwater acoustic signal denoising methods rely on convolutional neural networks, which leads to the loss of signal detail and fine-grained feature information, resulting in poor denoising performance.
A dual-path recurrent neural network structure is adopted. By acquiring the three-dimensional tensor features of the target underwater acoustic signal, the inter-group and intra-group recurrent neural networks are used to identify local and global signal features, and noise reduction is performed. Finally, the signal is restored to improve the noise reduction effect.
It effectively extracts the local and global features of underwater acoustic signals, improves the noise reduction effect, and enhances the underwater acoustic signals.
Smart Images

Figure CN119001692A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of underwater acoustic signal processing, and in particular to an underwater acoustic signal enhancement method based on a dual-path recurrent neural network structure. Background Art
[0002] Underwater acoustic research is an important foundation for passive sonar systems used by underwater vehicles to detect, track, locate and identify targets in aquatic environments. The mixing of multiple target signals, complex spectral components and intermittent noise signals are the reasons for the low signal-to-noise ratio of sonar received signals. Therefore, it is necessary to perform noise reduction on underwater acoustic signals.
[0003] Existing methods for denoising underwater acoustic signals rely on convolutional neural networks. Convolutional neural networks must first use the short-time Fourier transform method to convert the input signal into a two-dimensional spectrum. This conversion method loses more detail information and fine-grained feature information, resulting in poor denoising effects on underwater acoustic signals. Summary of the invention
[0004] The present invention provides a method for noise reduction of underwater acoustic signals, which is used to solve the defect of poor noise reduction effect of underwater acoustic signals in the prior art. By obtaining the three-dimensional tensor features corresponding to the target underwater acoustic signal, the three-dimensional tensor features are input into a trained dual-path recurrent neural network to obtain enhanced signal features, and the enhanced signal features are restored to obtain the enhanced underwater acoustic signal. The method can extract the local characteristics and global characteristics of the target underwater acoustic signal for noise reduction, improve the noise reduction effect, and realize the enhancement of the underwater acoustic signal.
[0005] The present invention provides an underwater acoustic signal enhancement method based on a dual-path recurrent neural network structure, comprising the following steps: Acquiring a three-dimensional tensor feature corresponding to a target hydroacoustic signal, wherein the three-dimensional tensor feature is determined according to a plurality of feature blocks corresponding to the target hydroacoustic signal; Inputting the three-dimensional tensor feature into a trained dual-path recurrent neural network to obtain an enhanced signal feature, wherein the trained dual-path recurrent neural network is used to identify corresponding local signal features and global signal features according to the three-dimensional tensor feature, and performing noise reduction according to the local signal features and the global signal features to obtain the enhanced signal feature; The enhanced signal characteristics are restored to obtain an enhanced underwater acoustic signal.
[0006] According to a method for enhancing underwater acoustic signals based on a dual-path recurrent neural network structure provided by the present invention, a three-dimensional tensor feature corresponding to a target underwater acoustic signal is obtained, including: The original underwater acoustic signal containing noise is subjected to feature extraction through a one-dimensional convolutional neural network to obtain the extracted features; Performing block processing on the extracted features to obtain the multiple feature blocks, wherein the multiple feature blocks have the same size; The multiple feature blocks are connected in series to obtain a three-dimensional tensor feature corresponding to the target underwater acoustic signal.
[0007] According to a method for underwater acoustic signal enhancement based on a dual-path recurrent neural network structure provided by the present invention, the trained dual-path recurrent neural network includes an inter-group recurrent neural network and an intra-group recurrent neural network, and the three-dimensional tensor feature is input into the trained dual-path recurrent neural network to obtain enhanced signal features, including: The inter-group recurrent neural network determines the local signal feature by iteratively processing feature information between the multiple feature blocks; The intra-group recurrent neural network determines the global signal feature by iteratively processing feature information in each feature block of the multiple feature blocks.
[0008] According to a method for enhancing underwater acoustic signals based on a dual-path recurrent neural network structure provided by the present invention, the inter-group recurrent neural network includes a hidden layer, a fully connected layer and a normalization layer, and the inter-group recurrent neural network determines the local signal features by iteratively processing the feature information between the multiple feature blocks, including: Using the first two-dimensional tensor features in the three-dimensional tensor features as first input data of the inter-group recurrent neural network, processing the first input data through the mapping function of the hidden layer, and outputting a processed feature vector; Performing dimension conversion on the processed feature vector by using the weight matrix and the offset vector of the fully connected layer to generate a converted feature vector; The converted feature vector is normalized by the normalization layer to obtain a normalized feature vector, which is the local signal feature.
[0009] According to a method for underwater acoustic signal enhancement based on a dual-path recurrent neural network structure provided by the present invention, the intra-group recurrent neural network determines the global signal feature by iteratively processing feature information in each feature block in the multiple feature blocks, including: Concatenating the normalized feature vector and the last-dimensional tensor feature in the three-dimensional tensor feature to obtain second input data; The second input data is input into the intra-group recurrent neural network for processing to obtain the enhanced signal features.
[0010] According to a method for enhancing underwater acoustic signals based on a dual-path recurrent neural network structure provided by the present invention, the signal characteristics of the enhanced signal are restored to obtain an enhanced underwater acoustic signal, including: The enhanced signal features are subjected to overlapping addition transformation to obtain the enhanced underwater acoustic signal.
[0011] The present invention also provides an underwater acoustic signal enhancement device based on a dual-path recurrent neural network structure, comprising the following modules: A feature acquisition module, used to acquire a three-dimensional tensor feature corresponding to a target hydroacoustic signal, wherein the three-dimensional tensor feature is determined according to a plurality of feature blocks corresponding to the target hydroacoustic signal; A feature denoising module, used for inputting the three-dimensional tensor feature into a trained dual-path recurrent neural network to obtain enhanced signal features, wherein the trained dual-path recurrent neural network is used for identifying corresponding local signal features and global signal features according to the three-dimensional tensor feature, and performing denoising according to the local signal features and the global signal features to obtain the enhanced signal features; The signal restoration module is used to restore the enhanced signal characteristics to obtain an enhanced underwater acoustic signal.
[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, an underwater acoustic signal enhancement method based on a dual-path recurrent neural network structure as described above is implemented.
[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for enhancing underwater acoustic signals based on a dual-path recurrent neural network structure as described above is implemented.
[0014] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for enhancing underwater acoustic signals based on a dual-path recurrent neural network structure.
[0015] The method for enhancing underwater acoustic signals based on a dual-path recurrent neural network structure provided by the present invention performs noise reduction processing by extracting local characteristics and global characteristics of target underwater acoustic signals, thereby improving the noise reduction effect and achieving enhancement of underwater acoustic signals. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0017] Figure 1It is a flow chart of the underwater acoustic signal enhancement method based on the dual-path recurrent neural network structure provided by the present invention.
[0018] Figure 2 It is a flowchart of the method for obtaining three-dimensional tensor features provided by the present invention.
[0019] Figure 3 It is a flowchart of the method for obtaining enhanced signal characteristics provided by the present invention.
[0020] Figure 4 It is a schematic diagram of the working process of the inter-group recurrent neural network provided by the present invention.
[0021] Figure 5 It is a flow chart of the processing method of the dual-path recurrent neural network provided by the present invention.
[0022] Figure 6 It is a schematic diagram of a scenario of an overlapping adding operation provided by the present invention.
[0023] Figure 7 Schematic diagram of the overall process of the underwater acoustic signal enhancement method based on the dual-path recurrent neural network structure provided by the embodiment of the present invention
[0024] Figure 8 It is a schematic diagram of the waveform diagram and the spectrum diagram before and after denoising provided by the present invention.
[0025] Fig. 9 A schematic diagram of the structure of an underwater acoustic signal enhancement device based on a dual-path recurrent neural network structure provided in an embodiment of the present invention.
[0026] Fig.10 It is a schematic diagram of the physical structure of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0027] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0028] Underwater acoustic research is an important foundation for passive sonar systems used by underwater vehicles to detect, track, locate and identify targets in aquatic environments. However, due to the diversity of natural environments, the volatility of sound sources and high noise intensity, modern sonar systems are full of challenges for feature extraction of non-underwater acoustic signals. The main causes of these problems are the mixing of multi-target signals, complex spectral components and intermittent noise signals, which reduce the signal-to-noise ratio of sonar received signals. Therefore, underwater acoustic signals must be denoised. At this stage, traditional denoising methods usually use transform analysis and time-frequency conversion methods to achieve signal enhancement. For example, amplitude-aware interchange entropy, wavelet and block threshold, multi-directional filter, minimum variance distortion less response, mutation method, wavelet transform, empirical mode decomposition, linear spectrum, singular value decomposition. However, the characteristics of background noise change with the changes in the ocean landscape, which may cause obvious dynamic changes at different moments. Underwater acoustic signals are also susceptible to interference from machines and the environment, which can weaken and distort their propagation, resulting in a relatively poor signal-to-noise ratio of underwater acoustic signals when they reach the detection system. The introduction of these problems has brought countless difficulties to traditional algorithms. Compared with the old methods, models based on deep learning have shown considerable potential in applications in various disciplines. Deep learning can learn the relationship between the previous and next time series through neural networks. In addition, the use of filters enables the network to automatically collect the key features of the signal, and the sequences are automatically encoded and compiled without manual extraction. Most importantly, the denoising effect of the model can be enhanced by modifying the network structure and parameter values. For example, existing methods at this stage include denoising representation recognition models, which convert spectrum into correlation coefficients to generate data for parallel training of convolutional denoising autoencoder models. In addition, a stacked convolutional sparse denoising autoencoder model can also be applied to complete the blind denoising task of underwater heterogeneous information data.
[0029] By comparing different denoising methods, it can be found that these methods mainly rely on convolutional neural networks to achieve the denoising and enhancement tasks of underwater acoustic signals. Among them, the one-dimensional convolutional network layer based on deep learning technology is the main technical means for underwater acoustic signal denoising. It uses filters to extract local time-frequency information from different types of underwater acoustic signals by constructing a deep network structure. Although, this has been proven to be very effective in many methods and can learn low-order information of different types of underwater acoustic signals. However, the convolutional neural network must first use the short-time Fourier transform method to convert the input signal into a two-dimensional spectrum. Compared with the traditional method of extracting features from time signals, this conversion method loses more detail information and fine-grained feature information. In addition, establishing a recurrent neural network to learn noise suppression methods and perform high-to-low dimensional conversion on different types of underwater acoustic signals is another core method. It focuses on extracting the change information of the signal in the time dimension to ensure that the network can learn the feature values under different conditions. These parameters must be learned repeatedly during the network training process. However, since these denoising techniques often require input sequences with many time steps, it is a challenge to model long time series.
[0030] In view of this, the present invention provides a method for enhancing underwater acoustic signals based on a dual-path recurrent neural network structure, comprising obtaining a three-dimensional tensor feature corresponding to a target underwater acoustic signal, inputting the three-dimensional tensor feature into a trained dual-path recurrent neural network to obtain an enhanced signal feature, and performing signal restoration on the enhanced signal feature to obtain an enhanced underwater acoustic signal. The method can extract the local characteristics and global characteristics of the target underwater acoustic signal for noise reduction, improve the noise reduction effect, and achieve the enhancement of the underwater acoustic signal.
[0031] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.
[0032] Figure 1 The figure is a flow chart of the method for enhancing underwater acoustic signals based on a dual-path recurrent neural network structure provided by the present invention. The method for enhancing underwater acoustic signals based on a dual-path recurrent neural network structure can be applied to electronic devices, which can be various types of devices with information processing capabilities during implementation. For example, the electronic device can include a personal computer, a laptop computer, a PDA or a server, etc.; the electronic device can also be a mobile terminal, for example, the mobile terminal can include a mobile phone, a car computer, a tablet computer or a projector, etc. Figure 1 As shown, the method may include the following steps 101 to 103: Step 101: Acquire a three-dimensional tensor feature corresponding to a target hydroacoustic signal, wherein the three-dimensional tensor feature is determined based on a plurality of feature blocks corresponding to the target hydroacoustic signal.
[0033] It should be noted that there are many ways to obtain the three-dimensional tensor features corresponding to the target hydroacoustic signal, such as feature conversion through a model or through a calculation method. The present invention is determined based on multiple feature blocks corresponding to the target hydroacoustic signal.
[0034] Among them, the multiple feature blocks corresponding to the target hydroacoustic signal can be obtained by dividing the features corresponding to the target hydroacoustic signal into multiple blocks. There are many ways to determine the three-dimensional tensor features based on the multiple feature blocks corresponding to the target hydroacoustic signal. For example, multiple feature blocks can be combined or spliced to obtain three-dimensional tensor features, etc. The present invention does not limit the method for determining the three-dimensional tensor features based on the multiple feature blocks corresponding to the target hydroacoustic signal.
[0035] Step 102: Input the three-dimensional tensor features into a trained dual-path recurrent neural network to obtain enhanced signal features. The trained dual-path recurrent neural network is used to identify corresponding local signal features and global signal features based on the three-dimensional tensor features, and perform noise reduction based on the local signal features and the global signal features to obtain the enhanced signal features.
[0036] It should be noted that the trained dual-path recurrent neural network is used to identify the corresponding local signal features and global signal features according to the three-dimensional tensor features, and to perform noise reduction according to the local signal features and the global signal features to obtain the enhanced signal features. The network structure of the trained dual-path recurrent neural network can be set according to demand, and the present invention does not limit the network structure of the trained dual-path recurrent neural network.
[0037] Step 103: performing signal restoration on the enhanced signal features to obtain an enhanced underwater acoustic signal.
[0038] It should be noted that the method for restoring the enhanced signal features can be through model processing or function processing, and signal restoration can remove noise and enhance the underwater acoustic signal. The present invention does not limit the method of restoring the enhanced signal features to obtain the enhanced underwater acoustic signal.
[0039] It can be understood that the underwater acoustic signal enhancement method based on the dual-path recurrent neural network structure provided by the present invention introduces global processing and local processing methods from the perspective of time to space transformation, and enhances the signal characteristics by acquiring the continuous characteristics and short-term change characteristics of the signal-noise, thereby improving the noise reduction effect and realizing the enhancement of the underwater acoustic signal.
[0040] In some embodiments, the three-dimensional tensor features corresponding to the target hydroacoustic signal can be obtained by processing the features extracted by the one-dimensional convolutional neural network.
[0041] Figure 2 FIG. 1 is a flow chart of the method for obtaining three-dimensional tensor features provided by the present invention. Figure 3 As shown, the acquisition of the three-dimensional tensor feature corresponding to the target hydroacoustic signal may include: Step 201: extract features from the original underwater acoustic signal containing noise through a one-dimensional convolutional neural network to obtain extracted features.
[0042] It should be noted that the one-dimensional convolutional neural network can extract local time-frequency information from different types of underwater acoustic signals. The one-dimensional convolutional neural network may include multiple one-dimensional convolutional layers, which can extract features of different scales from the underwater acoustic signal input sequence.
[0043] For example, for a noisy underwater acoustic signal , where N is the dimension after feature extraction through a one-dimensional convolutional neural network, and L is the time step.
[0044] Step 202: performing block processing on the extracted features to obtain the plurality of feature blocks, wherein the plurality of feature blocks have the same size.
[0045] It should be noted that there are many ways to perform block processing on the extracted features, which can be set according to needs.
[0046] For example, the block processing can convert the noisy underwater acoustic signal into The input sequence is divided into small blocks of length K and moving step length P. The beginning and end of the input sequence are filled with zeros. This ensures that each element in the input sequence appears K / P times in the block. In this way, a hydroacoustic signal sequence is divided into S blocks of the same size. . Among them, K>P can be set. In this case, each new feature block will partially overlap with the previous feature block, and the length of the overlap is K - P.
[0047] Step 203: Connect the multiple feature blocks in series to obtain a three-dimensional tensor feature corresponding to the target underwater acoustic signal.
[0048] It should be noted that a three-dimensional tensor feature can be obtained by connecting multiple feature blocks in series. Exemplarily, the three-dimensional tensor feature can be a three-dimensional tensor including time, frequency and feature dimensions.
[0049] For example, when a segment of an underwater acoustic signal sequence is divided into S feature blocks of the same size Afterwards, all feature blocks can be constructed into a three-dimensional tensor .
[0050] It can be understood that the above method can be used to conveniently and quickly obtain the three-dimensional tensor features corresponding to the target underwater acoustic signal and improve the noise reduction efficiency.
[0051] In some embodiments, the trained dual-path recurrent neural network may include an inter-group recurrent neural network and an intra-group recurrent neural network, and local signal features and global signal features are obtained through the two networks.
[0052] Figure 3 is a flow chart of the method for obtaining enhanced signal features provided by the present invention. The trained dual-path recurrent neural network includes an inter-group recurrent neural network and an intra-group recurrent neural network, such as Figure 3 As shown, the inputting the three-dimensional tensor feature into the trained dual-path recurrent neural network to obtain the enhanced signal feature may include: Step 301: The inter-group recurrent neural network determines the local signal feature by iteratively processing the feature information between the multiple feature blocks.
[0053] It should be noted that the inter-group recurrent neural network can be used to iteratively propagate information between feature blocks to identify changes in local signal parts affected by noise and obtain local signal features.
[0054] Step 302: The intra-group recurrent neural network determines the global signal feature by iteratively processing feature information in each feature block of the multiple feature blocks.
[0055] It should be noted that the intra-group recurrent neural network can be used to iteratively propagate information within each feature block to detect global changes in different signal types affected by the same noise segment.
[0056] It can be understood that the underwater acoustic signal enhancement method based on the dual-path recurrent neural network structure provided by the present invention combines the forward transmission method of the recurrent network with the time series processing method, introduces global processing and local processing methods from the perspective of time to space transformation, and obtains the persistence characteristics and short-term change characteristics of the signal-noise.
[0057] In some embodiments, the inter-group recurrent neural network can be processed through a hidden layer, a fully connected layer, and a normalization layer to obtain local signal features.
[0058] Figure 4 : is a schematic diagram of the working process of the inter-group recurrent neural network provided by the present invention, wherein the inter-group recurrent neural network includes a hidden layer, a fully connected layer and a normalized layer, such as Figure 4 As shown, the inter-group recurrent neural network determines the local signal feature by iteratively processing the feature information between the multiple feature blocks, which may include: Step 401: Use the first two-dimensional tensor features in the three-dimensional tensor features as the first input data of the inter-group recurrent neural network, process the first input data through the mapping function of the hidden layer, and output the processed feature vector.
[0059] It should be noted that the inter-group recurrent neural network can be a bidirectional recurrent neural network, and you can choose The N×K dimensions in are used as the input of the inter-group recurrent neural network, and the formula is: .in, is the output of the inter-group recurrent neural network, is the mapping function of the hidden layer, is the segment extracted from the i-th feature block.
[0060] In some embodiments, before processing the first input data by the mapping function of the hidden layer and outputting the processed feature vector, the method further comprises: After being input into the dual-path recurrent neural network, the inter-group recurrent neural network in the dual-path neural network structure converts the input three-dimensional tensor into a tensor with the same shape.
[0061] Step 402: Perform dimension conversion on the processed feature vector using the weight matrix and offset vector of the fully connected layer to generate a converted feature vector.
[0062] It should be noted that after obtaining the output of the inter-group recurrent neural network, a fully connected neural network can be used to convert The characteristic dimensions are: .in, is the output of the fully connected layer, is the weight of the fully connected layer, is the offset vector.
[0063] Step 403: normalizing the converted feature vector through the normalization layer to obtain a normalized feature vector, where the normalized feature vector is the local signal feature.
[0064] It should be noted that after obtaining the converted feature dimension, Input to the normalization layer, the formula is: .
[0065] in, , are the mean and variance values of the tensor, calculated as follows: ; .
[0066] in, and is a scaling factor that guarantees the model capacity and ensures that the output can be restored to the original input distribution.
[0067] It can be understood that by processing the first two-dimensional features of the three-dimensional tensor features through the bidirectional recurrent neural network as an inter-group recurrent neural network, the difference characteristics of the signal-noise in the high-dimensional space are extracted to extract the target mask and the noise mask, thereby improving the noise reduction effect and achieving the enhancement of the underwater acoustic signal.
[0068] In some embodiments, the intra-group recurrent neural network may act on the last dimension of the input three-dimensional tensor feature to extract global features.
[0069] In an embodiment of the present invention, the intra-group recurrent neural network determines the global signal feature by iteratively processing the feature information in each feature block in the multiple feature blocks, which may include: concatenating the normalized feature vector and the last-dimensional tensor feature in the three-dimensional tensor feature to obtain second input data; and inputting the second input data into the intra-group recurrent neural network for processing to obtain the enhanced signal feature.
[0070] It should be noted that the intra-group recurrent neural network can be a unidirectional recurrent neural network. And the last dimension tensor feature in the three-dimensional tensor feature Connect and use it as the input of the intra-group recurrent neural network. The formula is: + .
[0071] The intra-group recurrent neural network acts on the last dimension of the input, and the subsequent processing is the same as the fully connected layer and the normalized layer in the inter-group recurrent neural network. As shown below: . is the mapping function of the hidden layer, is the segment extracted from the i-th dimension, is the output of the intra-group recurrent neural network.
[0072] It can be understood that the dual-path structure recurrent neural network can retain the contextual information and global perception information in the feature sequence to the greatest extent, can extract the differences between the target signal and the ocean environment noise in the high-dimensional space, and improve the noise reduction ability of the network model when dealing with coupled signals.
[0073] Figure 5 is a flow chart of the processing method of the dual-path recurrent neural network provided by the present invention. Figure 5 As shown, the inter-group recurrent neural network is a bidirectional recurrent neural network. The N×K dimensions in the group serve as the input of the inter-group recurrent neural network: .in, is the output of the inter-group recurrent neural network, is the mapping function of the hidden layer, is the segment extracted from the i-th block. Then, a fully connected neural network is used to transform The characteristic dimensions are: .in, is the output of the fully connected layer, is the weight of the fully connected layer, is the offset vector. Then, Input to the normalization layer: .
[0074] in, , are the mean and variance values of the tensor, calculated as follows: ; ; and is a scaling factor to ensure the model capacity and to ensure that the output can be restored to the original input distribution. and Connect and use as input to the recurrent neural network within the group: + .
[0075] The intra-group recurrent neural network acts on the last dimension of the input, and the subsequent processing is the same as the fully connected layer and the normalization layer with the inter-group recurrent neural network. As shown below: .
[0076] In some embodiments, the signal restoration of the enhanced signal features may be performed using an overlap-addition operation.
[0077] In an embodiment of the present invention, the signal restoration of the enhanced signal features to obtain the enhanced underwater acoustic signal may include: performing overlapping and adding transformation on the enhanced signal features to obtain the enhanced underwater acoustic signal.
[0078] Figure 6 Schematic diagram of the overlapping addition operation provided by the present invention. Figure 6 As shown, the definition is the output of the dual-path recurrent neural network. The process of overlap-add operation is as follows Figure 6 For each subsequence, the overlap calculation formula is used to obtain the denoised signal. As shown below: ,in, It means adding the denoised results of each subsequence. is the number of slices, is the signal length.
[0079] The following describes an exemplary application of an embodiment of the present invention in a practical application scenario.
[0080] Figure 7 The overall flow chart of the underwater acoustic signal enhancement method based on the dual-path recurrent neural network structure provided by the embodiment of the present invention is as follows. The present invention uses a dual-path recurrent neural network to achieve underwater acoustic signal denoising, which is a method for modeling sound sequences. Figure 7 As shown, the method includes: using a one-dimensional convolutional neural network to extract features of the input sequence and divide it into smaller blocks. All the blocks are concatenated into a three-dimensional tensor and input into a dual-path recurrent neural network, which is a bidirectional neural network structure embedded in a recursive network. It iteratively applies operations within and between small signal blocks to obtain local and global features between target signals. The inter-group recurrent neural network is designed to identify changes in local signal parts affected by noise. In contrast, the intra-group recurrent neural network is designed to detect global changes in different signal types affected by the same segment of noise. After undergoing the overlap-add transformation, the target signal is restored to its original mode.
[0081] Among them, in the stacked DPRNN, the linear input size received by all blocks within each block or between blocks is the same.
[0082] In order to test the effect of dual-path recurrent neural network denoising, the present invention uses the underwater ship sound recording database ShipsEar for testing. The recordings of the ShipsEar dataset are made by hydrophones placed on the dock to record the ship speed and cavitation noise associated with docking or unhooking operations. These are real ship sounds recorded in a natural environment. Therefore, background noise from humans and nature, as well as vocalizations of marine mammals, are present. The recordings of 90 .wav files constitute the dataset, which is divided into five main categories. Each main category has one or more subcategories (for example, category A consists of dredgers, fish boats, trawlers, mussel boats, and tugboats), and the length of each audio clip is 15 seconds to 10 minutes. In order to better verify the denoising performance of the model. All signals are segmented at a fixed time of 5 seconds, resulting in 1956 labeled sound samples. Samples without noise categories are randomly extracted. The noise category is fused with these extracted samples to make the signal-to-noise ratio of the sample 0dB, and is divided into training, validation, and test sets at a ratio of 8:1:1. As shown in Table 1, the signal-to-noise ratio improvement of different methods is shown.
[0083] Table 1 SNR improvement of different methods
[0084] Furthermore, the memory space occupied during the training process is compared, as shown in Table 2, which shows the memory space occupied by different methods. The results show that less space can be occupied under the condition of unstable hardware resources.
[0085] Table 2 Memory space of different methods
[0086] Figure 8 Schematic diagram of the waveform diagram and spectrum diagram before and after denoising provided by the present invention. Figure 8 As shown, after comparing the waveform diagram and spectrum diagram before and after denoising, the results show that when the underwater acoustic signal enhancement method based on the dual-path recurrent neural network structure is used for denoising, the time-frequency characteristics of different underwater acoustic target signals can be highlighted.
[0087] The underwater acoustic signal enhancement method based on the dual-path recurrent neural network structure provided by the present invention proposes for the first time a method combining the dual-path structure and the recurrent neural network to study the separation operation of the target signal and the ocean noise. The dual-path recurrent neural network DPRNN uses the recurrent network, the bidirectional recurrent neural network and the dual-path structure as the base model, combines the forward transmission method of the recurrent network with the time series processing method, introduces the global processing and local processing methods from the perspective of time to space transformation, focuses on exploring the persistence characteristics and short-term change information of the signal-to-noise, extracts the difference characteristics and similarities and differences of the signal-to-noise in the high-dimensional space, and extracts the target mask and the noise mask. The fitting performance of the denoising model in multiple feature transformations and separation denoising processes is improved. Among them, one path selects the bidirectional recurrent neural network (Bi-RNN) to focus on learning the local persistence features from the first frame to the last frame and from the last frame to the first frame in the underwater acoustic signal, while the other path selects the unidirectional recurrent neural network (RNN) to focus on modeling the global features from the first frame to the last frame in the underwater acoustic signal. In the process of extracting local and global features of underwater acoustic signals, the recurrent neural network in the dual-path structure can retain the contextual information and global perception information in the feature sequence to the greatest extent, extract the differences between target signals and ocean environmental noise in high-dimensional space, and enhance the noise reduction capability of the network model when dealing with coupled signals.
[0088] Based on the foregoing embodiments, an embodiment of the present invention provides an underwater acoustic signal enhancement device based on a dual-path recurrent neural network structure. The modules included in the device and the units included in each module can be implemented by a processor; of course, they can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.
[0089] The underwater acoustic signal enhancement device based on the dual-path recurrent neural network structure provided by the present invention is described below. The underwater acoustic signal enhancement device based on the dual-path recurrent neural network structure described below and the underwater acoustic signal enhancement method based on the dual-path recurrent neural network structure described above can be referenced to each other.
[0090] Fig. 9 A schematic diagram of the structure of an underwater acoustic signal enhancement device based on a dual-path recurrent neural network structure provided by an embodiment of the present invention. Fig. 9 As shown, the device 500 includes a feature acquisition module 501, a feature noise reduction module 502 and a signal restoration module 503, wherein: A feature acquisition module 501 is used to acquire a three-dimensional tensor feature corresponding to a target hydroacoustic signal, wherein the three-dimensional tensor feature is determined according to a plurality of feature blocks corresponding to the target hydroacoustic signal; A feature denoising module 502 is used to input the three-dimensional tensor feature into a trained dual-path recurrent neural network to obtain enhanced signal features, wherein the trained dual-path recurrent neural network is used to identify corresponding local signal features and global signal features according to the three-dimensional tensor feature, and to perform denoising according to the local signal features and the global signal features to obtain the enhanced signal features; The signal restoration module 503 is used to restore the enhanced signal characteristics to obtain an enhanced underwater acoustic signal.
[0091] In some embodiments, the feature acquisition module 501 includes an extraction unit, a block division unit and a series connection unit, wherein:
[0092] The extraction unit is used to extract features from the original underwater acoustic signal containing noise through a one-dimensional convolutional neural network to obtain extracted features; The block division unit is used to perform block processing on the extracted features to obtain the multiple feature blocks, and the multiple feature blocks have the same size; The series connection unit is used to connect the multiple feature blocks in series to obtain the three-dimensional tensor feature corresponding to the target underwater acoustic signal.
[0093] In some embodiments, the trained dual-path recurrent neural network includes an inter-group recurrent neural network and an intra-group recurrent neural network, and the feature denoising module 502 includes an inter-group processing unit and an intra-group processing unit, wherein:
[0094] The inter-group processing unit is used for the inter-group recurrent neural network to iteratively process feature information between the multiple feature blocks to determine the local signal feature; The intra-group processing unit is used for the intra-group recurrent neural network to determine the global signal feature by iteratively processing feature information in each feature block of the multiple feature blocks.
[0095] In some embodiments, the inter-group recurrent neural network includes a hidden layer, a fully connected layer and a normalization layer, and the inter-group processing unit is specifically used to: use the first two-dimensional tensor features in the three-dimensional tensor features as the first input data of the inter-group recurrent neural network, process the first input data through the mapping function of the hidden layer, and output the processed feature vector; perform dimension conversion on the processed feature vector through the weight matrix and offset vector of the fully connected layer to generate a converted feature vector; normalize the converted feature vector through the normalization layer to obtain a normalized feature vector, and the normalized feature vector is the local signal feature.
[0096] In some embodiments, the intra-group processing unit is specifically used to: concatenate the normalized feature vector and the last-dimensional tensor feature in the three-dimensional tensor feature to obtain second input data; input the second input data into the intra-group recurrent neural network for processing to obtain the enhanced signal feature.
[0097] In some embodiments, the signal restoration module 503 is specifically used to: perform overlapping and adding transformation on the enhanced signal features to obtain the enhanced underwater acoustic signal.
[0098] In the embodiment of the present invention, the local characteristics and global features of the target underwater acoustic signal can be extracted to perform noise reduction, thereby improving the noise reduction effect and achieving enhancement of the underwater acoustic signal.
[0099] Fig.10 is a schematic diagram of the physical structure of the electronic device provided by the present invention, such as Fig.10As shown, the electronic device 600 may include: a processor 601, a communication interface 602, a memory 603 and a communication bus 604, wherein the processor 601, the communication interface 602 and the memory 603 communicate with each other through the communication bus 604. The processor 601 may call the logic instructions in the memory 603 to execute the underwater acoustic signal enhancement method based on the dual-path recurrent neural network structure, the method comprising: obtaining a three-dimensional tensor feature corresponding to the target underwater acoustic signal, the three-dimensional tensor feature is determined according to a plurality of feature blocks corresponding to the target underwater acoustic signal; inputting the three-dimensional tensor feature into a trained dual-path recurrent neural network to obtain an enhanced signal feature, the trained dual-path recurrent neural network is used to identify the corresponding local signal feature and global signal feature according to the three-dimensional tensor feature, and perform noise reduction according to the local signal feature and the global signal feature to obtain the enhanced signal feature; and performing signal restoration on the enhanced signal feature to obtain an enhanced underwater acoustic signal.
[0100] In addition, the logic instructions in the above-mentioned memory 603 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.
[0101] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the underwater acoustic signal enhancement method based on a dual-path recurrent neural network structure provided by the above-mentioned methods, and the method includes: obtaining a three-dimensional tensor feature corresponding to the target underwater acoustic signal, and the three-dimensional tensor feature is determined according to a plurality of feature blocks corresponding to the target underwater acoustic signal; inputting the three-dimensional tensor feature into a trained dual-path recurrent neural network to obtain an enhanced signal feature, and the trained dual-path recurrent neural network is used to identify the corresponding local signal feature and global signal feature according to the three-dimensional tensor feature, and perform noise reduction according to the local signal feature and the global signal feature to obtain the enhanced signal feature; and perform signal restoration on the enhanced signal feature to obtain an enhanced underwater acoustic signal.
[0102] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.
[0103] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the underwater acoustic signal enhancement method based on a dual-path recurrent neural network structure provided by the above-mentioned methods, the method comprising: obtaining a three-dimensional tensor feature corresponding to the target underwater acoustic signal, wherein the three-dimensional tensor feature is determined based on a plurality of feature blocks corresponding to the target underwater acoustic signal; inputting the three-dimensional tensor feature into a trained dual-path recurrent neural network to obtain an enhanced signal feature, wherein the trained dual-path recurrent neural network is used to identify the corresponding local signal feature and global signal feature based on the three-dimensional tensor feature, and perform noise reduction based on the local signal feature and the global signal feature to obtain the enhanced signal feature; and performing signal restoration on the enhanced signal feature to obtain an enhanced underwater acoustic signal.
[0104] The above-mentioned computer-readable storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.
[0105] Computer-readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0106] The program code embodied on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the foregoing.
[0107] Computer program code for performing the operations of the present specification may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).
[0108] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0109] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for underwater acoustic signal enhancement based on a dual-path recurrent neural network structure. It is characterized in that include: Acquiring a three-dimensional tensor feature corresponding to a target hydroacoustic signal, wherein the three-dimensional tensor feature is determined according to a plurality of feature blocks corresponding to the target hydroacoustic signal; Inputting the three-dimensional tensor feature into a trained dual-path recurrent neural network to obtain an enhanced signal feature, wherein the trained dual-path recurrent neural network is used to identify corresponding local signal features and global signal features according to the three-dimensional tensor feature, and performing noise reduction according to the local signal features and the global signal features to obtain the enhanced signal feature; The enhanced signal characteristics are restored to obtain an enhanced underwater acoustic signal.
2. According to the method for enhancing underwater acoustic signals based on a dual-path recurrent neural network structure according to claim 1, It is characterized in that Obtain the three-dimensional tensor features corresponding to the target underwater acoustic signal, including: The original underwater acoustic signal containing noise is subjected to feature extraction through a one-dimensional convolutional neural network to obtain the extracted features; Performing block processing on the extracted features to obtain the multiple feature blocks, wherein the multiple feature blocks have the same size; The multiple feature blocks are connected in series to obtain a three-dimensional tensor feature corresponding to the target underwater acoustic signal.
3. The underwater acoustic signal enhancement method based on the dual-path recurrent neural network structure according to claim 1, It is characterized in that The trained dual-path recurrent neural network includes an inter-group recurrent neural network and an intra-group recurrent neural network, and the three-dimensional tensor feature is input into the trained dual-path recurrent neural network to obtain enhanced signal features, including: The inter-group recurrent neural network determines the local signal feature by iteratively processing feature information between the multiple feature blocks; The intra-group recurrent neural network determines the global signal feature by iteratively processing feature information in each feature block of the multiple feature blocks.
4. The underwater acoustic signal enhancement method based on the dual-path recurrent neural network structure according to claim 3, It is characterized in that The inter-group recurrent neural network includes a hidden layer, a fully connected layer and a normalization layer. The inter-group recurrent neural network determines the local signal feature by iteratively processing the feature information between the multiple feature blocks, including: Using the first two-dimensional tensor features in the three-dimensional tensor features as first input data of the inter-group recurrent neural network, processing the first input data through the mapping function of the hidden layer, and outputting a processed feature vector; Performing dimension conversion on the processed feature vector by using the weight matrix and the offset vector of the fully connected layer to generate a converted feature vector; The converted feature vector is normalized by the normalization layer to obtain a normalized feature vector, which is the local signal feature.
5. The underwater acoustic signal enhancement method based on the dual-path recurrent neural network structure according to claim 4, It is characterized in that The intra-group recurrent neural network determines the global signal feature by iteratively processing feature information in each feature block in the plurality of feature blocks, including: Concatenating the normalized feature vector and the last-dimensional tensor feature in the three-dimensional tensor feature to obtain second input data; The second input data is input into the intra-group recurrent neural network for processing to obtain the enhanced signal features.
6. The underwater acoustic signal enhancement method based on a dual-path recurrent neural network structure according to claim 1, It is characterized in that The step of performing signal restoration on the enhanced signal feature to obtain an enhanced underwater acoustic signal includes: The enhanced signal features are subjected to overlapping addition transformation to obtain the enhanced underwater acoustic signal.
7. A water acoustic signal enhancement device based on a dual-path recurrent neural network structure, It is characterized in that include: A feature acquisition module, used to acquire a three-dimensional tensor feature corresponding to a target hydroacoustic signal, wherein the three-dimensional tensor feature is determined according to a plurality of feature blocks corresponding to the target hydroacoustic signal; A feature denoising module, used for inputting the three-dimensional tensor feature into a trained dual-path recurrent neural network to obtain enhanced signal features, wherein the trained dual-path recurrent neural network is used for identifying corresponding local signal features and global signal features according to the three-dimensional tensor feature, and performing denoising according to the local signal features and the global signal features to obtain the enhanced signal features; The signal restoration module is used to restore the enhanced signal characteristics to obtain an enhanced underwater acoustic signal.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the method for enhancing underwater acoustic signals based on a dual-path recurrent neural network structure as described in any one of claims 1 to 6 is implemented.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the method for enhancing underwater acoustic signals based on a dual-path recurrent neural network structure as described in any one of claims 1 to 6 is implemented.
10. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the method for enhancing underwater acoustic signals based on a dual-path recurrent neural network structure as described in any one of claims 1 to 6 is implemented.
Citation Information
Cited By
Target radar signal enhancement method and device based on DPRNN
CN119959922A
A target radar signal enhancement method and device based on DPRNN
CN119959922B
