Conformer-based underwater glider acoustic data processing method and storage medium

Abstract

The application discloses a kind of underwater glider acoustic data processing method and storage medium based on Conformer, belong to underwater acoustic signal processing technical field.The method includes: using MCR-AAD model to the original acoustic data collected by glider Effective signal detection, output the probability of each frame as effective signal, and using double threshold post-processing method to determine effective signal and noise signal;The detected effective signal is input into the causal BGformer noise reduction model, generates and reverses noise mask by audio feature fusion, DCC encoder, Conformer decoder and label embedding module, to obtain the audio result after noise reduction.MCR-AAD model is composed of mel feature extractor, multi-scale depth separable convolution, dynamic position coding, Conformer encoder and multi-task classification module.The application constructs efficient, real-time underwater acoustic data processing scheme, effectively improves the application performance of glider in marine observation and target identification.

Description

Technical Field

[0001] This invention belongs to the field of underwater acoustic signal processing technology, and particularly relates to a Conformer-based method for processing acoustic data of underwater gliders and a storage medium. Background Technology

[0002] Underwater gliders are buoyancy-driven unmanned underwater platforms capable of zigzag profile motion. They are characterized by low energy consumption and low operating costs, enabling long-term, large-scale marine environmental surveillance. Underwater acoustic gliders equipped with hydrophones can simultaneously acquire acoustic and marine environmental information. Furthermore, multi-glider collaborative observation can be conducted for hydrological observation, target identification, and tracking, showing broad application prospects in precise marine environmental observation. In recent years, both domestic and international research has focused on the integration of underwater gliders with acoustic systems, and the application of acoustic underwater gliders has gained increasing attention. During marine observation missions, acoustic underwater gliders equipped with hydrophones can measure acoustic signals present in the ocean. The study of these acoustic signals not only serves the military's needs for underwater battlefield reconnaissance and target location but can also be applied to marine biological resource surveys and ecological environmental protection. However, due to the sparsity of effective signals in the ocean, acoustic underwater gliders generate a large amount of invalid data during actual signal acquisition and are also affected by platform self-noise, leading to excessive data volume in underwater target identification. Furthermore, the unavoidable self-noise from the underwater glider itself significantly reduces the accuracy of underwater target identification. Therefore, effectively extracting and denoising the data collected by acoustic underwater gliders is crucial for ensuring the effectiveness and accuracy of underwater target identification.

[0003] Current methods for extracting effective acoustic data from underwater acoustic gliders rely on traditional signal processing techniques, the core of which is based on the short-term stationary nature of speech signals. Specifically, noise power is assessed through power statistics of the first few frames, and the power of subsequent frames is compared with the noise power to identify and output the speech frame. Subsequent methods have introduced other low-level features or combinations thereof, such as zero-crossing rate, spectral / cepheid features, pitch detection, and higher-order statistics. Current noise reduction methods still depend on manual extraction of acoustic features, using CNN neural networks to learn manually extracted features such as LOFAR or DEMON spectra to achieve noise separation.

[0004] Traditional effective signal extraction methods extract acoustic signal features that only reflect the statistical properties of speech, resulting in limited accuracy in complex noisy environments. While later attempts have combined likelihood ratio tests to improve this, these methods rely on prior assumptions about speech and noise distributions (such as Gaussian, Laplace, and Gamma distributions), and parameter estimation is limited by localized datasets, thus restricting performance improvement. Furthermore, in underwater glider detection missions, especially in complex sound fields with coupled multi-source noise in the ocean, traditional denoising methods based on manually extracted spectrum analysis struggle to meet the real-time processing requirements of underwater gliders. Existing denoising technologies face the dual challenges of signal distortion and computational load in complex ocean noise suppression. Traditional methods struggle to balance noise suppression and signal fidelity, while deep learning models, although performing well in laboratory environments, are data-driven and their high computational complexity makes them unsuitable for the low-power hardware of underwater gliders, resulting in insufficient real-time processing capabilities.

[0005] Therefore, systematically solving the bottlenecks in detection and noise reduction, and developing reliable and efficient processing methods for marine acoustic data collected by underwater gliders, is of key significance for improving the engineering application capabilities of underwater glider acoustic detection. Summary of the Invention

[0006] To address the problems of low accuracy in effective signal extraction, limited noise reduction effect, and difficulty in meeting the real-time processing requirements of low-power platforms in existing acoustic underwater gliders operating in complex marine noise environments, this invention provides a Conformer-based acoustic data processing method and storage medium for underwater gliders.

[0007] This invention is implemented as follows: a Conformer-based method for processing acoustic data of underwater gliders, characterized by the following steps:

[0008] S1. The raw acoustic data collected by the underwater glider is processed using the MCR-AAD model to detect effective signals, output the probability that each frame of signal is an effective signal, and use a double threshold post-processing method to determine the boundary between effective signals and noise signals.

[0009] S2. Input the valid signal detected in S1 into the BGformer noise reduction model, generate a noise mask through the audio feature fusion module, DCC encoder module, Conformer decoder module and tag embedding module, and restore the noise-reduced target audio after inverting the mask.

[0010] The MCR-AAD model includes a Mel feature extractor, a multi-scale depthwise separable convolution module, a dynamic positional encoding module, a Conformer encoder, and a multi-task classification module; the BGformer model is causal and supports real-time processing.

[0011] In the above technical solution, preferably, in the MCR-AAD model: the multi-scale depth-separable convolution module extracts multi-scale features through different convolution kernel sizes; the dynamic position encoding module adds temporal position information to the audio features; and the multi-task classification module jointly outputs effective signal classification results at the frame and segment levels.

[0012] In the above technical solution, preferably, the training of the MCR-AAD model adopts a joint loss function, including frame-level cross-entropy loss and fragment-level binary classification loss.

[0013] In the above technical solution, preferably, in the BGformer model: the audio feature fusion module fuses Mel spectrum features with waveform features; the DCC encoder uses dilated causal convolution to extract temporal features; and the label embedding module enhances the noise type features to improve the robustness of the model.

[0014] In the above technical solution, preferably, according to the method of claim 1 or 4, the mask network of the BGformer model generates a mask based solely on historical data through a causal structure, thereby achieving real-time noise reduction processing.

[0015] In the above technical solution, preferably, the dual-threshold post-processing method includes:

[0016] A high threshold and a low threshold are set. When the signal probability is higher than the high threshold, it is determined to be a valid signal, and when it is lower than the low threshold, it is determined to be a noise signal. For signal segments between the two thresholds, smoothing processing is performed by combining the determination results of the previous and next frames.

[0017] In the above technical solution, preferably, the hidden state dimension of the Conformer encoder in the MCR-AAD model is 128, the number of attention heads is 4, and the number of layers is 3; the hidden state dimension of the DCC encoder in the BGformer model is 256, the hidden state dimension of the Conformer decoder is 128, the number of attention heads is 4, and the number of layers is 1.

[0018] In the above technical solution, preferably, the data preprocessing of the method includes: normalizing the original acoustic data and scaling the amplitude to the interval [-1,1]; cutting the audio segment by a sliding window with a window length of 30 seconds and an overlap time of 5 seconds.

[0019] In the above technical solution, preferably, the method improves the SI-SNR index of the output audio after noise reduction to above 13.64dB.

[0020] This invention proposes a Conformer-based acoustic data processing method for underwater gliders. It addresses the problems of inaccurate effective signal extraction, poor noise reduction, and poor real-time performance in complex marine environments, and comprehensively improves the signal processing capabilities of underwater gliders in acoustic detection missions. It has significant advantages and practical application effects.

[0021] Firstly, regarding effective signal extraction, this invention introduces the MCR-AAD (Multi-scale Conformer Robust Audio Activity Detector), a multi-scale audio activity detection model based on the Conformer architecture, capable of accurately identifying effective signal segments from massive amounts of acoustic data. This model learns the differences between complex background noise and effective acoustic signals through deep learning, exhibiting strong feature representation capabilities and robustness. In the detection process, the model outputs the probability of each frame being a valid signal, and combined with a dual-threshold post-processing mechanism, accurately distinguishes between signal frames and noise frames, significantly reducing the false detection rate of invalid data. Compared to traditional manual algorithms based on energy statistics or low-level spectral features, MCR-AAD does not rely on complex preprocessing or prior assumptions, achieving high-accuracy end-to-end detection even under multi-source background noise interference. This significantly improves the accuracy and stability of effective signal extraction, thereby reducing manual intervention and the burden of subsequent data processing, laying a solid foundation for subsequent target recognition.

[0022] Secondly, regarding noise reduction, this invention further proposes a deep learning model called BGformer. This model integrates the excellent temporal modeling capabilities of the Conformer architecture with the sensitivity of convolutional neural networks (CNNs) to local features, possessing good noise modeling and real-time processing capabilities. Unlike non-causal neural networks, BGformer exhibits strict causality, ensuring its real-time deployment and operation on the glider, meeting the latency and computational resource constraints of low-power platforms. Through testing and verification with real sea trial data, BGformer achieved a performance of 13.64 dB on the SI-SNRi (signal enhancement ratio improvement index), demonstrating excellent noise reduction effects and effectively suppressing the interference of marine environmental noise and glider platform self-noise on the effective signal, significantly reducing the signal distortion rate.

[0023] In summary, this invention introduces two Conformer-based deep learning models, MCR-AAD and BGformer, to construct a complete intelligent processing method for underwater glider acoustic data. While ensuring processing accuracy, it achieves efficient and real-time data cleaning and noise reduction, significantly improving the overall performance of underwater gliders in tasks such as ocean observation and target identification. This provides key technical support for the engineering deployment and practical application of underwater glider acoustic detection systems.

[0024] A second objective of this invention is to provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the Conformer-based underwater glider acoustic data processing method described in any of the preceding claims. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the MCR-AAD model structure proposed in this invention;

[0026] Figure 2 This is the overall architecture diagram of the BGformer noise reduction model proposed in this invention;

[0027] Figure 3 This is a detailed schematic diagram of the mask network in the BGformer model;

[0028] Figure 4 This is a flowchart of the overall acoustic data processing for underwater gliders;

[0029] Figure 5 This is a diagram illustrating an effective method for creating audio tags;

[0030] Figure 6 This is an example image of the detection results from the MCR-AAD model;

[0031] Figure 7 This is a comparison chart of the noise reduction effects of BGformer. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0033] To address the problems of low accuracy in effective signal extraction, limited noise reduction effect, and inability to meet the real-time processing requirements of low-power platforms in existing acoustic underwater gliders operating in complex marine noise environments, this invention provides a Conformer-based acoustic data processing method and storage medium for underwater gliders. To further illustrate the structure of this invention, a detailed description is provided below in conjunction with the accompanying drawings:

[0034] A Conformer-based method for processing acoustic data from underwater gliders includes the following steps:

[0035] S1. The raw acoustic data collected by the underwater glider is used to detect effective signals through the MCR-AAD model, outputting the probability that each frame of signal is a valid signal, and the boundary between valid signals and noise signals is determined by the double threshold post-processing method.

[0036] S2. Input the valid signal detected in S1 into the BGformer noise reduction model, generate a noise mask through the audio feature fusion module, DCC encoder module, Conformer decoder module and tag embedding module, and restore the noise-reduced target audio after inverting the mask.

[0037] The MCR-AAD model includes a Mel feature extractor, a multi-scale depthwise separable convolution module, a dynamic positional encoding module, a Conformer encoder, and a multi-task classification module; the BGformer model is causal and supports real-time processing.

[0038] In the MCR-AAD model: the multi-scale depthwise separable convolution module extracts multi-scale features using different convolution kernel sizes; the dynamic position encoding module adds temporal position information to audio features; and the multi-task classification module jointly outputs effective signal classification results at the frame and segment levels. The training of the MCR-AAD model employs a joint loss function, including frame-level cross-entropy loss and segment-level binary classification loss. In the BGformer model: the audio feature fusion module fuses Mel-spectral features with waveform features; the DCC encoder uses dilated causal convolution to extract temporal features; and the label embedding module enhances noise type features to improve model robustness. The masking network of the BGformer model generates masks based solely on historical data through a causal structure, achieving real-time noise reduction. The dual-threshold post-processing method includes: setting a high threshold and a low threshold; when the signal probability is higher than the high threshold, it is determined as a valid signal; when it is lower than the low threshold, it is determined as a noise signal; for signal segments between the two thresholds, smoothing is performed by combining the determination results of the preceding and following frames.

[0039] This invention constructs a hybrid model that integrates an audio feature extractor, a multi-scale depthwise separable convolution (Multi-Scale DSConv) module, a dynamic positional encoding module, a Conformer encoder, and a multi-task classification module. The feature extractor adaptively extracts Mel features from the audio, then performs multi-scale depthwise separable convolution on them, incorporates the extracted features into dynamic positional encoding, and finally uses a joint frame-level and segment-level classifier to classify the audio, outputting the final effective audio classification result. The proposed MCR-AAD model structure is as follows: Figure 1 As shown.

[0040] This method can better identify the boundaries of valid audio, has better robustness and accuracy, and effectively reduces the false negative rate and false positive rate of valid signals.

[0041] After extracting the effective audio using the above model, the data still contains unstable noise that can affect the target recognition results. Therefore, it is necessary to continue inputting a noise reduction model to suppress the noise in the audio in order to further improve the accuracy of underwater target recognition.

[0042] This invention proposes a Conformer-based BGformer noise reduction model. The core modules of BGformer are an audio feature fusion module, a DCC encoder module, a Conformer decoder module, and a label embedding module. The audio feature fusion module adaptively fuses the Mel-spectral and waveform features of the audio to form more robust features. These features are then input into a masking network composed of a DCC (Dilated Causal Convolution) encoder and a Conformer decoder to generate a mask for background noise. The mask is then inverted to form a mask for the target sound source to be highlighted, ultimately restoring the audio. Because marine environmental noise and underwater glider self-noise are highly variable and non-stationary, a label embedding mechanism is used to enhance their features, making them more robust and facilitating subsequent learning from new data.

[0043] The BGformer model structure proposed in this invention is as follows: Figure 2 As shown, its mask network is as follows Figure 3 As shown. This method ensures real-time data processing through a causal masking network, enabling it to perform noise reduction by inferring from historical data alone. It also has stronger robustness and effectively reduces the distortion of the restored audio.

[0044] Specifically, the overall process verification of underwater glider acoustic data processing is as follows: Figure 4 As shown in the diagram, an audio segment of a marine mammal's call was first selected from a publicly available dataset and randomly added to the complete data collected by an underwater glider. To further verify the model's ability to process signals under extreme interference, the signal-to-noise ratio was set to a certain value, the audio sampling rate was 32.768 kHz, and the data was divided into 30-second speech segments using a sliding window with a 5-second window overlap time. Each audio segment was then input into the MCR-AAD model for valid signal detection. The identified audio segments were then input into the BGformer model for noise reduction, ultimately yielding the denoised audio data.

[0045] Since the MCR-AAD audio activity detection model requires frame-level and segment-level labels, these need to be created in advance. The creation method is as follows: Figure 5 As shown, the 1-second audio segment in the valid segment is divided into 100 frames, with each segment consisting of 20 frames. If it is active audio, it is marked as 1; otherwise, it is marked as 0.

[0046] Data normalization is an important preprocessing step in machine learning, which can improve the training efficiency and effectiveness of models. Since the amplitude of acoustic data varies, it is scaled to the interval [-1,1] to represent only waveform features.

[0047] In the proposed MCR-AAD architecture, the hidden states of the Conformer layer have a dimension of 128, with 4 heads and 3 layers. All linear layers use the tanh activation function and a learning rate of 3e-5. Adam is used as the model's optimizer. The detection results are as follows: Figure 6 As shown, it can be seen that it accurately detected the part that produces biological sounds.

[0048] In the BGformer model, the hidden dimension of the DCC encoder is set to 256, the hidden dimension of the Conformer decoder is set to 128, the number of attention heads is set to 4, and the number of layers is 1. The denoising results are as follows: Figure 7 As shown, compared with clean speech, its SI-SNR value is 13.64dB, and the model has successfully completed the noise reduction task.

[0049] As can be seen, the MCR-AAD audio activity detection model proposed in this invention effectively solves the problem of end-to-end detection of active speech in long audio files, saving manpower and subsequent computing resources, and providing a reliable processing method for the identification of acoustic targets on underwater gliders. The proposed BGformer hybrid architecture denoising model integrates the temporal modeling capabilities of Conformer with the local feature perception advantages of CNN. Its strict causal properties enable real-time noise processing on underwater gliders. Tested using actual sea trial data, its denoising effect reaches 13.64 dB on the SI-SNR index, providing a new approach to denoising acoustic signals detected by underwater gliders, in addition to mechanical denoising.

[0050] Example 2

[0051] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the Conformer-based underwater glider acoustic data processing method as proposed in Embodiment 1.

[0052] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A Conformer-based method for processing acoustic data of underwater gliders, characterized in that, Includes the following steps: S1. The raw acoustic data collected by the underwater glider is processed using the MCR-AAD model to detect effective signals, output the probability that each frame of signal is an effective signal, and use a double threshold post-processing method to determine the boundary between effective signals and noise signals. S2. Input the valid signal detected in S1 into the BGformer noise reduction model, generate a noise mask through the audio feature fusion module, DCC encoder module, Conformer decoder module and tag embedding module, and restore the noise-reduced target audio after inverting the mask. The MCR-AAD model includes a Mel feature extractor, a multi-scale depthwise separable convolutional module, a dynamic positional encoding module, a Conformer encoder, and a multi-task classification module; the BGformer model is causal and supports real-time processing. In the MCR-AAD model: the multi-scale depth-separable convolution module extracts multi-scale features through different convolution kernel sizes; the dynamic position encoding module adds temporal position information to the audio features; and the multi-task classification module jointly outputs effective signal classification results at the frame and segment levels. In the BGformer model: the audio feature fusion module fuses Mel-spectral features with waveform features; the DCC encoder uses dilated causal convolution to extract temporal features; and the label embedding module enhances the noise type features to improve the model's robustness. The dual-threshold post-processing method includes: A high threshold and a low threshold are set. When the signal probability is higher than the high threshold, it is determined to be a valid signal, and when it is lower than the low threshold, it is determined to be a noise signal. For signal segments between the two thresholds, smoothing processing is performed by combining the determination results of the previous and next frames.

2. The underwater glider acoustic data processing method based on Conformer according to claim 1, characterized in that, The MCR-AAD model is trained using a joint loss function, which includes frame-level cross-entropy loss and fragment-level binary classification loss.

3. The underwater glider acoustic data processing method based on Conformer according to claim 1, characterized in that, The masking network of the BGformer model generates masks based solely on historical data through a causal structure, achieving real-time noise reduction.

4. The underwater glider acoustic data processing method based on Conformer according to claim 1, characterized in that, In the MCR-AAD model, the hidden state dimension of the Conformer encoder is 128, the number of attention heads is 4, and the number of layers is 3; in the BGformer model, the hidden state dimension of the DCC encoder is 256, the hidden state dimension of the Conformer decoder is 128, the number of attention heads is 4, and the number of layers is 1.

5. The underwater glider acoustic data processing method based on Conformer according to claim 1, characterized in that, The data preprocessing of the method includes: normalizing the original acoustic data and scaling the amplitude to the interval [-1,1]; cutting the audio segment by a sliding window with a window length of 30 seconds and an overlap time of 5 seconds.

6. The underwater glider acoustic data processing method based on Conformer according to claim 1, characterized in that, The method improves the SI-SNR of the output audio after noise reduction to over 13.64 dB.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the Conformer-based underwater glider acoustic data processing method as described in any one of claims 1-6.

Images