Underwater sound target identification method and system based on soft label guidance and domain adversarial fine tuning
By using soft-label guidance and domain adversarial fine-tuning, combined with dynamic convolution and long short-term memory networks, the problems of low signal-to-noise ratio and low recognition accuracy of dynamically added categories in passive sonar target recognition are solved, achieving efficient recognition in complex tasks.
Patent Information
- Application Number
- CN202511663469.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-13
- Publication Date
- 2026-01-09
AI Technical Summary
Existing passive sonar target recognition technologies suffer from low recognition accuracy and poor adaptability in scenarios with low signal-to-noise ratios and dynamically added categories. In particular, deep learning models such as RNNs are prone to gradient vanishing or exploding when dealing with long-term dependencies, leading to performance degradation.
We adopt a method based on soft label guidance and domain adversarial fine-tuning. By constructing a teacher-student model system, combining dynamic convolutional feature extraction, residual-enhanced long short-term memory network and domain discriminator, and using soft label distillation loss and domain adversarial loss for joint training, we enhance the model's adaptive modeling ability and long-term sequence dependency modeling ability.
It significantly improves the accuracy and robustness of underwater acoustic target recognition, especially under small sample conditions, and effectively enhances the recognition effect, thus solving the performance bottleneck of deep learning models in complex tasks.
Smart Images
Figure CN121306193A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of underwater acoustic target recognition technology, specifically relating to an underwater acoustic target recognition method and system based on soft tag guidance and domain adversarial fine-tuning. Background Technology
[0002] Underwater acoustic target recognition is a crucial component of underwater acoustic signal processing and a vital technological support for underwater acoustic information acquisition and countermeasures. In recent years, deep convolutional neural networks (such as ResNet and DenseNet) have been applied to underwater acoustic target recognition, significantly improving recognition accuracy. However, in passive sonar target recognition, efficiently addressing the challenges of low signal-to-noise ratios and strong non-stationary signals remains a key challenge.
[0003] Deep learning models, especially pre-trained neural networks, have demonstrated superior feature extraction capabilities thanks to their training on large-scale datasets. However, these models still face performance bottlenecks in passive sonar target recognition, particularly in scenarios with dynamically added categories, where the class increment support of traditional models proves insufficient. Recurrent Neural Networks (RNNs) are deep learning models capable of capturing the temporal features of sequential data and are widely used in natural language processing, speech recognition, and time series prediction. However, traditional RNN models are prone to performance degradation due to gradient vanishing or exploding problems when handling long-term dependencies. This limitation restricts their application in more complex tasks. Summary of the Invention
[0004] This invention aims to solve the problems of low signal-to-noise ratio, poor adaptability, and low target recognition accuracy in existing passive sonar target recognition technologies, and proposes an underwater acoustic target recognition method and system based on soft tag guidance and domain adversarial fine-tuning.
[0005] In a first aspect, the present invention provides a method for underwater acoustic target recognition based on soft tag guidance and domain adversarial fine-tuning, the method comprising:
[0006] Construct a labeled underwater acoustic dataset, an unlabeled underwater acoustic dataset, and a source domain general audio dataset containing different audio data, and convert the audio data in the dataset into Mel spectrograms;
[0007] A teacher model and a student model are constructed. The student model includes a dynamically convolutional feature extraction module, a feature enhancement module, and a classification module connected in sequence. The dynamically convolutional feature extraction module includes a dynamically convolutional block and multiple dynamically inverted residual blocks connected in sequence. The feature enhancement module includes multiple residual-enhanced long short-term memory network modules connected in sequence. The classification module includes a classification layer. The classification layer obtains the output result of the student model based on the feature map output by the feature enhancement module.
[0008] In the residual enhanced long short-term memory network module, the input feature map is processed by batch normalization to obtain the initial feature map; the initial feature map is then processed by causal convolutional layers and activation functions in sequence, and the processing results are combined with the initial feature map and input into the enhanced long short-term memory network layer to obtain the intermediate feature map; the intermediate feature map is then processed by batch normalization, and the processing results are residually connected with the input feature map to obtain the output feature map of the residual enhanced long short-term memory network module;
[0009] The teacher model was trained using a source domain general audio dataset, and the student model was trained using different datasets and the trained teacher model by constructing a joint loss function that includes classification loss and soft label distillation loss.
[0010] The trained student model is used to identify the measured Mel spectrogram and obtain the underwater target corresponding to the measured Mel spectrogram.
[0011] Preferably, the classification module further includes a domain discriminator; the domain discriminator is used to obtain the audio signal type corresponding to the measured Mel spectrogram based on the feature map output by the feature enhancement module.
[0012] The joint loss function also includes domain adversarial loss; the domain adversarial loss is constructed by inputting the Mel spectrograms from the unlabeled underwater acoustic dataset and the source domain general audio dataset into the student model, and constructing the domain adversarial loss based on the discrimination results of the domain discriminator for the underwater acoustic data and the general audio data.
[0013] Preferably, the method for constructing the classification loss is as follows: input the Mel spectrogram from the labeled underwater acoustic dataset into the student model, and construct the classification loss based on the classification result of the classification layer in the student model and the label corresponding to the Mel spectrogram input into the student model;
[0014] The soft-label distillation loss is constructed as follows: Mel spectrograms from the unlabeled underwater acoustic dataset are input into the student model and the teacher model, respectively, and the soft-label distillation loss is constructed based on the classification results of the classification layer in the student model and the classification results of the teacher model.
[0015] Preferably, the dynamic inverted residual block includes a first convolutional block, a second convolutional block, a coordinate attention module, and a third convolutional block. In the dynamic inverted residual block, the input feature map is first convolved by the first and second convolutional blocks in sequence. Then, the coordinate attention module is used to perform attention weighting on the time dimension and frequency dimension of the output feature map of the second convolutional block, respectively. Finally, the output feature map of the coordinate attention module is processed by the third convolutional block, and the processing result is concatenated with the input feature map to obtain the output result of the dynamic inverted residual block.
[0016] Preferably, the first convolutional block includes a series of dynamic convolutional layers, a batch normalization layer, and an activation function; the second convolutional block includes a series of dynamic convolutional layers, a batch normalization layer, and a dynamic activation function; and the third convolutional block includes a series of dynamic convolutional layers and a batch normalization layer.
[0017] Preferably, the method for obtaining the Mel spectrogram is as follows: segmenting the audio data in the dataset and applying a Hanning window to the segmented data; then performing a Fast Fourier Transform on the windowed data to obtain the frequency domain components. The Mel spectrum characteristics after passing through the filter bank are calculated; the logarithm of the Mel spectrum characteristics is taken to obtain the logarithmic Mel spectrum, which is used as the Mel spectrum input to the model.
[0018] Preferably, during the acquisition of the Mel spectrogram, several frequency bands in the Mel filter bank are randomly selected and set to zero, and several time frames are randomly selected and set to zero.
[0019] Secondly, the present invention provides an underwater acoustic target recognition system based on soft tag guidance and domain adversarial fine-tuning, which uses the above-mentioned underwater acoustic target recognition method; the underwater acoustic target recognition system includes a data acquisition module, a Mel spectrogram extraction module, and an underwater acoustic target recognition module; the data acquisition module is used to acquire underwater acoustic audio; the Mel spectrogram extraction module is used to convert underwater acoustic audio into a Mel spectrogram; the underwater acoustic target recognition module is used to identify the corresponding target based on the measured Mel spectrogram.
[0020] Thirdly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: the memory stores the computer program; and the processor executes the above-described underwater acoustic target recognition method.
[0021] Fourthly, the present invention provides a readable storage medium storing a computer program; when the computer program is executed by a processor, it is used to implement the above-described underwater acoustic target recognition method.
[0022] The beneficial effects of this invention are:
[0023] 1. This invention enhances the model's adaptive modeling ability for input signal characteristics by introducing dynamic convolution and dynamic activation functions into the dynamic convolution feature extraction module, and improves the ability to capture key features of underwater acoustic signals by combining coordinate attention mechanism for in-depth modeling of time and frequency information. At the same time, this invention solves the problem of gradient vanishing or exploding in deep networks by introducing multi-layer enhanced long short-term memory network module and residual connection mechanism in feature enhancement module, and introduces exponential gating mechanism and new memory mixing method in enhanced long short-term memory network module, effectively enhancing the modeling ability for long-term sequence dependencies.
[0024] 2. This invention utilizes a combination of soft label guidance and domain adversarial mechanisms during the fine-tuning stage. On the one hand, it uses a pre-trained model to generate soft labels for unlabeled underwater acoustic data to convey fine-grained semantic knowledge. On the other hand, it forces the alignment of the feature distributions of general audio and underwater acoustics through a gradient inversion layer and a domain discriminator. Thus, while maintaining the inference efficiency of the original model, it significantly improves the accuracy and robustness of underwater acoustic target recognition in small samples. Attached Figure Description
[0025] Figure 1 This is a flowchart of the Mel spectrum feature extraction process in this invention.
[0026] Figure 2 This is a structural diagram of the teacher model and student model in this invention.
[0027] Figure 3 This is a structural diagram of the dynamically inverted residual block in this invention.
[0028] Figure 4 This is a structural diagram of the residual-enhanced long short-term memory network module in this invention. Detailed Implementation
[0029] The present invention will be further described below with reference to the accompanying drawings.
[0030] A method for underwater acoustic target recognition based on soft tag guidance and domain adversarial fine-tuning is proposed. The underwater acoustic target recognition system includes a data acquisition module, a Mel spectrogram extraction module, and an underwater acoustic target recognition module. The data acquisition module is used to acquire underwater acoustic audio. The Mel spectrogram extraction module is used to convert the underwater acoustic audio into a Mel spectrogram; the underwater acoustic target recognition module is used to identify the corresponding target based on the Mel spectrogram.
[0031] The underwater acoustic target identification method includes the following steps:
[0032] Step 1: Building the dataset
[0033] Different types of datasets were constructed based on the original audio signals: labeled underwater acoustic datasets, unlabeled underwater acoustic datasets, and source domain general audio datasets. The original audio data in the datasets was segmented into multiple sub-audio segments, each 5 seconds long. All sub-audio segments from the same segment were treated as a single sample, with each sample corresponding to a complete segment of the original audio data. The number of sub-audio segments in each sample was recorded. The dataset was then divided into training and validation sets in a 7:3 ratio, with the training set containing more samples than the validation set. In the partitioning of the labeled underwater acoustic dataset, samples from each category were randomly shuffled and proportionally allocated to the training and validation sets. This ensured that all sub-audio segments from the same segment were strictly assigned to the same subset, guaranteeing a balanced distribution of data between the training and validation sets and preventing information from the same audio segment from appearing in both sets, thus fundamentally eliminating data leakage.
[0034] In this embodiment, the source domain general audio dataset uses the AudioSet dataset, which consists of airborne sound data.
[0035] Step 2: Extract the Mel spectrum
[0036] like Figure 1 As shown, feature extraction and enhancement are performed on the original audio data in the dataset. By designing a Mel frequency scale filter bank, the feature information of the target signal is extracted and optimized, and finally the enhanced Mel spectrogram is obtained. The specific process is as follows:
[0037] 2-1. Divide the sub-audio segment into several sampling point sequences of length N according to time, and process each sampling point s[n] in the sampling point sequence as an analysis unit; where, .
[0038] 2-2. To avoid spectral leakage at signal boundaries, a Hanning window is used for windowing. Hanning window function. The expression is:
[0039]
[0040] Signal after adding a window Represented as:
[0041]
[0042] 2-3. Perform a Fast Fourier Transform (FFT) on the windowed signal to obtain its frequency domain components. Its expression is:
[0043]
[0044] Where N is the sampling point of the signal in the time domain; n is the time domain index; and k is the frequency index.
[0045] 2-4. Set up the Mel filter bank to map the spectrum to the Mel frequency domain. The expression is:
[0046]
[0047] in, This refers to the actual frequency.
[0048] To generate the Mel filter bank, define M filters, each with a center frequency of . Its expression is:
[0049]
[0050] in, and These are the lowest and highest frequencies of the Mel filter bank, respectively.
[0051] Mel filter response function The expression is:
[0052]
[0053] in, This is the actual frequency corresponding to the k-th frequency point.
[0054] 2-5. For each frame of audio signal, calculate its Mel-spectral characteristics after passing through the filter bank. Its expression is:
[0055]
[0056] To improve robustness, the logarithm of the Mel spectrum features is taken to obtain the log-Mel spectrum. The Mel spectrogram, which serves as the model input, is represented as follows:
[0057]
[0058] in, This is a smoothing parameter used to prevent numerical overflow during logarithmic operations.
[0059] 2-6. To improve the time-frequency robustness of the features, data augmentation is performed on the Mel spectral features. Several frequency bands in the Mel filter bank are randomly selected and zeroed, resulting in the loss of simulated frequencies. Several time frames are also randomly selected and zeroed, resulting in the loss of simulated time.
[0060] Step 3: Construct a teacher-student model
[0061] like Figure 2 As shown, teacher and student models are constructed respectively. The student model includes a dynamically convolutional feature extraction module, a feature enhancement module, and a classification module connected in sequence. The dynamically convolutional feature extraction module includes a dynamically convolutional block (7×7) and seven dynamically inverted residual blocks connected in sequence. The feature enhancement module includes a six-layer cascaded residual-enhanced long short-term memory network module (residual sLSTM). The classification module includes a classification layer and a domain discriminator in parallel; the classification layer is used to obtain the target recognition result corresponding to the measured Mel spectrogram based on the feature map output by the feature enhancement module; the domain discriminator is used to obtain the audio signal type (underwater acoustic data or airborne acoustic data) corresponding to the measured Mel spectrogram based on the feature map output by the feature enhancement module.
[0062] In this embodiment, the teacher model adopts Audio Mamba-FOBI (Feature-Only Bottleneck Integration).
[0063] In this embodiment, the classification layer includes multiple fully connected layers connected in sequence; the domain discriminator includes three MLP (Multilayer Perceptron) layers connected in sequence.
[0064] like Figure 3 As shown, the dynamically inverted residual block includes a first convolutional block, a second convolutional block, a coordinate attention module (CA), and a third convolutional block. In the dynamically inverted residual block, the input feature map is first convolved sequentially through the first and second convolutional blocks, allowing the convolutional kernels to dynamically adjust according to the input feature map to extract local and global features. Then, the coordinate attention module applies attention weights to the time and frequency dimensions of the output feature map of the second convolutional block, further enhancing the ability to capture time-series and spectral features. Finally, the third convolutional block processes the output feature map of the coordinate attention module, and the processing result is concatenated with the input feature map to obtain the output of the dynamically inverted residual block.
[0065] The first convolutional block consists of cascaded dynamic convolutional layers, batch normalization layers, and a ReLU activation function. The second convolutional block consists of cascaded dynamic convolutional layers, batch normalization layers, and a dynamic ReLU activation function. The third convolutional block consists of cascaded dynamic convolutional layers and batch normalization layers. By introducing dynamic convolutional layers and dynamic activation functions, the model's adaptive modeling ability to the characteristics of the input signal is enhanced.
[0066] In this embodiment, the sizes of the dynamic convolutional layers in the first, second, and third convolutional blocks of the first three dynamically inverted residual blocks are 1×1, 3×3, and 1×1, respectively; and the sizes of the dynamic convolutional layers in the first, second, and third convolutional blocks of the last four dynamically inverted residual blocks are 1×1, 5×5, and 1×1, respectively.
[0067] like Figure 4 As shown, in the residual enhanced long short-term memory network module, the input feature map is processed by batch normalization (LN) to obtain the initial feature map; the initial feature map is then processed by causal convolutional layers and the Swish activation function in sequence, and the processing results are combined with the initial feature map and input into the enhanced long short-term memory network layer (sLSTM layer) to obtain the intermediate feature map; the intermediate feature map is then processed by batch normalization, and the processing results are residually connected with the input feature map to obtain the output feature map of the residual enhanced long short-term memory network module.
[0068] Step 4: Training the model
[0069] The teacher model is pre-trained using a source-domain general audio dataset. After the teacher model is trained, the student model is trained using a joint loss function that guides parameter updates in a training batch of the student model, derived from a labeled underwater acoustic dataset, an unlabeled underwater acoustic dataset, and a source-domain general audio dataset. Represented as:
[0070]
[0071] in, For classification loss; For soft label distillation loss; Losses due to domain confrontation; Weighting for distillation loss; As the basic weight for domain adversarial operations; The dynamic coefficient is expressed as:
[0072]
[0073] in, Indicates the level of training. ; Indicates the current training batch; This indicates the total number of training batches.
[0074] The classification loss is constructed as follows: Mel spectrograms from the labeled underwater acoustic dataset are input into the student model, and the classification loss is constructed based on the classification results of the classification layer in the student model and the labels corresponding to the Mel spectrograms input into the student model. It is represented as:
[0075]
[0076] in, Indicates the output labels of the classification layer The classification probability; Indicates the output type of the classification layer The classification probability; The number of underwater target types; To output labels for the classification layer The classification probabilities are normalized.
[0077] The soft-label distillation loss is constructed as follows: Mel spectrograms from the unlabeled underwater acoustic dataset are input into the student model and the teacher model, respectively. The soft-label distillation loss is constructed based on the classification results of the classification layer in the student model and the classification results of the teacher model. It is represented as:
[0078]
[0079] in, Indicates the output type of the teacher model The classification probability; Indicates the output type of the classification layer The classification probability; This represents the classification probability output by the teacher model; This represents the classification probability output by the classification layer; Represents relative entropy.
[0080] The domain adversarial loss is constructed as follows: Mel spectrograms from the unlabeled underwater acoustic dataset and the source domain general audio dataset are input into the student model, respectively. The domain adversarial loss is constructed based on the discrimination results of the domain discriminator in the student model for the underwater acoustic data and the general audio data. , which is represented as
[0081]
[0082] in, For domain discriminators for general audio data The judgment result; For the domain discriminator for underwater acoustic data The judgment result; The number of general audio data in a single training batch; The number of underwater acoustic data in a single training batch.
[0083] In this embodiment, the Adam optimizer is used to minimize the domain adversarial loss. Learning rate Minimize the joint loss function using the AdamW optimizer. Learning rate .
[0084] Step 5: Use the trained student model to identify the measured Mel-ray spectrogram, and obtain the underwater target corresponding to the measured Mel-ray spectrogram based on the output of the classification layer in the student model. The underwater target identification results of different target identification methods are shown in Table 1.
[0085] Table 1. Recognition results of different target recognition methods
[0086] Model accuracy Overall accuracy Recall rate F1 score This invention 0.785 0.774 0.768 Yamnet 0.6953 0.6959 0.6929 VGGish 0.6685 0.6675 0.6656 CRNN9 0.7614 0.7649 0.7606 ResNet152 0.732 0.731 0.7318
[0087] As can be seen from Table 1, the overall accuracy of the recognition effect of the present invention reaches 78.5%, which is significantly higher than the accuracy of the current mainstream model under strict data segmentation conditions, demonstrating the effectiveness of the method.
Claims
1. A method for underwater acoustic target recognition based on soft tag guidance and domain adversarial fine-tuning, characterized in that: The method includes: Construct a labeled underwater acoustic dataset, an unlabeled underwater acoustic dataset, and a source domain general audio dataset containing different audio data, and convert the audio data in the dataset into Mel spectrograms; A teacher model and a student model are constructed. The student model includes a dynamically convolutional feature extraction module, a feature enhancement module, and a classification module connected in sequence. The dynamically convolutional feature extraction module includes a dynamically convolutional block and multiple dynamically inverted residual blocks connected in sequence. The feature enhancement module includes multiple residual-enhanced long short-term memory network modules connected in sequence. The classification module includes a classification layer. The classification layer obtains the output result of the student model based on the feature map output by the feature enhancement module. In the residual enhanced long short-term memory network module, the input feature map is processed by batch normalization to obtain the initial feature map; the initial feature map is then processed by causal convolutional layers and activation functions in sequence, and the processing results are combined with the initial feature map and input into the enhanced long short-term memory network layer to obtain the intermediate feature map; the intermediate feature map is then processed by batch normalization, and the processing results are residually connected with the input feature map to obtain the output feature map of the residual enhanced long short-term memory network module; The teacher model was trained using a source domain general audio dataset, and the student model was trained using different datasets and the trained teacher model by constructing a joint loss function that includes classification loss and soft label distillation loss. The trained student model is used to identify the measured Mel spectrogram and obtain the underwater target corresponding to the measured Mel spectrogram.
2. The underwater acoustic target recognition method based on soft tag guidance and domain adversarial fine-tuning according to claim 1, characterized in that: The classification module also includes a domain discriminator; the domain discriminator is used to obtain the audio signal type corresponding to the measured Mel spectrogram based on the feature map output by the feature enhancement module. The joint loss function also includes domain adversarial loss; the domain adversarial loss is constructed by inputting the Mel spectrograms from the unlabeled underwater acoustic dataset and the source domain general audio dataset into the student model, and constructing the domain adversarial loss based on the discrimination results of the domain discriminator for the underwater acoustic data and the general audio data.
3. The underwater acoustic target recognition method based on soft tag guidance and domain adversarial fine-tuning according to claim 1, characterized in that: The classification loss is constructed as follows: the Mel spectrogram in the labeled underwater acoustic dataset is input into the student model, and the classification loss is constructed based on the classification result of the classification layer in the student model and the label corresponding to the Mel spectrogram input into the student model. The soft-label distillation loss is constructed as follows: Mel spectrograms from the unlabeled underwater acoustic dataset are input into the student model and the teacher model, respectively, and the soft-label distillation loss is constructed based on the classification results of the classification layer in the student model and the classification results of the teacher model.
4. The underwater acoustic target recognition method based on soft tag guidance and domain adversarial fine-tuning according to claim 1, characterized in that: The dynamic inverted residual block includes a first convolutional block, a second convolutional block, a coordinate attention module, and a third convolutional block. In the dynamic inverted residual block, the input feature map is first convolved by the first and second convolutional blocks in sequence. Then, the coordinate attention module is used to perform attention weighting on the time and frequency dimensions of the output feature map of the second convolutional block. Finally, the output feature map of the coordinate attention module is processed by the third convolutional block, and the processing result is concatenated with the input feature map to obtain the output result of the dynamic inverted residual block.
5. The underwater acoustic target recognition method based on soft tag guidance and domain adversarial fine-tuning according to claim 4, characterized in that: The first convolutional block includes a series of dynamic convolutional layers, a batch normalization layer, and an activation function; the second convolutional block includes a series of dynamic convolutional layers, a batch normalization layer, and a dynamic activation function; the third convolutional block includes a series of dynamic convolutional layers and a batch normalization layer.
6. The underwater acoustic target recognition method based on soft tag guidance and domain adversarial fine-tuning according to claim 1, characterized in that: The method for obtaining the Mel spectrogram is as follows: the audio data in the dataset is segmented, and a Hanning window is used to window the segmented data; the windowed data is then subjected to a Fast Fourier Transform to obtain the frequency domain components. The Mel spectrum characteristics after passing through the filter bank are calculated; the logarithm of the Mel spectrum characteristics is taken to obtain the logarithmic Mel spectrum, which is used as the Mel spectrum input to the model.
7. The underwater acoustic target recognition method based on soft tag guidance and domain adversarial fine-tuning according to claim 1, characterized in that: During the acquisition of the Mel spectrogram, several frequency bands in the Mel filter bank are randomly selected and set to zero, and several time frames are randomly selected and set to zero.
8. A hydroacoustic target recognition system based on soft tag guidance and domain adversarial fine-tuning, characterized in that: The underwater acoustic target recognition method based on soft tag guidance and domain adversarial fine-tuning as described in claim 1 is used. The underwater acoustic target recognition system includes a data acquisition module, a Mel spectrogram extraction module, and an underwater acoustic target recognition module. The data acquisition module is used to acquire underwater acoustic audio. The Mel spectrogram extraction module is used to convert the underwater acoustic audio into a Mel spectrogram. The underwater acoustic target recognition module is used to identify the corresponding target based on the measured Mel spectrogram.
9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: The memory stores a computer program; the processor executes the underwater acoustic target recognition method as described in any one of claims 1-7.
10. A readable storage medium storing a computer program; characterized in that: When the computer program is executed by the processor, it is used to implement the underwater acoustic target recognition method as described in any one of claims 1-7.