Label-free sample assistance-based few-sample underwater acoustic target identification method and system

Abstract

The invention provides a few-sample underwater acoustic target identification method and system based on label-free sample assistance, and the method comprises the steps: obtaining a large-scale base data set, pre-training a model on the large-scale base data set, and completing the updating of network parameters in the pre-training process of the model; network parameters are used as initial parameters, and a label sample in a support set # imgabs0 # and a label-free sample fine tuning model of a current domain are used; and a feature extractor f phi, an environment feature adaptation module # imgabs 1 # and a classifier g theta are used to predict samples in the query set # imgabs 2 #, and an underwater acoustic target recognition result is obtained. The system comprises an obtaining unit and a processing unit, the obtaining unit obtains a label-free sample, the label-free sample is input into the processing unit, a model in the processing unit predicts the label-free sample, and an underwater acoustic target recognition result is output. According to the method, the fine tuning performance of the model in a few-sample scene is enhanced by using the label-free sample in the target domain; the model is helped to select features by comparing the features with environmental information in a feature extraction stage.

Description

Technical Field

[0001] The present invention belongs to the technical field of underwater acoustic target recognition, and in particular relates to a method and system for underwater acoustic target recognition with a small number of samples assisted by unlabeled samples. Background Art

[0002] Hydroacoustic target recognition technology is a key technology in marine engineering. It is of great significance in marine development and national defense security, and has become one of the research hotspots in the field of hydroacoustics. The radiation noise of underwater targets collected through passive sonar is the main source of information for hydroacoustic target recognition. Since the passive sonar system has good concealment and flexibility, how to use the radiation noise of ships obtained by passive sonar to detect and identify underwater targets has become an urgent problem to be solved. However, the underwater environment is complex and changeable, and accurate target recognition is challenging.

[0003] In recent years, with the continuous development of deep learning theory, researchers have applied CNN, Transformer and other models to underwater acoustic target recognition technology, achieving good recognition results. However, underwater acoustic target recognition methods based on deep learning are based on a large amount of data. In actual scenarios, underwater acoustic data collection is very difficult, and high-quality annotated data is often very scarce.

[0004] There are currently two main approaches to solving the data shortage problem: one is to use generative models for data enhancement, and the other is to solve this problem from the perspective of few-shot learning methods. In terms of generative models, some researchers use generative adversarial networks (GANs) to enhance samples. However, the training of generative adversarial networks (GANs) is unstable, and there may be a knowledge gap between the generated samples and the original samples, which affects the deployment of the model in real underwater environments. At the same time, few-shot learning methods can also be used in situations where data samples are limited, and have been successfully used in underwater missions. Although existing few-shot learning methods can alleviate the problem of data scarcity, they are still affected by limited application scenarios and poor performance.

[0005] For application scenarios with few samples, a large number of unlabeled samples can be easily obtained in real underwater acoustic applications. These large numbers of unlabeled samples include underwater sound signals from different environments and sources. Actively utilizing unlabeled samples can explore the intrinsic structure and pattern of underwater acoustic signals, reveal potential feature representations and correlations, and thus improve the recognition performance of the model. However, due to the lack of effective utilization methods, these unlabeled samples are often overlooked. Summary of the invention

[0006] The purpose of the present invention is to solve the shortcomings of the prior art that unlabeled samples cannot be effectively utilized, and to provide a few-sample underwater acoustic target recognition method and system based on the assistance of unlabeled samples, which enhances the fine-tuning performance of the model in the few-sample scenario by utilizing the unlabeled samples in the current domain.

[0007] To achieve the above purpose, the technical solution provided by the present invention is:

[0008] On the one hand, the present invention provides a method for underwater acoustic target recognition based on a small number of samples assisted by unlabeled samples, comprising:

[0009] Step 1: training model: obtaining a large-scale base data set, pre-training the model on the large-scale base data set, and completing the update of network parameters during the pre-training model, wherein: the model includes: feature extractor f φ , Environmental Characteristics Adaptation Module Frequency feature comparison module v ω and classifier g θ ;

[0010] Step 2: Fine-tune the model: Use the network parameters as initial parameters and use the new class data set to concentrate the support set Fine-tune the model using labeled samples in and unlabeled samples of the current domain;

[0011] Step 3: Use the model: Use the feature extractor f of the model after fine-tuning φ , Environmental Characteristics Adaptation Module and classifier g θ , query the new class data set The samples in are predicted to obtain the underwater acoustic target recognition results.

[0012] Based on one aspect, as a further limitation of the present invention, in the step 1, pre-training the model on the large-scale base data set includes:

[0013] In the base dataset The model is pre-trained, Mel-spectrogram features are extracted from the original audio, and the Mel-spectrogram is enhanced by applying data augmentation techniques;

[0014] The enhanced Mel-spectrogram is used as the input of the model, and the model’s feature extractor f φ Extract effective feature representation;

[0015] Input the effective feature representation to the environment feature adaptation module of the model And frequency feature comparison module v ω In the model, the classifier g θ Adapting environmental features to the module The output feature vectors are divided into different categories, and the frequency feature comparison module v ω comparing the similarity of two enhanced views of effective feature representation in corresponding frequency bands;

[0016] In the step 1, the updating of network parameters is completed in the pre-training model process, including:

[0017] By jointly optimizing the cross entropy loss Sum band contrast loss Complete the update of model parameters during pre-training;

[0018] When updating, the contrast loss in the frequency band Previously, the hyperparameter α was introduced, and its formula is:

[0019] Based on one aspect, as a further limitation of the present invention, in the step 2, using the network parameters as initial parameters includes:

[0020] The feature extractor f in the pre-trained model φ , Environmental Characteristics Adaptation Module And frequency feature comparison module v ω The parameters of are transferred to the model in the fine-tuning stage, and the classifier g is randomly initialized. θ Parameters;

[0021] In step 2, the new class data set is used to support the Fine-tune the model using labeled samples from the domain and unlabeled samples from the current domain, including:

[0022] Will support set The labeled samples in the current domain and the unlabeled samples in the current domain first extract the Mel spectrum features, and then obtain the enhanced Mel spectrum through data enhancement. The enhanced Mel spectrum is input into the environmental feature adaptation module. And frequency feature comparison module v ω ;

[0023] Use the environmental feature adaptation module and classifier g θ Calculating cross entropy loss Use frequency feature comparison module v ω Calculate the band contrast loss Cross Entropy Loss Sum band contrast loss Jointly supervise the fine-tuning process to make the model better adapt to the target domain.

[0024] Based on one aspect, as a further limitation of the present invention, in step 2, the environmental feature adaptation module Includes memory queue and attention pooling layer, where: support set The label sample in is denoted as d 1 ,d 2 ,...,d NK , where NK represents the support set The number of samples in the current domain; the unlabeled samples in the current domain are represented as x 1 ,x 2 ,...,x k , encoded into a vector e 1 ,e 2 ,..,e k , k represents the number of unlabeled samples in a batch;

[0025] Environmental characteristics adaptation module Will support set The feature vectors of the labeled samples and the feature vectors of a batch of unlabeled samples are taken as input, the two feature vectors are concatenated and input into the memory queue, and the feature vectors of the same number are removed from the memory queue to complete the update of the memory queue;

[0026] Output first N mq vectors as the input of the attention pooling layer, where N mq is the number of feature vectors stored in the memory queue; the attention pooling layer is N through two fully connected layers mq The scalar score α is calculated for each encoding vector in the vector i,c : In the formula, e i represents the feature representation of the i-th sample in the queue, W represents the first parameter of the first linear layer, b represents the second parameter of the first linear layer, and the information is projected into a smaller R-dimensional representation. f(·) represents the nonlinear activation function. After f(·), the information is activated by and k c Parameterized linear layer, get α i,c , where α i,c represents the attention score of the i-th sample in the feature dimension c;

[0027] Apply the softmax function to calculate the normalized score of each sample in the feature dimension c:

[0028]

[0029] In the formula, α′ i,c Indicates the importance of each sample in the feature dimension c, and calculates the weighted average μ within the feature dimension c c and weighted standard deviation σ c , the calculation formula is:

[0030]

[0031] Using the weighted average μ c and weighted standard deviation σ c Calculate the weighted average μ of each feature dimension c c and weighted standard deviation σ c , the obtained weighted average μ c and weighted standard deviation σ c Connect along the feature dimension c to form the final weighted mean vector μ and weighted standard deviation vector σ;

[0032] Will come from the support set The eigenvector d 1 ,d 2 ,...,d NK It is connected with the mean μ and standard deviation σ of the environmental information to form a vector e′, which is expressed as e′=[d 1 ,d 2 ,...,d NK ,μ,σ].

[0033] Based on one aspect, as a further limitation of the present invention, the frequency characteristic comparison module v of the model ω Including projection head and prediction head;

[0034] The frequency characteristic comparison module v ω The model is trained using labeled and unlabeled samples from the same domain; for each sample x, Mel-spectrogram features are extracted from the original audio and data augmentation techniques are applied to generate an enhanced spectrogram and enhanced spectrogram

[0035] Enhanced Spectrum and enhanced spectrogram Input to the model’s feature extractor f φ In the above example, we obtain the feature maps m∈R C×T×F and feature map m′∈R C×T×F , enhanced spectrum and enhanced spectrogram Pooling is performed along the time dimension T and the feature embedding e is obtained through the Reshape operation n ∈R F×C and feature embedding e n′ ∈R F×C ;

[0036] Extract feature maps of the same frequency dimension i and feature map Compare the similarity of the two spectrograms in the frequency dimension, and the extracted frequency features are mapped to the projection space by the projection head to obtain the projection vector and the projection vector Defined as:

[0037]

[0038] Use the prediction head to map the projection vector to the prediction space and get the prediction vector and the prediction vector

[0039]

[0040] Compute enhanced samples The prediction vector With another enhanced sample The projection vector The cosine distance between Among them, stopgrad(·) means that the model will not update the parameters according to the gradient obtained during the calculation of this tensor. The cosine distance calculation formula is:

[0041]

[0042] Symmetric loss for each sample pair is the sum of the distances on all frequency bands, expressed as:

[0043]

[0044] Based on one aspect, as a further limitation of the present invention, the classifier g of the model θ It consists of a linear layer and a softmax function.

[0045] Based on one aspect, as a further limitation of the present invention, the step three includes:

[0046] After fine-tuning, the feature extractor f in the fixed model φ , Environmental Characteristics Adaptation Module Classifier g θ Parameters and use test set samples Make predictions, where the query set The categories and support sets in The categories in the same random selection of multiple different support sets and queryset After the combination is fine-tuned and predicted, all prediction results are averaged as the final result.

[0047] On the other hand, the present invention further provides a few-sample underwater acoustic target recognition system based on unlabeled sample assistance. Based on the few-sample underwater acoustic target recognition method based on unlabeled sample assistance described in the above aspect, the few-sample underwater acoustic target recognition system includes:

[0048] An acquisition unit, wherein the acquisition unit acquires an unlabeled sample;

[0049] A processing unit, wherein unlabeled samples are input into the processing unit, a model in the processing unit predicts the unlabeled samples, and the processing unit outputs an underwater acoustic target recognition result.

[0050] The advantages of the present invention are:

[0051] 1. The present invention utilizes unlabeled samples in the target domain to enhance the fine-tuning performance of the model in the few-sample scenario.

[0052] 2. The present invention integrates the environmental information of unlabeled samples into the feature vector, and helps the model select features by comparing the features with the environmental information in the feature extraction stage. At the same time, the memory queue is combined to store and update the environmental feature vector, so as to better utilize the environmental information and enhance the adaptability of the model to the new environment.

[0053] 3. The frequency feature comparison module of the present invention compares the similarities between corresponding frequency bands in the enhanced view of unlabeled samples, which helps to reduce the interference of noise frequency components and enables the model to better understand the changes in different frequency bands, thereby learning the general feature representation of ship radiated noise.

[0054] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0056] Figure 1 :A flow chart of a method for underwater acoustic target recognition with few samples assisted by unlabeled samples provided by the present invention;

[0057] Figure 2 : Schematic diagram of a method for underwater acoustic target recognition with few samples assisted by unlabeled samples provided by the present invention;

[0058] Figure 3 : A diagram showing the calculation process of a method for underwater acoustic target recognition with a small number of samples assisted by unlabeled samples provided by the present invention;

[0059] Figure 4 : The data set division application table in the practical application provided by the present invention;

[0060] Figure 5 : A comparison data table of the present invention with the few-sample model in the same domain scenario;

[0061] Figure 6 : A comparison data table of the cross-domain scenario provided by the present invention and the few-sample model;

[0062] Figure 7 : A comparative data table of the number of unlabeled samples provided by the present invention;

[0063] Figure 8 : Comparative data table of ablation experiments provided by the present invention. DETAILED DESCRIPTION

[0064] Embodiments of the present invention are described in detail below. The embodiments are exemplary and intended to be used to explain the present invention, but should not be construed as limiting the present invention.

[0065] See also Figure 1 The embodiment of the present invention provides a method for underwater acoustic target recognition based on a small number of samples assisted by unlabeled samples, comprising the following steps:

[0066] Step 1: Training model: Obtain a large-scale base data set, pre-train the model on the large-scale base data set, and complete the update of network parameters during the pre-training model. The model includes: feature extractor f φ , Environmental Characteristics Adaptation Module Frequency feature comparison module v ω and classifier g θ .

[0067] See also Figure 2 In the above step 1 of the embodiment of the present invention, the model is pre-trained on a large-scale base data set, including:

[0068] In the base dataset The model is pre-trained in this way, the Mel-spectrogram features are extracted from the original audio, and the Mel-spectrogram is enhanced by applying data enhancement techniques (including SpecAugment and time-domain Gaussian interference); data enhancement techniques increase the robustness of the model and provide a basis for subsequent v ω Generate positive sample pairs.

[0069] Please continue reading Figure 2 In the above step 1 of the embodiment of the present invention, the enhanced Mel frequency spectrum is used as the input of the model, and the feature extractor f composed of ResNet12 in the model φ Extract effective feature representation; input the effective feature representation into the environmental feature adaptation module of the model And frequency feature comparison module v ω In the model, the classifier g θ Adapting environmental features to the module The output feature vectors are divided into different categories, and the frequency feature comparison module v ωThe similarity of the two enhanced views of the effective feature representations in the corresponding frequency bands is compared.

[0070] In the above step 1 of the embodiment of the present invention, the updating of network parameters is completed during the pre-training model, including: jointly optimizing the cross entropy loss Sum band contrast loss The model parameters are updated during the pre-training process; when updating, the frequency band contrast loss Previously, the hyperparameter α was introduced, and its formula is:

[0071] Step 2: Fine-tune the model: Use the network parameters as the initial parameters and use the new class data set to concentrate the support set Fine-tune the model using labeled samples from y and unlabeled samples from the current domain.

[0072] In the above step 2 of the embodiment of the present invention, using network parameters as initial parameters includes:

[0073] The feature extractor f in the pre-trained model φ , Environmental Characteristics Adaptation Module And frequency feature comparison module v ω The parameters of are transferred to the model in the fine-tuning stage, and the classifier g is randomly initialized. θ Parameters.

[0074] Please continue to read Figure 2 In the fine-tuning stage, the above model adapts to the environment feature module Includes memory queue and attention pooling layer, where: support set The label sample in is denoted as d 1 ,d 2 ,...,d NK , where NK represents the support set The number of samples in the current domain; the unlabeled samples in the current domain are represented as x 1 ,x 2 ,...,x k , encoded into a vector e 1 ,e 2 ,..,e k , k represents the number of unlabeled samples in a batch.

[0075] Environmental feature adaptation module of the embodiment of the present invention Will support set The feature vectors of the labeled samples and the feature vectors of a batch of unlabeled samples are taken as input, the two feature vectors are concatenated and input into the memory queue, and the same number of feature vectors are removed from the memory queue to complete the update of the memory queue.

[0076] For further information, please refer to Figure 2 In the fine-tuning stage of the above model, the embodiment of the present invention outputs the first N mq vectors as the input of the attention pooling layer, where N mq is the number of feature vectors stored in the memory queue; the attention pooling layer is N through two fully connected layers mq The scalar score α is calculated for each encoding vector in the vector i,c : In the formula, e i represents the feature representation of the i-th sample in the queue, W represents the first parameter of the first linear layer, b represents the second parameter of the first linear layer, and the information is projected into a smaller R-dimensional representation. f(·) represents the nonlinear activation function. After f(·), the information is activated by and k c Parameterized linear layer, get α i,c , where α i,c represents the attention score of the i-th sample in feature dimension c.

[0077] For details, please continue to see Figure 2 In the fine-tuning stage of the above model, the embodiment of the present invention applies the softmax function to calculate the normalized score of each sample in the feature dimension c:

[0078]

[0079] In the formula, α′ i,c Indicates the importance of each sample in the feature dimension c, and calculates the weighted average μ within the feature dimension c c and weighted standard deviation σ c , the calculation formula is:

[0080]

[0081] The embodiment of the present invention uses the weighted average value μ c and weighted standard deviation σ c Calculate the weighted average μ of each feature dimension c c and weighted standard deviation σ c , the obtained weighted average μ c and weighted standard deviation σ c Connect along the feature dimension c to form the final weighted mean vector μ and weighted standard deviation vector σ; The eigenvector d 1 ,d 2 ,...,d NK It is connected with the mean μ and standard deviation σ of the environmental information to form a vector e′, which is expressed as e′=[d 1 ,d 2 ,...,dNK ,μ,σ].

[0082] Among them, the frequency characteristic comparison module v of the above model in the embodiment of the present invention ω Including projection head and prediction head; frequency feature comparison module v ω The model is trained using labeled and unlabeled samples from the same domain; for each sample x, Mel-spectrogram features are extracted from the original audio and data augmentation techniques are applied to generate an enhanced spectrogram and enhanced spectrogram Enhanced Spectrum and enhanced spectrogram Input to the model’s feature extractor f φ In the above example, we obtain the feature maps m∈R C×T×F and feature map m′∈R C×T×F , enhanced spectrum and enhanced spectrogram Pooling is performed along the time dimension T and the feature embedding e is obtained through the Reshape operation n ∈R F×C and feature embedding e n′ ∈R F×C ; Extract feature maps of the same frequency dimension i and feature map Compare the similarity of the two spectrograms in the frequency dimension, and the extracted frequency features are mapped to the projection space by the projection head to obtain the projection vector and the projection vector Defined as:

[0083]

[0084] Use the prediction head to map the projection vector to the prediction space and get the prediction vector and the prediction vector

[0085]

[0086] The embodiment of the present invention calculates the enhanced sample The prediction vector With another enhanced sample The projection vector The cosine distance between Among them, stopgrad(·) means that the model will not update the parameters according to the gradient obtained during the calculation of this tensor. The cosine distance calculation formula is:

[0087]

[0088] Symmetric loss for each sample pair is the sum of the distances on all frequency bands, expressed as:

[0089]

[0090] Among them, the classifier g of the above model in the embodiment of the present invention θ It consists of a linear layer and a softmax function.

[0091] See also Figure 3 In the above step 2 of the embodiment of the present invention, the new class data set is used to support the set Fine-tune the model using labeled samples from and unlabeled samples from the current domain, including:

[0092] In the embodiment of the present invention, f in the pre-training model φ , and v ω The parameters of g are migrated to the model in the fine-tuning stage, and g is randomly initialized. θ The parameters of . Unlabeled samples in the current domain are introduced to make full use of them. The enhanced mel spectrograms of labeled samples and unlabeled samples are input into f φ To obtain their respective feature representations. Input these feature representations into and v ω , use and g θ Calculate, and Using v ω Computation,The two losses jointly supervise the fine-tuning process, enabling the model to better adapt to the target domain.

[0093] The embodiment of the present invention will support the collection The labeled samples in the current domain and the unlabeled samples in the current domain first extract the Mel spectrum features, and then obtain the enhanced Mel spectrum through data enhancement. The enhanced Mel spectrum is input into the environmental feature adaptation module. And frequency feature comparison module v ω .

[0094] The embodiment of the present invention uses an environmental feature adaptation module and classifier g θ Calculating cross entropy loss Use frequency feature comparison module v ω Calculate the band contrast loss Cross Entropy Loss Sum band contrast loss Jointly supervise the fine-tuning process to make the model better adapt to the target domain.

[0095] Step 3: Use the model: Use the feature extractor f of the fine-tuned model φ , Environmental Characteristics Adaptation Module and classifier g θ , query the new class data set The samples in are predicted to obtain the underwater acoustic target recognition results.

[0096] The above step 3 of the embodiment of the present invention includes: after the fine-tuning is completed, the feature extractor f in the fixed model is fixed. φ , Environmental Characteristics Adaptation Module Classifier g θ Parameters and use test set samples Make predictions, where the query set The categories and support sets in The categories in the same random selection of multiple different support sets and queryset After the combination is fine-tuned and predicted, all prediction results are averaged as the final result.

[0097] Based on the above-mentioned method for identifying a few-sample hydroacoustic target with the assistance of unlabeled samples, an embodiment of the present invention also provides a system for identifying a few-sample hydroacoustic target with the assistance of unlabeled samples. The system for identifying a few-sample hydroacoustic target includes an acquisition unit and a processing unit. The acquisition unit acquires unlabeled samples; the unlabeled samples are input into the processing unit, the model in the processing unit predicts the unlabeled samples, and the processing unit outputs the hydroacoustic target recognition results.

[0098] Please continue reading Figure 2 The embodiments of the present invention are verified using three public ship radiation noise datasets, ShipsEar, DeepShip and DanShip. The ShipsEar dataset contains 11 different types of ships, the DeepShip dataset contains the real underwater sounds of 265 different ships, which can be divided into four categories: tankers, tugboats, passenger ships and cargo ships; DanShip contains the sounds of three types of ships: fishing boats, cargo ships and speedboats. The ships travel along different routes at different speeds.

[0099] The audio in the three data sets of the embodiment of the present invention is resampled to 16kHz and divided into 2-second segments. When extracting the Mel-spectrogram features, the window length is set to 40ms and the window shift is set to 20ms. During fine-tuning, 100 unlabeled samples are randomly selected from the target domain for auxiliary training. During testing, 15 samples are taken from each class for prediction. In addition, N mq The capacity is set to 100, and the loss function The hyperparameter α in is set to 1, and the AdamW optimizer is used during model training.

[0100] The embodiments of the present invention complete experiments in the same-domain and cross-domain scenarios. For the same-domain scenario, multiple categories of ShipsEar are manually divided into two subsets, namely the base dataset and the And new class dataset exist Pre-training is done on Fine-tune on the basis of the classification. For this partitioning scheme, it is crucial to carefully consider the division of different categories. In this embodiment of the present invention, six categories are selected from ShipsEar as the base data set, and the remaining five categories are assigned to the new class data set. There are two main points to consider when making the partition. First, the categories with a large number of samples are assigned to the base data set to ensure that the model has sufficient data support to learn excellent feature representations during pre-training. Second, the similarity between samples in different subsets should be minimized as much as possible to ensure the difficulty of the few-sample task; please refer to Figure 4 , manually select three partitioning methods.

[0101] In order to ensure the reliability of the prediction, Figure 4 For each partitioning method listed in , 50 combinations of support sets and query sets are randomly selected. Then the 50 sets of experimental results are averaged to obtain the prediction results. Finally, the results under the three partitions are averaged as the final result of the model. The experimental results are comprehensively analyzed based on the accuracy (i.e., ACC), F1 score, and area under the receiver operating characteristic curve (i.e., AUC) evaluation indicators. For cross-domain scenarios, 11 ship categories are first used in ShipsEar for pre-training, and then DeepShip and DanShip datasets are used for fine-tuning, respectively. Due to the limitations of sample types in the dataset, DeepShip is a four-classification task, while DanShip is a three-classification task.

[0102] The model is pre-trained on a large-scale base data set to update the network parameters; wherein the model includes a feature extractor f φ , Environmental Characteristics Adaptation Module Frequency feature comparison module v ω and classifier g θ Extract Mel-spectrogram features from the original audio and apply data augmentation techniques, including SpecAugment and time-domain Gaussian interference, to preprocess the Mel-spectrogram. These data augmentation techniques increase the robustness of the model and provide a basis for subsequent v ω Generate positive sample pairs. The enhanced Mel-spectrogram is used as the input of the model, and the feature extractor f composed of ResNet12 φ Extract effective feature representations. Then input these feature representations into and v ω In. g θ Will The output feature vectors are divided into different classes. ω The similarity of two enhanced views of a sample in corresponding frequency bands is compared. The model parameters are updated by jointly optimizing the cross entropy loss Sum band contrast loss Before the band contrast loss, a hyperparameter α is introduced:

[0103] Use the parameters of the pre-training model as the initial parameters and use the support set The labeled samples of the pre-trained model and the unlabeled samples of the current domain are used to fine-tune the model. φ , and v ω The parameters of g are migrated to the model in the fine-tuning stage, and g is randomly initialized. θ The parameters of . Unlabeled samples in the current domain are introduced to make full use of them. The enhanced mel spectrograms of labeled samples and unlabeled samples are input into f φ To obtain their respective feature representations. Then input these feature representations into and v ω . use and g θ Calculate, and Using v ω Computation,The two losses jointly supervise the fine-tuning process, enabling the model to better adapt to the target domain.

[0104] Use the fine-tuned feature extractor f φ , Environmental Characteristics Adaptation Module and classifier g θ For query set Specifically, the fixed model f φ , g θ The parameters and the test set samples Make predictions, where The categories in The same category is in and After the combination is fine-tuned and predicted, all prediction results are averaged as the final result.

[0105] For verification of the technical effect of the embodiment of the present invention, please refer to Figure 5 and Figure 6 , Figure 5In order to compare the method of the present invention with advanced methods in few-shot classification tasks in the same domain scenario, EFFC is compared with other few-shot models, including RelationNet, ProtoNet, RFS, LabelHallu and CDCF, in 1-shot, 3-shot and 5-shot tasks respectively. In the 1-shot task, the recognition performance of all models is limited due to the scarcity of fine-tuning samples. As the number of samples increases, the model performance is significantly improved. In both 3-shot and 5-shot tasks, EFFC achieves the best performance. The experiment verifies that the embodiment of the present invention can use unlabeled samples to learn richer acoustic feature representations and achieve excellent performance.

[0106] Figure 6 For comparison with the few-shot model in cross-domain scenarios, EFFC achieved the highest accuracy in all tasks on the DeepShip dataset. In particular, in the 5-shot task, EFFC achieved an accuracy of 85.67%, which is significantly better than other few-shot learning models. This shows that EFFC exhibits strong generalization ability on cross-domain fine-tuning datasets and can better adapt to new domains and targets. EFFC also achieved the highest accuracy on the DanShip dataset, further verifying the excellent performance of EFFC on cross-domain datasets and proving its effective target classification ability. The embodiments of the present invention performed well in a variety of cross-domain scenarios, demonstrating its ability to use unlabeled samples for environmental feature adaptation and comparative self-supervised learning to adapt to new environments.

[0107] The present invention conducts comparative experiments to evaluate the impact of different numbers of unlabeled samples on model performance, and the corresponding results are recorded in Figure 7 Specifically, in the ShipsEar dataset, increasing the number of unlabeled samples can improve the accuracy of EFFC, but when the number of unlabeled samples exceeds 100, the accuracy improvement begins to decrease, with an increase of less than 1%. Adding more unlabeled samples does not significantly improve the performance of the model. In the cross-domain scenarios of the DeepShip and DanShip datasets, increasing the number of unlabeled samples also has a positive impact on the performance of EFFC, especially in DanShip, when the number of unlabeled samples reaches 200, EFFC achieves the best performance in all tasks. However, the performance of EFFC when the number of unlabeled samples is 100 is close to that when the number is 200, indicating that continuing to increase the number of unlabeled samples after saturation does not bring obvious performance improvement.

[0108] Increasing the number of unlabeled samples in the embodiments of the present invention has a certain impact on the performance of EFFC in both the same-domain and cross-domain scenarios. As the number of unlabeled samples increases, the accuracy of EFFC shows an overall upward trend, because more unlabeled samples can provide additional data information to help the model better learn data distribution and feature representation. However, when the number of unlabeled samples exceeds a certain threshold, further increasing the sample size has limited impact on model performance. In addition, the optimal number of unlabeled samples may vary for different data sets and tasks.

[0109] The present invention conducts ablation experiments to clarify the impact of different parts of the present invention on model performance. First, the impact of using a memory queue in the environment feature adaptation module (EF) on model performance is compared. Specifically, without using a memory queue, The feature vectors of the samples in the batch are combined with the feature vectors of the unlabeled samples in each batch for attention pooling. Secondly, each module is gradually integrated into the model and comparative experiments are conducted. Specifically, the effects of the EF module, the frequency feature contrast module (FC), and the global contrast module (GC) on the experimental results under 1-shot, 3-shot, and 5-shot tasks are evaluated. The GC module refers to a two-dimensional pooling operation on the feature tensor calculated by the feature extractor during the contrastive learning process to obtain a feature embedding with a dimension of C for similarity comparison. This module aims to compare and explore the effectiveness of contrasting similarities between different frequency bands in the FC module. The embodiments of the present invention experimented with different combinations under the DeepShip dataset, and the corresponding results are recorded in Figure 8 middle.

[0110] Specifically, compared with the baseline, when the EF module runs without the memory queue, it shows some improvement in 1-shot, 3-shot, and 5-shot tasks. This shows that attention pooling can utilize the unlabeled samples in the current domain to fine-tune the model. The length of the memory queue can exceed the batch size, enabling the model to adapt to the current environment by recording the information of the unlabeled samples in the queue and utilizing attention pooling. However, due to the differences in sample quality, caution should be exercised when using the information of the unlabeled samples in the queue, and over-reliance on the feature vectors stored in the current queue should be avoided. Continuously fine-tuning the model while updating the memory queue helps to alleviate overfitting to a certain extent and ensure that the weighted statistics better reflect the current environment information. On this basis, integrating and updating the memory queue in the EF module further enhances the recognition performance of the model. Although the GC module optimizes the accuracy of the model to a certain extent, the improvement effect is limited. Due to the unique characteristics of underwater acoustic signals, information in different frequency bands may have different importance. Pooling the features of all frequency bands will lead to information loss or confusion, thereby reducing the corresponding performance. In all tasks, the FC module significantly improves the performance of the model, indicating that it is effective to independently calculate the contrast similarity of different frequency bands. Finally, the combination of EF and FC modules (EF+FC) provides the best experimental results, further verifying the complementary effects of EF and FC modules. The experimental results show that by introducing EF and FC modules, the embodiment of the present invention achieves a significant performance improvement in the underwater acoustic target recognition task.

[0111] The few-sample underwater acoustic target recognition method and system disclosed in the embodiment of the present invention utilizes unlabeled samples in the target domain to enhance the fine-tuning performance of the model in the few-sample scenario. The embodiment of the present invention integrates the environmental information of the unlabeled samples into the feature vector; the module helps the model select features by comparing the features with the environmental information in the feature extraction stage. At the same time, a memory queue is also combined to store and update the environmental feature vector, so that the environmental information can be better utilized and the adaptability of the model to the new environment can be enhanced.

[0112] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should be included in the protection scope of the present invention.

Claims

1. A method for underwater acoustic target recognition based on few samples assisted by unlabeled samples, characterized in that: include: Step 1: training model: obtaining a large-scale base data set, pre-training the model on the large-scale base data set, and completing the update of network parameters during the pre-training model, wherein: the model includes: feature extractor f φ , Environmental Characteristics Adaptation Module Frequency feature comparison module v ω and classifier g θ ; Step 2: Fine-tune the model: Use the network parameters as initial parameters and use the new class data set to concentrate the support set Fine-tune the model using labeled samples in and unlabeled samples of the current domain; Step 3: Use the model: Use the feature extractor f of the model after fine-tuning φ , Environmental Characteristics Adaptation Module and classifier g θ , query the new class data set The samples in are predicted to obtain the underwater acoustic target recognition results.

2. The method for underwater acoustic target recognition based on few samples assisted by unlabeled samples according to claim 1 is characterized in that: In the step 1, pre-training the model on the large-scale base data set includes: In the base dataset The model is pre-trained, Mel-spectrogram features are extracted from the original audio, and the Mel-spectrogram is enhanced by applying data augmentation techniques; The enhanced Mel-spectrogram is used as the input of the model, and the model’s feature extractor f φ Extract effective feature representation; Input the effective feature representation to the environment feature adaptation module of the model And frequency feature comparison module v ω In the model, the classifier g θ Adapting environmental features to the module The output feature vectors are divided into different categories, and the frequency feature comparison module v ω comparing the similarity of two enhanced views of effective feature representation in corresponding frequency bands; In the step 1, the updating of network parameters is completed in the pre-training model process, including: By jointly optimizing the cross entropy loss Sum band contrast loss Complete the update of model parameters during pre-training; When updating, the contrast loss in the frequency band Previously, the hyperparameter α was introduced, and its formula is:

3. The method for underwater acoustic target recognition based on few samples assisted by unlabeled samples according to claim 1 is characterized in that: In the step 2, using the network parameters as initial parameters includes: The feature extractor f in the pre-trained model φ , Environmental Characteristics Adaptation Module And frequency feature comparison module v ω The parameters of are transferred to the model in the fine-tuning stage, and the classifier g is randomly initialized. θ Parameters; In step 2, the new class data set is used to support the Fine-tune the model using labeled samples from the domain and unlabeled samples from the current domain, including: Will support set The labeled samples in the current domain and the unlabeled samples in the current domain are firstly extracted to obtain the Mel-spectrogram features, and then the enhanced Mel-spectrogram is obtained by data enhancement. The enhanced Mel-spectrogram is input into the environmental feature adaptation module. And frequency feature comparison module v ω ; Use the environmental feature adaptation module and classifier g θ Calculating cross entropy loss Use frequency feature comparison module v ω Calculate the band contrast loss Cross Entropy Loss Sum band contrast loss Jointly supervise the fine-tuning process to make the model better adapt to the target domain.

4. The method for underwater acoustic target recognition based on few samples assisted by unlabeled samples according to claim 1 is characterized in that: In the step 2, the environmental feature adaptation module Includes memory queue and attention pooling layer, where: support set The labeled samples in are denoted as d1,d2,...,d NK , where NK represents the support set The number of samples in the current domain; the unlabeled samples in the current domain are represented as x1,x2,...,x k , encoded into vectors e1,e2,..,e k , k represents the number of unlabeled samples in a batch; Environmental characteristics adaptation module Will support set The feature vectors of the labeled samples and the feature vectors of a batch of unlabeled samples are taken as input, the two feature vectors are concatenated and input into the memory queue, and the feature vectors of the same number are removed from the memory queue to complete the update of the memory queue; Output first N mq vectors as the input of the attention pooling layer, where N mq is the number of feature vectors stored in the memory queue; the attention pooling layer is N through two fully connected layers mq Compute the scalar score for each encoded vector in the vector In the formula, e i represents the feature representation of the i-th sample in the queue, W represents the first parameter of the first linear layer, b represents the second parameter of the first linear layer, and the information is projected into a smaller R-dimensional representation. f(·) represents the nonlinear activation function. After f(·), the information is activated by and k c The parameterized linear layer gets α i,c , where α i,c represents the attention score of the i-th sample in the feature dimension c; Apply the softmax function to calculate the normalized score of each sample in the feature dimension c: In the formula, α i ' ,c Indicates the importance of each sample in the feature dimension c, and calculates the weighted average μ within the feature dimension c c and weighted standard deviation σ c , the calculation formula is: Using the weighted average μ c and weighted standard deviation σ c Calculate the weighted average μ of each feature dimension c c and weighted standard deviation σ c , the obtained weighted average μ c and weighted standard deviation σ c Connect along the feature dimension c to form the final weighted mean vector μ and weighted standard deviation vector σ; Will come from the support set The eigenvectors d1,d2,...,d NK It is connected with the mean μ and standard deviation σ of the environmental information to form a vector e′, which is expressed as e′=[d1,d2,...,d NK ,μ,σ].

5. The method for underwater acoustic target recognition based on few samples assisted by unlabeled samples according to claim 1 is characterized in that: The frequency feature comparison module v of the model ω Including projection head and prediction head; The frequency characteristic comparison module v ω Train the model using labeled and unlabeled samples from the same domain; For each sample x, extract the Mel spectrogram features from the original audio and apply data augmentation techniques to generate an enhanced spectrogram and enhanced spectrogram Enhanced Spectrum and enhanced spectrogram Input to the model’s feature extractor f φ In the above example, we obtain the feature maps m∈R C×T×F and feature map m′∈R C×T×F , enhanced spectrum and enhanced spectrogram Pooling is performed along the time dimension T and the feature embedding e is obtained through the Reshape operation n ∈R F×C and feature embedding e n′ ∈R F×C ; Extract feature maps of the same frequency dimension i and feature map Compare the similarity of the two spectrograms in the frequency dimension, and the extracted frequency features are mapped to the projection space by the projection head to obtain the projection vector z ni and the projection vector Defined as: Use the prediction head to map the projection vector to the prediction space and get the prediction vector and the prediction vector Compute enhanced samples The prediction vector With another enhanced sample The projection vector The cosine distance between Among them, stopgrad(·) means that the model will not update the parameters according to the gradient obtained during the calculation of this tensor. The cosine distance calculation formula is: Symmetric loss for each sample pair is the sum of the distances on all frequency bands, expressed as:

6. The method for underwater acoustic target recognition based on few samples assisted by unlabeled samples according to claim 1 is characterized in that: The classifier g of the model θ It consists of a linear layer and a softmax function.

7. The method for underwater acoustic target recognition with few samples assisted by unlabeled samples according to claim 1 is characterized in that: The step three comprises: After fine-tuning, the feature extractor f in the fixed model φ , Environmental Characteristics Adaptation Module Classifier g θ Parameters and use test set samples Make predictions, where the query set The categories and support sets in The categories in the same random selection of multiple different support sets and queryset After the combination is fine-tuned and predicted, all prediction results are averaged as the final result.

8. A small number of samples underwater acoustic target recognition system based on unlabeled sample assistance, based on a small number of samples underwater acoustic target recognition method based on unlabeled sample assistance according to any one of claims 1 to 7, characterized in that: The few-sample underwater acoustic target recognition system includes: An acquisition unit, wherein the acquisition unit acquires an unlabeled sample; A processing unit, wherein unlabeled samples are input into the processing unit, a model in the processing unit predicts the unlabeled samples, and the processing unit outputs a result of underwater acoustic target recognition.

Images