A few-shot underwater acoustic target recognition method based on sample mapping

Abstract

The application provides a few-shot underwater acoustic target recognition method based on sample mapping; comprising: step 1, training a feature extraction module on the base dataset for classification task to extract features of new classes; step 2, using the trained feature extraction module to extract features of the samples of new classes, and calculating the average features of each class, mapping the samples on the base dataset into the type space composed of new classes according to the similarity of the center features of each class to each sample, and labeling the pseudo labels; step 3, using the base dataset samples with pseudo labels and support set samples to fine-tune the model to fit the feature recognition of new classes. The method has higher accuracy compared with other models when detecting underwater targets in the case of few samples, and can also obtain good results in the case of adding noise.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of underwater acoustic targets, and particularly relates to a few-sample underwater acoustic target recognition method based on sample mapping. BACKGROUND

[0002] With the increasingly prominent strategic position of the ocean, developing and utilizing marine resources and space has become an important issue. However, it is difficult to obtain information underwater, and sound waves are the only form of energy that can propagate over long distances in the ocean. Since the 1960s, countries around the world have been investing heavily in research on underwater acoustic target recognition. In view of the demand for underwater detection in modern warfare, underwater target classification and recognition based on sound signals has become a research hotspot in the field of underwater detection.

[0003] Due to the advantages of passive sonar, such as strong concealment and long transmission distance, underwater target radiation noise collected by passive sonar is the main source of information for underwater acoustic target recognition. Passive sonar systems have good concealment and flexibility, so how to use the ship radiation noise obtained by passive sonar for underwater target detection and recognition has become the key to solving the problem. However, due to the application of various ship stealth technologies and the complexity of the marine environment, underwater target recognition based on ship radiation noise faces great challenges.

[0004] Traditional target recognition methods mainly use a certain method to extract features from the original signal based on the experience of researchers. However, this method has the following disadvantages: slow speed, high energy consumption, limited by complex environment and prior knowledge, weak generalization ability, and due to the instability of human physiological and psychological factors, it is difficult to achieve stable all-weather underwater target detection and recognition. Machine learning methods are essentially nonlinear feature transformations of data, such as multilayer perception models, support vector machine (SVM) models, and Gaussian mixture models. This method is efficient when processing small batches of data, but its computational efficiency is significantly insufficient when the data volume increases. In addition, the parameters of shallow learning cannot be self-adaptive, so its self-correction ability is lacking.

[0005] In recent years, deep learning has been developing, and many end-to-end models with strong feature extraction capabilities have emerged, which are widely used in various fields, and more and more people apply them to underwater acoustic target recognition. The method of deep learning converts the original space of data into a feature space, and then classifies it. The deep learning theory overcomes the shortcomings of shallow learning, such as dependence on feature engineering and lack of self-adaptation. It can perform parameter self-adaptive learning, and can represent high-dimensional complex functions and extract deep features of the target, so it can obtain more useful information about the target. This method uses experimental data or public data sets as training samples for training, but in actual application, it is impossible to obtain a large number of labeled underwater acoustic samples in a specific environment, and the application scene tends to be less sample recognition.

[0006] At present, there is little research on the few-shot classification of underwater acoustic signals. Few-shot classification needs to use the recognition ability learned from a large base data set to recognize new classes that have not been encountered, and each class will provide a small number of labeled samples, i.e. support set. In this case, if the network is pre-trained using the base data set and then fine-tuned using a small number of labeled samples, it will cause serious overfitting due to the small number of labeled samples. At the same time, the network trained under the base data set freezes the feature extraction module and only trains a simple linear classifier, which cannot completely fit the features of the new class and will cause underfitting. SUMMARY

[0007] The purpose of the present application is to provide a few-shot underwater acoustic target recognition method based on sample mapping. Although the classes on the base data set and the support set are disjoint, sometimes the samples of different classes have certain similarity, and the base data set samples can be used for fine-tuning and auxiliary training during training. Therefore, the method of the present application maps the samples on the base data set with pseudo-labels about the classes of the support set, and then fine-tunes the model with the base data set samples with pseudo-labels and the support set samples.

[0008] The present application is realized by the following technical solutions:

[0009] The present application relates to a few-shot underwater acoustic target recognition method based on sample mapping, comprising the following steps:

[0010] Step 1: training a feature extraction module on the base data set for classification task to extract features of new classes;

[0011] Step 2: using the trained feature extraction module to extract features of the samples of the new class, and calculating the average features of each class, and mapping the samples on the base data set to the type space composed of the new class according to the similarity of the sample to the center feature of each class, and labeling them with pseudo-labels;

[0012] Step 3, fine-tune the model with both the base dataset samples with pseudo labels and the support set samples to fit the feature recognition of the new class.

[0013] Preferably, the specific steps of step 1 are:

[0014] The first goal is to train a model on a large dataset with good feature extraction ability, which will be used to migrate to the downstream few-shot recognition task. In the base dataset Next, train a convolutional neural network composed of a feature extractor and a classification layer g φ using the cross-entropy loss function to optimize the model. and φ base are the parameters of the optimized feature extractor and classifier, respectively. After the training is completed, the feature extractor will be used for the downstream few-shot recognition task, and the classifier will be discarded. The formula used in this step is shown in (I):

[0015]

[0016] Preferably, the specific steps of step 2 are:

[0017] In order to label pseudo samples for the base dataset according to the new class, the invention uses the feature vector output by the feature extractor to perform similarity comparison. Specifically, first fix the parameters of the feature extractor to extract feature vectors e for the samples of class i in the support set i If there are multiple samples for each class, take the average as the feature vector of the class, where x k is the sample belonging to class i in the support set, as shown in formula (II):

[0018]

[0019] After obtaining the mean value of each class, it will be used to calculate the similarity with the feature vectors of the base dataset and mapped to the type space of the new class as the pseudo label of the sample. First, calculate the cosine similarity between the feature e j and the feature vector e i of class i in the support set as the probability of the sample being classified into the new class i as shown in formula (III):

[0020]

[0021] The probability concatenation of the n new classes in the support set is used as the pseudo label of the base sample j in the type space of the new class As shown in formula (IV):

[0022]

[0023] Preferably, the specific steps of step 3 are:

[0024] The entire model (including and g φ ) for fine-tuning. The entire model is optimized using a weighted sum of two loss functions, where the loss function of the base dataset is KL divergence. The loss function of the support set is cross entropy loss

[0025] In the sample mapping phase, the pseudo labels of the new classes are applied to the samples in the base dataset. α and β are hyperparameters that weigh the weights of the two loss functions. is the parameter optimized for the feature extraction module in the pre-training phase and continues to be optimized. φ' is the parameter reinitialized for the classification layer. During the training process, SpecAugment is used to amplify the data samples, as shown in formula (V):

[0026]

[0027] After fine-tuning, the model Fix the parameters and perform the test set To make predictions, randomly select multiple different and After fine-tuning and predicting the combination, all prediction results are averaged as the final result.

[0028] The present invention has the following advantages:

[0029] The method of the present invention labels the samples in the base data set with pseudo-labels about the support set categories through sample mapping, and then fine-tunes the model together with the base data set samples with pseudo-labels and the support set samples; the method of the present invention has higher accuracy than other models when detecting underwater targets with few samples, and can also achieve good results under noise conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a preview diagram of the overview of the few-sample underwater acoustic target recognition method based on sample mapping involved in the present invention;

[0031] Figure 2A process diagram of a few-shot underwater acoustic target recognition method based on sample mapping is involved in the present application. DETAILED DESCRIPTION

[0032] The present application will be described in detail below with specific examples. It should be noted that the following examples are only further illustrations of the present application, and the protection scope of the present application is not limited to the following examples.

[0033] Embodiment

[0034] The present embodiment relates to a few-shot underwater acoustic target recognition method based on sample mapping, as shown in Figure 1 and Figure 2 , comprising the following steps:

[0035] The present embodiment uses a public data set ShipsEar for training, divides the 11-class ship hull noise data contained therein, selects 6 classes as the base data set, and fine-tunes the model with the other 5 classes. First, the audio is divided into 2s length samples, and then the mel spectrogram is extracted, with a window length of 40ms, a window shift of 20ms, and n_mels = 80. In the training of the model, SpecAugment is used for audio data augmentation. In order to facilitate comparison with the work in few-shot classification, the present embodiment selects ResNet12 as the base network.

[0036] Step 1, train a feature extraction module on the base data set to extract features of new classes;

[0037] The first goal is to train a model on a large data set with good feature extraction capability, which will be used to migrate to the downstream few-shot recognition task. In the base data set , a convolutional neural network composed of a feature extractor and a classification layer g φ is trained using a multi-class training method, and the cross-entropy loss function is used to optimize the model. and φ base are the parameters of the optimized feature extractor and classifier, respectively. After the training is completed, the feature extractor will be used for the downstream few-shot recognition task, and the classifier will be discarded. The formula used in this step is shown in (I):

[0038]

[0039] Step 2, use the trained feature extraction module to extract features of the new class samples, and calculate the average features of each class. According to the similarity of the center features of each class to the samples, map the samples on the base data set to the type space composed of new classes, and label them with pseudo labels.

[0040] To label pseudo samples from the base dataset according to the new classes, a feature extractor is used The output feature vectors are compared for similarity. Specifically, first fix the parameters of the feature extractor and extract feature vectors e for the samples in the support set i If there are multiple samples per class, take the average as the feature vector for the class, where x k is the sample belonging to class i in the support set, as shown in equation (II):

[0041]

[0042] After obtaining the mean for each class, the feature vectors are used to compute similarity and map into the type space of the new classes as pseudo labels for the samples. First, feature e j is compared to the feature vectors e i of class i in the support set to compute the cosine similarity as the probability that the sample is classified into the new class i as shown in equation (III):

[0043]

[0044] The probabilities for the n new classes in the support set are concatenated as the pseudo label for base sample j in the type space of the new classes as shown in equation (IV):

[0045]

[0046] Step 3, use the base dataset samples with pseudo labels and the support set samples together to fine-tune the model to fit the feature recognition of the new classes. Use batches containing the same number of base dataset samples and support set samples to fine-tune the entire model (including and g φ ). Use a weighted sum of two loss functions to optimize the entire model, where the loss function for the base dataset is the KL divergence and the loss function for the support set is the cross-entropy loss

[0047] The pseudo labels for the new classes for the samples in the base dataset are mapped during the sample mapping phase, and a and b are hyperparameters that weigh the weights of the two loss functions, are the parameters optimized from the pre-training stage for the feature extraction module and continue to be optimized, and φ' is the reinitialized parameter of the classification layer. During the training process, SpecAugment is used to sample the data for expansion, as shown in equation (V):

[0048]

[0049] After the fine-tuning is completed, the model The fixed parameters are used to predict the test set samples are randomly selected from multiple different and combinations are fine-tuned and predicted, and all the prediction results are averaged as the final result.

[0050] Technical effect verification

[0051] (1) Comparison with classical models

[0052] Table 1 is a comparison of the information of the method of the present application and advanced methods in the few-shot classification task. In the present application, RelationNet, ProtoNet, RFS, LabelHallu and mymodel are selected for comparison in 1-shot, 3-shot and 5-shot tasks. The effect of RelationNet based on the Metalearning method is poor because the number of sample categories in the base data set is less than that of the image task. ProtoNet and RFS have a greater improvement compared with RelationNet, which shows that the method of extracting a good feature vector for class comparison is feasible. ProtoNet performs poorly in the 1-shot task, which shows that the method of comparing the distance between the to-be-predicted sample and the support set class center point is difficult to accurately express the class center point feature when the number of samples in each class is small. LabelHallu has good performance because it adds sample hallucination based on the extracted feature vector, which alleviates the overfitting phenomenon of the model during fine-tuning. The model involved in the method of the present application has better performance in the three tasks, which shows that the method of using the feature space of the base data set to support the set for mapping to mark the pseudo sample has a more effective pseudo sample to the ability of the model fine-tuning. At the same time, compared with the prediction results of other models, the advantage of the method of the present application is more obvious with the increase of the shot number, which also shows that when the number of samples increases, the method of marking the pseudo sample is more accurate and effective, and the help of fine-tuning the model is greater.

[0053] Table 1

[0054]

[0055] (2) Comparison of different strategies in downstream stage

[0056] Table 2 is the comparison data of training the model using different strategies in the downstream stage, wherein the pre-training stage of the base data set is the same.

[0057] Frozen in Table 2 is a traditional transfer learning method, that is, a linear regression model is trained using a small number of samples to freeze the parameters of the backbone learned from the base data set;

[0058] Finetune is to fine-tune the entire model only using the samples in the support set; Hard-label is a variant of the method of the application, that is, when labeling the samples on the base data set, the one-hot encoding is used for labeling, and then it is trained with the support set when fine-tuning the model.

[0059] For the four different training strategies, comparison is made on 1-shot, 3-shot and 5-shot tasks. First, it can be found that the effect of Finetune is obviously better than that of Frozen, which shows that fine-tuning the classifier only will cause serious underfitting. By using the pseudo-label samples to participate in the downstream fine-tuning, the method of Hard-label has better effect than Frozen and Finetune.

[0060] Soft-label can better fit the similarity between different samples in feature mapping by using soft label for labeling, so it has the highest accuracy, and the performance is improved by about 5% compared with Finetune on 1-shot, 3-shot and 5-shot tasks.

[0061] Table 2

[0062]

[0063] (3) Ablation experiment of fine-tuning different parameters of the model by pseudo samples

[0064] The ablation experiment is performed in the embodiment of the application, and then it is clear that which part of the model is beneficial to the pseudo sample of the base dataset in the fine-tuning stage. As shown in Table 3, whether the pseudo sample updates the parameters of the classification layer and the feature extraction module of the model is mainly controlled for comparison. The experiment without gradient on the feature extraction module and the classification layer is equivalent to using only the support set to fine-tune the model, and the experiment with gradient on the two parts is our model. As shown in the experimental results, compared with the fine-tuned model, when only the classification layer is updated by the base dataset, the experimental results are roughly the same, and when only the feature extraction module is updated, the experimental results are improved by about 2%. The data show that the improvement of the method involved in the embodiment of the application is partly due to the learning of the backbone feature extraction module using the base dataset. When the base dataset is used to update the parameters of the feature extraction module and the classification layer at the same time, the experimental results are greatly improved, which shows that the method involved in the embodiment of the application can better suppress the overfitting phenomenon in the fine-tuning process.

[0065] Table 3

[0066]

[0067] (4) Comparison experiment under different signal-to-noise ratios

[0068] In order to explore the prediction ability of the model involved in the application in a high-noise environment, the original dataset is added with ocean noise according to different signal-to-noise ratios, and the training and prediction of the artificially added noise samples are compared. In the experiment, 5-shot tasks are selected for comparison. As shown in Table 4, RelationNet, ProtoNet, RFS, LabelHallu and the model involved in the method of the application are selected for comparison. For the performance of a single model, the prediction results at SNR = -10 dB and -15 dB are not much different, and the results at SNR = -5 dB are improved. The effect of RelationNet is poor, the effects of ProtoNet and RFS are general, and the effect of LabelHallu is improved, which is similar to the experimental results of Table 1 without adding noise. The prediction accuracy of the model involved in the method of the application is improved by about 3% compared with the better LabelHallu at SNR = -5 dB, -10 dB and -15 dB. For this, it shows that the model involved in the method of the application has certain anti-noise ability, especially the prediction ability at SNR = -5 dB is outstanding.

[0069] Table 4

[0070] SNR (dB) -5 -10 -15 RelationNet 0.2594 0.2272 0.2160 ProtoNet 0.5299 0.4877 0.4768 RFS 0.5133 0.5085 0.4809 LabelHallu 0.5304 0.4880 0.4797 mymodel 0.5664 0.5160 0.5019

[0071] The method of the present application marks the samples on the base data set with pseudo labels about the support set categories in a sample mapping manner, and then fine tunes the model with the base data set samples with pseudo labels and the support set samples; compared with other models, the method of the present application has higher accuracy when detecting the underwater target under the condition of few samples, and can also obtain good results under the condition of adding noise.

[0072] The specific embodiments of the present application are described above. It should be understood that the present application is not limited to the specific embodiments described above, and various modifications or changes can be made by those skilled in the art within the scope of the claims, which do not affect the essence of the present application.

Claims

1. A sample underwater acoustic target recognition method based on sample mapping, characterized in that: The following steps are involved: Step 1: Perform a classification task on the base dataset and train a feature extraction module to extract features of the new class. Step 2: Use the trained feature extraction module to extract features from samples of the new class. After obtaining the mean of each class, it will be used to calculate the similarity with the feature vector of the base dataset and mapped to the type space of the new class as the pseudo label of the sample; Step 3: Use pseudo-labeled base dataset samples and support set samples to fine-tune the model to fit the feature recognition of the new class; Among them, the support set samples and base data set samples are selected from the Shipsear data set. The base data set samples include 6 types of samples selected from the Shipsear data set; the support set samples are selected from the remaining 5 types of samples; the number of samples in each type of support set samples is 1, 3 or 5; The specific steps of step 1 are: In the base dataset Next, use the multi-classification training method to train a feature extractor and classification layer A convolutional neural network composed of Optimize the model; base and base are the parameters of the optimized feature extractor and classifier, respectively. After training, the feature extractor will be used for downstream sample recognition tasks, while the classifier will be discarded. The formula used in this step is shown in (I): （I）； The specific steps of step 2 are: fixed The parameter support set Upper category The sample extraction feature vector , if there are multiple samples for each class, the average value is taken as the feature vector of the class, where For the support set belonging to the class The sample is as shown in formula (II): (II); After obtaining the mean of each class, it will be used to compare with the feature vector of the base data set. Calculate the similarity and map it to the type space of the new class as the pseudo label of the sample. First, the feature Classes with support set The eigenvector of Calculate the cosine similarity as the sample is classified into the new class Probability , as shown in formula (III): (III); Support will be focused The probability of new classes is concatenated as the base sample Pseudo-labels in the type space of the new class , as shown in formula (IV): (IV); The specific steps of step 3 are: The entire model is optimized by weighted summation of two loss functions, where the loss function of the base data set is KL divergence. , the loss function of the support set is cross entropy loss ; The pseudo labels of the new classes are given to the samples in the base dataset during the sample mapping phase. and is a hyperparameter that weighs the weights of the two loss functions. are the parameters optimized for the feature extraction module during the pre-training phase. is the parameter for reinitializing the classification layer, and SpecAugment is used to amplify the data, as shown in formula (V): （V） After fine-tuning, the model Fix the parameters and perform the test set To make predictions, randomly select multiple different and After fine-tuning and predicting the combination, all prediction results are averaged as the final result.

Images