Few-sample underwater acoustic target recognition method based on weighted hybrid generation and adversarial domain adaptation

Abstract

The invention discloses a few-sample underwater acoustic target recognition method based on weighted mixing generation and adversarial domain adaptation. The method comprises the following steps: step 1, carrying out category balance operation based on weighted mixing on an original signal; 2, extracting a Mel power spectrum of the underwater acoustic signal through short-time Fourier transform and Mel filtering; step 3, performing further enhancement operations such as masking on the obtained Mel power spectrum so as to improve the generalization ability; 4, performing feature extraction on information in the frequency spectrum by using an encoder based on a convolutional neural network; step 5; classifying the extracted features by using a linear classifier, and calculating loss based on a classification result for training; step 6, performing feature extraction on the unlabeled data and the training data of the target domain by using the pre-trained convolutional neural network; step 7, trying to distinguish data sources by using a discriminator realized based on a multi-layer perceptron; 8, calculating the loss of an output result of the discriminator, and carrying out the adversarial domain adaptation fine tuning of the pre-trained convolutional neural network according to the result; 9, carrying out the test verification through employing the fine-tuned convolution to cooperate with a pre-trained linear classifier; and step 10, storing the trained model and performing deployment.

Description

A method for few-shot underwater acoustic target recognition based on weighted hybrid generation and adversarial domain adaptation Technical Field

[0001] This invention relates to a method for identifying underwater acoustic targets with few samples, and more particularly to a method for identifying underwater acoustic targets with few samples based on weighted hybrid generation and adversarial domain adaptation. Background Technology

[0002] Underwater acoustic identification is a technology that utilizes acoustic signal processing and artificial intelligence to extract, analyze, and classify the acoustic characteristics of underwater targets such as submarines, unmanned underwater vehicles, and marine life. It is a core component of the marine sensing system. In the underwater environment, due to severe electromagnetic wave attenuation and weak visible light penetration, sound waves have become a crucial means of long-distance underwater information transmission and target detection. Its importance is not only reflected in civilian fields such as marine resource development and marine environmental monitoring, but also occupies an irreplaceable strategic position in the field of national defense and security, making it an indispensable technology for ensuring the safety of my country's long coastline.

[0003] Underwater acoustic signal processing technology has developed to a certain level over the years. Underwater acoustic signals are very different from traditional time series analysis. Underwater acoustic signals are often very long sequences, and direct analysis of them is not only computationally intensive but also ineffective. Therefore, researchers usually extract the corresponding spectrum from the sound signal and then conduct further analysis around the extracted spectrum. The specific analysis techniques are mainly divided into two categories: methods based on traditional machine learning and methods based on deep learning.

[0004] Traditional machine learning methods were early approaches for processing underwater acoustic signals. One such method (see: Y. Tong, X. Zhang and Y. Ge, "Classification and Recognition of Underwater Target Based on MFCC Feature Extraction," 2020 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), Macau, China, 2020, pp. 1-4) first extracts the underwater acoustic signal of the target into its corresponding Mel spectrum, and then uses a K-Nearest Neighbors (KNN) classifier model for prediction and classification. The KNN classifier is a simple and effective supervised learning algorithm. In classification tasks, it classifies based on the class of its nearest neighbor training samples in the feature space. Specifically, for a given unknown sample, the KNN classifier determines its class by calculating the classes of its N nearest neighbor training samples. This method is fast and effective, and because the classification result of the KNN classifier is determined by the distance between the training data and the test data, as well as the specific label, it does not require an explicit training process. However, K-nearest neighbor (KNN) classifiers often require a large amount of data and have high requirements for data quality. If the training set data is insufficient, or if there is a significant difference in distribution between the training and test sets, the performance of the KNN classifier will be unsatisfactory. Traditional machine learning classification models such as decision trees (DT) or linear classifiers (LC) also have similar effects. These methods are characterized by fast computational efficiency but weak learning ability and are highly dependent on data quality and the effectiveness of feature extraction.

[0005] With the breakthrough progress of deep learning based on neural networks in the field of speech recognition technology (Reference: Yang Lufei, Zhang Xinhua, Wu Bingkun, Li Lanrui. Deep learning classification method for passive underwater acoustic targets based on MFCC features [J]. Ship Science and Technology, 2020, 42(10)), a simple replacement of the K-nearest neighbor classifier model with a neural network was considered. This method can indeed achieve very good results for ordinary datasets. However, on some highly imbalanced datasets with a single data distribution and a large difference between the training set and the test set, this method is still difficult to achieve good results.

[0006] In reality, due to cost and other special factors, underwater acoustic recognition, especially for ships, cannot collect large amounts of target data anytime and anywhere; that is, the number of target-containing samples is very small. Furthermore, the sea area where the target appears may be completely different each time, meaning that the environmental noise during actual use may differ significantly from that at the time of data collection. These factors make underwater acoustic-based ship identification tasks extremely difficult. Summary of the Invention

[0007] Purpose of the invention: The technical problem to be solved by the present invention is that the generalization ability of the model is insufficient due to the lack of target samples and the large differences in marine environmental noise in actual use scenarios. To this end, a specially designed sample mixing generation method and a supporting domain adaptation fine-tuning system are provided.

[0008] To address the aforementioned technical problems, this invention discloses a few-sample underwater acoustic target recognition method based on weighted hybrid generation and adversarial domain adaptation, comprising the following steps:

[0009] Step 1, train the underwater acoustic dataset D, which has a high class imbalance. ori Let D be the data containing the target and the data not containing the target. target and D no_target , give D target D with higher weight and lower weight no_target Randomly mix to generate new target samples D′ ori ;

[0010] Step 2, for the balanced training dataset D′ ori Frequency domain feature transformation is performed, and a linear power spectrum is obtained based on short-time Fourier transform. This power spectrum is then mapped through a Mel filter bank to a Mel power spectrum that conforms to auditory perception characteristics.

[0011] Step 3: Analyze the extracted Mel power spectrum. Perform zero-mean normalization, and then obtain the result through data augmentation operations such as masking and translation. This is to prevent overfitting and improve the generalization ability of subsequent models;

[0012] Step 4: Utilize the feature encoder f implemented based on a convolutional neural network. CNN Enhanced Mel power spectrum Feature extraction is performed to obtain the feature encoding v of the Mel power spectrum. i ;

[0013] Step 5, using a linear classifier f C For the extracted features v i Classify the data and calculate the loss against f using the binary cross-entropy loss BCELoss. CNN and fC Perform pre-training;

[0014] Step 6: Utilize the convolutional neural network f trained for classification CNN Extract the source domain D′ for training respectively ori Data characteristics and the target domain data D to be deployed subsequently new Features Let the set of the two be...

[0015] Step 7 ， Create a new multilayer perceptron f MLP As a discriminator, it attempts to distinguish whether the input encoding v comes from the source domain or the target domain used for deployment;

[0016] Step 8, calculate the multilayer perceptron discriminator f MLP The discriminative loss is applied to the pre-trained convolutional neural network f through backpropagation of the loss gradient. CNN Perform adversarial fine-tuning to achieve domain adaptation to the target domain;

[0017] Step 9, fine-tune f CNN Re-applied to linear classifier f C Composing an underwater acoustic recognition model, in D new The algorithm was tested and verified to confirm its effectiveness.

[0018] Step 10: If the test results meet expectations, save the fine-tuned convolutional neural network f. CNN With linear classifier f C The model parameters are determined and then deployed.

[0019] In step 1 of this invention, underwater acoustic data D is extracted from the database. ori It is divided into underwater acoustic data D containing the target. target and underwater acoustic data without target D no_target Based on the data characteristics, a weighted hybrid method for generating target-containing data was adopted to generate new target-containing samples to expand data diversity and balance the number of samples from different categories, resulting in a class-balanced dataset D′. ori ;

[0020] The steps for generating weighted mixed samples include: creating two random samplers, which randomly select underwater acoustic data x containing the target. target and underwater acoustic data without target x no_target A random number between 0.7 and 1 is generated using the random number generator in the Python random library and used as the mixing weight r; the weight r is then assigned to the sampled underwater acoustic data x containing the target. target Assign 1-r to x no_targetA new sample x containing the target is obtained. target_gen Repeat this step to generate new samples containing the target until the total number of samples in this class equals the number of samples without the target.

[0021] In step 2 of this invention, the class-balanced dataset D′ generated in step 1 is... ori As input to the training set, and for the training data x i Frequency domain feature transformation is performed to obtain the Mel power spectrum that conforms to the characteristics of human auditory perception. The specific calculation formula is as follows:

[0022]

[0023] The frequency domain transformation step includes: converting the time-domain underwater acoustic signal... (sampling frequency is f) s According to the preset frame length N L With frame shift N M Divided into T frames, the signal of the t-th frame is x i (m, t); Applying a Hamming window to the framing results makes the time-domain signal better meet the periodicity requirement and reduces leakage, resulting in Perform a short-time Fourier transform on the windowed single-frame time-domain signal to map the time-domain signal to the frequency domain, and calculate the frequency scale as N. FFT Complex frequency domain signal X i (k, t) is further calculated and converted into a real power spectrum P. i (k, t); Construct a Mel filter bank H with a preset number of K. k (k′)(Only the positive frequency part is taken, i.e., k′=0,1,...,N) FFT / 2), the power value S after Mel filtering is obtained by weighted summation of the linear power spectrum using the Mel filter bank. i (k, t); then, logarithmic calculation is used to compress the dynamic range, improve the discriminative power of low-energy features, make them conform to the input characteristics of neural networks, and generate Mel power spectrum.

[0024] In step 3 of this invention, the Mel power spectrum obtained in step 2 is transformed... Perform random data augmentation operations Random(scroll, mask) t ,mask f(Reference: Park, DS, Chan, W., Zhang, Y., Chiu, C.-C., Zoh, B., Cubuk, ED, & Le, QV (2019). SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition. Interspeech 2019.) This improves the model's generalization ability and ultimately yields...

[0025]

[0026] The data augmentation operation Random(scroll, mask) t ,mask f The steps include: processing the extracted Mel power spectrum. Zero-mean normalization is obtained Randomly mask k1 time points along the time axis on the Mel power spectrum, corresponding to the mask. t Operation: Using prior knowledge, specify the most important frequency scales for identifying the target. Randomly select k2 frequencies from the remaining frequency scales and perform zero masking, corresponding to the mask. f The scroll operation is a random scrolling operation, scrolling randomly along the time axis by ±t ticks, and filling the empty spaces after scrolling with zeros.

[0027] In step 4 of this invention, the enhanced product obtained in step 3 is... As model input, the input is an improved convolutional neural network f adapted to Mel power spectrum features. CNN It effectively extracts spectral features through two rounds of convolution and pooling layer operations. Finally, the extracted features are flattened and compressed and integrated via fully connected layers fc1 and fc2 to extract the feature code v. i The specific calculation process is as follows:

[0028] f CNN (x)=fc2(ReLU(fc1(Conv(x)))))

[0029] Conv(x)=pool2(ReLU(conv2(pool1(ReLU(conv1(x))))))

[0030]

[0031] In this layer, conv1 is the first convolutional layer with a kernel size of 5 and a stride of 1, which can effectively extract local spectral features. ReLU is the activation function, which mainly distorts linearity and improves the nonlinear fitting ability of the network. pool1 is the first pooling layer with a kernel size of 2 and a stride of 2, which performs dimensionality reduction and feature selection on the activated feature map. conv2 and pool2 have the same structure as conv1 and pool1, and further improve the learning ability of the network by increasing the number of layers. The fully connected layers fc1 and fc2 realize feature encoding compression and integration.

[0032] In step 5 of this invention, the feature code v extracted in step 4 is... i Input linear classifier f C The final identification result is obtained from this. in like It is then assumed that the underwater sound contains target information, denoted as... The classification loss is calculated by comparing the data with the true labels using the binary cross-entropy loss BCELoss and then adjusting f using the back propagation algorithm (BP). CNN and f C The parameters are calculated until convergence, where BCELoss is calculated as follows:

[0033] f C (x)=Softmax(W C x+b C )

[0034]

[0035]

[0036] Among them W c b is the weight matrix of the linear classifier, used to implement linear mapping; C is the bias vector of the linear classifier, which realizes the offset in the mapping space; Softmax is the activation function used for probability output, which maps the linear transformation output to probability values ​​of [0, 1] in the classification task, realizing the distinction between features belonging to the source domain or the target domain.

[0037] In step 6 of this invention, inspired by Adversarial Discriminative Domain Adaptation (ADDA) (see: E. Tzeng, J. Hoffman, K. Saenko and T. Darrell, "Adversarial Discriminative Domain Adaptation", 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 2962-2971), unlabeled underwater acoustic signals D from the waters where the field deployment will be carried out will be collected in advance. new and the source domain dataset D′ used for training ori Together, they form dataset D for subsequent adversarial domain adaptation fine-tuning. adda Temporarily freeze f after step 5 training. C The parameters will f CNN Switch to training mode for further fine-tuning; similar to step 2, obtain dataset D. adda Mel power spectrum of medium sample and use f CNN Perform feature extraction on it to obtain and

[0038] In step 7 of this invention, the material from step 6 is... and The inputs are respectively to a multilayer perceptron discriminator f consisting of two linear layers. MLP In the middle, let f MLP Try to distinguish the source of the data (D′) ori Or D new ):

[0039] f MLP (v)=fc4(ReLU(fc3(v))))

[0040] Where fc3 and fc4 are fully connected layers, and ReLU is the activation function; MLP The output is a two-dimensional vector. like The surface discriminator then considers feature v′ to be... i From D′ ori Conversely, feature v′ is considered to be... i From target domain D new .

[0041] In step 8 of this invention, the binary cross-entropy loss BCELoss is defined as the loss function of the model, and the model's loss in D is calculated.adda The discriminative loss on the given surface is calculated, and the corresponding gradient is backpropagated; so that f CNN To improve the "deception ability" of data from different domain sources where the encoding distribution tends to be consistent, thus making f MLP It is more sensitive to the different encodings of data from different domains, thus improving its discrimination ability; ultimately enabling the realization of f CNN Unsupervised fine-tuning improves its performance in the target domain.

[0042] In step 10 of this invention, the trained model f is saved. CNN and f C As needed, it can be deployed on a microcontroller or server for application.

[0043] Beneficial effects:

[0044] 1. This invention designs a sample mixing generation method that is highly suitable for underwater sound recognition, improving the diversity of positive samples while balancing the samples. The excessively high imbalance ratio in this project meant that the classifier would fail directly if no data balancing was performed. We tried directly modifying the classification weights and using loss functions specifically designed for class imbalance problems, such as Focalloss (reference: T.-Y. Lin, P. Goyal, R. Girshick, K. He and P. Dollár, "Focal Loss for Dense Object Detection," 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017, pp. 2999-3007). However, the results were not as expected. When the requirement was to identify at least 21 out of 30 target samples in the test set (to ensure target recall in practical applications), none of the methods could improve the overall accuracy to above 70%. However, after balancing the data using the specially designed DataMixGen mixed data generation technology, the model achieved a classification performance of 73% while ensuring that 22 / 30 samples with targets could be identified.

[0045] 2. Because this invention uses a fine-tuning architecture similar to adversarial discriminative domain adaptation, it effectively overcomes the problem of poor generalization ability of the same model in different waters. When testing with real military data, after domain alignment in the second stage, the overall accuracy was improved to 78.05% while ensuring that 27 / 30 target samples could be identified. Both recall and precision were significantly improved. Attached Figure Description

[0046] The following further specific descriptions of the present invention will be made in conjunction with the accompanying drawings and specific embodiments, making the technical details clearer.

[0047] Figure 1 is a schematic flow chart of the present invention.

[0048] Figure 2 is a detailed process of the adversarial domain adaptation fine-tuning convolutional feature extractor in the second stage.

[0049] Figure 3 is a visual comparison of the Mel power spectrum and the conventional linear spectrum used in the present invention.

[0050] Figure 4 is the specific result of the ablation experiment of the method proposed in the present invention. Specific Embodiments

[0051] In order to make the purpose, technical solutions, and advantages of the present invention clearer and more distinct, this chapter will further describe the invention in detail in conjunction with the accompanying drawings and embodiments.

[0052] Figure 1 is a flow chart of a few-shot underwater acoustic target recognition method based on weighted hybrid generation and adversarial domain adaptation proposed in the present invention, including 3 stages and 10 steps.

[0053] The first stage of the underwater acoustic recognition algorithm proposed in the present invention is the pre-training stage based on classification loss, corresponding to steps 1-5.

[0054] In step 1 of the present invention, a total of N0 underwater acoustic data D is extracted from the database ori , and according to the labels, it is divided into underwater acoustic data D target containing targets and underwater acoustic data D no_target without targets, where D target = {x1, x2,..., x k}, and k < N0; according to the data characteristics, a weighted hybrid target-containing data generation method is used to generate new target-containing samples, and is constructed so that k1 = N0 - k, expanding the data diversity and balancing the number of samples in different categories, and obtaining a class-balanced dataset D' ori；

[0055] Among them, the steps of weighted hybrid sample generation include: creating two random samplers, randomly screening underwater acoustic data x target containing targets and underwater acoustic data x no_target without targets respectively; using the random number generator in the python random library to generate a random number between 0.7 and 1 as the mixing weight r; assigning the weight r to the sampled underwater acoustic data x target , and assigning 1 - r to x no_target , to obtain a new target-containing sample x target_genRepeat this step N0-2k times to generate new target samples, ultimately constructing a new dataset of target samples with a total number of N0-k samples.

[0056] In step 2 of this invention, the class-balanced dataset D′ generated in step 1 is... ori ={D′ target D no_target} as input to the training set, and on the training data x i Frequency domain feature transformation is performed to obtain the Mel power spectrum that conforms to the characteristics of human auditory perception. Includes the following steps:

[0057] Step 2-1, convert the time-domain underwater acoustic signal (sampling frequency is f) s According to the preset frame length N L With frame shift N M Divided into T frames, of which The signal in frame t is The specific calculation formula is as follows:

[0058] x i (m, t) = x i (t×N M +m), m=0,1,...,N L -1; t = 0, 1, ..., T-1

[0059] Step 2-2, for the frame segmentation result x i Applying a Hamming window w to (m,t) better satisfies the periodicity requirement of the time-domain signal and reduces leakage, the result after corresponding calculation is obtained. The specific calculation method is as follows:

[0060]

[0061] Where π is the constant of pi, and cos is the cosine function;

[0062] Steps 2-3 involve performing a short-time Fourier transform on the windowed single-frame time-domain signal to map the time-domain signal to the frequency domain, and calculating the frequency scale as N. FFT Complex frequency domain signals Further calculations yielded the real power spectrum. The specific calculation formula is as follows:

[0063]

[0064] Where e is the natural constant, and j is the imaginary unit (ji). 2 =-1), |X i (k, t)|2 The operation represented is X i The square of the amplitude of (k, t); N FFT The number of points in the Fast Fourier Transform is a core parameter that determines the resolution and calculation range of frequency domain analysis. It is usually specified by us based on experience and needs.

[0065] Steps 2-4: Construct a Mel filter bank H with a preset number of K. k (k′), the power value after Mel filtering is obtained by weighted summation of the linear power spectrum using the Mel filter bank.

[0066] Step 2-4-1, constructing the Mel filter bank requires the sharp frequency f and the Mel frequency f. mel Mapping relationship between them:

[0067]

[0068] Step 2-4-2: Calculate the frequency range covered by the triangular filter on the linear power spectrum, and weight and accumulate the power spectrum values ​​within this range to obtain a "Mel band energy". Calculate the linear frequency point m corresponding to the center frequency of the k-th Mel filter. k The specific formula is as follows:

[0069]

[0070] Where f min The lowest linear frequency, f, is usually taken as 0. max The highest linear frequency is equal to f s / 2;

[0071] Step 2-4-3: Using the inverse transform formula between Mel frequency and linear frequency, obtain the linear frequency value corresponding to the center Mel frequency, and then map it to the frequency point index f(k) corresponding to the Fast Fourier Transform. The calculation formula is as follows:

[0072]

[0073] Step 2-4-4: Construct a Mel filter bank H with a preset number of K. k (k′)(Only the positive frequency part is taken, i.e., k′=0,1,...,N) FFT / 2), the power value S after Mel filtering is obtained by weighted summation of the linear power spectrum using the Mel filter bank. i (k, t), the specific calculation formula is as follows:

[0074]

[0075] Steps 2-4-5 utilize logarithmic calculations to compress the dynamic range, improving the discriminative power of low-energy features and making them conform to the input characteristics of neural networks, thereby generating the Mel power spectrum. The calculation formula is:

[0076]

[0077] Where ε = 1e-10, to prevent S i (k, t) = 0 cannot be calculated.

[0078] In step 3 of this invention, the Mel power spectrum obtained in step 2 is transformed... Perform random data augmentation operations Random(scroll, mask) t mask f (Reference: Park, DS, Chan, W., Zhang, Y., Chiu, C.-C., Zoph, B., Cubuk, ED, & Le, QV (2019). SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition. Interspeech 2019.) This improves the model's generalization ability and ultimately yields...

[0079]

[0080] The data augmentation operation Random(scroll,mask) t ,mask f The steps include: processing the extracted Mel power spectrum. Zero-mean normalization is obtained Randomly mask k1 time points along the time axis on the Mel power spectrum, corresponding to the mask. t Operation: Using prior knowledge, specify the most important frequency scales for identifying the target. Randomly select k2 frequencies from the remaining frequency scales and perform zero masking, corresponding to the mask. f The scroll operation is a random scrolling operation, scrolling randomly along the time axis by ±t ticks, and filling the empty spaces after scrolling with zeros.

[0081] In step 4 of this invention, the enhanced product obtained in step 3 is... As model input, the input is an improved convolutional neural network f adapted to Mel power spectrum features. CNNIt effectively extracts spectral features through two rounds of convolution and pooling layer operations. Finally, the extracted features are flattened and compressed and integrated through fully connected layers fc1 and fc2 to extract the feature code. (hid2 is the output dimension of fc2), the specific calculation process is as follows:

[0082] f CNN (x)=fc2(ReLU(fc1(Conv(x)))))

[0083] Conv(x)=pool2(ReLU(conv2(pool1(ReLU(conv1(x))))))

[0084]

[0085] Here, conv1 is the first convolutional layer with a kernel size of 5, a stride of 1, and C1 output channels, which can effectively extract local spectral features and output... ReLU is the activation function, whose main function is to distort linearity and improve the nonlinear fitting ability of the network; pool1 is the first max pooling layer with a size of 2 and a stride of 2, which performs dimensionality reduction and feature selection on the activated feature map, and outputs... Conv2 has the same size as conv1, with C2 output channels. Increasing the number of layers further improves the network's learning ability, resulting in the following output: pool2 has the same size as pool1, and its output is... After flattening, a length of [length missing] is obtained. The vector, via the weight matrix is The fully connected layer fc1 outputs a hidden layer vector of length hid1, which is activated by the ReLU function and then processed by the weight matrix. The fully connected layer fc2 further compresses and integrates the feature encoding, outputting a feature vector of length hid2.

[0086] In step 5 of this invention, the feature code extracted in step 4 is... Input linear classifier f C The final identification result is obtained from this. in like Then it is assumed that the sound of the water contains target information, and this is recorded. The value is 1 if the prediction is true and 0 otherwise; use the prediction results The classification loss is calculated by comparing the data with the true labels using the binary cross-entropy loss BCELoss and then adjusting f using the back propagation algorithm (BP).CNN and f C The parameters are calculated until convergence, where BCELoss is calculated as follows:

[0087] f C (x)=Softmax(W C x+b C )

[0088]

[0089]

[0090] in b is the weight matrix of the linear classifier, used to implement linear mapping; C is the bias vector of the linear classifier, which realizes the offset in the mapping space; Softmax is the activation function used for probability output, which maps the linear transformation output to probability values ​​of [0, 1] in the classification task, realizing the distinction between features belonging to the source domain or the target domain.

[0091] The second stage of the underwater acoustic recognition algorithm proposed in this invention is based on adversarial domain adaptation fine-tuning, corresponding to steps 6-8, and the specific framework is shown in Figure 2.

[0092] In step 6 of this invention, inspired by Adversarial Discriminative Domain Adaptation (ADDA) (see: E. Tzeng, J. Hoffman, K. Saenko and T. Darrell, "Adversarial Discriminative Domain Adaptation", 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 2962-2971), unlabeled underwater acoustic signals from the waters where the field deployment will be carried out are collected in advance. and the source domain dataset D′ used for training ori ={D′ target D no_target Together, they form the dataset D used for subsequent adversarial domain adaptation fine-tuning. adda Temporarily freeze the linear classifier f trained in step 5. C The parameters will be used by the feature extractor f, which is implemented based on a convolutional neural network. CNN Switch to training mode for further fine-tuning; similar to step 2, obtain dataset D.adda Mel power spectrum of medium sample and use f CNN Perform feature extraction on it to obtain and

[0093] In step 7 of this invention, the material from step 6 is... and The inputs are respectively to a multilayer perceptron discriminator f consisting of two linear layers. MLP In the middle, let f MLP Try to distinguish the source of the data (D) ori Or D new ):

[0094] f MLP (v)=fc4(ReLU(fc3(v))))

[0095] fc3 and fc4 are fully connected layers, and their feature matrices are respectively and ReLU is the activation function; f MLP The output is a two-dimensional vector. like The surface discriminator then considers feature v′ to be... i From D′ ori Conversely, feature v′ is considered to be... i From target domain D new .

[0096] In step 8 of this invention, the binary cross-entropy loss BCELoss is defined as the loss function of the model, and the model's loss in D is calculated. adda The discriminative loss on the given surface is calculated, and the corresponding gradient is backpropagated; so that f CNN To improve the "deception ability" of data from different domain sources where the encoding distribution tends to be consistent, thus making f MLP It is more sensitive to the different encodings of data from different domains, thus improving its discrimination ability; ultimately enabling the realization of f CNN Unsupervised fine-tuning improves its performance in the target domain.

[0097] The third stage of the underwater acoustic recognition algorithm proposed in this invention is the verification and deployment stage, corresponding to steps 9-10.

[0098] In step 9 of this invention, the pre-trained model f is used on the labeled target domain dataset. CNN and f C The model is tested, and its performance metrics such as precision, recall, and F1 score are given to evaluate whether the results after training are stable and sufficient for practical application.

[0099] In step 10 of this invention, the already trained model f... CNN and f C The parameters are processed, the model is quantified and saved, and then deployed on a microcontroller or server for application as needed.

[0100] Example

[0101] To verify the effectiveness of this invention, we used two real-world datasets provided by our collaborative research team. These included a source domain dataset for training and a dataset collected from other sea areas to verify the generalization performance of the scheme. The data were obtained by slicing two audio segments using a sliding window, with a slice length of 5 seconds and a step size of 1 second. The source domain dataset for training contained 1200 target data points and 18740 ocean noise data points; the target domain dataset for testing came from different sea environments, containing 30 target data points and 1770 ocean noise data points. Due to the difficulty in collecting data from other countries' ships in real-world scenarios, the data provided to us was of poor quality. The data provided was highly repetitive, with poor data distribution diversity, unclear effective information, and significant noise. Furthermore, the distribution of the validation data and training data used for testing differed greatly, placing high demands on the algorithm's generalization ability. General deep learning methods struggled to address this issue, posing a significant challenge to our algorithm research. This embodiment uses real-world data with a small number of effective samples and significant differences between the test and training sets as an example, and performs target identification according to the following steps:

[0102] 1. Class balancing is performed on highly imbalanced source domain underwater acoustic data. Unlabeled samples are treated as real environmental noise, and a mixed sample generation method is used to randomly weight and mix them with real labeled data to generate labeled data that combines diversity and realism. This data, together with the original data, constitutes the training set X. train (Approximately 40,000 data entries).

[0103] 2. Extract X using a pre-designed frequency domain transformation algorithm in conjunction with a Mel filter bank. train The frequency domain characteristics of underwater acoustic data were analyzed to obtain a Mel power spectrum with obvious low-frequency features that conforms to auditory perception characteristics.

[0104] 3. Due to the small differences between different samples in this underwater acoustic data, a data augmentation operation similar to SpecAugment was designed to prevent model overfitting. First, the Mel power spectrum was normalized to zero mean. Then, enhancement operations such as time-axis masking, frequency domain scale masking, and translation were used to deliberately mask some non-critical features, resulting in...

[0105] 4. Use the enhanced two-dimensional spectrum as the convolutional neural network f CNNThe input, a single input tensor of size [B, 1, T, F], i.e., a batch size of B, a single audio channel, and T×F spectral features, processed by f CNN The compressed and extracted features are transformed into a vector of length d; then a linear classifier f is used. C The extracted feature vector d is classified, and the classification results are... Compared with the actual result y i The comparisons are performed, and the loss is calculated using the binary cross-entropy loss BCELoss. The gradient backpropagation architecture built into PyTorch is then used to evaluate f. CNN and f C The parameters are pre-trained.

[0106] 6. To ensure the model performs well on the test set X test Generalization performance on (underwater acoustic data collected from different water areas), without leaking X test In the case of real labels, it is used for unsupervised fine-tuning of the model. Specifically, it records the source of the data as a pseudo-label for the micro-task, using f. CNN Features v from data from different sources were extracted, and a discriminator f was constructed using a multilayer perceptron. MLP The training process attempts to identify which dataset (which body of water) the feature encoding v originates from; during this training process, f CNN and f MLP With opposite optimization objectives, the loss function will attempt to guide f. CNN By converging the codes of different water bodies (learning the same distribution across different water bodies), an attempt is made to fool f. MLP , and f MLP The loss target and f CNN Conversely, f MLP It will continuously improve its discrimination ability and try to more accurately distinguish the source of data; through this adversarial training method, we can achieve results without relying on the test set X. test Labeling, i.e., can improve the generalization performance of the model without re-collecting target samples, only by re-collecting noise from new water areas.

[0107] 7. After completing the two-stage training, readjust the f... CNN Re-applied to linear classifier f C A water acoustic recognition model was constructed using the test set X. test Test the various metrics of the model to confirm the effectiveness of the algorithm. After the partner approves it, use the torch.jit.trace() function built into the PyTorch architecture to package the model and deploy it.

[0108] After testing, by using the hybrid sample generation operation in this invention, the overall accuracy of the model was improved from 57.4% to 73%. When the model could identify 22 / 30 samples with the target, the classification precision reached 73%. After further improving the generalization ability by introducing a fine-tuning architecture that adapts to the class adversarial discriminant domain, the model ensured that when 27 out of 30 samples containing the target were identified, the overall accuracy was further improved to 78.05%, and both recall and precision were significantly optimized.

[0109] Figure 2 illustrates the detailed process of the adversarial domain adaptation fine-tuning in the second stage of the proposed method. Source and target domain data are input into a pre-trained convolutional neural network (sharing parameters) to extract cross-domain feature vectors. These feature vectors are then fed into a discriminator based on a multilayer perceptron, which must distinguish whether the features originate from the source or target domain. The loss optimization objective of the convolutional network is to make the entire architecture indistinguishable from the data source, i.e., to encode underwater acoustic data from two different sources into a consistent hidden layer space, removing irrelevant ocean noise information, and allowing the target domain features to "confuse" the discriminator as much as possible. The loss optimization objective of the discriminator is to accurately classify the data sources with different encodings, i.e., to accurately identify the "deception tactics" of the convolutional neural network. Finally, this adversarial fine-tuning process achieves the transfer and adaptation of source domain knowledge to the target domain.

[0110] Figure 3 illustrates the difference between Mel power spectrum and linear spectrum. As can be seen from the figure, the frequency axis of the linear spectrum obtained through Fast Fourier Transform is uniformly distributed, with low-frequency features concentrated only in a narrow strip at the bottom of the image. In contrast, the Mel power spectrum allocates more dimensions to low frequencies (with a larger proportion in the vertical direction), making the low-frequency details of the underwater acoustic signal clearer. The effective information of underwater acoustic signals (such as ship radiated noise) is mostly concentrated in the low-frequency band. The Mel power spectrum aligns with the "low-frequency sensitivity, high-frequency compression" characteristic of human underwater acoustic perception, better preserving low-frequency feature details while compressing redundant high-frequency information. Therefore, it is more suitable for underwater acoustic recognition tasks and has become the preferred choice in this field.

[0111] Figure 4 shows the specific data from the ablation experiments of the method proposed in this invention. This table comprehensively compares the performance of the Baseline, DataMixGen, and DataMixGen+ADDA methods. Accuracy is the proportion of correct predictions made by the model as a whole; Recall is the proportion of samples correctly identified; Precision is the proportion of predicted results that are actually of that class; AUC (area under the curve) can comprehensively reflect the model's ability to distinguish between positive and negative classes; and F1-score is the harmonic mean of precision and recall. From these various metrics, it can be seen that the hybrid data generation and adversarial domain adaptation architecture used in this invention are well-suited for underwater acoustic recognition tasks with few samples and large differences in domain noise.

[0112] This invention provides a few-sample underwater acoustic target recognition method based on weighted hybrid generation and adversarial domain adaptation. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A method for few-sample underwater acoustic target recognition based on weighted hybrid generation and adversarial domain adaptation, characterized in that, Includes the following steps: Step 1, train the underwater acoustic dataset D, which has a high class imbalance. ori Let D be the data containing the target and the data not containing the target. target and D no_target , give D target D with higher weight and lower weight no_target Randomly mix to generate new target samples D′ ori Step 2, for the balanced training dataset D′ ori Frequency domain feature transformation is performed, and a linear power spectrum is obtained based on short-time Fourier transform. This power spectrum is then mapped through a Mel filter bank to a Mel power spectrum that conforms to auditory perception characteristics. Step 3: Analyze the extracted Mel power spectrum. Perform zero-mean normalization, and then obtain the result through data augmentation operations such as masking and translation. To prevent overfitting and improve the generalization ability of subsequent models; Step 4, utilize the feature encoder f implemented based on a convolutional neural network. CNN Enhanced Mel power spectrum Feature extraction is performed to obtain the feature encoding v of the Mel power spectrum. i Step 5: Use a linear classifier f C For the extracted features v i Classify the data and calculate the loss against f using the binary cross-entropy loss BCELoss. CNN and f C Perform pre-training; Step 6, utilize the classification-trained convolutional neural network f CNN Extract the source domain D′ for training respectively ori Data characteristics and the target domain data D to be deployed subsequently new Features Let the set of the two be... Step 7, create a new multilayer perceptron f MLP As a discriminator, it attempts to distinguish whether the input code v comes from the source domain or the target domain used for deployment; step 8, compute the multilayer perceptron discriminator f. MLP The discriminative loss is applied to the pre-trained convolutional neural network f through backpropagation of the loss gradient. CNN Perform adversarial fine-tuning to achieve domain adaptation to the target domain; Step 9, adjust the fine-tuned f... CNN Re-applied to linear classifier f C Composing an underwater acoustic recognition model, in D new The algorithm is then tested and verified to confirm its effectiveness. Step 10: If the test results meet expectations, the fine-tuned convolutional neural network f is saved. CNN With linear classifier f C The model parameters are determined and then deployed.

2. The method for few-sample underwater acoustic target recognition based on weighted hybrid generation and adversarial domain adaptation as described in claim 1, characterized in that, In step 1, underwater acoustic data D is extracted from the database. ori It is divided into underwater acoustic data D containing the target. target and underwater acoustic data D without a target no_target Based on the data characteristics, a weighted hybrid method for generating target-containing data was adopted to generate new target-containing samples to expand data diversity and balance the number of samples from different categories, resulting in a class-balanced dataset D′. ori The steps for generating weighted mixed samples include: creating two random samplers, which randomly select underwater acoustic data x containing the target. target and underwater acoustic data without target x n_target A random number between 0.7 and 1 is generated using the random number generator in the Python random library and used as the mixing weight r; the weight r is then assigned to the sampled underwater acoustic data x containing the target. target Assign 1-r to x no_target A new sample x containing the target is obtained. target_gen Repeat this step to generate new samples containing the target until the total number of samples in this class equals the number of samples without the target.

3. The method for few-sample underwater acoustic target recognition based on weighted hybrid generation and adversarial domain adaptation as described in claim 2, characterized in that, In step 2, the class-balanced dataset D′ generated in step 1 is... ori As input to the training set, and for the training data x i Frequency domain feature transformation is performed to obtain the Mel power spectrum that conforms to the characteristics of human auditory perception. The specific calculation formula is as follows: The frequency domain transformation step includes: converting the time-domain underwater acoustic signal... (sampling frequency is f) s According to the preset frame length N L With frame shift N M Divided into T frames, the signal of the t-th frame is x i (m, t); Applying a Hamming window to the framing results makes the time-domain signal better meet the periodicity requirement and reduces leakage, resulting in Perform a short-time Fourier transform on the windowed single-frame time-domain signal to map the time-domain signal to the frequency domain, and calculate the frequency scale as N. FFT Complex frequency domain signal X i (k, t) is further calculated and converted into a real power spectrum P. i (k, t); Construct a Mel filter bank H with a preset number of K. k (k′)(Only the positive frequency part is taken, i.e., k′=0,1,…,N) FFT / 2), the power value S after Mel filtering is obtained by weighted summation of the linear power spectrum using the Mel filter bank. i (k, t); then, logarithmic calculation is used to compress the dynamic range, improve the discriminative power of low-energy features, make them conform to the input characteristics of neural networks, and generate Mel power spectrum.

4. The method for few-sample underwater acoustic target recognition based on weighted hybrid generation and adversarial domain adaptation as described in claim 3, characterized in that, In step 3, the Mel power spectrum obtained in step 2 is transformed. Perform random data augmentation operations Random(scroll, mask) t mask f This improves the model's generalization ability and ultimately yields... The data augmentation operation Random(scroll, mask) t The steps of (maskf) include: processing the extracted Mel power spectrum Zero-mean normalization is obtained Randomly mask k1 time points along the time axis on the Mel power spectrum, corresponding to the mask. t Operation: Using prior knowledge, specify the most important frequency scales for identifying the target. Randomly select k2 frequencies from the remaining frequency scales and perform zero masking, corresponding to the mask. f The scroll operation is a random scrolling operation, scrolling randomly along the time axis by ±t ticks, and filling the empty positions after scrolling with zeros.

5. A few-sample underwater acoustic target recognition method based on weighted hybrid generation and adversarial domain adaptation as described in claim 4, characterized in that... In step 4, the enhancement obtained in step 3 is... As model input, the input is an improved convolutional neural network f adapted to Mel power spectrum features. CNN It effectively extracts spectral features through two rounds of convolution and pooling layer operations. Finally, the extracted features are flattened and compressed and integrated via fully connected layers fc1 and fc2 to extract the feature code v. i The specific calculation process is as follows: f CNN (x)=fc2(ReLU(fc1(Conv(x)))))Conv(x)=pool z (ReLU(conv2(pool1(ReLU(conv1(x))))) In this layer, conv1 is the first convolutional layer with a kernel size of 5 and a stride of 1, which can effectively extract local spectral features. ReLU is the activation function, which mainly distorts linearity and improves the nonlinear fitting ability of the network. pool1 is the first pooling layer with a kernel size of 2 and a stride of 2, which performs dimensionality reduction and feature selection on the activated feature map. conv2 and pool2 have the same structure as conv1 and pool1, and further improve the learning ability of the network by increasing the number of layers. The fully connected layers fc1 and fc2 realize feature encoding compression and integration.

6. The method for few-sample underwater acoustic target recognition based on weighted hybrid generation and adversarial domain adaptation as described in claim 5, characterized in that, In step 5, the feature code v extracted in step 4 is... i Input linear classifier f C The final identification result is obtained from this. in like It is then assumed that the underwater sound contains target information, denoted as... The classification loss is calculated by comparing the data with the true labels using the binary cross-entropy loss BCELoss and then adjusting f using the back propagation algorithm (BP). CNN and f C The parameters are calculated until convergence, where BCELoss is calculated as follows: f C (x)=Softmax(W C x+b C ) Among them W c b is the weight matrix of the linear classifier, used to implement linear mapping; C is the bias vector of the linear classifier, which realizes the offset in the mapping space; Softmax is the activation function used for probability output, which maps the linear transformation output to probability values ​​of [0, 1] in the classification task, realizing the distinction between features belonging to the source domain or the target domain.

7. The method for few-sample underwater acoustic target recognition based on weighted hybrid generation and adversarial domain adaptation as described in claim 6, characterized in that, In step 6, drawing inspiration from Adversarial Discriminative Domain Adaptation (ADDA), unlabeled underwater acoustic signals D from the waters where the deployment will take place are collected in advance. new and the source domain dataset D′ used for training ori - This forms the dataset D for subsequent adversarial domain adaptation fine-tuning. ddda Temporarily freeze f after step 5 training. C The parameters will f CNN Switch to training mode for further fine-tuning; similar to step 2, obtain dataset D. adda Mel power spectrum of medium sample and use f CNN Perform feature extraction on it to obtain and 8. The method for few-sample underwater acoustic target recognition based on weighted hybrid generation and adversarial domain adaptation as described in claim 7, characterized in that, In step 7, the steps from step 6 are... and The inputs are respectively to a multilayer perceptron discriminator f consisting of two linear layers. MLP In the middle, let f MLP Try to distinguish the source of the data (D′) ori Or D new ):f MLP (v)=fc4(ReLU(fc3(v)))) where f c3 and f c4 For a fully connected layer, ReLU is the activation function; f MLP The output is a two-dimensional vector. like The surface discriminator then considers feature v′ to be... i From D′ ori Conversely, feature v is considered to be... i From target domain D new .

9. The method for few-sample underwater acoustic target recognition based on weighted hybrid generation and adversarial domain adaptation as described in claim 8, characterized in that, In step 8, the binary cross-entropy loss BCELoss is defined as the loss function of the model, and the model's loss in D is calculated. adda The discriminative loss on the given surface is calculated, and the corresponding gradient is backpropagated; so that f CNN To improve the "deception ability" of data from different domain sources by making the encoding distribution more consistent; thus, f MLP It is more sensitive to the different encodings of data from different domains, thus improving its discrimination ability; ultimately enabling the realization of f CNN Unsupervised fine-tuning improves its performance in the target domain.

10. A few-sample underwater acoustic target recognition method based on weighted hybrid generation and adversarial domain adaptation according to claim 9, characterized in that, In step 10, save the trained model f. CNN and f C As needed, it can be deployed on a microcontroller or server for application.

Images