A method for underwater acoustic target recognition based on multi-target contrastive learning

Abstract

The application discloses a kind of underwater acoustic target identification methods based on multi-target contrast learning, including original data is preprocessed, and is divided into labeled dataset and unlabeled dataset;The frequency domain characteristics and timing characteristics of the noise data after analysis and processing are constructed contrast learning agent task, and then the frequency domain and timing information self-supervised feature extraction network is constructed;Two self-supervised feature extraction networks are fused to construct MTC framework;Unlabeled dataset is used to pretrain MTC framework in self-supervision;The feature extraction results of the frequency domain information and timing information of the MTC framework after self-supervised pretraining are spliced, and a fully connected neural network is added to construct a classifier;Supervised training is carried out on downstream classification task using labeled dataset, the classification accuracy of MTC framework in downstream task is obtained, and the hyperparameters of the framework are adjusted.The application significantly reduces the cost of labeled data in the task of underwater acoustic target feature extraction and identification, and improves the data labeling efficiency.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of digital signal processing and oceanography, and specifically relates to a method for underwater acoustic target recognition based on multi-target contrastive learning. Background Technology

[0002] Sound signals travel long distances and experience minimal attenuation in water, making sonar the most effective and widely used method for underwater target identification. However, the marine environment is complex, and the interference factors present are constantly increasing with the development of human society. Manually listening and analyzing signals by sonar operators to accurately identify targets is becoming increasingly difficult and costly.

[0003] Traditional mathematical modeling methods and manually constructed feature vectors can achieve good target recognition results under specific environments and conditions, with relatively low overall computational complexity. However, limited by the distribution of training data and the size of the model, their overall generalization ability and scalability are low, limiting their application value. In the contemporary era of explosive growth in the number of sensors and the total amount of data, these traditional methods are insufficient to efficiently learn target features from massive amounts of data, making it difficult to obtain a target recognition system that can operate stably in different marine environments. Summary of the Invention

[0004] The technical problem to be solved by this invention is to propose an underwater acoustic target recognition method based on multi-target contrastive learning. By training the MTC (Multi-Target Contrastive, end-to-end multi-target learning) framework with a large amount of unlabeled data, the model has a certain representation ability for sample data. Then, only a small amount of labeled data is needed to fine-tune the model to achieve a good recognition effect, thereby reducing the workload of data labeling and improving the training efficiency of the model.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] A method for underwater acoustic target recognition based on multi-target contrastive learning includes the following steps:

[0007] S1. Preprocess the raw data of underwater acoustic target radiation noise by frame segmentation, divide the data into labeled datasets and unlabeled datasets according to whether they are labeled, and further divide the unlabeled datasets and labeled datasets.

[0008] S2. Analyze the frequency domain characteristics and temporal characteristics of the preprocessed underwater acoustic target radiated noise data, and construct contrastive learning agent tasks accordingly.

[0009] S3. Based on the proxy tasks of frequency domain features and time series features, construct a self-supervised feature extraction network for frequency domain information and a self-supervised feature extraction network for time series information, respectively.

[0010] S4. Use appropriate weights to jointly optimize the agent task objective, fuse the two self-supervised feature extraction networks from step S3 to construct the MTC framework, and achieve simultaneous training.

[0011] S5. Use the pre-processed unlabeled dataset to perform self-supervised pre-training of the MTC framework.

[0012] S6. The feature extraction results of frequency domain information and time series information in the self-supervised pre-trained MTC framework are concatenated, and a fully connected neural network layer is added to build a classifier.

[0013] S7. Use the preprocessed labeled dataset to conduct supervised training on downstream classification tasks, obtain the classification accuracy of the MTC framework in downstream tasks, and adjust the hyperparameters of the framework based on the evaluation results.

[0014] S8. Input the sample to be tested into the classifier to obtain the predicted category label and complete the recognition.

[0015] Furthermore, in step S1, data preprocessing includes the following:

[0016] The data is divided into frames: After reading the data according to the original sampling rate, the target radiation noise data for each frame is taken at a set time, and a certain amount of overlap is set between adjacent frames.

[0017] Furthermore, in step S1, the dataset partitioning includes the following:

[0018] Data that inherently contains label information needs to have its label information discarded after data is read during self-supervised training. Based on the concept of training models using a large amount of unlabeled data and a small amount of labeled data, the unlabeled dataset used for self-supervised learning is divided into the unlabeled training dataset SSL_train and the unlabeled validation dataset SSL_valid; the labeled dataset used for supervised learning of downstream tasks is divided into the labeled training dataset SUP_train and the labeled validation dataset SUP_valid.

[0019] Furthermore, in step S2, the task of constructing the contrastive learning agent includes the following:

[0020] For frequency domain features, the similarity between spectra of samples of the same class is significantly greater than the similarity between spectra of samples of different classes. Using this property, according to the definition of spectral similarity, different segments (clips) cut from the same sample are positive examples, and different segments (clips) cut from different samples are negative examples. The constructed proxy task is to maximize the similarity between clips of the same sample and minimize the similarity between clips of different samples.

[0021] Based on the steady-state characteristics of underwater acoustic target radiated noise over a short period of time, several frames of the same sample are arranged in time as positive examples, and the last frame or several frames are replaced with other sample data as negative examples. The constructed proxy task is to maximize the similarity of positive examples and minimize the similarity between negative examples.

[0022] Furthermore, in step S3, a frequency domain information self-supervised feature extraction network is constructed, which is formally based on the COLA (Contrastive Learning of representations) model, and includes the following sub-steps:

[0023] (1) The input noise data is cropped online to 1 / 2, 1 / 4 or even shorter than the original length, and converted into Mel spectrum using Fourier transform and cepstral operation. Log-Mel (LogarithmicMel Spectrogram) spectrum is obtained after logarithmic operation.

[0024] Construct an input layer that uses an array to receive the Log-Mel spectrum of the input.

[0025] (2) The encoder f(h) is constructed using the EfficientNet-B0 lightweight convolutional neural network architecture, which includes an input layer, a first convolutional layer, repeatedly stacked MBConv (mobile inverted bottleneck convolution) modules, a second convolutional layer, a global average pooling layer, and a fully connected layer.

[0026] The convolutional layer includes a batch normalization layer and a switch function.

[0027] (3) In the encoder layer, the same input spectrum is truncated twice with random starting positions and the same length to obtain two cropped spectra with the same dimensions. Random Gaussian noise is added to one of them as a positive sample pair. The result is input into the encoder f(·) to obtain the intermediate feature h. a .

[0028] (4) In the first convolutional layer, the input spectrum is convolved, multiple convolutional layers are stacked, and DSC (Depthwise Separable Convolution) technology is used for each convolutional layer to reduce the amount of computation. The data that has completed the convolution operation is batch normalized and activated.

[0029] (5) In the first part of the repeatedly stacked MBConv module, the data output by the first convolutional layer is depthwise convolved using MBConv with an expansion ratio of 1, and compression and excitation operations are performed.

[0030] (6) In the second part of the repeatedly stacked MBConv module, the data is convolved pointwise using MBConv with a larger expansion ratio. The channel dimension is increased to several times the original according to the expansion ratio, and batch normalization and activation function processing are performed. Depth convolution is performed, and compression and activation operations are performed. The channel dimension is restored using 1×1 pointwise convolution.

[0031] (7) In the third part of the repeatedly stacked MBConv module, the data is convolved pointwise using MBConv with a larger expansion ratio and a kernel stride of 1. The channel dimension is increased to several times the original according to the expansion ratio, and batch normalization and activation function processing are performed. Depth convolution is performed, and connection deactivation and skip connection operations are performed. The channel dimension is restored using 1×1 pointwise convolution.

[0032] (8) In the fourth part of the repeatedly stacked MBConv modules, repeat operation steps (6)-(7).

[0033] (9) In the second convolutional layer, a 1×1 convolutional kernel is used to perform convolution operation on the input. Multiple convolutional layers are stacked, and depth-separable convolution is used on each convolutional layer. Batch normalization and activation function processing are performed on the data after the convolution operation.

[0034] (10) In the global average pooling layer, the height and width of the feature map are reduced to 1 by performing global average pooling on the feature map, and the features of each channel are averaged.

[0035] (11) In the fully connected layer, the feature vector after global average pooling is mapped to the intermediate feature h. a .

[0036] (12) Using a single-layer fully connected neural network with the same shared weights as the projection part g(·), and utilizing the linear transformation at h a This yields the final representation vector z. t .

[0037] (13) Construct the output layer and output the similarity matrix of the representation vectors. Calculate the similarity between the representation vectors of positive and negative examples using the dot product form and temperature coefficient. The specific formula is as follows:

[0038]

[0039] Where s(x,y) represents the similarity, x and y are both feature vectors, τ represents the temperature coefficient, and ∥∥ represents the norm.

[0040] (14) Use multi-class cross-entropy as the loss function. The specific formula is as follows:

[0041]

[0042] Where p(x) represents the true value, i.e., the similarity label; q(x) represents the predicted value, i.e., the output similarity; L spec The loss function represents the frequency domain information; χ represents a set of samples.

[0043] (15) The input layer, the first convolutional layer, the repeatedly stacked MBConv modules, the second convolutional layer, the global average pooling layer and the fully connected layer encoder, projection part and output layer constitute the frequency domain information self-supervised feature extraction network.

[0044] Furthermore, in step S3, a self-supervised feature extraction network for temporal information is constructed, which borrows from the CPC (Contrastive Predictive Coding) model in form, including the following sub-steps:

[0045] (1) Construct an input layer to receive the input Log-Mel spectrum.

[0046] (2) Construct a nonlinear encoder g with shared weights using the EfficientNet-B0 lightweight convolutional neural network. enc The structure of this encoder is the same as that of encoder f(·); it encodes the input data into a representation vector z. t =g enc (x).

[0047] (3) A recurrent neural network is constructed using four GRUs (Gated Recurrent Units) for z. t Extract context information c t All four GRU units are configured to enable output.

[0048] (4) The outputs of the four gated loop units are c t The time sequence information of the input segments is concatenated. Based on the time sequence information of the positive examples, the representation vector z of the negative examples is processed through a single parallel fully connected neural network.t Make a prediction and obtain the prediction result.

[0049] (5) Construct the output layer to output the similarity matrix of the representation vectors. Calculate the similarity between the representation vectors of positive and negative examples using the dot product form and a temperature coefficient. The specific formula is as follows:

[0050]

[0051] (6) and z t The flattening operation is performed using multi-class cross-entropy as the loss function, with the specific formula as follows:

[0052]

[0053] Among them, L temp The loss function representing time series information.

[0054] (7) The input layer, nonlinear encoder, recurrent neural network consisting of 4 GRUs, a fully connected neural network in parallel, and output layer constitute a time-series information self-supervised feature extraction network.

[0055] Furthermore, in step S4, constructing the end-to-end multi-objective learning framework includes the following sub-steps:

[0056] S401. Construct an input terminal to receive the input Log-Mel spectrum.

[0057] S402. Construct a parallel feature extraction module, and connect a frequency domain and time series information self-supervised feature extraction network in parallel to process the cropped and transformed input noise data simultaneously.

[0058] S403. At the output of the end-to-end multi-objective learning framework, the loss functions output by the frequency domain and temporal information self-supervised feature extraction networks are combined into a single overall loss using a weighted summation method. The weight of the loss function for the temporal network is higher. The specific formula is as follows:

[0059] L total =f t ·L temp +L spec

[0060] Among them, f t f represents the scaling factor. t >1; L total This indicates the overall loss.

[0061] S404. Construct the output terminal to output the similarity. Adjust the similarity of the frequency domain and time series information self-supervised feature extraction network. The specific formula is as follows:

[0062]

[0063]

[0064] in, Z represents the similarity of self-supervised feature extraction networks for temporal information. b This represents the true value of the feature vector of sample b in the self-supervised feature extraction network for temporal information. Z represents the predicted value of the feature vector of sample b in the time-series information self-supervised feature extraction network, τ1 represents the temperature coefficient of the time-series information self-supervised feature extraction network, and Z represents the temperature coefficient of the time-series information self-supervised feature extraction network. b T Z represents b transpose, s spec (Z ta Z tb Z represents the similarity of the frequency domain information self-supervised feature extraction network. ta Z represents the true value of the feature vector of sample a in the frequency domain information self-supervised feature extraction network. tb The true value of the feature vector of sample b in the frequency domain information self-supervised feature extraction network, and τ2 represents the temperature coefficient of the frequency domain information self-supervised feature extraction network.

[0065] S405. Complete the fusion of the two self-supervised feature extraction networks. The fused network is the MTC framework, which includes an input end, a parallel feature extraction module, and an output end.

[0066] Furthermore, step S5, the self-supervised pre-training process includes the following sub-steps:

[0067] S501. Input the data in SSL_train into the input terminal of the MTC framework, perform online cropping, and convert it into Log-Mel spectrum.

[0068] S502. Input the Log-Mel spectrum into the parallel feature extraction module of the MTC framework and output the similarity.

[0069] S503. For a sample in a batch, all other samples are designated as negative examples of that sample. Extract a. i and b i Two cropped spectra are used as a pair of positive examples to obtain the corresponding representation vectors. and thereby obtaining and Similarity s between ij Form a similarity matrix, s ij As the element in the i-th row and j-th column; where i, j∈[1, N], and N represents the total number of samples in a batch.

[0070] When i = j, it indicates that the calculation results of the same cropped spectrum are output with high similarity; other positions output with low similarity. The representation vector of the data after encoder processing is retained as the feature vector for downstream tasks.

[0071] S504. Input the Log-Mel spectrum into the time-series information self-supervised feature extraction module of MTC and output the similarity.

[0072] S505. Calculate and output the overall loss of the self-supervised feature extraction network for frequency domain and temporal information. This loss represents the model's comprehensive representation ability in the temporal and frequency domains. Combine the corresponding output loss function value to optimize the network parameters.

[0073] Furthermore, in step S6, a classifier is constructed and trained in a supervised manner using a labeled dataset. The specific method is as follows:

[0074] Selecting context information c from the output of the time-series self-supervised feature extraction network GRU t The intermediate features h output by the encoder of the frequency domain information self-supervised feature extraction network a The model concatenates the temporal and frequency domain features extracted from the input data, and adds a fully connected neural network layer for feature projection. This involves using a fully connected neural network layer to perform a nonlinear transformation, mapping the original features to a new space to complete the subsequent classification task. Thus, the classifier is constructed by adding a softmax classification layer to the fully connected neural network.

[0075] Furthermore, in step S7, supervised training is performed using a labeled dataset, specifically as follows:

[0076] S301, The classifier extracts intermediate features h a Context information c t Finally, establish a mapping relationship with the classification labels, which are the labels for different categories to be distinguished, and complete the classification task.

[0077] S302. The data is processed by the downstream classifier to obtain the corresponding softmax cross-entropy loss function output, which measures the difference between the model's prediction results and the true labels.

[0078] S303. Next, the backpropagation algorithm is used to update the model parameters based on the gradient information of the loss function in order to minimize the loss function.

[0079] The present invention adopts the above technical solution, and compared with the prior art, its significant technical effects are as follows:

[0080] 1. This invention constructs an end-to-end multi-target contrastive learning framework, which simultaneously performs representation learning on unlabeled data using two self-supervised feature extraction networks in the frequency domain and time series. Subsequently, supervised model fine-tuning is performed on a small amount of labeled data, which can achieve the goal of obtaining high target recognition accuracy with a small amount of labeled data. This greatly reduces the dependence of model training on labeled data and reduces data labeling costs. It also saves the time of training two models separately and then fusing them, and reduces the number of parameters and code in the model fusion.

[0081] 2. The strategy of using unlabeled data for self-supervised training of the model and then fine-tuning it with a small amount of labeled data in this invention can improve model performance, enhance overall classification accuracy, and prevent overfitting compared to supervised training models using the same amount of labeled data.

[0082] 3. The method of splicing different representations in this invention and then projecting features using a fully connected layer has better downstream task performance in most cases compared to using the frequency domain and time-series information self-supervised feature extraction networks separately.

[0083] 4. The strategy of this invention, which uses underwater acoustic targets extracted through self-supervised training for downstream classification tasks, improves classification accuracy compared to directly using traditional artificially constructed features such as Log-Mel spectrum and MFCC (Mel-scale Frequency Cepstral Coefficients) features. It can achieve a high accuracy when the amount of labeled data increases. Attached Figure Description

[0084] Figure 1 This is a schematic diagram illustrating the overall implementation of the present invention.

[0085] Figure 2 This is a schematic diagram of the time-series information self-supervised feature extraction network of the present invention.

[0086] Figure 3 This is a schematic diagram of the overall framework of the MTC of the present invention. Detailed Implementation

[0087] The technical solution of the present invention will be described in detail below, but the scope of protection of the present invention is not limited to the embodiments described.

[0088] This invention proposes an underwater acoustic target recognition method based on multi-target contrastive learning, such as... Figure 1 As shown, it includes the following steps:

[0089] S1. Preprocess the raw underwater acoustic target radiated noise data through frame segmentation, dividing the data into labeled and unlabeled datasets based on whether they are labeled. The specific steps for dividing the labeled and unlabeled datasets are as follows:

[0090] After reading the dataset at the original sampling rate of 52734Hz, the frame length l is taken. frame =32768, Frame shift l hop =16384, which means that each frame takes about 0.62s of target radiated noise audio data and sets 50% overlap between adjacent frames.

[0091] Data that inherently contains label information needs to have that label information discarded after data is read during self-supervised training. Based on the concept of training models using a large amount of unlabeled data and a small amount of labeled data, the unlabeled dataset used for self-supervised learning is divided into the unlabeled training dataset SSL_train and the unlabeled validation dataset SSL_valid; the labeled dataset used for supervised learning in downstream tasks is divided into the labeled training dataset SUP_train and the labeled validation dataset SUP_valid. The allocation ratio of these four datasets is 7:1:1:1.

[0092] S2. Analyze the frequency domain and temporal characteristics of the preprocessed underwater acoustic target radiated noise data, and construct contrastive learning agent tasks accordingly. The specific content is as follows:

[0093] For frequency domain features, the similarity between spectra of samples of the same class is significantly greater than the similarity between spectra of samples of different classes. Using this property, different clips segmented from the same sample are positive examples, and different clips segmented from different samples are negative examples, according to the definition of spectral similarity. The constructed proxy task is to maximize the similarity between clips of the same sample and minimize the similarity between clips of different samples.

[0094] Based on the steady-state characteristics of underwater acoustic target radiated noise over a short period of time, several frames of the same sample are arranged in time as positive examples, and the last frame or several frames are replaced with other sample data as negative examples. The constructed proxy task is to maximize the similarity of positive examples and minimize the similarity between negative examples.

[0095] S3. Based on frequency domain features and time series features, a proxy task is implemented by constructing a self-supervised feature extraction network for frequency domain information and a self-supervised feature extraction network for time series information, respectively. The specific content is as follows:

[0096] A frequency domain information self-supervised feature extraction network is constructed, which is modeled after the COLA model in form, and includes the following sub-steps:

[0097] (1) The input noise data is cropped online and converted into Mel spectrum using Fourier transform and cepstral operation. Log-Mel spectrum is obtained after logarithmic operation.

[0098] Construct an input layer that uses an array to receive the Log-Mel spectrum of the input.

[0099] (2) The encoder f(·) is constructed using the EfficientNet-B0 lightweight convolutional neural network architecture, wherein the encoder includes an input layer, a first convolutional layer, repeatedly stacked MBConv modules, a second convolutional layer, a global average pooling layer, and a fully connected layer.

[0100] The convolutional layer includes a batch normalization layer and a switch function.

[0101] (3) In the encoder input layer, the same input spectrum is truncated twice with random starting positions and the same length to obtain two cropped spectra with the same dimension. Random Gaussian noise is added to one of them as a positive sample pair. The result is input into the encoder f(·) to obtain the intermediate feature h. a .

[0102] (4) In the first convolutional layer, the input spectrum is convolved, multiple convolutional layers are stacked, and DSC technology is used for each convolutional layer to reduce the amount of computation. The data that has completed the convolution operation is batch normalized and activated.

[0103] (5) In the first part of the repeatedly stacked MBConv module, the data output by the first convolutional layer is depthwise convolved using MBConv with an expansion ratio of 1, and compression and excitation operations are performed.

[0104] (6) In the second part of the repeatedly stacked MBConv module, the data is convolved pointwise using MBConv with a larger expansion ratio. The channel dimension is increased to several times the original according to the expansion ratio, and batch normalization and activation function processing are performed. Depth convolution is performed, and compression and activation operations are performed. The channel dimension is restored using 1×1 pointwise convolution.

[0105] (7) In the third part of the repeatedly stacked MBConv module, the data is convolved pointwise using MBConv with a larger expansion ratio and a kernel stride of 1. The channel dimension is increased to several times the original according to the expansion ratio, and batch normalization and activation function processing are performed. Depth convolution is performed, and connection deactivation and skip connection operations are performed. The channel dimension is restored using 1×1 pointwise convolution.

[0106] (8) In the fourth part of the repeatedly stacked MBConv modules, repeat operation steps (6)-(7).

[0107] (9) In the second convolutional layer, a 1×1 convolutional kernel is used to perform convolution operation on the input. Multiple convolutional layers are stacked, and depth-separable convolution is used on each convolutional layer. Batch normalization and activation function processing are performed on the data after the convolution operation.

[0108] (10) In the global average pooling layer, the height and width of the feature map are reduced to 1 by performing global average pooling on the feature map, and the features of each channel are averaged.

[0109] (11) In the fully connected layer, the feature vector after global average pooling is mapped to the intermediate feature h. a .

[0110] (12) Using a single-layer fully connected neural network with the same shared weights as the projection part g(·), and utilizing the linear transformation at h a This yields the final representation vector z. t .

[0111] (13) Construct the output layer and output the similarity matrix of the representation vectors. Calculate the similarity between the representation vectors of positive and negative examples using the dot product form and temperature coefficient. The specific formula is as follows:

[0112]

[0113] Where s(x, y) represents the similarity, x and y are both feature vectors, τ represents the temperature coefficient, and || represents the norm.

[0114] (14) Use multi-class cross-entropy as the loss function. The specific formula is as follows:

[0115]

[0116] Where p(x) represents the true value, i.e., the similarity label; q(x) represents the predicted value, i.e., the output similarity; L spec The loss function represents the frequency domain information; χ represents a set of samples.

[0117] (15) The input layer, the first convolutional layer, the repeatedly stacked MBConv modules, the second convolutional layer, the global average pooling layer and the fully connected layer encoder, projection part and output layer constitute the frequency domain information self-supervised feature extraction network.

[0118] In this embodiment, the same input spectrum is truncated twice, with random starting positions and the same length (half the original data length, i.e., 16384), resulting in two cropped spectra with the same dimensions. The dimension of the intermediate features is determined by the encoder output dimension, and the dimension of the representation vector is 512.

[0119] like Figure 2As shown, a self-supervised feature extraction network for time-series information is constructed, which borrows from the CPC model in form and includes the following sub-steps:

[0120] (1) Construct an input layer to receive the input Log-Mel spectrum.

[0121] (2) Construct a nonlinear encoder g with shared weights using the EfficientNet-B0 lightweight convolutional neural network. enc The structure of this encoder is the same as that of encoder f(·); it encodes the input data into a representation vector z. t =g enc (x).

[0122] (3) A recurrent neural network is constructed using four GRUs for z. t Extract context information c t All four GRU units are configured to enable output.

[0123] (4) The outputs of the four gated loop units are c t The time sequence information of the input segments is concatenated. Based on the time sequence information of the positive examples, the representation vector z of the negative examples is processed through a single parallel fully connected neural network. t Make a prediction and obtain the prediction result.

[0124] (5) Construct the output layer to output the similarity matrix of the representation vectors. Calculate the similarity between the representation vectors of positive and negative examples using the dot product form and a temperature coefficient. The specific formula is as follows:

[0125]

[0126] (6) and z t The flattening operation is performed using multi-class cross-entropy as the loss function, with the specific formula as follows:

[0127]

[0128] Among them, L temp The loss function representing time series information.

[0129] (7) The input layer, nonlinear encoder, recurrent neural network consisting of 4 GRUs, a fully connected neural network in parallel, and output layer constitute a time-series information self-supervised feature extraction network.

[0130] In this embodiment, in the time-series information self-supervised feature extraction network, the upper and lower halves of the network input are obtained by truncating the same sample twice, and random Gaussian noise is added to one of them. For the equal-length input spectral features on the left and right sides, they are each truncated into four equal segments, resulting in a total of eight frames. A non-linear encoder g with shared weights is used to process the four frames derived from the truncated segments a and b. enc Encode the data to obtain two sets of intermediate features.

[0131] Each GRU output has a dimension of 256. The outputs of all units are concatenated into a 1024-dimensional feature as the context information for cropped segment a. The intermediate features of cropped segment b are then predicted using a parallel fully connected neural network.

[0132] S4, such as Figure 3 As shown, by using appropriate weights to jointly optimize the proxy task objective, the two self-supervised feature extraction networks in step S3 are fused to construct the MTC framework, enabling simultaneous training. The specific details are as follows:

[0133] S401. Construct an input terminal to receive the input Log-Mel spectrum.

[0134] S402. Construct a parallel feature extraction module, and connect a frequency domain and time series information self-supervised feature extraction network in parallel to process the cropped and transformed input noise data simultaneously.

[0135] S403. At the output of the end-to-end multi-objective learning framework, the loss functions output by the frequency domain and temporal information self-supervised feature extraction networks are combined into a single overall loss using a weighted summation method. The weight of the loss function for the temporal network is higher. The specific formula is as follows:

[0136] L total =f t ·L temp +L spec

[0137] Among them, f t f represents the scaling factor. t >1; L total This indicates the overall loss.

[0138] S404. Construct the output terminal to output the similarity. Adjust the similarity of the frequency domain and time series information self-supervised feature extraction network. The specific formula is as follows:

[0139]

[0140]

[0141] in, Z represents the similarity of self-supervised feature extraction networks for temporal information. b This represents the true value of the feature vector of sample b in the self-supervised feature extraction network for temporal information. Z represents the predicted value of the feature vector of sample b in the time-series information self-supervised feature extraction network, τ1 represents the temperature coefficient of the time-series information self-supervised feature extraction network, and Z represents the temperature coefficient of the time-series information self-supervised feature extraction network. b T Z represents b transpose, s spec (Z ta Z tb Z represents the similarity of the frequency domain information self-supervised feature extraction network. ta Z represents the true value of the feature vector of sample a in the frequency domain information self-supervised feature extraction network. tb The true value of the feature vector of sample b in the frequency domain information self-supervised feature extraction network, and τ2 represents the temperature coefficient of the frequency domain information self-supervised feature extraction network.

[0142] S405. Complete the fusion of the two self-supervised feature extraction networks. The fused network is the MTC framework, which includes an input end, a parallel feature extraction module, and an output end.

[0143] In this embodiment, the temperature coefficients are taken as τ1 = 0.22, τ2 = 0.10, and f t =1000.

[0144] S5. Perform self-supervised pre-training on the MTC framework using a pre-processed unlabeled dataset. The specific steps are as follows:

[0145] S501. Input the data in SSL_train into the input terminal of the MTC framework, perform online cropping, and convert it into Log-Mel spectrum.

[0146] S502. Input the Log-Mel spectrum into the parallel feature extraction module of the MTC framework and output the similarity.

[0147] S503. For a sample in a batch, all other samples are designated as negative examples of that sample. Extract a. i and b i Two cropped spectra are used as a pair of positive examples to obtain the corresponding representation vectors. and thereby obtaining and Similarity s between ij Form a similarity matrix, s ij As the element in the i-th row and j-th column; where i, j∈[1, N], and N represents the total number of samples in a batch.

[0148] When i = j, it indicates that the calculation results of the same cropped spectrum are output with high similarity; other positions output with low similarity. The representation vector of the data after encoder processing is retained as the feature vector for downstream tasks.

[0149] S504. Input the Log-Mel spectrum into the time-series information self-supervised feature extraction module of MTC and output the similarity.

[0150] Retrieve context information c from the GRU network output t It is used for downstream classification tasks.

[0151] S505. Calculate and output the overall loss of the self-supervised feature extraction network for frequency domain and temporal information. This loss represents the model's comprehensive representation ability in the temporal and frequency domains. Combine the corresponding output loss function value to optimize the network parameters.

[0152] S6. Concatenate the feature extraction results of frequency domain information and time series information from the self-supervised pre-trained MTC framework, and add a fully connected neural network layer to build a classifier. The specific content is as follows:

[0153] Selecting context information c from the output of the time-series self-supervised feature extraction network GRU t The intermediate features h output by the encoder of the frequency domain information self-supervised feature extraction network a The model concatenates the temporal and frequency domain features extracted from the input data, and adds a fully connected neural network layer for feature projection. This involves using a fully connected neural network layer to perform a nonlinear transformation, mapping the original features to a new space to complete the subsequent classification task. Thus, the fully connected neural network, along with a softmax classification layer, completes the construction of the classifier.

[0154] S7. Supervised training of the downstream classification task is performed using the preprocessed labeled dataset to obtain the classification accuracy of the MTC framework in the downstream task. Based on the evaluation results, the hyperparameters of the framework are adjusted. The specific content is as follows:

[0155] S301, The classifier extracts intermediate features h a Context information c t Finally, establish a mapping relationship with the classification labels, which are the labels for different categories to be distinguished, and complete the classification task.

[0156] S302. The data is processed by the downstream classifier to obtain the corresponding softmax cross-entropy loss function output, which measures the difference between the model's prediction results and the true labels.

[0157] S303. Next, the backpropagation algorithm is used to update the model parameters based on the gradient information of the loss function in order to minimize the loss function.

[0158] S8. Input the sample to be tested into the classifier to obtain the predicted category label and complete the recognition.

[0159] Table 1 shows the results of the effectiveness verification experiment of the MTC framework. The experiment used equal amounts of labeled data to train a network with the same structure and parameter settings from scratch in a supervised manner, comparing its performance with that of a network pre-trained under self-supervised supervision. In the previous dataset splitting, 10% of the data SUP_train was used for training downstream tasks, and 10% of the data SUP_valid was used for testing downstream tasks. Furthermore, the 10% data used for training can be further split, taking half of it, which is equivalent to using 5% of the total data for training, while the testing still uses the 10% non-cross-test dataset. In this embodiment, statistical comparisons were performed for cases where the amount of SUP_train data accounted for 1% to 10% of the total, and the results are shown in Table 1. When the amount of labeled data reaches 4% or more, the classification accuracy of MTC is higher than 90%, but the accuracy of supervised learning methods is far less than 85%. However, when the amount of labeled data is reduced to 1%, MTC can still achieve a downstream task accuracy higher than 75%, while the supervised learning model can only achieve a classification accuracy of less than 50%. This demonstrates that models pre-trained using the MTC framework achieve better results. Even with limited labeled data, pre-trained models can still achieve high accuracy in downstream tasks. Conversely, if a model is trained directly from scratch using only a small amount of labeled data, its deeper layers prevent it from learning better representations, thus hindering high classification performance.

[0160] Table 1 Results of the MTC framework effectiveness verification experiment

[0161]

[0162] Table 2 presents the results of the ablation experiments. This embodiment verifies the importance of the temporal information self-supervised feature extraction network (Temp) and the frequency domain information self-supervised feature extraction network (Spec) in the MTC framework for the overall model. The MTC framework comprises two networks, temporal and frequency domain, and these two networks are parallel in structure. Therefore, the ablation experiments can compare the model's performance in downstream tasks when using both networks simultaneously versus using only one of them. As shown in the table, with varying amounts of labeled data (1%–10%), using the complete MTC framework generally yields better downstream task performance, showing an improvement of approximately 0.4%–2% compared to using only temporal information, and a greater improvement compared to using only frequency domain information. This demonstrates that using the complete model, thanks to the higher-dimensional feature vectors and more complete time-frequency features, allows the model to represent the input audio data more comprehensively and effectively in downstream tasks, achieving better classification performance. Another noteworthy feature is that when the amount of labeled data is small, less than 5%, the performance difference between the three network types in the table is relatively large in downstream tasks, with a maximum difference of about 6 percentage points. However, as the amount of labeled data gradually increases, the difference between the three will narrow.

[0163] Table 2 Results of the ablation experiment

[0164]

[0165] Table 3 shows the results of the comparative experiments between the MTC framework and Log-Mel spectral and MFCC features. The experiments compared the performance of underwater acoustic targets extracted through self-supervised pre-training with traditional artificially constructed features in downstream classification tasks, using the MTC framework and two common artificially constructed features: Log-Mel spectral and MFCC features. In the downstream task training of the MTC framework, after representation through both temporal and frequency domain networks and concatenating the representations to obtain a feature vector, a fully connected layer was used as a classifier to output a five-class classification result. For Log-Mel spectral and MFCC features, after extracting the two features from the input data, a fully connected layer with the same number of nodes was used for classification prediction, and the classification accuracy was statistically analyzed on the test data. Experiments were conducted and results were statistically analyzed with labeled data amounts of 1%, 5%, and 10%. The training parameters of the model remained consistent with the above experiments, i.e., the learning rate was set to 1×10⁻⁶. -4The training was conducted for 200 epochs. As shown in the table, directly using Log-Mel spectrum or MFCC features, with only a single fully connected layer as the classifier, resulted in low classification accuracy, and this accuracy did not increase with the amount of labeled data. In contrast, using the MTC framework after self-supervised pre-training resulted in higher classification accuracy on downstream tasks with varying amounts of labeled data. Specifically, when the labeled data increased from 1% to 10%, the accuracy of MTC improved from 75.29% to 92.10%, while the accuracy of the model using manually constructed features never exceeded 53%.

[0166] Table 3 Comparative experimental results of different methods

[0167]

[0168] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for underwater acoustic target recognition based on multi-target contrastive learning, characterized in that, include: S1. Preprocess the raw data of underwater acoustic target radiated noise by frame segmentation, divide the data into labeled datasets and unlabeled datasets according to whether they are labeled, and further divide them. S2. Analyze the frequency domain characteristics and temporal characteristics of the preprocessed underwater acoustic target radiated noise data, and construct contrastive learning agent tasks respectively. S3. Based on the proxy task of frequency domain features and time series features, construct a frequency domain information self-supervised feature extraction network and a time series information self-supervised feature extraction network respectively; S4. Use weights to jointly optimize the agent task objective, and fuse the two self-supervised feature extraction networks in step S3 to construct an end-to-end multi-objective learning framework. S5. Use a pre-processed unlabeled dataset to perform self-supervised pre-training on the end-to-end multi-objective learning framework; S6. Construct a classifier using frequency domain features and temporal features; S7. Use the preprocessed labeled dataset to supervise the training of the downstream classification task, obtain the classification accuracy of the end-to-end multi-objective learning framework in the downstream task, and adjust the hyperparameters of the framework. S8. Input the sample to be tested into the classifier to obtain the predicted category label and complete the recognition.

2. The underwater acoustic target recognition method based on multi-target contrastive learning according to claim 1, characterized in that, In step S1, data preprocessing includes the following: After reading the data at the original sampling rate, target radiation noise data for a set time is taken for each frame, and an appropriate amount of overlap is set between adjacent frames.

3. The underwater acoustic target recognition method based on multi-target contrastive learning according to claim 1, characterized in that, In step S1, the dataset partitioning includes the following: The unlabeled dataset used for self-supervised learning is divided into the unlabeled training dataset SSL_train and the unlabeled validation dataset SSL_valid; the labeled dataset used for supervised learning of downstream tasks is divided into the labeled training dataset SUP_train and the labeled validation dataset SUP_valid.

4. The underwater acoustic target recognition method based on multi-target contrastive learning according to claim 3, characterized in that, In step S2, the task of constructing the contrastive learning agent includes the following: For frequency domain features, different segments segmented from the same sample are defined as positive examples, and different segments segmented from different samples are defined as negative examples. The constructed proxy task is to maximize the similarity between segments of the same sample and minimize the similarity between segments of different samples. Based on the steady-state characteristics of underwater acoustic target radiated noise over a short period of time, several frames of the same sample are arranged in time as positive examples, and the last frame or several frames are replaced with other sample data as negative examples. The constructed proxy task is to maximize the similarity of positive examples and minimize the similarity between negative examples.

5. The underwater acoustic target recognition method based on multi-target contrastive learning as described in claim 4, characterized in that, In step S3, constructing the frequency domain information self-supervised feature extraction network includes the following sub-steps: (1) The input noise data is cropped online to extract segments of a set length, and then converted into Mel spectrum using Fourier transform and cepstral operation. Logarithmic Mel spectrum is obtained after logarithmic operation. The input layer is constructed and the input logarithmic Mel spectrum is received using an array. (2) Construct an encoder f(·) using a lightweight convolutional neural network architecture, wherein the encoder includes an input layer, a first convolutional layer, a repeatedly stacked moving-flipping bottleneck convolutional module, a second convolutional layer, a global average pooling layer, and a fully connected layer. The convolutional layer includes a batch normalization layer and a switch function; (3) In the encoder layer, the same input spectrum is truncated twice with random starting positions and the same length to obtain two cropped spectra with the same dimensions. Random Gaussian noise is added to one of them as a positive sample pair. The result is input into the encoder to obtain the intermediate feature h. a ; (4) In the first convolutional layer, the input spectrum is convolved, multiple convolutional layers are stacked, and depth-separable convolution is used for each convolutional layer. The data that has completed the convolution operation is batch normalized and activated. (5) In the first part of the repeatedly stacked moving-flip bottleneck convolution module, the data output by the first convolutional layer is depthwise convolved using a moving-flip bottleneck convolution with an expansion ratio of 1, and compression and excitation operations are performed. (6) In the second part of the repeatedly stacked moving-flipping bottleneck convolution module, the moving-flipping bottleneck convolution is used to perform pointwise convolution on the data, and the channel dimension is increased to several times the original according to the expansion ratio, and batch normalization and activation function processing are performed; depth convolution is performed, and compression and activation operations are performed, and 1×1 pointwise convolution is used to restore the channel dimension. (7) In the third part of the repeatedly stacked moving-flipping bottleneck convolution module, the data is convolved pointwise using a moving-flipping bottleneck convolution with a kernel stride of 1. The channel dimension is increased to several times the original according to the expansion ratio, and batch normalization and activation function processing are performed. Depth convolution is performed, and connection deactivation and skip connection operations are performed. The channel dimension is restored using 1×1 pointwise convolution. (8) In the fourth part of the repeatedly stacked moving flip bottleneck convolutional module, repeat operation steps (6)-(7); (9) In the second convolutional layer, a 1×1 convolutional kernel is used to perform convolution operation on the input. Multiple convolutional layers are stacked, and depthwise separable convolution is used on each convolutional layer. Batch normalization and activation function processing are performed on the data after the convolution operation. (10) In the global average pooling layer, the height and width of the feature vector are reduced to 1 by performing global average pooling on the feature vector, and the features of each channel are averaged. (11) In the fully connected layer, the feature vector after global average pooling is mapped to the intermediate feature h. a ; (12) Using a single-layer fully connected neural network with the same shared weights as the projection part g(·), and utilizing the linear transformation at h a This yields the final representation vector z. t ; (13) Construct the output layer and output the similarity matrix of the representation vectors. Calculate the similarity between the representation vectors of positive and negative examples using the dot product form and temperature coefficient. The specific formula is as follows: Where s(x,y) represents the similarity, x and y are both feature vectors, τ represents the temperature coefficient, and ∥∥ represents the norm; (14) Use multi-class cross-entropy as the loss function. The specific formula is as follows: Where p(x) represents the true value, i.e., the similarity label; q(x) represents the predicted value, i.e., the output similarity; L spec The loss function represents the frequency domain information; χ represents a set of samples; (15) The input layer, the first convolutional layer, the repeatedly stacked moving and flipping bottleneck convolutional module, the second convolutional layer, the global average pooling layer and the fully connected layer encoder, projection part and output layer constitute the frequency domain information self-supervised feature extraction network.

6. The underwater acoustic target recognition method based on multi-target contrastive learning according to claim 5, characterized in that, In step S3, constructing the self-supervised feature extraction network for temporal information includes the following sub-steps: (1) Construct an input layer to receive the input log-Mel spectrum; (2) Constructing a nonlinear encoder g using a lightweight convolutional neural network enc The structure of this encoder is the same as that of encoder f(·); it encodes the input data into a representation vector z. t =g enc (x); (3) A recurrent neural network is constructed using four gated recurrent units for z. t Extract context information c t The four gated loop units are configured to enable output. (4) The outputs of the four gated loop units are c t The time sequence information of the input segments is concatenated. Based on the time sequence information of the positive examples, the representation vector z of the negative examples is processed through a single parallel fully connected neural network. t Make a prediction and obtain the prediction result. (5) Construct the output layer to output the similarity matrix of the representation vectors. Calculate the similarity between the representation vectors of positive and negative examples using the dot product form and a temperature coefficient. The specific formula is as follows: (6) and z t The flattening operation is performed using multi-class cross-entropy as the loss function, with the specific formula as follows: Among them, L temp The loss function representing time-series information; (7) The time-series information self-supervised feature extraction network consists of an input layer, a nonlinear encoder, a recurrent neural network composed of four gated recurrent units, a fully connected neural network in parallel, and an output layer.

7. The underwater acoustic target recognition method based on multi-target contrastive learning according to claim 6, characterized in that, Step S4, constructing the end-to-end multi-objective learning framework, includes the following sub-steps: S401. Construct an input terminal to receive the input log-Mel spectrum; S402. Construct a parallel feature extraction module, and connect a frequency domain and time series information self-supervised feature extraction network in parallel to process the cropped and transformed input noise data simultaneously. S403. The loss functions output by the self-supervised feature extraction networks for both frequency domain and time series information are combined into a single overall loss function using a weighted summation approach. The specific formula is as follows: L total =f t ·L temp +L spec Among them, f t L represents the scaling factor. total Indicates the overall loss; S404. Construct the output terminal to output the similarity. Adjust the similarity of the frequency domain and time series information self-supervised feature extraction network. The specific formula is as follows: in, Z represents the similarity of self-supervised feature extraction networks for temporal information. b This represents the true value of the feature vector of sample b in the self-supervised feature extraction network for temporal information. Z represents the predicted value of the feature vector of sample b in the time-series information self-supervised feature extraction network, τ1 represents the temperature coefficient of the time-series information self-supervised feature extraction network, and Z represents the temperature coefficient of the time-series information self-supervised feature extraction network. b T Z represents b transpose, s spec (Z ta Z tb Z represents the similarity of the frequency domain information self-supervised feature extraction network. ta Z represents the true value of the feature vector of sample a in the frequency domain information self-supervised feature extraction network. tb The true value of the feature vector of sample b in the frequency domain information self-supervised feature extraction network, where τ2 represents the temperature coefficient of the frequency domain information self-supervised feature extraction network; S405. Complete the fusion of the two self-supervised feature extraction networks. The fused network is the end-to-end multi-objective learning framework, which includes an input end, a parallel feature extraction module, and an output end.

8. The underwater acoustic target recognition method based on multi-target contrastive learning according to claim 4, characterized in that, Step S5, the self-supervised pre-training process includes the following sub-steps: S501. Input the data in SSL_train into the input of the end-to-end multi-objective learning framework, perform online pruning, and convert it into a log-Mel spectrum; S502. Input the log-Mel spectrum into the parallel feature extraction module of the end-to-end multi-objective learning framework, and output the similarity of the frequency domain information self-supervised feature extraction network. S503. For a given sample in a batch, all other samples are designated as negative examples of that sample. Extract a. i and b i Two cropped spectra are used as a pair of positive examples to obtain the corresponding representation vectors. and thereby obtaining and Similarity s between ij The similarity matrix formed by s ij Let i represent the element in the i-th row and j-th column; where i,j∈[1,N], and N represents the total number of samples in a batch. When i = j, it indicates that the calculation results of the same clipped spectrum are output with high similarity; other positions output with low similarity. S504. Input the log-Mel spectrum into the parallel feature extraction module of the end-to-end multi-objective learning framework, and output the similarity of the time-series information self-supervised feature extraction network. S505. Calculate and output the overall loss of the frequency domain and time-series information self-supervised feature extraction network, and optimize the network parameters by combining the corresponding output loss function values.

9. The underwater acoustic target recognition method based on multi-target contrastive learning according to claim 7, characterized in that, In step S6, constructing the classifier includes the following: Selecting context information c from the output of the gated recurrent unit in the time-series self-supervised feature extraction network t The intermediate features h of the encoder output in the frequency domain information self-supervised feature extraction network a The time-series and frequency-domain features extracted from the input data are concatenated, and a fully connected neural network layer is added to perform a nonlinear transformation, mapping the original features to a new space; the fully connected neural network and the softmax classification layer constitute the classifier.

10. The underwater acoustic target recognition method based on multi-target contrastive learning according to claim 9, characterized in that, In step S7, supervised training includes the following sub-steps: S301, The classifier extracts intermediate features h a Context information c t Finally, establish mapping relationships using category labels; S302. The data is processed by the downstream classifier to obtain the corresponding softmax cross-entropy loss function output, which measures the difference between the model's prediction results and the true labels. S303. Using the backpropagation algorithm, update the model parameters based on the gradient information of the loss function to minimize the loss function.

Images