Underwater target identification method based on multi-dimensional time-frequency feature fusion

Abstract

The invention discloses an underwater target recognition method based on multi-dimensional time-frequency feature fusion, and relates to the technical field of underwater target recognition, in particular to the underwater target recognition method based on the multi-dimensional time-frequency feature fusion. The objective of the invention is to solve the problem of low recognition accuracy caused by feature redundancy and non-significant target category information in conventional underwater sound target recognition. The method comprises the following steps: step 1, constructing a training data set and a test data set based on an underwater target radiation noise database; 2, constructing a CAF-ViT model, wherein the CAF-ViT model sequentially comprises a feature extraction module, a feature fusion module and a target recognition module; the CAF-ViT model is trained on the basis of the training data set, and a trained CAF-ViT model is obtained; and step 3, predicting the data in the test data set based on the trained CAF-ViT model, and identifying the type of the underwater target.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater target recognition, and in particular to an underwater target recognition method based on multi-dimensional time-frequency feature fusion. Background Art

[0002] The purpose of underwater acoustic target recognition is to distinguish target types by analyzing sonar received signals, and it has important applications in multiple fields. In the search and rescue field, it can identify wrecked ships and personnel and improve the efficiency of search and rescue; in the marine exploration field, it can be used to identify and monitor seabed resources, providing support for research such as energy extraction. In the process of underwater acoustic target recognition, feature extraction and classifier design are two key steps. Traditional methods have certain limitations in both aspects, which affect the accuracy and robustness of recognition. Specifically, the feature extraction link often relies on artificially designed features, the data dimension is usually low-dimensional, and lacks the ability to capture deep-level features. In addition, these features are usually based on specific physical environments or target types, the robustness of the features is insufficient, and they lack the ability to adaptively adjust. The classifier design link usually uses traditional machine learning classifiers, which lack the ability to generalize to changing environments and the ability to learn large-scale data. In addition, due to structural limitations, traditional classifiers lack the ability to process spatiotemporal joint data and feature fusion processing capabilities.

[0003] Vision Transformer (ViT) is currently the best deep learning framework in the field of image recognition, and has the momentum to replace ResNet, the representative model of convolutional neural networks. ViT can directly capture the global dependencies between pixels at different positions in the image through the self-attention mechanism, and has more efficient global feature processing capabilities than ResNet. The time-frequency structural features of underwater acoustic targets include but are not limited to LOFAR features, Mel features, and wavelet packet features. These time-frequency structural features are redundant, and the key target category information is not significant. Therefore, by calculating the self-attention features of these time-frequency structural features, the feature quality can be further improved and the recognition accuracy can be improved. In addition, using the attention mechanism to achieve information fusion of multiple features is also an effective way to enhance the robustness of underwater target features and further improve the target recognition accuracy. Summary of the invention

[0004] The present invention aims to propose an underwater target recognition method based on multi-dimensional time-frequency feature fusion, so as to solve the problem of low recognition accuracy caused by feature redundancy and insignificant target category information in conventional underwater acoustic target recognition. Aiming at time-frequency features such as LOFAR, Mel and wavelet packets, a time-frequency information self-attention feature extraction method for underwater target radiation noise is proposed to achieve a deeper level of extraction of these time-frequency structural features, thereby improving the target recognition accuracy based on single-dimensional time-frequency features. On this basis, by constructing a cross-attention fusion module based on feature pairs, a fusion algorithm of multi-dimensional time-frequency features is implemented, thereby achieving information complementarity between multi-dimensional features and improving the overall performance of underwater target recognition.

[0005] A method for underwater target recognition based on multi-dimensional time-frequency feature fusion. The specific process is as follows:

[0006] Step 1: Construct training data set and test data set based on underwater target radiation noise database;

[0007] Step 2: Construct the CAF-ViT model. The CAF-ViT model includes a feature extraction module, a feature fusion module, and a target recognition module in sequence.

[0008] Train the CAF-ViT model based on the training data set to obtain a trained CAF-ViT model;

[0009] Step 3: Based on the trained CAF-ViT model, predict the data in the test dataset and identify the underwater target category.

[0010] The beneficial effects of the present invention are:

[0011] The present invention constructs an underwater acoustic target fusion recognition model based on multi-dimensional time-frequency feature input. The model uses LOFAR spectrum, Mel spectrum and wavelet packet spectrum as input, and performs target recognition based on self-attention mechanism and cross-attention mechanism. This method optimizes the self-attention feature extraction process of time-frequency structural features, realizes the refinement of category attribute features, and thus improves the target recognition accuracy based on single-dimensional feature input. On this basis, the present invention proposes a multi-feature fusion recognition method, which realizes information interaction between features of different dimensions through a cross-attention mechanism, and finally generates a target recognition result based on a comprehensive decision of multi-dimensional features. The experimental results based on a real underwater database show that the proposed fusion recognition method is significantly better than the single-dimensional feature recognition method in terms of accuracy. In addition, as an end-to-end deep learning method, the model does not require manual design of feature extraction and feature fusion algorithms and parameters, and is simple to operate and easy to implement. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1A method for underwater target feature fusion recognition with multi-dimensional time-frequency feature input;

[0013] Figure 2 Error bar plot for the CAF-ViT variant model. DETAILED DESCRIPTION

[0014] Specific implementation method 1: This implementation method is a method for underwater target recognition based on multi-dimensional time-frequency feature fusion. The specific process is as follows:

[0015] Step 1: Construct training data set and test data set based on underwater target radiation noise database;

[0016] Step 2: Construct the CAF-ViT model. The CAF-ViT model includes a feature extraction module, a feature fusion module, and a target recognition module in sequence.

[0017] Train the CAF-ViT model based on the training data set to obtain a trained CAF-ViT model;

[0018] Step 3: Based on the trained CAF-ViT model, predict the data in the test dataset and identify the underwater target category.

[0019] Specific implementation method 2: This implementation method is different from the specific implementation method 1 in that: in step 1, a training data set and a test data set are constructed based on the underwater target radiation noise database; the specific process is:

[0020] Step 1.1, by sample category, divide the audio files in the DeepShip database into cargo ships, passenger ships, tankers, and tugboats;

[0021] Through appropriate deletion, ensure that the total duration difference of the sound signal data in each type of audio file does not exceed 20%;

[0022] Step 1.2, segmenting the sound signal of each audio file in each type of audio file according to a fixed time length T=3s, to generate a large number of segmented audio signals;

[0023] All segmented audio signals are randomly divided into training data set and test data set in a ratio of 8:2;

[0024] It is required that the segmented audio signals generated by the same audio file cannot appear in both the training dataset and the test dataset;

[0025] Step 1.3: Repeat step 1.2 N times to generate training data set and test data set; N is a positive integer.

[0026] The other steps and parameters are the same as those in the first embodiment.

[0027] Specific implementation method three: This implementation method is different from specific implementation method one or two in that: in step 2, a CAF-ViT model is constructed, and the CAF-ViT model includes a feature extraction module, a feature fusion module, and a target recognition module in sequence;

[0028] Train the CAF-ViT model based on the training data set to obtain a trained CAF-ViT model;

[0029] The specific process is:

[0030] Step 21: Perform LOFAR spectrum, Mel spectrum and wavelet packet spectrum processing on each segmented audio signal in the training data set to obtain the LOFAR spectrum, Mel spectrum and wavelet packet spectrum corresponding to each segmented audio signal;

[0031] Step 22: Input each of the LOFAR spectrum, the Mel spectrum, and the wavelet packet spectrum into the feature extraction module of the CAF-ViT model, and the feature extraction module outputs the self-attention features of each of the LOFAR spectrum, the Mel spectrum, and the wavelet packet spectrum;

[0032] Step 23: Combine the self-attention features obtained in step 22 in pairs and input them into the feature fusion module, which outputs the cross-attention features.

[0033] Step 24: Input each cross attention result into the target recognition module respectively. The target recognition module is MLPHead. Each MLPHead outputs the prediction result of the category.

[0034] Calculate the confidence of the prediction results of each MLPHead output category;

[0035] The prediction score of category k′ is obtained based on the prediction results and confidence of each MLPHead output category;

[0036] The final underwater target recognition result is obtained based on the prediction score of each category;

[0037] Step 25: Repeat steps 22 to 24 until the maximum number of iterations is reached to obtain the trained CAF-ViT model.

[0038] The other steps and parameters are the same as those in the first or second embodiment.

[0039] Specific implementation method 4: This implementation method is different from any one of specific implementation methods 1 to 3 in that: in step 21, each segmented audio signal in the training data set is processed by LOFAR spectrum, Mel spectrum and wavelet packet spectrum respectively to obtain the LOFAR spectrum, Mel spectrum and wavelet packet spectrum corresponding to each segmented audio signal; the specific process is:

[0040] Step 211: divide the segmented audio signal sequence x(n) with a duration of T=3s into frames to obtain the i-th frame signal x i (n);

[0041] Among them, the i-th frame signal x i (n) contains L sampling points;

[0042] n is the nth sampling point, n=1,2,…,L;

[0043] Under this parameter, LOFAR spectrum, Mel spectrum and wavelet packet spectrum are generated respectively;

[0044] Step 212: for the i-th frame signal x i (n) is processed to obtain a LOFAR spectrum; the specific process is:

[0045] Step 2121: for the i-th frame signal x i (n) Perform normalization processing to obtain the normalized i-th frame signal u i (n); expressed as

[0046]

[0047] Step 2122: Remove signal u i The DC component in (n) is obtained by removing the DC component from the i-th frame signal X i (n); expressed as

[0048]

[0049] Among them, u i (j) is the i-th frame signal after normalization, j is the j-th sampling point, j = 1, 2, ..., L;

[0050] Step 2123, X i (n) Perform fast Fourier transform to obtain the spectrum of the i-th frame signal;

[0051] S i (k) = FFT[X i (n)] (3)

[0052] Where, FFT is fast Fourier transform, S i (k) is the i-th frame signal x i (n) spectrum;

[0053] Step 2124: obtaining a power spectrum of the i-th frame signal based on the frequency spectrum of the i-th frame signal, and synthesizing the power spectrum of each frame signal into a LOFAR spectrogram in chronological order;

[0054] Step 213: for the i-th frame signal x i(n) is processed to obtain a Mel spectrum; the specific process is:

[0055] Step 2131: Design a triangular filter so that the frequency band of the mth triangular filter satisfies

[0056] f0(m)=f l (m+1)=f h (m-1) (4)

[0057] In the formula, f0, f l 、f h Respectively represent the center frequency, lower limit frequency, and upper limit frequency; m=40;

[0058] f0(m) represents the center frequency of the mth triangular filter, f l (m+1) represents the lower limit frequency of the m+1th triangular filter, f h (m-1) represents the upper limit frequency of the m-1th triangular filter;

[0059] Step 2132: Calculate the response of the mth triangular filter based on the frequency band of the mth triangular filter; the expression is

[0060]

[0061] Where f represents the frequency, and H(f) represents the response of the triangular filter;

[0062] Step 2133: for the i-th frame signal x i (n) Perform fast Fourier transform to obtain the spectrum S of the i-th frame signal i (f);

[0063] Based on the response H of the mth triangular filter m (f) and the spectrum S of the i-th frame signal i (f), get the result E of the i-th frame signal i (m); the expression is:

[0064]

[0065] In the formula, f l (m) represents the lower limit frequency of the mth triangular filter, f h (m) represents the upper frequency limit of the m-th triangular filter;

[0066] Step 2134, synthesize the results of each frame signal into a Mel spectrum in time sequence;

[0067] Step 214: for the i-th frame signal x i (n) Perform wavelet packet transform to obtain a wavelet packet spectrum; the specific process is:

[0068] Step 2141: Apply Daubechies as the basic wavelet to the i-th frame signal x i (n) Perform wavelet packet transform to transform the i-th frame signal x i (n) is decomposed into B levels, where B is 5; the process is expressed as:

[0069] d iu (n) = WP(x i (n)) (7)

[0070] Where i is the frame index, u is the node index of the current decomposition level (1-32), and d iu (n) represents a wavelet packet node, WP() represents a wavelet packet transform;

[0071] Step 2142: Set the node d of the wavelet packet of the i-th frame iu (n) Add and get the wavelet packet result E of the i-th frame iu ; The process is expressed as:

[0072]

[0073] Step 2143: The wavelet packet result E of each frame iu Synthesize wavelet packet spectrograms in time sequence.

[0074] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.

[0075] Specific implementation method five: This implementation method is different from any one of specific implementation methods one to four in that: in step 22, each of the LOFAR spectrogram, the Mel spectrogram, and the wavelet packet spectrogram is input into the feature extraction module of the CAF-ViT model, and the feature extraction module outputs the self-attention features of each of the LOFAR spectrogram, the Mel spectrogram, and the wavelet packet spectrogram;

[0076] The specific process is:

[0077] The CAF-ViT model extracts spectral self-attention features through multiple parallel structures;

[0078] Step 221: Slice each spectrogram, that is, consider the slices of each spectrogram in a single frame as feature vectors, and arrange the feature vectors into a feature matrix in time sequence.

[0079] Where N is the total number of time frames, C is the dimension of the feature, represents the field of real numbers;

[0080] Step 222: Add category information and position information to the feature matrix to obtain a feature matrix x0 containing category information and position information; expressed as:

[0081] x0=[x class ||x slice ]+x pos (9)

[0082] in, is a randomly initialized category vector; || is a matrix concatenation operation; x pos Encode the position of the random initialization;

[0083] x0 represents the feature matrix containing category information and position information;

[0084] Step 223, the feature extraction module of the CAF-ViT model extracts self-attention features from the input x0.

[0085] The other steps and parameters are the same as those in Specific Embodiments 1 to 4.

[0086] Specific implementation method 6: This implementation method is different from any one of the specific implementation methods 1 to 5 in that: the feature extraction module of the CAF-ViT model in step 223 extracts the self-attention feature from the input x0; the specific process is:

[0087] The feature extraction module of the CAF-ViT model includes M layers in sequence;

[0088] Each layer contains an attention module, a multi-layer perceptron module and layer normalization LN;

[0089] The working process of the feature extraction module of the CAF-ViT model is as follows:

[0090] The feature matrix x0 is sequentially input into the first layer to the Mth layer of the feature extraction module of the CAF-ViT model, and the output of the Mth layer is the self-attention feature extracted by the feature extraction module of the CAF-ViT model.

[0091] The output features of the first layer are input into the second layer, the output features of the second layer are input into the third layer, and so on until the output features of the M-1 layer are input into the Mth layer;

[0092] The feature extraction module is composed of multiple repeated layers in series, each layer contains an attention module MSA and a multi-layer perceptron module MLP.

[0093] The other steps and parameters are the same as those in Specific Implementation Methods 1 to 5.

[0094] Specific implementation method 7: This implementation method is different from any one of specific implementation methods 1 to 6 in that: the output of the kth layer of the feature extraction module of the CAF-ViT model is expressed as:

[0095]

[0096] In the formula, y k represents the feature matrix of the MSA output of the kth layer of the feature extraction module, x k represents the feature matrix of the MLP output of the kth layer of the feature extraction module, x k-1 It represents the feature matrix of the MLP output of the k-1th layer of the feature extraction module, LN() represents layer normalization, MSA() represents the attention module, and MLP() represents the multi-layer perceptron module.

[0097] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

[0098] Specific implementation eight: This implementation differs from any one of specific implementations one to seven in that: in step 23, the self-attention features obtained in step 22 are combined in pairs and input into a feature fusion module, and the feature fusion module outputs cross-attention features; the specific process is:

[0099] Step 231: combine the self-attention features into pairs; the specific process is:

[0100] Define the self-attention features corresponding to the LOFAR spectrum, Mel spectrum, and wavelet packet spectrum as the first self-attention feature f1, the second self-attention feature f2, and the third self-attention feature f3 respectively;

[0101] Based on the first self-attention feature f1, the second self-attention feature f2 and the third self-attention feature f3, generate 6 feature pairs p1 = (f1, f2), p2 = (f1, f3), p3 = (f2, f1), p4 = (f2, f3), p5 = (f3, f1) and p6 = (f3, f2);

[0102] Step 232: Calculate feature pair p θ , cross attention with θ=1,2,...,6.

[0103] The other steps and parameters are the same as those in Specific Implementation 1 to 7-1.

[0104] Specific implementation method 9: This implementation method is different from any one of the specific implementation methods 1 to 8 in that: in step 232, the feature pair p is calculated. θ ,θ=1,2,...,6 cross attention; the specific process is:

[0105] Step 2321: Perform linear mapping, and then Concatenate to get a new attention matrix x n* ; expressed as:

[0106]

[0107] in is the category token output by the mth self-attention feature;

[0108] It is the slice tokens output by the nth self-attention feature;

[0109] m is the first self-attention feature f1, the second self-attention feature f2 or the third self-attention feature f3;

[0110] n is the first self-attention feature f1, the second self-attention feature f2 or the third self-attention feature f3;

[0111] m≠n;

[0112] x n* Include category token and slice tokens

[0113] Step 2322, based on the new attention matrix x n* Calculate the cross attention; expressed as:

[0114]

[0115] in, is the training parameter, C m yes The dimension of ; the superscript T indicates transposition; A map is the attention map, CA() is the cross attention, q, k, v are all vectors;

[0116] Step 2323, calculate feature pair p θ The cross attention result z θ ; The expression is:

[0117]

[0118] in, is the intermediate variable, is the intermediate variable, f nm () is a linear mapping.

[0119] The other steps and parameters are the same as those in Specific Implementation 1 to 8-1.

[0120] Specific implementation ten: This implementation differs from any one of specific implementations one to nine in that: in step 24, each cross attention result is input into MLPHead respectively, and each MLPHead outputs a prediction result of a category;

[0121] Calculate the confidence of the prediction results of each MLPHead output category;

[0122] The prediction score of category k′ is obtained based on the prediction results and confidence of each MLPHead output category;

[0123] The final underwater target recognition result is obtained based on the prediction score of each category;

[0124] The specific process is:

[0125] Step 241: Combine each cross attention result z θ Input MLPHead respectively, and each MLPHead outputs the prediction results of the category; the specific process is:

[0126]

[0127] Among them, a θ is the prediction result of each category, For z θ The category token vector of ;

[0128] Step 242: Calculate the confidence of the prediction results of each MLPHead output category; the specific process is:

[0129] For the prediction result a θ ,

[0130]

[0131] in, Represents the probability that the prediction result of the category output by the θth MLPHead belongs to category k′;

[0132] make

[0133] Indicates the confidence that the prediction result of the category output by the θth MLPHead belongs to category k′;

[0134] Step 243: Based on the prediction results and confidence levels of each MLPHead output category, the prediction score of category k′ is obtained; the expression is:

[0135]

[0136] Step 244: Obtain the final underwater target recognition result based on the prediction score of each category; the expression is:

[0137]

[0138] The other steps and parameters are the same as those in Specific Embodiments 1 to 9.

[0139] The following examples are used to verify the beneficial effects of the present invention:

[0140] Embodiment 1:

[0141] The present invention provides an underwater acoustic target recognition method based on a cross-attention mechanism, and the model is a deep learning model based on a neural network. This embodiment introduces the training and testing methods of the model, and the specific implementation steps are as follows:

[0142] Step 1: Dataset construction.

[0143] The dataset is built using the open source database Deepship.

[0144] The database contains 47 hours and 4 minutes of radiated noise data from 265 different ships, which are divided into four categories: Cargo, Passenger Ship, Tanker, and Tug. To ensure that the data of the same audio item does not appear in the training set and the test set at the same time, the files were first sampled 10 times in a ratio of 80%:20%. Subsequently, each audio file was processed as follows: the audio was downsampled to 16kHz, divided into 3s long segments, and slid every 1s. The segmented segments formed a data set, and a total of 10 data sets were obtained.

[0145] Each sample in the data set is subjected to time-frequency analysis to obtain LOFAR spectrum, Mel spectrum and wavelet packet spectrum. In order to accelerate the learning speed of the network, each spectrum is normalized so that the sample data of each spectrum follows a distribution with a mean of 0 and a variance of 1.

[0146] Step 2: Model hyperparameter setting.

[0147] Hyperparameters are non-training parameters of the model. The key model hyperparameters of the present invention include slice embedding method, encoder depth and number of multi-head attention, expansion factor of MLP_Head layer, and Dropout parameter.

[0148] Step 3: Network model training.

[0149] The deep learning network was built using the Pytorch deep learning framework. The software environment was Python 3.8, Cuda 10.0, and the hardware configuration was GPU: Nvidia Rtx3060, CPU: Intel i512500h. Adam was uniformly selected as the network optimization algorithm, with an initial learning rate of 0.005, which decayed as the training cycle increased. The batch size of each training batch was set to 256. The maximum training cycle was 150.

[0150] Figure 1 The figure shows the overall structure of the network. First, the three groups of features of the samples in the training set are input into the network in batches. After passing through the feature extraction module, a parallel feature extraction stream is formed. The network obtains the self-attention features of the multi-stream output through forward propagation. In the feature fusion module, the multi-stream self-attention features are fused with each other to obtain the cross-attention output of 6 branches. In the target recognition module, the multiple cross-attention branches give their own category prediction results and the cost functions of each branch respectively. The gradient of the front layer is calculated by the average cost function, and the gradient of each layer of the model is updated through back propagation.

[0151] Set the early stopping strategy for training, take 10% of the samples in the test set as the validation set, and do not participate in the test. When the accuracy of the model on the validation set does not improve for more than 5 training cycles, save the network parameters of the cycle corresponding to the highest validation set accuracy in the last 5 cycles as the final training result.

[0152] Step 4: Network model test results.

[0153] Use the trained network to test the test set data. Statistical analysis of the accuracy, macro recall, macro precision, and macro F1 score of the prediction results under the test set.

[0154] Embodiment 2:

[0155] The present invention provides an underwater acoustic target recognition method based on the cross attention mechanism. The hyperparameters of the model of the present invention need to be optimized. For this purpose, the following experiment is designed. The specific implementation steps are as follows:

[0156] Step 1: Set the hyperparameters and fixed parameters to be optimized.

[0157] It is known from experience that the expansion factor of the Mlp layer in the encoder and the Dropout parameter of the network have little effect on the network performance, so these parameters are fixed, and the parameters such as the slice embedding method, encoder depth and the number of multi-head attention are optimized. Table 1 provides the detailed parameters of each model. These models are represented in the format of CAF-ViT-X00. In the naming convention, "X" represents the Slice Embedding method: "I" represents the identity operation, that is, the feature input after the Slice operation has not undergone any additional transformation; "L" represents the linear transformation; "C" represents the one-dimensional convolution operation. The first number in "00" represents the number of layers of the feature extraction module, and the second number represents the number of multi-heads.

[0158] Table 1 Variants of the proposed network model

[0159]

[0160] Step 2: Optimize hyperparameters based on the statistical results of each variant structure.

[0161] Figure 2 Error bar plots for CAF-ViT variant models. Results are based on statistical analysis of 10 random data sets. The table gives the mean and standard deviation of each performance metric for CAF-ViT-I82, CAF-ViT-I42, CAF-ViT-I22, CAF-ViT-L42, and CAF-ViT-C42. The mean is marked below the bar graph, and the standard deviation corresponds to the length of the error bar, which is marked on the bar graph. The numbers in brackets reflect the change in performance without using multi-head attention (the number of heads is 1).

[0162] By comparing the performance indicators of various models, we can see that:

[0163] (1) The “slice embedding” approach significantly impacts model performance, with determining the projection of the markers during the embedding process producing the greatest improvement.

[0164] (2) Appropriately increasing the number of layers of the feature extraction module can improve the accuracy of the model. In this example, the optimal number of layers is 4.

[0165] (3) Multi-head attention does not significantly improve the performance of CAF ViT models, especially in CAF-ViT-I22, CAF-ViT-L42, and CAF-ViT-C42. This may be related to the smaller token dimensions we used. Our token dimensions are 100, 40, and 32, respectively, which are consistent with the conventional dimensions of single-head attention.

[0166] The following conclusions were drawn:

[0167] (1) The choice of “slice embedding” method has a significant impact on model performance.

[0168] (2) Appropriately increasing the number of layers of the feature extraction module can improve the accuracy of the model. In this example, the optimal number of layers is 4.

[0169] (3) Multi-head attention does not significantly improve the performance of CAF-ViT models, especially in CAF ViT-I22, CAFViT-L42, and CAF ViT-C42. This is related to the fact that we used a smaller token dimension. Our token dimensions are 100, 40, and 32, respectively. This is comparable to the single attention head dimension used in the image recognition field.

[0170] Embodiment three:

[0171] The CAF-ViT algorithm proposed in this paper extracts various features based on the multi-stream 1D-ViT model. Among them, the cross-attention fusion module and the decision fusion module are the key components for realizing fusion recognition. Through comparative experiments, the performance improvement of CAF-ViT compared with the 1D-ViT single feature recognition algorithm is analyzed. To this end, the following experiment is designed, and the specific steps are as follows:

[0172] Step 1: There are 4 algorithms for comparison, namely CAF-ViT, 1D-ViT with LOFAR as input, 1D-ViT with Mel spectrum as input, and 1D-ViT with wavelet spectrum as input. The accuracy of each algorithm, the precision, recall and F1 score in each category, and the macro-average index are calculated respectively. The experimental results are shown in Table 2. Compared with 1D-ViT, CAF-ViT shows excellent performance. Specifically, the accuracy of CAF ViT is 6.0% higher than the highest accuracy of 1D-ViT (LOFAR input), and the precision, recall and F1 score under the macro-average are all the highest values. It is worth noting that the classification performance of cargo and tanker categories is significantly improved. For cargo ships, the accuracy, recall and F1 score are improved by 6.0%, 9.2% and 7.6% respectively compared with the highest index of 1D-ViT. For tankers, these indicators are improved by 5.6%, 6.4% and 6.0% respectively. These results show that the CAF-ViT model effectively integrates the features of LOFAR, Mel spectrum and wavelet packet, thereby achieving richer feature representation and enhancing category prediction capabilities.

[0173] Table 2 Performance comparison between fusion algorithm and single-dimensional feature algorithm

[0174]

[0175] Step 2: Analyze the performance of CAF-ViT when the prediction results of 1D-ViT (LOFAR) are wrong. The samples predicted incorrectly by 1D-ViT (LOFAR) are screened out from the experimental results in step 1, and the prediction results of these samples under CAF-ViT, 1D-ViT (Mel) and 1D-ViT (WP) are statistically analyzed, as shown in Table 3. Among the 7468 samples predicted incorrectly by LOFAR features, CAF-ViT correctly predicted the categories of 2030 samples. It is worth noting that when both 1D-ViT (Mel) and 1D-ViT (WP) are predicted correctly, the correction rate of CAF-ViT reaches 75.6%. In addition, when the wavelet packet (WP) feature is predicted correctly and both LOFAR and Mel features are predicted incorrectly, the correction rate of CAF-ViT is 48.0%, and it achieves the maximum accuracy contribution, which improves the accuracy of the test set by about 3.7%.

[0176] Table 31 Prediction results of each algorithm under D-ViT (LOFAR) error prediction

[0177]

[0178] The CAF-ViT algorithm proposed in the present invention uses the Vision Transformer architecture with the best performance in recent years as the feature extraction skeleton, and uses the cross-attention mechanism and decision fusion theory to achieve fusion recognition. Through comparative experiments, the superior performance of the CAF-ViT module is specifically verified. The comparison algorithms select Joint-Model, SNANet, and the single time-frequency feature recognition models DenseNet121, ConvNeXt and the baseline algorithm SCAE of the database used in the present invention. The joint model fuses TF feature input and wave signal input to improve recognition accuracy. SNANet uses adaptive weights based on forward weights and backward weights to fuse features based on spectral component analysis. The remaining single feature recognition methods are all recognition algorithms for image input.

[0179] Table 4 shows that the accuracy, macro precision, macro recall, and macro F1 score of CAF-ViT are 83.7%, 84.0%, 83.5%, and 83.7%, respectively. This performance surpasses the comparison algorithms and is 2.3%, 2.3%, 2.7%, and 2.8% higher than the suboptimal indicators, respectively. Among the fusion recognition algorithms, SNANet performs slightly better than the Joint-Model model. The performance of the single feature recognition algorithms DenseNet121, SCAE, and ConvNeXt is significantly weaker than the fusion recognition model. DenseNet121 achieves the best performance with an accuracy of only 74.4%.

[0180] Table 4 Performance comparison of the proposed method and existing underwater target recognition algorithms

[0181]

[0182] The present invention may also have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. An underwater target recognition method based on multi-dimensional time-frequency feature fusion, characterized in that: The specific process of the method is: Step 1: Construct training data set and test data set based on underwater target radiation noise database; Step 2: Construct the CAF-ViT model. The CAF-ViT model includes a feature extraction module, a feature fusion module, and a target recognition module in sequence. Train the CAF-ViT model based on the training data set to obtain a trained CAF-ViT model; Step 3: Based on the trained CAF-ViT model, predict the data in the test dataset and identify the underwater target category.

2. The underwater target recognition method based on multi-dimensional time-frequency feature fusion according to claim 1 is characterized in that: In the step 1, a training data set and a test data set are constructed based on the underwater target radiation noise database; The specific process is: Step 1.1, by sample category, divide the audio files in the DeepShip database into cargo ships, passenger ships, tankers, and tugboats; The total duration difference of the sound signal data in each type of audio file does not exceed 20%; Step 1.2, segmenting the sound signal of each audio file in each type of audio file according to a fixed time length T=3s to generate a segmented audio signal; All segmented audio signals are randomly divided into training data set and test data set in a ratio of 8:2; It is required that the segmented audio signals generated by the same audio file cannot appear in both the training dataset and the test dataset; Step 1.3: Repeat step 1.2 N times to generate training data set and test data set.

3. The underwater target recognition method based on multi-dimensional time-frequency feature fusion according to claim 2 is characterized in that: In step 2, a CAF-ViT model is constructed, and the CAF-ViT model includes a feature extraction module, a feature fusion module, and a target recognition module in sequence; Train the CAF-ViT model based on the training data set to obtain a trained CAF-ViT model; The specific process is: Step 21: Perform LOFAR spectrum, Mel spectrum and wavelet packet spectrum processing on each segmented audio signal in the training data set to obtain the LOFAR spectrum, Mel spectrum and wavelet packet spectrum corresponding to each segmented audio signal; Step 22: Input each of the LOFAR spectrum, the Mel spectrum, and the wavelet packet spectrum into the feature extraction module of the CAF-ViT model, and the feature extraction module outputs the self-attention features of each of the LOFAR spectrum, the Mel spectrum, and the wavelet packet spectrum; Step 23: Combine the self-attention features obtained in step 22 in pairs and input them into the feature fusion module, which outputs the cross-attention features. Step 24: Input each cross attention result into the target recognition module respectively. The target recognition module is MLPHead. Each MLPHead outputs the prediction result of the category. Calculate the confidence of the prediction results of each MLPHead output category; The prediction score of category k′ is obtained based on the prediction results and confidence of each MLPHead output category; The final underwater target recognition result is obtained based on the prediction score of each category; Step 25: Repeat steps 22 to 24 until the maximum number of iterations is reached to obtain the trained CAF-ViT model.

4. The underwater target recognition method based on multi-dimensional time-frequency feature fusion according to claim 3 is characterized by: In step 21, each segmented audio signal in the training data set is processed by LOFAR spectrum, Mel spectrum and wavelet packet spectrum to obtain the LOFAR spectrum, Mel spectrum and wavelet packet spectrum corresponding to each segmented audio signal; the specific process is: Step 211: divide the segmented audio signal sequence x(n) with a duration of T=3s into frames to obtain the i-th frame signal x i (n); Among them, the i-th frame signal x i (n) contains L sampling points; n is the nth sampling point, n=1,2,…,L; Step 212: for the i-th frame signal x i (n) is processed to obtain a LOFAR spectrum; the specific process is: Step 2121: for the i-th frame signal x i (n) Perform normalization processing to obtain the normalized i-th frame signal u i (n); expressed as Step 2122: Remove signal u i The DC component in (n) is obtained by removing the DC component from the i-th frame signal X i (n); expressed as Among them, u i (j) is the i-th frame signal after normalization, j is the j-th sampling point, j = 1, 2, ..., L; Step 2123, X i (n) Perform fast Fourier transform to obtain the spectrum of the i-th frame signal; S i (k)=FFT[X i (n)] (3) Where, FFT is fast Fourier transform, S i (k) is the i-th frame signal x i (n) spectrum; Step 2124: obtaining a power spectrum of the i-th frame signal based on the frequency spectrum of the i-th frame signal, and synthesizing the power spectrum of each frame signal into a LOFAR spectrogram in chronological order; Step 213: for the i-th frame signal x i (n) is processed to obtain a Mel spectrum; the specific process is: Step 2131: Design a triangular filter so that the frequency band of the mth triangular filter satisfies f0(m)=f l (m+1)=f h (m-1) (4) In the formula, f0, f l 、f h Respectively represent the center frequency, lower limit frequency, and upper limit frequency; f0(m) represents the center frequency of the mth triangular filter, f l (m+1) represents the lower limit frequency of the m+1th triangular filter, f h (m-1) represents the upper limit frequency of the m-1th triangular filter; Step 2132: Calculate the response of the mth triangular filter based on the frequency band of the mth triangular filter; the expression is Where f represents the frequency, and H(f) represents the response of the triangular filter; Step 2133: for the i-th frame signal x i (n) Perform fast Fourier transform to obtain the spectrum S of the i-th frame signal i (f); Based on the response H of the mth triangular filter m (f) and the spectrum S of the i-th frame signal i (f), get the result E of the i-th frame signal i (m); the expression is: In the formula, f l (m) represents the lower limit frequency of the mth triangular filter, f h (m) represents the upper frequency limit of the m-th triangular filter; Step 2134, synthesize the results of each frame signal into a Mel spectrum in time sequence; Step 214: for the i-th frame signal x i (n) Perform wavelet packet transform to obtain a wavelet packet spectrum; the specific process is: Step 2141: for the i-th frame signal x i (n) Perform wavelet packet transform to transform the i-th frame signal x i (n) Decomposed into B level; the process is expressed as: d iu (n)=WP(x i (n)) (7) Where i is the frame index, u is the node index of the current decomposition level (1-32), and d iu (n) represents a wavelet packet node, WP() represents a wavelet packet transform; Step 2142: Set the node d of the wavelet packet of the i-th frame iu (n) Add and get the wavelet packet result E of the i-th frame iu ; The process is expressed as: Step 2143: The wavelet packet result E of each frame iu Synthesize wavelet packet spectrograms in time sequence.

5. The underwater target recognition method based on multi-dimensional time-frequency feature fusion according to claim 4 is characterized in that: In the step 22, each of the LOFAR spectrogram, the Mel spectrogram, and the wavelet packet spectrogram is input into a feature extraction module of the CAF-ViT model, and the feature extraction module outputs the self-attention features of each of the LOFAR spectrogram, the Mel spectrogram, and the wavelet packet spectrogram; The specific process is: Step 221: Slice each spectrogram, that is, consider the slices of each spectrogram in a single frame as feature vectors, and arrange the feature vectors into a feature matrix in time sequence. Where N is the total number of time frames, C is the dimension of the feature, represents the field of real numbers; Step 222: Add category information and position information to the feature matrix to obtain a feature matrix x0 containing category information and position information; expressed as: x0=[x class ||x slice ]+x pos (9) in, is a randomly initialized category vector; || is a matrix concatenation operation; x pos Encode the position of the random initialization; x0 represents the feature matrix containing category information and position information; Step 223, the feature extraction module of the CAF-ViT model extracts self-attention features from the input x0.

6. The underwater target recognition method based on multi-dimensional time-frequency feature fusion according to claim 5 is characterized by: The feature extraction module of the CAF-ViT model in step 223 extracts the self-attention feature from the input x0; the specific process is: The feature extraction module of the CAF-ViT model includes M layers in sequence; Each layer contains an attention module, a multi-layer perceptron module and layer normalization LN; The working process of the feature extraction module of the CAF-ViT model is as follows: The feature matrix x0 is sequentially input into the first layer to the Mth layer of the feature extraction module of the CAF-ViT model, and the output of the Mth layer is the self-attention feature extracted by the feature extraction module of the CAF-ViT model.

7. The underwater target recognition method based on multi-dimensional time-frequency feature fusion according to claim 6 is characterized by: The output of the kth layer of the feature extraction module of the CAF-ViT model is expressed as: In the formula, y k represents the feature matrix of the MSA output of the kth layer of the feature extraction module, x k represents the feature matrix of the MLP output of the kth layer of the feature extraction module, x k-1 It represents the feature matrix of the MLP output of the k-1th layer of the feature extraction module, LN() represents layer normalization, MSA() represents the attention module, and MLP() represents the multi-layer perceptron module.

8. The underwater target recognition method based on multi-dimensional time-frequency feature fusion according to claim 7 is characterized by: In step 23, the self-attention features obtained in step 22 are combined in pairs and input into a feature fusion module, and the feature fusion module outputs a cross-attention feature; The specific process is: Step 231: combine the self-attention features into pairs; the specific process is: Define the self-attention features corresponding to the LOFAR spectrum, Mel spectrum, and wavelet packet spectrum as the first self-attention feature f1, the second self-attention feature f2, and the third self-attention feature f3 respectively; Based on the first self-attention feature f1, the second self-attention feature f2 and the third self-attention feature f3, generate 6 feature pairs p1 = (f1, f2), p2 = (f1, f3), p3 = (f2, f1), p4 = (f2, f3), p5 = (f3, f1) and p6 = (f3, f2); Step 232: Calculate feature pair p θ , cross attention with θ=1,2,...,6.

9. The underwater target recognition method based on multi-dimensional time-frequency feature fusion according to claim 8 is characterized by: In step 232, the feature pair p is calculated. θ ,θ=1,2,...,6 cross attention; the specific process is: Step 2321: Perform linear mapping, and then Concatenate to get a new attention matrix x n* ; expressed as: in is the category token output by the mth self-attention feature; It is the slice tokens output by the nth self-attention feature; m is the first self-attention feature f1, the second self-attention feature f2 or the third self-attention feature f3; n is the first self-attention feature f1, the second self-attention feature f2 or the third self-attention feature f3; m≠n; x n* Include categories and slice Step 2322, based on the new attention matrix x n* Calculate the cross attention; expressed as: in, is the training parameter, C m yes The dimension of ; the superscript T indicates transposition; A map is the attention map, CA() is the cross attention, q, k, v are all vectors; Step 2323, calculate feature pair p θ The cross attention result z θ ; The expression is: in, is the intermediate variable, is the intermediate variable, f nm () is a linear mapping.

10. The underwater target recognition method based on multi-dimensional time-frequency feature fusion according to claim 9, characterized in that: In step 24, each cross attention result is input into MLPHead respectively, and each MLPHead outputs a prediction result of a category; Calculate the confidence of the prediction results of each MLPHead output category; The prediction score of category k′ is obtained based on the prediction results and confidence of each MLPHead output category; The final underwater target recognition result is obtained based on the prediction score of each category; The specific process is: Step 241: Combine each cross attention result z θ Input MLPHead respectively, and each MLPHead outputs the prediction results of the category; the specific process is: Among them, a θ is the prediction result of each category, For z θ The category token vector of ; Step 242: Calculate the confidence of the prediction results of each MLPHead output category; the specific process is: For the prediction result a θ , in, Indicates the probability that the prediction result of the category output by the θth MLPHead belongs to category k′; make Indicates the confidence that the prediction result of the category output by the θth MLPHead belongs to category k′; Step 243: Based on the prediction results and confidence levels of each MLPHead output category, the prediction score of category k′ is obtained; the expression is: Step 244: Obtain the final underwater target recognition result based on the prediction score of each category; the expression is:

Images