Underwater acoustic signal recognition method based on multi-branch external attention network

Abstract

The present invention relates to a method for underwater acoustic signal recognition based on a multi-branch external attention network. First, the input data is passed through multiple parallel backbone network branches to extract feature information at different levels of the underwater acoustic signal. Second, the channel and spatial dimensions of the underwater acoustic signal are weighted separately using channel and spatial attention modules to adjust the importance of different channels and spatial positions for feature representation. Finally, the external attention module is integrated to guide the network's feature extraction and prediction using external memory units and additional computations, thereby significantly improving the model's recognition rate and robustness. Extensive experiments have confirmed the superiority of the algorithm of the present invention over existing methods.

Description

Technical Field

[0001] The present invention belongs to the field of underwater acoustic signal recognition, is used for identifying ship radiation noise signals, and specifically relates to an underwater acoustic signal recognition method based on a multi-branch external attention network. Background Art

[0002] Underwater acoustic signal recognition has attracted much attention in recent years. Due to the time-varying and space-varying characteristics of the ocean underwater acoustic channel, the complex sound source characteristics of underwater acoustic targets, and the unique propagation mode of underwater sound, the underwater acoustic signal recognition task faces huge challenges. In the early days of underwater acoustic signal recognition, traditional underwater acoustic signal recognition basically adopted a shallow learning-based method [1], which includes four steps: preprocessing, feature extraction, feature selection, and classifier design. The feature extraction method is the top priority of the underwater acoustic signal recognition task, and its effectiveness directly affects the recognition performance of the underwater acoustic signal. Traditional underwater acoustic signal recognition methods are difficult to fully obtain the representation information of the target and do not have good noise resistance. The recognition effect needs to be improved. Summary of the Invention

[0003] The purpose of the present invention is to overcome the limitations of existing underwater acoustic signal recognition and provide an underwater acoustic signal recognition method based on a multi-branch backbone external attention network. The invention adopts a multi-branch backbone network structure to better capture the local details and global features of the input data, while accelerating the convergence speed of the model. It is supplemented by channel attention and spatial attention modules to reduce the feature weights of ocean background noise and multi-target noise, and better capture the spatial structure and contextual information in the input data. Finally, the invention also adds an external attention module before the fully connected layer. Under the premise of lower complexity than self-attention, this module can mine the potential relationships of the entire underwater acoustic data set, enhance the perception and generalization capabilities of the model, and thus improve the recognition effect of the invention. At the same time, different data enhancement methods are applied to one-dimensional time domain, frequency domain data and two-dimensional time-frequency diagrams respectively. The experimental results show that the MEANet proposed by the present invention can effectively improve the accuracy of underwater acoustic signal recognition.

[0004] To achieve the above objectives, the technical solution of the present invention is: a method for underwater acoustic signal recognition based on a multi-branch external attention network, which first passes the input data through multiple parallel backbone network branches to extract feature information at different levels of the underwater acoustic signal; secondly, the channel and spatial dimensions of the underwater acoustic signal are weighted separately with the assistance of channel and spatial attention modules to adjust the importance of different channels and spatial positions to feature representation; finally, the external attention module is integrated to guide the network's feature extraction and prediction with external memory units and additional calculations.

[0005] In one embodiment of the present invention, the input data needs to undergo data preprocessing and data enhancement operations before being input into multiple parallel backbone network branches. That is, the input data undergoes data preprocessing operations to remove blank segments and slice it into time domain signals, frequency domain signals, and time-frequency domain signals. Then, after corresponding data enhancement, it is input into multiple parallel backbone network branches for training.

[0006] In one embodiment of the present invention, the data preprocessing is specifically implemented as follows:

[0007] For time domain input, MATLAB is used to slice the audio file into time domain signals. The length of each sample of the input data is 2048. The total number of samples is obtained from formula (1). After generating samples, 80% of the total samples are used as training sets and 20% of the total samples are used as test sets.

[0008]

[0009] Where L is the length of each signal, N is the total number of samples, and floor means rounding towards negative infinity;

[0010] For frequency domain input, use the FFT function to transform each sample x of the input data into i Converting from the time domain to the frequency domain as shown in equation (2), the data length is halved and the new sample is expressed as:

[0011]

[0012] The operator FFT(·) represents the transformation of x i Transform to the frequency domain and take the first half of the result;

[0013] For time-frequency domain input, for each sample x i Apply the short-time Fourier transform (STFT) to obtain the time-frequency representation shown in equation (3). Use a Hanning window with a window length of 64 to generate the time-frequency representation as follows:

[0014]

[0015] The operator STFT(·) represents the transformation of x i Transform to the time-frequency domain.

[0016] In one embodiment of the present invention, for time-frequency domain input, the signal size of the STFT is adjusted to 330×330, and the time-domain and frequency-domain inputs remain unchanged.

[0017] In one embodiment of the present invention, the data enhancement is specifically implemented as follows:

[0018] One-dimensional input data augmentation

[0019] RandomAddGaussian: This strategy randomly adds Gaussian noise to the input signal. The formula is as follows:

[0020] x:=x+n (4)

[0021] Where x is a one-dimensional input signal, and n is generated by Gaussian distribution N(0,0.01);

[0022] RandomScale: This strategy randomly multiplies the input signal with a random factor. Its formula is as follows:

[0023] x:=β*x (5)

[0024] Where x is the one-dimensional input signal, β is the scaler after Gaussian distribution N(0,0.01);

[0025] RandomStretch: This strategy resamples the signal into random proportions and ensures equal length by zeroing and truncating;

[0026] RandomCrop: This strategy randomly covers part of the signal. Its formula is as follows:

[0027] x:=mask*x (6)

[0028] Where x is the one-dimensional input signal, and mask is a binary sequence with subsequences of zeros at random positions;

[0029] 2D input data augmentation

[0030] RandomScale: This strategy randomly multiplies the input signal with a random factor. Its formula is as follows:

[0031] x:=β*x (7)

[0032] Where x is the two-dimensional input signal, β is the scaler after Gaussian distribution N(0,0.01);

[0033] RandomCrop: This strategy randomly covers part of the signal. Its formula is as follows:

[0034] x:=mask*x (8)

[0035] Where x is the two-dimensional input signal and mask is a binary sequence with subsequences of zeros at random positions.

[0036] In one embodiment of the present invention, the plurality of parallel backbone network branches include a multi-branch backbone network and an attention module; wherein,

[0037] The multi-branch backbone network consists of convolution branches, identity connections, and batch normalization layers. The convolution kernel sizes of the convolution branches are 1×1 and 3×3, respectively. In each branch, the convolution layer is followed by a batch normalization layer. The 3×3 convolution layer is used to extract more detailed features, and the 1×1 convolution layer is used to reduce the dimension of the features and improve computational efficiency. The features extracted by each branch are fused and integrated to obtain a richer and more complete feature representation. On the one hand, the results of the multi-branch backbone network are summarized and flow into the attention mechanism network. On the other hand, the multi-branch backbone network skips the attention module and proceeds to the next step.

[0038] The attention module includes the channel attention module CAM, the spatial attention module SAM and the external attention module EAM; the channel attention module CAM and the spatial attention module SAM are used in series, and the features output by the previous layer are used as the input of the channel attention module CAM. The weights obtained after passing through the channel attention module CAM are used as the input of the spatial attention module SAM to obtain new feature values ​​to continue participating in model training.

[0039] In one embodiment of the present invention, the channel attention module CAM uses the output of the previous layer network as the input H×W×C of the channel attention module CAM, and obtains new features through the average pooling layer AvgPool and the maximum pooling layer MaxPool, as shown in formula (9), and outputs meaningful features in the original ship signal:

[0040]

[0041] Among them, M c The size of (F) is the number of channels of F, that is, C; the size of the F feature is C×H×W, where C is the number of channels, H is the height, and W is the width; MLP is a multi-layer perceptron operation; σ() represents the Sigmoid activation function; W0 is , r is the reduction factor used to reduce the complexity and computational complexity of MLP; W1 is The matrix, is the C-dimensional global description vector obtained after the average pooling operation, is the C-dimensional global description vector obtained after the maximum pooling operation.

[0042] In one embodiment of the present invention, the spatial attention module SAM obtains the channel attention module CAM features as input, and then selects the pooling layer in the average pooling layer AvgPool and the maximum pooling layer MaxPool for pooling and then performs convolution to obtain the value of the regional features that need to be paid attention to in the original ship signal, as shown in formula (10):

[0043]

[0044] Among them, M sIn (F), F is the input feature map, σ() represents the Sigmoid activation function, and f 7×7 () represents the convolution operation and 7×7 is the convolution kernel size, represents the global average pooling in the channel dimension, Represents the maximum pooling of the channel dimension.

[0045] In one embodiment of the present invention, the external attention module (EAM) consists of two parts: one or more learnable query vectors and an external memory unit that is compared with the input data. First, attention weights are generated by calculating the relationship between the query vector and each memory vector in the external memory unit. Then, the attention weights are applied to different dimensions of the input data using weighted averaging or weighted pooling to generate new adjusted feature representations. The calculation formula is as follows:

[0046] A=(α) i,j =Norm(FM T ) (11)

[0047]

[0048] Among them: a i,j is the similarity between the i-th pixel and the j-th row of M, M is a learnable parameter independent of the input that acts as a memory for the entire training dataset, middle, S It is a flexible parameter. Controlling its size can change the complexity of attention. A is the attention map derived from the dataset-level prior knowledge learned. Norm() is the normalization function. Finally, the input features of M are updated by the similarity in A. F out The updated features.

[0049] Use two different memory cells M k and M v As key and value, Norm() is the normalization function. At this time, the calculation formula of the external attention module EAM is:

[0050] A=Norm(FM k T ) (13)

[0051] F out =AM v (14)

[0052] Double normalization is used to normalize rows and columns. This double normalization is expressed as:

[0053]

[0054]

[0055]

[0056] Compared to existing technologies, the present invention has the following advantages: It proposes a novel underwater acoustic signal recognition method based on a Multi-Branch External Attention Network (MEANet). To overcome the limitations of traditional underwater acoustic signal recognition methods based on shallow learning, the present invention adopts a multi-branch optimal architecture design to fully extract features from different samples and introduces a residual structure to fully utilize time-frequency features to avoid gradient vanishing or gradient exploding. The present invention also introduces channel and spatial attention modules to enhance the diversity of features across different channels, thereby suppressing the interference of ocean background noise and improving the model's applicability. To improve classification recognition rate, the present invention also introduces a novel, lightweight but effective external attention module using two external memory units. This module can learn more representative features from the input, enhancing the model's perception and generalization capabilities, while also reducing computational cost compared to the self-attention module. The present invention also explores the impact of input format and data augmentation on underwater acoustic signal recognition, comparing recognition results for three representative input formats: time domain, frequency domain, and time-frequency domain, as well as several effective data augmentation methods. Experimental results show that in a noisy underwater acoustic background environment, the MEANet proposed in this paper can effectively improve the accuracy of underwater acoustic signal recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 The present invention is summarized.

[0058] Figure 2 This is the time domain diagram of the underwater acoustic signal.

[0059] Figure 3 It is the frequency domain diagram of underwater acoustic signal.

[0060] Figure 4 It is the time-frequency diagram of underwater acoustic signal.

[0061] Figure 5 This is the MEANet model structure diagram.

[0062] Figure 6 Comparison of recognition rates between different models. DETAILED DESCRIPTION

[0063] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0064] The present invention provides an underwater acoustic signal recognition method based on a multi-branch external attention network. First, the input data is passed through multiple parallel backbone network branches to extract feature information at different levels of the underwater acoustic signal. Secondly, the channel and spatial dimensions of the underwater acoustic signal are weighted separately with the assistance of channel and spatial attention modules to adjust the importance of different channels and spatial positions to feature representation. Finally, the external attention module is integrated to guide the network's feature extraction and prediction with external memory units and additional calculations.

[0065] The following is a specific implementation process of the present invention.

[0066] 1. Method Overview

[0067] like Figure 1 As shown in the figure, the method of the present invention mainly includes four steps: data preprocessing, data enhancement, MEANet network training, and final classification results. First, the data is preprocessed to remove blank segments and sliced ​​into time domain signals, frequency domain signals, and time-frequency domain signals. After corresponding data enhancement, these signals are input into the MEANet network for training, and finally the classification results are output.

[0068] 2. Data preprocessing

[0069] like Figure 2 As shown, for time domain input, MATLAB is used to slice the audio file into time domain signals. In the present invention, the length of each sample is 2048, and the total number of samples can be obtained from formula (1). After generating samples, 80% of the total samples are used as the training set and 20% of the total samples are used as the test set.

[0070]

[0071] Where L is the length of each signal, N is the total number of samples, and floor means rounding towards negative infinity.

[0072] like Figure 3 As shown, for frequency domain input, use the FFT function to transform each sample x i Convert from the time domain to the frequency domain as shown in equation (2). After this operation, the data length will be halved and the new sample can be expressed as:

[0073]

[0074] The operator FFT(·) represents the transformation of x i Transform to the frequency domain and take the first half of the result.

[0075] like Figure 4 As shown, for time-frequency domain input, for each sample x iApply the Short-Time Fourier Transform (STFT) to obtain the time-frequency representation shown in Equation (3). Use a Hanning window with a window length of 64. After this operation, the time-frequency representation (a 33×33 image) is generated as follows:

[0076]

[0077] The operator STFT(·) represents the transformation of x i Transform to the time-frequency domain.

[0078] If a very small two-dimensional signal is input to the two-dimensional convolutional neural network, the model will not be able to extract appropriate features. Therefore, in the present invention, the signal size of the STFT is adjusted to 330×330. The time domain and frequency domain inputs remain unchanged.

[0079] 3. Data Augmentation

[0080] 3.1 One-dimensional input data enhancement

[0081] RandomAddGaussian: This strategy randomly adds Gaussian noise to the input signal. The formula is as follows:

[0082] x:=x+n (4)

[0083] Where x is the one-dimensional input signal and n is generated by Gaussian distribution N(0,0.01).

[0084] RandomScale: This strategy randomly multiplies the input signal with a random factor. Its formula is as follows:

[0085] x:=β*x (5)

[0086] Where x is the one-dimensional input signal and β is the scaler following the Gaussian distribution N(0,0.01).

[0087] RandomStretch: This strategy resamples the signal into random proportions and ensures equal length by zeroing and truncating.

[0088] RandomCrop: This strategy randomly covers part of the signal. Its formula is as follows:

[0089] x:=mask*x (6)

[0090] Where x is a one-dimensional input signal, and mask is a binary sequence whose subsequences at random positions are zero. In the present invention, the length of the subsequence is 10.

[0091] 3.2 Two-dimensional input data enhancement

[0092] RandomScale: This strategy randomly multiplies the input signal with a random factor. Its formula is as follows:

[0093] x:=β*x (7)

[0094] Where x is the two-dimensional input signal and β is the scaler following the Gaussian distribution N(0,0.01).

[0095] RandomCrop: This strategy randomly covers part of the signal. Its formula is as follows:

[0096] x:=mask*x (8)

[0097] Where x is a two-dimensional input signal, and mask is a binary sequence whose subsequences at random positions are zero. In the present invention, the length of the subsequence is 20.

[0098] 4. Model Architecture

[0099] In order to fully acquire the characteristics of underwater acoustic signals and identify them in complex ocean environments, the present invention designs a multi-branch network to extract underwater acoustic signal information, which mainly consists of a backbone network based on a multi-branch structure and an attention module.

[0100] 4.1 Multi-branch backbone network

[0101] like Figure 5 As shown in , the multi-branch backbone consists of convolution branches, identity connections, and batch normalization layers. The convolution kernel sizes of the convolution branches are 1×1 and 3×3, respectively. In each branch, the convolution layer is followed by a batch normalization layer. The 3×3 convolution layer can be used to extract more detailed features, and the 1×1 convolution layer can be used to reduce the dimension of the features and improve computational efficiency. In this way, each branch can extract some unique features, and then fuse and integrate these features to obtain a richer and more complete feature representation, thereby improving the performance of the network and improving modularity and scalability. At the same time, the results summarized by the backbone network can flow into the attention mechanism network. On the other hand, it skips the attention module and enters the next operation. The structure of the multi-branch backbone can be stacked multiple times, such as Figure 5 Stage 1 to stage 4 in the figure are stacked with 4, 6, 16 and 1 layers respectively.

[0102] 4.2 Attention Mechanism Module

[0103] The attention mechanism simulates the human visual system, selectively focusing on a part of key information and ignoring other redundant information. It has gradually become a development direction for improving deep neural networks. So far, the attention module has been proven to help improve the accuracy of various tasks. [1][2][3][4]The present invention couples the attention module into the backbone network. It includes the channel attention module (CAM), the spatial attention module (SAM) and the external attention module (EAM) [5] .

[0104] In the present invention, the channel attention and spatial attention modules are used in series. The features output by the previous layer are used as the input of the channel attention module. The weights obtained after the channel attention module are used as the input of the spatial attention module to obtain new feature values ​​to continue participating in model training.

[0105] Channel attention module: In convolutional neural networks, the number of channels is a critical factor, and the model usually relies on a large number of channel features for learning. However, in practical applications, some of the features in the channel may be redundant noise or other useless information. The channel attention module adaptively learns the weight of each channel and weights the channels of the input features, so that the network pays more attention to those important feature channels, thereby improving the performance of the network. Specifically, the output of the previous layer of the network is used as the input H×W×C of the channel attention module, and new features are obtained through the average pooling layer (AvgPool) and the maximum pooling layer (MaxPool), as shown in formula (9), outputting meaningful features in the original ship signal:

[0106]

[0107] Among them, M c The size of (F) is the number of channels of F, that is, C; the size of the F feature is C×H×W; σ() represents the Sigmoid activation function; W0 is The matrix of The matrix of .

[0108] Spatial attention module: This module focuses on the importance of different spatial positions on the original feature map. Based on the attention mechanism, it adaptively learns the weight of each spatial position to enhance the feature information of those important positions. This module mainly obtains the channel attention module features as input, and then selects the appropriate pooling layer in the average pooling layer (AvgPool) and the maximum pooling layer (MaxPool) for pooling and then performs convolution. The obtained value can indicate which regional features in the original ship signal are worthy of attention, as shown in formula (10):

[0109]

[0110] Among them, σ() represents the Sigmoid activation function, f 7×7 () represents the convolution operation and 7×7 is the convolution kernel size.

[0111] External attention module: Unlike channel attention and spatial attention, external attention is not generated inside the feature map, but relies on external information and additional computation to guide the network's feature extraction and prediction. Specifically, external attention consists of two parts: one or more learnable query vectors and an external memory unit that is compared with the input data. First, attention weights are generated by calculating the relationship between the query vector and each memory vector in the external memory unit. Then, the attention weights are applied to different dimensions of the input data using weighted averaging or weighted pooling to generate new adjusted feature representations. The calculation formula is as follows:

[0112] A=(α) i,j =Norm(FM T ) (11)

[0113]

[0114] Among them: a i,j is the similarity between the i-th pixel and the j-th row of M, where M is a learnable parameter independent of the input and acts as a memory for the entire training dataset. A is the attention map derived from the dataset-level prior knowledge learned from it. Finally, the input features of M are updated by the similarity in A.

[0115] In the present invention, two different storage units M are used. k and M v As key and value to improve the performance of the network. At this time, the calculation formula of external attention is:

[0116] A=Norm(FM k T ), (13)

[0117] F out =AM v . (14)

[0118] Unlike cosine similarity, since the attention map is calculated by matrix multiplication, the attention map is sensitive to the scale of the input features. [6] Softmax is used to double normalize the attention map so that ∑ j α i,j = 1. To avoid this problem, we choose the literature [7] The double normalization proposed in

[15] normalizes both rows and columns. This double normalization is expressed as:

[0119]

[0120]

[0121]

[0122] 5. Experimental data and evaluation

[0123] All underwater acoustic signal data are from the ShipsEar dataset [8]. This dataset provides the sounds of many different types of ships recorded by researchers on the Atlantic coast of Spain during 2012 and 2013. A total of 91 data points are included for 11 types of ships and one background noise. The length of each underwater acoustic signal data is between 15 seconds and 10 minutes, and the sampling frequency is 52734Hz. After removing the blank segments and the excessive interference, the audio signal is converted into a time domain signal using MATLAB, and all the sound segments are segmented according to a fixed 2048 sampling points, resulting in a total of 93409 labeled sound samples. The detailed classification of the radiated noise of different types of ships is shown in Table 1.

[0124] Table 1 Dataset classification

[0125]

[0126]

[0127] Many researchers use signal processing methods to map time series to different domains to improve performance. However, which input type is most suitable for underwater acoustic target recognition remains an open question. This paper discusses the impact of different input types on model performance. When the input is time-domain data and frequency-domain data, the present invention rewrites the model by rewriting the two-dimensional convolutional layers, pooling layers, and normalization layers into one-dimensional versions to adapt to the input changes. Experimental results show that the present method is still effective when the input is one-dimensional data.

[0128] 5.1 Qualitative comparison

[0129] Comparison of Different Input Formats: We first compared the performance of the model using the time domain, frequency domain, and time-frequency domain as input. We divided 80% of the 93,409 data items from five categories into a training set and 20% into a test set. The training settings were as follows: batch_size of 64 for one-dimensional signals and 32 for two-dimensional signals. The optimizer was adamW, with an initial learning rate of 0.0001 and 100 iterations. We compared CNN, BiLSTM, and ResNet. The results are shown in Table 2.

[0130] Table 2 Recognition results of different input formats

[0131]

[0132] Table 2 shows that when using CNN and BiLSTM as the classification methods, the time-domain signal as input achieves the best classification results, at 92.28% and 95.27%, respectively. Meanwhile, when using ResNet and MEANet as the classification methods, the time-frequency domain input (STFT) achieves the best results, at 97.76% and 98.84%, respectively. This means that shallower models like CNN and BiLSTM are better at processing one-dimensional time-domain signals and are less effective with more complex images. Deeper ResNet and MEANet models, on the other hand, are able to extract more useful information from more informative two-dimensional images and achieve higher recognition rates.

[0133] 5.2 Quantitative comparison

[0134] Comparison of Data Augmentation with and without Data Augmentation: Tables 3 through 5 show that data augmentation outperforms the results without data augmentation in both the time and time-frequency domains, while the results in the frequency domain deviate. When using the frequency domain as input, the CNN model performs better with data augmentation, while BiLSTM, ResNet, and MEANet perform worse than without data augmentation. This may be because the frequency domain signal data segments are too short, resulting in less significant features. Adding noise, multiplying by random factors, and covering transformations can mislead the model, resulting in a decrease in recognition rate.

[0135] Table 3 Comparison of time domain signal data enhancement

[0136]

[0137] Table 4 Comparison of frequency domain signal data enhancement

[0138]

[0139] Table 5 Comparison of signal data enhancement in time and frequency domains

[0140]

[0141] 5.3 Application Test

[0142] Table 6 Recognition results without data enhancement

[0143]

[0144] Table 7 Recognition results with data enhancement

[0145]

[0146] Comparison between models: When comparing the performance between models, we additionally introduce ResNet50 for comparison when the input type is a time-frequency graph. From Tables 6, 7, and Figure 6 It can be seen that the model proposed in this invention achieves the best results in all three input types and whether data enhancement is performed. When STFT is used as input and data enhancement is adopted, the recognition rate reaches 98.84%, which fully demonstrates that the MEANet proposed in this invention has better performance and the effectiveness of data enhancement in most cases.

[0147] References:

[0148] [1]Xie S, Liu S, Chen Z, et al. Attentional shapecontextnet for pointcloud recognition [C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 4606-4615.

[0149] [2]Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2018:7132-7141.

[0150] [3]Woo S,Park J,Lee JY,et al.Cbam:Convolutional block attentionmodule[C] / / Proceedings of the European conference on computer vision(ECCV).2018:3-19.

[0151] [4]Chen L, Zhang H, Xiao J, et al. Sca-cnn: Spatial and channel-wiseattention in convolutional networks for image captioning[C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2017:5659-5667.

[0152] [5]Guo MH, Liu ZN, Mu TJ, et al.Beyond self-attention:Externalattention using two linear layers for visual tasks[J]. IEEE Transactions onPattern Analysis and Machine Intelligence, 2022.

[0153] [6]Shaw P,Uszkoreit J,Vaswani A.Self-attention with relative positionrepresentations[J].arXiv preprint arXiv:1803.02155,2018.

[0154] [7]Guo MH,Cai JX,Liu ZN,et al.Pct:Point cloud transformer[J].Computational Visual Media,2021,7:187-199.

[0155] [8] Santos-Domínguez D, Torres-Guijarro S, Cardenal-López A, etal. Ships Ear: An underwater vessel noise database [J]. Applied Acoustics, 2016, 113: 64-69.

[0156] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions and effects do not exceed the scope of the technical solution of the present invention, shall fall within the scope of protection of the present invention.

Claims

1. A method for underwater acoustic signal recognition based on a multi-branch external attention network, characterized in that: First, the input data is passed through multiple parallel backbone network branches to extract feature information at different levels of the underwater acoustic signal; Secondly, the channel and spatial attention modules are used to weight the channel and spatial dimensions of the underwater acoustic signal respectively, and adjust the importance of different channels and spatial positions to feature representation; finally, the external attention module is integrated to guide the feature extraction and prediction of the network with external memory units and additional calculations; the multiple parallel backbone network branches include a multi-branch backbone network and an attention module; wherein, The multi-branch backbone network consists of convolution branches, identity connections, and batch normalization layers. The convolution kernel sizes of the convolution branches are 1×1 and 3×3, respectively. In each branch, the convolution layer is followed by a batch normalization layer. The 3×3 convolution layer is used to extract more detailed features, and the 1×1 convolution layer is used to reduce the dimension of the features and improve computational efficiency. The features extracted by each branch are fused and integrated to obtain a richer and more complete feature representation. On the one hand, the results of the multi-branch backbone network are summarized and flow into the attention mechanism network. On the other hand, the multi-branch backbone network skips the attention module and proceeds to the next step. The attention module includes the channel attention module CAM, the spatial attention module SAM and the external attention module EAM; the channel attention module CAM and the spatial attention module SAM are used in series, and the features output by the previous layer are used as the input of the channel attention module CAM. The weights obtained after passing through the channel attention module CAM are used as the input of the spatial attention module SAM to obtain new feature values ​​to continue participating in model training.

2. The underwater acoustic signal recognition method based on a multi-branch external attention network according to claim 1 is characterized in that: Before being input into multiple parallel backbone network branches, the input data needs to undergo data preprocessing and data enhancement operations. That is, the input data undergoes data preprocessing operations to remove blank segments and slice it into time domain signals, frequency domain signals, and time-frequency domain signals. After corresponding data enhancement, it is input into multiple parallel backbone network branches for training.

3. The underwater acoustic signal recognition method based on a multi-branch external attention network according to claim 2 is characterized in that: The data preprocessing is specifically implemented as follows: For time domain input, MATLAB is used to slice the audio file into time domain signals. The length of each sample of the input data is 2048. The total number of samples is obtained from formula (1). After generating samples, 80% of the total samples are used as training sets and 20% of the total samples are used as test sets. Where L is the length of each signal, N is the total number of samples, and floor means rounding towards negative infinity; For frequency domain input, use the FFT function to transform each sample x of the input data into i Converting from the time domain to the frequency domain as shown in equation (2), the data length is halved and the new sample is expressed as: The operator FFT(·) represents the transformation of x i Transform to the frequency domain and take the first half of the result; For time-frequency domain input, for each sample x i Apply the short-time Fourier transform (STFT) to obtain the time-frequency representation shown in equation (3). Use a Hanning window with a window length of 64 to generate the time-frequency representation as follows: The operator STFT(·) represents the transformation of x i Transform to the time-frequency domain.

4. The underwater acoustic signal recognition method based on a multi-branch external attention network according to claim 3 is characterized in that: For the time-frequency domain input, the signal size of STFT is adjusted to 330×330, and the time domain and frequency domain inputs remain unchanged.

5. The underwater acoustic signal recognition method based on a multi-branch external attention network according to claim 2 is characterized in that: The data enhancement is specifically implemented as follows: One-dimensional input data augmentation RandomAddGaussian: This strategy randomly adds Gaussian noise to the input signal. The formula is as follows: x:=x+n (4) Where x is a one-dimensional input signal, and n is generated by Gaussian distribution N(0,0.01); RandomScale: This strategy randomly multiplies the input signal with a random factor. Its formula is as follows: x:=β*x (5) Where x is the one-dimensional input signal, β is the scaler after Gaussian distribution N(0,0.01); RandomStretch: This strategy resamples the signal into random proportions and ensures equal length by zeroing and truncating; RandomCrop: This strategy randomly covers part of the signal. Its formula is as follows: x:=mask*x (6) Where x is the one-dimensional input signal, and mask is a binary sequence with subsequences of zeros at random positions; 2D input data augmentation RandomScale: This strategy randomly multiplies the input signal with a random factor. Its formula is as follows: x:=β*x (7) Where x is the two-dimensional input signal, β is the scaler after Gaussian distribution N(0,0.01); RandomCrop: This strategy randomly covers part of the signal. Its formula is as follows: x:=mask*x (8) Where x is the two-dimensional input signal and mask is a binary sequence with subsequences of zeros at random positions.

6. The underwater acoustic signal recognition method based on a multi-branch external attention network according to claim 1 is characterized in that: The channel attention module CAM takes the output of the previous layer of network as the input H×W×C of the channel attention module CAM, and obtains new features through the average pooling layer AvgPool and the maximum pooling layer MaxPool, as shown in formula (9), and outputs meaningful features in the original ship signal: Among them, M c The size of (F) is the number of channels of F, that is, C; the size of the F feature is C×H×W, where H is the height and W is the width; MLP is the multi-layer perceptron operation; σ() represents the Sigmoid activation function; W0 is , r is the reduction factor; W1 is The matrix, is the C-dimensional global description vector obtained after the average pooling operation, is the C-dimensional global description vector obtained after the maximum pooling operation.

7. The underwater acoustic signal recognition method based on a multi-branch external attention network according to claim 6 is characterized in that: The spatial attention module SAM obtains the channel attention module CAM features as input, and then selects the pooling layer in the average pooling layer AvgPool and the maximum pooling layer MaxPool for pooling and then performs convolution to obtain the value of the regional features that need to be paid attention to in the original ship signal, as shown in formula (10): Among them, M s In (F), F is the input feature map, σ() represents the Sigmoid activation function, and f 7×7 () represents the convolution operation and 7×7 is the convolution kernel size, represents the global average pooling in the channel dimension, Represents the maximum pooling of the channel dimension.

8. The underwater acoustic signal recognition method based on a multi-branch external attention network according to claim 7 is characterized in that: The external attention module (EAM) consists of two parts: one or more learnable query vectors and an external memory unit that is compared with the input data. First, attention weights are generated by calculating the relationship between the query vector and each memory vector in the external memory unit. Then, the attention weights are applied to different dimensions of the input data using weighted averaging or weighted pooling to generate new adjusted feature representations. The calculation formula is as follows: A= (a) i,j =Norm(FM T ) (11) Where: (α) i,j is the similarity between the i-th pixel and the j-th row of M, M is a learnable parameter independent of the input that acts as a memory for the entire training dataset, middle, S is a flexible parameter, A is the attention map derived from the dataset-level prior knowledge learned, Norm() is the normalization function, and finally, the input features of M are updated by the similarity in A, F out For the updated features; Use two different memory cells M k and M v As key and value, Norm() is the normalization function. At this time, the calculation formula of the external attention module EAM is: A=Norm(FM k T ) (13) F out =AM v (14) Double normalization is used to normalize rows and columns. This double normalization is expressed as:

Images