Underwater acoustic target recognition method, device, equipment, medium and product based on joint neural network of 3D fusion features

Abstract

This application discloses a method, apparatus, device, medium, and product for underwater acoustic target recognition based on a joint neural network using 3D fusion features, relating to the field of underwater acoustic recognition. The underwater acoustic target recognition method includes: extracting features from underwater audio signals to obtain 3D Mel-frequency scale features and 3D Mel-frequency cepstral coefficient features; fusing the 3D Mel-frequency scale features and 3D Mel-frequency cepstral coefficient features to obtain 3D fused features; and inputting the 3D fused features into a trained joint neural network model for underwater acoustic target recognition. This application achieves high accuracy in underwater acoustic recognition.

Description

Technical Field

[0001] The present application relates to the field of underwater acoustic recognition, and in particular to an underwater acoustic target recognition method, device, equipment, medium and product based on a joint neural network with three-dimensional fusion features. Background Art

[0002] Underwater acoustic target recognition primarily relies on the analysis of ship-radiated noise. This noise signal contains rich target feature information. Through in-depth analysis of its characteristics, underwater target classification and identification can be effectively achieved. Traditional underwater acoustic target recognition methods based on ship-radiated noise utilize artificially designed features and shallow classifiers to classify ship types, focusing on feature extraction and the development of nonlinear classifiers. For feature extraction, techniques such as Mel-frequency cepstral coefficients (MFCC), constant Q transform (CQT), wavelet features, detection of envelope modulation on noise (DEMON), low frequency analysis and recording (LOFAR) spectra, and advanced spectral features have been used. Existing feature extraction techniques focus on static features and ignore dynamic features, which can reduce underwater acoustic recognition accuracy.

[0003] Furthermore, deep learning can directly absorb features from the original signal, effectively filtering out redundant data and reducing the computational requirements of subsequent models. Existing research on passive underwater acoustic target feature recognition based on deep learning is still in its early stages, primarily focusing on theoretical exploration and small-scale experiments. Existing deep neural networks have a large number of parameters and require integration with big data to ensure performance. However, in underwater acoustic recognition applications, it is often difficult to collect sufficient data for model training, which significantly limits the performance of deep neural networks and makes it impossible to guarantee the accuracy of underwater acoustic recognition. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, equipment, medium and product for underwater acoustic target recognition based on a joint neural network of 3D fusion features, which can improve the accuracy of underwater acoustic recognition.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides an underwater acoustic target recognition method based on a joint neural network with 3D fusion features, the underwater acoustic target recognition method comprising:

[0007] Feature extraction is performed on the underwater audio signal to obtain 3D Mel-spectrogram features and 3D Mel-frequency cepstral coefficient features; the 3D Mel-spectrogram features include: Mel-spectrogram and the first-order difference and second-order difference of the Mel-spectrogram; the 3D Mel-frequency cepstral coefficient features include: Mel-frequency cepstral coefficient and the first-order difference and second-order difference of the Mel-frequency cepstral coefficient;

[0008] The 3D Mel spectrum features and the 3D Mel frequency cepstral coefficient features are fused to obtain the 3D fusion features;

[0009] The 3D fusion features are input into a trained joint neural network model for underwater acoustic target recognition; the joint neural network model includes a serially connected multi-scale depth-separable convolutional network module, a multi-scale channel attention mechanism and a long short-term memory network module.

[0010] Optionally, feature extraction is performed on the underwater audio signal to obtain 3D Mel-spectrogram features and 3D Mel-frequency cepstral coefficient features, specifically including:

[0011] The underwater audio signal is framed using the Hanning window to obtain multiple underwater audio segments;

[0012] Perform fast Fourier transform on each underwater acoustic audio band to obtain the frequency domain characteristics of each underwater acoustic audio band;

[0013] The Mel filter is used to capture the frequency domain features of each underwater sound frequency band and obtain the Mel spectrum of each underwater sound frequency band.

[0014] Calculate the first-order difference and second-order difference of the Mel-spectrogram of each underwater sound frequency band;

[0015] Perform logarithmic calculation and discrete Fourier transform on the Mel frequency spectrum of each underwater sound frequency band to obtain the Mel frequency cepstrum coefficient of each underwater sound frequency band;

[0016] Calculate the first-order and second-order differences of the Mel-frequency cepstral coefficients of each underwater acoustic audio band.

[0017] Optionally, the joint neural network model further includes an input layer, a two-dimensional convolutional layer, a maximum pooling layer, a flattening layer, a reshaping layer, a first dropout layer, a first fully connected layer, a second dropout layer, and a second fully connected layer;

[0018] The input layer is connected to the two-dimensional convolutional layer, the two-dimensional convolutional layer is connected to the multi-scale depth-wise separable convolutional network module, the multi-scale depth-wise separable convolutional network module is connected to the maximum pooling layer, the maximum pooling layer is connected to the flattening layer, the flattening layer is connected to the reshaping layer, the reshaping layer is connected to the long short-term memory network module, the long short-term memory network module is connected to the first dropout layer, the first dropout layer is connected to the first fully connected layer, the first fully connected layer is connected to the second dropout layer, and the second dropout layer is connected to the second fully connected layer.

[0019] Optionally, the multi-scale depth-wise separable convolutional network module includes three multi-scale depth-wise separable convolutional network layers, and the long short-term memory network module includes three long short-term memory network layers.

[0020] Optionally, the multi-scale channel attention mechanism includes: a global average pooling layer, three one-dimensional convolutional layers, an additive fusion layer, an activation function layer, and a multiplicative fusion layer;

[0021] The three one-dimensional convolutional layers are arranged in parallel, the input ends of the three one-dimensional convolutional layers are connected to the global average pooling layer, the output ends of the three one-dimensional convolutional layers are connected to the additive fusion layer, the additive fusion layer is connected to the activation function layer, and the activation function layer is connected to the multiplicative fusion layer;

[0022] The multiplication fusion layer is used to multiply the attention of each channel output by the activation function layer with the input feature map to obtain an attention feature map; the input feature map is the feature map input to the multi-scale channel attention mechanism.

[0023] In a second aspect, the present application provides an underwater acoustic target recognition device based on a joint neural network with 3D fusion features, wherein the underwater acoustic target recognition device applies the above-mentioned underwater acoustic target recognition method, and the underwater acoustic target recognition device comprises:

[0024] A feature extraction module is used to extract features from underwater audio signals to obtain 3D Mel-spectrogram features and 3D Mel-frequency cepstral coefficient features; the 3D Mel-spectrogram features include: Mel-spectrogram and the first-order and second-order differences of the Mel-spectrogram; the 3D Mel-frequency cepstral coefficient features include: Mel-frequency cepstral coefficients and the first-order and second-order differences of the Mel-frequency cepstral coefficients;

[0025] The feature fusion module is used to fuse the 3D Mel-frequency spectrum features and the 3D Mel-frequency cepstral coefficient features to obtain 3D fusion features;

[0026] The target recognition module is used to input the 3D fusion features into a trained joint neural network model for underwater acoustic target recognition; the joint neural network model includes a serially connected multi-scale depth-separable convolutional network module, a multi-scale channel attention mechanism and a long short-term memory network module.

[0027] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned underwater acoustic target recognition method.

[0028] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned underwater acoustic target recognition method when executed by a processor.

[0029] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the above-mentioned underwater acoustic target recognition method when executed by a processor.

[0030] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0031] The present application provides a method, device, equipment, medium and product for underwater acoustic target recognition based on a joint neural network of 3D fusion features. The original underwater acoustic audio signal of the present application is converted into a 3D fusion feature (3D fusion feature) after fusing 3D Mel spectrum features (3D-Mel features) and 3D Mel frequency cepstral coefficient features (3D-MFCC features), thereby capturing the original signal target and its dynamic information. In the classification task, a multi-scale deep separable convolutional network replaces the CNN in the serial architecture of the traditional convolutional neural network (Cable News Network, CNN) and the long short-term memory network (Long Short-Term Memory, LSTM) to reduce the number of parameters of the model, thereby improving computational efficiency and reducing the number of samples required for training. In addition, in order to further enhance the recognition ability of the model, a multi-scale channel attention mechanism (Multi-Scale Channel Attention, MSCA) is integrated into the above-mentioned optimized serial architecture, so that the model can more accurately capture key speech features and enhance the overall performance of the model. The present application can improve the accuracy of underwater acoustic recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0033] Figure 1 This is an application environment diagram of an underwater acoustic target recognition method based on a joint neural network of 3D fusion features in one embodiment of the present application;

[0034] Figure 2 A flowchart of an underwater acoustic target recognition method based on a joint neural network with 3D fusion features provided in one embodiment of the present application;

[0035] Figure 3 A schematic diagram of a process for feature extraction and fusion of underwater audio signals provided in one embodiment of the present application;

[0036] Figure 4 A schematic diagram of the overall structure of a joint neural network model provided in one embodiment of the present application;

[0037] Figure 5 A schematic diagram of the structure of a multi-scale depth-separable convolutional network layer provided in one embodiment of the present application;

[0038] Figure 6 A schematic diagram of the structure of a multi-scale channel attention mechanism provided in one embodiment of the present application;

[0039] Figure 7 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0041] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0042] The underwater acoustic target recognition method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown, the terminal 102 communicates with the server 104 via a network. The data storage system can store data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send an underwater acoustic audio signal to the server 104. The server 104 receives the underwater acoustic audio signal and performs feature extraction on the underwater acoustic audio signal to obtain 3D Mel-spectrogram features and 3D Mel-frequency cepstral coefficient features. The server 104 fuses the 3D Mel-spectrogram features and 3D Mel-frequency cepstral coefficient features to obtain 3D fused features. The 3D fused features are input into a trained joint neural network model for underwater acoustic target recognition. The server 104 can feed back the obtained underwater acoustic target recognition results to the terminal 102. In addition, in some embodiments, the underwater acoustic target recognition method can also be implemented independently by the server 104 or the terminal 102. For example, the terminal 102 can directly process the underwater acoustic audio signal, or the server 104 can obtain the underwater acoustic audio signal from the data storage system and process the underwater acoustic audio signal.

[0043] Terminal 102 may include, but is not limited to, various desktop computers, laptops, smartphones, tablet computers, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, and smart car devices. Portable wearable devices may include smart watches, smart bracelets, and head-mounted devices. Server 104 may be implemented as a standalone server or a server cluster consisting of multiple servers, or may be a cloud server.

[0044] In an exemplary embodiment, Figure 2 As shown, a method for underwater acoustic target recognition is provided. The method is executed by a computer device, specifically a computer device such as a terminal or a server, or a terminal and a server. In the embodiment of the present application, the method is applied to Figure 1 Taking the server 104 in FIG. 1 as an example, the method includes the following steps 201 to 203. In which:

[0045] Step 201 , extract features of the underwater audio signal to obtain 3D Mel-spectrogram features and 3D Mel-frequency cepstral coefficient features; the 3D Mel-spectrogram features include: the Mel-spectrogram and the first-order difference and second-order difference of the Mel-spectrogram; the 3D Mel-frequency cepstral coefficient features include: the Mel-frequency cepstral coefficient and the first-order difference and second-order difference of the Mel-frequency cepstral coefficient.

[0046] Step 202 : Fusing the 3D Mel-spectrogram feature and the 3D Mel-frequency cepstral coefficient feature to obtain a 3D fused feature.

[0047] In step 203, the three-dimensional fusion features are input into a trained joint neural network model for underwater acoustic target recognition; the joint neural network model includes a serially connected multi-scale depth-separable convolutional network module, a multi-scale channel attention mechanism, and a long short-term memory network module.

[0048] Implementing the above steps 201 to 203 can improve the accuracy of underwater acoustic recognition.

[0049] A comparative analysis found that 3D dynamic Mel-frequency Cepstrum Coefficient (3D MFCC) outperformed MFCC, Mel-spectrogram, 3D dynamic Mel-spectrogram, and CQT in terms of performance. This application adds 3D-Mel to 3D-MFCC feature extraction as a supplement to enrich the feature representation of speech signals.

[0050] like Figure 3 As shown, feature extraction is performed on the underwater acoustic audio signal to obtain 3D Mel spectrum features and 3D Mel frequency cepstral coefficient features, specifically including: using a Hanning window to frame the underwater acoustic audio signal to obtain multiple underwater acoustic audio segments; performing fast Fourier transform on each underwater acoustic audio segment to obtain the frequency domain features of each underwater acoustic audio segment; using a Mel filter to capture the frequency domain features of each underwater acoustic audio segment to obtain the Mel spectrum of each underwater acoustic audio segment; calculating the first-order difference and second-order difference of the Mel spectrum of each underwater acoustic audio segment; performing logarithm calculation and discrete Fourier transform on the Mel spectrum of each underwater acoustic audio segment to obtain the Mel frequency cepstral coefficient of each underwater acoustic audio segment; and calculating the first-order difference and second-order difference of the Mel frequency cepstral coefficient of each underwater acoustic audio segment.

[0051] Specifically, during the feature extraction process, the original underwater audio signal saved in WAV format was preprocessed by removing blank segments and reducing noise, and the sample audio length was cut into 5-second segments. By using Python libraries such as the operating system library os, the file matching library glob, the scientific computing library numpy, the audio processing library librosa, and the audio reading and writing library soundfile, we wrote related functions to extract features from the audio file and calculate the following features:

[0052] Mel-spectrogram (Mel-spectrogram), first-order and second-order differences of Mel-spectrogram (delta and double delta of log-mel spectrogram), Mel-frequency cepstral coefficients (MFCC), first-order and second-order differences of MFCC (delta and double delta of MFCC).

[0053] Among them, the Mel spectrogram is a spectrogram commonly used in the field of audio processing. It is closely related to the Mel scale. The Mel spectrogram is a representation of an audio signal, which maps the spectral content of the signal to the Mel scale.

[0054] The Mel frequency scale is a perceptual frequency scale used for audio signals that simulates the human ear's frequency perception. The human ear's sensitivity to sounds of different frequencies is not linear, especially in the low-frequency region. The Mel frequency scale converts Hertz (Hz) to Mel (Mel) using the following formula:

[0055]

[0056] Where M is the mel frequency and f is the frequency in Hertz.

[0057] The main feature of this scale is that it is relatively dense in the low-frequency area and relatively sparse in the high-frequency area, which matches the human ear's auditory perception of frequency.

[0058] The Mel frequency scale provides the theoretical basis and implementation framework for the construction of the Mel spectrogram, which is a visualization tool that re-represents the audio signal according to the Mel frequency scale.

[0059] The Mel frequency scale is a nonlinear frequency scale based on human frequency perception. The human ear perceives low and high frequencies differently, with low-frequency variations being more noticeable. The y-axis of a Mel spectrogram uses the Mel spectrum instead of the traditional frequency to better simulate the human auditory characteristics.

[0060] During the feature fusion process, the numpy.dstack function is used to stack the original Mel-frequency spectrogram, first-order differences, and second-order differences into a 3D array, resulting in 3D Mel-frequency scale features. The numpy.dstack function is also used to stack the MFCCs and their differences into another 3D array, resulting in 3D Mel-frequency cepstral coefficient features. This stacking method allows the model to take into account both the static and dynamic characteristics of the audio signal. Finally, the 3D Mel-frequency scale features and the 3D Mel-frequency cepstral coefficient features are merged in the last dimension to form a larger feature array. This operation is the key to feature fusion, as it integrates features of different types and dimensions as input to subsequent models, providing the model with richer information.

[0061] Finally, the extracted 3D fusion features of each audio signal and their corresponding labels are saved as NPY-formatted feature files as input for subsequent recognition modules. The first two modules aim to provide a comprehensive feature representation for subsequent audio recognition analysis tasks, thereby improving the performance and generalization capabilities of the model.

[0062] Mel-spectrograms and Mel-frequency cepstral coefficients are commonly used acoustic feature extraction methods. They simulate human auditory perception of sound frequencies and capture key speech characteristics. However, traditional spectral features primarily capture static spectral envelope information and lack dynamic signal representation. This embodiment bridges this gap by calculating the first- and second-order differences of the Mel-spectrogram and MFCC.

[0063] First, the delta (first-order difference) and double_delta_ (second-order difference) features of the Mel spectrum and MFCCs are extracted to construct a three-dimensional feature space. Then, the extracted 3D-MFCC and 3D-Mel features are fused to form a more comprehensive and rich feature representation. The whole process is as follows Figure 3 As shown in Figure 2, the detailed steps of feature extraction are shown.

[0064] Since the original features are static, incremental features and dual incremental features are added to add dynamic information to the static features, forming multidimensional dynamic features. This is performed by locally estimating the difference operation of the input features along the time axis. Incremental features and dual incremental features provide dynamic information about the original features over time.

[0065] The delta coefficient of the Mel-spectrogram can be obtained by formula (1):

[0066]

[0067] Among them, c tis the static Mel spectrum cepstral coefficient of frame t, and n is the number of adjacent frames. Through the difference operation, D t is determined by the static coefficient c t+n and c t-n The calculated delta coefficient of the Mel-spectrogram. N represents the difference dimension of the time-spectrogram. The same formula can also be used to calculate the delta-delta coefficient, but they are calculated based on delta. As mentioned above, the delta coefficient of the Mel-spectrogram at frame t is D t , then the delta-delta coefficient D of the corresponding mel-spectrogram t ' is defined as follows:

[0068] D t '=D t+n -D t-n (2)

[0069] Where n represents the number of adjacent frames. t ' represents the delta-delta coefficient of the Mel-spectrogram at the tth frame,

[0070] The three-dimensional spectrogram of the Mel-spectrogram is defined as:

[0071] Mel_3D∈R F×T×C

[0072] Where F represents the number of Mel filter bins, T represents the number of time dimensions, and C represents the dimensionality of the spectrogram. When C = 1, it represents a one-dimensional Mel-spectrogram, C = 2 represents a first-order Mel-spectrogram, and C = 3 represents a second-order Mel-spectrogram.

[0073] Similar to the above Mel difference operation, the difference operation of MFCCs (Mel-frequency cepstral coefficients) features along the time axis can obtain delta MFCC (first-order difference of Mel-frequency cepstral coefficients) and double-delta MFCC (second-order difference of Mel-frequency cepstral coefficients).

[0074] During the feature extraction process, the frame length is set to 2048, and the Hanning window function is applied before performing the fast Fourier transform (FFT) on each frame signal to reduce spectral leakage. The length of the applied Hanning window is the same as the frame length, that is, 2048 samples. In addition, a filter group containing 40 filters is configured to capture different frequency ranges. After these processing steps, the 3D fusion features finally obtained by this application have a shape of (40×34×6). Table 1 shows the dimensional information of these features in detail.

[0075] Table 1 Dimension information of features

[0076]

[0077] In the process of underwater acoustic signal recognition, the feature file saved in NPY format is used as input. The feature file stores the above-mentioned 3D fusion features. The 3D fusion features and their labels are randomly divided into a training set and a test set in a ratio of 4:1.

[0078] The joint neural network model of this embodiment replaces the traditional CNN with a multi-scale depth-separable convolutional network (MSDC) based on the serial architecture of the convolutional network (CNN) and the long short-term memory network (LSTM), and adds a multi-scale channel attention mechanism (MSCA). Its structure is as follows: Figure 4 The parameters of the model are shown in Table 2.

[0079] Table 2 Joint network model

[0080]

[0081]

[0082] The joint neural network model proposed in this embodiment includes:

[0083] Input layer (Input): The input data shape is (None, 40, 34, 6), where None represents the batch size (data set outside the model does not affect the data structure within the model, similar to the number of loops), 40 and 34 are the height and width of the image respectively, and 6 is the number of channels.

[0084] Conv2D: There is no additional padding, so the spatial dimensions (height and width) are reduced by 2 (because the convolution kernel covers the edges of the input), and the output shape becomes (None, 38, 32, 64). The number of input channels is 6, and the number of output channels is set to 64.

[0085] Three multi-scale depthwise separable convolutional network layers (MSDM, MSDM_1, MSDM_2): In the MSDM layer, the number of channels does not change. The MSDM layer consists of several branches, each of which is output through a ReLU activation function, but there is no operation that changes the number of channels between branches or at the end of a branch.

[0086] The MSDM consists of four sub-branches, each of which operates on its input:

[0087] Branch 1: Using 7x7 depth-wise separable convolution, the spatial dimension of the output is halved to (None, 19, 16, 1*64l).

[0088] Branch 2 and branch 3: Similar to branch 1, but using 5x5 and 3x3 convolution kernels.

[0089] Branch 4: Use 3x3 maximum pooling, and the output shape is the same as branch 1.

[0090] The outputs of all branches are concatenated in the channel dimension. Since the output channel is 64, the output shape after concatenation is (None, 19, 16, 64).

[0091] MSDM_1: Repeats the operation of the MSDM layer, but the input spatial dimensions have been halved, so the output shape becomes (None, 10, 8, 64).

[0092] MSDM_2: Repeat the operation of the MSDM layer again, and the output shape becomes (None, 5, 4, 64).

[0093] Multi-Scale Channel Attention (MSCA): Mainly used for channel attention, it does not change the number of channels. The MSCA layer adjusts the channel weights through 1x1 convolution and Sigmoid activation function, but the number of channels in the final output is the same as the input, which is 64.

[0094] Flatten layer: Flattens the output of the MSCA layer along the last dimension, and the output shape becomes (None, 256).

[0095] Reshape: Reshapes the flattened output to (None, 64, 4). This simply modifies the data format to match the input structure below.

[0096] Three Long Short-Term Memory (LSTM) layers (LSTM, LSTM_1, and LSTM_2): LSTM layers can process sequential data and have their own internal state (cell state and hidden state). However, the number of output channels in each LSTM layer is determined by the number of cells in the layer. The model structure has a three-dimensional input.

[0097] The number of LSTM output channels is 64, LSTM_1 is 128, and LSTM_2 is 256. These channel numbers correspond to the number of units in each LSTM layer.

[0098] It also includes the first dropout layer (Dropout), the first fully connected layer (Dense), the second dropout layer (Dropout_1) and the second fully connected layer (Dense_1).

[0099] This embodiment introduces a Multi-Scale Depthwise Convolutional Network (MSDC) to extend depthwise separable convolution. By integrating depthwise separable convolution and multi-scale feature extraction technology, the MSDC module not only improves the efficiency of parameters and calculations, but also enhances the model's ability to process multi-scale features. This makes the MSDC module extremely valuable in building lightweight and efficient deep learning models.

[0100] In MSDC, the input feature map is divided into three regions along the channel dimension, and then processed by 3×3, 5×5, and 7×7 two-dimensional depth convolutions respectively. Finally, these three feature maps are connected and merged into a fused feature map. The choice of two-dimensional depth convolution instead of traditional two-dimensional convolution is to reduce the complexity and computational cost of the network; at the same time, the initial channel segmentation also helps to reduce the computational cost. The structure of the MSDC network is as follows: Figure 5 shown.

[0101] Multi-Scale Channel Attention (MSCA) leverages an attention mechanism to capture nonlinear information across channels in feature maps. While channel attention can be implemented in a variety of ways, Squeeze-and-Excitation Networks (SE Nets) have garnered widespread attention. SE Nets consist of two components: compression and excitation. Compression reduces the feature map dimensionality to a 1-D vector via global average pooling (GAP), while excitation alternately reduces and then expands the channel dimension via a cascade of two fully connected layers (FC layers). In SE Nets, channel attention is implemented by scaling the original feature map by each component of the excitation vector channel-by-channel. The recently proposed Efficient Channel Attention Network (ECA-Net) replaces the FC layer in SE Net with a single one-dimensional convolutional layer, avoiding information loss caused by dimensionality reduction, reducing model complexity, and enhancing cross-channel information interaction. ECA-Net uses a single convolutional kernel whose length is chosen based on the number of channels in the input feature map.

[0102] In the multi-scale channel attention mechanism proposed in this embodiment, ECA-Net is improved. Unlike the single convolution used in ECA-Net, this embodiment adopts three independent one-dimensional convolutions, each with a different kernel length. Specifically, MSCA first compresses the feature map spatially through the GAP operation; then, one-dimensional convolutions with kernel lengths of 3, 5, and 7 are applied in parallel. These feature vectors are fused by addition and processed by the S-shaped activation function. Finally, channel attention is generated channel by channel and multiplied with the input feature map, as shown in Figure 2. Figure 6 shown.

[0103] To address the current unsatisfactory performance of underwater acoustic target recognition, a joint neural network model based on a channel-attention mechanism and multi-scale depthwise separable convolutions is proposed. By fusing 3D-MFCC and 3D-Mel features, this model effectively addresses the shortcomings of a single 3D feature set in representing target information. This method was compared with state-of-the-art underwater acoustic recognition network models on the Deep Ship dataset, demonstrating superior recognition performance.

[0104] Based on the same inventive concept, embodiments of the present application also provide an underwater acoustic target recognition device based on a joint neural network and 3D fusion features for implementing the aforementioned underwater acoustic target recognition method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more of the following embodiments of the underwater acoustic target recognition device can be found in the aforementioned limitations of the underwater acoustic target recognition method and will not be further elaborated here.

[0105] In an exemplary embodiment, an underwater acoustic target recognition device based on a joint neural network of 3D fusion features is provided, comprising:

[0106] The feature extraction module is used to extract features of the underwater audio signal to obtain 3D Mel-frequency scale features and 3D Mel-frequency cepstral coefficient features; the 3D Mel-frequency scale features include: Mel-frequency scale and the first-order difference and second-order difference of the Mel-frequency scale; the 3D Mel-frequency cepstral coefficient features include: Mel-frequency cepstral coefficient and the first-order difference and second-order difference of the Mel-frequency cepstral coefficient.

[0107] The feature fusion module is used to fuse the 3D Mel-frequency scale feature and the 3D Mel-frequency cepstral coefficient feature to obtain a 3D fused feature.

[0108] The target recognition module is used to input the 3D fusion features into a trained joint neural network model for underwater acoustic target recognition; the joint neural network model includes a serially connected multi-scale depth-separable convolutional network module, a multi-scale channel attention mechanism and a long short-term memory network module.

[0109] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data determined by complementarity. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for underwater acoustic target recognition is implemented.

[0110] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0111] In an exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned underwater acoustic target recognition method embodiment when executing the computer program.

[0112] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, which, when executed by a processor, implements the steps in the above-mentioned underwater acoustic target recognition method embodiment.

[0113] In an exemplary embodiment, a computer program product is provided, including a computer program, which implements the steps in the above-mentioned underwater acoustic target recognition method embodiment when executed by a processor.

[0114] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0115] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, and the like.

[0116] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0117] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for underwater acoustic target recognition based on a joint neural network with 3D fusion features, characterized in that: The underwater acoustic target recognition method comprises: Feature extraction is performed on the underwater audio signal to obtain 3D Mel-spectrogram features and 3D Mel-frequency cepstral coefficient features; the 3D Mel-spectrogram features include: Mel-spectrogram and the first-order difference and second-order difference of the Mel-spectrogram; the 3D Mel-frequency cepstral coefficient features include: Mel-frequency cepstral coefficient and the first-order difference and second-order difference of the Mel-frequency cepstral coefficient; The 3D Mel-spectrogram features and the 3D Mel-frequency cepstral coefficient features are fused to obtain 3D fused features. In the feature fusion process, the original Mel-spectrogram, first-order difference and second-order difference are stacked into a 3D array using the numpy.dstack function to obtain the 3D Mel-frequency scale features. At the same time, the numpy.dstack function is also used to stack the MFCC and its difference into another 3D array to obtain the 3D Mel-frequency cepstral coefficient features. Finally, the 3D Mel-frequency scale features and the 3D Mel-frequency cepstral coefficient features are merged in the last dimension to form a 3D fused feature. Inputting the 3D fusion features into a trained joint neural network model for underwater acoustic target recognition; the joint neural network model includes a serially connected multi-scale depthwise separable convolutional network module, a multi-scale channel attention mechanism, and a long short-term memory network module; The joint neural network model further includes: an input layer, a two-dimensional convolutional layer, a maximum pooling layer, a flattening layer, a reshaping layer, a first dropout layer, a first fully connected layer, a second dropout layer, and a second fully connected layer; The input layer is connected to the two-dimensional convolutional layer, the two-dimensional convolutional layer is connected to the multi-scale depth-wise separable convolutional network module, the multi-scale depth-wise separable convolutional network module is connected to the maximum pooling layer, the maximum pooling layer is connected to the flattening layer, the flattening layer is connected to the reshaping layer, the reshaping layer is connected to the long short-term memory network module, the long short-term memory network module is connected to the first dropout layer, the first dropout layer is connected to the first fully connected layer, the first fully connected layer is connected to the second dropout layer, and the second dropout layer is connected to the second fully connected layer; The multi-scale depth-separable convolutional network module includes three multi-scale depth-separable convolutional network layers, and the long short-term memory network module includes three long short-term memory network layers; The multi-scale depth-wise separable convolutional network layer consists of 4 branches: Branch 1: Use 7x7 depth-separable convolution; Branch 2 and branch 3: use 5x5 and 3x3 convolution kernels; Branch 4: Use 3x3 maximum pooling; The outputs of all branches are concatenated in the channel dimension; The three multi-scale depthwise separable convolutional network layers repeat the same operation three times.

2. The underwater acoustic target recognition method based on a joint neural network with 3D fusion features according to claim 1 is characterized in that: Feature extraction is performed on the underwater audio signal to obtain 3D Mel-spectrogram features and 3D Mel-frequency cepstral coefficient features, specifically including: The underwater audio signal is framed using the Hanning window to obtain multiple underwater audio segments; Perform fast Fourier transform on each underwater acoustic audio band to obtain the frequency domain characteristics of each underwater acoustic audio band; The Mel filter is used to capture the frequency domain features of each underwater sound frequency band and obtain the Mel spectrum of each underwater sound frequency band. Calculate the first-order difference and second-order difference of the Mel-spectrogram of each underwater sound frequency band; Perform logarithmic calculation and discrete Fourier transform on the Mel frequency spectrum of each underwater sound frequency band to obtain the Mel frequency cepstrum coefficient of each underwater sound frequency band; Calculate the first-order and second-order differences of the Mel-frequency cepstral coefficients of each underwater acoustic audio band.

3. The underwater acoustic target recognition method based on a joint neural network with 3D fusion features according to claim 1 is characterized in that: The multi-scale channel attention mechanism includes: a global average pooling layer, three one-dimensional convolutional layers, an additive fusion layer, an activation function layer, and a multiplicative fusion layer; The three one-dimensional convolutional layers are arranged in parallel, the input ends of the three one-dimensional convolutional layers are connected to the global average pooling layer, the output ends of the three one-dimensional convolutional layers are connected to the additive fusion layer, the additive fusion layer is connected to the activation function layer, and the activation function layer is connected to the multiplicative fusion layer; The multiplication fusion layer is used to multiply the attention of each channel output by the activation function layer with the input feature map to obtain an attention feature map; the input feature map is the feature map input to the multi-scale channel attention mechanism.

4. An underwater acoustic target recognition device based on a joint neural network with 3D fusion features, characterized in that: The underwater acoustic target recognition device applies the underwater acoustic target recognition method according to any one of claims 1 to 3, and the underwater acoustic target recognition device comprises: A feature extraction module is used to extract features from underwater audio signals to obtain 3D Mel-spectrogram features and 3D Mel-frequency cepstral coefficient features; the 3D Mel-spectrogram features include: Mel-spectrogram and the first-order and second-order differences of the Mel-spectrogram; the 3D Mel-frequency cepstral coefficient features include: Mel-frequency cepstral coefficients and the first-order and second-order differences of the Mel-frequency cepstral coefficients; The feature fusion module is used to fuse the 3D Mel-frequency spectrum features and the 3D Mel-frequency cepstral coefficient features to obtain 3D fusion features; The target recognition module is used to input the 3D fusion features into a trained joint neural network model for underwater acoustic target recognition; the joint neural network model includes a serially connected multi-scale depth-separable convolutional network module, a multi-scale channel attention mechanism and a long short-term memory network module.

5. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the underwater acoustic target recognition method according to any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the underwater acoustic target recognition method according to any one of claims 1 to 3 is implemented.

7. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the underwater acoustic target recognition method according to any one of claims 1 to 3 is implemented.

Images