Underwater sound target recognition method, device and equipment based on three-dimensional fusion feature combined neural network, medium and product

By using a joint neural network with 3D fusion features, combined with a multi-scale deep separable convolutional network and a channel attention mechanism, the problem of static features ignoring dynamic features in underwater acoustic recognition is solved, thereby improving recognition accuracy and model efficiency.

CN119028373AActive Publication Date: 2024-11-26ZHONGBEI UNIV

Patent Information

Application Number
CN202411117642.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-14
Publication Date
2024-11-26
Estimated Expiration
2044-08-14

AI Technical Summary

Technical Problem

Existing underwater acoustic target recognition methods rely on static features and ignore dynamic features, resulting in reduced recognition accuracy. Deep neural networks cannot guarantee performance due to insufficient data.

Method used

A joint neural network based on 3D fusion features is adopted to extract 3D Mel spectrogram features and 3D Mel frequency cepstral coefficient features, and combine a multi-scale deep separable convolutional network and a long short-term memory network with a multi-scale channel attention mechanism for recognition.

Benefits of technology

It improves the accuracy of underwater acoustic recognition, reduces the number of model parameters, lowers the training sample requirements, and enhances the model's recognition ability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119028373A_ABST
    Figure CN119028373A_ABST
Patent Text Reader

Abstract

The invention discloses an underwater acoustic target recognition method, device and equipment based on a three-dimensional fusion feature joint neural network, a medium and a product, and relates to the field of underwater acoustic recognition. The underwater sound target identification method comprises the following steps: performing feature extraction on an underwater sound audio signal to obtain a three-dimensional Mel frequency scale feature and a three-dimensional Mel frequency cepstral coefficient feature; fusing the three-dimensional Mel frequency scale feature and the three-dimensional Mel frequency cepstrum coefficient feature to obtain a three-dimensional fusion feature; and inputting the three-dimensional fusion features into a trained joint neural network model for underwater acoustic target recognition. According to the invention, the accuracy of underwater sound recognition is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of underwater acoustic recognition, and in particular to an underwater acoustic target recognition method, device, equipment, medium and product based on a joint neural network of 3D fusion features. Background Art

[0002] Hydroacoustic target recognition mainly relies on the analysis of ship radiated noise. These noise signals contain rich target feature information. Through in-depth analysis of their characteristics, underwater targets can be effectively classified and identified. The traditional method of hydroacoustic target recognition based on ship radiated noise is to classify ship types using artificially designed features and shallow classifiers, focusing on feature extraction and the development of nonlinear classifiers. For feature extraction, Mel-frequency cepstral coefficients (MFCC), constant Q transform (CQT), wavelet features, detection of envelope modulation on noise (DEMON) and low frequency analysis and recording (LOFAR) spectrum, as well as advanced spectrum features are cited. The existing feature extraction technology focuses on static features and ignores dynamic features, which will lead to a decrease in the accuracy of hydroacoustic recognition.

[0003] In addition, deep learning can also directly absorb features from the original signal. This method effectively filters out redundant data, thereby reducing the computational requirements of subsequent models. Existing studies on passive underwater acoustic target feature recognition based on deep learning are all in their early stages, mainly focusing on theoretical exploration and small-scale experiments. Existing deep neural networks have a large number of parameters and need to be combined with big data to ensure their performance. In the application of underwater acoustic recognition, it is usually difficult to collect a sufficient amount of data for model training, which greatly limits the performance of deep neural networks and cannot guarantee the accuracy of underwater acoustic recognition. Summary of the invention

[0004] The purpose of this application is to provide a method, device, equipment, medium and product for underwater acoustic target recognition based on a joint neural network with three-dimensional fusion features, which can improve the accuracy of underwater acoustic recognition.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a method for underwater acoustic target recognition based on a joint neural network with 3D fusion features, the method comprising:

[0007] The underwater acoustic audio signal is feature extracted to obtain 3D Mel spectrum features and 3D Mel frequency cepstrum coefficient features; the 3D Mel spectrum features include: Mel spectrum and the first-order difference and second-order difference of the Mel spectrum; the 3D Mel frequency cepstrum coefficient features include: Mel frequency cepstrum coefficient and the first-order difference and second-order difference of the Mel frequency cepstrum coefficient;

[0008] The 3D Mel frequency spectrum feature and the 3D Mel frequency cepstral coefficient feature are fused to obtain the 3D fusion feature;

[0009] The 3D fusion features are input into a trained joint neural network model for underwater acoustic target recognition; the joint neural network model includes a serially connected multi-scale depth-separable convolutional network module, a multi-scale channel attention mechanism and a long short-term memory network module.

[0010] Optionally, feature extraction is performed on the underwater audio signal to obtain 3D Mel-spectrogram features and 3D Mel-frequency cepstrum coefficient features, specifically including:

[0011] The underwater acoustic audio signal is framed using a Hanning window to obtain multiple underwater acoustic audio segments;

[0012] Performing fast Fourier transform on each underwater acoustic frequency band respectively to obtain the frequency domain characteristics of each underwater acoustic frequency band;

[0013] The Mel filter is used to capture the frequency domain features of each underwater sound frequency band, and the Mel spectrum of each underwater sound frequency band is obtained;

[0014] Calculate the first-order difference and second-order difference of the Mel-spectrogram of each underwater sound frequency band;

[0015] Performing logarithmic calculation and discrete Fourier transformation on the Mel frequency spectrum of each underwater acoustic audio band to obtain the Mel frequency cepstrum coefficient of each underwater acoustic audio band;

[0016] Calculate the first-order difference and second-order difference of the Mel-frequency cepstral coefficients of each underwater acoustic audio band.

[0017] Optionally, the joint neural network model further includes an input layer, a two-dimensional convolutional layer, a maximum pooling layer, a flattening layer, a reshaping layer, a first dropout layer, a first fully connected layer, a second dropout layer, and a second fully connected layer;

[0018] The input layer is connected to the two-dimensional convolutional layer, the two-dimensional convolutional layer is connected to the multi-scale depth-separable convolutional network module, the multi-scale depth-separable convolutional network module is connected to the maximum pooling layer, the maximum pooling layer is connected to the flattening layer, the flattening layer is connected to the reshaping layer, the reshaping layer is connected to the long short-term memory network module, the long short-term memory network module is connected to the first drop layer, the first drop layer is connected to the first fully connected layer, the first fully connected layer is connected to the second drop layer, and the second drop layer is connected to the second fully connected layer.

[0019] Optionally, the multi-scale depth-separable convolutional network module includes three multi-scale depth-separable convolutional network layers, and the long short-term memory network module includes three long short-term memory network layers.

[0020] Optionally, the multi-scale channel attention mechanism includes: a global average pooling layer, three one-dimensional convolutional layers, an additive fusion layer, an activation function layer and a multiplicative fusion layer;

[0021] The three one-dimensional convolutional layers are arranged in parallel, the input ends of the three one-dimensional convolutional layers are all connected to the global average pooling layer, the output ends of the three one-dimensional convolutional layers are all connected to the addition fusion layer, the addition fusion layer is connected to the activation function layer, and the activation function layer is connected to the multiplication fusion layer;

[0022] The multiplication fusion layer is used to multiply the attention of each channel output by the activation function layer with the input feature map to obtain an attention feature map; the input feature map is the feature map input to the multi-scale channel attention mechanism.

[0023] In a second aspect, the present application provides an underwater acoustic target recognition device based on a joint neural network of 3D fusion features, wherein the underwater acoustic target recognition device applies the above-mentioned underwater acoustic target recognition method, and the underwater acoustic target recognition device comprises:

[0024] A feature extraction module is used to extract features of underwater acoustic audio signals to obtain 3D Mel spectrum features and 3D Mel frequency cepstrum coefficient features; the 3D Mel spectrum features include: Mel spectrum and the first-order difference and second-order difference of the Mel spectrum; the 3D Mel frequency cepstrum coefficient features include: Mel frequency cepstrum coefficient and the first-order difference and second-order difference of the Mel frequency cepstrum coefficient;

[0025] A feature fusion module is used to fuse the 3D Mel frequency spectrum features and the 3D Mel frequency cepstral coefficient features to obtain 3D fusion features;

[0026] The target recognition module is used to input the 3D fusion features into the trained joint neural network model for underwater acoustic target recognition; the joint neural network model includes a serially connected multi-scale depth-separable convolutional network module, a multi-scale channel attention mechanism and a long short-term memory network module.

[0027] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned underwater acoustic target recognition method.

[0028] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned underwater acoustic target recognition method when executed by a processor.

[0029] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the above-mentioned underwater acoustic target recognition method when executed by a processor.

[0030] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0031] The present application provides a method, device, equipment, medium and product for underwater acoustic target recognition based on a joint neural network with 3D fusion features. The original underwater acoustic audio signal of the present application is converted into a 3D fusion feature (3D fusion feature) after fusion of a 3D Mel spectrum feature (3D-Mel feature) and a 3D Mel frequency cepstral coefficient feature (3D-MFCC feature), thereby capturing the original signal target and its dynamic information. In the classification task, a multi-scale deep separable convolutional network replaces the CNN in the serial architecture of the traditional convolutional neural network (Cable News Network, CNN) and the long short-term memory network (Long Short-Term Memory, LSTM) to reduce the number of parameters of the model, thereby improving the computational efficiency and reducing the number of samples required for training. In addition, in order to further enhance the recognition ability of the model, a multi-scale channel attention mechanism (Multi-Scale Channel Attention, MSCA) is integrated into the above-mentioned optimized serial architecture, so that the model can more accurately capture the key speech features and enhance the overall performance of the model. The present application can improve the accuracy of underwater acoustic recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 This is an application environment diagram of an underwater acoustic target recognition method based on a joint neural network of 3D fusion features in one embodiment of the present application;

[0034] Figure 2 A flowchart of a method for underwater acoustic target recognition based on a joint neural network with 3D fusion features provided in one embodiment of the present application;

[0035] Figure 3 A schematic diagram of a process for extracting and fusing features of underwater audio signals provided in an embodiment of the present application;

[0036] Figure 4 A schematic diagram of the overall structure of a joint neural network model provided in one embodiment of the present application;

[0037] Figure 5 A schematic diagram of the structure of a multi-scale depth-separable convolutional network layer provided in an embodiment of the present application;

[0038] Figure 6 A schematic diagram of the structure of a multi-scale channel attention mechanism provided in an embodiment of the present application;

[0039] Figure 7 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0041] In order to make the above-mentioned objects, features and advantages of the present application more obvious and understandable, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0042] The underwater acoustic target recognition method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the underwater acoustic audio signal to the server 104, and the server 104 receives the underwater acoustic audio signal. The server 104 extracts the features of the underwater acoustic audio signal to obtain a 3D Mel spectrum feature and a 3D Mel frequency cepstral coefficient feature; the 3D Mel spectrum feature and the 3D Mel frequency cepstral coefficient feature are fused to obtain a 3D fusion feature; the 3D fusion feature is input into the trained joint neural network model for underwater acoustic target recognition. The server 104 can feedback the obtained underwater acoustic target recognition result to the terminal 102. In addition, in some embodiments, the underwater acoustic target recognition method can also be implemented by the server 104 or the terminal 102 alone, such as the terminal 102 can directly process the underwater acoustic audio signal, or the server 104 can obtain the underwater acoustic audio signal from the data storage system and process the underwater acoustic audio signal.

[0043] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0044] In an exemplary embodiment, Figure 2 As shown, a method for underwater acoustic target recognition is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate, including the following steps 201 to 203. Among them:

[0045] Step 201, extract features of the underwater acoustic audio signal to obtain 3D Mel-frequency spectrum features and 3D Mel-frequency cepstral coefficient features; the 3D Mel-frequency spectrum features include: Mel-frequency spectrum and the first-order difference and second-order difference of the Mel-frequency spectrum; the 3D Mel-frequency cepstral coefficient features include: Mel-frequency cepstral coefficients and the first-order difference and second-order difference of the Mel-frequency cepstral coefficients.

[0046] Step 202, fuse the 3D Mel-frequency spectrum feature and the 3D Mel-frequency cepstral coefficient feature to obtain a 3D fused feature.

[0047] Step 203, input the 3D fusion features into the trained joint neural network model for underwater acoustic target recognition; the joint neural network model includes a serially connected multi-scale depth-separable convolutional network module, a multi-scale channel attention mechanism and a long short-term memory network module.

[0048] The implementation of the above steps 201 to 203 can improve the accuracy of underwater acoustic recognition.

[0049] In the comparative analysis, it is found that 3D dynamic Mel-frequency Cepstrum Coefficient (3D MFCC) exceeds MFCC, Mel spectrogram, 3D dynamic Mel spectrogram and CQT in terms of effectiveness. This application adds 3D-Mel as a supplement to 3D-MFCC feature extraction to enrich the feature representation of speech signals.

[0050] like Figure 3 As shown, feature extraction is performed on the underwater acoustic audio signal to obtain 3D Mel spectrum features and 3D Mel frequency cepstrum coefficient features, specifically including: using a Hanning window to frame the underwater acoustic audio signal to obtain multiple underwater acoustic audio bands; performing fast Fourier transform on each underwater acoustic audio band to obtain the frequency domain features of each underwater acoustic audio band; using a Mel filter to capture the frequency domain features of each underwater acoustic audio band to obtain the Mel spectrum of each underwater acoustic audio band; calculating the first-order difference and second-order difference of the Mel spectrum of each underwater acoustic audio band; taking logarithm calculation and discrete Fourier transform on the Mel spectrum of each underwater acoustic audio band to obtain the Mel frequency cepstrum coefficient of each underwater acoustic audio band; calculating the first-order difference and second-order difference of the Mel frequency cepstrum coefficient of each underwater acoustic audio band.

[0051] Specifically, in the process of feature extraction, the original underwater audio signal saved in WAV format is preprocessed by removing blank segments and reducing noise, and the length of the sample audio is cut into 5 seconds. The operating system library os, the file matching library glob, the scientific computing library numpy, the audio processing library librosa, and the audio reading and writing library soundfile in the Python library are used to write related functions, aiming to extract features from audio files and calculate the following features:

[0052] Mel-spectrogram (Mel-spectrogram), first-order and second-order differences of Mel-spectrogram (delta and doubledeltaoflog-mel spectrogram), Mel-frequency cepstral coefficients (MFCC), first-order and second-order differences of MFCC (delta and doubledeltaofMFCC).

[0053] Among them, the Mel spectrogram is a spectrogram commonly used in the field of audio processing. It is closely related to the Mel frequency scale. The Mel spectrogram is a representation of an audio signal, which maps the spectral content of the signal to the Mel frequency scale.

[0054] The Mel frequency scale is a perceptual frequency scale for audio signals that simulates the human ear's perception of frequency. The human ear is not linearly sensitive to sounds of different frequencies, especially in the low-frequency region. The Mel frequency scale converts Hertz (Hz) to Mel (Mel) using the following formula:

[0055]

[0056] Where M is the mel frequency and f is the frequency in hertz.

[0057] The main feature of this scale is that it is dense in the low-frequency area and sparse in the high-frequency area, which matches the human ear's auditory perception of frequency.

[0058] The Mel frequency scale provides a theoretical basis and implementation framework for the construction of the Mel spectrogram, which is a visualization tool that re-represents the audio signal according to the Mel frequency scale.

[0059] The Mel frequency scale is a nonlinear frequency scale based on human perception of frequency, that is, the human ear perceives low and high frequencies differently, and low-frequency changes are more obvious to the human ear. The Y-axis of the Mel spectrum graph uses the Mel spectrum instead of the traditional frequency, which can better simulate the auditory characteristics of the human ear.

[0060] In the process of feature fusion, the numpy.dstack function is used to stack the original Mel frequency spectrum, first-order difference and second-order difference into a 3D array to obtain the 3D Mel frequency scale feature. At the same time, the numpy.dstack function is also used to stack the MFCC and its difference into another 3D array to obtain the 3D Mel frequency cepstral coefficient feature. This stacking method allows the model to take into account both the static and dynamic characteristics of the audio signal. Finally, the 3D Mel frequency scale feature and the 3D Mel frequency cepstral coefficient feature are merged in the last dimension to form a larger feature array. This operation is the key to feature fusion. It integrates features of different types and dimensions together as the input of the subsequent model, providing the model with richer information.

[0061] Finally, the extracted 3D fusion features of each audio signal and its corresponding labels are saved as feature files in NPY format as input for subsequent recognition modules. The first two modules aim to provide a comprehensive feature representation for subsequent audio recognition analysis tasks, thereby improving the performance and generalization ability of the model.

[0062] Mel-spectrogram and Mel-frequency cepstral coefficient are commonly used acoustic feature extraction methods. Mel-spectrogram and MFCC simulate human auditory perception of sound frequency and capture key speech features. However, traditional spectrum features mainly capture static spectrum envelope information and lack dynamic signal representation. This embodiment bridges this gap by calculating the first-order and second-order differences of Mel-spectrogram and MFCC.

[0063] First, the delta (first-order difference) and double_delta_ (second-order difference) features of the Mel spectrum and MFCCs are extracted to construct a three-dimensional feature space. Then, the extracted 3D-MFCC and 3D-Mel features are fused to form a more comprehensive and rich feature representation. The whole process is as follows Figure 3 As shown, the detailed steps of feature extraction are shown.

[0064] Since the original features are static, in order to add dynamic information to the static features, incremental features and double incremental features are added to form multi-dimensional dynamic features, which are performed by locally estimating the differential operation of the input features along the time axis. Incremental features and double incremental features provide dynamic information about the original features changing over time.

[0065] The delta coefficient of the Mel-spectrogram can be obtained by formula (1):

[0066]

[0067] Among them, c tis the static Mel-spectrogram cepstral coefficient of frame t, and n is the number of adjacent frames. Through differential operation, D t is the static coefficient c t+n and c t-n The calculated delta coefficient of the Mel-spectrogram. N represents the difference dimension of the time-spectrogram. The same formula can also be used to calculate the delta-delta coefficients, but they are calculated based on delta. As mentioned above, the delta coefficient of the Mel-spectrogram at frame t is D t , then the corresponding delta-delta coefficient D of the mel-spectrogram t 'The definition is as follows:

[0068] D t '=D t+n -D t-n (2)

[0069] Where n represents the number of adjacent frames. t ' represents the delta-delta coefficient of the Mel-spectrum graph in the tth frame,

[0070] The three-dimensional spectrogram of the Mel-spectrogram is defined as:

[0071] Mel_3D∈R F×T×C

[0072] Where F represents the number of Mel filter bins, T represents the number of time dimensions, and C represents the dimension of the spectrogram. When C=1, it represents a one-dimensional Mel-spectrogram, C=2 represents a primary Mel-spectrogram, and C=3 represents a secondary Mel-spectrogram.

[0073] Similar to the above-mentioned Mel difference operation, the difference operation of MFCCs (Mel frequency cepstral coefficients) features along the time axis can obtain delta MFCC (first-order difference of Mel frequency cepstral coefficients) and Double-delta MFCC (second-order difference of Mel frequency cepstral coefficients).

[0074] During the feature extraction process, the frame length is set to 2048, and the Hanning window function is applied before the fast Fourier transform (FFT) of each frame signal to reduce spectral leakage. The length of the applied Hanning window is the same as the frame length, that is, 2048 samples. In addition, a filter group containing 40 filters is configured to capture different frequency ranges. After these processing steps, the 3D fusion features finally obtained by this application have a shape of (40×34×6). Table 1 shows the dimensional information of these features in detail.

[0075] Table 1 Dimension information of features

[0076]

[0077] In the process of underwater acoustic signal recognition, the feature file saved in NPY format is used as input. The feature file stores the above-mentioned 3D fusion features. The 3D fusion features and their labels are randomly divided into a training set and a test set at a ratio of 4:1.

[0078] The joint neural network model of this embodiment replaces the traditional CNN with a multi-scale depth separable convolutional network (MSDC) based on the serial architecture model of the convolutional network (CNN) and the long short-term memory network (LSTM), and adds a multi-scale channel attention mechanism (MSCA). The structure is as follows: Figure 4 The parameters of the model are shown in Table 2.

[0079] Table 2 Joint network model

[0080]

[0081]

[0082] The joint neural network model proposed in this embodiment includes:

[0083] Input layer (Input): The input data shape is (None, 40, 34, 6), where None represents the batch size (data set outside the model, which does not affect the data structure within the model, similar to the number of cycles), 40 and 34 are the height and width of the image respectively, and 6 is the number of channels.

[0084] Conv2D: There is no additional padding, so the spatial dimensions (height and width) are reduced by 2 (because the kernel covers the edges of the input), and the output shape becomes (None, 38, 32, 64). The number of input channels is 6, and the number of output channels is set to 64.

[0085] Three multi-scale depth-wise separable convolutional network layers (MSDM, MSDM_1, MSDM_2): In the MSDM layer, the number of channels does not change. The MSDM layer consists of several branches, each of which is finally output through the ReLU activation function, but there is no operation to change the number of channels between branches or at the end of the branch.

[0086] MSDM consists of four sub-branches, each of which operates on its input:

[0087] Branch 1: Using 7x7 depth-wise separable convolution, the spatial dimension of the output is halved to (None, 19, 16, 1*64l).

[0088] Branch 2 and Branch 3: Similar to Branch 1, but using 5x5 and 3x3 convolution kernels.

[0089] Branch 4: Use 3x3 max pooling, the output shape is the same as branch 1.

[0090] The outputs of all branches are concatenated in the channel dimension. Since the output channel is 64, the output shape after concatenation is (None, 19, 16, 64).

[0091] MSDM_1: Repeats the operation of the MSDM layer, but the input spatial dimension has been halved, so the output shape becomes (None, 10, 8, 64).

[0092] MSDM_2: Repeat the operation of the MSDM layer again, and the output shape becomes (None, 5, 4, 64).

[0093] Multi-scale channel attention mechanism (MSCA): Mainly used for channel attention, it does not change the number of channels. The MSCA layer adjusts the weight of the channel through 1x1 convolution and Sigmoid activation function, but the number of channels of the final output is the same as the input, which is 64 here.

[0094] Flatten layer: Flattens the output of the MSCA layer in the last dimension, and the output shape becomes (None, 256).

[0095] Reshape: Reshape the flattened output to (None, 64, 4). Simply modify the data format to match the input structure below.

[0096] Three LSTM layers (LSTM, LSTM_1, LSTM_2): LSTM layers can process sequence data. They have their own internal states (cell states and hidden states), but the number of output channels of each LSTM layer is determined by the number of its units. The model structure input is three-dimensional.

[0097] The number of LSTM output channels is 64, LSTM_1 is 128, and LSTM_2 is 256. These channel numbers correspond to the number of units in each LSTM layer.

[0098] It also includes the first dropout layer (Dropout), the first fully connected layer (Dense), the second dropout layer (Dropout_1) and the second fully connected layer (Dense_1).

[0099] This embodiment introduces a multi-scale depth separable convolutional network (Multi-Scale Depthwise Convolutional Network, MSDC for short) to expand the depth separable convolution. The MSDC module not only improves the efficiency of parameters and calculations by integrating depth separable convolution and multi-scale feature extraction technology, but also enhances the model's ability to process multi-scale features. This makes the MSDC module extremely valuable in building lightweight and efficient deep learning models.

[0100] In MSDC, the input feature map is divided into three regions along the channel dimension, and then processed by 3×3, 5×5, and 7×7 2D depth convolutions. Finally, the three feature maps are connected and merged into a fused feature map. The 2D depth convolution is chosen instead of the traditional 2D convolution to reduce the complexity and computational cost of the network; at the same time, the initial channel segmentation also helps to reduce the computational cost. The structure of the MSDC network is shown in Figure 2. Figure 5 shown.

[0101] Multi-Scale Channel Attention (MSCA) uses attention mechanism to capture nonlinear information between channels across feature maps. Although channel attention can be implemented in many ways, Squeeze-and-Excitation Networks (SE Net) have attracted widespread attention. SE Net consists of two components: compression and excitation: compression reduces the dimension of feature maps to 1-D vectors through global average pooling (GAP), while excitation alternately reduces and then expands the channel dimension through the cascade of two fully connected layers (FC layers). In SE Net, channel attention is implemented by scaling the original feature map channel by channel for each component of the excitation vector. The recently proposed Efficient Channel Attention Network (ECA-Net) replaces the FC layer in SE Net with a single 1D convolutional layer, avoiding the information loss caused by dimensionality reduction, reducing the model complexity, and enhancing cross-channel information interaction. ECA-Net uses a single convolution kernel whose length is selected according to the number of channels of the input feature map.

[0102] In the multi-scale channel attention mechanism proposed in this embodiment, ECA-Net is improved. Unlike the single convolution used in ECA-Net, this embodiment uses three independent one-dimensional convolutions, each with a different kernel length. Specifically, MSCA first compresses the feature map spatially through the GAP operation; then, one-dimensional convolutions with kernels of lengths 3, 5, and 7 are applied in parallel. These feature vectors are fused by addition and processed by the S-shaped activation function, and finally channel attention is generated channel by channel and multiplied with the input feature map, as shown in Figure 6 shown.

[0103] In view of the current unsatisfactory effect of underwater acoustic target recognition, a joint neural network model of multi-scale depth separable convolution based on channel attention mechanism is proposed, and by fusing 3D-MFCC and 3D-Mel features, the shortcomings of a single three-dimensional feature set in expressing target information are effectively solved. On the Deep Ship dataset, this method is compared with the current advanced underwater acoustic recognition network model, and the results show that the model achieves better recognition effect.

[0104] Based on the same inventive concept, the embodiment of the present application also provides a water acoustic target recognition device based on a joint neural network of 3D fusion features for implementing the water acoustic target recognition method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more water acoustic target recognition device embodiments provided below can refer to the limitations of the water acoustic target recognition method above, and will not be repeated here.

[0105] In an exemplary embodiment, a water acoustic target recognition device based on a joint neural network of 3D fusion features is provided, comprising:

[0106] The feature extraction module is used to extract features of the underwater acoustic audio signal to obtain 3D Mel-frequency scale features and 3D Mel-frequency cepstral coefficient features; the 3D Mel-frequency scale features include: Mel-frequency scale and the first-order difference and second-order difference of the Mel-frequency scale; the 3D Mel-frequency cepstral coefficient features include: Mel-frequency cepstral coefficient and the first-order difference and second-order difference of the Mel-frequency cepstral coefficient.

[0107] The feature fusion module is used to fuse the 3D Mel frequency scale feature and the 3D Mel frequency cepstral coefficient feature to obtain a 3D fused feature.

[0108] The target recognition module is used to input the 3D fusion features into the trained joint neural network model for underwater acoustic target recognition; the joint neural network model includes a serially connected multi-scale depth-separable convolutional network module, a multi-scale channel attention mechanism and a long short-term memory network module.

[0109] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data determined by complementarity. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for underwater acoustic target recognition is implemented.

[0110] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0111] In an exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned underwater acoustic target recognition method embodiment when executing the computer program.

[0112] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, which, when executed by a processor, implements the steps in the above-mentioned underwater acoustic target recognition method embodiment.

[0113] In an exemplary embodiment, a computer program product is provided, including a computer program, which implements the steps in the above-mentioned underwater acoustic target recognition method embodiment when executed by a processor.

[0114] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0115] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, etc., but is not limited thereto.

[0116] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0117] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for underwater acoustic target recognition based on a joint neural network with 3D fusion features, characterized in that: The underwater acoustic target recognition method comprises: The underwater acoustic audio signal is feature extracted to obtain 3D Mel spectrum features and 3D Mel frequency cepstrum coefficient features; the 3D Mel spectrum features include: Mel spectrum and the first-order difference and second-order difference of the Mel spectrum; the 3D Mel frequency cepstrum coefficient features include: Mel frequency cepstrum coefficient and the first-order difference and second-order difference of the Mel frequency cepstrum coefficient; The 3D Mel frequency spectrum feature and the 3D Mel frequency cepstral coefficient feature are fused to obtain the 3D fusion feature; The 3D fusion features are input into a trained joint neural network model for underwater acoustic target recognition; the joint neural network model includes a serially connected multi-scale depth-separable convolutional network module, a multi-scale channel attention mechanism and a long short-term memory network module.

2. The underwater acoustic target recognition method based on a joint neural network with 3D fusion features according to claim 1 is characterized in that: The underwater audio signal is feature extracted to obtain 3D Mel spectrum features and 3D Mel frequency cepstrum coefficient features, including: The underwater acoustic audio signal is framed using a Hanning window to obtain multiple underwater acoustic audio segments; Performing fast Fourier transform on each underwater acoustic frequency band respectively to obtain the frequency domain characteristics of each underwater acoustic frequency band; The Mel filter is used to capture the frequency domain features of each underwater sound frequency band, and the Mel spectrum of each underwater sound frequency band is obtained; Calculate the first-order difference and second-order difference of the Mel-spectrogram of each underwater sound frequency band; Performing logarithmic calculation and discrete Fourier transformation on the Mel frequency spectrum of each underwater acoustic audio band to obtain the Mel frequency cepstrum coefficient of each underwater acoustic audio band; Calculate the first-order difference and second-order difference of the Mel-frequency cepstral coefficients of each underwater acoustic audio band.

3. The underwater acoustic target recognition method based on a joint neural network with 3D fusion features according to claim 1 is characterized in that: The joint neural network model also includes: an input layer, a two-dimensional convolution layer, a maximum pooling layer, a flattening layer, a reshaping layer, a first drop layer, a first fully connected layer, a second drop layer, and a second fully connected layer; The input layer is connected to the two-dimensional convolutional layer, the two-dimensional convolutional layer is connected to the multi-scale depth-separable convolutional network module, the multi-scale depth-separable convolutional network module is connected to the maximum pooling layer, the maximum pooling layer is connected to the flattening layer, the flattening layer is connected to the reshaping layer, the reshaping layer is connected to the long short-term memory network module, the long short-term memory network module is connected to the first drop layer, the first drop layer is connected to the first fully connected layer, the first fully connected layer is connected to the second drop layer, and the second drop layer is connected to the second fully connected layer.

4. The underwater acoustic target recognition method based on a joint neural network with three-dimensional fusion features according to claim 1 or 3 is characterized in that: The multi-scale depth-separable convolutional network module includes three multi-scale depth-separable convolutional network layers, and the long short-term memory network module includes three long short-term memory network layers.

5. The underwater acoustic target recognition method based on a joint neural network with three-dimensional fusion features according to claim 1 or 3 is characterized in that: The multi-scale channel attention mechanism includes: a global average pooling layer, three one-dimensional convolutional layers, an addition fusion layer, an activation function layer and a multiplication fusion layer; The three one-dimensional convolutional layers are arranged in parallel, the input ends of the three one-dimensional convolutional layers are all connected to the global average pooling layer, the output ends of the three one-dimensional convolutional layers are all connected to the addition fusion layer, the addition fusion layer is connected to the activation function layer, and the activation function layer is connected to the multiplication fusion layer; The multiplication fusion layer is used to multiply the attention of each channel output by the activation function layer with the input feature map to obtain an attention feature map; the input feature map is the feature map input to the multi-scale channel attention mechanism.

6. An underwater acoustic target recognition device based on a joint neural network with 3D fusion features, characterized in that: The underwater acoustic target recognition device applies the underwater acoustic target recognition method according to any one of claims 1 to 5, and the underwater acoustic target recognition device comprises: A feature extraction module is used to extract features of underwater acoustic audio signals to obtain 3D Mel spectrum features and 3D Mel frequency cepstrum coefficient features; the 3D Mel spectrum features include: Mel spectrum and the first-order difference and second-order difference of the Mel spectrum; the 3D Mel frequency cepstrum coefficient features include: Mel frequency cepstrum coefficient and the first-order difference and second-order difference of the Mel frequency cepstrum coefficient; A feature fusion module is used to fuse the 3D Mel frequency spectrum features and the 3D Mel frequency cepstral coefficient features to obtain 3D fusion features; The target recognition module is used to input the 3D fusion features into the trained joint neural network model for underwater acoustic target recognition; the joint neural network model includes a serially connected multi-scale depth-separable convolutional network module, a multi-scale channel attention mechanism and a long short-term memory network module.

7. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for underwater acoustic target recognition according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the underwater acoustic target recognition method described in any one of claims 1 to 5 is implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the underwater acoustic target recognition method described in any one of claims 1 to 5 is implemented.

Citation Information

Patent Citations

  • Passive underwater acoustic signal identification method and system based on MFCC characteristics

    CN117009781A

  • Novel weighted Mel frequency cepstrum feature fusion method

    CN117251822A

  • Underwater target sound automatic classification method based on lightweight multi-scale convolution attention neural network

    CN117831572A

  • Underwater sound target identification method and system based on neural network and multi-dimensional feature fusion

    CN117854540A

  • Acoustic scene classification method based on multi-dimensional convolutional neural network

    CN117975948A

Cited By

  • Underwater sound target identification method and system based on expert hybrid model

    CN119724240A

  • Ship noise signal detection method, equipment and medium

    CN120892796A

  • Vehicle abnormal sound detection method and device based on convolutional neural network, equipment and medium

    CN121148424A