Underwater sound target identification method based on knowledge injection deep learning

Abstract

The invention relates to an underwater acoustic target recognition method based on knowledge injection deep learning, and the method comprises the following steps: S1, collecting target echo data through employing an active sonar, or collecting target radiation noise data through employing a passive sonar, and carrying out the multi-stage labeling of a target according to an actual demand, and obtaining a multi-stage label tree; s2, constructing an underwater acoustic target recognition deep learning network model based on knowledge injection, wherein the model comprises two parts: a feature extraction module and a knowledge injection module based on multistage labels; according to the method, the neural network learns the reasoning process of a person, the confidence of sonar target recognition is improved through layer-by-layer judgment while the deep learning nonlinear feature representation capability is fully utilized, and the explanatability of deep learning model output is also improved through multi-level labels.

Description

Technical field:

[0001] The present invention belongs to the technical field of underwater acoustic engineering, and in particular relates to an underwater acoustic target recognition method based on knowledge injection deep learning. Background technology:

[0002] Hydroacoustic target recognition uses active sonar to emit sound waves and receive scattered echoes from the target, or uses passive sonar to receive radiated noise from the target and analyze the received data. It can then determine the types of underwater targets, such as surface ships, underwater fish schools, underwater vehicles, and reefs. It has very important applications in marine environment monitoring, biological surveys, and underwater early warning.

[0003] Traditional underwater acoustic target recognition generally includes two parts: feature extraction and classifier design. By extracting the identifiable features of the target, the target type is judged through fuzzy logic classification systems. The rules in the fuzzy logic system are designed based on expert experience. For example, people will first judge surface and underwater targets based on certain features, and then judge moving and stationary targets, etc., and derive the target type through layer-by-layer reasoning. With the rapid development of deep learning technology, deep learning technology has also been introduced into the field of underwater acoustic target recognition. By using the powerful high-dimensional nonlinear feature representation ability of deep learning, the separable features of the target can be directly mined from the data, which can improve the recognition performance to a certain extent.

[0004] At present, there are still the following problems in underwater acoustic target recognition:

[0005] First, as the sonar range increases significantly, the number of targets detected by sonar increases, but the resulting reverberation and noise interference also increase. In addition, affected by the channel propagation effect of the random fluctuations of the ocean channel, the received data is distorted and the signal-to-noise ratio is low, resulting in a decrease in the performance of underwater acoustic target recognition based on traditional feature extraction methods.

[0006] Second, deep learning is often viewed as a black box that directly outputs category results, and the reasoning process of the neural network cannot be seen, resulting in poor interpretability of underwater acoustic target recognition based on deep learning. Summary of the invention:

[0007] The technical problem to be solved by the present invention is to provide a method for underwater acoustic target recognition based on knowledge injection deep learning. This method integrates the human multi-level reasoning process into the neural network, allowing the neural network to learn the human reasoning process, which can greatly improve the interpretability of underwater acoustic target recognition based on deep learning, and at the same time utilize the feature mining capabilities and layer-by-layer judgment of deep learning to improve recognition performance and confidence.

[0008] The technical solution of the present invention is to provide a method for underwater acoustic target recognition based on knowledge injection deep learning, comprising the following steps:

[0009] S1: Use active sonar to collect target echo data, or use passive sonar to collect target radiation noise data, and perform multi-level labeling on the target according to actual needs to obtain a multi-level label tree. The depth of the label tree is N, and the nth level label is Among them, M n is the number of categories of the n-th level label;

[0010] S2: Construct a deep learning network model for underwater acoustic target recognition based on knowledge injection. The model consists of two parts: a feature extraction module and a knowledge injection module based on multi-level labels.

[0011] The feature extraction module is used to extract high-dimensional features from the target echo or radiation noise data. If the number of target samples is large, a backbone network can be used to directly extract features from the echo or radiation noise. If the number of target samples is small, the echo or radiation noise can be transformed into multiple domains. The multi-domain spectra are spliced ​​in the channel dimension, and then the backbone network is used to extract features from the spliced ​​multi-channel feature spectra.

[0012] The output of the feature extraction module is the last layer output feature f before the fully connected layer of the backbone network. r ;

[0013] The multi-level label-based knowledge injection module is used to enable the network to learn multi-level label reasoning. It contains N branches, where N is the depth of the multi-level label tree.

[0014] The first branch consists of a convolutional layer Conv1, two fully connected layers, and a Softmax layer. The input is f r , the output is the probability value of all labels in the first level;

[0015] The nth branch (n>1) includes a convolutional layer Conv n , two fully connected layers, a Softmax layer, and the input is the output of the convolutional layer in the n-1th branch Conv n-1 (f r ) and f r Concat(Conv n-1 (f r ),f r ), where Concat(g) represents the feature concatenation operation, which allows the label inference at this level to inherit the features of the previous level inference, and the output is the probability value of all labels at the nth level;

[0016] The loss function in each branch adopts the cross entropy loss function;

[0017] S3: The model is trained in stages. In the first stage, all model parameters are trained. After the model converges, the second stage training begins. The second stage training locks the parameters of the backbone network and fine-tunes the parameters of the branch modules with a small learning rate.

[0018] S4: Use the trained model to predict the new data. The N branches of the model generate the probability values ​​w1, w2 of the corresponding level labels respectively. n ,L,w N , where w n represents the output probability value of the nth branch, Starting from the first layer, traverse all paths to the bottom leaf nodes of the label tree from top to bottom, multiply the probability values ​​on the paths, and get M1·M2·LM N probability values, and select the label on the path corresponding to the maximum probability value as the output multi-level label of the echo.

[0019] Preferably, in step S2, the backbone network may be a convolutional neural network, including but not limited to ResNet, InceptionNet, Unet, and transformer.

[0020] Preferably, when a convolutional neural network is used, the two-dimensional convolution kernel needs to be modified into a one-dimensional convolution kernel.

[0021] Preferably, in step S2, the echo can obtain echo highlights in the spatiotemporal domain, a time-frequency spectrum in the time-frequency domain, and a Mel-frequency cepstrum in the auditory perception domain, and the radiated noise can be transformed to obtain a LOFAR spectrum, a DEMON spectrum, and an MFCC feature spectrum.

[0022] Compared with the prior art, the present invention has the following advantages:

[0023] The underwater acoustic target recognition method based on knowledge injection deep learning provided by the present invention performs multi-level labeling of target categories through expert knowledge and integrates it into the deep learning network model, allowing the neural network to learn the human reasoning process. While making full use of the nonlinear feature representation capability of deep learning, the confidence of sonar target recognition is improved through layer-by-layer judgment, and the multi-level labeling also improves the interpretability of the deep learning model output. Description of the drawings:

[0024] Figure 1 It is a work flow chart of the present invention;

[0025] Figure 2 This is an example of a secondary label tree for an active sonar target category in an embodiment of the present invention.

[0026] Figure 3It is a schematic diagram of a neural network model for active sonar target classification under a secondary label tree according to an embodiment of the present invention.

[0027] Figure 4 It is a schematic diagram of a method for calculating the probability value of the final output of the network model according to an embodiment of the present invention. Specific implementation method:

[0028] The present invention will be further described below with reference to the accompanying drawings:

[0029] Figure 1 The flowchart of the method for underwater acoustic target recognition based on knowledge injection deep learning of the present invention is as follows. This embodiment takes active sonar target echo as an example to describe the present invention, and includes the following steps:

[0030] Step 1: Collect active sonar target echo data and perform multi-level labeling on the active sonar target echo according to actual needs. Here we take the second-level label as an example. Figure 2 An example of a second-level label tree for active sonar target categories is given.

[0031] The depth of the label tree is 2. The first-level labels are surface targets and underwater targets. The number of categories is 2. The second-level labels corresponding to surface targets are surface ships and merchant ships. The second-level labels corresponding to underwater targets are underwater vehicles, fish schools, and reefs. The total number of categories of the second-level labels is 5.

[0032] Step 2: Construct a deep learning model for active sonar target recognition based on knowledge injection. The model consists of two parts: an echo feature extraction module and a multi-level label-based knowledge injection module. Figure 3 A schematic diagram of a deep learning neural network with two branches is given.

[0033] The echo feature extraction module is used to extract high-dimensional features from active sonar target echoes. If there are a large number of echoes, a backbone network can be used to directly extract features from the echoes. The backbone network can be a convolutional neural network, such as ResNet, InceptionNet, Unet, Transformer, etc. If a convolutional neural network such as ResNet is used, the two-dimensional convolution kernel in the model needs to be modified to a one-dimensional convolution kernel; if the number of echoes is small, the echoes can be transformed into multiple domains, such as echo highlights in the spatiotemporal domain, time-frequency spectrograms in the time-frequency domain, and Mel cepstrum in the auditory perception domain. The multi-domain spectrograms are spliced ​​in the channel dimension, and then the spliced ​​multi-channel feature spectra are extracted using backbone networks such as ResNet, InceptionNet, and UnetTransformer networks.

[0034] Figure 3In the paper, multi-domain transformation is performed on the active target echo to generate spatiotemporal echo highlights, STFT time-frequency spectrum, CWT time-frequency spectrum, WVD time-frequency spectrum and MFCC spectrum. Each spectrum is normalized to 255×255 and spliced ​​in the channel dimension. The dimension of the spliced ​​feature spectrum is 5×255×255, which is used as the input of the backbone network.

[0035] The output of the echo feature extraction module is the last layer output feature f before the fully connected layer of the backbone network. r

[0036] The multi-level label-based knowledge injection module is used to enable the network to learn multi-level label reasoning and contains two branches.

[0037] The first branch consists of a convolutional layer Conv1, two fully connected layers, and a Softmax layer. The input is f r , the output is the probability value of all labels in the first level.

[0038] The second branch includes a convolutional layer Conv2, two fully connected layers, and a Softmax layer. The input is the output Conv1 (f r ) and f r Concatenation of Concat(Conv1(f r ),f r ), where Concat(g) represents the feature concatenation operation. The purpose of concatenation is to allow the second-level label inference to inherit the features used for the first-level inference. The output is the probability value of all the second-level labels.

[0039] The loss function in each branch adopts the cross entropy loss function.

[0040] Step 3: Train the model in stages. In the first stage, all model parameters are trained with a learning rate of 1e-5. The first stage of training is mainly used to obtain the parameters of the backbone network that meet the requirements of multi-level classification. After the first stage of training converges, the second stage of training begins. The second stage of training locks the parameters of the backbone network and fine-tunes the parameters of the two branch modules with a small learning rate of 1e-6 to obtain the parameters of the knowledge injection module that are adapted to the labels at this level.

[0041] Step 4: Use the trained model to predict the new echo data. Figure 4The model predicts the probability values ​​of the new echo data at each level of labels. It can be seen that the first branch outputs a probability value of a1 = [0.4, 0.6], which corresponds to the first-level two labels. The model has a higher probability of judging that it is an underwater target. The second branch outputs a probability value of a2 = [0.42, 0.11, 0.41, 0.04, 0.02], which corresponds to the second-level five labels. The model has a certain ambiguity in judging whether it is a surface ship or an underwater vehicle. The probability of all paths of the two-layer labels is multiplied, and a total of 10 probability values ​​are obtained. The largest probability value is the label path corresponding to the underwater vehicle ( Figure 4 The solid line path with bold arrows in the middle) is 0.6×0.41=0.246, which is greater than the probability value of the label path corresponding to the surface ship 0.4×0.42=0.168, so the model output label is: underwater target, underwater vehicle.

[0042] By allowing the model to make multi-level decisions, the interpretability of the model output results is increased, while the impact of judgment errors at a certain level on the final results is effectively reduced, thereby increasing recognition confidence.

[0043] The above is only an explanation of the preferred embodiments of the present invention, which should not be construed as a limitation on the claims. Any equivalent process changes made using the present invention specification are included in the patent protection scope of the present invention.

Claims

1. A method for underwater acoustic target recognition based on knowledge injection deep learning, characterized in that: The following steps are included: S1: Use active sonar to collect target echo data, or use passive sonar to collect target radiation noise data, and perform multi-level labeling on the target according to actual needs to obtain a multi-level label tree. The depth of the label tree is N, and the nth level label is Among them, M n is the number of categories of the n-th level label; S2: Construct a deep learning network model for underwater acoustic target recognition based on knowledge injection. The model consists of two parts: a feature extraction module and a knowledge injection module based on multi-level labels. The feature extraction module is used to extract high-dimensional features from the target echo or radiation noise data. If the number of target samples is large, a backbone network can be used to directly extract features from the echo or radiation noise. If the number of target samples is small, the echo or radiation noise can be transformed into multiple domains. The multi-domain spectra are spliced ​​in the channel dimension, and then the backbone network is used to extract features from the spliced ​​multi-channel feature spectra. The output of the feature extraction module is the last layer output feature f before the fully connected layer of the backbone network. r ; The multi-level label-based knowledge injection module is used to enable the network to learn multi-level label reasoning. It contains N branches, where N is the depth of the multi-level label tree. The first branch consists of a convolutional layer Conv1, two fully connected layers, and a Softmax layer. The input is f r , the output is the probability value of all labels in the first level; The nth branch (n>1) includes a convolutional layer Conv n , two fully connected layers, a Softmax layer, and the input is the output of the convolutional layer in the n-1th branch Conv n-1 (f r ) and f r Concat(Conv n-1 (f r ),f r ), where Concat(g) represents the feature concatenation operation, which allows the label inference at this level to inherit the features of the previous level inference, and the output is the probability value of all labels at the nth level; The loss function in each branch adopts the cross entropy loss function; S3: The model is trained in stages. In the first stage, all model parameters are trained. After the model converges, the second stage training begins. The second stage training locks the parameters of the backbone network and fine-tunes the parameters of the branch modules with a small learning rate. S4: Perform model prediction on new data. Use the trained model to predict new data. The N branches of the model generate probability values ​​w1, w2 for the corresponding level labels respectively. n ,L,w N , where w n represents the output probability value of the nth branch, Starting from the first layer, traverse all paths to the leaf nodes at the bottom of the label tree from top to bottom, multiply the probability values ​​on the paths, and get M1·M2·LM N probability values, and select the label on the path corresponding to the maximum probability value as the output multi-level label of the echo.

2. The underwater acoustic target recognition method based on knowledge injection deep learning according to claim 1 is characterized in that: In step S2, the backbone network may be a convolutional neural network, including but not limited to ResNet, InceptionNet, Unet, and transformer.

3. The underwater acoustic target recognition method based on knowledge injection deep learning according to claim 2 is characterized in that: When using a convolutional neural network, the two-dimensional convolution kernel needs to be modified into a one-dimensional convolution kernel.

4. The underwater acoustic target recognition method based on knowledge injection deep learning according to claim 1 is characterized in that: In step S2, the echo can obtain the echo highlights in the spatiotemporal domain, the time-frequency spectrum in the time-frequency domain, and the Mel-frequency cepstrum in the auditory perception domain.

5. The underwater acoustic target recognition method based on knowledge injection deep learning according to claim 1 is characterized in that: In step S2, the radiated noise can be transformed to obtain a LOFAR spectrum, a DEMON spectrum, and an MFCC characteristic spectrum.

Images