Multi-underwater acoustic target separation and identification method based on deep learning

Abstract

The multi-underwater acoustic target separation and identification method based on deep learning comprises the steps that sonar equipment receives noise signals radiated by various underwater acoustic targets; filtering and noise reduction processing is carried out on the received signals, and after a large number of noise signals of a single target are obtained, n kinds of noise are randomly mixed into a composite signal x; a multi-noise target separation model is constructed, the input of the model is the composite signal x, and the output of the model is a separated single target noise signal set; training a multi-noise target separation model; the method comprises the following steps: constructing an underwater sound target identification model, inputting two sections of noise signals into the underwater sound target identification model, outputting a similar value of the two sections of signals, and when the output value is closer to 1, indicating that the two sections of signals are more likely to come from the same type of equipment, so as to judge which kind or several kinds of equipment receives the underwater sound signals sent by the sonar equipment; and deploying the multi-underwater acoustic target separation model and the target identification model to a computer or other computing equipment for separating and identifying any composite signal in a real scene.

Description

Technical Field

[0001] The present invention relates to the field of underwater acoustic target recognition, and in particular to a method for separating and recognizing multiple underwater acoustic targets based on deep learning. Background Art

[0002] In underwater acoustic confrontation, whether it is early warning and avoidance of underwater weapons such as torpedoes and mines or targeted attacks on targets, it is inseparable from the identification of specific underwater acoustic targets. The development of intelligent and automated underwater acoustic weapons in the new era is inseparable from the research on automatic target identification technology. With the continuous development of technologies such as electronic jamming, acoustic stealth, and acoustic decoys, whether sonar equipment can accurately identify underwater targets such as submarines, surface ships, UUVs, torpedoes, and make quick decisions, and take the initiative to gain an advantage is the key to defeating the enemy.

[0003] With the advancement of the informatization and intelligentization of naval equipment, future naval battles will be based on the coordinated use of multiple weapon systems and joint operations of formations, and the battlefield environment will become more complex and changeable. Whether underwater targets can be identified and analyzed in a timely and accurate manner and accurate battlefield information can be provided to command systems at all levels has become the key to the comprehensive combat effectiveness of surface ships and surface ship formations and their victory over the enemy. In recent years, artificial intelligence technology (AI) represented by machine learning (ML) and deep learning (DL) has developed rapidly, providing new ideas and development directions for underwater target recognition technology. Through big data, pattern recognition, and machine learning technology, in-depth research and mining of underwater acoustic target signal characteristics, and informatization and intelligent upgrading of the recognition system have become research hotspots in underwater target recognition technology.

[0004] Underwater target recognition technology is mainly divided into two categories: active recognition and passive recognition. Active recognition uses active sonar to emit sound signals and makes judgments on target attributes based on the characteristics of the received echo signals. The advantage is that the received echo signals carry a large amount of information that is conducive to classification and recognition and reflects the essential characteristics of the target; but there is an obvious disadvantage, that is, it is easy to be exposed and not conducive to self-protection. Passive recognition technology uses passive sonar to receive target radiation noise for classification and recognition. It has the characteristics of high security and strong concealment. It is suitable for the classification and recognition of long-range targets. It can provide a strong guarantee for the enemy to discover and implement effective attacks first, making it better than active recognition in terms of its own information protection. Therefore, it is the current research focus.

[0005] At present, countries around the world are vigorously promoting the development of related technologies for intelligent underwater acoustic target recognition. For example, the United States first launched the BQQ-3~BQQ-5 submarine sonar target classification system, which can accurately analyze the low-frequency line spectrum; Stanford University developed the HASP system, which introduced the concept of artificial intelligence into underwater acoustic target signal processing, and then developed a more intelligent SIAP recognition system, and applied it to ships, submarines and other equipment; MITRE has developed a fourth-generation sonar system, which has a certain underwater acoustic target recognition function; JCHassab and CHChen scholars developed the CLMA classification system based on the analysis of fuzzy theory; APRusso scholar developed a recognition and analysis system based on neural networks. Other countries in the world have also followed suit and carried out corresponding research. For example, the detection equipment based on the principle of artificial neural network developed by British scholar CRGent can distinguish two types of target signals under certain environmental noise interference; Thomson Company of France uses the DEMON spectrum characteristics of target signals for identification and analysis, and on this basis developed the TMS-8202 sonar system; Canada has developed the INTERSENSOR expert system, which can perform relatively accurate identification and analysis of target radiation noise; in addition, there are also India's RETSENSOR, Japan's SK-8 and other identification and early warning systems, as well as my country's EXPLOREO identification system. Many achievements also show that countries pay close attention to and invest heavily in new marine technologies. With the development and use of new equipment in various countries, it has brought great challenges to the current accurate identification. Therefore, it is urgent to develop a hydroacoustic target recognition system based on advanced recognition algorithms.

[0006] At present, the main research methods for underwater acoustic target recognition at home and abroad can be divided into two categories: the first is based on modern signal and information processing theory and statistical pattern recognition methods, and the method of analyzing line spectrum features based on traditional signal processing methods and then solving physical parameters. The second method generally obtains the approximate category of the target by solving the target propeller parameters and low-frequency line spectrum features and comparing them with the target parameter database. In recent years, with the rapid development of deep learning technology and its superior performance in all walks of life, a large number of underwater target recognition methods based on deep learning have emerged. This intelligent achievement also studies how to use deep learning technology to achieve multi-water acoustic target recognition.

[0007] Regarding the use of artificial intelligence technology to achieve efficient underwater target recognition, a large number of patents have provided some typical methods, which can be generally summarized as methods based on traditional machine learning, such as clustering, and methods based on deep learning, mainly including methods based on recurrent neural networks (RNN), convolutional neural networks (CNN), Transformer, etc., or a combination of multiple neural networks. The progress of submarine noise reduction technology, the rapid development of underwater unmanned vehicles, the diversification of underwater weapons such as torpedoes, and the increasingly harsh underwater complex environment have made it increasingly difficult to accurately identify underwater acoustic targets. Most of the existing methods have proposed the following main ways to solve this problem: in the data preprocessing stage, signal processing technology is used to remove environmental noise and interference in the target signal; at the feature extraction level, the multi-domain characteristics of underwater acoustic signals are fully utilized, such as time domain waveforms, frequency domain analysis, and auditory characteristics; when designing deep learning algorithms, more complex models and better training strategies are used to improve recognition accuracy; when making model decisions, the outputs of multiple models are combined to further improve system performance.

[0008] Although these methods have made great progress, they are only aimed at single hydroacoustic target recognition. In theory, their single input can only be the noise radiated by a specific underwater target, that is, the input signal can only contain the noise radiated by one equipment. However, in actual application scenarios, the navy will inevitably face multiple enemies and simultaneous attacks by multiple marine equipment and weapons in battles. At this time, the sonar equipment will simultaneously receive mixed noise signals radiated by multiple equipment, such as different hydroacoustic signals radiated by different types of torpedoes, or the radiation noise of multiple equipment such as submarines, surface ships, UUVs, torpedoes, etc. In such a scenario, the existing hydroacoustic target recognition system will not be able to directly process this mixed signal. Although the mixed signal can be separated into multiple hydroacoustic signals by traditional signal processing technology, and then each signal is input into the recognition model to obtain the classification result. However, the final recognition accuracy of the model at this time depends not only on the accuracy of the classification model itself, but also on the accuracy of the separation of the mixed signal. Underwater sound in complex environments not only includes ocean environmental noise, but is also a unified entity of radiation noise from multiple equipment. It is difficult to decompose it precisely using only traditional signal processing technology, which inevitably leads to the recognition system being unable to accurately identify underwater targets in complex scenarios.

[0009] In addition, most of the existing deep learning-based underwater acoustic target recognition models treat this problem as a classification problem. This approach can only identify a limited number of target types, and the types of targets to be identified must be included in the model training data set. However, in actual combat environments, the navy often faces new types of combat equipment. When the recognition model receives underwater target noise that has never been learned, the classifier will usually fail to make a decision or even make a wrong decision, causing immeasurable losses to naval operations. Summary of the Invention

[0010] The object of the present invention is to separate the mixed noise signals radiated simultaneously by multiple underwater acoustic targets and to identify the radiation sources of these signals. To achieve this goal, a method for separating and identifying multiple underwater acoustic targets based on deep learning is proposed. This method has high flexibility and provides more possibilities for the intelligent underwater acoustic recognition system to be applied to actual combat scenarios.

[0011] To achieve the object of the present application, the following technical solutions are adopted in the present application:

[0012] A method for separating and identifying multiple underwater acoustic targets based on deep learning according to the present invention, which includes the following steps:

[0013] (1) The sonar device receives the noise signals radiated by multiple underwater acoustic targets and performs filtering and noise reduction processing on the received signals. The steps are as follows:

[0014] (1.1) Divide the noise signal into equal-duration samples;

[0015] (1.2) Perform spatial domain filtering according to the equal-duration samples;

[0016] (1.3) Perform frequency domain filtering on the result of step (1.2) to complete filtering and noise reduction;

[0017] (1.4) Divide the processed noise signals into K types according to the types of noise-emitting devices

[0018] (2) After obtaining a large number of noise signals of single targets in step (1), randomly mix n types of noises into a composite signal x, where 1 < n ≤ K, to replace the complex noise signals radiated simultaneously by multiple underwater acoustic targets received by the sonar in the actual scenario

[0019] (2.1) Determine the number of targets K, which is the number of types of underwater acoustic signals received by the sonar device in step (1);

[0020] (2.2) Randomly select n different types of samples from the sample set and mix them into a composite signal. To simulate the underwater acoustic fusion in the real scenario as much as possible and increase the diversity of data, each time the n types of noises are mixed into the composite signal x in one of the following three ways;

[0021] (a) Synthesize using Adobe Audition professional audio editing software

[0022] Import n segments of signals into the software, adjust the volume and position of each audio track to align them, then mix all the audio tracks into the main audio track, and finally export the mixed signal;

[0023] (b) Use programming mixing method

[0024] Use Python language to call ffmpeg's pydub library or librosa library to achieve the superposition and mixing of multiple audio segments;

[0025] (c) Using a hybrid signal processing approach

[0026] First, each noise segment is converted into a digital signal through ADC, then all signals are time-aligned and amplitude-adjusted, and the adjusted signals are added in the time domain to obtain a mixed signal;

[0027] Each time, randomly select one of the three methods (a), (b) and (c) above to mix n single noise signals into a composite signal;

[0028] (2.3) Repeat steps (2.1) and (2.2) until the number of generated mixed samples reaches the training quantity requirement;

[0029] (III) Construct a multi-noise target separation model. The input of the model is the composite signal x mentioned above, and the output is a separated single target noise signal set У={y n}, which can be described as the formula:

[0030] F θ (x) = {y n}

[0031] Where θ represents the trainable parameters of the model;

[0032] (IV) Use the data from step (II) to train a multi-noise target separation model, with the composite signal x as the input of the model and the corresponding number of synthesized noise signals and mixed signal types n as labels

[0033] (4.1) Divide all the data in step (2) into a training set, a test set, and a validation set in proportion. Each piece of data consists of: the mixed composite signal x; the n single noise signals mixed; the number of mixed signals n;

[0034] (4.2) Train the parameters of the multi-noise target separation model on the training set, and optimize the counting module for predicting the number of targets to minimize the cross entropy loss function, as shown in the formula:

[0035]

[0036] Where 1 represents the exponential function, p(|У|)=k represents the mixed input signal, and x is the predicted probability of k noise targets;

[0037] For the decoder of the separated features, a permutation invariant loss function (uPit) is used:

[0038]

[0039] Among them k represents the output of the kth decoder,

[0040] uPIT(У,У k ) is formula (3):

[0041]

[0042] where π represents the arrangement of speaker channels, SI-SNR represents scale-invariant signal-to-noise ratio,

[0043] When training the model, two different loss functions are used, introducing To balance the influence of the two, the loss function of the entire network training is:

[0044]

[0045] (4.3) After each iteration of the training set, the separation accuracy is tested on the test set;

[0046] (4.4) After every 5 rounds of training set iterations, the separation accuracy is tested on the validation set. When the separation accuracy is greater than the separation accuracy of the last validation, the multi-noise target separation model is saved to ensure that the saved multi-noise target separation model is the current optimal multi-noise target separation model.

[0047] (4.5) Repeat steps (4.3) and (4.4) until the separation accuracy of step (4.3) on the test set does not improve or the loss function value does not decrease, then stop training;

[0048] (V) Construct an underwater acoustic target recognition model, input two noise signals into the underwater acoustic target recognition model, and output the similarity value of the two signals. The closer the output value is to 1, the more likely it is that the two signals are from the same type of equipment;

[0049] (5.1) The frequency spectra of two segments of arbitrary underwater acoustic target noise signals are respectively input into two encoders of the same structure of the above-mentioned underwater acoustic target recognition model to obtain two encoding features. The above-mentioned two encoders are twin encoders, and the encoder is a classic convolutional neural network resnet50, and the output is a 7×7 feature tensor;

[0050] (5.2) Reorganize the two 7×7 feature tensors in step (5.1) into two one-dimensional vectors, respectively.

[0051] is α={x1,x2,...,x n} and β={y1,y2,...,y n};---Formula (5)

[0052] (5.3) Calculate the Euclidean distance between the two vectors α and β in step (5.2) using the following formula:

[0053]

[0054] The calculation result is the similarity value of the two sections of hydroacoustic signals;

[0055] (VI) Use the noise signal in step (I) to train the underwater acoustic target recognition model. The input is a section of underwater acoustic signal and another section of underwater acoustic signal randomly selected from the data set. When the two come from the same target, they are positive samples and the corresponding label is 1; otherwise, it is 0, that is, a negative sample.

[0056] (6.1) Divide the data obtained in step (i) into training set, test set and validation set in proportion.

[0057] (6.2) During training, for each piece of training data, another piece of data is randomly selected from the training set and sent to the underwater acoustic target recognition model in pair. When the underwater acoustic targets corresponding to the two pieces of data are the same, the label is 1, otherwise the label is 0. The loss function during training is the mean square error loss function, and its formula is:

[0058]

[0059] Where n is the number of samples, y is the true value, and y is the model predicted value;

[0060] (6.3) After each iteration of the training set, the recognition accuracy is tested on the test set;

[0061] (6.4) After every 5 rounds of training set iterations, the recognition accuracy is tested on the validation set. When the accuracy is greater than the accuracy of the last validation, the underwater acoustic target recognition model is saved and replaced with the previous underwater acoustic target recognition model to ensure that the saved underwater acoustic target recognition model is the current optimal one.

[0062] (6.5) Repeat steps (6.3) and (6.4) until the accuracy of step (6.3) on the test set does not improve or the loss function value does not decrease, then stop training;

[0063] (VII) Deploy the model to a computer to separate and identify any mixed signals

[0064] The trained multi-noise target separation model and hydroacoustic target recognition model are deployed on the computer. When the sonar device receives the hydroacoustic signal, the signal is transmitted to the computer and input into the multi-noise target separation model to obtain the number of hydroacoustic targets and the corresponding single clean target hydroacoustic set. The separated signals and the noise signals of all known hydroacoustic equipment are input into the hydroacoustic target recognition model in turn to obtain the corresponding similarity values. Based on the similarity values, it is inferred which equipment’s characteristics are most similar to the characteristics of the unknown hydroacoustic target, so as to make a more accurate response to determine which equipment or equipment the hydroacoustic signal received by the sonar device is emitted by.

[0065] The present invention provides a method for separating and identifying multiple underwater acoustic targets based on deep learning, wherein: in step (iii), the constructed multiple noise target separation model includes the following steps:

[0066] (3.1) The mixed noise signal is passed through an encoder to obtain the output encoding features. The encoder is composed of a one-dimensional convolutional layer (1-D CNN), a nonlinear activation function (ReLU) and several MulCat modules in series. The content of this module is:

[0067] Each MulCat module contains an odd block and an even block, and the input dimensions of the two are different. In the odd block, the input vector is first input into the bidirectional LSTM along the second dimension, and the result is multiplied element by element, and then concatenated with the original signal along the third dimension. Finally, a linear layer is applied to obtain features of the same size as the input, and then added element by element to the original signal to obtain the output features of the odd block; the output features of the odd block are transposed in the first and second dimensions through scale transformation, and the transposed features are input into the even block. In the even block, the input vector is first input into the bidirectional LSTM along the second dimension, and the result is multiplied element by element, and then concatenated with the original signal along the third dimension. Finally, a linear layer is applied to obtain features of the same size as the input, and then added element by element to the original signal to obtain the output features of the even block. The first and second dimensions of the output of the even block are transposed and then input into the next MulCat module;

[0068] (3.2) The encoded features infer the number of targets contained in the composite signal through a counting module

[0069] First, the output features of the encoder are globally averaged and pooled, then a ReLU activation function is used, and finally, the result is passed through a linear layer to obtain a one-dimensional vector of length K, and then a SoftMax activation function is used to obtain the prediction result of the target quantity;

[0070] (3.3) According to the target number predicted in step (3.2), select the corresponding decoder from the decoder group

[0071] When the number of noise targets is k, the kth decoder is selected, and the input is the high-dimensional features encoded by the encoder. Assuming the dimension is N, PreLU and 1×1 convolution layers are first applied to expand the dimension to N×k, and then it is divided into k vectors, each with a dimension of N. Finally, the k vectors are restored to noise signals, and the clean noise signals of k targets are obtained.

[0072] The present invention provides a method for separating and identifying multiple underwater acoustic targets based on deep learning, wherein: in the multiple noise target separation model, the ratio of the training set, the test set and the validation set is 7:2:1.

[0073] The present invention provides a method for separating and identifying multiple underwater acoustic targets based on deep learning, wherein: in the underwater acoustic target recognition and separation model, the ratio of the training set, the test set and the validation set is 7:2:1.

[0074] The present invention provides a method for separating and identifying multiple underwater acoustic targets based on deep learning, wherein: in step (1), the equal-time samples are equal-time samples of 2 seconds to 6 seconds.

[0075] The invention discloses a method for separating and identifying multiple underwater acoustic targets based on deep learning, wherein: the plurality of MulCat modules are 2-5 MulCat modules. The invention discloses a method for separating and identifying multiple underwater acoustic targets based on deep learning, which has the following beneficial effects:

[0076] At present, underwater acoustic target recognition technology has been widely studied, and there are a large number of intelligent recognition systems based on deep learning technology. However, most methods require that the input underwater acoustic signal can only contain the radiation noise of one target. This premise will become a great restriction for the application of intelligent recognition systems in actual combat scenarios. This intellectual achievement proposes for the first time to use deep learning technology to separate multi-target underwater acoustic mixed signals into single underwater acoustic signals according to target types. The separated signals can not only be used for target characteristic analysis, but also can use existing underwater acoustic target recognition technology to infer the type of equipment currently attacking, so as to respond in time and take the initiative to defeat the enemy.

[0077] In addition, most of the existing intelligent recognition systems divide water acoustic target recognition into a classification problem. Such a recognition system can only recognize the water sounds of known targets that have been trained. When the system encounters water acoustic noise that it has never seen before, the system will make wrong decisions. This will also become a major limitation for the implementation of intelligent recognition systems. This intellectual achievement cleverly converts the water acoustic target recognition system into a regression problem, which can be used to calculate the similarity value between the water sound of an unknown target and any other known water sound. The larger the value, the higher the similarity between the unknown target and the known target. At this time, the navy takes similar measures to fight back. This approach will greatly increase the flexibility of operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 Schematic diagram of the overall process of a multi - underwater acoustic target separation and recognition method based on deep learning according to the present invention;

[0079] Figure 2 Schematic diagram of the structure of a multi - noise target separation model;

[0080] Figure 3 Schematic diagram of the structure of the MulCat module

[0081] Figure 4 Schematic diagram of the structure of an underwater acoustic target recognition model. Specific implementation manners

[0082] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings:

[0083] A multi - underwater acoustic target separation and recognition method based on deep learning aims to separate the noise signals radiated by an unknown number of underwater acoustic targets into several single - target signals, and can calculate the similarity value between the separated signals and any known signal, so as to infer the characteristics of the targets.

[0084] Figure 1 The overall idea of the method is described. First, the radiation noise data received by the sonar is pre - processed, then the multi - underwater acoustic signal separator is used to decompose it into single - target signals, and finally the underwater acoustic recognition model is used to calculate the similarity value between the single - target signal and the known underwater acoustic signal, and the similarity degree between the target and the characteristics of a certain known equipment is judged by the magnitude of the similarity value.

[0085] A multi - underwater acoustic target separation and recognition method based on deep learning according to the present invention includes the following steps:

[0086] (1) The sonar device receives the noise signals radiated by multiple underwater acoustic targets, and performs filtering and noise reduction processing on the received signals. The steps are as follows:

[0087] (1.1) Divide the noise signal into equal - duration samples;

[0088] (1.2) Perform spatial domain filtering according to the equal - duration samples;

[0089] (1.3) Perform frequency domain filtering on the result of step (1.2) to complete filtering and noise reduction;

[0090] (1.4) Divide the processed noise signals into K types according to the types of the noise - emitting devices

[0091] (2) After obtaining a large number of single - target noise signals in step (1), randomly mix n types of noises into a composite signal x, where 1 < n ≤ K, to replace the complex noise signals simultaneously radiated by multiple underwater acoustic targets received by the sonar in the actual scenario

[0092] (2.1) Determine the number of targets K, which is the number of types of underwater acoustic signals received by the sonar equipment in step (1);

[0093] (2.2) Randomly select n different types of samples from the sample set and mix them into a composite signal. In order to simulate the underwater acoustic fusion in the real scene as much as possible and increase the diversity of data, one of the following three methods is used each time to mix n types of noise into a composite signal x;

[0094] (a) Synthesize using Adobe Audition professional audio editing software

[0095] Import n segments of signal into the software, adjust the volume and position of each track to align them, then mix all tracks into the master track, and finally export the mixed signal;

[0096] (b) Using a mixed programming approach

[0097] Use Python language to call ffmpeg's pydub library or librosa library to achieve the superposition and mixing of multiple audio segments;

[0098] (d) Using a hybrid signal processing approach

[0099] First, each noise segment is converted into a digital signal through ADC, then all signals are time-aligned and amplitude-adjusted, and the adjusted signals are added in the time domain to obtain a mixed signal;

[0100] Each time, randomly select one of the three methods (a), (b) and (c) above to mix n single noise signals into a composite signal;

[0101] (2.3) Repeat steps (2.1) and (2.2) until the number of generated mixed samples reaches the training quantity requirement;

[0102] (III) Figure 2 As shown in the figure, a multi-noise target separation model is constructed. The input of the model is the composite signal x mentioned above, and the output is a separated single target noise signal set У={y n}, which can be described as the formula:

[0103] F θ (x) = {y n}

[0104] Where θ represents the trainable parameters of the model;

[0105] (3.1) The mixed noise signal is passed through an encoder to obtain the output encoding features. The encoder is composed of a one-dimensional convolutional layer (1-D CNN) with a kernel size of L and a stride of 2, followed by a nonlinear activation function (ReLU) and several MulCat modules in series. The size of L is set according to the length of the mixed signal in the specific case, followed by 2-5 MulCat modules. The content of this module is:

[0106] like Figure 3 As shown, each MulCat module contains an odd block and an even block, and the input dimensions of the two are different. In the odd block, the input vector is first input into the bidirectional LSTM along the second dimension, and the result is multiplied element by element, and then concatenated with the original signal along the third dimension, and finally a linear layer is applied to obtain features of the same size as the input, and then the features are added element by element to obtain the output features of the odd block; the output features of the odd block are transposed by scaling the first and second dimensions, and the transposed features are input into the even block. In the even block, the input vector is first input into the bidirectional LSTM along the second dimension, and the result is multiplied element by element, and then concatenated with the original signal along the third dimension, and finally a linear layer is applied to obtain features of the same size as the input, and then the features are added element by element to obtain the output features of the even block. The first and second dimensions of the output of the even block are transposed and then input into the next MulCat module;

[0107] (3.2) The encoded features infer the number of targets contained in the composite signal through a counting module

[0108] First, the output features of the encoder are globally averaged and pooled, then a ReLU activation function is used, and finally, the result is passed through a linear layer to obtain a one-dimensional vector of length K, and then a SoftMax activation function is used to obtain the prediction result of the target quantity;

[0109] (3.3) According to the target number predicted in step (3.2), select the corresponding decoder from the decoder group

[0110] When the number of noise targets is k, select the kth decoder, and input the high-dimensional features encoded by the encoder. Assuming the dimension is N, first apply PreLU and 1×1 convolution layers to expand the dimension to N×k, and then divide it into k vectors, each with a dimension of N. Finally, restore the k vectors to noise signals, and get the clean noise signals of k targets.

[0111] Implement the multi-noise target separation model based on Pytorch framework programming;

[0112] (IV) Use the data from step (II) to train a multi-noise target separation model, with the composite signal x as the input of the model and the corresponding number of synthesized noise signals and mixed signal types n as labels

[0113] (4.1) All the data in step (2) are divided into training set, test set and validation set in a ratio of 7:2:1. Each data is composed of: the mixed composite signal x; the n single noise signals mixed; the number of mixed signals n;

[0114] (4.2) The parameters of the multi-noise multi-noise target separation model are trained on the training set, and the counting module for predicting the number of targets is optimized to minimize the cross entropy loss function, as shown in the formula:

[0115]

[0116] Where 1 represents the exponential function, p(|У|)=k represents the mixed input signal, and x is the predicted probability of k noise targets;

[0117] For the decoder of the separated features, a permutation invariant loss function (uPit) is used:

[0118]

[0119] Among them k represents the output of the kth decoder,

[0120] uPIT(У,У k ) is formula (3):

[0121]

[0122] where π represents the arrangement of speaker channels, SI-SNR represents scale-invariant signal-to-noise ratio,

[0123] When training the model, two different loss functions are used, introducing To balance the influence of the two, the loss function of the entire network training is:

[0124]

[0125] The separation model was trained on a Linux computer equipped with a GeForce RTX 3090Ti graphics card using the training set divided in step 4.1. During training, the Adam optimizer was used to optimize the model parameters, and the default parameters (β1=0.9, β2=0.999) were used. The initial learning rate was set to 0.001, and the batch size was set to 32. The loss balance parameter α was optimized through experiments. It was first set to 0.1 and increased by 0.1 each time until 0.9, which was equivalent to verifying 9 values. The final value of α was determined based on the highest accuracy on the verification set.

[0126] (4.3) After each iteration of the training set, the separation accuracy is tested on the test set;

[0127] (4.4) After every 5 rounds of training set iterations, the separation accuracy is tested on the validation set. When the separation accuracy is greater than the separation accuracy of the last validation, the multi-noise target separation model is saved to ensure that the saved multi-noise target separation model is the current optimal multi-noise target separation model.

[0128] (4.5) Repeat steps (4.3) and (4.4) until the separation accuracy of step (4.3) on the test set does not improve or the loss function value does not decrease, then stop training;

[0129] (V) Figure 4 As shown, an underwater acoustic target recognition model is constructed, two noise signals are input into the underwater acoustic target recognition model, and the similarity value of the two signals is output. The closer the output value is to 1, the more likely it is that the two signals are from the same type of equipment.

[0130] (5.1) The frequency spectra of two segments of arbitrary underwater acoustic target noise signals are respectively input into two encoders of the same structure of the above-mentioned underwater acoustic target recognition model to obtain two encoding features. The above-mentioned two encoders are twin encoders, and the encoder is a classic convolutional neural network resnet50, and the output is a 7×7 feature tensor;

[0131] (5.2) Reorganize the two 7×7 feature tensors in step (5.1) into two one-dimensional vectors, respectively.

[0132] is α={x1,x2,...,x n} and β={y1,y2,...,y n};---Formula (5)

[0133] (5.3) Calculate the Euclidean distance between the two vectors α and β in step (5.2) using the following formula:

[0134]

[0135] The calculation result is the similarity value of the two sections of hydroacoustic signals;

[0136] The Pytorch framework is still used to implement the underwater acoustic target recognition model.

[0137] (VI) Use the noise signal in step (I) to train the underwater acoustic target recognition model. The input is a section of underwater acoustic signal and another section of underwater acoustic signal randomly selected from the data set. When the two come from the same target, they are positive samples and the corresponding label is 1; otherwise, it is 0, that is, a negative sample.

[0138] (6.1) Divide the data obtained in step (i) into training set, test set and validation set in the ratio of 7:2:1.

[0139] (6.2) During training, for each piece of data, another piece of data is randomly selected from the training set to be paired with it as the input of the model. Before inputting it into ResNet50, a short-time Fourier transform is performed on it to obtain the signal spectrum. The specific steps are as follows:

[0140] 1. Windowing: Use a window function (such as a Hamming window) to divide the signal into a series of overlapping short-time segments.

[0141] 2. Fourier transform: Apply Fast Fourier Transform (FFT) to each segment to convert the time domain signal into the frequency domain signal.

[0142] 3. Spectrum calculation: calculate the amplitude spectrum and power spectrum of each short-time segment;

[0143] 4. Save the spectrum: Arrange the spectrum results of all short-time segments in chronological order to obtain a two-dimensional matrix, which is the spectrum diagram.

[0144] The converted spectrogram is input into the recognition model for training. When the two pieces of data correspond to the same underwater acoustic target, the label is 1, otherwise the label is 0.

[0145] The loss function during training is the mean square error loss function, and its formula is:

[0146]

[0147] Where n is the number of samples, y is the true value, and y is the model prediction value.

[0148] The separation model was trained on a Linux computer equipped with a GeForce RTX 3090Ti graphics card using the training set processed in step 6.1. During training, the Adam optimizer was used to optimize the model parameters, and the default parameters (β1=0.9, β2=0.999) were used. The initial learning rate was set to 0.001, and the batch size was set to 32.

[0149] (6.3) After each iteration of the training set, the recognition accuracy is tested on the test set;

[0150] (6.4) After every 5 rounds of training set iterations, the recognition accuracy is tested on the validation set. When the accuracy is greater than the accuracy of the last validation, the underwater acoustic target recognition model is saved and replaced with the previous underwater acoustic target recognition model to ensure that the saved underwater acoustic target recognition model is the current optimal one.

[0151] (6.5) Repeat steps (6.3) and (6.4) until the accuracy of step (6.3) on the test set does not improve or the loss function value does not decrease, then stop training;

[0152] (VII) Deploy the model to a computer to separate and identify any mixed signals

[0153] The trained multi-noise target separation model and hydroacoustic target recognition model are deployed on a computer or other computing device, which is equipped with a GeForce RTX 3090Ti graphics card, and the system can be a domestic operating system. When the sonar device receives the hydroacoustic signal, the signal is transmitted to the computer and input into the multi-noise target separation model to obtain the number of hydroacoustic targets and the corresponding single clean target hydroacoustic set. The separated signals and the noise signals of all known hydroacoustic equipment are input into the hydroacoustic target recognition model in turn to obtain the corresponding similarity values. A visual interface is written in Python language, which displays the number of separated signals and the key characteristics of each signal. In addition, the known equipment noise signals with a similarity greater than 0.5 to the separated signals should also be displayed. According to their similarity values, the unknown hydroacoustic target characteristics are inferred to be most similar to which known equipment characteristics, so as to make a more accurate response. In this way, it can be inferred which one or which equipment sends the composite hydroacoustic signal received by the sonar device. The operator can decide how to deal with the unknown target based on the characteristics of these known equipment. This method is more flexible and persuasive.

[0154] The above specific examples are used to illustrate the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the art, according to the concept of the present invention, some simple deductions, modifications or substitutions can be made.

Claims

1. A method for separation and recognition of multiple underwater acoustic targets based on deep learning, characterized in that: It includes the following steps: (1). The sonar device receives the noise signals radiated by multiple underwater acoustic targets, and performs filtering and noise reduction processing on the received signals. The steps are as follows: (1.1) Divide the noise signals into equal-duration samples; (1.2) Perform spatial domain filtering according to the equal-duration samples; (1.3) Perform frequency domain filtering on the result of step (1.2) to complete filtering and noise reduction; (1.4) Divide the processed noise signals into K types according to the types of the noise-emitting devices. (2). After obtaining a large number of noise signals of single targets in step (1), randomly mix n types of noises into a composite signal x, where 1 < n ≤ K, to replace the complex noise signals simultaneously radiated by multiple underwater acoustic targets received by the sonar in the actual scenario (2.1) Determine the number of targets K, which is the number of types of underwater acoustic signals received by the sonar device in step (1); (2.2) Randomly select n samples of different types from the sample set and mix them into a composite signal. In order to simulate the underwater acoustic fusion in the real scenario as much as possible and increase the diversity of data, each time use one of the following three methods to mix n types of noises into the composite signal x; (a) Synthesize using Adobe Audition professional audio editing software Import n segments of signals into the software, adjust the volume and position of each track to align them, then mix all the tracks into the main track, and finally export the mixed signal; (b) Use the programming mixing method Use the python language to call the pydub library or librosa library of ffmpeg to achieve the superposition and mixing of multiple segments of audio; (c) Use the signal processing mixing method First convert each segment of noise into a digital signal through ADC, then perform time alignment and amplitude adjustment on all signals, and add the adjusted signals in the time domain to obtain the mixed signal; Each time randomly select one of the above methods (a), (b) and (c) to mix n types of single noise signals into a composite signal; (2.3) Repeat steps (2.1) and (2.2) until the number of generated mixed samples reaches the requirement of the training quantity; (III) Construct a multi-noise target separation model. The input of the model is the composite signal x mentioned above, and the output is a separated single target noise signal set У={y n }, which can be described as the formula: F θ (x)={y n } where θ represents the trainable parameters of the model; (4). Use the data in step (2) to train the multi-noise target separation model, with the composite signal x as the input of the model, and the corresponding synthesized noise signals and the number n of the mixed signal types as the labels (4.1) Divide all the data in step (2) into a training set, a test set and a validation set according to a ratio. The composition of each piece of data is: the mixed composite signal x; the n single noise signals mixed; the number n of the mixed signals; (4.2) Train the parameters of the multi-noise target separation model on the training set, and optimize the counting module for predicting the number of targets to minimize the cross-entropy loss function, as shown in the formula: where 1 represents the exponential function, p(|У|) = k represents the mixed input signal, and x is the predicted probability of k noise targets; For the decoder of the separated features, use the permutation invariant loss function (uPit): Among them k represents the output of the kth decoder, uPIT(У,У k ) is formula (3): where π represents the arrangement of the speaker channels, and SI-SNR represents the scale invariant signal-to-noise ratio, When training the model, two different loss functions are used, introducing To balance the influence of the two, the loss function of the entire network training is: (4.3) After each iteration of the training set, the separation accuracy is tested on the test set; (4.4) After every 5 rounds of training set iterations, the separation accuracy is tested on the validation set. When the separation accuracy is greater than the separation accuracy of the last validation, the multi-noise target separation model is saved to ensure that the saved multi-noise target separation model is the current optimal multi-noise target separation model. (4.5) Repeat steps (4.3) and (4.4) until the separation accuracy of step (4.3) on the test set no longer improves or the loss function value no longer decreases, then stop training; (V) Construct an underwater acoustic target recognition model, input two noise signals into the underwater acoustic target recognition model, and output the similarity value of the two signals. The closer the output value is to 1, the more likely it is that the two signals are from the same type of equipment; (5.1) The frequency spectra of two segments of arbitrary underwater acoustic target noise signals are respectively input into two encoders of the same structure of the above-mentioned underwater acoustic target recognition model to obtain two encoding features. The above-mentioned two encoders are twin encoders, and the encoder is a classic convolutional neural network resnet50, and the output is a 7×7 feature tensor; (5.2) Reconstruct the two 7×7 feature tensors in step (5.1) into two one-dimensional vectors, α = {x1, x2, ..., x n } and β={y1,y2,...,y n };---Formula (5) (5.3) Calculate the Euclidean distance between the two vectors α and β in step (5.2) using the following formula: The calculation result is the similarity value of the two sections of hydroacoustic signals; (VI) Use the noise signal in step (I) to train the underwater acoustic target recognition model. The input is a section of underwater acoustic signal and another section of underwater acoustic signal randomly selected from the data set. When the two come from the same target, they are positive samples and the corresponding label is 1; otherwise, it is 0, that is, a negative sample. (6.1) Divide the data obtained in step (i) into training set, test set and validation set in proportion. (6.2) During training, for each piece of training data, another piece of data is randomly selected from the training set and sent to the underwater acoustic target recognition model in pair. When the underwater acoustic targets corresponding to the two pieces of data are the same, the label is 1, otherwise the label is 0. The loss function during training is the mean square error loss function, and its formula is: Where n is the number of samples, y is the true value, and y is the model predicted value; (6.3) After each iteration of the training set, the recognition accuracy is tested on the test set; (6.4) After every 5 rounds of training set iterations, the recognition accuracy is tested on the validation set. When the accuracy is greater than the accuracy of the last validation, the underwater acoustic target recognition model is saved and replaced with the previous underwater acoustic target recognition model to ensure that the saved underwater acoustic target recognition model is the current optimal one. (6.5) Repeat steps (6.3) and (6.4) until the accuracy of step (6.3) on the test set does not improve or the loss function value does not decrease, then stop training; (VII) Deploy the model to a computer to separate and identify any mixed signals The trained multi-noise target separation model and hydroacoustic target recognition model are deployed on the computer. When the sonar device receives the hydroacoustic signal, the signal is transmitted to the computer and input into the multi-noise target separation model to obtain the number of hydroacoustic targets and the corresponding single clean target hydroacoustic set. The separated signals and the noise signals of all known hydroacoustic equipment are input into the hydroacoustic target recognition model in turn to obtain the corresponding similarity values. Based on the similarity values, it is inferred which equipment’s characteristics are most similar to the characteristics of the unknown hydroacoustic target, so as to make a more accurate response to determine which equipment or equipment the hydroacoustic signal received by the sonar device is emitted by.

2. The method for separating and identifying multiple underwater acoustic targets based on deep learning according to claim 1, characterized in that: In step (iii), the constructed multi-noise target separation model includes the following steps: (3.1) The mixed noise signal is passed through an encoder to obtain the encoded features. The encoder is composed of a one-dimensional convolutional layer (1-D CNN), a nonlinear activation function (ReLU) and several MulCat modules in series. The content of this module is: Each MulCat module contains an odd block and an even block, and the input dimensions of the two are different. In the odd block, the input vector is first input into the bidirectional LSTM along the second dimension, and the result is multiplied element by element, and then concatenated with the original signal along the third dimension. Finally, a linear layer is applied to obtain features of the same size as the input, and then added element by element to the original signal to obtain the output features of the odd block; the output features of the odd block are transposed in the first and second dimensions through scale transformation, and the transposed features are input into the even block. In the even block, the input vector is first input into the bidirectional LSTM along the second dimension, and the result is multiplied element by element, and then concatenated with the original signal along the third dimension. Finally, a linear layer is applied to obtain features of the same size as the input, and then added element by element to the original signal to obtain the output features of the even block. The first and second dimensions of the output of the even block are transposed and then input into the next MulCat module; (3.2) The encoded features infer the number of targets contained in the composite signal through a counting module First, the output features of the encoder are globally averaged and pooled, then a ReLU activation function is used, and finally, the result is passed through a linear layer to obtain a one-dimensional vector of length K, and then a SoftMax activation function is used to obtain the prediction result of the target quantity; (3.3) According to the target number predicted in step (3.2), select the corresponding decoder from the decoder group When the number of noise targets is k, the kth decoder is selected, and the input is the high-dimensional features encoded by the encoder. Assuming the dimension is N, PreLU and 1×1 convolution layers are first applied to expand the dimension to N×k, and then it is divided into k vectors, each with a dimension of N. Finally, the k vectors are restored to noise signals, and the clean noise signals of k targets are obtained.

3. The method for separating and identifying multiple underwater acoustic targets based on deep learning as claimed in claim 2, characterized in that: In the multi-noise target separation model, the ratio of the training set, the test set and the validation set is 7:2:

1.

4. The method for separating and identifying multiple underwater acoustic targets based on deep learning as claimed in claim 3, characterized in that: In the underwater acoustic target recognition separation model, the ratio of the training set, the test set and the validation set is 7:2:

1.

5. The method for separating and identifying multiple underwater acoustic targets based on deep learning as claimed in claim 4, characterized in that: In step (1), the equal-time samples are equal-time samples of 2 seconds to 6 seconds.

6. The method for separating and identifying multiple underwater acoustic targets based on deep learning as claimed in claim 5, characterized in that: The plurality of MulCat modules are 2-5 MulCat modules.

Images