A Deep Neural Network-Driven Supervised Learning Method for Sound Source Localization Driven by Underwater Acoustic Propagation Model

By using a deep neural network self-supervised learning method driven by a physical model of underwater acoustic propagation, the problem of insufficient environmental adaptability of underwater acoustic target localization under sparse measured data is solved, and high-precision sound source localization is achieved in complex environments.

CN117949954BActive Publication Date: 2026-11-17THE 715TH RES INST OF CHINA SHIPBUILDING IND CORP
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410134431.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-01-31
Publication Date
2026-11-17
Estimated Expiration
2044-01-31

AI Technical Summary

Technical Problem

In underwater acoustic target localization, existing technologies are difficult to effectively extend to the unobserved parameter space under sparse measured data conditions, resulting in model overfitting and insufficient environmental adaptability.

Method used

A deep neural network self-supervised learning method driven by a physical model of underwater acoustic propagation is adopted. By constructing a self-supervised deep learning model, simulation data generated by the physical model is used for pre-training, and training is carried out in the closed-loop iteration process. Combining forward prediction and backward perception, self-supervised learning is achieved.

Benefits of technology

It enhances the environmental adaptability and generalizability of deep neural networks in complex environments, and improves the accuracy of sound source localization.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117949954B_ABST
    Figure CN117949954B_ABST
Patent Text Reader

Abstract

This invention discloses a deep neural network self-supervised learning target localization method driven by an underwater acoustic propagation model. The method includes steps such as constructing a physical model, generating simulation data, constructing and pre-training a deep neural network, constructing a self-supervised deep learning model, iteratively training the self-supervised deep learning model, stopping training, and obtaining the distance and depth of the sound source. This invention forms a closed loop through forward physical model prediction and backward network model perception, using a self-supervised learning approach that iteratively coordinates forward prediction and backward perception to train the deep neural network model. This achieves a self-learning mode even with a limited number of actual observation samples, enhancing the environmental adaptability and generalization of the deep neural network model, and improving the accuracy of sound source localization in complex environments. It can be applied to autonomous detection and localization of manned and unmanned underwater platforms, as well as underwater acoustic buoys and other similar devices.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to an intelligent underwater acoustic target detection and localization technology, which is a deep neural network self-supervised learning sound source localization method driven by an underwater acoustic propagation model. Background Technology

[0002] This invention incorporates a physical model of underwater acoustic propagation into the deep learning model construction and training process to form a self-supervised learning mechanism. The construction of the physical model of underwater acoustic propagation is a forward prediction process that describes the changes and coupling laws of complex ocean acoustic field environments and simulates and predicts observed signals; the deep learning model is a reverse perception process that uses the received underwater acoustic field expression information as a basis to perceive the depth and distance of the target through the sound field excited by the target.

[0003] When measured data is sparse and the corresponding changes in underwater acoustic physical quantities are insufficient to cover the parameter space, the relationship between the target and environmental parameter spaces established by the physical model and the observed data supports the cognitive model in expanding its learning to the unobserved parameter space. The relationship between the observed signals, underwater acoustic field physical quantities, and target and environmental parameters described by the physical model can serve as a guide for constructing intelligent learning models, ensuring a theoretically guaranteed mapping relationship between model inputs and outputs. The physical model provides supervisory information during the cognitive model training process and is crucial for guiding the model's learning direction. The target and environmental parameters output by the reverse sensing model can be used to calculate the underwater acoustic field observation signals through the forward prediction model. The accuracy of the calculation results can be explained by comparing them with the observed signals. By introducing the forward prediction model, the performance of the reverse sensing model is no longer evaluated by comparison with manually labeled data, but rather by the cognitive model itself, avoiding the overfitting problem in model learning and reducing the application risks of the cognitive model.

[0004] By leveraging the errors between actual observation information and network prediction information, a closed loop is formed through forward prediction and backward perception processes. The deep neural network model is trained through iterative, collaborative, and self-supervised learning of the physical model's forward prediction and backward perception. This learning method introduces knowledge constraints into the solution space of the objective parameters. It makes robust target and environmental perception possible even with limited observational information. Summary of the Invention

[0005] The purpose of this invention is to provide a deep neural network self-supervised learning method for sound source localization driven by an underwater acoustic propagation model. This learning process is a self-learning mode under the condition of a small number of actual observation samples, which can establish a model foundation for underwater acoustic target perception and localization in complex environments, and greatly enhance the environmental adaptability and generalizability of the model.

[0006] The technical solution of this invention is as follows:

[0007] A deep neural network self-supervised learning method for sound source localization driven by an underwater acoustic propagation model includes the following steps:

[0008] 1) Construct a physical model: Based on the actual sonar array type, construct a physical model of underwater acoustic propagation and coupling with the sonar array;

[0009] 2) Generate simulation data: Using the physical model constructed in step 1), generate a large amount of simulation data under different environments;

[0010] 3) Construct and pre-train a deep neural network: Construct a deep neural network and pre-train it using the data generated in step 2) simulation;

[0011] 4) Construct a self-supervised deep learning model: Construct a self-supervised deep learning model using the physical model constructed in step 1) and the deep neural network pre-trained in step 3);

[0012] 5) Iterative training of the self-supervised deep learning model: Input the underwater acoustic field observation data into the self-supervised deep learning model constructed in step 4), and iteratively train the deep neural network model;

[0013] 6) Stop training and obtain sound source distance and depth: Set decision criteria to stop the deep neural network from iterating and obtain the distance and depth of the target sound source.

[0014] The features of this invention are:

[0015] 1. Using deep neural network models and underwater acoustic propagation physical models as the core, and sound field observation and environmental perception results as input, a self-supervised deep learning model driven by the underwater acoustic propagation physical model is constructed.

[0016] 2. The sound field observations reconstructed from the physical model are used as training data, and the estimation values ​​from the deep neural network are used as supervision to achieve iterative training of the self-supervised deep learning model;

[0017] 3. Introduce a stopping criterion for the closed-loop self-supervised deep learning model. When the model stops iterating, the output of the deep neural network is the distance and depth of the target sound source.

[0018] This invention forms a closed loop through the forward physical model prediction and the backward network model perception process. It trains the deep neural network model by iteratively co-training and self-supervised learning of forward prediction and backward perception, and completes the self-learning mode under the condition of a small number of actual observation samples. This enhances the environmental adaptability and generalization of the deep neural network model and improves the accuracy of sound source localization in complex environments. Attached Figure Description

[0019] Figure 1 This is a physical model diagram of the coupling between underwater acoustic propagation and sonar array in this invention;

[0020] Figure 2 This is a flowchart illustrating the iterative training process of the self-supervised deep learning model in this invention.

[0021] Figure 3 This is a flowchart illustrating the deep neural network workflow used in this invention. Detailed Implementation

[0022] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0023] This invention proposes a deep neural network self-supervised learning method for sound source localization driven by an underwater acoustic propagation model. The method includes constructing a self-supervised deep learning model based on an underwater acoustic propagation physical model and a deep neural network, enabling closed-loop self-supervised iterative training of the physical and network models, selecting the timing for stopping the iterative training of the closed-loop self-supervised deep learning model, and obtaining the distance and depth of the target sound source. The specific steps are as follows:

[0024] 1. Construct a physical model for the coupling of underwater acoustic propagation and sonar array. Commonly used sonar array types include horizontal arrays and vertical arrays. Acoustic propagation models include ray models, normal mode models, and parabolic equation models. The inputs to the acoustic propagation model include the sound source distance, sound source depth, receiving depth, signal frequency, sound velocity profile, water depth, sound velocity in the seabed sediment, and seabed sediment density. The output of the acoustic propagation model is the frequency domain sound pressure. Different acoustic propagation models have different applicable conditions and ranges. This invention follows the following rules when selecting an acoustic propagation model:

[0025] Table 1. Applicable Conditions of Sound Propagation Model

[0026]

[0027] In the table, “√” indicates that the model has physical meaning and is computationally feasible; “×” indicates that the model is not applicable or feasible; “-” indicates that there are limitations in computation speed or accuracy, and it is optional but not recommended.

[0028] The structure of the physical model for coupling underwater acoustic propagation with the sonar array in this invention is as follows: Figure 1 As shown. The sonar platform model needs to consider array size, element coordinates, number of elements, etc., as these parameters determine the receiving distance and receiving depth of each element. Simultaneously, considering the inconsistency in amplitude and phase response between elements in an actual array, phase deviation, amplitude deviation, and other factors need to be taken into account in the simulation. Let the transfer function (frequency domain) of the nth element in the array be F. n Then we have:

[0029] F n =B n ×A n ×exp(js n )

[0030] Among them, B n For the transfer function of the array without considering amplitude and phase offset, A n For the amplitude offset of the array element, s n This refers to the phase shift of the array elements.

[0031] according to Figure 1 The frequency domain sound pressure signal matrix P at the receiving depth and distance of the array, calculated by the sound propagation model, and the frequency domain transfer function matrix F of the sonar platform array are coupled, multiplied, and then subjected to an inverse Fourier transform to obtain the coupled array element time-domain sound pressure signal P. t The formula is:

[0032] P t =ifft(P·F).

[0033] 2. Using the physical model, generate simulation data under different environments. Using the physical model constructed in step 1, generate a large number of simulated sonar array element domain sound pressure signals under different parameters such as sound source distance, sound source depth, receiving depth, signal frequency, sound velocity profile, water depth, sound velocity of the bottom sediment, and bottom sediment density. At the same time, save the sound source distance and sound source depth information of the corresponding simulated sound pressure signals.

[0034] 3. Construct a deep neural network and pre-train it using the data generated in step 2. Based on the actual application scenario and requirements, construct a suitable deep neural network, such as a convolutional neural network or a recurrent neural network, and ensure that the constructed network is a supervised learning network. For cases requiring simultaneous estimation of multiple underwater acoustic physical parameters, single-output and multi-output neural networks can be designed. If the neural network is a single-output network, multiple networks need to be constructed according to the number of parameters to be estimated.

[0035] Specifically, this invention adjusts the classic residual network ResNet18 to design single-output or multi-output neural networks as needed. Figure 3 This is a 2-output deep neural network used in this invention to simultaneously estimate the distance and depth of a sound source. Here, "conv" represents a convolutional layer, "maxpool" represents a max pooling layer, "avgpool" represents an average pooling layer, "3×3" represents the size of the convolutional kernel, "64", "128", etc. represent the number of convolutional kernels, and " / 2" represents a stride of 2 for the convolutional kernel or pooling kernel. If not labeled, it means the stride is 1.

[0036] The input to the constructed deep neural network is the preprocessed sonar array element signal, which is first normalized before being input into the network.

[0037]

[0038] Where f is the signal frequency, i is the array element number, and N is the total number of array elements.

[0039] After the simulation data is preprocessed, it is used as input to a deep neural network, and the corresponding sound source distance and depth information are used as supervision labels to train the deep neural network.

[0040] 4. Construct a self-supervised deep learning model. The structure of a self-supervised deep learning model is as follows: Figure 2 As shown, its core consists of a deep neural network model and an underwater acoustic propagation physics model. The inputs to the deep neural network model include underwater acoustic field observation results and underwater acoustic field observation reconstruction results, corresponding to the two states of deep neural network underwater acoustic physics parameter estimation and iterative training, respectively. The sound source depth and distance estimation results are the outputs of the deep neural network model in the parameter estimation state, which are the inputs to the underwater acoustic propagation physics model and also the supervision labels for the deep neural network in the training state.

[0041] The inputs to the underwater acoustic propagation physical model include source distance, source depth, receiving depth, signal frequency, sound velocity profile, water depth, sound velocity in the seabed sediment, and seabed sediment density. Source distance and source depth are estimated quantities and are also the outputs of the deep neural network. Other underwater acoustic physical parameters, such as receiving depth, signal frequency, sound velocity profile, water depth, sound velocity in the seabed sediment, and seabed sediment density, are treated as known physical quantities during self-supervised learning. These known physical quantities, along with the source depth and distance output by the deep neural network, are input into the underwater acoustic propagation physical model to reconstruct the sound field observation results. The reconstructed sound field observation results can be used as input to the deep neural network during training and can also be used to evaluate the completion of the deep neural network training.

[0042] 5. Input the underwater acoustic field observation data into the self-supervised deep learning model and iteratively train the deep neural network model. The working mode of the self-supervised deep learning model is as follows: Figure 2 As shown, the sound field observation results are input into a pre-trained deep neural network model. The network outputs the sound source distance and depth. The network estimation results, along with the known physical quantities from step 4, are then input into the physical model of underwater acoustic propagation and sonar coupling to obtain the reconstructed sound field observation values. These reconstructed observation values ​​are used as input to the deep neural network, and the sound source distance and depth output by the deep neural network are used as supervision labels to train and update the pre-trained deep neural network. After training, the sound field observation results are input into a retrained deep neural network, and the above process is repeated to complete another round of iteration. In the above iterative training process, the required supervision information is provided by the deep neural network itself, which is a self-supervised training and learning process.

[0043] 6. Set decision criteria to stop the deep neural network from iterating and obtain the distance and depth of the target sound source. After multiple iterations of training on the current batch of underwater acoustic field observation data, training can be stopped according to the decision criteria. The evaluation criteria for judging the completion of deep neural network training include the following two:

[0044] (1) Once the similarity between the sound field observation results and the sound field observation reconstruction results (or some features of both) reaches a certain standard, it can be determined that the network has completed training under this batch of sound field observation data. Specifically, this invention judges the similarity by the cross-correlation coefficient between the observed signal and the reconstructed signal, and sets the corresponding cross-correlation coefficient threshold according to the positioning accuracy requirements.

[0045] (2) After completing the set number of iterations, the closed-loop self-supervised deep learning model determines that the network has completed training on the sound field observation data of that batch.

[0046] Once the network stops training according to the evaluation criteria, the output of the deep neural network is the estimated distance and depth of the target sound source.

[0047] This invention provides a closed-loop self-supervised learning mechanism that incorporates a physical model of underwater acoustic propagation into the construction and training process of a deep learning model, enabling sound source localization in complex environments. The construction of the underwater acoustic propagation physical model is a forward prediction process used to reconstruct the underwater acoustic field; the deep learning model is a backward sensing process used to perceive the distance and depth of the sound source. By forming a closed loop through the forward prediction and backward sensing processes, the deep neural network model is trained through iterative and collaborative self-supervised learning of the physical model's forward prediction and backward sensing. This achieves a self-learning mode with a limited number of actual observation samples, establishing a model foundation for underwater acoustic target perception and localization in complex environments, and greatly enhancing the environmental adaptability and generalizability of the deep model. The method of this invention can be effectively applied to the autonomous detection and localization of manned and unmanned underwater platforms, as well as underwater acoustic buoys and other similar devices.

[0048] It should be understood that the embodiments and examples discussed herein are merely illustrative and may be modified or altered by those skilled in the art, and all such modifications and alterations should fall within the scope of protection of the claims to which this invention pertains.

Claims

1. A deep neural network self-supervised learning method for sound source localization driven by an underwater acoustic propagation model, characterized in that, Includes the following steps: 1) Construct a physical model: Based on the actual sonar array type, construct a physical model of underwater acoustic propagation and coupling with the sonar array; 2) Generate simulation data: Using the physical model constructed in step 1), generate a large amount of simulation data under different environments; 3) Construct and pre-train a deep neural network: Construct a deep neural network and pre-train it using the data generated in step 2) simulation; 4) Construct a self-supervised deep learning model: Construct a self-supervised deep learning model using the physical model constructed in step 1) and the deep neural network pre-trained in step 3); 5) Iterative training of the self-supervised deep learning model: Input the underwater acoustic field observation data into the self-supervised deep learning model constructed in step 4), and iteratively train the deep neural network model; 6) Stop training and obtain sound source distance and depth: Set decision criteria to stop the deep neural network from iterating and obtain the distance and depth of the target sound source; In step 4), the core of the constructed self-supervised deep learning model consists of a deep neural network model and an underwater acoustic propagation physical model. The inputs of the deep neural network model include underwater acoustic field observation results and underwater acoustic field observation reconstruction results. The estimated results of sound source depth and sound source distance are the outputs of the deep neural network model under parameter estimation state, which are the inputs of the underwater acoustic propagation physical model and also the supervision labels of the deep neural network under training state. The inputs of the underwater acoustic propagation physical model include sound source distance, sound source depth, receiving depth, signal frequency, sound velocity profile, water depth, sound velocity of the bottom sediment, and bottom sediment density. Among them, sound source distance and sound source depth are the outputs of the deep neural network model, while receiving depth, signal frequency, sound velocity profile, water depth, sound velocity of the bottom sediment, and bottom sediment density are known physical quantities in self-supervised learning. After inputting the known physical quantities together with the output of the deep neural network model into the underwater acoustic propagation physical model, the sound field observation results are reconstructed. The reconstructed sound field observation results are used as the inputs of the deep neural network under training state and also used to evaluate the degree of completion of deep neural network training.

2. The deep neural network self-supervised learning sound source localization method driven by the underwater acoustic propagation model according to claim 1, characterized in that: In step 1), the sound propagation model includes a ray model, a normal wave model, and a parabolic equation model. The inputs of the sound propagation model include the sound source distance, sound source depth, receiving depth, signal frequency, sound velocity profile, water depth, sound velocity of the bottom sediment, and density of the bottom sediment. The output of the sound propagation model is the frequency domain sound pressure.

3. The deep neural network self-supervised learning sound source localization method driven by the underwater acoustic propagation model according to claim 1, characterized in that: In step 1), under shallow sea and low frequency conditions, the normal mode model is selected regardless of distance, and the parabolic equation model is selected depending on distance; under shallow sea and high frequency conditions, the normal mode model is selected regardless of distance, and the ray model is selected depending on distance; under deep sea and low frequency conditions, the normal mode model is selected regardless of distance, and the parabolic equation model is selected depending on distance; under deep sea and high frequency conditions, the ray model is selected regardless of whether it is independent of distance or related to distance.

4. The deep neural network self-supervised learning sound source localization method driven by the underwater acoustic propagation model according to any one of claims 1 to 3, characterized in that: In step 1), let the transfer function of the nth element in the array be... F n Then we have: , in, B n The transfer function of the array without considering amplitude and phase offset. A n For the amplitude offset of the array element, s n For the phase shift of the array elements; The frequency domain acoustic pressure signal matrix P at the receiving depth and distance of the array, calculated based on the selected underwater acoustic propagation model, and the frequency domain transfer function matrix F of the sonar platform array, are coupled, multiplied, and then subjected to an inverse Fourier transform to obtain the coupled array element time-domain acoustic pressure signal P. t , 。 5. The deep neural network self-supervised learning sound source localization method driven by the underwater acoustic propagation model according to claim 1, characterized in that: In step 2), the physical model constructed in step 1) is used to generate a large number of simulated sonar array element domain sound pressure signals with different sound source distances, sound source depths, receiving depths, signal frequencies, sound velocity profiles, water depths, sound velocity of the bottom sediment, and density of the bottom sediment. At the same time, the sound source distance and sound source depth information of the corresponding simulated sound pressure signals are saved.

6. The deep neural network self-supervised learning sound source localization method driven by the underwater acoustic propagation model according to claim 1, characterized in that: In step 3), the deep neural network is a convolutional neural network or a recurrent neural network, and the constructed network is a supervised learning network. The designed neural network is a single-output neural network or a multi-output neural network, wherein the single-output neural network constructs multiple networks according to the estimated number of parameters.

7. The deep neural network self-supervised learning sound source localization method driven by the underwater acoustic propagation model according to claim 1, characterized in that: In step 3), the input to the constructed deep neural network is the preprocessed sonar array element signal, which is normalized before being input into the network. The formula is: , in, f For signal frequency, i The array element ordinal number. N This represents the total number of array elements. After the simulation data is preprocessed, it is used as input to a deep neural network, and the corresponding sound source distance and depth information are used as supervision labels to train the deep neural network.

8. The deep neural network self-supervised learning sound source localization method driven by the underwater acoustic propagation model according to claim 1, characterized in that: In step 5), the sound field observation results are first input into the pre-trained deep neural network model, and the network outputs the sound source distance and sound source depth. Then, the network estimation results and the known physical quantities from step 4) are input into the underwater acoustic propagation physical model coupled with sonar to obtain the reconstructed sound field observation values. The reconstructed sound field observation values ​​are used as the input of the deep neural network model, and the sound source distance and sound source depth output by the deep neural network model are used as supervision labels to train and update the pre-trained deep neural network. After training is completed, the sound field observation results are input into the retrained deep neural network, and the above process is repeated to complete another round of iterative updates.

9. The deep neural network self-supervised learning sound source localization method driven by the underwater acoustic propagation model according to claim 1, characterized in that: In step 6), the evaluation criteria for determining the completion of deep neural network training include two types: the similarity between the sound field observation results and the sound field observation reconstruction results, or the similarity of some features of the sound field observation results reaching a certain standard, or the number of iterations reaching a set value. After training stops, the output of the deep neural network is the estimated distance and depth of the target sound source.

Citation Information

Patent Citations

  • Signal modeling method for coupling of ray sound propagation model and circular array transfer function

    CN116992648A

  • Model-based reinforcement learning for behavior prediction

    US20220138568A1