An underwater sound source positioning method and system based on underwater acoustic-positioning text generation alignment

Abstract

The application discloses an underwater sound source positioning method and system based on underwater acoustic-positioning text generation alignment, which acquires multi-channel underwater acoustic signals and corresponding structured positioning text labels, and performs Tokenization processing on the positioning text labels; an underwater acoustic multi-modal converter model with an encoder-decoder structure is constructed, a generative alignment of underwater positioning features and decoding end text features is realized through a cross attention mechanism, and a positioning text sequence containing depth and distance information is generated in a self-recurrence manner. During training, the model is used as a student model, an exponential moving average copy thereof is constructed as a teacher model, a loss is calculated based on clean input and noise disturbance input, and distance numerical Token and depth numerical Token are subjected to a weight higher than that of ordinary structure Token. During reasoning, the generated structured positioning text sequence is parsed to obtain underwater sound source depth values and distance values, so that the positioning result is determined.

Description

Technical Field

[0001] This invention relates to the field of underwater sound source localization and deep learning technology, specifically to an underwater sound source localization method and system based on underwater acoustic-localization text generation alignment. Background Technology

[0002] Underwater sound source localization is a key technology for estimating the location of underwater sound sources using hydrophone arrays and other underwater acoustic devices. It is widely used in fields such as marine exploration, target monitoring, marine resource development, and underwater safety. The underwater sound propagation environment is complex. Factors such as sound velocity profiles, seabed boundaries, sea surface undulations, multipath propagation, and environmental noise can all affect the characteristics of the received signal, making accurate estimation of the sound source depth and distance quite challenging.

[0003] Traditional underwater acoustic source localization methods mainly include matched field processing, beamforming, and time difference of arrival (TDOA) estimation. Among these, the matched field processing method typically requires establishing a relatively accurate marine environment model and relies on prior information such as sound velocity profiles, seabed parameters, and array positions. When the actual marine environment deviates from the pre-set model, environmental mismatch can easily occur, leading to location peak shifts, increased sidelobe interference, or decreased location accuracy. Therefore, in practical application scenarios where environmental parameters are difficult to obtain accurately or underwater acoustic propagation conditions vary significantly, the adaptability and robustness of traditional methods are limited.

[0004] In recent years, deep learning methods have been increasingly applied to underwater acoustic source localization tasks. These methods typically utilize neural networks to automatically extract features from underwater acoustic signals and output the sound source's depth, distance, or location category. Compared to traditional model-driven methods, this approach reduces reliance on precise environmental parameters to some extent. However, existing deep learning localization schemes usually treat sound source depth estimation and distance estimation as separate prediction tasks. This can easily lead to problems such as inconsistent convergence speeds between the two tasks, significant differences in loss magnitude, and mutual interference between parameter update directions. This necessitates the design of additional task weights, loss balancing, or gradient adjustment strategies, increasing the complexity of model training and engineering parameter tuning.

[0005] Furthermore, existing deep learning-based localization methods, such as classification or regression, also have certain limitations. Classification methods typically require dividing the sound source depth and distance space into discrete grids. If the grid is too coarse, the localization resolution is insufficient; if the grid is too fine, the number of categories increases and the sample distribution becomes sparse, leading to increased difficulty in model training. Although regression methods can directly output continuous values, the prediction results are prone to significant deviations under low signal-to-noise ratios, strong noise disturbances, or changing environmental conditions, making it difficult to guarantee output stability and localization accuracy.

[0006] Therefore, there is a need for an underwater acoustic source localization method that can reduce the complexity of multi-task localization parameter tuning and improve the accuracy and robustness of joint depth and distance estimation in complex underwater acoustic environments and under low signal-to-noise ratio conditions. Summary of the Invention

[0007] Purpose of the invention: To address the shortcomings of existing technologies, this invention provides an underwater sound source localization method and system based on underwater acoustic-localization text generation alignment, which improves the accuracy, robustness, and training stability of joint localization of underwater sound sources based on depth and distance.

[0008] Technical solution: An underwater sound source localization method based on underwater acoustic-localization text generation alignment, comprising the following steps:

[0009] A training sample set is obtained, which includes multi-channel underwater acoustic signal samples and their corresponding location text labels. The location text labels are structured text sequences that at least represent the depth and distance information of the underwater sound source. The location text labels are then tokenized to obtain a target location text token sequence.

[0010] An underwater acoustic multimodal converter (HMT) model is constructed, which adopts an encoder-decoder structure. The encoder is used to encode the features of multi-channel underwater acoustic signals to obtain underwater acoustic localization features. The decoder is used to obtain the decoding text features based on the target localization text token sequence during the training phase and to obtain the decoding text features based on the generated preceding tokens during the inference phase. The decoding text features are then generatively aligned with the underwater acoustic localization features through a cross-attention mechanism to generate a localization text sequence containing depth and distance information in an autoregressive manner.

[0011] The HMT model is used as the student model, and its exponential moving average replica is constructed as the teacher model. Clean input paths and noisy perturbation input paths are constructed for the same multi-channel underwater acoustic signal sample. The weighted label smoothing cross-entropy loss is calculated based on the clean input path. The KL divergence consistency loss between the probability distributions of the location text tokens output by the teacher model and the student model is calculated based on the noisy perturbation input path, and the student model parameters are updated with the weighted sum of the two. When calculating the weighted label smoothing cross-entropy loss, the loss weights are set according to the field category and token type of the target location text token, so that the weight of the distance numerical token is greater than the weight of the depth numerical token, and both are greater than the weight of the ordinary structure token.

[0012] The system acquires multi-channel underwater acoustic signals from the underwater sound source to be located, inputs them into the trained HMT model to generate a structured localization text sequence, parses the depth and distance tokens in the sequence to obtain the depth and distance values ​​of the underwater sound source, and determines the localization result of the underwater sound source.

[0013] Furthermore, the encoder of the HMT model includes a convolutional encoder and a Transformer coding layer; the convolutional encoder is used to downsample the multi-channel underwater acoustic signal and extract local acoustic features to obtain the underwater acoustic signal embedding sequence; the Transformer coding layer is used to perform global feature modeling on the underwater acoustic signal embedding sequence to obtain the correlation features between different underwater acoustic channels and between different time segments.

[0014] Furthermore, the location text tag is tokenized to obtain the target location text token sequence, which includes a start character, an end character, a padding character, an unknown character, a structure token, a field identifier token, a depth value token, and a distance value token.

[0015] Furthermore, the location text labels are structured text sequences in JSON format, with depth and distance information represented by numeric characters;

[0016] In this process, numeric characters in the depth field are converted into depth value tokens with depth field identifiers, and numeric characters in the distance field are converted into distance value tokens with distance field identifiers, in order to distinguish the semantics of the same numeric characters in different physical quantities. The depth value tokens include d0 to d9, and the distance value tokens include r0 to r9. The depth field and the distance field are respectively filled to a fixed length so that the corresponding numerical positions in different positioning text labels are in the same token position.

[0017] Furthermore, the weighted label smoothing cross-entropy loss is obtained by weighted summation of the label smoothing cross-entropy losses at each position in the output sequence;

[0018]

[0019] Where N is the total number of tokens in the sequence. For the token weight function, For label smoothing cross-entropy loss, For the student model, output logits at position i. The target Token ID; the label smooth cross-entropy loss. The calculation formula is:

[0020]

[0021] in V is the smoothing coefficient, and V is the vocabulary size. For the student model, the target token The predicted probability, PAD is the Token ID corresponding to the padding character, and c is the traversal sequence number of all Tokens with IDs from 0 to V-1 in the vocabulary;

[0022] The token weighting function is constructed to apply a depth weight to depth-value tokens, a distance weight to distance-value tokens, and a default weight to ordinary structure tokens, satisfying the following condition: distance weight > depth weight > default weight.

[0023] Furthermore, the noise disturbance input path obtains a noisy underwater acoustic signal by adding Gaussian noise to the clean underwater acoustic signal;

[0024] The teacher model and the student model perform forward propagation on the noisy underwater acoustic signal to obtain the first localization text token probability distribution output by the teacher model and the second localization text token probability distribution output by the student model.

[0025] The KL divergence consistency loss is calculated based on the probability distribution of the first location text token and the probability distribution of the second location text token, and the probability distribution of the first location text token output by the teacher model does not participate in gradient backpropagation.

[0026] An underwater acoustic source localization system based on underwater acoustic-localization text generation alignment includes:

[0027] The training sample acquisition module is used to acquire a training sample set, which includes multi-channel underwater acoustic signal samples and their corresponding positioning text labels. The positioning text labels are structured text sequences that at least represent the depth and distance information of the underwater sound source. The positioning text labels are tokenized to obtain a target positioning text token sequence.

[0028] The model building module is used to build an underwater acoustic multimodal converter (HMT) model. The HMT model adopts an encoder-decoder structure. The encoder is used to encode the features of multi-channel underwater acoustic signals to obtain underwater acoustic localization features. The decoder is used to obtain the decoding text features based on the target localization text token sequence during the training phase and to obtain the decoding text features based on the generated preceding tokens during the inference phase. The decoding text features are then generatively aligned with the underwater acoustic localization features through a cross-attention mechanism to generate a localization text sequence containing depth and distance information in an autoregressive manner.

[0029] The model training module uses the HMT model as the student model and constructs an exponential moving average copy of it as the teacher model. For the same multi-channel underwater acoustic signal sample, a clean input path and a noisy perturbation input path are constructed. A weighted label-smooth cross-entropy loss is calculated based on the clean input path. A KL divergence consistency loss is calculated between the probability distributions of the location text tokens output by the teacher model and the student model based on the noisy perturbation input path. The student model parameters are updated using the weighted sum of the two. When calculating the weighted label-smooth cross-entropy loss, loss weights are set according to the field category and token type of the target location text token, ensuring that the weight of distance numerical tokens is greater than the weight of depth numerical tokens, and both are greater than the weight of ordinary structure tokens.

[0030] The localization result generation module is used to acquire multi-channel underwater acoustic signals of the underwater sound source to be located, input them into the trained HMT model to generate a structured localization text sequence, parse the depth and distance tokens in it, obtain the depth and distance values ​​of the underwater sound source, and determine the localization result of the underwater sound source.

[0031] The present invention also provides an electronic device, characterized in that it includes: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the underwater sound source localization method based on underwater acoustic-localization text generation alignment as described above.

[0032] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the underwater sound source localization method based on underwater acoustic-localization text generation alignment as described above.

[0033] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the underwater sound source localization method based on underwater acoustic-localization text generation alignment as described above.

[0034] Beneficial effects:

[0035] (1) This invention constructs the depth and distance information corresponding to the multi-channel underwater acoustic signals into structured localization text labels, and directly generates localization text sequences containing depth and distance through an underwater acoustic multimodal converter model in an autoregressive text generation mode. This transforms the traditional multi-task localization problem of depth prediction and distance prediction into a single text generation task, avoiding the problems of gradient conflict, inconsistent loss magnitude and repeated adjustment of task weights in multi-task learning, and reducing the complexity of model training.

[0036] (2) The present invention extracts underwater acoustic localization features through an encoder, and obtains decoding end text features by using the token sequence corresponding to the localization text label during the training phase and the generated preceding token during the inference phase through a decoder. It also achieves generative alignment between underwater acoustic localization features and decoding end text features by using a cross-attention mechanism, so that the model can make full use of depth and distance related information in the underwater acoustic signal when generating each localization text token, thereby improving the expressive ability between underwater acoustic signal and localization result.

[0037] (3) In the training phase, the present invention constructs a teacher model and a student model, and sets a clean input path and a noise-perturbed input path for the same underwater acoustic sample. The weighted label smooth cross-entropy loss ensures that the model learns the real localization text label. The KL divergence consistency loss constrains the output distribution of the teacher model and the student model to be consistent under noise perturbation, so that the model learns underwater acoustic localization features that are not sensitive to noise changes, thereby improving the localization robustness in complex underwater environments and low signal-to-noise ratio conditions.

[0038] (4) In the weighted label smooth cross-entropy loss, the present invention sets different loss weights according to the field category and token type of the target localization text token, so that the weight of the distance numerical token is greater than the weight of the depth numerical token, and both are greater than the weight of the ordinary structure token, so that the model training process pays more attention to the key numerical tokens that have a greater impact on the final localization error, and reduces the localization deviation caused by distance and depth number generation errors.

[0039] (5) Experimental results show that the present invention achieves a significant performance improvement in distance prediction accuracy (≤100m), increasing from 27.57% of the baseline HMT model to 92.35% of the complete solution of the present invention, i.e., the distance prediction accuracy is improved to 3.35 times that of the baseline, and the mean absolute distance error is reduced from 275.6m to 51.4m. At the same time, the depth prediction accuracy (≤2m) is improved from 91.57% of the baseline to 97.47%. This fully verifies the effectiveness and synergistic effect of the consistency regularization and numerical token weighting strategy proposed in the present invention. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the environmental parameters used in constructing the simulated underwater acoustic scene and training sample set in an embodiment of the present invention.

[0041] Figure 2 This is a schematic diagram of the overall structure of the underwater acoustic multimodal converter model in an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of the dual-path training process with consistency regularization during training in an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram comparing the positioning performance of the method described in this embodiment of the invention with that of the baseline method under different signal-to-noise ratio conditions. Detailed Implementation

[0044] To provide a clearer understanding of the features and advantages of the technical solution of the present invention, the composition and implementation of the specific solution are described below in conjunction with the accompanying drawings.

[0045] The underwater sound source localization method based on underwater acoustic-localization text generation alignment proposed in this invention includes the following steps:

[0046] S1. Obtain multi-channel underwater acoustic signals and corresponding structured positioning text labels, construct a training sample set, and tokenize the positioning text labels;

[0047] The training sample set includes multi-channel underwater acoustic signal samples and corresponding location text labels for each multi-channel underwater acoustic signal sample. The multi-channel underwater acoustic signals can be acquired by a hydrophone array or obtained through simulation based on an underwater acoustic propagation model. Each signal sample corresponds to a sound source depth value and a sound source distance value. In this embodiment of the invention, the simulation dataset is constructed using the KRAKEN reference SwellEx-96 experiment. Figure 1 The image shows the sound velocity profile and related environmental parameters. The upper part is the sound velocity profile, with a water depth of 216.5 m. The lower part is the seabed: the uppermost layer of the seabed is a 23.5 m thick sedimentary layer, which is adjacent to the water layer and has a density of 1.76 g / cm³. 3 The attenuation coefficient is 0.2 dB / kmHz, and the top velocity of sound is... The velocity of sound at the bottom is 1572.3 m / s. The velocity is 1593.0 m / s; below the sedimentary layer lies an 800 m deep mudstone layer with a density of 2.06 g / cm³. 3 The attenuation coefficient is 0.06 dB / kmHz, and the sound velocity at the top of the mudstone layer is... The speed of sound is 1881 m / s, while at the bottom of this layer, the speed of sound increases to... =3245m / s; the lowest point is the seabed semi-space, with a density of 2.66 g / cm³. 3The attenuation coefficient was 0.02 dB / kmHz, and the speed of sound was 5200 m / s. The experimental scenario was set as a vertical linear array (VLA) containing 21 elements. The distribution of the elements was consistent with the actual layout of the undamaged elements in the SwellEx-96 experiment in event S5 (the elements were evenly distributed at equal intervals between depths of 94.125 m and 212.25 m, comprising a total of 22 elements, with the 8th element damaged). When generating data, the distance range was set to 5-10000 m, with sampling every 5 m; the depth range was set to 2-200 m, with sampling every 2 m. A total of 2000 × 100 simulation samples were obtained, with each sample lasting 1 s and a sampling rate of 1500 Hz. 15% of the total data was randomly selected for the training set, and the validation and test sets each contained 4000 randomly selected samples. After adding additional noise to the test set, it was divided into five signal-to-noise ratio stages: clean, 20dB, 15dB, 10dB, and 5dB, with 800 samples in each stage.

[0048] The location text tags are structured text sequences in JSON format, such as {"target":"1","depth":"152","distance":"3740"}, where depth and distance are represented by pure numeric characters. The location text tags are tokenized by segmenting the JSON text at the character level and extracting each character and its corresponding semantic category, including special characters (start character BOS, end character EOS, padding character PAD, unknown character UNK), JSON structure characters (curly braces, quotation marks, colons, commas), field identifier characters (Chinese characters corresponding to "target", "depth", and "distance"), depth numeric characters (numbers 0 to 9, segmented and prefixed with "d" to form depth numeric tokens: d0, d1, ..., d9, e.g., depth value "152" is converted to depth numeric tokens "d1", "d5", "d2"), and distance numeric characters (numbers 0 to 9, segmented and prefixed with "r" to form distance numeric tokens: r0, r1, ..., r9). By adding different prefixes, the semantics of the same numeric characters in different physical quantities are distinguished, and the depth and distance fields each reach a fixed number of tokens (3 numeric tokens for depth and 4 numeric tokens for distance), thus ensuring that corresponding numeric bits in different localization text labels are in the same token position. The tokenized target localization text token sequence is the target sequence input to the decoder during training, used as known label input during training, and generated by the model's autoregression during inference. After inference, the localization text token sequence generated by the model is parsed, the depth and distance numeric tokens are extracted, the "d" and "r" prefixes are removed, and the numeric characters are concatenated to obtain the depth and distance values ​​of the underwater sound source.

[0049] Table 1 shows the mapping relationship between each Token and the Token ID after the localization text label is processed by Tokenization: the BOS start symbol is mapped to ID 0, the EOS end symbol is mapped to ID 1, the PAD padding symbol is mapped to ID 2, the UNK unknown symbol is mapped to ID 3, JSON structure characters (braces, quotation marks, colons, commas) are sequentially assigned IDs 4 to 7 in lexicographical order, depth value Tokens d0 to d9 are mapped to IDs 8 to 17, distance value Tokens r0 to r9 are mapped to IDs 18 to 27, and Chinese characters ("mu", "biao", "shen", "du", "ju", "li") are sequentially assigned IDs 28 to 35 in lexicographical order. The vocabulary size V=36.

[0050] Table 1 Mapping Relationship Between Localization Text Tokens and Token IDs

[0051]

[0052] S2. Construct an encoder-decoder structured Hydroacoustic Multimodal Transformer (HMT) model, which realizes generative alignment of hydroacoustic localization features and decoder-side text features through a cross-attention mechanism, and generates a localization text sequence containing depth and distance information in an autoregressive manner.

[0053] Different from traditional classification-based localization methods and regression-based localization methods, the present invention represents the underwater sound source localization result as a structured localization text sequence, generates the localization text containing depth information and distance information in an autoregressive text generation manner through the hydroacoustic multimodal transformer HMT model, and obtains the depth value and distance value of the underwater sound source by parsing the localization text.

[0054] Figure 2The architecture of the HMT model is shown, including an encoder module, a decoder module, and a generator module. The encoder module can include a convolutional encoder and a Transformer encoding layer. The convolutional encoder is used to downsample the multi-channel underwater acoustic signal and extract local acoustic features, while the Transformer encoding layer is used to model the correlations between different channels and different time segments. The decoder module obtains decoding text features from the text input and combines them with the underwater acoustic localization features output by the encoder to generate a localization text sequence. During the training phase, the decoder obtains decoding text features based on the target localization text token sequence corresponding to the localization text label; during the inference phase, the decoder obtains decoding text features based on the generated preceding tokens. The decoder uses a cross-attention mechanism to perform attention fusion and generative alignment of the decoding text features and underwater acoustic localization features, enabling the model to progressively generate localization text sequences containing depth and distance information in an autoregressive manner. The generator module maps the features output by the decoder to logits of the vocabulary size and obtains the probability distribution of each candidate target token.

[0055] During the training phase, the target localization text token sequence input to the decoder is formatted as follows: a BOS start token followed by a tokenized localization text token sequence. In this sequence, the depth token has a "d" prefix, and the distance token has an "r" prefix. A complete sequence example is: BOS → "{" → """ → "target" → """ →:" → """ → "1" → """ → "," → """ → "depth" → "degree" → """ → ":" → """ → "d1" → "d5" → "d2" → """ → "," → """ → "distance" → "distance" → """ → ":" → """ → "r3" → "r7" → "r4" → "r0" → """ → "}" → EOS. During the inference phase, the decoder autoregressively generates subsequent tokens starting from the BOS start token until the EOS end token is generated. After inference is completed, the generated location text token sequence is parsed, and the numerical tokens with the prefixes "d" and "r" are extracted. After removing the prefixes, the numerical characters are concatenated to obtain the sound source depth value and distance value, thus completing the underwater sound source localization.

[0056] In this embodiment of the invention, the specific parameter configuration of the HMT model is as follows: encoder layer N=4, decoder layer N=4, model hidden dimension d_model=768, feedforward network dimension d_ff=2048, multi-head attention head h=8, and Dropout ratio=0.3. The source input is a 21-channel underwater acoustic signal, which is downsampled to a 768-dimensional feature representation at 48 time steps by a Contrastive Predictive Coding (CPC) style convolutional encoder; the decoder receives a text vector, avoids future information leakage through masked self-attention, and then performs cross-attention calculation with the encoder output to achieve modal alignment between signal features and text features. The target output is a structured JSON format text sequence, with the template format: {"target":"1","depth":"dXdXdX","distance":"rXrXrXrX"}.

[0057] Specifically, the encoder's calculation process is as follows: After the 21-channel underwater acoustic signal is input into the convolutional encoder, it first passes through the first one-dimensional convolutional layer (with a kernel size of 9, a stride of 4, and 768 output channels) to obtain the downsampled feature sequence, which is then processed by channel normalization and the ReLU activation function. Subsequently, it passes through multiple convolutional layers for further downsampling and local acoustic feature extraction, finally outputting a 768-dimensional underwater acoustic signal embedding sequence with 48 time steps. The underwater acoustic signal embedding sequence is input into an encoder consisting of 4 Transformer coding layers, each containing a multi-head self-attention sublayer and a feedforward neural network sublayer. The self-attention mechanism models the global correlation between different underwater acoustic channels and between different time segments to obtain underwater acoustic localization features.

[0058] The decoder's computation process is as follows: the target localization text token sequence is processed by the embedding layer and the position encoding layer to obtain the text feature vector, which is input to the decoder consisting of four Transformer decoding layers. Each layer contains a masked multi-head self-attention sub-layer, a cross-attention sub-layer, and a feedforward neural network sub-layer. Among them, the masked multi-head self-attention sub-layer ensures that the prediction of each position depends only on the generated preceding token through causal masking, realizing the autoregressive generation of the text sequence. The cross-attention sub-layer uses the decoder's text features as the query (Q) and the encoder's output underwater acoustic localization features as the key (K) and value (V), calculates the attention weights, and performs weighted aggregation of the underwater acoustic localization features to achieve generative alignment between the underwater acoustic localization features and the decoder's text features, so that the model can fully pay attention to the depth and distance related information in the underwater acoustic signal when generating each localization text token.

[0059] S3. During training, the model is used as a student model, and its exponential moving average copy is constructed as a teacher model. The loss is calculated based on clean input and noisy perturbation input, and the distance value token and the depth value token are given higher weights than the ordinary structure token.

[0060] like Figure 3 As shown, in each training batch, a clean input path and a noisy perturbation input path are constructed for the same multi-channel underwater acoustic signal sample. The clean input path is used to maintain the model's supervised learning ability on the real localization text labels, while the noisy perturbation input path is used to improve the model's output consistency under noisy conditions. A weighted label smoothing cross-entropy loss is calculated based on the clean input path; a KL divergence consistency loss between the probability distributions of the localization text tokens output by the teacher model and the student model is calculated based on the noisy perturbation input path, and the student model parameters are updated using the weighted sum of the two.

[0061] S31. After the HMT student model is initialized, the teacher model is created through deep copy:

[0062]

[0063] A gradient detach operation is performed on all parameters of the teacher model, freezing all parameters of the teacher model so that they do not participate in gradient updates during backpropagation. A deep copy refers to creating an independent copy in memory that is identical to the original model in terms of structure and parameter values. The two models are independent of each other, and subsequent updates to the student model's parameters will not be automatically passed to the teacher model.

[0064] S32. The student model performs standard forward propagation on the original clean underwater acoustic signal input:

[0065]

[0066] The weighted label smoothing cross-entropy loss is calculated using the following formula:

[0067]

[0068] Where N is the total number of tokens in the sequence. For the token weight function, For label smoothing (LabelSmoothing) cross-entropy loss, For the student model, output logits at position i. The target token ID is used. The label-based smooth cross-entropy loss is applied. The calculation formula is:

[0069]

[0070] in The smoothing coefficient is 0.1, and V is the vocabulary size, which is 36 in this example. For the student model, the target token The predicted probability is given by PAD, which is the Token ID corresponding to the padding character and has a value of 2. c iterates through all Tokens in the vocabulary (ID from 0 to V-1), but excludes the true target Token at the current position (the one the model should predict correctly) and the padding character Token (which is not included in the loss calculation).

[0071] When calculating the weighted label smoothed cross-entropy loss, different loss weights are set according to the field category and token type of the target localization text tokens. Specifically, default weights are applied to ordinary structure tokens, depth weights are applied to depth numerical tokens, and distance weights are applied to distance numerical tokens. The loss weight of distance numerical tokens is greater than that of depth numerical tokens, and both are greater than the loss weight of ordinary structure tokens. As a result, the model training process can pay more attention to numerical tokens that have a greater impact on the final localization error, especially key numerical tokens in the distance field.

[0072] In this embodiment of the invention, the depth token corresponds to vocabulary IDs 8~17, represented as d0~d9, and the depth weight... The value range is 1.0~3.0; the distance token corresponds to vocabulary ID 18~27, represented as r0~r9, and the distance weight... The value ranges from 1.0 to 5.0. A default weight of 1.0 is applied to other tokens.

[0073] The weighting function Defined as:

[0074]

[0075]

[0076]

[0077] S33. The specific implementation of the noise input path is as follows: Gaussian noise is added to the original clean input sample, and the calculation formula is:

[0078]

[0079] in For clean underwater acoustic signal input, For signal power, The signal-to-noise ratio used for consistency regularization ranges from 5 to 25 dB. This indicates that the mean is 0 and the variance is 0. The noise value is generated by random sampling from the Gaussian (normal) distribution.

[0080] In this embodiment, a fixed signal-to-noise ratio is added to the clean underwater acoustic signal input. The formula for calculating Gaussian noise in dB is consistent with the theoretical formula mentioned above. That is, the noise power is calculated based on the given signal-to-noise ratio, Gaussian noise is generated and superimposed on the original signal.

[0081] Forward propagation (gradient detach) of the teacher model on noisy input:

[0082]

[0083] Forward propagation of the student model on noisy input:

[0084]

[0085] Both the teacher and student models receive the following four parameters for forward propagation calculations:

[0086] (1) Noisy underwater acoustic signals : As input to the encoder, extract underwater acoustic localization features;

[0087] (2) Target localization text token sequence trg: as input to the decoder, used to guide the model to generate the corresponding text representation;

[0088] (3) Source mask src_mask: Its dimension corresponds to the input multi-channel underwater acoustic signal (including noise), and is used to shield the invalid features of the filling part in the underwater acoustic signal to ensure that the encoder only focuses on the valid acoustic features;

[0089] (4) Target end mask trg_mask: Its dimension corresponds to the target location text Token sequence. It is used to apply causal constraints in the self-attention calculation of the decoder to ensure that when the decoder predicts the i-th Token, the attention mechanism only acts on the i-th and previous Tokens and does not refer to subsequent Tokens, so as to avoid the leakage of future information and ensure the logical correctness of the autoregressive generation.

[0090] The KL divergence consistency loss is calculated using the following formula:

[0091]

[0092] Where N is the total number of tokens in the sequence, and V is the vocabulary size. The output of the teacher model, after undergoing gradient blocking operations, does not participate in gradient backpropagation. The output for the student model is indicated by the superscript (c), which represents the c-th token category in the vocabulary.

[0093] S34. The formula for calculating total loss is:

[0094]

[0095] in This is the consistency weighting coefficient, with a value ranging from 0.1 to 5.0.

[0096] Based on the calculated total loss The model parameters are backpropagated to calculate the gradient. The optimizer then updates the model parameters based on the calculated gradient and clears the gradient to zero after the update is complete, in order to prepare for the next round of iteration.

[0097] The teacher model is updated using an exponential moving average (EMA), with the following update formula:

[0098]

[0099] in This is the EMA attenuation factor, with a value ranging from 0.95 to 0.999. For teacher model parameters, These are the parameters for the student model.

[0100] This invention refers to the above training method as Consistent Regularization (CR) training. Through this training method, on the one hand, the student model learns the mapping relationship between underwater acoustic signals and localization text labels, and on the other hand, the model maintains a stable localization text generation distribution under noise perturbation conditions, thereby improving the localization robustness in low signal-to-noise ratio environments.

[0101] The specific training parameter configurations in this embodiment of the invention are as follows: batch_size=32, learning rate lr=5e-5, weight decay=1e-3, and label smoothing coefficient. The early stopping mechanism has a patience value of 15, the learning rate scheduler is ReduceLROnPlateau (mode=min, factor=0.5, patience=2, min_lr=1e-7), and the maximum sequence length is max_length=50. Consistency regularization parameters are also specified. EMA attenuation coefficient CR noise SNR = 10dB. Token weighting parameter: distance weight. Depth weights .

[0102] S4. During inference, the generated structured localization text sequence is parsed to obtain the depth and distance values ​​of the underwater sound source, thereby determining the localization result.

[0103] After model training is complete, the underwater sound source localization stage begins. Multi-channel underwater acoustic signals from the underwater sound source to be located are acquired and input into the trained underwater acoustic multimodal converter model. The encoder performs feature encoding on the underwater acoustic signals to obtain underwater acoustic localization features; the decoder starts from the start token (BOS) and gradually generates subsequent tokens based on the generated preceding tokens and underwater acoustic localization features until the end token (EOS) is generated, thus obtaining a structured localization text sequence. Subsequently, the structured localization text sequence is parsed, and the depth and distance tokens are extracted and converted into underwater sound source depth and distance values, respectively, thereby determining the underwater sound source localization result.

[0104] To verify the performance of the method proposed in this invention, comparative experiments were conducted. Figure 4 The graph shows the performance comparison curves of the proposed method and the baseline method on the test set at different signal-to-noise ratio (SNR) stages. (a) shows the depth prediction accuracy (ACC) of the two methods (≤2m), (b) shows the distance prediction accuracy of the two methods (≤100m), (c) shows the mean absolute error (MAE) of depth prediction of the two methods, and (d) shows the mean absolute error of distance prediction of the two methods. The blue curve HMT-CR-RW-DW (Proposed) represents the proposed method, where RW and DW indicate distance and depth token weighting operations. The red curve represents the baseline (HMT only) method. The ACC values ​​for each stage are shown in Table 2 below.

[0105] Table 2 Comparison of identification accuracy between this method and the baseline method

[0106]

[0107] Depend on Figure 4 As can be seen from Table 2 above, the method of the present invention outperforms the baseline method in both depth and distance prediction tasks during the varying SNR phase of the test (from clean with no noise, 20dB noise, and noise intensity gradually increasing to 5dB), especially in distance prediction, where it shows a significant improvement over the baseline method.

[0108] In addition, this invention conducted a series of ablation experiments to explore the individual and combined contributions of consistency regularization (CR) and numerical token weighting. RW indicates that only the following methods are used. DW indicates the use of The evaluation results are shown in Table 3 below:

[0109] Table 3 Ablation Experiment Evaluation Results

[0110]

[0111] As shown in Table 3, the independent contributions of consistency regularization (CR) training are as follows: Comparing HMT and HMT+CR: the latter maintains a depth ACC (≤2m) of 95.95% (a slight improvement from the baseline of 91.57%) while increasing the range ACC from 27.57% to 56.65%. This indicates that CR does indeed improve the model's robustness to noise perturbations without compromising depth prediction performance.

[0112] The independent contribution of distance-to-token (RW) weighting is as follows: Compared to HMT and HMT+RW, the latter significantly improves the range ACC from 27.57% to 80.05%, fully demonstrating the beneficial effect of token weighting. However, the cost is that the depth ACC decreases from 91.57% to 78.45%. This is because the loss weights are skewed towards the distance token, and the gradient distribution is concentrated in the distance direction, relatively weakening the optimization of the depth token.

[0113] Let's examine the synergistic effect of CR and RW. Comparing HMT+RW and HMT+CR+RW: the latter further improves the Range ACC from 80.05% to 95.60% based on RW, while the Depth ACC recovers from 78.45% to 84.30%. The consistency constraint of CR alleviates the mutual constraint between depth and distance caused by RW, enabling the model to achieve better generalization ability in both directions.

[0114] The final balance of Depth Token Weighted (DW): Comparing HMT+CR+RW and HMT+CR+RW+DW: The latter sets the depth weight to 1.5, while maintaining the Range ACC at 92.35% (only a slight decrease of 3.25 percentage points), and increasing the Depth ACC from 84.30% to 97.47%.

[0115] The results show that the HMT-CR-RW-DW method of the present invention surpasses the HMT baseline in both depth and distance directions—the depth ACC is improved from 91.57% to 97.47%, and the range ACC is significantly improved from 27.57% to 92.35% (an improvement of 64.78 percentage points), with the accuracy improved to 3.35 times that of the baseline, fully verifying the breakthrough effect of the present invention in distance prediction.

[0116] Based on the same technical concept as the method embodiments, the present invention also provides an underwater sound source localization system based on underwater acoustic-localization text generation alignment, comprising:

[0117] The training sample acquisition module is used to acquire a training sample set, which includes multi-channel underwater acoustic signal samples and their corresponding positioning text labels. The positioning text labels are structured text sequences that at least represent the depth and distance information of the underwater sound source. The positioning text labels are tokenized to obtain a target positioning text token sequence.

[0118] The model building module is used to build an underwater acoustic multimodal converter (HMT) model. The HMT model adopts an encoder-decoder structure. The encoder is used to encode the features of multi-channel underwater acoustic signals to obtain underwater acoustic localization features. The decoder is used to obtain the decoding text features based on the target localization text token sequence during the training phase and to obtain the decoding text features based on the generated preceding tokens during the inference phase. The decoding text features are then generatively aligned with the underwater acoustic localization features through a cross-attention mechanism to generate a localization text sequence containing depth and distance information in an autoregressive manner.

[0119] The model training module uses the HMT model as the student model and constructs an exponential moving average copy of it as the teacher model. For the same multi-channel underwater acoustic signal sample, a clean input path and a noisy perturbation input path are constructed. A weighted label-smooth cross-entropy loss is calculated based on the clean input path. A KL divergence consistency loss is calculated between the probability distributions of the location text tokens output by the teacher model and the student model based on the noisy perturbation input path. The student model parameters are updated using the weighted sum of the two. When calculating the weighted label-smooth cross-entropy loss, loss weights are set according to the field category and token type of the target location text token, ensuring that the weight of distance numerical tokens is greater than the weight of depth numerical tokens, and both are greater than the weight of ordinary structure tokens.

[0120] The localization result generation module is used to acquire multi-channel underwater acoustic signals of the underwater sound source to be located, input them into the trained HMT model to generate a structured localization text sequence, parse the depth and distance tokens in it, obtain the depth and distance values ​​of the underwater sound source, and determine the localization result of the underwater sound source.

[0121] The present invention also provides an electronic device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the underwater sound source localization method based on underwater acoustic-localization text generation alignment as described above.

[0122] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the underwater sound source localization method based on underwater acoustic-localization text generation alignment as described above.

[0123] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the underwater sound source localization method based on underwater acoustic-localization text generation alignment as described above.

Claims

1. A method for underwater sound source localization based on underwater acoustic-localization text generation alignment, characterized in that, Includes the following steps: A training sample set is obtained, which includes multi-channel underwater acoustic signal samples and their corresponding location text labels. The location text labels are structured text sequences that at least represent the depth and distance information of the underwater sound source. The location text labels are then tokenized to obtain a target location text token sequence. A multi-mode underwater acoustic converter (HMT) model is constructed, which adopts an encoder-decoder structure. The encoder is used to perform feature encoding on the multi-channel underwater acoustic signals to obtain underwater acoustic localization features. The decoder is used to obtain decoding text features based on the target localization text token sequence during the training phase, and to obtain decoding text features based on the generated preceding tokens during the inference phase. It also uses a cross-attention mechanism to generatively align the decoding text features with the underwater acoustic localization features, and generates a localization text sequence containing depth and distance information in an autoregressive manner. The HMT model is used as the student model, and its exponential moving average replica is constructed as the teacher model. Clean input paths and noisy perturbation input paths are constructed for the same multi-channel underwater acoustic signal sample. The weighted label smoothing cross-entropy loss is calculated based on the clean input path. The KL divergence consistency loss between the probability distributions of the location text tokens output by the teacher model and the student model is calculated based on the noisy perturbation input path, and the student model parameters are updated with the weighted sum of the two. When calculating the weighted label smoothing cross-entropy loss, the loss weights are set according to the field category and token type of the target location text token, so that the weight of the distance numerical token is greater than the weight of the depth numerical token, and both are greater than the weight of the ordinary structure token. The system acquires multi-channel underwater acoustic signals from the underwater sound source to be located, inputs them into the trained HMT model to generate a structured localization text sequence, parses the depth and distance tokens in the sequence to obtain the depth and distance values ​​of the underwater sound source, and determines the localization result of the underwater sound source.

2. The method according to claim 1, characterized in that, The encoder of the HMT model includes a convolutional encoder and a Transformer encoding layer. The convolutional encoder is used to downsample the multi-channel underwater acoustic signal and extract local acoustic features to obtain the underwater acoustic signal embedding sequence. The Transformer encoding layer is used to perform global feature modeling on the underwater acoustic signal embedding sequence to obtain the correlation features between different underwater acoustic channels and between different time segments.

3. The method according to claim 1, characterized in that, After the location text tag is tokenized, a target location text token sequence is obtained. The target location text token sequence includes a start character, an end character, a padding character, an unknown character, a structure token, a field identifier token, a depth value token, and a distance value token.

4. The method according to claim 3, characterized in that, The location text labels are structured text sequences in JSON format, with depth and distance information represented by numeric characters; In this process, numeric characters in the depth field are converted into depth value tokens with depth field identifiers, and numeric characters in the distance field are converted into distance value tokens with distance field identifiers, in order to distinguish the semantics of the same numeric characters in different physical quantities. The depth value tokens include d0 to d9, and the distance value tokens include r0 to r9. The depth field and the distance field are respectively filled to a fixed length so that the corresponding numerical positions in different positioning text labels are in the same token position.

5. The method according to claim 4, characterized in that, The weighted label smooth cross-entropy loss is obtained by weighting and summing the label smooth cross-entropy losses at each position in the output sequence; Where N is the total number of tokens in the sequence. For the token weight function, For label smoothing cross-entropy loss, For the student model, output logits at position i. The target Token ID; the label smooth cross-entropy loss. The calculation formula is: in V is the smoothing coefficient, and V is the vocabulary size. For the student model, the target token The predicted probability, PAD is the Token ID corresponding to the padding character, and c is the traversal sequence number of all Tokens with IDs from 0 to V-1 in the vocabulary; The token weighting function is constructed to apply a depth weight to depth-value tokens, a distance weight to distance-value tokens, and a default weight to ordinary structure tokens, satisfying the following condition: distance weight > depth weight > default weight.

6. The method according to claim 1, characterized in that, The noise disturbance input path obtains a noisy underwater acoustic signal by adding Gaussian noise to a clean underwater acoustic signal; The teacher model and the student model perform forward propagation on the noisy underwater acoustic signal to obtain the first localization text token probability distribution output by the teacher model and the second localization text token probability distribution output by the student model. The KL divergence consistency loss is calculated based on the probability distribution of the first location text token and the probability distribution of the second location text token, and the probability distribution of the first location text token output by the teacher model does not participate in gradient backpropagation.

7. An underwater sound source localization system based on underwater acoustic-localization text generation alignment, characterized in that, include: The training sample acquisition module is used to acquire a training sample set, which includes multi-channel underwater acoustic signal samples and their corresponding positioning text labels. The positioning text labels are structured text sequences that at least represent the depth and distance information of the underwater sound source. The positioning text labels are tokenized to obtain a target positioning text token sequence. The model building module is used to build an underwater acoustic multimodal converter (HMT) model, which adopts an encoder-decoder structure. The encoder is used to perform feature encoding on multi-channel underwater acoustic signals to obtain underwater acoustic localization features. The decoder is used to obtain decoding text features based on the target localization text token sequence during the training phase, and to obtain decoding text features based on the generated preceding tokens during the inference phase. It also uses a cross-attention mechanism to generatively align the decoding text features with the underwater acoustic localization features, and generates a localization text sequence containing depth and distance information in an autoregressive manner. The model training module uses the HMT model as the student model and constructs an exponential moving average copy of it as the teacher model. For the same multi-channel underwater acoustic signal sample, a clean input path and a noisy perturbation input path are constructed. A weighted label-smooth cross-entropy loss is calculated based on the clean input path. A KL divergence consistency loss is calculated between the probability distributions of the location text tokens output by the teacher model and the student model based on the noisy perturbation input path. The student model parameters are updated using the weighted sum of the two. When calculating the weighted label-smooth cross-entropy loss, loss weights are set according to the field category and token type of the target location text token, ensuring that the weight of distance numerical tokens is greater than the weight of depth numerical tokens, and both are greater than the weight of ordinary structure tokens. The localization result generation module is used to acquire multi-channel underwater acoustic signals of the underwater sound source to be located, input them into the trained HMT model to generate a structured localization text sequence, parse the depth and distance tokens in it, obtain the depth and distance values ​​of the underwater sound source, and determine the localization result of the underwater sound source.

8. An electronic device, characterized in that, include: One or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the underwater sound source localization method based on underwater acoustic-localization text generation alignment as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the underwater sound source localization method based on underwater acoustic-localization text generation alignment as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the underwater sound source localization method based on underwater acoustic-localization text generation alignment as described in any one of claims 1-6.

Images