Conditional generative adversarial network data enhancement method and system based on multi-feature constraint

Abstract

The invention discloses a conditional generative adversarial network data enhancement method and system based on multi-feature constraint, and the method comprises the following steps: S1, obtaining a target underwater acoustic signal, and constructing a multi-dimensional feature extraction and fusion network; s2, a multi-constraint-condition generative adversarial network is constructed to generate a simulation target underwater acoustic signal, the multi-constraint-condition generative adversarial network comprises a multi-loss constraint generator and a discriminator, and the target underwater acoustic signal is used as input to train the multi-constraint-condition generative adversarial network; and S3, performing underwater acoustic signal generation based on the trained multi-constraint-condition generative adversarial network.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of underwater target recognition, in particular to a conditional adversarial generation network data enhancement method and system based on multi-feature constraint. BACKGROUND

[0002] In underwater ship target recognition, the training of the noise classifier is a crucial link, and its recognition performance and generalization ability directly depend on the quality and diversity of the collected underwater ship noise data. However, the collection of underwater ship noise data faces many challenges, resulting in limited number of available samples and uneven distribution. In addition, China's maritime rights and interests have been challenged repeatedly, and the underwater confrontation situation is becoming increasingly prominent. The existing sample library in the actual confrontation environment is insufficient in coverage, and it is difficult to meet the underwater reconnaissance tasks in the actual confrontation environment. Therefore, it is of great significance to carry out research on the generation and recognition of hostile underwater target underwater acoustic signal samples in the field of underwater confrontation. Data generation methods can be divided into two categories: traditional data enhancement methods (such as noise addition, time domain transformation) and data-driven generation models (such as GAN, VAE).

[0003] The generative adversarial network (GAN) model contains two basic structures: generator G and discriminator D. In the training phase, G tries to make the data it generates consistent with the samples from the real training set, while D tries to distinguish whether the current sample is generated by the generator or taken from the real training set. This mutual confrontation and iterative optimization process can continuously improve the performance of both the generator G and the discriminator D, and eventually D will be unable to correctly distinguish the source of the data. At this time, it can be considered that the generator G has learned the distribution of the real data. The samples generated by the generator and the discriminator through mutual confrontation and iterative optimization generally achieve the effect of "faking the real", but there are uncontrollable and unpredictable shortcomings in generating data, and GAN is difficult to achieve fine-grained control of generated data.

[0004] The traditional GAN model adopts a min-max game training strategy, and the gradient updates of the discriminator and the generator are dynamically unbalanced, which easily leads to gradient disappearance or gradient explosion phenomenon, resulting in unstable model training and mode collapse problems.

[0005] The MelGAN model uses Mel spectrum-based features as input, gradually up-samples to the length of the speech, adds a convolutional block between the up-sampling to calculate the frequency domain to time domain transformation, and finally outputs the fixed frame number of speech. It takes the entire up-sampling process as a generator part, nests it in a GAN framework for training, and adjusts the discriminator and the objective function according to the unique properties of speech, making the training more stable and effective. It has fast generation speed, light model, and is suitable for real-time applications, but often appears over-smoothing phenomenon, high-frequency sub-band and narrow-band details are blurred, and it is difficult to deal with high-frequency detail preservation. Although HiFiGAN brings more sharp details, the noise floor is lifted and accompanied by granular artifacts and cross-domain instability. At the same time, the MelGAN and HiFiGAN models will cause error accumulation and domain mismatch through the method of "first converting to Mel spectrum and then restoring through vocoder". Moreover, the feature analysis of MelGAN and HiFiGAN models is limited to a single feature field, which limits the robustness and generalization ability of the classifier in complex underwater environments.

[0006] Therefore, the existing method still has problems in signal feature extraction accuracy, authenticity of the generated model, and effective fusion of multi-dimensional information. The traditional GAN model is unstable in training and has problems such as mode collapse. SUMMARY

[0007] To solve these problems, the present application proposes a multi-feature constraint-based conditional adversarial generative network data enhancement method and system, aiming to solve the problems of low signal feature extraction accuracy, low accuracy of the generated model, and effective fusion of multi-dimensional information of traditional GAN models and their improved models in an adversarial environment. A multi-dimensional feature extraction and fusion network and an effective feature fusion strategy are proposed. Aiming at the difficulty of simulating underwater target radiation noise signals and the problems of unstable training and mode collapse of traditional adversarial generative networks, a multi-constraint-based adversarial generative network is proposed.

[0008] According to an aspect of the present application, a multi-feature constraint-based conditional adversarial generative network data enhancement method is proposed, comprising the following steps:

[0009] S1, obtaining a target underwater acoustic signal, constructing a multi-dimensional feature extraction and fusion network, the multi-dimensional feature extraction and fusion network comprising a time-frequency feature extractor, a spectral feature extractor and a feature fusion device;

[0010] S2, constructing a multi-constraint condition generative adversarial network to generate a simulated target underwater acoustic signal, the multi-constraint condition generative adversarial network comprising a multi-loss constraint generator and a discriminator, the multi-constraint condition generative adversarial network being trained with the target underwater acoustic signal as input, wherein the multi-loss constraint generator is trained to minimize a multi-loss constraint function, the multi-loss constraint function comprising an auditory perception constraint function, a time-frequency combination constraint function, and an adversarial generative loss function, the multi-dimensional feature extraction and fusion network being used to calculate the time-frequency combination constraint function based on multi-dimensional features of the target underwater acoustic signal and the simulated target underwater acoustic signal respectively;

[0011] S3, generating an underwater acoustic signal based on the multi-constraint condition generative adversarial network after training.

[0012] Specifically, the time-frequency feature extractor is configured to extract time-frequency features of the target underwater acoustic signal with the target underwater acoustic signal as input, the spectral feature extractor is configured to extract spectral features of the target underwater acoustic signal with a mel spectrogram of the target underwater acoustic signal as input, and the feature fusioner is configured to receive the time-frequency features and the spectral features and perform feature fusion based on a cross-attention mechanism to obtain multi-dimensional features of the target underwater acoustic signal.

[0013] The time-frequency feature extractor and the spectral feature extractor are designed. The former realizes an end-to-end feature extraction process from waveform input to high-dimensional time-frequency feature representation, ensuring the integrity of phase information and time-domain structure, and the latter can effectively capture the harmonic structure and spectral energy distribution in the underwater acoustic signal. On this basis, the internal complementarity between different features is combined to build a complete information chain while avoiding redundancy risks as much as possible.

[0014] Specifically, the time-frequency feature extractor comprises a normalization layer, two cascaded residual convolution layers, a pooling layer, a two-dimensional convolution layer, a batch normalization layer, a channel attention layer, and a GLformer module.

[0015] The target underwater acoustic signal is subjected to amplitude normalization after being input into the normalization layer, and the output of the normalization layer is sequentially processed by the two cascaded residual convolution layers to extract time-domain and local frequency domain features, and then subjected to down-sampling by the pooling layer to obtain waveform features, and the output of the normalization layer is subjected to full-band fast Fourier transform processing to obtain spectral features.

[0016] After the spectral features and the waveform features are spliced, they are processed by the two-dimensional convolution layer for preliminary feature fusion, then subjected to batch normalization processing by the batch normalization layer, then subjected to attention weighting in the channel dimension by the channel attention layer, and finally subjected to processing by the GLformer module to obtain the time-frequency features.

[0017] The time-frequency feature extractor based on the Transformer architecture realizes an end-to-end feature extraction process from waveform input to high-dimensional time-frequency feature representation, ensuring the integrity of phase information and time-domain structure.

[0018] Specifically, the spectral feature extractor includes a batch normalization layer, three residual convolution layers connected in parallel and having different scales, a feature fusion and attention layer, a convolution layer with a 1*1 convolution kernel, a pooling layer, and a fully connected layer.

[0019] The Mel spectrogram of the target underwater acoustic signal is input into the batch normalization layer and is subjected to batch normalization processing, the output of the normalization layer is subjected to processing by the three residual convolution layers connected in parallel and having different scales to obtain three parallel features, the three parallel features are subjected to processing by the feature fusion and attention layer to splice and aggregate in the channel dimension, then are subjected to channel compression and nonlinear mapping by processing by the convolution layer with a 1*1 convolution kernel, are subjected to down-sampling by processing by the pooling layer, and finally are subjected to processing by the fully connected layer to obtain the spectral feature.

[0020] The spectral feature extractor can effectively capture the harmonic structure and spectral energy distribution in the underwater acoustic signal through the synergistic effect of multi-scale convolution and attention mechanism, and provide stable and discriminative frequency domain feature representation for downstream recognition and generation tasks.

[0021] Specifically, in S1, after the target underwater acoustic signal is acquired, noise is added to the target underwater acoustic signal before the target underwater acoustic signal is input into the multi-dimensional feature extraction and fusion network, and the added noise is one or more of Gaussian noise, uniform noise, impulse noise, salt and pepper noise, and multipath interference noise.

[0022] The underwater acoustic signal is significantly affected by multi-source interference (such as sensor thermal noise, environmental background noise, mechanical impact, and multipath reflection), resulting in complex noise components, strong time-varying nature, and heterogeneous distribution. The noise enhancement strategy aims to construct training samples with diverse and close-to-real sea noise interference distribution, so that the generator not only learns signal reconstruction, but also learns the ability to remove irrelevant interference and retain stable features when facing different noise disturbances.

[0023] Specifically, the multi-loss constraint function is defined as:

[0024] L G =λ1L mel +λ2L feat +λ 3adv

[0025] Wherein, L mel , L feat , L featrespectively represent the hearing perceptual constraint function, the time-frequency combined constraint function and the adversarial generation loss function, and λ1, λ2, λ3 represent corresponding weights respectively;

[0026] The hearing perceptual constraint function L mel is defined as L mel =||Mel(x gen )-Mel(x real )||, where Mel(x gen ) represents a mel spectrogram of the simulation target underwater acoustic signal generated by the multi-loss constraint generator, and Mel(x real ) represents a real mel spectrogram of the target underwater acoustic signal.

[0027] The time-frequency combined constraint function L feat is defined as f gen represents a multi-dimensional feature of the simulation target underwater acoustic signal generated by the multi-loss constraint generator, and f real represents a multi-dimensional feature of the target underwater acoustic signal.

[0028] The adversarial generation loss function L adv is defined as D(G(x)) represents a discrimination result of the discriminator on the simulation target underwater acoustic signal generated by the multi-loss constraint generator, and G(x) represents the simulation target underwater acoustic signal generated by the multi-loss constraint generator, represents an expectation of calculating a distribution sample of the multi-loss constraint generator.

[0029] After the corresponding features of the signal are extracted and fused, the multi-constraint condition adversarial generation neural network training is started, and multiple loss functions are jointly used in the training process to take into account the time domain reconstruction accuracy, spectral consistency and perceptual authenticity.

[0030] Specifically, the multi-constraint condition adversarial generation network is trained in an alternating optimization manner of the multi-loss constraint generator and the discriminator, where the loss function of the discriminator is defined as D(x) represents a discrimination result of the discriminator on the target underwater acoustic signal, represents an expectation of calculating a distribution sample of the target underwater acoustic signal.

[0031] The parameters of the multi-loss constraint generator and the discriminator are updated in the "turn alternation" manner to form a benign adversarial cycle. At the same time, unlike the traditional binary classification adversarial loss, the discriminator minimizes the squared difference distance between the generated sample and the real sample, so that the training process is smoother and the gradient is more stable.

[0032] Specifically, the multi-loss constraint generator is a coding-decoding network based on an improved one-dimensional U-Net network, multi-scale residual convolution modules and SE channel attention modules are added to the down-sampling structure of the encoding part and the up-sampling structure of the decoding part and are symmetrically arranged, the up-sampling structure of the decoding part is connected to the features output by the down-sampling structure of the encoding part at the same level through a skip connection, and the multi-scale residual convolution module is a plurality of convolution layers with different scale convolution kernels arranged in parallel;

[0033] The discriminator is a one-dimensional convolution network structure, which extracts the distribution features of the signal input into the discriminator through a plurality of convolution layers and a global average pooling layer and discriminates the classification of the signal input into the discriminator.

[0034] The multi-loss constraint generator realizes the joint modeling of local texture and global time sequence features through multi-scale convolution encoding and symmetric decoding of underwater acoustic waveforms, captures the multi-time scale features of the target underwater acoustic signal through a plurality of convolution layers with different scale convolution kernels, and realizes the selective enhancement of key band features through the SE channel attention module to learn the importance weight of different channels.

[0035] According to an aspect of the present application, a multi-feature constraint-based conditional adversarial generative network data enhancement system is provided, according to the method of any one of the first aspect, comprising the following modules:

[0036] The multi-dimensional feature extraction and fusion network construction module is configured to obtain a target underwater acoustic signal, construct a multi-dimensional feature extraction and fusion network, and the multi-dimensional feature extraction and fusion network comprises a time-frequency feature extractor, a spectral feature extractor and a feature fusioner;

[0037] The conditional adversarial generative network training module is configured to construct a multi-constraint conditional adversarial generative network to generate a simulated target underwater acoustic signal, the multi-constraint conditional adversarial generative network comprises a multi-loss constraint generator and a discriminator, and the multi-constraint conditional adversarial generative network is trained with the target underwater acoustic signal as input, wherein the multi-loss constraint generator is trained to minimize a multi-loss constraint function, the multi-loss constraint function comprises an auditory perception constraint function, a time-frequency combination constraint function and an adversarial generative loss function, and the time-frequency combination constraint function is calculated based on the multi-dimensional feature extraction and fusion network to obtain the multi-dimensional features of the target underwater acoustic signal and the simulated target underwater acoustic signal;

[0038] The underwater acoustic signal generation module is configured to generate underwater acoustic signals based on the multi-constraint conditional adversarial generative network after training.

[0039] According to an aspect of the present application, a computer program product is provided, which stores a computer program, and the computer program implements the method of the first aspect when executed by a processor.

[0040] The present application has the advantages of:

[0041] (1) Multi-feature extractor design and feature fusion strategy: Break through the limitation of the frequency domain, break through the limitation of traditional single feature analysis, and propose a multi-dimensional feature modeling method integrating time domain, frequency domain and spatial information. At the same time, a series of comparative experiments are carried out combined with task requirements to select the most appropriate feature fusion strategy. The problems of low feature extraction accuracy, low model accuracy and effective fusion of multi-dimensional information are solved, and the classification and recognition ability of underwater target radiation noise is effectively improved.

[0042] (2) Multi-constraint condition generative adversarial neural network: End-to-end optimization is carried out in the form of multi-loss fusion, realizing the generation process from the original waveform to the high-fidelity reconstructed signal, solving the problems of unstable training and mode collapse of traditional generative network model, and realizing high-quality and faithful generation and enhancement of underwater target radiation noise waveform. BRIEF DESCRIPTION OF DRAWINGS

[0043] The accompanying drawings are included to provide a further understanding of embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and serve the purpose of explaining principles of the application. Other embodiments and many of the intended advantages of the embodiments will be readily appreciated as the same becomes better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale relative to each other. Like reference numerals designate corresponding similar parts.

[0044] Figure 1 A flowchart of a multi-feature constraint conditional adversarial generative network data enhancement method according to the present application is shown;

[0045] Figure 2 A simplified architecture diagram of a multi-dimensional feature extraction and fusion network and a multi-feature constraint conditional adversarial generative network according to the present application is shown;

[0046] Figure 3 A schematic diagram of the overall architecture of a time-frequency feature extractor according to the present application is shown;

[0047] Figure 4 A schematic diagram of the overall architecture of a spectral feature extractor according to the present application is shown;

[0048] Figure 5 A classification accuracy diagram using single features is shown;

[0049] Figure 6 A classification accuracy diagram based on multi-feature fusion is shown;

[0050] Figure 7(a) shows a schematic diagram of the overall architecture of a conditional adversarial generative network with multi-feature constraints according to the present invention;

[0051] Figure 7(b) shows a schematic diagram of the model structure of the multi-loss constraint generator according to the present invention;

[0052] Figure 7(c) shows a schematic diagram of the network structure of the downsampling structure of the multi-loss constraint generator according to the present invention;

[0053] Figure 7(d) shows a schematic diagram of the network structure of the upsampling structure of the multi-loss constraint generator according to the present invention;

[0054] Figure 8(a) shows the original sample of the target underwater acoustic signal, the target underwater acoustic signal under the condition of added noise input, and the simulated target underwater acoustic signal generated by the target underwater acoustic signal under the condition of added noise input in the Mel spectrum and STFT, respectively.

[0055] Figure 8(b) shows the original sample of the target underwater acoustic signal, the target underwater acoustic signal under the condition of added noise input, and the simulation target underwater acoustic signal generated by the target underwater acoustic signal under the condition of added noise input in the LOFAR spectrum and CQT spectrum, respectively.

[0056] Figure 9(a) shows the target underwater acoustic signal, and the simulated target underwater acoustic signal generated by the MelGAN algorithm, the HiFiGAN algorithm and the method proposed in this invention, respectively, in the Mel spectrum and STFT.

[0057] Figure 9(b) shows the target underwater acoustic signal, and the visualizations of the simulated target underwater acoustic signal generated by the MelGAN algorithm, the HiFiGAN algorithm and the method proposed in this invention in the LOFAR spectrum and CQT spectrum, respectively.

[0058] Figure 10(a) shows a comparison of the original amplitude spectrum and the generated amplitude spectrum corresponding to the method proposed in this invention;

[0059] Figure 10(b) shows a comparison of the original amplitude spectrum and the generated amplitude spectrum corresponding to the MelGAN algorithm;

[0060] Figure 10(c) shows a comparison of the original amplitude spectrum and the generated amplitude spectrum corresponding to the HiFiGAN algorithm;

[0061] Figure 11(a) shows the original sample of the target underwater acoustic signal, the target underwater acoustic signal under noisy input, and the visualization of the simulated target underwater acoustic signal generated by the method proposed in this invention in Mel spectrum and STFT, respectively.

[0062] Figure 11(b) shows the original sample of the target underwater acoustic signal, the target underwater acoustic signal under noisy input, and the simulated target underwater acoustic signal generated by the method proposed in this invention in the LOFAR spectrum and CQT spectrum, respectively.

[0063] Figure 12(a) shows the feature distribution of the Deepship dataset after sample generation and augmentation using the method proposed in this invention;

[0064] Figure 12(b) shows the feature distribution of the Shipsear dataset after sample generation and augmentation using the method proposed in this invention;

[0065] Figure 13(a) shows the performance difference and performance improvement of the Deepship dataset before and after sample generation and expansion using the method proposed in this invention;

[0066] Figure 13(b) shows the performance difference and performance improvement of the Shipsear dataset before and after sample generation and expansion using the method proposed in this invention;

[0067] Figure 14 A schematic diagram of the structure of a conditional adversarial generative network data augmentation system based on multi-feature constraints according to the present invention is shown.

[0068] Figure 15 A schematic diagram of a computer system architecture suitable for implementing the embodiments of this application is shown. Detailed Implementation

[0069] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0070] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0071] Figure 1 This paper presents a proposed data augmentation method for conditional adversarial generative networks based on multi-feature constraints, comprising the following steps:

[0072] S1, acquire the target underwater acoustic signal, and construct a multi-dimensional feature extraction and fusion network, which includes a time-frequency feature extractor, a spectral feature extractor, and a feature fusion network;

[0073] S2, Construct a multi-constraint adversarial generative network to generate a simulated target underwater acoustic signal. The multi-constraint adversarial generative network includes a multi-loss constraint generator and a discriminator. The multi-constraint adversarial generative network is trained with the target underwater acoustic signal as input. The multi-loss constraint generator is trained with the goal of minimizing the multi-loss constraint function. The multi-loss constraint function includes an auditory perception constraint function, a time-frequency combined constraint function, and an adversarial generative loss function. The time-frequency combined constraint function is calculated based on the multi-dimensional feature extraction and fusion network to obtain the multi-dimensional features of the target underwater acoustic signal and the simulated target underwater acoustic signal.

[0074] S3, underwater acoustic signal generation is performed based on the multi-constraint adversarial generative network after training is completed.

[0075] First, such as Figure 2 As shown, in the first stage, a temporal feature extractor and a spectral feature extractor are proposed, which together form a multi-feature extractor, realizing a multi-dimensional feature modeling method integrating temporal, frequency, and spatial information. After acquiring multi-dimensional features, the second stage proceeds by constructing various constraint mechanisms to optimize the training process of the generative adversarial network. Finally, the proposed conditional generative adversarial network is used to generate the target underwater acoustic signal.

[0076] Specifically, the time-frequency feature extractor takes the target underwater acoustic signal as input to extract the time-frequency features of the target underwater acoustic signal, the spectral feature extractor takes the Mel spectrum of the target underwater acoustic signal as input to extract the spectral features of the target underwater acoustic signal, and the feature fusion unit receives the time-frequency features and the spectral features and performs feature fusion based on the cross-attention mechanism to obtain the multidimensional features of the target underwater acoustic signal.

[0077] like Figure 3 As shown, a novel signal feature extraction method—the time-frequency feature extractor—is proposed based on the Transformer architecture. This feature extractor takes the waveform input of the target underwater acoustic signal as its starting point, and its overall structure includes: a normalization layer, two cascaded residual convolutional layers, a pooling layer, a two-dimensional convolutional layer, a batch normalization layer, a channel attention layer, and a GLformer module.

[0078] In a specific embodiment, after receiving raw waveform data of length N = 60000, the normalization layer standardizes the amplitude of the input waveform to reduce dynamic differences between different samples. Residual convolutional layers 1 and 2 employ a two-stage cascaded residual structure to extract temporal and local frequency domain features, enhancing the nonlinear expressive power and stability of the features. The kernel size of both residual convolutional layers 1 and 2 is 3. Then, a pooling layer performs downsampling to compress redundant information and highlight the main energy distribution, outputting waveform features with a dimension of N = 1 × 30000, where the pooling window of the pooling layer is 2 and the stride is 2.

[0079] The output of the normalization layer is processed by a full-band Fast Fourier Transform, resulting in a spectral feature with a dimension of N = 1 × 30000. This spectral feature and the waveform feature are concatenated and then processed by a two-dimensional convolutional layer with kernel size (2, 3) and a batch normalization layer. Local convolution and normalization operations are performed on the time-frequency plane to provide a smoother feature distribution for the subsequent attention mechanism. Then, a channel attention layer adaptively adjusts the importance of different channels, strengthening the response in key time-frequency regions, outputting waveform and spectral features with a dimension of N = 1 × 30000. A lightweight Transformer-structured GLformer module is then introduced to balance local details and global dependencies, improving the temporal modeling capability of the features. Finally, a time-frequency feature with a dimension of 512 is obtained, providing input for the subsequent multimodal fusion and generation module.

[0080] The structure of the spectral feature extractor is as follows: Figure 4 As shown, it uses the Mel spectrum as input and focuses on extracting auditory-related features such as frequency distribution, harmonic structure, and energy envelope. This spectral feature extractor, through the synergistic effect of multi-scale convolution and attention mechanisms, can effectively capture the harmonic structure and spectral energy distribution in underwater acoustic signals, providing a stable and discriminative frequency domain feature representation for downstream recognition and generation tasks. The spectral feature extractor network structure mainly consists of the following parts: batch normalization layer, four cascaded modules (each module consists of three parallel-connected residual convolutional layers of different scales, a feature fusion and attention layer, and a convolutional layer with a 1×1 kernel), pooling layer, and fully connected layer.

[0081] The Mel spectrum of the target underwater acoustic signal is input into the batch normalization layer for batch normalization processing to reduce the dynamic range differences between different frequency bands and stabilize the training process. The spectral feature extractor includes: a batch normalization layer, four cascaded modules, a pooling layer, and a fully connected layer. Each cascaded module consists of three parallel residual convolutional layers of different scales, a feature fusion and attention layer, and a convolutional layer with a 1×1 kernel.

[0082] The Mel-spectrum map of the target underwater acoustic signal is input into the batch normalization layer for batch normalization. The output of the normalization layer is processed sequentially through the four cascaded modules. The input of each cascaded module is processed by three parallel residual convolutional layers of different scales to obtain three parallel features. The kernel sizes of the three parallel residual convolutional layers of different scales are k=3, 5, and 7 from top to bottom, which are used to extract spectral patterns under different receptive fields to capture local and global frequency domain features. The three parallel features are concatenated and weighted in the channel dimension through feature fusion and attention layer processing to enhance the model's response to key frequency bands. Then, channel compression and nonlinear mapping are performed through the 1×1 convolutional layer. The output of the last cascaded module is then downsampled through the pooling layer to extract high-level semantic features. Finally, the fused spectral features are mapped into a 512-dimensional feature vector through a fully connected layer, maintaining the same dimensionality as the output of the time-frequency feature extractor for subsequent feature fusion and joint modeling.

[0083] A cross-attention mechanism is used to fuse the features output by the time-frequency feature extractor and the spectral feature extractor to form a multi-dimensional feature signal. Its working principle is to achieve deep interaction between the two feature sequences through a bidirectional cross-attention mechanism.

[0084] First, the key-value pairs of feature1 (the feature output by the time-frequency feature extractor) are queried using feature1 (the feature output by the spectral feature extractor). Then, feature2 is queried using feature1. The results from both directions are concatenated and fused after residual connection and layer normalization, ultimately outputting a fused feature that enhances bidirectional semantic information. Its effectiveness is based on the inherent complementarity between different features.

[0085] like Figure 5 and Figure 6 As shown, this invention compares the classification results of multiple feature fusion methods. Experimental data comparison reveals that the classification accuracy obtained through feature fusion is nearly 10% higher than that of the original time-domain waveform, fully demonstrating the importance of multi-angle identification of target radiated noise signals. Different fusion methods yield varying accuracy rates; ultimately, this invention selects "waveform + spectrum + Mel" as the feature extraction and fusion scheme.

[0086] After extracting and fusing the corresponding features of the signal, the training of the multi-constraint conditional adversarial generative network begins. During the training process, multiple loss functions are used in combination to balance temporal reconstruction accuracy, spectral consistency and perceptual realism. The overall architecture of the conditional adversarial generative network based on multi-feature constraints is shown in Figure 7(a).

[0087] As shown in Figure 7(b), the multi-loss constraint generator adopts an improved one-dimensional U-Net structure. Through multi-scale convolutional encoding and symmetric decoding of underwater acoustic waveforms, it achieves joint modeling of local texture and global temporal features. The multi-loss constraint generator includes a one-dimensional convolutional layer (Conv 1D), a batch normalization layer (BatchNorm), an activation function layer (ReLU), six downsampling structures (Down Mixer) and upsampling structures (Up Mixer) connected by a bottleneck layer (BottleNeck), a one-dimensional convolutional layer with a 3×1 kernel, an activation function layer (ReLU), a one-dimensional convolutional layer (Conv 1D), an activation function layer (Tanh), and an interpolate interpolation function.

[0088] As shown in Figure 7(c), in the encoding stage, a downsampling structure is adopted. The multi-loss constraint generator uses convolutional kernels of different scales in parallel to capture the multi-timescale features of the target underwater acoustic signal. Then, the features are fused through a concat layer, and then local correlations in the channel / sequence dimension are captured through a one-dimensional convolutional layer, mapping the features to a more suitable dimensional space. After normalization through a batch normalization layer, the features are then transformed nonlinearly by introducing a ReLU activation function. Simultaneously, to further enhance the feature representation capability, a Squeeze-and-Excitation (SE) channel attention module is introduced in each encoding layer. This module selectively enhances key frequency band features by adaptively learning the importance weights of different channels. Finally, average pooling is used to reduce the spatial dimension of the features before output.

[0089] As shown in Figure 7(d), the decoding stage employs an upsampling structure symmetrical to the encoding stage. Upsampling is performed through a transposed convolutional layer (ConvTranspose) to gradually restore temporal resolution. Skip connections are made between the features at the same level as those in the encoding stage at the Concat layer to preserve fine-grained temporal information. Next, a one-dimensional convolutional layer extracts local features in the channel / sequence dimension of the concatenated features, capturing feature correlations within this dimension. Simultaneously, the feature dimension is adjusted to better suit the space of subsequent computations. Then, batch normalization is performed, followed by a ReLU activation function for non-linear transformation. Next, multi-scale feature extraction is performed through three parallel convolutional layers with kernels of different scales. The multi-scale concatenated features are then subjected to channel / sequence dimension feature extraction and dimension adjustment again through a Concat layer. This process of one-dimensional convolutional layer, batch normalization, and ReLU activation is repeated. Finally, SE Attention assigns differentiated importance weights to features in different channels for output.

[0090] The final output layer of the multi-loss constraint generator uses the Tanh activation function to constrain the waveform amplitude to [-1,1] to achieve waveform reconstruction output. Then, the output simulated underwater acoustic target signal is corrected to the same sample points as the underwater acoustic target signal through the interpolate function.

[0091] The multi-loss constraint function is defined as follows:

[0092] L G =λ1L mel +λ2L feat +λ3L adv

[0093] Among them, L mel L feat L feat Let λ1, λ2, and λ3 represent the auditory perception constraint function, the time-frequency combination constraint function, and the adversarial generation loss function, respectively, and let λ1, λ2, and λ3 represent the corresponding weights.

[0094] During training, a warm-up dynamic weighting strategy is introduced to gradually increase the weight of the adversarial term in the initial stage to prevent instability in the early training process. The optimizer chosen is AdamW, which decouples the weight decay term from the first-order moment estimation, making weight decay independent of gradient updates, thus enabling more accurate control of the model's regularization strength. The learning rate scheduling employs the ReduceLROnPlateau strategy, which automatically decays the learning rate proportionally when the validation set loss does not show a significant decrease within several rounds. Unlike common fixed-periodic scheduling methods such as StepLR or Cosine Annealing, ReduceLROnPlateau adaptively adjusts the learning rate based on the model's actual convergence state. For example, it reduces the learning rate when the model is close to the optimum to promote fine-tuning, and maintains a larger learning rate when the model is still in a rapid decline phase, thereby accelerating overall convergence. This strategy effectively prevents oscillations and premature convergence in the later stages of training. To prevent overfitting, the model employs an early stopping mechanism, and the minimum comprehensive generative loss on the validation set is used as the model selection criterion.

[0095] The auditory perception constraint function, the time-frequency combined constraint function, and the adversarial generation loss function are defined based on the spectral distance L, the eigenvector distance C, and the adversarial generation loss D(G(x)) of the original target underwater acoustic signal and the generated simulated target underwater acoustic signal, respectively.

[0096] Wherein, the auditory perception constraint function L mel Defined as L mel =||Mel(x) gen )-Mel(x real )||,Mel(x genMel(x) represents the Mel spectrum of the simulated target underwater acoustic signal generated by the multi-loss constraint generator. real The ) represents the true Mel spectrum of the target underwater acoustic signal, which is used to maintain the similarity of the generated signal and the input signal in spectral structure by using L1 loss of the Mel spectrum.

[0097] The feature extraction module of the multi-loss constraint generator employs a pre-trained multi-dimensional feature extraction and fusion network. During the training process of the multi-loss constraint generator, the frozen multi-dimensional feature extraction and fusion network does not participate in gradient updates; it is only used to calculate the cosine similarity loss between the generated signal and the real signal in the feature space. This feature consistency constraint helps maintain the semantic alignment of the generated waveform in terms of spectral structure and energy distribution, compensating for the shortcomings of using only time-domain or frequency-domain reconstruction losses. The time-frequency combined constraint function L... feat Defined as f gen f represents the multidimensional features of the simulated target underwater acoustic signal generated by the multi-loss constraint generator. real This represents the multidimensional characteristics of the target underwater acoustic signal.

[0098] The adversarial generation loss function L adv Defined as D(G(x)) represents the discrimination result of the discriminator on the simulated target underwater acoustic signal generated by the multi-loss constraint generator, and G(x) represents the simulated target underwater acoustic signal generated by the multi-loss constraint generator. This represents the expectation of the distributed sampling calculated by the multi-loss constrained generator. Adversarial loss is used to improve the realism of the generated signal.

[0099] Specifically, the multi-constraint adversarial generative network is trained by alternately optimizing the multi-loss constraint generator and the discriminator, wherein the loss function of the discriminator is defined as... D(x) represents the discrimination result of the discriminator on the target underwater acoustic signal. This represents the expected value of the target underwater acoustic signal distribution sampling.

[0100] The conditional adversarial generative network data augmentation method based on multi-feature constraints overcomes the limitations of the spectral domain and breaks through the limitations of traditional single-feature analysis. It proposes a multi-dimensional feature modeling method integrating time-domain, frequency-domain, and spatial information, improving the classification and identification capabilities of underwater target radiated noise. This research not only enhances the multi-angle identification of target radiated noise signals but also provides new technical support for target detection in complex environments in practical applications. Finally, this invention combines multiple constraint mechanisms to optimize the training process of the generative network, shifting from generating the Mel spectrum to generating the waveform, overcoming the limitations of current generation methods, improving the realism and diversity of radiated noise signal generation, and providing a more accurate radiated noise generation model.

[0101] Underwater acoustic signals are significantly affected by multi-source interference (such as sensor thermal noise, ambient background noise, mechanical shock, and multipath reflections), resulting in complex, time-varying, and heterogeneous noise components. Single noise models often fail to accurately characterize these non-stationary, sparse, and abrupt noise features. The noise enhancement strategy aims to construct diverse training samples that closely approximate the distribution of real-world underwater acoustic interference, enabling the generator not only to learn signal reconstruction but also to learn to remove irrelevant interference and retain stable features when faced with different noise disturbances. Essentially, it is a feature robustness regularization that effectively mitigates performance degradation in low signal-to-noise ratio (SNR) scenarios. Compared to generators trained solely on clean samples, noisy training significantly improves the model's stability and spectral fidelity in low SNR environments. To simulate various interference forms that may occur in real-world underwater acoustic signals, five noise models were designed: Gaussian noise, uniform noise, impulse noise, salt-and-pepper noise, and multipath interference noise. Through this mechanism, the model's generation results under high-noise conditions are smoother and the spectral shape is more stable, achieving a similar effect to feature purification.

[0102] In S1, after acquiring the target underwater acoustic signal, noise is added to the target underwater acoustic signal before it is input into the multidimensional feature extraction and fusion network. The added noise is one or more of Gaussian noise, uniform noise, impulse noise, salt-and-pepper noise, and multipath interference noise.

[0103] Given a clean original waveform signal x, adding several types of noise, such as k types of noise, can be represented as follows:

[0104]

[0105] Where, n i Let α represent the i-th type of noise. i It is its amplitude scaling factor.

[0106] The function and specific form of the added noise are shown below:

[0107] (1) Gaussian noise n g

[0108] Used to simulate sensor thermal noise and random background disturbances, defined as:

[0109] n g (t)~N(0,1)

[0110]

[0111] x represents the original clean waveform signal, α g This is its amplitude scaling factor. The parameters in the following formulas follow the same pattern.

[0112] (2) Uniform noise n u

[0113] Used to simulate random background electromagnetic interference, with values ​​following a uniform distribution:

[0114] n u (t)~U(-1,1)

[0115]

[0116] (3) Impulse noise n p

[0117] For each sampling point, a pulse is triggered once with probability p. After triggering, the point value takes a random high amplitude value to simulate sudden impact disturbances (such as propeller blade impacts), randomly generating sparse spikes in the time domain.

[0118]

[0119] Where p = 0.005 represents the probability of pulse occurrence, and this parameter controls the sparsity of the pulse noise on the time axis. An excessively large value will make the noise no longer sparse, affecting the semantics of the model training signal.

[0120] (4) Salt and pepper noise n sp

[0121] Certain sampling points are set to +1 or -1 with probability p to simulate sudden extreme values ​​(sudden saturation or breakpoint phenomena) caused by ADC saturation, sensor transient errors, or zero-point drift. This is defined as:

[0122]

[0123] Where p = 0.005 controls the noise sparsity.

[0124] (5) Multipath interference noise n m

[0125] Delayed and attenuated signal copies are superimposed. Given the delay and attenuation, this is used to simulate the multipath reflection effect of signal propagation in seawater. The delay superposition form is as follows:

[0126] n m (t)=α m x(t-τ)

[0127] Where τ is the number of random delayed sample points (range [50, 400]), α m ∈[0.2,0.3] is the attenuation coefficient, which can simulate the delayed echo and superposition caused by multipath reflection in seawater. This term has a significant effect at low frequencies.

[0128] Figure 8(a) shows the visualizations of the original target underwater acoustic signal, the target underwater acoustic signal with added noise, and the simulated target underwater acoustic signal generated by the target underwater acoustic signal with added noise in the Mel spectrum and STFT (Short Time Fourier Transform), respectively. Figure 8(b) shows the visualizations of the original target underwater acoustic signal, the target underwater acoustic signal with added noise, and the simulated target underwater acoustic signal generated by the target underwater acoustic signal with added noise in the LOFAR spectrum and CQT spectrum, respectively. In the legends of Figures 8(a) and 8(b), "original" represents the original sample, i.e., the target underwater acoustic signal. "noisy" (Augment) represents the target underwater acoustic signal with added noise, and "generated" (Fake) represents the simulated target underwater acoustic signal generated by the target underwater acoustic signal with added noise. The experimental results show that this strategy significantly improves the convergence stability and reconstruction fidelity of the generation system in low signal-to-noise ratio environments. This result indicates that the current generation network has good steady-state statistical modeling and cross-domain semantic consistency capabilities, and can effectively improve the discriminability of time-frequency representation without compromising the main semantic information.

[0129] Figure 9(a) shows the target underwater acoustic signal, and visualizations of the simulated target underwater acoustic signal generated by the MelGAN algorithm, HiFiGAN algorithm, and the method proposed in this invention in the Mel spectrum and STFT, respectively. Figure 9(b) shows the target underwater acoustic signal, and visualizations of the simulated target underwater acoustic signal generated by the MelGAN algorithm, HiFiGAN algorithm, and the method proposed in this invention in the LOFAR spectrum and CQT, respectively. As shown in Figures 9(a) and 9(b), compared with the classic speech generation methods MelGAN and HiFiGAN, the spectrum generated by this method is more readable and separable, and shows a more significant advantage in the structural fidelity and detail representation of the Mel spectrum.

[0130] Figures 10(a), 10(b), and 10(c) show a comparison of the original and generated amplitude spectra of the three algorithms. The comparison results show that the generation algorithm proposed in this invention is most consistent with the overall trend of the original spectrum in the 0-3.8kHz range. In the transition band near the upper sampling limit (3.6-4.0kHz) at the high-frequency end, the original signal is significantly attenuated due to front-end anti-aliasing and system roll-off. The underfitting of this roll-off by this algorithm is significantly improved compared to MelGAN and HiFiGAN, exhibiting excessive high-frequency energy retention. In summary, the generation algorithm proposed in this invention achieves a better balance between low bias, low artifacts, and detail fidelity, effectively suppressing noise while maintaining discriminative structures such as line spectra and dips, making it more favorable for subsequent type identification and parameter inversion.

[0131] Meanwhile, the generative model proposed in this invention has a certain denoising effect and also realizes the generation process from the original waveform to a high-fidelity reconstructed signal. As shown in Figures 11(a) and 11(b), regardless of whether the original sample or the noisy sample is used as input, the current model can effectively restore the main frequency band profile and temporal coherence of underwater acoustic radiation noise, and significantly suppress broadband noise, so that the overall spectral shape returns to the original distribution. The generated data exhibits high realism and effectiveness in the feature space, and is not isolated or deviated, but seamlessly integrated into the feature manifold of the real data.

[0132] Figures 12(a) and 12(b) show the feature distributions of the Deepship and Shipsear datasets after sample generation augmentation. This high degree of distribution consistency indicates that the generated samples introduced through data augmentation techniques have a very high similarity to the real samples at the feature level. This effectively expands the training set and provides high-quality and consistently distributed diverse data for subsequent model training, which is expected to significantly improve the model's generalization ability and robustness.

[0133] As shown in Figures 13(a) and 13(b), with the sample size increased by 100%, the overall performance of various features and their fusion schemes in the DeepShip dataset is significantly improved, and the trend of multi-domain fusion outperforming single-domain features is more prominent. The classification accuracy of various features and their combinations in the ShipsEar dataset also shows varying degrees of improvement.

[0134] According to one aspect of the present invention, a conditional adversarial generative network data augmentation system based on multi-feature constraints is proposed, comprising the following modules according to any one of the first aspects:

[0135] The multidimensional feature extraction and fusion network construction module 141 is configured to acquire the target underwater acoustic signal and construct a multidimensional feature extraction and fusion network, wherein the multidimensional feature extraction and fusion network includes a time-frequency feature extractor, a spectral feature extractor and a feature fusion unit;

[0136] The conditional adversarial generative network training module 142 is configured to construct a multi-constraint conditional adversarial generative network to generate a simulated target underwater acoustic signal. The multi-constraint conditional adversarial generative network includes a multi-loss constraint generator and a discriminator. The multi-constraint conditional adversarial generative network is trained with the target underwater acoustic signal as input. The multi-loss constraint generator is trained with the goal of minimizing the multi-loss constraint function. The multi-loss constraint function includes an auditory perception constraint function, a time-frequency combined constraint function, and an adversarial generative loss function. The time-frequency combined constraint function is calculated based on the multi-dimensional features of the target underwater acoustic signal and the simulated target underwater acoustic signal obtained by the multi-dimensional feature extraction and fusion network.

[0137] The underwater acoustic signal generation module 143 is configured to generate underwater acoustic signals based on the multi-constraint adversarial generative network after it has been trained.

[0138] The following is for reference. Figure 15 It shows a schematic diagram of the structure of a computer system 1500 suitable for implementing electronic devices according to embodiments of the present application. Figure 15 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0139] like Figure 15 As shown, the computer system 1500 includes a central processing unit (CPU) 1501, which performs various appropriate actions and processes based on programs stored in read-only memory (ROM) 1502 or programs loaded from storage portion 1509 into random access memory (RAM) 1504. The RAM 1504 also stores various programs and data required for the operation of the system 1500. The CPU 1501, ROM 1502, ROM 1503, and RAM 1504 are interconnected via a bus 1505. An input / output (I / O) interface 1506 is also connected to the bus 1505.

[0140] The following components are connected to I / O interface 1506: an input section 1507 including a keyboard, mouse, etc.; an output section 1508 including a liquid crystal display (LCD) and speakers, etc.; a storage section 1509 including a hard disk, etc.; and a communication section 1510 including a network interface card such as a LAN card and a modem, etc. The communication section 1510 performs communication processing via a network such as the Internet. A drive 1511 is also connected to I / O interface 1506 as needed. A removable medium 1512, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 1511 as needed so that computer programs read from it can be installed into storage section 1509 as needed.

[0141] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts are implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program is downloaded and installed from a network via communication section 1510, and / or installed from removable medium 1512. When the computer program is executed by central processing unit (CPU) 1501, it performs the functions defined in the methods of this application.

[0142] It should be noted that the computer-readable storage medium of this application is a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium is, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium is any tangible medium containing or storing a program that is used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium includes a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals take various forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium or any computer-readable storage medium other than a computer-readable storage medium may transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0143] Computer program code for performing the operations of this application is written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code executes entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer is connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or connected to an external computer (e.g., via the Internet using an Internet service provider).

[0144] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram represents a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually execute substantially in parallel, and they may sometimes execute in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, is implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0145] The modules described in the embodiments of this application are implemented in software or hardware.

[0146] In another respect, this application also provides a computer-readable storage medium that is included in the electronic device described in the above embodiments; or that exists independently and is not assembled into the electronic device. The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by the electronic device, the electronic device causes the following to occur: S1, acquire a target underwater acoustic signal, and construct a multi-dimensional feature extraction and fusion network, wherein the multi-dimensional feature extraction and fusion network includes a time-frequency feature extractor, a spectral feature extractor, and a feature fusion unit; S2, construct a multi-constraint adversarial generative network to generate a simulated target underwater acoustic signal, wherein the multi-constraint adversarial generative network includes a multi-loss constraint generator and a discriminator, and trains the multi-constraint adversarial generative network with the target underwater acoustic signal as input, wherein the multi-loss constraint generator is trained with the goal of minimizing the multi-loss constraint function, wherein the multi-loss constraint function includes an auditory perception constraint function, a time-frequency combination constraint function, and an adversarial generative loss function, and calculates the time-frequency combination constraint function based on the multi-dimensional features acquired by the multi-dimensional feature extraction and fusion network for the target underwater acoustic signal and the simulated target underwater acoustic signal; S3, generate an underwater acoustic signal based on the trained multi-constraint adversarial generative network.

[0147] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A data augmentation method for conditional adversarial generative networks based on multi-feature constraints, characterized in that, Includes the following steps: S1, acquire the target underwater acoustic signal, and construct a multi-dimensional feature extraction and fusion network, which includes a time-frequency feature extractor, a spectral feature extractor, and a feature fusion network; S2, Construct a multi-constraint adversarial generative network to generate a simulated target underwater acoustic signal. The multi-constraint adversarial generative network includes a multi-loss constraint generator and a discriminator. The multi-constraint adversarial generative network is trained with the target underwater acoustic signal as input. The multi-loss constraint generator is trained with the goal of minimizing the multi-loss constraint function. The multi-loss constraint function includes an auditory perception constraint function, a time-frequency combined constraint function, and an adversarial generative loss function. The time-frequency combined constraint function is calculated based on the multi-dimensional feature extraction and fusion network to obtain the multi-dimensional features of the target underwater acoustic signal and the simulated target underwater acoustic signal. S3, underwater acoustic signal generation is performed based on the multi-constraint adversarial generative network after training is completed.

2. The data augmentation method for conditional adversarial generative networks based on multi-feature constraints according to claim 1, characterized in that, The time-frequency feature extractor takes the target underwater acoustic signal as input to extract the time-frequency features of the target underwater acoustic signal, the spectral feature extractor takes the Mel spectrum of the target underwater acoustic signal as input to extract the spectral features of the target underwater acoustic signal, and the feature fusion unit receives the time-frequency features and the spectral features and performs feature fusion based on the cross-attention mechanism to obtain the multidimensional features of the target underwater acoustic signal.

3. The data augmentation method for conditional adversarial generative networks based on multi-feature constraints according to claim 2, characterized in that, The time-frequency feature extractor includes: a normalization layer, two cascaded residual convolutional layers, a pooling layer, a two-dimensional convolutional layer, a batch normalization layer, a channel attention layer, and a GLformer module; The target underwater acoustic signal is input into the normalization layer for amplitude normalization. The output of the normalization layer is processed by the two cascaded residual convolutional layers to extract time-domain and local frequency-domain features. Then, it is processed by the pooling layer for downsampling to obtain waveform features. The output of the normalization layer is processed by full-band fast Fourier transform to obtain spectral features. After the spectral features and the waveform features are concatenated, they are processed by the two-dimensional convolutional layer for preliminary feature fusion, then batch normalization is performed by the batch normalization layer, then channel attention weighting is performed by the channel attention layer, and finally the time-frequency features are obtained by the GLformer module.

4. The data augmentation method for conditional adversarial generative networks based on multi-feature constraints according to claim 2, characterized in that, The spectral feature extractor includes: a batch normalization layer, four cascaded modules, a pooling layer, and a fully connected layer. Each cascaded module consists of three parallel residual convolutional layers of different scales, a feature fusion and attention layer, and a convolutional layer with a 1×1 kernel. The Mel spectrum of the target underwater acoustic signal is input into the batch normalization layer for batch normalization. The output of the batch normalization layer is processed by the four cascaded modules and then input into the pooling layer for downsampling to extract high-level semantics. Finally, the spectral features are obtained through the fully connected layer. The input of the cascaded modules is processed by the three parallel residual convolutional layers of different scales to obtain three parallel features of spectral patterns under different receptive fields. The three parallel features are concatenated and weighted in the channel dimension through feature fusion and attention layer processing. Then, the channel compression and nonlinear mapping are performed by the convolutional layer with a 1×1 kernel before output.

5. The data augmentation method for conditional adversarial generative networks based on multi-feature constraints according to claim 1, characterized in that, In S1, after acquiring the target underwater acoustic signal, noise is added to the target underwater acoustic signal before it is input into the multidimensional feature extraction and fusion network. The added noise is one or more of Gaussian noise, uniform noise, impulse noise, salt-and-pepper noise, and multipath interference noise.

6. The data augmentation method for conditional adversarial generative networks based on multi-feature constraints according to claim 1, characterized in that, The multi-loss constraint function is defined as follows: L G =λ1L mel +λ2L feat +λ3L adv Among them, L mel L feat L feat Let λ1, λ2, and λ3 represent the auditory perception constraint function, the time-frequency combination constraint function, and the adversarial generation loss function, respectively, and let λ1, λ2, and λ3 represent the corresponding weights. Wherein, the auditory perception constraint function L mel Defined as L mel =||Mel(x) gen )-Mel(x real )||,Mel(x gen Mel(x) represents the Mel spectrum of the simulated target underwater acoustic signal generated by the multi-loss constraint generator. real () represents the true Mel spectrum of the target underwater acoustic signal; The time-frequency combined constraint function L feat Defined as f gen f represents the multidimensional features of the simulated target underwater acoustic signal generated by the multi-loss constraint generator. real This represents the multidimensional features of the target underwater acoustic signal; The adversarial generation loss function L adv Defined as D(G(x)) represents the discrimination result of the discriminator on the simulated target underwater acoustic signal generated by the multi-loss constraint generator, and G(x) represents the simulated target underwater acoustic signal generated by the multi-loss constraint generator. This represents the expectation of calculating the distributed sampling of the multi-loss constraint generator.

7. The data augmentation method for conditional adversarial generative networks based on multi-feature constraints according to claim 6, characterized in that, The multi-constraint adversarial generative network is trained by alternating optimization of the multi-loss constraint generator and the discriminator, wherein the loss function of the discriminator is defined as D(x) represents the discrimination result of the discriminator on the target underwater acoustic signal. This represents the expected value of the target underwater acoustic signal distribution sampling.

8. The data augmentation method for conditional adversarial generative networks based on multi-feature constraints according to claim 1, characterized in that, The multi-loss constraint generator is an encoder-decoder network based on an improved one-dimensional U-Net network. The downsampling structure of the encoding part and the upsampling structure of the decoding part are combined with a multi-scale residual convolution module and an SE channel attention module and are set symmetrically. The upsampling structure of the decoding part and the features output by the downsampling structure of the encoding part at the same level are skipped. The multi-scale residual convolution module is a multi-layer convolutional layer with convolutional kernels of different scales set in parallel. The discriminator is a one-dimensional convolutional network structure that extracts the distribution features of the signal input to the discriminator through multiple convolutions and global average pooling, and then classifies the signal input to the discriminator.

9. A conditional adversarial generative network data augmentation system based on multi-feature constraints, characterized in that, The method according to any one of claims 1 to 8 comprises the following modules: A multidimensional feature extraction and fusion network construction module is configured to acquire target underwater acoustic signals and construct a multidimensional feature extraction and fusion network, wherein the multidimensional feature extraction and fusion network includes a time-frequency feature extractor, a spectral feature extractor, and a feature fusion unit. The conditional adversarial generative network training module is configured to construct a multi-constraint conditional adversarial generative network to generate a simulated target underwater acoustic signal. The multi-constraint conditional adversarial generative network includes a multi-loss constraint generator and a discriminator. The multi-constraint conditional adversarial generative network is trained with the target underwater acoustic signal as input. The multi-loss constraint generator is trained with the goal of minimizing the multi-loss constraint function. The multi-loss constraint function includes an auditory perception constraint function, a time-frequency combined constraint function, and an adversarial generative loss function. The time-frequency combined constraint function is calculated based on the multi-dimensional features of the target underwater acoustic signal and the simulated target underwater acoustic signal obtained by the multi-dimensional feature extraction and fusion network. The underwater acoustic signal generation module is configured to generate underwater acoustic signals based on the multi-constraint adversarial generative network after it has been trained.

10. A computer program product, characterized in that, It stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-8.

Images