Unmanned underwater vehicle sound signal synthesis method and system based on generative adversarial network and related device

Abstract

The invention provides an unmanned underwater vehicle sound signal synthesis method and system based on a generative adversarial network, and a related device. The method comprises the following steps: 1, obtaining a to-be-processed unmanned underwater vehicle sound signal data set and input parameters used for describing an underwater detection scene; step 2, performing feature extraction on the obtained unmanned underwater vehicle sound signal data set to obtain voiceprint features; encoding the obtained input parameters used for describing the underwater detection scene to obtain environmental parameter semantic representation; 3, performing residual quantization processing on the obtained voiceprint features to obtain quantized sparse voiceprint features corresponding to the sound signals of the unmanned underwater vehicle; step 4, carrying out feature fusion on the obtained environmental advisory semantic representation and the quantized sparse voiceprint features to obtain fusion features; 5, decoding the obtained fusion features, and generating an unmanned underwater vehicle sound signal; according to the method, a generative adversarial network framework is utilized, residual vector quantization and feature fusion technologies are combined, unmanned underwater vehicle sound signals which are high in fidelity and adapt to different underwater environments can be efficiently synthesized under the condition of data scarcity, the problems that a traditional method is high in data dependence and complex scene modeling is difficult are effectively solved, and the method is suitable for being applied to unmanned underwater vehicle sound signals. And the method is of great significance in improving the underwater target detection and recognition capability.

Description

Technical Field

[0001] This invention belongs to the field of acoustic signal synthesis and relates to a method, system and related device for acoustic signal synthesis of unmanned underwater vehicles based on generative adversarial networks. Background Technology

[0002] Sonar is a device that uses sound waves to detect underwater objects. It determines an object's location, distance, speed, and other information by emitting sound waves and receiving their reflected signals. Sonar equipment mainly consists of a transmitter and a receiver, and its working principle is similar to radar, but it uses sound waves instead of electromagnetic waves. Due to advancements in silent technology for unmanned underwater vehicles and increased maritime traffic, the importance of sonar has become increasingly prominent. However, the multipath propagation of sound waves and the low signal-to-noise ratio caused by multiple clutter elements make it difficult to use sonar to detect, track, and identify underwater targets. To address this problem, machine learning and deep learning technologies, which have received considerable attention in recent years, have been applied to this field, but these technologies require large amounts of data. To supplement insufficient sonar data, traditional methods based on mathematical modeling are mainly employed. However, mathematical modeling-based methods have limitations in accurately simulating complex underwater scenarios. Summary of the Invention

[0003] The purpose of this invention is to provide a method, system, and related apparatus for synthesizing acoustic signals of unmanned underwater vehicles based on generative adversarial networks, which solves the problem that existing deep learning-based sound synthesis methods suffer from data scarcity, making it difficult to track and identify underwater targets.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides a method for synthesizing acoustic signals of an unmanned underwater vehicle based on generative adversarial networks, comprising the following steps: Step 1: Obtain the acoustic signal dataset of the unmanned underwater vehicle to be processed, as well as the input parameters used to describe the underwater exploration scenario; Step 2: Extract features from the acquired acoustic signal dataset of the unmanned underwater vehicle to obtain acoustic signature features; encode the acquired input parameters used to describe the underwater exploration scenario to obtain semantic representations of environmental parameters. Step 3: Perform residual quantization on the obtained voiceprint features to obtain the quantized sparse voiceprint features corresponding to the acoustic signal of the unmanned underwater vehicle. Step 4: The obtained semantic representation of environmental adoption number and quantized sparse voiceprint features are fused to obtain fused features; Step 5: Decode the obtained fused features to generate the acoustic signal of the unmanned underwater vehicle.

[0005] Preferably, the input parameters used to describe the underwater detection scenario include hull angle, target size, target distance, horizontal azimuth, vertical azimuth, sound speed, temperature, salinity, pH value, and water depth.

[0006] Preferably, the acoustic signal of the unmanned underwater vehicle is extracted using an encoder to obtain acoustic signature features; The encoder includes multiple convolutional layers, each followed by batch normalization and a sine activation function.

[0007] Preferably, the input parameters used to describe the underwater detection scenario are encoded using a sonar parameter embedding module to obtain a semantic representation of the environmental parameters; The sonar parameter embedding module is a T5 text encoder unit.

[0008] Preferably, the obtained acoustic features are subjected to residual quantization processing using a residual vector quantization module to obtain quantized sparse acoustic features corresponding to the acoustic signal of the unmanned underwater vehicle. The residual vector quantization module includes several sub-layers, wherein the first sub-layer is configured to quantize a vector in the latent representation to obtain a quantized residual; the remaining sub-layers are configured to iteratively quantize the quantized residual of the previous sub-layer to obtain a set of quantized vectors, and the obtained set of quantized vectors is used as quantized sparse voiceprint features.

[0009] Preferably, the obtained semantic representation of the environmental adoption number and the quantized sparse voiceprint features are fused using a Transformer-based decoder unit to obtain fused features.

[0010] Secondly, the present invention provides an acoustic signal synthesis system for unmanned underwater vehicles based on generative adversarial networks, comprising: The data acquisition unit is used to acquire the acoustic signal dataset of the unmanned underwater vehicle to be processed, as well as the input parameters used to describe the underwater exploration scenario; The data processing unit is used to extract features from the acquired acoustic signal dataset of the unmanned underwater vehicle to obtain acoustic signature features; and to encode the acquired input parameters used to describe the underwater exploration scenario to obtain semantic representations of environmental parameters. The feature residual processing unit is used to perform residual quantization processing on the obtained acoustic features to obtain the quantized sparse acoustic features corresponding to the acoustic signal of the unmanned underwater vehicle. The feature fusion processing unit is used to fuse the obtained semantic representation of the environmental adoption number and the quantized sparse voiceprint features to obtain fused features; The feature decoding unit is used to decode the obtained fused features to generate the acoustic signal of the unmanned underwater vehicle.

[0011] Thirdly, the present invention provides an electronic device including a processor and a memory, wherein the memory stores computer instructions, and when the computer instructions are executed by the processor, the electronic device performs the method described thereon.

[0012] Fourthly, the present invention provides a computer program product, the computer program product including computer-executable instructions, which, when executed, implement the method described.

[0013] Fifthly, the present invention provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the method described herein.

[0014] Compared with the prior art, the beneficial effects of the present invention are: This invention provides a method for synthesizing acoustic signals of unmanned underwater vehicles (UUVs) based on generative adversarial networks (GANs). This method addresses the scarcity of acoustic signal data in underwater target detection. Traditional methods rely on large amounts of measured data or simulations based on mathematical modeling. The former is difficult and costly to obtain, while the latter has limitations in simulating complex and variable underwater environments, resulting in insufficient realism and adaptability of the synthesized signals. This invention, by introducing a GAN framework, can efficiently synthesize high-quality UUV acoustic signals with limited sample size, effectively alleviating the constraints of data scarcity on model training and performance improvement. The method incorporates residual vector quantization (RVC) technology to discretize continuous acoustic features, enhancing feature expressiveness and compression efficiency. Simultaneously, through an encoder-decoder structure and a Transformer module, it achieves multi-level fusion of acoustic features and environmental parameters, enabling the synthesized signal to not only retain the target's acoustic characteristics but also adapt to different underwater detection scenarios (such as different hull angles, distances, water temperatures, and salinity).

[0015] In summary, this invention utilizes a generative adversarial network framework, combined with residual vector quantization and feature fusion techniques, to efficiently synthesize high-fidelity acoustic signals of unmanned underwater vehicles that are adaptable to different underwater environments under conditions of data scarcity. This effectively solves the problems of strong data dependence and difficulty in modeling complex scenes in traditional methods, and is of great significance for improving the detection and identification capabilities of underwater targets. Attached Figure Description

[0016] Figure 1 This is an overall structural design diagram related to an embodiment of the present invention; Figure 2 This is a schematic diagram of the encoder network structure according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the decoder network structure according to an embodiment of the present invention; Figure 4This is a schematic diagram of the residual vector quantization module algorithm involved in an embodiment of the present invention. Detailed Implementation

[0017] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0018] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0019] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0020] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0021] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0022] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0023] Example 1 This embodiment provides a method for synthesizing acoustic signals of unmanned underwater vehicles based on generative adversarial networks, including the following steps: Step 1: Obtain the acoustic signal dataset of the unmanned underwater vehicle to be processed, as well as the input parameters used to describe the underwater exploration scenario; Step 2: Extract features from the acquired acoustic signal dataset of the unmanned underwater vehicle to obtain acoustic signature features; encode the acquired input parameters used to describe the underwater exploration scenario to obtain semantic representations of environmental parameters. Step 3: Perform residual quantization on the obtained voiceprint features to obtain the quantized sparse voiceprint features corresponding to the acoustic signal of the unmanned underwater vehicle. Step 4: The obtained semantic representation of environmental adoption number and quantized sparse voiceprint features are fused to obtain fused features; Step 5: Decode the obtained fused features to generate the acoustic signal of the unmanned underwater vehicle.

[0024] Example 2 See Figures 1 to 4 This embodiment provides a method for synthesizing acoustic signals of unmanned underwater vehicles based on generative adversarial networks (GANs). This method utilizes GAN technology to achieve efficient signal synthesis with a limited number of samples. Specifically, it includes the following steps: Step 1: Obtain the dataset, which includes an unmanned underwater vehicle (UUV) acoustic signal dataset and corresponding sonar equipment environmental parameters. The UUV acoustic signal dataset is used to train the encoder module, and the sonar equipment environmental parameters are used to train the sonar parameter embedding module. If the UUV acoustic signal originates from a simulation experiment, the sonar equipment environmental parameters are already obtained as initial conditions; if the UUV acoustic signal originates from measured data, the sonar equipment environmental parameters need to be obtained through UUV array signal processing.

[0025] Preprocessing can include, but is not limited to, data cleaning, data augmentation, label generation, and dataset partitioning. Specifically, cleaning the raw acoustic waveform of the unmanned underwater vehicle (UUV) removes potential gaps, missing data, or other interference noise. This can be achieved by applying filtering techniques or other signal processing methods. Due to environmental factors or the sonar equipment itself, UUV acoustic waveform data may contain large gaps or excessive noise that overwhelms the target signal. Removing such data helps improve the model's generalization ability, and clean acoustic data helps the model better learn real-world scenarios.

[0026] Step 2: Input the acoustic signal dataset from the unmanned underwater vehicle into the encoder module. The encoder module extracts acoustic features through a series of convolutional downsampling operations. The specific implementation of the encoder module is as follows: The encoder module employs a multi-layer convolutional network to extract acoustic features. This network can extract features from the time-domain signal to obtain compressed acoustic features. After each convolution operation, batch normalization and ReLU activation are performed. The convolution formula is as follows:

[0027] In the formula: For the output feature map, the first The spatial location of each channel The value; For the first input feature map Each channel is located in The value; denoted as the weights of the convolution kernel, where u and v are the kernel spatial indices, c′ is the input channel index, and k is the output channel index; This is the bias term for the k-th output channel; Input the number of channels; and These represent the height and width of the convolution kernel, respectively.

[0028] In some embodiments, the encoder module includes a lower-level one-dimensional convolution, four convolutional blocks, and an upper-level one-dimensional convolution. The acoustic signal from the unmanned underwater vehicle is input to the lower-level one-dimensional convolution module to obtain a multi-channel lower-level feature map. The lower-level feature map is input to the convolution blocks, each of which includes several residual units that use dilated convolutions to increase the receptive field and capture longer contextual relationships. Simultaneously, the initial input is added to the output in the form of residual connections to avoid gradient vanishing or gradient exploding. The output of the residual units is input to a downsampling convolution. The stride of the downsampling convolution layer is greater than 1 to achieve the downsampling process. During downsampling, the number of channels is doubled to obtain a middle-level feature map. The middle-level feature map is input to the upper-level one-dimensional convolution module to compress the feature map dimension to a suitable size, obtaining a compressed feature map. To ensure realism, all convolutions use causal convolution. ReLU is used as the activation function, and layer normalization is employed.

[0029] Step 3: The acoustic features are input into the residual vector quantization module to obtain quantized latent features. These quantized latent features discretize the continuous acoustic features. The implementation of the residual vector quantization module is as follows: Several cascaded codebooks are used to approximate the quantization of acoustic features. At each step, the original vector is quantized by one codebook, and the quantization error (i.e., the residual vector) is calculated. Subsequent codebooks are then used to quantize the residual. Ultimately, the original vector is represented as a combination of multiple codebook indices, thus achieving efficient compression. The number of codebooks is pre-set as a hyperparameter, and the codebook size and number of layers can be adjusted according to requirements, flexibly balancing compression ratio and reconstruction quality.

[0030] Suppose we have an input vectorx The goal of residual vector quantization is to quantize stepwise using M codebook vectors. x The formula is:

[0031] In the formula: For the first m Layer Codebook One of the code words.

[0032] Residual vector recursive update: initial residual is residual of the m-th layer Final residual It should be as close to zero as possible.

[0033] In some embodiments, the residual vector quantization module can adopt two methods to improve the utilization rate of the vector quantization layer codebook. First, instead of using random initialization, the codebook is initialized using the k-means algorithm to cluster latent features, with the cluster centers used as the initial codebook. Second, codebook vectors with low utilization rates are replaced with a specific latent feature. The vector quantization layer structure contains several sub-layers, each quantizing the quantization residual of the previous layer, thereby discretizing continuous audio features. Furthermore, by discarding several quantization vectors from the last layer, the feature length can be changed without altering the encoder structure, achieving variable bit rate compression.

[0034] Step 3: Input the environmental target parameters (water depth, target distance, target heading angle, unmanned underwater vehicle frequency, bandwidth, waveform, etc.) into the sonar parameter embedding module to obtain semantic representation.

[0035] The sonar parameter embedding module is based on the Transformer model, where the self-attention mechanism maps active sonar operating parameters, environmental characteristics, and target parameters into a semantically dense feature vector. Its core idea is to generate a representation that integrates global information by dynamically calculating the dependency weights of each element in the sequence on all other elements.

[0036] Input sequence ( For sequence length, (For the feature dimension) is linearly transformed using three learnable weight matrices to generate query, key, and value vectors:

[0037] In the formula: These are the association requirements, associated characteristics, and actual information content of the captured elements; They are respectively The weight matrix; For input features.

[0038] After calculating the potential characteristics of environmental parameters After three matrices, calculate using the dot product. The similarity between them is used to obtain attention weights. The attention score is calculated using the following formula:

[0039] Where: scaling factor prevent Larger time dot product results can lead to Softmax gradient saturation.

[0040] Apply Softmax to each row (a single query) to convert the scores into a probability distribution:

[0041] The weight matrix is ​​summed with V to generate a new representation Z:

[0042] In the formula: Z is the final weighted feature output; The product of the transposes of the Query matrix and the Key matrix represents the feature similarity at different positions. Let V be the dimension of the Key vector; V is the Value matrix. This step achieves information filtering and fusion.

[0043] In some embodiments, the sonar parameter embedding module may use a pre-trained BERT or T5 model.

[0044] Step 4: The quantized acoustic compression representation and the semantic representation are fused using a cross-attention mechanism.

[0045] In some embodiments, the feature fusion module employs a Transformer decoder structure, uses a cross-attention mechanism to fuse acoustic compression representations and semantic representations, and generates sound features autoregressively.

[0046] Step 5: Input the fused features into the decoder module. The decoder module reconstructs the acoustic signal of the unmanned underwater vehicle based on the fused features. The specific implementation of the decoder module is as follows: The decoder module primarily uses a multi-layer deconvolution network to perform deconvolution operations on the fused features, restoring the fused target features to a time-domain signal. After each deconvolution operation, batch normalization and ReLU activation are performed. The deconvolution formula is as follows:

[0047] In the formula: To represent the input feature map; To represent the output feature map; Let represent the convolution kernel weight matrix.

[0048] The output size of deconvolution is determined by , , Three parameters determine this, among which, This represents the number of upsampling channels. , Calculated using the following formula:

[0049] In the formula: To indicate the input height; To indicate the input width; To indicate the step size (Stride); To represent the kernel size; To indicate padding; To indicate the output height; This indicates the output width.

[0050] In some embodiments, the decoder module mirrors the encoder structure and uses deconvolution for upsampling to reconstruct the acoustic features into unmanned underwater vehicle signals.

[0051] Step 6: Construct the discriminator module and define the loss function and optimization conditions.

[0052] The discriminator can be viewed as a classifier that learns the features of the data to divide the input data into different categories. In the generative adversarial model framework, the discriminator's main task is to distinguish between real and generated samples, and to continuously improve the quality of generated data through adversarial training against the generator. The discriminator module in this method employs two types of discriminators: one based on time-domain waveforms and the other on short-time Fourier transform. Both discriminators are fully convolutional.

[0053] In some embodiments, multiple discriminators with different resolutions can be configured. When training the discriminators, the following loss function is minimized:

[0054] In the formula: To combat the losses; To represent the expected calculation; To indicate batch size; For the first k Number of time steps per batch; For the discriminator in the first k The first batch, the first Each time step relative to real data The output; For the discriminator in the first k The first batch, the first t Each time step generates fake data. The output.

[0055] At the same time, minimize the decoder adversarial loss:

[0056] In the formula: To combat loss in generators; Calculate the expected value; Batch size; For the first k Number of time steps per batch; For the discriminator in the first k The first batch, the first t Each time step generates fake data. The output.

[0057] Design the feature loss function for the decoder and calculate the output reconstruction loss for each inner layer of the discriminator:

[0058] In the formula: The feature loss of the generator; Calculate the expected value; Batch size; For the first k The first batch, the first Number of time steps per feature layer; The number of feature layers; For the discriminator in the first The first batch, the first The time step, the first Each feature layer on real data The output; For the discriminator in the first The first batch, the first The time step, the first Generating fake data at each feature layer The output.

[0059] Design the STFT loss function for the decoder and calculate the frequency domain reconstruction loss:

[0060] In the formula: The reconstruction loss of the generator; To verify the actual data x In time t ,scale s STFT output below; For generator The generated data ( In time t ,scale s The STFT output below.

[0061] The total loss of the decoder is a weighted sum of the loss functions described above:

[0062] In the formula: The total loss function of the generator; This is a hyperparameter used to adjust the weight of the generator's adversarial loss in the total loss; To combat loss in generators; This is a hyperparameter used to adjust the weight of the generator feature loss in the total loss; The feature loss of the generator; This is a hyperparameter used to adjust the weight of the generator reconstruction loss in the total loss; This represents the reconstruction loss of the generator.

[0063] Example 3 This embodiment provides an acoustic signal synthesis system for unmanned underwater vehicles based on generative adversarial networks, comprising: The data acquisition unit is used to acquire the acoustic signal dataset of the unmanned underwater vehicle to be processed, as well as the input parameters used to describe the underwater exploration scenario; The data processing unit is used to extract features from the acquired acoustic signal dataset of the unmanned underwater vehicle to obtain acoustic signature features; and to encode the acquired input parameters used to describe the underwater exploration scenario to obtain semantic representations of environmental parameters. The feature residual processing unit is used to perform residual quantization processing on the obtained acoustic features to obtain the quantized sparse acoustic features corresponding to the acoustic signal of the unmanned underwater vehicle. The feature fusion processing unit is used to fuse the obtained semantic representation of the environmental adoption number and the quantized sparse voiceprint features to obtain fused features; The feature decoding unit is used to decode the obtained fused features to generate the acoustic signal of the unmanned underwater vehicle.

[0064] Example 4 This embodiment also provides a computing device. The computing device includes a bus, a processor, a memory, and a communication interface. The processor, memory, and communication interface communicate with each other via the bus. The computing device can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memory in the computing device.

[0065] A bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, a bus can include a path for transmitting information between various components of a computing device (e.g., memory, processor, communication interfaces).

[0066] The processor may include any one or more of the following: central processing unit (CPU), graphics processing unit (GPU), tensor processing unit (TPU), application specific integrated circuit (ASIC), field-programmable gate array (FPGA), microprocessor (MP), or digital signal processor (DSP).

[0067] Memory can include volatile memory, such as random access memory (RAM). Processors can also include non-volatile memory. volatile memory, such as read-only memory (ROM). ROM (memory only), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0068] The memory stores executable program code, which the processor executes to implement the functions of the aforementioned units, thereby achieving, for example, the method described in Embodiment 1. That is, the memory may store instructions for the methods and functions relating to the computing device in any of the above embodiments.

[0069] The communication interface uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between computing devices and other devices or communication networks.

[0070] Example 5 This embodiment also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, cause the processor to perform the methods and functions of the computing device involved in any of the above embodiments.

[0071] Generally, the various embodiments of this disclosure can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software, which can be executed by a controller, microprocessor, or other computing device. Although various aspects of the embodiments of this disclosure are shown and described as block diagrams, flowcharts, or represented using some other illustration, it should be understood that the blocks, apparatuses, systems, techniques, or methods described herein can be implemented as, as non-limiting examples, in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0072] Example 6 This embodiment provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods as described above with reference to the accompanying drawings. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.

[0073] Computer program code used to implement the methods of this disclosure may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, such that when executed by the computer or other programmable data processing apparatus, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be performed. The program code may be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0074] In the context of this disclosure, computer program code or related data may be carried on any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and so on. Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0075] Computer-readable media can be any tangible medium that contains or stores programs for or relating to an instruction execution system, apparatus, or device, or a data storage device such as a data center containing one or more available media. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. More detailed examples of computer-readable storage media include electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0076] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for synthesizing acoustic signals of an unmanned underwater vehicle based on generative adversarial networks, characterized in that, Includes the following steps: Step 1: Obtain the acoustic signal dataset of the unmanned underwater vehicle to be processed, as well as the input parameters used to describe the underwater exploration scenario; Step 2: Extract features from the acquired acoustic signal dataset of the unmanned underwater vehicle to obtain acoustic signature features; The obtained input parameters used to describe the underwater exploration scenario are encoded to obtain a semantic representation of the environmental parameters; Step 3: Perform residual quantization on the obtained voiceprint features to obtain the quantized sparse voiceprint features corresponding to the acoustic signal of the unmanned underwater vehicle. Step 4: The obtained semantic representation of environmental adoption number and quantized sparse voiceprint features are fused to obtain fused features; Step 5: Decode the obtained fused features to generate the acoustic signal of the unmanned underwater vehicle.

2. The method for synthesizing acoustic signals of unmanned underwater vehicles based on generative adversarial networks according to claim 1, characterized in that, The input parameters used to describe the underwater exploration scenario include hull angle, target size, target distance, horizontal azimuth, vertical azimuth, sound speed, temperature, salinity, pH value, and water depth.

3. The method for synthesizing acoustic signals of unmanned underwater vehicles based on generative adversarial networks according to claim 1, characterized in that, The acoustic signal of the unmanned underwater vehicle was extracted using an encoder to obtain the acoustic signature features; The encoder includes multiple convolutional layers, each followed by batch normalization and a sine activation function.

4. The method for synthesizing acoustic signals of unmanned underwater vehicles based on generative adversarial networks according to claim 1, characterized in that, The input parameters used to describe the underwater detection scenario are encoded using a sonar parameter embedding module to obtain a semantic representation of the environmental parameters; The sonar parameter embedding module is a T5 text encoder unit.

5. The method for synthesizing acoustic signals of an unmanned underwater vehicle based on a generative adversarial network according to claim 1, characterized in that, The residual vector quantization module is used to perform residual quantization processing on the obtained acoustic features to obtain the quantized sparse acoustic features corresponding to the acoustic signal of the unmanned underwater vehicle. The residual vector quantization module includes several sub-layers, wherein the first sub-layer is configured to quantize a vector in the latent representation to obtain a quantized residual; The remaining sub-layers are configured to iteratively quantize the quantization residuals of the previous sub-layer to obtain a set of quantization vectors, which are then used as quantized sparse voiceprint features.

6. The method for synthesizing acoustic signals of an unmanned underwater vehicle based on a generative adversarial network according to claim 1, characterized in that, The obtained semantic representation of the environmental adoption number and the quantized sparse voiceprint features are fused using a Transformer-based decoder unit to obtain fused features.

7. A system for synthesizing acoustic signals of an unmanned underwater vehicle based on generative adversarial networks, characterized in that, include: The data acquisition unit is used to acquire the acoustic signal dataset of the unmanned underwater vehicle to be processed, as well as the input parameters used to describe the underwater exploration scenario; The data processing unit is used to extract features from the acquired acoustic signal dataset of the unmanned underwater vehicle to obtain acoustic signature features; The obtained input parameters used to describe the underwater exploration scenario are encoded to obtain a semantic representation of the environmental parameters; The feature residual processing unit is used to perform residual quantization processing on the obtained acoustic features to obtain the quantized sparse acoustic features corresponding to the acoustic signal of the unmanned underwater vehicle. The feature fusion processing unit is used to fuse the obtained semantic representation of the environmental adoption number and the quantized sparse voiceprint features to obtain fused features; The feature decoding unit is used to decode the obtained fused features to generate the acoustic signal of the unmanned underwater vehicle.

8. An electronic device, characterized in that, It includes a processor and a memory, the memory storing computer instructions that, when executed by the processor, cause the electronic device to perform the method of any one of claims 1 to 6.

9. A computer program product, characterized in that, The computer program product includes computer-executable instructions that, when executed, implement the method of any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when executed by a processor, implement the method of any one of claims 1 to 6.

Images