A sonar image database augmentation device and method based on recurrent adversarial generative networks

Abstract

This invention discloses a sonar image database augmentation device and method based on a recurrent adversarial generative network (ROGGN). It includes a raw data acquisition hardware module; a raw data acquisition software module; a sonar image preprocessing module with histogram equalization, contour clustering and pixel filling, size slicing, and image filtering units; and a sonar image synthesis module with a discrete cosine transform RUGGN, pre-configured with a loss function described by the following formula, used for training constraints during the adversarial generation of sonar images from real sonar images and remote sensing images. This invention can obtain high-quality sonar images suitable for detection, segmentation, and other tasks, greatly alleviating the problem of sonar data scarcity in underwater missions and providing highly realistic training data for underwater missions.

Description

Technical Field

[0001] This invention relates to the field of underwater acoustic imaging technology, and in particular to a sonar image database augmentation device and method based on a recurrent adversarial generative network. Background Technology

[0002] Underwater acoustic imaging research has received increasing attention in recent years, with a wide range of applications including seabed resource location, seabed morphology modeling and mapping, underwater salvage and rescue, and seabed target location and identification. Currently, underwater acoustic tasks such as target detection mostly employ traditional image processing methods based on handcrafted features. Due to their poor generalization ability, these methods are often unable to handle detection tasks in complex environments. To improve the efficiency and completion rate of underwater acoustic operations, the introduction of deep learning is an inevitable trend in the development of underwater operations.

[0003] Underwater research is inseparable from underwater images. Deep learning-based underwater image processing and analysis techniques have become an emerging research direction, such as underwater image segmentation, underwater target detection, underwater image enhancement, and underwater image synthesis.

[0004] However, in the real world, the underwater environment is extremely harsh. Poor underwater lighting, high concentrations of suspended matter, and abundant aquatic life often lead to low visibility and difficulties in exploration, preventing the widespread adoption of optical sensors in underwater environments. To address underwater exploration missions and accelerate underwater research, researchers often use acoustic sensors as the primary underwater detection equipment.

[0005] Sound waves possess strong penetrating power, slow attenuation, and long range in underwater environments. Therefore, sonar sensors are widely used in underwater detection due to their high adaptability to underwater environments. However, acquiring raw sonar data is difficult because sonar sensors are expensive, making underwater data acquisition costly. Furthermore, the acquisition process requires the design and installation of sonar-mounted equipment, and the complex underwater environment presents significant obstacles and challenges, typically consuming substantial time, manpower, and resources. Deep learning methods for underwater tasks often require massive amounts of training data, and the limited availability of real sonar data is often insufficient to support the desired performance of deep learning models. To address the problem of insufficient training data for deep learning methods, more and more researchers are focusing on image generation, using synthetic data to alleviate the data burden on deep learning models.

[0006] Currently, the main technologies for synthesizing sonar images include: 1) using 3D modeling software to simulate sonar images, and 2) using neural network methods to synthesize sonar images.

[0007] Existing technology employs a 3D modeling software technique to simulate sonar data. This technique first establishes a 3D model of the seabed topography and target objects based on seabed topographic point cloud information and target object information. Second, this 3D model is imported into Unity3D 3D modeling software to construct a simulated seabed scene. Subsequently, the sound waves emitted by the sonar are modeled as ray clusters, forming a sound wave scanning area. Finally, the reflection of sound waves on the target objects is modeled as collisions between the rays and the 3D model of the target objects. The shape and spacing of the target objects are obtained based on the echo time generated by the collisions to further simulate sonar images. However, the process of constructing the seabed environment requires point cloud information of the seabed topography and relevant data of the target objects. This data needs to be collected and processed in advance, increasing the requirements for modeling. During the simulation, the changes in the simulation model deviate from the actual operation of the equipment, and the simulation data is overly idealistic and does not conform to reality.

[0008] Existing technologies also include techniques that use non-parametric sampling and synthetic data generated by neural networks to augment training data. This technique first obtains preliminary sonar images through non-parametric sampling. Next, the seabed region and target region are segmented from the real sonar images as templates for synthesizing the target image. The preliminary sonar images are then filtered, and seabed and target regions are randomly selected as texture image training templates to establish a U-net deep learning network model. Subsequently, the training samples are used as input, and the difference image between the target image and the training sample for each training sample is used as the target image to train the deep learning model. Finally, the output result is obtained by merging the image to be synthesized with the output result to obtain the final sonar image.

[0009] However, non-parametric methods require high-density computation, which can severely impact computational speed. Furthermore, non-parametric methods have poor generalization ability, which can affect the initial synthesized sonar images. During the training phase of deep models, aligned data pairs are usually required, but paired data may not be available in many real-world situations. Without using aligned data pairs, there is often a lack of constraints on structural consistency, and the neural network may lose geometric information. Summary of the Invention

[0010] The purpose of this invention is to provide a sonar image database augmentation device and method based on recurrent adversarial generative networks, which can enrich underwater acoustic image datasets, alleviate the predicament of underwater sonar data lack faced by underwater researchers, and promote the development and application of underwater target detection and segmentation tasks based on deep learning methods.

[0011] To achieve the above objectives, the present invention provides a sonar image database augmentation device based on a recurrent adversarial generative network, comprising:

[0012] The raw data acquisition hardware module is used to acquire real sonar images;

[0013] The raw data acquisition software module is used to analyze and read real sonar images;

[0014] The sonar image preprocessing module has the following features:

[0015] The histogram equalization unit is used to obtain a sonar image containing the target object of a preset size based on remote sensing images and real sonar images;

[0016] The contour clustering and pixel filling unit is used to enhance the quality of the sonar image output by the histogram equalization unit 31, extract the contour information of the target object from the enhanced sonar image, and finally fill the pixels in the contour to obtain the candidate sonar image.

[0017] The size slicing unit and the image filtering unit are used to filter out sonar images that meet the preset conditions from the candidate sonar images, and classify and store them according to the category corresponding to the target object in the sonar image, so as to obtain real sonar images that can be directly used for network model training.

[0018] The sonar image synthesis module has a discrete cosine transform cyclic adversarial generative network, which has a pre-set loss function described by the following formula. Used to provide training constraints during the process of training real sonar images and remote sensing images to generate adversarial sonar images;

[0019]

[0020] In the formula, G1 is the first generator, used to generate sonar image G1(x) based on remote sensing image x in remote sensing image set X, and G2 is the second generator, used to generate remote sensing image G2(y) based on sonar image y in real sonar image set Y. This is the global discriminator for G1, used to calculate the global loss of the sonar images generated by G1. This is the global discriminator for G2, used to calculate the global loss of the remote sensing image X generated by G2. This is the local discriminator for G1, used to calculate the local loss of the sonar image generated by G1. This is the local discriminator of G2, used to calculate the local loss of the remote sensing image X generated by G2. For G1 and Global combat losses, For G2 and Global combat losses, For G1 and Localized combat losses, For G2 and Localized combat losses, L c (G1,G2) represents the discrete cosine transform cycle consistency loss. λ1 is the weight of the local loss, and λ2 is the weight of the cycle consistency loss.

[0021] Furthermore,

[0022] In the formula, Let Y be the expected distribution of the sonar image dataset. Let X be the expected distribution of the remote sensing image dataset.

[0023] Furthermore,

[0024] In the formula, ||*||2 represents L2 regularization. For a local part of G1(x), This is a partial view of a real sonar image.

[0025] Furthermore,

[0026] In the formula, Dct x (x), Dct y (y) represents the result of encoding x and y after the discrete cosine transform changes, ||*||2 represents L2 regularization, G2(G1(x)) represents the reconstructed image of remote sensing image x, and G1(G2(y)) represents the reconstructed image of sonar image y.

[0027] This invention also provides a method for augmenting a sonar image database based on a recurrent adversarial generative network, comprising:

[0028] Step 1: Acquire real sonar images;

[0029] Step 2: Based on the remote sensing image and the actual sonar image, obtain a sonar image containing the target object of a preset size;

[0030] Step 3: Enhance the quality of the sonar image output in Step 2, extract the contour information of the target object from the enhanced sonar image, and finally fill the pixels within the contour to obtain the candidate sonar image.

[0031] Step 4: Select sonar images that meet the preset conditions from the candidate sonar images, and classify and store them according to the category corresponding to the target object in the sonar images to obtain real sonar images that can be directly used for network model training.

[0032] Step 5: Use the Discrete Cosine Transform (DCT) cyclic adversarial generator network to generate a loss function pre-defined by the following formula. Training constraints are applied during the process of training real sonar images and remote sensing images to generate sonar images adversarially.

[0033]

[0034] In the formula, G1 is the first generator, used to generate sonar image G1(x) based on remote sensing image x in remote sensing image set X, and G2 is the second generator, used to generate remote sensing image G2(y) based on sonar image y in real sonar image set Y. This is the global discriminator for G1, used to calculate the global loss of the sonar images generated by G1. This is the global discriminator for G2, used to calculate the global loss of the remote sensing image X generated by G2. This is the local discriminator for G1, used to calculate the local loss of the sonar image generated by G1. This is the local discriminator of G2, used to calculate the local loss of the remote sensing image X generated by G2. For G1 and Global combat losses, For G2 and Global combat losses, For G1 and Localized combat losses, For G2 and Localized combat losses, L c (G1,G2) represents the discrete cosine transform cycle consistency loss. λ1 is the weight of the local loss, and λ2 is the weight of the cycle consistency loss.

[0035] Furthermore,

[0036] In the formula, Let Y be the expected distribution of the sonar image dataset. Let X be the expected distribution of the remote sensing image dataset.

[0037] Furthermore,

[0038] In the formula, ||*||2 represents L2 regularization. For a local part of G1(x), This is a partial view of a real sonar image.

[0039] Furthermore,

[0040] In the formula, Dct x (x), Dct y(y) represents the result of encoding x and y after the discrete cosine transform changes, ||*||2 represents L2 regularization, G2(G1(x)) represents the reconstructed image of remote sensing image x, and G1(G2(y)) represents the reconstructed image of sonar image y.

[0041] This invention acquires real sonar images through a sonar data acquisition platform for training a recurrent adversarial generative network (ROGGN). The RPGN then converts remote sensing images into sonar images, thus expanding the sonar data. In this system, for the sonar data acquisition platform module, this invention constructs a sonar data acquisition platform to collect raw sonar data. For the raw sonar data preprocessing module, this invention performs preprocessing on the previously acquired sonar data, including slicing, filtering, and image enhancement, to obtain high-quality 256×256 sonar images with clear targets, minimal noise impact, for RPGGN training. For the Discrete Cosine Transform (DCT) RPGGN, this invention defines a new cyclic consistency loss using the DCT of the image to ensure structural consistency. Furthermore, an attention module and a local discriminator are used to improve the consistency of foreground objects, ensuring that the foreground objects and background in the sonar images maintain a consistent style. With existing sonar image synthesis technology, this method can obtain high-quality sonar images that can be used for tasks such as detection and segmentation, promoting the rapid development of the underwater field, greatly alleviating the problem of sonar data scarcity in underwater missions, and providing highly realistic data for underwater missions to be used for training. Attached Figure Description

[0042] Figure 1 This is a schematic flowchart of a sonar image database augmentation device based on a recurrent adversarial generative network provided in an embodiment of the present invention.

[0043] Figure 2 for Figure 1 A schematic diagram of the structure of an embodiment of the loss function in the diagram.

[0044] Figure 3 for Figure 1 A schematic diagram of the structure of an embodiment of the generator. Detailed Implementation

[0045] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0046] This invention provides a low-cost, high-efficiency sonar database augmentation device, which involves building a hardware and software platform for acquiring original sonar data, performing preprocessing such as slicing, filtering, and image enhancement on real sonar images, and finally using CycleGAN image synthesis technology to synthesize underwater sonar images from remote sensing images and acquired real sonar images to augment the sonar image dataset.

[0047] like Figure 1 As shown, the present invention is a privately provided sonar image database augmentation device based on a recurrent adversarial generative network, comprising a raw data acquisition hardware module 1, a raw data acquisition software module 2, a sonar image preprocessing module 3, and a sonar image synthesis module 4, wherein:

[0048] The raw data acquisition hardware module 1 includes a sonar device 11 and an AUV (Aerial Vehicle) untethered underwater robot 12. The AUV untethered underwater robot 12 uses the sonar device 11 to acquire realistic sonar images of the underwater scene. The sonar device 11 can be a multi-beam dual-frequency sonar instrument or other types of sonar instruments. It should be noted that a large amount of sonar image data of the underwater scene can be acquired on-site using sonar equipment, and the sonar image data can also be simulated by building underwater scene and object models through 3D modeling and simulation.

[0049] The raw data acquisition software module 2 includes a cloud service operating system 21 and a sonar raw data parsing and reading unit 22. The cloud service operating system 21 integrates the environment required for various software operations, including subsequent data parsing and reading, sonar image preprocessing, and network training and synthesis modules. The sonar raw data parsing and reading unit 22 is used to parse and read the acquired sonar data.

[0050] The sonar image preprocessing module 3 includes a histogram equalization unit 31, a contour clustering and pixel filling unit 32, a size slicing unit, and an image filtering unit 33, wherein:

[0051] Histogram equalization unit 31 is used to obtain a sonar image containing the target object of a preset size based on remote sensing images and real sonar images. The preset size is required to closely match the size of the target object, which is beneficial for subsequent network training; for example, it can be 256×256 or other suitable sizes. Remote sensing images can be obtained from publicly available open-source datasets.

[0052] The contour clustering and pixel filling unit 32 is used to enhance the quality of the sonar image output by the histogram equalization unit 31. Then, the contour information of the target object is extracted from the enhanced sonar image using methods such as k-means clustering algorithm. Finally, the pixels in the contour are filled using methods such as bilinear interpolation to obtain the candidate sonar image.

[0053] The size slicing unit and image filtering unit 33 are used to filter out sonar images that meet preset conditions from candidate sonar images, and classify and store them according to the category corresponding to the target object in the sonar image, so as to obtain real sonar images that can be directly used for network model training. Therefore, sonar images with clear targets, low noise impact, and high image quality can be filtered out.

[0054] The real sonar image shows clear targets and accurate classification.

[0055] Sonar image synthesis module 4 has a discrete cosine transform cyclic adversarial generative network 41, which has a loss function pre-set in the following equation (1). Used to provide training constraints during the process of training real sonar images and remote sensing images to generate adversarial sonar images.

[0056]

[0057] In the formula, G1 is the first generator, used to generate sonar image G1(x) based on remote sensing image x in remote sensing image set X, and G2 is the second generator, used to generate remote sensing image G2(y) based on sonar image y in real sonar image set Y. This is the global discriminator for G1, used to calculate the global loss of the sonar images generated by G1. This is the global discriminator for G2, used to calculate the global loss of the remote sensing image X generated by G2. This is the local discriminator for G1, used to calculate the local loss of the sonar image generated by G1. This is the local discriminator of G2, used to calculate the local loss of the remote sensing image X generated by G2. For G1 and Global combat losses, For G2 and Global combat losses, For G1 and Localized combat losses, For G2 and Localized combat losses, L c (G1,G2) represents the discrete cosine transform cycle consistency loss. λ1 is the weight of the local loss, and λ2 is the weight of the cycle consistency loss.

[0058] This invention utilizes the discrete cosine transform of an image to define a new cyclic consistency loss to ensure structural consistency.

[0059] In the above embodiments, the following three evaluation metrics for G1(x) and G2(y) all exceed the performance of existing methods in this regard:

[0060] Evaluation Metric 1: Inception Score (IS).

[0061] Evaluation metric two, Fréchet Inception Distance (FID), Fréchet Inception distance.

[0062] Evaluation metric three: Kernel Inception Distance (KID), the distance to Kernel Inception.

[0063] In one embodiment, This can be achieved using the following formula. Alternatively, it can be achieved by evaluating the overall style and global feature consistency of the image, thereby establishing a global loss function to impose global constraints and encourage the model to establish a stable and consistent mapping between the target and source domains. Other existing methods can also be used.

[0064]

[0065] In the formula, Let Y be the expected distribution of the sonar image dataset. Let X be the expected distribution of the remote sensing image dataset.

[0066] In one embodiment, This can be achieved using the following formula for inverse mapping. It can also be achieved through other methods, such as learning detailed features specific to sonar or remote sensing images and using these features to distinguish between real and fake samples, thereby establishing a local loss function to impose local constraints and encourage the generator to produce more realistic images.

[0067]

[0068] In the formula, ||*||2 represents L2 regularization. It is a local part of G1(x). This is a partial view of a real sonar image.

[0069] In one embodiment, L c (G1,G2) can be implemented using the following formula. It can also be implemented using other methods, such as: establishing a cyclic consistency loss based on the discrete cosine transform of the image to perform transform constraints, transforming the image from the spatial domain to the frequency domain, then discarding the low-frequency components (planar regions with grayscale variations) of the image, while retaining the high-frequency components (contour boundaries and texture information), and finally encoding the processed image to centrally distinguish the structural consistency of the target, ensuring a close match between the reconstructed image and the original image.

[0070]

[0071] In the formula, Dct x (x), Dct y (y) represents the result of encoding x and y after the discrete cosine transform changes, ||*||2 represents L2 regularization, G2(G1(x)) represents the reconstructed image of remote sensing image x, and G1(G2(y)) represents the reconstructed image of sonar image y.

[0072] Finally, the obtained sonar images are processed, and images that do not meet the requirements, such as those with blurred outlines or high noise, are removed.

[0073] In this embodiment, during the deep neural network training phase of sonar images, aligned data pairs are typically required. However, paired data may not be available in many real-world situations. Without aligned data pairs, there is often a lack of constraints on structural consistency, and the neural network may lose geometric information. Therefore, this embodiment can ensure the structural consistency between the produced sonar images and real sonar images, greatly enhancing the synthesis effect and practicality of the sonar images. Furthermore, this embodiment also utilizes an attention module and a local discriminator to improve the consistency of foreground objects, ensuring that the foreground objects and background in the sonar images maintain a consistent style.

[0074] This invention also provides a method for augmenting a sonar image database based on a recurrent adversarial generative network, comprising:

[0075] Step 1: Acquire real sonar images;

[0076] Step 2: Based on the remote sensing image and the actual sonar image, obtain a sonar image containing the target object of a preset size;

[0077] Step 3: Enhance the quality of the sonar image output in Step 2, extract the contour information of the target object from the enhanced sonar image, and finally fill the pixels within the contour to obtain the candidate sonar image.

[0078] Step 4: Select sonar images that meet the preset conditions from the candidate sonar images, and classify and store them according to the category corresponding to the target object in the sonar images to obtain real sonar images that can be directly used for network model training.

[0079] Step 5: Use the Discrete Cosine Transform (DCT) cyclic adversarial generator network to generate a loss function pre-defined by the following formula. Training constraints are applied during the process of training real sonar images and remote sensing images to generate sonar images adversarially.

[0080]

[0081] In the formula, G1 is the first generator, used to generate sonar image G1(x) based on remote sensing image x in remote sensing image set X, and G2 is the second generator, used to generate remote sensing image G2(y) based on sonar image y in real sonar image set Y. This is the global discriminator for G1, used to calculate the global loss of the sonar images generated by G1. This is the global discriminator for G2, used to calculate the global loss of the remote sensing image X generated by G2. This is the local discriminator for G1, used to calculate the local loss of the sonar image generated by G1. This is the local discriminator of G2, used to calculate the local loss of the remote sensing image X generated by G2. For G1 and Global combat losses, For G2 and Global combat losses, For G1 and Localized combat losses, For G2 and Localized combat losses, L c (G1,G2) represents the discrete cosine transform cycle consistency loss. λ1 is the weight of the local loss, and λ2 is the weight of the cycle consistency loss.

[0082] like Figure 3 As shown, Figure 3 The diagram illustrates the network structure of a generator based on an attention module. The input remote sensing image first passes through a reflectionPad layer, a 7*7 Conv layer, a Batch Normalization layer, and a ReLU activation function layer. Then, it passes through two downsampling modules to obtain a 64*64 feature map with 256 channels. This feature map undergoes feature extraction using a 3-layer residual network module and a 6-layer attention mechanism module. It is then transformed into a 256*256 feature map with 64 channels through two upsampling layers. Finally, the generated sonar image is obtained through a reflectionPad layer, a 3*4 Conv layer, a Batch Normalization layer, and a Tanh activation function layer.

[0083] In the above embodiments, both the global and local discriminators use 70x70 PatchGANs, but their parameters differ due to their different tasks. The global discriminator aims to encourage the model to establish a stable and consistent mapping between the target and source domains by evaluating the overall style and global feature consistency of the image. On the other hand, the local discriminator focuses on learning detailed features specific to sonar or remote sensing images and uses these features to distinguish between real and fake samples. This helps encourage the generator to produce more realistic images.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Those skilled in the art should understand that modifications can be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A sonar image database augmentation device based on a recurrent adversarial generative network, characterized in that, include: The raw data acquisition hardware module (1) is used to acquire real sonar images; The raw data acquisition software module (2) is used to analyze and read real sonar images; The sonar image preprocessing module (3) has the following features: Histogram equalization unit (31) is used to obtain a sonar image containing the target object of a preset size based on remote sensing image and real sonar image; The contour clustering and pixel filling unit (32) is used to enhance the quality of the sonar image output by the histogram equalization unit (31), and then the k-means clustering algorithm is used to extract the contour information of the target object from the enhanced sonar image. Finally, the bilinear interpolation method is used to fill the pixels in the contour to obtain the candidate sonar image. The size slicing unit and the image filtering unit (33) are used to filter out sonar images that meet the preset conditions from the candidate sonar images, and classify and store them according to the category corresponding to the target object of the sonar image, so as to obtain real sonar images that can be directly used for network model training. The sonar image synthesis module (4) has a discrete cosine transform cyclic adversarial generative network (41), which has a loss function pre-set in it as described by the following formula. Used to provide training constraints during the process of training real sonar images and remote sensing images to generate adversarial sonar images; In the formula, G1 is the first generator, used to generate sonar image G1(x) based on remote sensing image x in remote sensing image set X, and G2 is the second generator, used to generate remote sensing image G2(y) based on sonar image y in real sonar image set Y. This is the global discriminator for G1, used to calculate the global loss of the sonar images generated by G1. This is the global discriminator for G2, used to calculate the global loss of the remote sensing image X generated by G2. This is the local discriminator for G1, used to calculate the local loss of the sonar image generated by G1. This is the local discriminator of G2, used to calculate the local loss of the remote sensing image X generated by G2. For G1 and Global combat losses, For G2 and Global combat losses, For G1 and Localized combat losses, For G2 and Localized combat losses, L c (G1, G2) represents the discrete cosine transform cycle consistency loss. λ1 is the weight of the local loss, and λ2 is the weight of the cycle consistency loss. The generator employs a network structure based on an attention module. The input remote sensing image is first processed through a reflectionPad layer, a 7*7 Conv layer, a Batch Normalization layer, and a ReLU activation function layer. Then, it passes through two downsampling modules to obtain a feature map of size 64*64 and 256 channels. This feature map is then used for feature extraction through a 3-layer residual network module and a 6-layer attention mechanism module. Next, it is converted into a feature map of size 256*256 and 64 channels through two upsampling layers. Finally, the generated sonar image is obtained through a reflectionPad layer, a 3*4 Conv layer, a Batch Normalization layer, and a Tanhactivation function layer.

2. The sonar image database augmentation device based on a recurrent adversarial generative network as described in claim 1, characterized in that, In the formula, Let Y be the expected distribution of the sonar image dataset. Let X be the expected distribution of the remote sensing image dataset.

3. The sonar image database augmentation device based on a recurrent adversarial generative network as described in claim 1, characterized in that, In the formula, ||*||2 represents L2 regularization. For a local part of G1(x), This is a partial view of a real sonar image. Let X be the expected distribution of the remote sensing image dataset.

4. The sonar image database augmentation device based on a recurrent adversarial generative network as described in claim 1, characterized in that, In the formula, Dct x (x), Dct y (y) represents the result of encoding x and y after the discrete cosine transform is changed, ||*||2 represents L2 regularization, G2(G1(x)) represents the reconstructed image of remote sensing image x, and G1(G2(y)) represents the reconstructed image of sonar image y. Let Y be the expected distribution of the sonar image dataset. Let X be the expected distribution of the remote sensing image dataset.

5. A method for augmenting a sonar image database based on a recurrent adversarial generative network, characterized in that, The sonar image database augmentation device based on a recurrent adversarial generative network as described in any one of claims 1-4 includes: Step 1: Acquire real sonar images; Step 2: Based on the remote sensing image and the actual sonar image, obtain a sonar image containing the target object of a preset size; Step 3: Enhance the quality of the sonar image output in Step 2, extract the contour information of the target object from the enhanced sonar image, and finally fill the pixels within the contour to obtain the candidate sonar image. Step 4: Select sonar images that meet the preset conditions from the candidate sonar images, and classify and store them according to the category corresponding to the target object in the sonar images to obtain real sonar images that can be directly used for network model training. Step 5: Use the Discrete Cosine Transform (DCT) cyclic adversarial generator network to generate a loss function pre-defined by the following formula. Training constraints are applied during the process of training real sonar images and remote sensing images to generate sonar images adversarially. In the formula, G1 is the first generator, used to generate sonar image G1(x) based on remote sensing image x in remote sensing image set X, and G2 is the second generator, used to generate remote sensing image G2(y) based on sonar image y in real sonar image set Y. This is the global discriminator for G1, used to calculate the global loss of the sonar images generated by G1. This is the global discriminator for G2, used to calculate the global loss of the remote sensing image X generated by G2. This is the local discriminator for G1, used to calculate the local loss of the sonar image generated by G1. This is the local discriminator of G2, used to calculate the local loss of the remote sensing image X generated by G2. For G1 and Global combat losses, For G2 and Global combat losses, For G1 and Localized combat losses, For G2 and Localized combat losses, L c (G1, G2) represents the discrete cosine transform cycle consistency loss. λ1 is the weight of the local loss, and λ2 is the weight of the cycle consistency loss.

6. The sonar image database augmentation method based on recurrent adversarial generative networks as described in claim 5, characterized in that, In the formula, Let Y be the expected distribution of the sonar image dataset. Let X be the expected distribution of the remote sensing image dataset.

7. The sonar image database augmentation method based on recurrent adversarial generative networks as described in claim 5, characterized in that, In the formula, ||*||2 represents L2 regularization. For a local part of G1(x), This is a partial view of a real sonar image. Let X be the expected distribution of the remote sensing image dataset.

8. The sonar image database augmentation method based on recurrent adversarial generative networks as described in claim 5, characterized in that, In the formula, Dct x (x), Dct y (y) represents the result of encoding x and y after the discrete cosine transform is changed, ||*||2 represents L2 regularization, G2(G1(x)) represents the reconstructed image of remote sensing image x, and G1(G2(y)) represents the reconstructed image of sonar image y. Let Y be the expected distribution of the sonar image dataset. Let X be the expected distribution of the remote sensing image dataset.

Images