A sonar image simulation method based on a generative adversarial network

Abstract

The application discloses a sonar image simulation method based on a generative adversarial network, and specifically comprises the following steps: S1: taking an original sonar image as a high-resolution image, preprocessing the original sonar image to obtain a low-resolution image, and combining the high-resolution image and the low-resolution image to form a data set; S2: creating a generative adversarial network model, wherein the generative adversarial network model comprises a sonar image generation network module and a sonar image discrimination network module, the sonar image generation network module generates a super-resolution image based on the low-resolution image in the data set, and the sonar image discrimination network module is used for outputting a discrimination result based on the generated super-resolution image and the high-resolution image in the data set; S3: training the generative adversarial network model based on the data set to obtain a trained generative adversarial network model; and S4: simulating a sonar image based on the trained generative adversarial network model. The constructed generative adversarial network model can improve the clarity of an existing sonar image and generate a sonar image with higher resolution.

Description

Technical Field

[0001] This invention relates to the field of sonar image simulation technology, and in particular to a sonar image simulation method based on generative adversarial networks. Background Technology

[0002] Research on image simulation began in the 1960s and has progressed through various stages, from the synthesis of simple line segments and regular shapes, to the synthesis of regular images, such as texture images and face images, and then to the synthesis of complex natural images, such as the image synthesis in the ImageNet dataset. As the amount of image data has gradually increased and the computing power of modern computers has improved, natural image simulation technology has matured, and the quality of simulated natural images has continuously improved. However, due to the complexity of the underwater environment, sonar images are relatively scarce, and sonar image simulation technology remains relatively lacking.

[0003] The purpose of underwater sonar image simulation is to study a method for generating high-quality underwater sonar images. Image generation is essentially probabilistic modeling of images, which can be summarized as the application of a generative model. Most generative models are based on maximum likelihood estimation, calculating appropriate and correct parameters for a selected model to maximize the likelihood function value of the data in the training dataset. Traditional sonar image simulation methods utilize computer simulation technology to model and simulate based on the sonar's imaging mechanism and image features. Therefore, both methods rely on model building and require specialized domain knowledge. When the scene containing the simulation object lacks a usable geometric model, it is difficult to generate sonar images using these traditional methods. Furthermore, the simulation effect is highly dependent on the accuracy of the model construction and the selection of parameters. The diverse structures, complex imaging processes, and clutter inherent in the simulation object itself make parameter optimization difficult, resulting in low-quality generated sonar images. Summary of the Invention

[0004] This invention provides a sonar image simulation method based on generative adversarial networks to overcome the problem that when performing image simulation using existing technologies, the simulation results are highly dependent on the accuracy of model construction and the selection of parameters, and the parameters of the simulation model are affected by the simulation object and are difficult to optimize, resulting in low quality of generated sonar images.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] A sonar image simulation method based on generative adversarial networks, the specific steps of which are as follows:

[0007] S1: The original sonar image is used as a high-resolution image. The original sonar image is preprocessed to obtain a low-resolution image. The high-resolution image and the low-resolution image are combined to form a dataset.

[0008] S2: Create a generative adversarial network model, which includes a sonar image generation network module and a sonar image discrimination network module. Based on the low-resolution images in the dataset, the sonar image generation network module generates super-resolution images. Based on the generated super-resolution images and the high-resolution images in the dataset, the sonar image discrimination network module outputs discrimination results.

[0009] S3: Train the generative adversarial network model based on the dataset to obtain the trained generative adversarial network model;

[0010] S4: Simulation of sonar images based on the trained generative adversarial network model.

[0011] Specifically, in S2, the sonar image generation network module includes a first input module, a feature extraction module, a feature learning module, and a first output module;

[0012] The first input module is used to extract the edge and texture features of the input low-resolution image, output a first feature map, and transmit the first feature map to the feature extraction module;

[0013] The feature extraction module is used to extract the edge and texture features of the first feature map and output the second feature map, and then transmit the second feature map to the feature learning module;

[0014] The feature learning module is used to extract the shape and overall structure features of the object from the second feature map, output the third feature map, and transmit the third feature map to the first output module.

[0015] The first output module is used to upsample the third feature map to output a super-resolution image, and transmit the generated super-resolution image to the sonar image discrimination network module;

[0016] The sonar image discrimination network module includes a second input module, several convolutional modules, and a second output module;

[0017] The second input module is used to extract the edge and texture features of the super-resolution image and the high-resolution image in the dataset, and outputs the fourth feature map and the fifth feature map respectively and transmits them to the convolution module;

[0018] The convolution module is used to extract the shape and overall structural features of the object from the fourth and fifth feature maps, output the sixth and seventh feature maps respectively, and transmit them to the second output module;

[0019] The second output module is used to calculate the Wasserstein distance between the sixth feature map and the seventh feature map.

[0020] Specifically, the first input module includes a first convolutional layer and a first PReLU activation function layer;

[0021] The feature extraction module includes a multi-scale convolution module, which includes a first branch, a second branch, a third branch, a fourth branch, and a fifth branch.

[0022] The first branch includes a 1×1 first branch convolutional layer and a first branch dilated convolutional layer, used to extract edge and texture features of the first feature map with a receptive field size of 3×3.

[0023] The second branch includes a 1×1 second branch convolutional layer, a second branch asymmetric convolutional layer, and a second branch dilated convolutional layer, used to extract edge and texture features of the first feature map with a receptive field size of 9×9.

[0024] The third branch includes a 1×1 third branch convolutional layer, a third branch asymmetric convolutional layer, and a third branch dilated convolutional layer, used to extract edge and texture features of the first feature map with a receptive field size of 9×9.

[0025] The fourth branch includes a 1×1 fourth branch convolutional layer, a fourth branch asymmetric convolutional layer, and a fourth branch dilated convolutional layer, used to extract edge and texture features of the first feature map with a receptive field size of 15×15.

[0026] The fifth branch concatenates the feature maps output by the above four branches along the channel dimension to obtain a feature map with twice the original number of channels. Then, a 1×1 fifth branch convolutional layer is used to adjust the feature map with twice the original number of channels to a feature map with one times the original number of channels. The feature map is then summed with the first feature map along the channel dimension through a skip connection. Finally, the second feature map is output through a ReLU activation function layer.

[0027] The feature learning module includes L densely connected residual blocks, where L≥2. Each residual block includes a second convolutional layer and a second PReLU activation function layer. The first residual block takes the second feature map as input, and the Lth residual block takes the feature map output by summing the second feature map with the feature maps output by the previous L-1 residual blocks as input and outputs the third feature map.

[0028] The first output module includes a second convolutional block, several upsampling modules, and a third convolutional block;

[0029] The second convolutional block includes a third convolutional layer and a first batch normalization layer. The third convolutional layer maps the third feature map to the high-resolution image in the dataset according to the rule that makes the texture, edge and context information of the third feature map correspond to the high-resolution image. The first batch normalization layer is used to output a fourth feature map after batch normalization of the output of the third convolutional layer and transmit it to the upsampling module.

[0030] Each of the upsampling modules includes a fourth convolutional layer, a sub-pixel convolutional layer, and a third PReLU activation function layer. The fourth convolutional layer is used to split the fourth feature map and output r. 2 A low-resolution feature map, the sub-pixel convolutional layer is based on r 2 Subpixel convolution is performed on each low-resolution feature map to generate a generator of size r. 2 The high-resolution feature map, where r is the magnification factor, is then passed through the third PReLU activation function layer and transmitted to the third convolutional block.

[0031] The third convolutional block includes a fifth convolutional layer, which is used to transform the abstract high-level features contained in the high-resolution feature map output by the upsampling module into the pixel representation of the final output image and output a super-resolution image. The abstract high-level features include shape and appearance features related to object categories.

[0032] The second input module includes a sixth convolutional layer and a fourth PReLU activation function layer. The sixth convolutional layer is used to extract the edge and texture features of the super-resolution image and the high-resolution image in the dataset, and then outputs the fourth feature map and the fifth feature map through the fourth PReLU activation function layer.

[0033] Each of the convolutional modules includes a seventh convolutional layer, a second batch normalization layer, and a first Leaky ReLU activation function layer. The seventh convolutional layer is used to extract the shape and overall structural features of the object in the fourth and fifth feature maps. It is then processed by batch normalization through the second batch normalization layer and outputs the sixth and seventh feature maps after passing through the first Leaky ReLU activation function layer.

[0034] The second output module includes a first fully connected layer, a second Leaky ReLU activation function layer, and a second fully connected layer connected in sequence. The first fully connected layer is used to output the feature vectors of the sixth feature map and the seventh feature map. The second Leaky ReLU activation function layer adds the feature vectors of the sixth feature map and the seventh feature map pixel by pixel and then outputs them. The second fully connected layer calculates the Wasserstein distance based on the feature vectors output by the second Leaky ReLU activation function layer, which is a real value, and outputs it.

[0035] Specifically, in S3, the specific training steps of the generative adversarial network model include:

[0036] S31: Initialize the sonar image generation network module and the sonar image discrimination network module, input the low-resolution images in the dataset into the sonar image generation network module, generate super-resolution images through the sonar image generation network module, train the sonar image discrimination network module based on the super-resolution images and the high-resolution images in the dataset, minimize the adversarial loss function of the sonar image discrimination network module, and update the parameters of the sonar image discrimination network module;

[0037] S32: Based on the updated parameters of the sonar image discrimination network module, train the sonar image generation network module, minimize the content loss function of the sonar image generation network module, and update the parameters of the sonar image generation network module;

[0038] S33: Repeat the training process. When the loss function composed of the adversarial loss function and the content loss function converges, the training ends and the trained generative adversarial network model is obtained.

[0039] Specifically, in S31, when the sonar image discrimination network module is trained based on the super-resolution image and the high-resolution image in the dataset, it determines the difference between the high-resolution image in the dataset and the generated super-resolution image distribution by judging the Wasserstein distance between them, and sets an adversarial loss function, which is expressed as:

[0040]

[0041] In the formula, For Wasserstein distance, For gradient penalty, D(I) HR ) represents the discrimination network score for high-resolution sonar images; D(G(I) represents the score for the discrimination network for high-resolution sonar images. LR )) represents the discriminant network score of the generated super-resolution image, λ GP The weights represent the gradient penalty coefficients, and E is the expectation operator. It is the discriminant network module for high-resolution image I HR The output of I HR The gradient, 1 is used to reference the target value of the gradient norm.

[0042] Specifically, in S32, based on the updated parameters of the sonar image discrimination network module, the sonar image generation network module is trained. Feature extraction is performed on the high-resolution images in the dataset and the generated super-resolution images using a pre-trained VGG19 network. The content loss between the generated super-resolution image and the high-resolution image in the dataset is calculated, and the content loss function is set as follows:

[0043]

[0044] In the formula, G(I) LR ) is a super-resolution image generated by a generative network, I LR and I HR W represents low-resolution and high-resolution images, respectively. i,j and H i,j Φ represents the size of the feature map of the j-th convolutional layer before max pooling in the i-th layer of the VGG19 network, Φ represents the feature extraction function in the VGG19 network, and x and y represent the coordinate positions on the feature map.

[0045] Specifically, in S33, training ends when the loss function composed of the adversarial loss function and the content loss function converges. The loss function is expressed as:

[0046]

[0047] Beneficial Effects: This invention constructs a generative adversarial network model for sonar image simulation. Compared to traditional methods, it is not dependent on a dataset, and can generate high-quality images even when the dataset is limited, thus expanding the existing sonar image dataset. Furthermore, compared to traditional methods, it uses deep learning, reducing the reliance on manual parameter adjustments. Simultaneously, it improves the clarity of existing sonar images, enabling the generation of higher-resolution sonar images. It solves the problem that in image simulation using existing technologies, the simulation results are highly dependent on the accuracy of the model construction and the selection of parameters, while the parameters of the simulation model are affected by the simulation object and are difficult to optimize, resulting in low-quality generated sonar images. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a schematic diagram of the structure of the generative adversarial network model in this invention;

[0050] Figure 2 This is a schematic diagram of low-resolution images in the dataset in an embodiment of the present invention;

[0051] Figure 3 This is a schematic diagram illustrating the super-resolution images generated from low-resolution images in the dataset in an embodiment of the present invention after processing by a generative adversarial network model. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] GAN is a powerful data generator. With future improvements in network architecture and algorithms, GAN is expected to generate higher quality images, music, videos, and text, injecting new vitality into the fields of natural language processing and computer vision. GAN is still an area of ​​continuous development. How to increase the diversity of generated samples and explore more effective evaluation methods are important challenges facing GAN. Therefore, how to use deep learning generation algorithms to generate clearer sonar image samples that can be used directly from raw sonar images is an urgent problem to be solved.

[0054] This embodiment provides a sonar image simulation method based on generative adversarial networks, and the specific steps are as follows:

[0055] S1: The original sonar image is used as a high-resolution image. The original sonar image is preprocessed to obtain a low-resolution image. The high-resolution image and the low-resolution image are combined to form a dataset.

[0056] S11: Obtain the sample image dataset. The dataset uses the acoustic competition dataset from the underwater robot competition. Process the dataset into a .mat file format. The .mat file contains three variables: x, y, and digitStruct. Read the BMP image and save it to the x variable. Save the label information and position information to the y variable and the digitStruct variable, respectively. Perform bicubic interpolation on the sample images in the dataset as high-resolution images to obtain low-resolution images. Combine the low-resolution image and high-resolution image datasets. In this embodiment, the resolution of the high-resolution image is 1024×1024, and the resolution of the low-resolution image is 500×500.

[0057] S2: Create a generative adversarial network model, which includes a sonar image generation network module and a sonar image discrimination network module. Based on the low-resolution images in the dataset, the sonar image generation network module generates super-resolution images. Based on the generated super-resolution images and the high-resolution images in the dataset, the sonar image discrimination network module outputs discrimination results.

[0058] Specifically, in S2, such as Figure 1 As shown, the sonar image generation network module includes a first input module, a feature extraction module, a feature learning module, and a first output module;

[0059] The first input module is used to extract the edge and texture features of the input low-resolution image, output a first feature map, and transmit the first feature map to the feature extraction module;

[0060] Specifically, the first input module includes a 9×9 first convolutional layer and a first PReLU activation function layer;

[0061] The feature extraction module is used to extract the edge and texture features of the first feature map and output the second feature map, and then transmit the second feature map to the feature learning module;

[0062] Specifically, the feature extraction module includes a multi-scale convolution module, which includes a first branch, a second branch, a third branch, a fourth branch, and a fifth branch. The first branch includes a 1×1 first branch convolutional layer and a 3×3 first branch dilated convolutional layer. The dilation rate of the dilated convolutional layer is 1, which is used to extract edge and texture features of the first feature map with a receptive field size of 3×3.

[0063] The second branch includes a 1×1 second branch convolutional layer, a 1×3 second branch asymmetric convolutional layer, and a 3×3 second branch dilated convolutional layer. The dilation rate of the dilated convolutional layer is 3, which is used to extract edge and texture features of the first feature map with a receptive field size of 9×9.

[0064] The third branch includes a 1×1 third branch convolutional layer, a 3×1 third branch asymmetric convolutional layer, and a 3×3 third branch dilated convolutional layer. The dilation rate of the dilated convolutional layer is 3, which is used to extract edge and texture features of the first feature map with a receptive field size of 9×9.

[0065] The fourth branch includes a 1×1 fourth branch convolutional layer, two 1×3 fourth branch asymmetric convolutional layers, and a 3×3 fourth branch dilated convolutional layer. The dilated convolutional layer has an expansion rate of 5 and is used to extract edge and texture features of the first feature map with a receptive field size of 15×15.

[0066] The fifth branch concatenates the feature maps output by the above four branches along the channel dimension to obtain a feature map with twice the original number of channels. Then, the 1×1 convolutional layer in the fifth branch adjusts the feature map with twice the original number of channels to a feature map with one times the original number of channels. The feature map is then summed with the first feature map along the channel dimension through a skip connection, and finally output as a second feature map through a ReLU activation function layer.

[0067] Specifically, this embodiment introduces a multi-scale convolution module, which can better preserve the detailed information of small objects in large images and ensure the quality of the generated image.

[0068] The feature learning module is used to extract the shape and overall structure features of the object from the second feature map, output the third feature map, and transmit the third feature map to the first output module.

[0069] Specifically, the feature learning module includes L densely connected residual blocks, where L≥2. Each residual block includes a second convolutional layer and a second PReLU activation function layer. The first residual block takes the second feature map as input. The second residual block takes the feature map output by summing the second feature map with the output feature map of the first residual block as input. The Lth residual block takes the feature map output by summing the second feature map with the feature maps output by the previous L-1 residual blocks as input and outputs a third feature map.

[0070] Specifically, the feature learning module in this embodiment includes 16 densely connected residual blocks. Each residual block includes a 3×3 convolutional layer, a second PReLU activation function layer, and a 3×3 convolutional layer connected in sequence. In this embodiment, the residual block structure is modified by removing the BN layer in the residual block, which reduces the introduction of noise, increases the feature reuse rate, and reduces the resource consumption of a single residual block. Furthermore, the output of the Lth residual block is batch normalized by the first BN layer after processing by the convolutional layer, which can stretch the contrast of the image and prevent the model from overfitting.

[0071] The first output module is used to upsample the third feature map to output a super-resolution image, and transmit the generated super-resolution image to the sonar image discrimination network module;

[0072] Specifically, the first output module includes a second convolutional block, several upsampling modules, and a third convolutional block;

[0073] The second convolutional block includes a 3×3 third convolutional layer and a first batch normalization (BN) layer. The third convolutional layer maps the third feature map to the high-resolution image in the dataset according to the rule that the texture, edge, and context information of the third feature map correspond to the high-resolution image, thereby ensuring that the texture, edge, and context information of the third feature map correspond to the high-resolution image. The first BN layer is used to perform batch normalization processing on the output of the third convolutional layer and output a fourth feature map, which is then transmitted to the upsampling module.

[0074] Each of the upsampling modules includes a 3×3 fourth convolutional layer, two sub-pixel convolutional layers, and a third PReLU activation function layer. The fourth convolutional layer is used to split the fourth feature map and output r. 2 A low-resolution feature map, the sub-pixel convolutional layer is based on r 2 Subpixel convolution is performed on each low-resolution feature map to generate a generator of size r. 2 The high-resolution feature map, where r is the magnification factor, is then passed through the third PReLU activation function layer and transmitted to the third convolutional block. The third PReLU activation function layer processes the input pixel by pixel, introducing a negative slope when the input is less than zero to avoid the generation of dead neurons. In this embodiment, preferably, it includes two upsampling modules.

[0075] The third convolutional block includes three 9×9 fifth convolutional layers, which are used to transform the abstract high-level features contained in the high-resolution feature map into the pixel representation of the final output image and output a 3-channel super-resolution image. The abstract high-level features include shape and appearance features related to object categories.

[0076] In this embodiment, preferably, r = 2, and 2 is generated through the fourth convolutional layer. 2 A low-resolution feature map with 2 channels, where the feature channels of a pixel are split into 2... 2 The system generates a high-resolution feature map by rearranging the sub-channels sequentially. The high-resolution feature map is then processed by three 9×9 fifth convolutional layers to output a 3-channel super-resolution image, which is four times the size of the original image.

[0077] The sonar image discrimination network module includes a second input module, several convolutional modules, and a second output module;

[0078] The second input module is used to extract the edge and texture features of the super-resolution image and the high-resolution image in the dataset, and outputs the fourth feature map and the fifth feature map respectively and transmits them to the convolution module;

[0079] Specifically, the second input module includes a 3×3 sixth convolutional layer and a fourth PReLU activation function layer. The sixth convolutional layer is used to extract edge and texture features of the super-resolution image and the high-resolution image in the dataset. The fourth PReLU activation function layer then outputs a fourth feature map and a fifth feature map. The fourth PReLU activation function layer is used to introduce a negative slope, enabling the network to adapt more flexibly to different types of features.

[0080] The convolution module is used to extract the shape and overall structural features of the object from the fourth and fifth feature maps, output the sixth and seventh feature maps respectively, and transmit them to the second output module;

[0081] Specifically, each convolutional module includes a 3×3 seventh convolutional layer, a second BN layer, and a first LeakyReLU activation function layer. The seventh convolutional layer is used to extract the shape and overall structural features of the object in the fourth and fifth feature maps. Batch normalization is performed through the second BN layer. After passing through the first LeakyReLU activation function layer, the sixth and seventh feature maps are output. Preferably, this embodiment includes 7 convolutional modules.

[0082] The second output module is used to calculate the Wasserstein distance between the sixth feature map and the seventh feature map.

[0083] The second output module includes a first fully connected layer, a second Leaky ReLU activation function layer, and a second fully connected layer connected in sequence. The first fully connected layer outputs a 1×1024 feature vector from the sixth and seventh feature maps. The second Leaky ReLU activation function layer sums the feature vectors of the sixth and seventh feature maps pixel by pixel and outputs a 1×1024 feature vector. The second Leaky ReLU activation function introduces non-linearity and utilizes its good preservation properties in the negative range, allowing backpropagation even for negative input values. The second fully connected layer multiplies the 1×1024 feature vector output from the second Leaky ReLU activation function layer with the weights obtained during training to obtain the Wasserstein distance, i.e., a real value, and outputs it. In this embodiment, the weights are initialized to 0.01, and subsequent weights are learned through network training.

[0084] Traditional generative adversarial networks (GANs) use a discriminator network module that inputs the output into a sigmoid activation function to obtain a probability value. This probability value is used to determine whether the input image is a real sample or a fake sample generated by the generator. For some input values, the derivative of the sigmoid activation function will be close to 0, causing gradient vanishing and hindering gradient propagation. Therefore, this embodiment does not use the sigmoid activation function. Instead, the discriminator outputs the difference between the real image and the generated image, i.e., the Wasserstein distance between the two image distributions.

[0085] S3: Train the generative adversarial network model based on the dataset to obtain the trained generative adversarial network model;

[0086] S4: Simulation of sonar images based on the trained generative adversarial network model.

[0087] Specifically, in S3, the specific training steps of the generative adversarial network model include:

[0088] S31: Initialize the sonar image generation network module and the sonar image discrimination network module, input the low-resolution images in the dataset into the sonar image generation network module, output the super-resolution images through the sonar image generation network module, train the sonar image discrimination network module based on the super-resolution images and the high-resolution images in the dataset, minimize the adversarial loss function of the sonar image discrimination network module, and update the parameters of the sonar image discrimination network module;

[0089] Specifically, in S31, when the sonar image discrimination network module is trained based on the super-resolution image and the high-resolution image in the dataset, it determines the difference between the high-resolution image in the dataset and the generated super-resolution image distribution by judging the Wasserstein distance between them, and sets an adversarial loss function, which is expressed as:

[0090]

[0091] In the formula, For Wasserstein distance, For gradient penalty, D(I) HR ) represents the discrimination network score for high-resolution sonar images; D(G(I) represents the score for the discrimination network for high-resolution sonar images. LR )) represents the discriminant network score of the generated super-resolution image, λ GP The weights represent the gradient penalty coefficients, and E is the expectation operator. It is the discriminant network module for high-resolution image I HR The output of I HRThe gradient, 1 is used to reference the target value of the gradient norm.

[0092] S32: Based on the updated parameters of the sonar image discrimination network module, train the sonar image generation network module, minimize the content loss function of the sonar image generation network module, and update the parameters of the sonar image generation network module;

[0093] Specifically, in S32, based on the updated parameters of the sonar image discrimination network module, the sonar image generation network module is trained. Feature extraction is performed on the high-resolution images in the dataset and the generated super-resolution images using a pre-trained VGG19 network. The content loss between the generated super-resolution image and the high-resolution image in the dataset is calculated, and the content loss function is set as follows:

[0094]

[0095] In the formula, G(I) LR ) is a super-resolution image generated by a generative network, I LR and I HR W represents low-resolution and high-resolution images, respectively. i,j and H i,j Φ represents the size of the feature map of the j-th convolutional layer before max pooling in the i-th layer of the VGG19 network, Φ represents the feature extraction function in the VGG19 network, and x and y represent the coordinate positions on the feature map.

[0096] S33: Repeat the training process. When the loss function composed of the adversarial loss function and the content loss function converges, the training ends and the trained generative adversarial network model is obtained.

[0097] Specifically, in this embodiment, Wasserstein distance and gradient penalty are used to improve the loss function as adversarial loss, ensuring stable training performance and reducing model crashes and oscillations during training.

[0098] Specifically, in S33, training ends when the loss function composed of the adversarial loss function and the content loss function converges. The loss function is expressed as:

[0099]

[0100] Minimizing the loss function is used to train the generative network module to generate high-quality super-resolution images. The adversarial loss function encourages the generative network module to generate realistic images, while the content loss function ensures that the generated images retain important structural and semantic features. The adversarial and content loss functions are weighted and summed to balance the generative network module's demands on image quality and visual detail. If more attention is paid to the visual detail of the generated images, the weight of the content loss is increased; if more attention is paid to the realism and fidelity of the generated images, the weight of the adversarial loss is increased.

[0101] In this embodiment, the learning rate is set to r = 0.002, the batch size to m = 4, the number of iterations to epochs = 100, and the number of iterations n for the discriminant network is set to n for each iteration of the generator network. critic =5, gradient penalty factor λ GP =10; for example Figure 2 The image shown is a low-resolution image from the dataset in this embodiment. Figure 3 This is a high-resolution image generated by the generative adversarial network model in this embodiment. As can be seen from the image, the sonar image generated by this method is improved in terms of detail, image quality and clarity compared to the original image, and can be applied.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A sonar image simulation method based on generative adversarial networks, characterized in that, The specific steps are as follows: S1: The original sonar image is used as a high-resolution image. The original sonar image is preprocessed to obtain a low-resolution image. The high-resolution image and the low-resolution image are combined to form a dataset. S2: Create a generative adversarial network model, which includes a sonar image generation network module and a sonar image discrimination network module. Based on the low-resolution images in the dataset, the sonar image generation network module generates super-resolution images. Based on the generated super-resolution images and the high-resolution images in the dataset, the sonar image discrimination network module outputs discrimination results. S3: Train the generative adversarial network model based on the dataset to obtain the trained generative adversarial network model; S4: Simulation of sonar images based on the trained generative adversarial network model; The sonar image generation network module includes a first input module, a feature extraction module, a feature learning module, and a first output module; The first input module includes a first convolutional layer and a first PReLU activation function layer; the first input module is used to extract edge and texture features of the input low-resolution image, output a first feature map, and transmit the first feature map to the feature extraction module; The feature extraction module is used to extract the edge and texture features of the first feature map and output the second feature map, and then transmit the second feature map to the feature learning module; The feature extraction module includes a multi-scale convolution module, which includes a first branch, a second branch, a third branch, a fourth branch, and a fifth branch. The first branch includes a 1×1 first branch convolutional layer and a first branch dilated convolutional layer, used to extract edge and texture features of the first feature map with a receptive field size of 3×3. The second branch includes a 1×1 second branch convolutional layer, a second branch asymmetric convolutional layer, and a second branch dilated convolutional layer, used to extract edge and texture features of the first feature map with a receptive field size of 9×9. The third branch includes a 1×1 third branch convolutional layer, a third branch asymmetric convolutional layer, and a third branch dilated convolutional layer, used to extract edge and texture features of the first feature map with a receptive field size of 9×9. The fourth branch includes a 1×1 fourth branch convolutional layer, a fourth branch asymmetric convolutional layer, and a fourth branch dilated convolutional layer, used to extract edge and texture features of the first feature map with a receptive field size of 15×15. The fifth branch concatenates the feature maps output by the above four branches along the channel dimension to obtain a feature map with twice the original number of channels. Then, a 1×1 fifth branch convolutional layer is used to adjust the feature map with twice the original number of channels to a feature map with one times the original number of channels. The feature map is then summed with the first feature map along the channel dimension through a skip connection. Finally, the second feature map is output through a ReLU activation function layer. The feature learning module is used to extract the shape and overall structure features of the object from the second feature map, output the third feature map, and transmit the third feature map to the first output module. The feature learning module includes L densely connected residual blocks, where L≥2. Each residual block includes a second convolutional layer and a second PReLU activation function layer. The first residual block takes the second feature map as input, and the Lth residual block takes the feature map output by summing the second feature map with the feature maps output by the previous L-1 residual blocks as input and outputs the third feature map. The first output module includes a second convolutional block, several upsampling modules, and a third convolutional block; the first output module is used to upsample the third feature map to output a super-resolution image, and transmit the generated super-resolution image to the sonar image discrimination network module; The second convolutional block includes a third convolutional layer and a first batch normalization layer. The third convolutional layer maps the third feature map to the high-resolution image in the dataset according to the rule that makes the texture, edge and context information of the third feature map correspond to the high-resolution image. The first batch normalization layer is used to output a fourth feature map after batch normalization of the output of the third convolutional layer and transmit it to the upsampling module. Each of the upsampling modules includes a fourth convolutional layer, a sub-pixel convolutional layer, and a third PReLU activation function layer. The fourth convolutional layer is used to split the fourth feature map and output r. 2 A low-resolution feature map, the sub-pixel convolutional layer is based on r 2 Subpixel convolution is performed on each low-resolution feature map to generate a generator of size r. 2 The high-resolution feature map, where r is the magnification factor, is then passed through the third PReLU activation function layer and transmitted to the third convolutional block. The third convolutional block includes a fifth convolutional layer, which is used to transform the abstract high-level features contained in the high-resolution feature map output by the upsampling module into the pixel representation of the final output image and output a super-resolution image. The abstract high-level features include shape and appearance features related to object categories. The sonar image discrimination network module includes a second input module, several convolutional modules, and a second output module; The second input module is used to extract the edge and texture features of the super-resolution image and the high-resolution image in the dataset, and outputs the fourth feature map and the fifth feature map respectively and transmits them to the convolution module; The second input module includes a sixth convolutional layer and a fourth PReLU activation function layer. The sixth convolutional layer is used to extract the edge and texture features of the super-resolution image and the high-resolution image in the dataset, and then outputs the fourth feature map and the fifth feature map through the fourth PReLU activation function layer. Each convolutional module includes a seventh convolutional layer, a second batch normalization layer, and a first Leaky ReLU activation function layer. The seventh convolutional layer is used to extract the shape and overall object structure features of the fourth and fifth feature maps, performs batch normalization processing through the second batch normalization layer, and outputs the sixth and seventh feature maps after passing through the first Leaky ReLU activation function layer. The convolutional module is used to extract the shape and overall object structure features of the fourth and fifth feature maps, output the sixth and seventh feature maps respectively, and transmit them to the second output module. The second output module is used to calculate the Wasserstein distance between the sixth feature map and the seventh feature map; The second output module includes a first fully connected layer, a second Leaky ReLU activation function layer, and a second fully connected layer connected in sequence. The first fully connected layer is used to output the feature vectors of the sixth feature map and the seventh feature map. The second Leaky ReLU activation function layer adds the feature vectors of the sixth feature map and the seventh feature map pixel by pixel and then outputs them. The second fully connected layer calculates the Wasserstein distance based on the feature vectors output by the second Leaky ReLU activation function layer, which is a real value, and outputs it.

2. The sonar image simulation method based on generative adversarial networks according to claim 1, characterized in that, In S3, the specific training steps of the generative adversarial network model include: S31: Initialize the sonar image generation network module and the sonar image discrimination network module, input the low-resolution images in the dataset into the sonar image generation network module, generate super-resolution images through the sonar image generation network module, train the sonar image discrimination network module based on the super-resolution images and the high-resolution images in the dataset, minimize the adversarial loss function of the sonar image discrimination network module, and update the parameters of the sonar image discrimination network module; S32: Based on the updated parameters of the sonar image discrimination network module, train the sonar image generation network module, minimize the content loss function of the sonar image generation network module, and update the parameters of the sonar image generation network module; S33: Repeat the training process. When the loss function composed of the adversarial loss function and the content loss function converges, the training ends and the trained generative adversarial network model is obtained.

3. The sonar image simulation method based on generative adversarial networks according to claim 2, characterized in that, In S31, when the sonar image discrimination network module is trained based on the super-resolution image and the high-resolution image in the dataset, it determines the difference between the high-resolution image in the dataset and the generated super-resolution image distribution by judging the Wasserstein distance between them, and sets an adversarial loss function, which is expressed as: ； In the formula, For Wasserstein distance, For gradient penalty, D ( I HR ) represents the discrimination network score of a high-resolution sonar image; D ( G ( I LR )) represents the discrimination network score of the generated super-resolution image. The weight coefficients representing the gradient penalty. E It is the expected value operator. D ( I HR () is the discrimination network module for high-resolution images I HR The output about I HR The gradient, 1 is used to reference the target value of the gradient norm.

4. The sonar image simulation method based on generative adversarial networks according to claim 3, characterized in that, In S32, based on the updated parameters of the sonar image discrimination network module, the sonar image generation network module is trained. Feature extraction is performed on the high-resolution images in the dataset and the generated super-resolution images using a pre-trained VGG19 network. The content loss between the generated super-resolution image and the high-resolution image in the dataset is calculated, and the content loss function is set as follows: , In the formula, It is a super-resolution image generated by a generative network. I LR and I HR These represent low-resolution and high-resolution images, respectively. and The first in the VGG19 network i The first layer before max pooling j The size of the feature maps of each convolutional layer This represents the feature extraction function in the VGG19 network. x and y This indicates the coordinate position on the feature map.

5. The sonar image simulation method based on generative adversarial networks according to claim 4, characterized in that, In S33, training ends when the loss function composed of the adversarial loss function and the content loss function converges. The loss function is expressed as: 。

Images