Sonar image super-resolution reconstruction method and system based on generative adversarial network

Abstract

The invention discloses a sonar image super-resolution reconstruction method and system based on a generative adversarial network, and the method comprises the steps: constructing the generative adversarial network which comprises a generator and a discriminator; acquiring a first sonar image data set, and preprocessing the first sonar image data set to obtain a second sonar image data set; inputting the second sonar image data set into the generative adversarial network for training, and performing parameter updating on the generative adversarial network through a preset total loss function to obtain a trained super-resolution reconstruction model; and obtaining a sonar image to be detected, and inputting the sonar image to be detected into the super-resolution reconstruction model to obtain a super-resolution reconstruction result. The method can accurately capture the detail features of the underwater sonar image, further recovers the real texture details of the underwater sonar image, and can be widely applied to the technical field of sonar image reconstruction.

Description

Technical Field

[0001] The present application relates to the technical field of sonar image reconstruction, and in particular to a sonar image super-resolution reconstruction method and system based on a generative adversarial network. Background Art

[0002] At present, with the increasing importance of marine resource development, sonar is an indispensable sensor for underwater detection, providing rich underwater observation data for the rational development and utilization of marine resources. However, due to the principle of underwater acoustics, sound waves are affected by a variety of environmental factors during propagation, such as water depth, flow rate, temperature, etc., sonar images often have distortion and low resolution problems, which affects the visual effect of sonar images and is difficult to meet application requirements. There are two main ways to enhance the visual effect of sonar images: the first method is to enhance the visual effect from the bottom layer by improving the performance of sonar equipment, but this method has very high requirements on the manufacturing level and is very expensive; the other is to start from the algorithm level and process the collected sonar images to enhance their visual effects.

[0003] Super-resolution (SR), as a method to improve image quality technology, aims to recover realistic high-resolution (HR) images from their low-resolution (LR) image counterparts. High-resolution (HR) images provide improved reconstruction details of scenes and constituent objects, which are crucial for many real-world fields, such as remote sensing images, medical applications, face recognition, and sonar images. However, due to the complex underwater background and the large variation in texture and structural information of sonar images, existing image super-resolution algorithms often produce blurred underwater sonar images with limited ability to recover details and textures. In addition, the noise in sonar images is very complex, such as speckle noise. This makes it difficult for existing super-resolution methods to generalize to find the mapping relationship between high-resolution (HR) sonar images and low-resolution (LR) sonar images. Summary of the invention

[0004] In order to solve the above technical problems, the purpose of the present invention is to provide a sonar image super-resolution reconstruction method and system based on a generative adversarial network, which can restore the real texture details of underwater sonar images.

[0005] To achieve the above object, an embodiment of the present application proposes a sonar image super-resolution reconstruction method based on a generative adversarial network, comprising the following steps:

[0006] Constructing a generative adversarial network, wherein the generative adversarial network includes a generator and a discriminator;

[0007] Acquire a first sonar image data set, and preprocess the first sonar image data set to obtain a second sonar image data set;

[0008] Inputting the second sonar image data set into the generative adversarial network for training, and updating the parameters of the generative adversarial network through a preset total loss function to obtain a trained super-resolution reconstruction model;

[0009] The sonar image to be measured is acquired, and the sonar image to be measured is input into the super-resolution reconstruction model to obtain a super-resolution reconstruction result.

[0010] In some embodiments, constructing a generative adversarial network specifically includes:

[0011] Constructing the generator, which includes a shallow feature extraction module, a deep feature extraction module and an image reconstruction module;

[0012] The discriminator is constructed, and the discriminator includes an encoder and a decoder.

[0013] In some embodiments, the shallow feature extraction module includes a convolution layer, the deep feature extraction module includes a residual self-calibration attention group, a convolution layer and a self-calibration enhanced attention module, and the image reconstruction module includes an upsampler and a convolution layer.

[0014] In some embodiments, the residual self-calibration attention group includes a residual block, a convolutional layer, and the self-calibration enhanced attention module, and the self-calibration enhanced attention module includes a self-calibration enhanced attention block and an enhanced attention block.

[0015] In some embodiments, the acquiring of the first sonar image dataset and preprocessing of the first sonar image dataset to obtain the second sonar image dataset specifically includes:

[0016] Acquire the first sonar image dataset, where the first sonar image dataset includes a plurality of first sonar image samples;

[0017] Performing data enhancement on each of the first sonar image samples to obtain a plurality of second sonar image samples;

[0018] performing unsharp masking on each of the second sonar image samples to obtain a plurality of third sonar image samples;

[0019] Denoising each of the third sonar image samples to obtain a plurality of fourth sonar image samples;

[0020] Down-sampling each of the fourth sonar image samples is performed to obtain the second sonar image data set.

[0021] In some embodiments, the second sonar image dataset includes a first-resolution sonar image and a second-resolution sonar image, and the second sonar image dataset is input into the generative adversarial network for training, and the parameters of the generative adversarial network are updated by a preset total loss function to obtain a trained super-resolution reconstruction model, specifically including:

[0022] Inputting the first resolution sonar image into the generator to obtain a reconstruction result;

[0023] Inputting the second resolution sonar image into the discriminator, and then comparing the second resolution sonar image with the reconstruction result through the discriminator to obtain discrimination information;

[0024] Updating parameters of the generative adversarial network according to the discrimination information and the total loss function;

[0025] When the number of iterations reaches a preset first threshold or the test accuracy reaches a preset second threshold, the training is stopped to obtain the trained super-resolution reconstruction model.

[0026] In some embodiments, the super-resolution reconstruction method further includes the step of presetting the total loss function, wherein the setting of the total loss function specifically includes:

[0027] Use L1 loss function as pixel loss function;

[0028] Constructing a perceptual loss function based on the feature map generated by the generative adversarial network and the L1 loss function;

[0029] Construct an adversarial loss function based on the generator loss and the discriminator loss;

[0030] The pixel loss function, the perceptual loss function and the adversarial loss function are weightedly summed to obtain the total loss function.

[0031] To achieve the above object, another aspect of the embodiment of the present application proposes a sonar image super-resolution reconstruction system based on a generative adversarial network, comprising:

[0032] A network construction module, used to construct a generative adversarial network, wherein the generative adversarial network includes a generator and a discriminator;

[0033] A data set acquisition module, used to acquire a first sonar image data set, and preprocess the first sonar image data set to obtain a second sonar image data set;

[0034] A model training module, used for inputting the second sonar image data set into the generative adversarial network for training, and updating the parameters of the generative adversarial network through a preset total loss function to obtain a trained super-resolution reconstruction model;

[0035] The super-resolution reconstruction module is used to obtain the sonar image to be measured, input the sonar image to be measured into the super-resolution reconstruction model, and obtain the super-resolution reconstruction result.

[0036] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes an electronic device, which includes a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the sonar image super-resolution reconstruction method based on the generative adversarial network as described above is realized.

[0037] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium for computer-readable storage, and the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the sonar image super-resolution reconstruction method based on the generative adversarial network as described above.

[0038] The beneficial effects of the present invention are as follows: the method and system for super-resolution reconstruction of sonar images based on a generative adversarial network of the present invention first constructs a generative adversarial network, which includes a generator and a discriminator, and then obtains a first sonar image data set, pre-processes the first sonar image data set, obtains a second sonar image data set, then inputs the second sonar image data set into the generative adversarial network for training, and updates the parameters of the generative adversarial network through a preset total loss function to obtain a trained super-resolution reconstruction model, and finally obtains a sonar image to be tested, inputs the sonar image to be tested into the super-resolution reconstruction model, and obtains a super-resolution reconstruction result. The present invention uses a super-resolution reconstruction model based on a generative adversarial network to perform super-resolution reconstruction on underwater sonar images, which can accurately capture the detailed features of underwater sonar images and further restore the real texture details of underwater sonar images. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solution in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solution of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 A flowchart of a sonar image super-resolution reconstruction method based on a generative adversarial network provided by an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of the structure of a generator provided by an embodiment of the present invention;

[0042] Figure 3 A schematic diagram of the structure of a residual self-calibration attention group provided by an embodiment of the present invention;

[0043] Figure 4 A schematic diagram of the structure of a self-calibration enhanced attention module provided in an embodiment of the present invention;

[0044] Figure 5 A schematic diagram of the structure of a discriminator provided in an embodiment of the present invention;

[0045] Figure 6 A schematic diagram of the structure of a sonar image super-resolution reconstruction system based on a generative adversarial network provided by an embodiment of the present invention;

[0046] Figure 7 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0048] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0049] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0050] At present, with the increasing importance of marine resource development, sonar is an indispensable sensor for underwater detection, providing rich underwater observation data for the rational development and utilization of marine resources. However, due to the principle of underwater acoustics, sound waves are affected by a variety of environmental factors during propagation, such as water depth, flow rate, temperature, etc., sonar images often have distortion and low resolution problems, which affects the visual effect of sonar images and is difficult to meet application requirements. There are two main ways to enhance the visual effect of sonar images: the first method is to enhance the visual effect from the bottom layer by improving the performance of sonar equipment, but this method has very high requirements on the manufacturing level and is very expensive; the other is to start from the algorithm level and process the collected sonar images to enhance their visual effects.

[0051] Super-resolution (SR), as a method to improve image quality technology, aims to recover realistic high-resolution (HR) images from their low-resolution (LR) counterparts. High-resolution (HR) images provide improved reconstruction details of scenes and constituent objects, which are crucial for many real-world fields, such as remote sensing images, medical applications, face recognition, and sonar images. However, due to the complex underwater background and the large variation in texture and structural information of sonar images, existing image super-resolution algorithms often produce blurred underwater sonar images with limited ability to recover details and textures. In addition, the noise in sonar images is very complex, such as speckle noise. This makes it difficult for existing super-resolution methods to generalize to find the mapping relationship between high-resolution (HR) sonar images and low-resolution (LR) sonar images.

[0052] To this end, an embodiment of the present invention proposes a sonar image super-resolution reconstruction method based on a generative adversarial network, a sonar image super-resolution reconstruction method and system based on a generative adversarial network, firstly constructing a generative adversarial network, the generative adversarial network including a generator and a discriminator, and then obtaining a first sonar image data set, preprocessing the first sonar image data set to obtain a second sonar image data set, then inputting the second sonar image data set into the generative adversarial network for training, and updating the parameters of the generative adversarial network through a preset total loss function to obtain a trained super-resolution reconstruction model, and finally obtaining a sonar image to be tested, inputting the sonar image to be tested into the super-resolution reconstruction model to obtain a super-resolution reconstruction result. The present invention uses a super-resolution reconstruction model based on a generative adversarial network to perform super-resolution reconstruction on underwater sonar images, which can accurately capture the detailed features of underwater sonar images and further restore the real texture details of underwater sonar images.

[0053] Reference Figure 1 , Figure 1 A flowchart of a sonar image super-resolution reconstruction method based on a generative adversarial network is provided in an embodiment of the present invention. The embodiment of the present invention proposes a sonar image super-resolution reconstruction method based on a generative adversarial network, and the method includes steps S101 to S104:

[0054] S101. Construct a generative adversarial network, which includes a generator and a discriminator.

[0055] Specifically, the generative adversarial network consists of a generator and a Unet-based discriminator.

[0056] As an optional implementation, the step of constructing a generative adversarial network can be specifically divided into the following steps S1011 and S1012:

[0057] S1011, constructing a generator, the generator comprising a shallow feature extraction module, a deep feature extraction module and an image reconstruction module;

[0058] Further as an optional implementation, the shallow feature extraction module includes a convolution layer, the deep feature extraction module includes a residual self-calibration attention group, a convolution layer and a self-calibration enhanced attention module, and the image reconstruction module includes an upsampler and a convolution layer.

[0059] Specifically, Figure 2 The figure is a schematic diagram of the generator structure. The generator consists of three parts: shallow feature extraction module, deep feature extraction module and image reconstruction module. Among them, the shallow feature extraction module uses a convolutional layer with a kernel size of 3*3 and a step size of 1. The shallow feature extraction module is responsible for widening the number of channels of the input sonar image and extracting shallow features of the sonar image.

[0060] Furthermore, the image reconstruction module consists of an upsampler and a convolutional layer. The upsampler consists of a convolutional layer and a pixel reshuffle, which varies with the scale factor. It is used to upgrade the feature map from a low-resolution space to a high-resolution space, and the last convolutional layer with a kernel size of 3*3 and a stride of 1 produces a super-resolution image.

[0061] Further as an optional implementation, the residual self-calibration attention group includes a residual block, a convolutional layer, and a self-calibration enhanced attention module, and the self-calibration enhanced attention module includes a self-calibration enhanced attention block and an enhanced attention block.

[0062] In some optional embodiments, the deep feature extraction module is composed of a plurality of residual self-calibrated attention groups (RSAGs), a convolutional layer, and a self-calibrated enhanced attention module (SEAB). In addition, a skip connection is introduced between the start part and the input to reduce the difficulty of model learning.

[0063] like Figure 3 The figure shows the structure of the residual self-calibration attention group. Each residual self-calibration attention group (RSAG) consists of multiple basic residual blocks (RBs), a convolutional layer, and a self-calibration enhanced attention module (SEAB), plus a jump connection. Specifically, after the shallow features are obtained by the shallow feature extraction module, the shallow features pass through multiple residual self-calibration attention groups (RSAGs), and the convolutional layer splits its output into two feature maps with the same number of channels. One feature map is processed by the convolutional layer to generate a feature map in the original scale space, and the other feature map is processed by the convolutional layer and then generated by the self-calibration enhanced attention module (SEAB). The two feature maps are then connected across channels to form the output.

[0064] Among them, the Self-Aligned Enhanced Attention Module (SEAB) is used as an attention mechanism to enhance the perceptual field, enabling any spatial location to adaptively encode contextual information from distant areas, such as Figure 4 Shown is a structural schematic diagram of a self-calibration enhanced attention module, which includes a self-calibration enhanced attention block and an enhanced attention block.

[0065] It should be noted that the embodiment of the present invention introduces a residual self-calibration attention group in the generator, which contains a self-calibration enhanced attention block and an enhanced attention block, which can increase the feature extraction information in the input sonar image, thereby accurately capturing the detailed features of the entire sonar image, and can generate a sonar image with good texture details.

[0066] S1012: construct a discriminator, which includes an encoder and a decoder.

[0067] In some optional embodiments, such as Figure 5 The figure shows the structure of the discriminator. The Unet-based discriminator is mainly composed of an encoder and a decoder. The encoder continuously processes the super-resolution image I SR Downsampling is performed to obtain global information, and finally responds to the overall image reality. While the decoder is committed to judging the authenticity of local information, it continuously performs progressive upsampling operations and outputs the same image as the super-resolution image I SRPixel-by-pixel authenticity at the same resolution. In addition, the skip connection facilitates the flow of information between different levels, allowing the network to learn more features. Such a structural design forces the discriminator to focus on the differences in structure and semantic information between true and false samples, and pursues the accuracy of the global context and local information of the reconstruction results. Finally, it is input into the convolutional layer. In addition, both the encoding layer and the decoding layer include spectral normalization regularization (Spectral Norm). Spectral normalization constrains the Lipschitz constant of the discriminator by limiting the spectral norm (i.e., the maximum singular value) of the weight matrix of each layer of the neural network of the discriminator, so that it satisfies the 1-Lipschitz condition. This operation can reduce the sensitivity of the discriminator to small changes in the input, thereby helping to stabilize the training process and suppress artifacts.

[0068] It should be noted that the embodiment of the present invention introduces a Unet-based discriminator in the generative adversarial network to improve the perception performance and enable the generator to focus on the challenging areas of the sonar image for reconstruction. The constructed discriminator can distinguish low-resolution images from high-resolution images at the detail level and is used to guide the generator to generate realistic textures in complex environments.

[0069] S102, acquiring a first sonar image data set, and preprocessing the first sonar image data set to obtain a second sonar image data set;

[0070] As an optional implementation, the step of obtaining the first sonar image dataset and preprocessing the first sonar image dataset to obtain the second sonar image dataset can be specifically divided into the following steps S1021 to S1025:

[0071] S1021, acquiring a first sonar image data set, where the first sonar image data set includes a plurality of first sonar image samples;

[0072] In some optional embodiments, firstly, a first sonar image dataset of various scenes is collected. The first sonar image dataset of the embodiment of the present invention is mainly a SCTD dataset. SCTD constructs a completely open sonar dataset that can be used for sonar image research based on widely collected sonar images. The dataset contains three typical targets: underwater wreckage, wreckage of a crashed aircraft, and victims, with a total of 596 samples.

[0073] S1022, performing data enhancement on each first sonar image sample to obtain a plurality of second sonar image samples;

[0074] S1023, performing unsharp masking on each second sonar image sample to obtain a plurality of third sonar image samples;

[0075] S1024, denoising each third sonar image sample to obtain a plurality of fourth sonar image samples;

[0076] S1025. Downsample each fourth sonar image sample to obtain a second sonar image data set.

[0077] It is understandable that the underwater environment is very complex, and sonar images often face problems such as low resolution and high noise, resulting in very rare clean sonar images. If the original sonar images are directly downsampled to generate HR-LR data pairs, the performance will be degraded in actual scenarios. Therefore, the embodiments of the present invention adopt a variety of preprocessing techniques to alleviate the problem of poor sonar image quality.

[0078] Specifically, firstly, since too few first sonar image samples are collected, there is a risk of model overfitting. To address this risk, the second sonar image sample is obtained by performing data augmentation operations such as rotation, scaling, and cropping on the first sonar image sample. The data augmentation operation can expand the number of images, improve the generalization ability of the network, and avoid overfitting. Subsequently, the second sonar image sample is subjected to unsharp masking (USM), and the original second sonar image sample is Gaussian blurred and subtracted from the original image, and then an enhanced sharpening filter is applied to the result to enhance the details and edges of the image, thereby obtaining a third sonar image sample. Subsequently, the third sonar image sample is subjected to bicubic downsampling with a scale factor of 2 to remove noise and make the image clearer, thereby obtaining a fourth sonar image sample. Next, the processed fourth sonar image sample is used as a high-resolution sonar image I. HR , after downsampling, a low-resolution image I is formed LR , and obtain the second sonar image dataset.

[0079] It should be noted that the embodiment of the present invention uses a targeted data preprocessing method to expand the original underwater sonar image dataset, which can improve the robustness of the method in complex underwater environments, such as low contrast, high noise and blurred resolution, to a certain extent.

[0080] S103, inputting the second sonar image data set into a generative adversarial network for training, and updating the parameters of the generative adversarial network through a preset total loss function to obtain a trained super-resolution reconstruction model;

[0081] Further as an optional implementation, the second sonar image dataset includes a first-resolution sonar image and a second-resolution sonar image, the second sonar image dataset is input into a generative adversarial network for training, and the parameters of the generative adversarial network are updated by a preset total loss function to obtain a trained super-resolution reconstruction model. This step can be specifically divided into the following steps S1031 to S1034:

[0082] S1031, inputting the first resolution sonar image into a generator to obtain a reconstruction result;

[0083] S1032, inputting the second resolution sonar image into a discriminator, and then comparing the second resolution sonar image with the reconstruction result through the discriminator to obtain discrimination information;

[0084] Specifically, the second sonar image dataset consists of low-resolution sonar images I LR (i.e., first resolution sonar image) and high resolution sonar image I HR The high-quality sonar image dataset (i.e., the second-resolution sonar image dataset) is composed of two sets of sonar images with different resolutions, and the high-quality sonar image dataset (i.e., the second-resolution sonar image dataset) is input into the constructed generative adversarial network for training. The low-resolution sonar image is input into the generator to obtain the reconstruction result, and the reconstruction result and the high-resolution sonar image are simultaneously input into the discriminator to obtain the discrimination information, and finally the discrimination information is transmitted back to the generator.

[0085] S1033, updating the parameters of the generative adversarial network according to the discriminant information and the total loss function;

[0086] S1034. When the number of iterations reaches a preset first threshold or the test accuracy reaches a preset second threshold, stop training to obtain a trained super-resolution reconstruction model.

[0087] Specifically, based on the discriminant information and the total loss function, the back propagation algorithm and gradient descent can be used to update the parameters of the generator and discriminator in the generative adversarial network. After a few iterations, a trained super-resolution reconstruction model can be obtained, so that the generator gradually learns to generate images that are closer to real samples, and the discriminator is better at distinguishing between real and fake samples. The specific number of iterations can be set in advance, or the training is considered complete when the test set meets the accuracy requirements.

[0088] As an optional implementation, the super-resolution reconstruction method further includes the step of presetting a total loss function. The step of setting the total loss function can be specifically divided into the following steps D100 to D400:

[0089] D100, use L1 loss function as pixel loss function;

[0090] D200, construct the perceptual loss function based on the feature map generated by the generative adversarial network and the L1 loss function;

[0091] D300, construct an adversarial loss function based on the generator loss and the discriminator loss;

[0092] D400. Perform weighted summation of the pixel loss function, the perceptual loss function, and the adversarial loss function to obtain the total loss function.

[0093] Specifically, the loss function consists of a pixel loss function, a perceptual loss function, and an adversarial loss function.

[0094] The pixel loss function is an important evaluation index in image processing tasks. It measures the pixel-by-pixel difference between the real high-resolution sonar image and the reconstructed low-resolution sonar image. It is a basic component of the loss function and plays an irreplaceable role in ensuring network convergence. Since the L1 loss has good robustness, the embodiment of the present invention chooses to use the L1 loss function as the pixel loss function and defines the pixel loss function as follows:

[0095]

[0096] Where C, H and W represent the height, width and number of channels of the generated image respectively. G(·) represents the generator function, I LR represents a low-resolution sonar image, I HR Represents a high-resolution sonar image.

[0097] The perceptual loss function measures the similarity between the generated image and the real image by comparing their differences in depth features. The perceptual loss function can improve the visual quality of the generated image and make the generated super-resolution image more realistic. In this embodiment of the present invention, a VGG network truncated at the Conv5-4 layer is used to extract the depth features of the low-resolution sonar image and the high-resolution sonar image. The extracted features are combined with the L1 loss function for comparison, and the perceptual loss function is defined as follows:

[0098]

[0099] Among them, C, H and W are the height, width and number of channels of the VGG19-54 feature map respectively, and F VGG54 Represents the 4th convolution feature map and the 5th maximum pooling layer feature map before VGG19 network activation.

[0100] The adversarial loss function is used to promote the generation of realistic high-resolution sonar images. In the Relative Generative Adversarial Network (RGAN) framework, the generator adversarial loss and the discriminator loss are expressed as:

[0101] L adv = -Ε[log(1-D RGAN (I HR ,G(I LR )))]-Ε[log(D RGAN (G(I LR ),I HR ))]

[0102] L D = -Ε[log(D RGAN(I HR ,G(I LR )))]-Ε[log(1-D RGAN (G(I LR ),I HR ))]

[0103] Among them, L adv and L D Denote the generator loss and the discriminator loss respectively. E[·] denotes the average of a mini-batch, D RGAN Indicates that the discriminator type is a relative discriminator (Relativistic GAN).

[0104] Finally, the pixel loss function, perceptual loss function, and adversarial loss function are weighted and summed to obtain the total loss function L of the self-calibration generator. G :

[0105] L G =αL pixel +βL perc +γL adv

[0106] Among them, α, β, and γ are weighted parameters used to balance different loss terms.

[0107] S104, obtaining a sonar image to be measured, and inputting the sonar image to be measured into a super-resolution reconstruction model to obtain a super-resolution reconstruction result.

[0108] The above is a description of the sonar image super-resolution reconstruction method based on a generative adversarial network according to an embodiment of the present invention. It can be appreciated that, compared with the super-resolution reconstruction method in the prior art, the embodiment of the present invention has the following advantages:

[0109] First, the super-resolution reconstruction model based on the generative adversarial network is used to reconstruct the underwater sonar images, which can accurately capture the detailed features of the underwater sonar images and further restore the real texture details of the underwater sonar images;

[0110] Second, the residual self-calibration attention group is introduced in the generator, which contains the self-calibration enhanced attention block and the enhanced attention block, which can increase the feature extraction information in the input sonar image, thereby accurately capturing the detailed features of the entire sonar image and generating sonar images with good texture details;

[0111] Third, a Unet-based discriminator is introduced in the generative adversarial network to improve the perceptual performance and enable the generator to focus on the challenging areas of the sonar image for reconstruction. The constructed discriminator can distinguish between low-resolution and high-resolution images at the detail level and is used to guide the generator to generate realistic textures in complex environments;

[0112] Fourth, using targeted data enhancement methods to expand the original underwater sonar image dataset can improve the robustness of this method in complex underwater environments such as low contrast, high noise and blurred resolution to a certain extent.

[0113] Reference Figure 6 The embodiment of the present invention further provides a sonar image super-resolution reconstruction system based on a generative adversarial network, comprising:

[0114] A network building module is used to build a generative adversarial network, which includes a generator and a discriminator.

[0115] A data set acquisition module is used to acquire a first sonar image data set, preprocess the first sonar image data set, and obtain a second sonar image data set;

[0116] A model training module is used to input the second sonar image data set into a generative adversarial network for training, and update the parameters of the generative adversarial network through a preset total loss function to obtain a trained super-resolution reconstruction model;

[0117] The super-resolution reconstruction module is used to obtain the sonar image to be tested, input the sonar image to be tested into the super-resolution reconstruction model, and obtain the super-resolution reconstruction result.

[0118] The contents of the above-mentioned embodiments of the sonar image super-resolution reconstruction method based on a generative adversarial network are all applicable to the embodiments of the sonar image super-resolution reconstruction system based on a generative adversarial network. The functions specifically implemented by the embodiments of the sonar image super-resolution reconstruction system based on a generative adversarial network are the same as those of the above-mentioned embodiments of the sonar image super-resolution reconstruction method based on a generative adversarial network, and the beneficial effects achieved are also the same as the beneficial effects achieved by the above-mentioned embodiments of the sonar image super-resolution reconstruction method based on a generative adversarial network.

[0119] The embodiment of the present invention also provides an electronic device, which includes: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and when the program is executed by the processor, the sonar image super-resolution reconstruction method based on the generative adversarial network is realized. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0120] like Figure 7 FIG. 1 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention, referring to FIG. Figure 7 , an embodiment of the present invention provides an electronic device, including:

[0121] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0122] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the sonar image super-resolution reconstruction method based on a generative adversarial network according to the embodiment of the present invention;

[0123] Input / output interface 1003, used to implement information input and output;

[0124] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0125] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0126] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0127] An embodiment of the present invention also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned sonar image super-resolution reconstruction method based on generative adversarial network.

[0128] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0129] The embodiment of the present invention also discloses a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 1 The method shown.

[0130] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0131] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified to the contrary, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0132] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the above methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0133] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0134] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0135] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0136] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A sonar image super-resolution reconstruction method based on a generative adversarial network, characterized in that: The following steps are involved: Constructing a generative adversarial network, wherein the generative adversarial network includes a generator and a discriminator; Acquire a first sonar image data set, and preprocess the first sonar image data set to obtain a second sonar image data set; Inputting the second sonar image data set into the generative adversarial network for training, and updating the parameters of the generative adversarial network through a preset total loss function to obtain a trained super-resolution reconstruction model; The sonar image to be measured is acquired, and the sonar image to be measured is input into the super-resolution reconstruction model to obtain a super-resolution reconstruction result.

2. The sonar image super-resolution reconstruction method based on a generative adversarial network according to claim 1, characterized in that: The constructing of a generative adversarial network specifically includes: Constructing the generator, which includes a shallow feature extraction module, a deep feature extraction module and an image reconstruction module; The discriminator is constructed, and the discriminator includes an encoder and a decoder.

3. The sonar image super-resolution reconstruction method based on a generative adversarial network according to claim 2 is characterized in that: The shallow feature extraction module includes a convolution layer, the deep feature extraction module includes a residual self-calibration attention group, a convolution layer and a self-calibration enhanced attention module, and the image reconstruction module includes an upsampler and a convolution layer.

4. The sonar image super-resolution reconstruction method based on a generative adversarial network according to claim 3 is characterized in that: The residual self-calibration attention group includes a residual block, a convolutional layer and the self-calibration enhanced attention module, and the self-calibration enhanced attention module includes a self-calibration enhanced attention block and an enhanced attention block.

5. The sonar image super-resolution reconstruction method based on a generative adversarial network according to claim 1, characterized in that: The acquiring of the first sonar image data set and preprocessing of the first sonar image data set to obtain the second sonar image data set specifically includes: Acquire the first sonar image dataset, where the first sonar image dataset includes a plurality of first sonar image samples; Performing data enhancement on each of the first sonar image samples to obtain a plurality of second sonar image samples; performing unsharp masking on each of the second sonar image samples to obtain a plurality of third sonar image samples; Denoising each of the third sonar image samples to obtain a plurality of fourth sonar image samples; Down-sampling each of the fourth sonar image samples is performed to obtain the second sonar image data set.

6. The sonar image super-resolution reconstruction method based on a generative adversarial network according to claim 1, characterized in that: The second sonar image data set includes first-resolution sonar images and second-resolution sonar images. The second sonar image data set is input into the generative adversarial network for training, and the parameters of the generative adversarial network are updated by a preset total loss function to obtain a trained super-resolution reconstruction model, specifically including: Inputting the first resolution sonar image into the generator to obtain a reconstruction result; Inputting the second resolution sonar image into the discriminator, and then comparing the second resolution sonar image with the reconstruction result through the discriminator to obtain discrimination information; Updating the parameters of the generative adversarial network according to the discrimination information and the total loss function; When the number of iterations reaches a preset first threshold or the test accuracy reaches a preset second threshold, the training is stopped to obtain the trained super-resolution reconstruction model.

7. A sonar image super-resolution reconstruction method based on a generative adversarial network according to any one of claims 1 to 6, characterized in that: The super-resolution reconstruction method further comprises the step of presetting the total loss function, wherein the step of setting the total loss function specifically comprises: Use L1 loss function as pixel loss function; Constructing a perceptual loss function based on the feature map generated by the generative adversarial network and the L1 loss function; Construct an adversarial loss function based on the generator loss and the discriminator loss; The pixel loss function, the perceptual loss function and the adversarial loss function are weightedly summed to obtain the total loss function.

8. A sonar image super-resolution reconstruction system based on a generative adversarial network, characterized in that: include: A network construction module, used to construct a generative adversarial network, wherein the generative adversarial network includes a generator and a discriminator; A data set acquisition module, used to acquire a first sonar image data set, and preprocess the first sonar image data set to obtain a second sonar image data set; A model training module, used for inputting the second sonar image data set into the generative adversarial network for training, and updating the parameters of the generative adversarial network through a preset total loss function to obtain a trained super-resolution reconstruction model; The super-resolution reconstruction module is used to obtain the sonar image to be measured, input the sonar image to be measured into the super-resolution reconstruction model, and obtain the super-resolution reconstruction result.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the sonar image super-resolution reconstruction method based on a generative adversarial network as described in any one of claims 1 to 7 are realized.

10. A storage medium, the storage medium being a computer-readable storage medium, used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the sonar image super-resolution reconstruction method based on a generative adversarial network as described in any one of claims 1 to 7.

Images