Contrast learning fused generative adversarial network sonar image denoising method and device

Abstract

The invention discloses a generative adversarial network sonar image denoising method and device fused with comparative learning, and the method comprises the steps: constructing a generative network, inputting a noisy image, and carrying out the image denoising processing through a fusion attention mechanism module and a comparative learning module; constructing a discrimination network, inputting a noise-free image without noise and a de-noised image generated by the generation network, and carrying out image discrimination; s3, training an adversarial network composed of the generative network and the discriminant network, and storing final parameters after training is finished; and S4, inputting the noise image into the trained generative network to obtain a denoised image. The fusion attention mechanism module and the contrast learning module can improve the ability of the generative network to understand and process the input image, enhance the ability of the network to capture and retain the image details, and accurately restore the main body features and detail information of the noise image.

Description

Technical Field

[0001] The present invention belongs to the field of image processing and underwater detection, and relates to a sonar image denoising method and device using a generative adversarial network integrating contrast learning. Background Art

[0002] Sonar is a technology that uses the propagation of sound waves in water and the principle of echoes for detection and imaging. It is widely used in ocean exploration and underwater navigation. Since light waves decay quickly in water, sound waves have stronger penetration and longer propagation distances, making them the preferred means of underwater imaging. Sonar generates images of seabed topography or underwater objects by emitting sound waves and receiving echo signals reflected from the seabed or underwater objects. However, due to factors such as water flow, waves, and the movement of the sonar equipment itself, the echo signals are often affected by noise and distortion, resulting in distortion and noise in the imaging results, which poses a challenge to subsequent data analysis and processing.

[0003] The noise and distortion in sonar images seriously affect the image quality and the accuracy of subsequent processing. Therefore, image denoising becomes a key step in the sonar data processing process. The goal of denoising technology is to remove noise in the image as much as possible while retaining the detailed features of the target object. Traditional image denoising methods such as Gaussian filtering and median filtering can eliminate noise to a certain extent, but they often weaken important information in the image at the same time. In recent years, denoising methods based on deep learning have received widespread attention. These methods can more effectively separate noise and effective signals by learning a large amount of sample data, thereby improving the image quality and processing effect. However, in sonar image analysis, the collection of sonar image data is relatively complex and is restricted by environmental conditions. At the same time, sonar image data may contain sensitive information, and data collection and sharing are restricted. Therefore, the difficulty of data collection and the limited number of labeled samples make small sample learning an urgent problem to be solved.

[0004] As a self-supervised learning method, contrastive learning allows the model to learn the similarities and differences between samples by constructing sample pairs, which can effectively improve the performance of the model on small sample data. In addition, the generative adversarial network also performs well in image generation and denoising. It consists of a generator and a discriminator. Through adversarial training between the two, the generator can gradually generate realistic images, and the discriminator continuously improves its ability to distinguish between true and false images. This two-way optimization mechanism enables the network to play an important role in denoising tasks and provides a new solution for sonar image denoising. Summary of the invention

[0005] The purpose of the present invention is to reduce the impact of insufficient sample size due to difficulties in sonar image acquisition, which leads to poor neural network image denoising effect, and to provide a generative adversarial network sonar image denoising method and device that integrates contrast learning.

[0006] The present invention can autonomously learn the optimal parameter configuration and model strategy on a given small sample sonar data set, and improve the sonar image denoising effect through comparative learning, attention mechanism, and game confrontation.

[0007] To solve the above technical problems, the first aspect of the present invention relates to a sonar image denoising method using a generative adversarial network fused with contrast learning, which specifically comprises the following steps:

[0008] S1, builds a generative network, inputs a noisy image, and performs image denoising after the fusion attention mechanism module and contrast learning module;

[0009] S2, construct a discriminant network, input a noise-free image without noise and a denoised image generated by a generative network, and perform image discrimination;

[0010] S3, train the adversarial network composed of the generating network and the discriminating network, and save the final parameters at the end of the training;

[0011] S4, input the noisy image into the trained generation network to obtain the denoised image.

[0012] Step S1 also includes the following steps:

[0013] S11. Design a feature extraction network. The network consists of three convolutional layers to form an encoding module. It extracts image features and transforms the input original noisy image P into a multi-layer feature map. Represents, where i represents the i-th layer feature;

[0014] S12. Design a network that integrates attention mechanisms. That is, the last layer of the feature extraction network outputs the feature map F as the network input, and the feature map F last ∈P H×W×C The feature map F′ is obtained by processing the channel attention mechanism module, and then the feature map F″ is obtained by processing the spatial attention mechanism module. By fusing the attention mechanism network, the feature map F″ of the mixed channel and spatial two-dimensional information is obtained. The calculation formula is as follows:

[0015]

[0016] Among them, M c and M sThey represent the channel attention mechanism module and the spatial attention mechanism module respectively. The channel attention mechanism module processes the input feature map F to obtain F′, and the spatial attention mechanism module processes the input feature map F′ to obtain F″:

[0017] M c (F)=ξ(C(C(AvgP(F)))+C(C(MaxP(F)))) (2)

[0018] M s (F')=ξ(C(Cat(AvgP(F')+MaxP(F')))) (3)

[0019] Where ξ represents the Sigmoid function, AvgP represents average pooling, MaxP represents maximum pooling, C represents the convolution layer, and Cat represents the concatenation function;

[0020] S13. Design a contrastive learning network. The network consists of contrastive learning modules. Select the generator G encoder The feature layer of the encoding is generated through two MLP networks H, and the cross entropy loss is calculated. The contrastive learning network is used to constrain the generation network. The contrastive learning module designed in step S13 autonomously selects different feature maps for cross entropy calculation. The cross entropy calculation formula is:

[0021]

[0022] The contrastive learning feature is expressed as:

[0023]

[0024] in represents the output feature of the i-th layer, t i Represents the i-th layer feature, i represents the number of layers;

[0025] S14. Design an image restoration network. The network consists of multiple groups of MLP networks. Each group of MLP networks consists of two deconvolution layers and one convolution layer. Through the image restoration network, the feature map F” is restored to the denoised image P’ generated by the generation network.

[0026] The second aspect of the present invention relates to a sonar image denoising device for generative adversarial network fusion of contrastive learning, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement a sonar image denoising method for generative adversarial network fusion of contrastive learning of the present invention.

[0027] A third aspect of the present invention relates to a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements a generative adversarial network sonar image denoising method integrating contrastive learning of the present invention.

[0028] The working principle of the present invention is:

[0029] During the training stage, considering the scarcity of sonar images due to the difficulty in collecting them, the existing denoising models do not have a good effect on sonar image denoising. The generative adversarial network is used to enable the generator to have the ability to extract features of noise-free images in the target domain, so that the generated denoised images have the characteristics of noise-free images. At the same time, the contrastive learning module is used to constrain the content distribution of the generated denoised images, so that the denoised images are consistent with the input noisy images in content distribution. Finally, a noise-free image with the same main content as the noisy image is obtained.

[0030] The innovative features of the present invention are:

[0031] A denoising method for forward-looking sonar images using a generative adversarial network is proposed; a network model training method combining fusion attention mechanism and contrastive learning for small sample image denoising is proposed.

[0032] The present invention has the following beneficial effects:

[0033] 1. To solve the problem of small number of sonar image samples, the present invention proposes a sonar image denoising method based on a generative adversarial network that integrates contrastive learning. The denoising network can be trained using an unpaired data set, i.e., without data pairs consisting of noisy images and noise-free images, thereby simplifying the data preparation process and improving the practicability and universality of the algorithm.

[0034] 2. To solve the problem that neural networks have poor denoising effects on sonar images due to the particularity of sonar images, the present invention proposes a sonar image denoising method that integrates contrast learning and generative adversarial networks. While removing noise, it effectively retains the content and detail information of the image, thereby improving the denoising effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a flow chart of the method of the present invention.

[0036] Figure 2 It is a schematic diagram of the residual network structure of the present invention integrating the attention mechanism.

[0037] Figure 3 It is a flow chart of the comparative learning module of the present invention.

[0038] Figure 4 It is a structural diagram of the device of the present invention. DETAILED DESCRIPTION

[0039] In order to better understand the above technical solution, the technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0040] Example 1

[0041] like Figure 1 As shown, a sonar image denoising method based on a generative adversarial network integrating contrastive learning is implemented, and the implementation process specifically includes the following steps:

[0042] S1. Build a generative network, input a noisy image, and perform image denoising after fusing the attention mechanism module and the contrastive learning module.

[0043] S11. Design a feature extraction network. The network consists of a coding module consisting of three convolutional layers. c represents the convolution kernel size, s represents the convolution step size, p represents padding, and k represents the number of convolution kernels. Conv1: c7s1k64, Conv2: c3s2p1k128, and Conv3: c3s2p1k256 are used as feature extraction networks to transform the input original noisy image P into a multi-layer feature. Represents, where i represents the i-th layer feature;

[0044] S12. Design a fusion attention mechanism network, which consists of nine cascaded residual networks. That is, the last layer of the feature extraction network outputs the feature map F as the network input, passes through two cascaded convolutional layers Conv1: c3s1k64, Conv2: c3s1k64, after instance normalization and ReLU activation function, first passes through the channel attention mechanism module and then passes through the spatial attention mechanism module to obtain a feature map F that mixes the information of the two dimensions of space and channel. Its structure is as follows Figure 2 As shown;

[0045] Feature map F last ∈P H×W×C The feature map F′ is obtained by processing the channel attention mechanism module, and then the feature map F″ is obtained by processing the spatial attention mechanism module. By fusing the attention mechanism network, the feature map F″ of the mixed channel and spatial two-dimensional information is obtained. The calculation formula is as follows:

[0046]

[0047] Among them, M c and M s They represent the channel attention mechanism module and the spatial attention mechanism module respectively. The channel attention mechanism module processes the input feature map F to obtain F′, and the spatial attention mechanism module processes the input feature map F′ to obtain F″:

[0048] M c(F)=ξ(C(C(AvgP(F)))+C(C(MaxP(F)))) (2)

[0049] M s (F')=ξ(C(Cat(AvgP(F')+MaxP(F')))) (3)

[0050] Where ξ represents the Sigmoid function, AvgP represents average pooling, MaxP represents maximum pooling, C represents the convolution layer, and Cat represents the concatenation function;

[0051] S13. Design a contrastive learning network. The network consists of contrastive learning modules. Select the generator G encoder The encoded feature layer is passed through two MLP networks H to generate contrastive learning features, calculate the cross entropy loss, and constrain the generation network through the contrastive learning network. The contrastive learning feature is expressed as:

[0052]

[0053] in represents the output feature of the i-th layer, t i Represents the i-th layer feature, i represents the number of layers;

[0054] S14. Design an image restoration network. The network consists of multiple groups of MLP networks. Each group of MLP networks consists of two deconvolution layers DeConv1: c3s0.5p1k128, DeConv2: c3s0.5p1k64 and one convolution layer Conv: c7s1k3. Through the image restoration network, the feature map F” is restored to the denoised image P’ generated by the generation network.

[0055] S2. Construct a discriminant network, input the noise-free image and the denoised image generated by the generative network, and perform image discrimination. The discriminant network consists of five groups of convolutional layers, including Conv1: c4s2p1k64, Conv2: c4s2p1k128, Conv3: c4s2p1k256, Conv4: c4s1p1k512, Conv: c4s1p1k1. Each convolutional layer contains batch normalization and Leaky ReLU activation function. The discriminant network finally outputs a value between 0 and 1, which represents the probability that the input image is a noise-free image, which is used to measure the denoising effect.

[0056] S3. Train the adversarial network composed of the generative network and the discriminative network, and save the final parameters at the end of the training. During the training of the generative adversarial network, the overall training loss is composed of the adversarial loss and the contrastive learning loss.

[0057] Adversarial loss is used to ensure that the denoised image G(X) generated by the generator G is visually similar to the real noise-free image Y, so that the denoised image is as similar as possible to the noise-free image, so that the generator G can generate a sufficiently realistic denoised image, so that the discriminator D cannot distinguish between the denoised image and the noise-free image, thereby achieving the denoising effect. The adversarial loss formula is as follows:

[0058]

[0059] Contrastive learning loss is used to constrain the generative network and solve the distortion problem that may occur in the process of generating images, so that the distribution of the generated image G(X) is consistent with the distribution of the noisy image X. Contrastive learning loss is composed of cross entropy loss;

[0060]

[0061] Where t represents the number of patches in each layer of the feature map t∈{1,2,..,S I}, where S i Indicates that the i-th layer has S i spatial location, Indicates that the dimension of the feature vector for the sth patch of the i-th layer is C i , Indicates that in the i-th layer, the features of s are removed from all patchesS;

[0062] The cross entropy loss is calculated as shown in formula (9):

[0063]

[0064] like Figure 3 As shown, the positive example, output, and N negative examples are mapped to K-dimensional vectors t,t + ,t - , normalize the vector to a unit sphere to prevent space collapse or expansion, and calculate the cross entropy loss l, where τ is a proportional hyperparameter, often called a temperature coefficient, which indicates the probability of a positive sample being selected.

[0065] S4. Input the noisy image into the trained generative network to obtain a denoised image.

[0066] The fused attention mechanism of the present invention can further extract features, improve the generator's ability to understand and process input images, and at the same time enhance the network's ability to capture and retain image details; the generative adversarial network can support small sample sonar image denoising, solving the problem that traditional sonar image denoising requires paired data sets; the contrastive learning module can suppress possible distortion and erroneous information in the generated image, constrain the generated image, and accurately restore the main features and detail information of the noisy image.

[0067] Example 2

[0068] like Figure 4 This embodiment relates to a sonar image denoising device for generating adversarial networks that integrate contrastive learning, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement a sonar image denoising method for generating adversarial networks that integrate contrastive learning in Example 1.

[0069] Example 3

[0070] The present embodiment relates to a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, a generative adversarial network sonar image denoising method integrating contrast learning of embodiment 1 is implemented.

[0071] Finally, it should be noted that the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the implementation methods of the present invention. Technical personnel in this profession may make slight changes and modifications to the technical contents described without departing from the technical scope of the present invention. However, all modifications and variations made without departing from the spirit and principles of the present invention should fall within the scope of the technical solution of the present invention.

Claims

1. A sonar image denoising method based on a generative adversarial network integrating contrastive learning, characterized by: The steps include: S1, builds a generative network, inputs a noisy image, and performs image denoising after the fusion attention mechanism module and contrast learning module; S2, construct a discriminant network, input a noise-free image without noise and a denoised image generated by a generative network, and perform image discrimination; S3, train the adversarial network composed of the generating network and the discriminating network, and save the final parameters at the end of the training; S4, input the noisy image into the trained generation network to obtain the denoised image.

2. The sonar image denoising method of integrating contrastive learning with a generative adversarial network as claimed in claim 1, characterized in that: Step S1 also includes: S11. Design a feature extraction network. The network consists of three convolutional layers to form an encoding module. It extracts image features and transforms the input original noisy image P into a multi-layer feature map. Represents, where i represents the i-th layer feature; S12. Design a network that integrates attention mechanisms. That is, the last layer of the feature extraction network outputs the feature map F as the network input, and the feature map F last ∈P H×W×C The feature map F′ is processed by the channel attention mechanism module, and the feature map F″ is processed by the spatial attention mechanism module. The feature map F” of mixed channel and spatial information is obtained by fusing the attention mechanism network. S13. Design a contrastive learning network. The network consists of contrastive learning modules. Select the generator G encoder The encoded feature layer is passed through two MLP networks H to generate contrastive learning features, calculate the cross entropy loss, and constrain the generation network through the contrastive learning network; S14. Design an image restoration network. The network consists of multiple groups of MLP networks. Each group of MLP networks consists of two deconvolution layers and one convolution layer. Through the image restoration network, the feature map F” is restored to the denoised image P’ generated by the generation network.

3. The sonar image denoising method of a generative adversarial network fused with contrast learning as claimed in claim 2, characterized in that: The fusion attention mechanism network designed in step S12 is composed of nine cascaded residual networks. The residual network consists of two cascaded convolutional layers. After instance normalization and ReLU activation function, it first passes through the channel attention mechanism module and then passes through the spatial attention mechanism module to obtain the final feature map.

4. The sonar image denoising method of integrating contrastive learning with a generative adversarial network as claimed in claim 2, characterized in that: The contrast learning module designed in step S13 autonomously selects different feature maps for cross entropy calculation. The cross entropy calculation formula is: The contrastive learning feature is expressed as: in represents the output feature of the i-th layer, t i Represents the i-th layer feature, and i represents the number of layers.

5. The sonar image denoising method of integrating contrastive learning with a generative adversarial network as claimed in claim 1, characterized in that: The discriminant network described in step S2 is composed of five groups of convolutional layers. Each convolutional layer contains batch normalization and LeakyReLU activation function. The discriminant network finally outputs a value between 0 and 1, which represents the probability that the input image is a noise-free image, which is used to measure the denoising effect.

6. The sonar image denoising method of integrating contrastive learning with a generative adversarial network as claimed in claim 1, characterized in that: In step S3, during the training of the generative adversarial network, the overall training loss is composed of the sum of the adversarial loss and the contrastive learning loss; Adversarial loss is used to ensure that the denoised image G(X) generated by the generator G is visually similar to the real noise-free image Y. The denoised image is made as similar to the noise-free image as possible so that the generator G can generate a sufficiently realistic denoised image and the discriminator D cannot distinguish between the denoised image and the noise-free image, thereby achieving the denoising effect. Contrastive learning loss is used to constrain the generative network and solve the distortion problem that may occur in the process of generating images, so that the distribution of the generated image G(X) is consistent with the distribution of the noisy image X.

7. The sonar image denoising method of integrating contrastive learning with a generative adversarial network as claimed in claim 6, characterized in that: The adversarial loss formula is as follows: The contrastive learning loss consists of the cross entropy loss; Where t represents the number of patches in each layer of the feature map t∈{1,2,..,S I }, where S i Indicates that the i-th layer has S i spatial location, Indicates that the dimension of the feature vector for the sth patch of the i-th layer is C i , It means that in the i-th layer, the features of s are removed from all patchesS.

8. A sonar image denoising device based on a generative adversarial network integrating contrastive learning, characterized in that: The invention comprises a memory and one or more processors, wherein the memory stores executable codes, and when the one or more processors execute the executable codes, the method for denoising sonar images by generating adversarial networks by integrating contrastive learning as described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, a generative adversarial network sonar image denoising method integrating contrast learning as described in any one of claims 1 to 6 is implemented.

Images