GOCI image inpainting and denoising method based on self-attention mechanism and joint dilated convolution

Abstract

The application provides a GOCI image restoration and denoising method based on a self-attention mechanism and a joint dilated convolution, comprising: S1, a training stage, training a generator; S2, an application stage, using the trained generator to restore GOCI images. The S1 training stage specifically comprises: S11, selecting a training image and an original complete image thereof; S12, an encoder extracting semantic information features; S13, a self-attention module extracting high-level semantic information features; S14, a decoder up-sampling the semantic information features, using a joint dilated convolution operation to refine the restoration and denoising results, and forming a preliminary result image; S15, a discriminator combining the preliminary result image and the original complete image, calculating an adversarial loss and returning the generator. The application reduces mask preprocessing and overcomes the complicated steps of the prior art; the self-attention mechanism is used to promote the reasoning and restoration of semantic information by the encoder during the restoration process; and the joint dilated convolution module is used to further enhance the complexity of multi-sampling.

Description

TECHNICAL FIELD

[0001] The application relates to the field of image inpainting and denoising, and particularly relates to a GOCI image inpainting and denoising method based on a self-attention mechanism and a joint dilated convolution. BACKGROUND

[0002] GOCI (Geostationary Ocean Color Imager) is a Korean ocean color remote sensor, which has very high precision and strong application value. However, due to factors such as noise and cloud cover, a large area of image is missing, and pixel color abnormalities are easily caused, which has an adverse effect on subsequent related applications, and therefore needs to be repaired and denoised.

[0003] Although much progress has been made in the research on image denoising and inpainting in natural scenes, the effect is not very ideal when these algorithms are directly used for denoising and inpainting of ocean remote sensing water color images. The mainstream image denoising and inpainting algorithms at present are mainly CNN-based neural network models, which have a wide range of applications in face and building inpainting scenes. These algorithms generally use an end-to-end CNN inpainting model to extract the features of an image from coarse to fine through a deep convolutional layer, and then realize image inpainting through upsampling. The main reason why these mainstream image inpainting algorithms in natural scenes do not work well for ocean remote sensing water color images is that the missing area of the ocean remote sensing water color images is generally high, and the image denoising and inpainting algorithms in natural scenes lack a mechanism for reconstructing the semantic information of the large-area missing ocean water color images to be inpainted. SUMMARY

[0004] The application aims to solve the problem that the existing image inpainting and denoising method does not work well in the application of GOCI images.

[0005] The technical scheme adopted by the application to solve the above technical problem is to provide a GOCI image inpainting and denoising method based on a self-attention mechanism and a joint dilated convolution, and the steps are as follows.

[0006] S1, training stage, training a generator;

[0007] S2, application stage, using the trained generator to inpaint a GOCI image, inputting the GOCI image to be inpainted and denoised into the generator, and outputting an inpainted and denoised image by the generator;

[0008] The S1 training stage is specifically as follows.

[0009] S11, selecting a training image and an original complete image thereof, and inputting the training image into a generator to form a data set;

[0010] S12, the encoder in the generator discriminates the region with missing and noise in the training image, and extracts the semantic information feature of the training image;

[0011] S13, the self-attention module in the generator extracts the high-level semantic information feature of the training image;

[0012] S14, the decoder in the generator up-samples the extracted semantic information feature, and uses the joint dilated convolution operation to refine and repair the denoising result in the last stage of the up-sampling, to form a preliminary result image;

[0013] S15, the discriminator receives the preliminary result image output by the generator, combines the original complete image, calculates the adversarial loss, and returns to the generator.

[0014] Preferably, the training image in S11 has three sub-images, which are training images with only noise, training images with only missing, and training images with both noise and missing, and the image size is 224*224.

[0015] Preferably, in S12, the encoder adopts five groups of structures, the first group is one layer of input and two layers of convolution, and the second to fifth groups are one layer of pooling and two layers of convolution; the convolution kernel size of the convolution layer of the first group is increased from 3 to 64, so that the number of semantic information features becomes 64, the semantic information features output by the first group of the encoder are stacked with the semantic information features output by the fourth group of the decoder, and the semantic information features output by the second group, the third group and the fourth group of the encoder are stacked with the semantic information features output by the third group, the second group and the first group of the decoder respectively; the size of the semantic information features output by the encoder is down-sampled from 224*224 to 112*112, 56*56 in turn, and the convolution kernel size of the convolution layer is increased to 128, 256 in turn; the fourth group and the fifth group perform maximum pooling and convolution processing on the semantic information features, the size of the semantic information features is down-sampled to 28*28, and the convolution kernel size of the convolution layer is iterated to 1024, outputting 1024 feature maps with a size of 14*14.

[0016] Preferably, in S13, the self-attention module is connected to the end of the encoder, and after the semantic information features output by the encoder change the dimension, they are input to the self-attention module, which extracts high-level semantic information features, specifically: the self-attention module effectively extracts the high-level semantic information features in the complete region of the training image, and effectively reasons the high-level semantic information features in the missing region of the training image.

[0017] Preferably, the self-attention module is provided with four self-attention heads; the self-attention module is provided with three linear layers to meet the input of the query feature Q, the key-value feature K and the content feature V; the corresponding relationship mapping of the combination of the query feature Q and the key-value feature K and the content feature V is output through a scaling dot product operation, and after feature fusion, linear operation is performed in the linear layer; the attention weight between the query feature Q and the key-value feature K is calculated, and then the content feature V is acted on to obtain the weight of the content feature V and the output, that is, the degree of attention required by each region of the content feature V.

[0018] Preferably, the decoder in S14 performs an upsampling operation on the extracted semantic information feature, and the decoder adopts a five-group structure, the first group outputs the semantic information feature of the self-attention module through upsampling and convolution layers, then performs stacking processing with the semantic information feature output by the fourth group of the encoder, and then passes through a residual module to upsample the semantic information feature of the image to 28*28; the second, third and fourth groups of the decoder also adopt stacking processing, and perform stacking with the semantic information features output by the third, second and first groups of the encoder, respectively, to sequentially upsample the semantic information feature of the image to 56*56, 112*112 and 224*224, so that each layer structure of the decoder can obtain the semantic information feature output by the previous layer structure of the decoder and the semantic information feature output by the encoder.

[0019] Preferably, the joint dilated convolution in S14 is composed of five convolution layers with different dilation rates, and the dilation rates are 1, 2, 4, 8 and 16, respectively; the convolution layer includes batch normalization and an activation function ReLU, the convolution kernel size of the convolution layer is 4, the stride is 2, and the padding size is 1; the joint dilated convolution module obtains recovery features with larger spatial resolution and more global semantic information by using convolution layers with different dilation rates, which is helpful to repair the integrity of the image and the global texture of the image; a residual mechanism is also designed between adjacent two convolution layers, so that the joint dilated convolution modules are connected in a hierarchical manner; finally, the convolution results are superimposed, and the output result image is output after the activation function Sigmoid.

[0020] Preferably, the residual mechanism is composed of two convolution layers with a convolution kernel of 3*3, a stride of 1 and a padding size of 1, two activation layers and a residual connection.

[0021] Preferably, the S15 further comprises: calculating a pixel-level loss between the preliminary result image and its original complete image; inputting the preliminary result image and its original complete image into a VGG16 feature extractor, and the VGG16 feature extractor calculates a feature-level perceptual loss and a style loss between the images; the pixel-level loss, the perceptual loss and the style loss are returned to the training generator.

[0022] Preferably, the discriminator in S15 is a Markov discriminator type, which receives a preliminary result image, discriminates to obtain an adversarial loss according to the preliminary result image and its original complete image, and further supervises the training of the generator; the Markov discriminator mainly consists of four convolutional layers, three activation functions LeakyReLU and one activation function Sigmoid layer; the convolution kernel size of the first three convolutional layers is 4, the stride is 2, the padding size is 1, the neurons are activated using the activation function LeakyRelu, the activation function is set to a negative slope of 0.2, and the implace parameter is selected to perform an in-place operation; the last convolutional layer has a convolution kernel size of 4, a stride of 1, and a padding size of 1, and the neurons are activated using the activation function Sigmoid; finally, local discrimination is performed on the preliminary result image to ensure effective recovery of low-frequency local texture structure.

[0023] The beneficial effects in the application are:

[0024] 1. The GOCI image repair and denoising method based on the self-attention mechanism and the joint dilated convolution proposed in the application adopts end-to-end processing in the design aspect, directly inputs the image to be repaired and denoised, and outputs the result image after repair and denoising, and does not need to adopt a mask preprocessing process in the input aspect, so that the cumbersome process of the prior art is overcome.

[0025] 2. In order to enhance the extraction of high-level semantic features, the GOCI image repair and denoising method based on the self-attention mechanism and the joint dilated convolution proposed in the application adopts the self-attention mechanism, promotes the reasoning and restoration of semantic information by the encoder in the repair process, and is more in line with the repair requirements of the GOCI image.

[0026] 3. The GOCI image repair and denoising method based on the self-attention mechanism and the joint dilated convolution proposed in the application further enhances the complexity of multiple sampling by adopting the joint dilated convolution module, so as to improve the reconstruction ability of the algorithm to noise and the learning ability of the algorithm to global semantics with a wider field of view. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 The flowchart of the training stage of the embodiment of the application is shown in the figure;

[0028] Figure 2 The training image of the embodiment of the application is shown in the figure, Figure 2 (a) the training image only containing noise, Figure 2 (b) the training image only containing missing, Figure 2 (c) the training image containing both noise and missing;

[0029] Figure 3 The structural schematic diagram of the embodiment of the application is shown in the figure;

[0030] Figure 4 Structure diagram of a self-attention module of an embodiment of the application;

[0031] Figure 5 Structure diagram of a joint dilated convolution module of an embodiment of the application;

[0032] Figure 6 Diagram of a residual mechanism of an embodiment of the application;

[0033] Figure 7 Flowchart of an application stage of an embodiment of the application;

[0034] Figure 8 Comparison diagram of repair denoising results of an embodiment of the application, (a) is an image to be repaired, (b) is a position mark of an image missing area and noise, (c) is an image processed by using the algorithm of the application, and (d) is a real complete image corresponding to the missing image. DETAILED DESCRIPTION

[0035] The schemes in the embodiments of the application will be described in detail below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only some of the embodiments of the application, but not all the embodiments of the application.

[0036] The GOCI image repair denoising method based on the self-attention mechanism and the joint dilated convolution of the application has the following steps:

[0037] S1, training stage, training a generator;

[0038] S2, application stage, using the trained generator to repair GOCI images, inputting a GOCI image to be repaired and denoised into the generator, and outputting a repaired and denoised image by the generator;

[0039] The S1 training stage specifically includes the following steps:

[0040] S11, selecting a training image and an original complete image thereof, and inputting the training image into a data set to a generator;

[0041] S12, an encoder in the generator discriminates a region with missing and noise in the training image, and extracts semantic information features of the training image;

[0042] S13, a self-attention module in the generator extracts high-level semantic information features of the training image;

[0043] S14, a decoder in the generator performs up-sampling on the extracted semantic information features, and uses a joint dilated convolution operation to refine repair and denoising results in the last stage of the up-sampling, to form a preliminary result image;

[0044] S15, the discriminator receives the preliminary result image output by the generator, combines it with the original complete image, calculates the adversarial loss, and returns it to the generator.

[0045] For details, see Figure 1 The diagram shows a flowchart of the training phase in an embodiment of the present invention. In this embodiment, the selected training images are screenshots of GOCI images with missing regions and noise obtained from the NASA website. These images are cropped to 224*224 pixels to create a dataset. In this embodiment, operation S15 further includes: calculating the pixel-level loss between the preliminary result image and its original complete image; inputting the preliminary result image and its original complete image into a feature extractor, which calculates the perceptual loss and style loss at the feature level between the images; and returning the pixel-level loss, perceptual loss, and style loss to the training generator.

[0046] For details, see Figure 2 As shown, the training image in S11 of this embodiment has 3 sub-images. Figure 2 (a) is a training image containing only noise. Figure 2 (b) represents training images with only missing values. Figure 2 (c) represents training images that simultaneously contain noise and missing data.

[0047] For details, see Figure 3 The diagram shown is a structural schematic of an embodiment of the present invention, including a generator and a discriminator. The generator further includes an encoder, a self-attention module, a decoder, a joint dilated convolution module, and a VGG16 feature extractor.

[0048] In S12 of this embodiment, the encoder adopts a five-group structure. The first group consists of one input layer and two convolutional layers, while the second to fifth groups consist of one pooling layer and two convolutional layers. The kernel size of the convolutional layer in the first group is increased from 3 to 64, resulting in 64 semantic information features. The semantic information features output by the first group of the encoder are stacked with the semantic information features output by the fourth group of the decoder. The second, third, and fourth groups of the encoder are stacked with the semantic information features output by the third, second, and first groups of the decoder, respectively. The size of the semantic information features output by the encoder is downsampled from 224*224 to 112*112 and 56*56, respectively, and the kernel size of the convolutional layers is increased to 128 and 256, respectively. The fourth and fifth groups perform max pooling and convolution on the semantic information features, downsampling the size of the semantic information features to 28*28, and iterating the kernel size of the convolutional layers to 1024, outputting 1024 feature maps of size 14*14.

[0049] In S13 of this embodiment, the self-attention module is connected to the end of the encoder. After the semantic information features output by the encoder are changed in dimension, they are input to the self-attention module. The self-attention module extracts high-level semantic information features. Specifically, the self-attention module effectively extracts high-level semantic information features in complete regions of the training image, and at the same time performs effective reasoning on high-level semantic information features in missing regions of the training image.

[0050] In this embodiment, the decoder in S14 performs upsampling on the extracted semantic information features. The decoder employs a five-group structure. The first group outputs the semantic information features from the attention module through upsampling and convolutional layers, then stacks them with the semantic information features output from the fourth group of the encoder. After passing through a residual module, the semantic information features of the image are upsampled to 28*28. The second, third, and fourth groups of the decoder also undergo stacking, stacking with the semantic information features output from the third, second, and first groups of the encoder, respectively, upsampling the semantic information features of the image to 56*56, 112*112, and 224*224 sequentially. This ensures that each layer of the decoder can acquire both the semantic information features output from the previous layer and the semantic information features output from the encoder. The decoder's end is connected to a joint dilated convolutional module to refine the output, obtaining a preliminary result image.

[0051] In this embodiment, the discriminator in S15 is a Markov discriminator. The Markov discriminator receives a preliminary result image and, based on the preliminary result image and its original complete image, performs discriminative processing to obtain adversarial loss, thereby supervising the training of the generator. The Markov discriminator mainly consists of four convolutional layers, three LeakyReLU activation functions, and one Sigmoid activation function layer. The first three convolutional layers have a kernel size of 4, a stride of 2, and a padding size of 1. The LeakyReLU activation function is used to activate neurons, with a negative slope of 0.2 and the implace parameter selected for in-situ operation. The last convolutional layer has a kernel size of 4, a stride of 1, and a padding size of 1. The Sigmoid activation function is used to activate neurons. Finally, local discrimination is performed using the preliminary result image, ensuring the effective recovery of low-frequency local texture structures. While the preliminary result image and its original complete image are input into the discriminator, the pixel-level loss between the preliminary result image and its original complete image is also calculated. The preliminary result image and its original complete image are then input into the VGG16 feature extractor, which calculates the perceptual loss and style loss at the feature level between the images. The pixel-level loss, perceptual loss, and style loss, together with the adversarial loss output by the discriminator, are returned to the training generator.

[0052] Specifically, the Markov discriminator is a texture discriminator.

[0053] For details, seeFigure 4 As shown in the figure, it is a structural schematic diagram of a self-attention module of an embodiment of the present application, the self-attention module is provided with four self-attention heads; the self-attention module is provided with three linear layers to meet the input of query features Q, key value features K and content features V; the corresponding relationship mapping of the combination of the query features Q and the key value features K and the content features V is output through a scaling dot multiplication operation, and after feature fusion, linear operation is performed in a linear layer; the attention weight between the query features Q and the key value features K is calculated, and then the content features V are acted on to obtain the weight of the content features V and the output, the weight is the degree of attention required by each region of the content features V.

[0054] Specifically, referring to Figure 5 As shown in the figure, it is a structural schematic diagram of a joint dilated convolution module of an embodiment of the present application, the joint dilated convolution in the S14 is composed of five convolution layers with different expansion rates, and the expansion rates are 1, 2, 4, 8 and 16 respectively; the convolution layer includes batch normalization and an activation function ReLU, the convolution kernel size of the convolution layer is 4, the stride is 2, and the padding size is 1; the joint dilated convolution module obtains recovery features with larger spatial resolution by using convolution layers with different expansion rates, and obtains more global semantic information, which is helpful to repair the integrity of the image and the global texture of the image; a residual mechanism is also designed between adjacent two convolution layers, so that the joint dilated convolution modules are connected in a hierarchical manner; finally, the convolution results are superimposed, and the output result image is obtained after the activation function Sigmoid.

[0055] Specifically, referring to Figure 6 As shown in the figure, it is a schematic diagram of a residual mechanism of an embodiment of the present application, the residual mechanism is composed of two convolution layers with a convolution kernel of 3*3, a stride of 1 and a padding size of 1, two activation layers and a residual connection.

[0056] Specifically, referring to Figure 7 As shown in the figure, it is an application stage flow chart of an embodiment of the present application, the application of the trained generator to repair and cover noise of GOCI marine water color images, the image to be repaired and denoised is input into the trained generator, and the generator outputs the repaired and denoised image.

[0057] Referring to Figure 8 As shown in the figure, it is a repair and denoising result comparison diagram of an embodiment of the present application. The (a) is an image to be repaired, the (b) is a position mark of an image missing area and noise, the (c) is an image processed by using the algorithm of the present application, and the (d) is a real complete image corresponding to the missing image.

[0058] It can be seen that the GOCI image repairing and denoising method based on the self-attention mechanism and the joint dilated convolution can well realize repairing and denoising of the GOCI image with missing and / or noise. The joint loss is used to control the recovery effect of the model; the self-attention mechanism is used to enhance the extraction ability of high-level semantic features; the self-attention connection is used between feature extraction and feature decoding to ensure that the features of the missing image can be effectively extracted in the feature extraction branch and more attention is paid to the recovery of high-level texture information in the feature decoding; in order to prevent the problem of network parameter explosion in feature extraction and feature decoding, residual connection is used for each level; the joint dilated convolution module is used in the upsampling stage to further enhance the complexity of multi-sampling, so as to improve the reconstruction ability of the network to noise and the learning ability of global semantics; the joint dilated convolution module is only used in the last layer of upsampling, which avoids the problem that the use of the joint dilated convolution module in each layer leads to too large parameter quantity and the network is difficult to train, and ensures that the image noise can be effectively removed and the image texture structure can be reconstructed under different spatial resolutions; the efficient repairing of the large-area missing image often appears the phenomenon of blur and artifact, which is caused by the inconsistency between the global and local texture structure of the image, and the inability to capture low-frequency texture information; in order to enhance the consistency of the global and local texture structure of the repaired image, the block-based adversarial network is used to realize the effective recovery of the local texture information, so that the local texture structure information of the image is consistent with the global texture structure information.

Claims

1. A GOCI image inpainting and denoising method based on self-attention mechanism and joint dilated convolution, characterized in that, The steps are as follows: S1, training stage, training generator; S2, application stage, using the trained generator to repair the GOCI image, inputting the GOCI image to be repaired and denoised into the generator, and outputting the repaired and denoised image by the generator; The S1 training stage is specifically: S11, selecting a training image and its original complete image, and inputting the training image into a data set to a generator; S12, an encoder in the generator discriminates the area with missing and noise in the training image, and extracts semantic information features of the training image; S13, a self-attention module in the generator extracts high-level semantic information features of the training image; S14, a decoder in the generator performs up-sampling on the extracted semantic information features, and uses a joint dilated convolution operation to refine the repair and denoising result in the last stage of the up-sampling, to form a preliminary result image; S15, the discriminator receives the preliminary result image output by the generator, combines the original complete image, calculates the adversarial loss, and returns to the generator; In S12, the encoder adopts five groups of structures, the first group is one layer of input and two layers of convolution, and the second to fifth groups are one layer of pooling and two layers of convolution; the convolution kernel size of the convolution layer of the first group is increased from 3 to 64, so that the number of semantic information features becomes 64, the semantic information features output by the first group of the encoder are stacked with the semantic information features output by the fourth group of the decoder, the semantic information features output by the second group, the third group and the fourth group of the encoder are stacked with the semantic information features output by the third group, the second group and the first group of the decoder respectively, and the size of the semantic information features output by the encoder is sequentially down-sampled from 224*224 to 112*112 and 56*56, and the convolution kernel size of the convolution layer is sequentially increased to 128 and 256; the fourth group and the fifth group perform maximum pooling and convolution processing on the semantic information features, the size of the semantic information features is down-sampled to 28*28, the convolution kernel size of the convolution layer is iterated to 1024, and 1024 feature maps with a size of 14*14 are output; The joint dilated convolution in S14 is composed of five convolution layers with different dilation rates, and the dilation rates are 1, 2, 4, 8 and 16 respectively; the convolution layer includes batch normalization and an activation function ReLU, the convolution kernel size of the convolution layer is 4, the stride is 2, and the padding size is 1; the joint dilated convolution obtains recovery features with larger spatial resolution by using convolution layers with different dilation rates, and obtains more global semantic information, which is helpful to the integrity of the repaired image and the global texture of the image; a residual mechanism is also designed between adjacent two convolution layers, so that the joint dilated convolution module is connected in a hierarchical manner; finally, the convolution results are superimposed, and the output result image is output after the activation function Sigmoid.

2. The GOCI image inpainting and denoising method based on self-attention mechanism and joint dilated convolution according to claim 1, characterized in that, The training image in S11 has three sub-images, which are a training image with only noise, a training image with only missing, and a training image with both noise and missing, and the image size is 224*224.

3. The GOCI image inpainting and denoising method based on self-attention mechanism and joint dilated convolution according to claim 1, characterized in that, In the S13, the self-attention module is connected to the end of the encoder, and the semantic information features output by the encoder are input into the self-attention module after the dimension of the semantic information features is changed, and the self-attention module extracts high-level semantic information features, specifically: the self-attention module effectively extracts the high-level semantic information features in the complete region of the training image, and effectively reasons the high-level semantic information features in the missing region of the training image.

4. The GOCI image inpainting and denoising method based on self-attention mechanism and joint dilated convolution according to claim 3, characterized in that, The self-attention module is provided with four self-attention heads; the self-attention module is provided with three linear layers to meet the input of the query feature Q, the key value feature K and the content feature V; the corresponding relationship mapping of the combination of the query feature Q and the key value feature K and the content feature V is output through the scaling dot multiplication operation, and after feature fusion, linear operation is performed in the linear layer; the attention weight between the query feature Q and the key value feature K is calculated, and then the content feature V is acted on to obtain the weight and output of the content feature V, and the weight is the degree of attention required by each region of the content feature V.

5. The GOCI image inpainting and denoising method based on self-attention mechanism and joint dilated convolution according to claim 1, characterized in that, The decoder in the S14 performs upsampling operation on the extracted semantic information features, and the decoder adopts five groups of structures, the first group outputs the semantic information features of the self-attention module through upsampling and convolution layers, and then performs stacking processing with the semantic information features output by the fourth group of the encoder, and then passes through a residual module to upsample the semantic information features of the image to 28*28; the second, third and fourth groups of the decoder also adopt stacking processing, and are stacked with the semantic information features output by the third, second and first groups of the encoder, respectively, so as to sequentially upsample the semantic information features of the image to 56*56, 112*112 and 224*224, so that each layer structure of the decoder can obtain the semantic information features output by the previous layer structure of the decoder and the semantic information features output by the encoder.

6. The GOCI image inpainting and denoising method based on self-attention mechanism and joint dilated convolution according to claim 1, characterized in that, The residual mechanism is composed of two convolution kernels with a size of 3*3, a stride of 1 and a padding size of 1, two activation layers and a residual connection.

7. The GOCI image inpainting and denoising method based on self-attention mechanism and joint dilated convolution according to claim 1, characterized in that, The S15 further comprises: calculating the pixel-level loss between the preliminary result image and its original complete image; inputting the preliminary result image and its original complete image into the VGG16 feature extractor, and the VGG16 feature extractor calculates the perceptual loss and the style loss between the features of the images; the pixel-level loss, the perceptual loss and the style loss are returned to the training generator.

8. The GOCI image inpainting and denoising method based on self-attention mechanism and joint dilated convolution according to claim 1, characterized in that, The discriminator in the S15 is a Markov discriminator type, which receives a preliminary result image, discriminates and processes to obtain an adversarial loss according to the preliminary result image and an original complete image, and further supervises the training of the generator; the Markov discriminator mainly consists of four convolutional layers, three activation functions LeakyReLU and one activation function Sigmoid layer; the convolution kernel size of the first three convolutional layers is 4, the stride is 2, the padding size is 1, the neurons are activated by using the activation function LeakyRelu, the activation function is set to a negative slope of 0.2, and the implace parameter is selected to perform an in-place operation; the convolution kernel size of the last convolutional layer is 4, the stride is 1, the padding size is 1, and the neurons are activated by using the activation function Sigmoid; finally, the preliminary result image is used for local discrimination, so that the effective recovery of the low-frequency local texture structure is ensured.

Images