A method for image edge and local consistency restoration based on prior semantics

Abstract

The present invention discloses a method for image edge and local consistency restoration based on prior semantics, comprising: using a training data set to train an image restoration model, and performing a reconstruction loss L g and adversarial loss L adv A weighted sum is performed to obtain the restoration loss L; the Adam optimizer is used to optimize the image restoration model according to the restoration loss L, and the iteration is repeated until the set number of iterations C is reached to output the trained image restoration model; a fitting image of the damaged image is generated through the pre-trained GAN model, and the fitting image is input into the pre-trained image restoration model to obtain the restoration image; the rationality and authenticity of the restoration content are ensured, and the restoration of edge parts and texture details is better completed.

Description

Technical Field

[0001] The present invention belongs to the technical field of image restoration, and in particular relates to an image edge and local consistency restoration method based on prior semantics. Background Art

[0002] Restoring missing image regions is a challenging problem in image processing and a key factor affecting the accuracy and robustness of visual tasks such as object recognition and tracking. Image restoration, a key underlying visual task, relies on the principle of predicting unknown information from known information in an image. It has widespread applications in various fields, such as restoring damaged drawings or photographs and removing objects in film and television special effects.

[0003] Based on the current research landscape, most inpainting methods still employ encoder-decoder designs, with hybrid frameworks combining CNNs and GANs being the mainstream. Furthermore, with the widespread use of coarse-to-fine models, multi-scale neural networks, and multiple discriminators, inpainting strategies are no longer simply end-to-end direct inpainting, but rather a multi-stage, multi-functional process. The entire inpainting framework is also complex, typically consisting of multiple neural networks.

[0004] Methods based on deep neural networks have certain advantages in extracting complex features, but there are still many problems, mainly including three aspects: (1) Lack of effective image prior knowledge. Since the content of the missing part of the image is uncertain, it will present a variety of restoration results. However, there is currently a lack of effective image prior knowledge to constrain the solution range, resulting in the restoration of images that lack rationality and authenticity. (2) It is difficult to utilize long-range image context information. Due to the limited receptive field, convolutional neural networks find it difficult to utilize long-range image features, so they cannot model the correlation between long-range context information and the missing area, and therefore usually generate blurred and distorted structures at the boundaries of the restoration area. (3) The restored image lacks local consistency. Although current deep learning models can fill in missing areas, they usually produce overly smooth results. Although they maintain global consistency, they lack local consistency. Global consistency is to make the content of the image restoration fit the context, and local consistency is to make the texture details of the local area richer. Summary of the Invention

[0005] The present invention provides an image edge and local consistency restoration method based on prior semantics, which ensures the rationality and authenticity of the restoration content and better completes the restoration of edge parts and texture details.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] A first aspect of the present invention provides an image edge and local consistency restoration method based on prior semantics, comprising:

[0008] Generate a fitting image of the damaged image through the pre-trained GAN model, and input the fitting image into the pre-trained image restoration model to obtain the restoration image;

[0009] The training process of the image restoration model includes:

[0010] Fix the pre-trained discriminator parameters, obtain the real image and multiply it with the binary mask representing the damaged area to obtain the damaged simulated image, which is constructed as the training data set;

[0011] The image restoration model is trained using the training data set, and the damaged simulated image is input into the image restoration model to obtain the restored simulated image. The reconstruction loss L is calculated based on the restored simulated image and the damaged simulated image. g ; The discriminant result of the pre-trained discriminator on the repaired simulated image is used to calculate the adversarial loss L based on the discriminant result adV ; For reconstruction loss L g and adversarial loss L adv Perform weighted summation to obtain the restoration loss L; use the Adam optimizer to optimize the image restoration model according to the restoration loss L, and repeat the iteration until the set number of iterations C is reached to output the trained image restoration model.

[0012] Preferably, the method of inputting the damaged simulated image into the image restoration model to obtain the restored simulated image includes:

[0013] Inputting the damaged simulated image into an image restoration model, wherein the image restoration model adopts a U-Net structure; the image restoration model includes a feature extraction network, an image reconstruction network, a dilated convolution and an attention module;

[0014] The damaged simulated image is input into the feature extraction network, and the feature map U is obtained by the kth extraction layer of the feature extraction network. k ;

[0015] The feature map U k Input into the dilated convolution and image reconstruction network in sequence, and the kth upsampling layer of the image reconstruction network outputs the feature map U′ k ;

[0016] The feature map U of the same size k and feature map U′ k At the same time, it is input into the kth attention module for feature fusion to obtain the feature map H k , fuse the features to obtain the feature map H k Input to the k+1th upsampling layer of the image reconstruction network to obtain the feature map U′ k+1 , repeat feature fusion until the simulated image is restored by the output of the last upsampling layer in the image reconstruction network.

[0017] Preferably, the training process of the GAN model includes:

[0018] The GAN model is divided into generator G1 and generator G2; the vector z randomly initialized by generator G1 n , and the feature map T′ formed after the generator G1 n ;

[0019] The feature map T′ n Input to the generator G2, which is the feature map T' n After assigning adaptive weights, feature fusion is performed to obtain the feature map x gt ; Calculate feature map x gt and the fitting loss of the corrupted image According to the fitting loss The parameters of the GAN model are optimized and iterated repeatedly until the set number of iterations A is reached to output the trained GAN model.

[0020] Preferably, the training process of the discriminator includes:

[0021] Fix the parameters of the image restoration model; input the real image into the image restoration model, and the image restoration model generates a virtual image; supervise the discriminator through the virtual image and the real image, and calculate the discriminant loss L of the discriminator through the binary cross entropy loss function s , according to the discriminant loss L s The parameters of the discriminator are optimized and iterated repeatedly until the set number of iterations B is reached to output the trained discriminator.

[0022] Preferably, the vector z randomly initialized by generator G1 is n , N feature maps T′ formed after generator G1 n The methods include:

[0023] The vector z randomly initialized by generator G1 n , N feature maps T formed after generator G1 n , for the feature map T n Add channel weights to get feature map T′ n , the expression formula is:

[0024] T′ n =G1(z n )⊙α n

[0025] In the formula, ⊙ represents the Hadamard operation; α n Expressed as channel weight.

[0026] Preferably, it is the characteristic graph T′n After assigning adaptive weights, feature fusion is performed to obtain the feature map x gt The calculation formula is:

[0027]

[0028] In the formula, β n Represented as feature map T′ n Adaptive weight of; N is represented by the feature map T′ n The number of

[0029] Preferably, the feature map x is calculated gt and the fitting loss of the corrupted image The methods include:

[0030] Calculate the feature map x using mean square error MSE gt and the pixel-level reconstruction error between the damaged image; the feature map x gt The distance between the perceptual features extracted from the damaged image input to the VGG16 network and the perceptual error is calculated;

[0031] The fitting loss is obtained by summing the pixel-level reconstruction error and the perceptual error The expression formula is:

[0032]

[0033] In the formula, Expressed as pixel-level reconstruction error; is the perceptual error; x is the damaged image, Represented as VGG16 network function.

[0034] Preferably, according to the fitting loss Methods for optimizing the parameters of the GAN model include:

[0035]

[0036] Among them, β represents the feature map T′ n The adaptive weight set of θ is represented by the parameters of the GAN model; z n , β, α n , θ are optimized respectively β * 、 and θ * .

[0037] Preferably, the reconstruction loss L is calculated based on the repaired simulated image and the damaged simulated image. g , the expression formula is:

[0038] L g =λ1||Igt ⊙MI out ⊙M||2+λ2||I gt ⊙(1-M)-I out ⊙(1-M)||2

[0039] I out =I gt ⊙M+G(I gt ⊙M)⊙(1-M)

[0040] In the formula, I gt represents the real image, I out It is represented as the repair simulation image, M is the binary mask map, G is the repair network function of the image repair model; λ1 and λ2 are the set influence coefficients.

[0041] A second aspect of the present invention provides an image edge and local consistency restoration system based on prior semantics, comprising:

[0042] The repair module is used to generate a fitting image of the damaged image through the pre-trained GAN model, and input the fitting image into the pre-trained image repair model to obtain the repaired image;

[0043] A construction module is used to fix the pre-trained discriminator parameters, obtain the real image, and multiply it with the binary mask representing the damaged area to obtain the damaged simulated image, which is constructed as the training data set;

[0044] The loss calculation module is used to train the image restoration model using the training data set, input the damaged simulated image into the image restoration model to obtain the restored simulated image, and calculate the reconstruction loss L based on the restored simulated image and the damaged simulated image. g ; The discriminant result of the pre-trained discriminator on the repaired simulated image is used to calculate the adversarial loss L based on the discriminant result adv ; For reconstruction loss L g and adversarial loss L adv Perform weighted summation to obtain the restoration loss L;

[0045] The optimization module is used to optimize the image restoration model according to the restoration loss L using the Adam optimizer, and repeat the iteration until the set number of iterations C is reached to output the trained image restoration model.

[0046] Preferably, the discriminator includes a local discriminator and a global discriminator, the local discriminator is used to discriminate the local restoration result of the restoration simulation image; the global discriminator is used to discriminate the global restoration result of the restoration simulation image.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] This paper uses the rich semantic information in the GAN model pre-trained on a large-scale dataset as the image prior, and the vector z randomly initialized by the generator G1 is n , and the feature map T′ formed after the generator G1 n ; The feature map T′ n Input to the generator G2, which is the feature map T' n After assigning adaptive weights, feature fusion is performed to obtain the feature map x gt , thereby more effectively mining the rich image semantic information in the pre-trained model, and using perceptual loss to ensure more detailed texture generation, which is conducive to enhancing the rationality and authenticity of the restoration results of the image restoration model.

[0049] The present invention reconstructs the loss L g and adversarial loss L adv A weighted summation is performed to obtain the restoration loss L; the Adam optimizer is used to optimize the image restoration model based on the restoration loss L, and a fitting image of the damaged image is generated through the pre-trained GAN model. The fitting image is then input into the pre-trained image restoration model to obtain the restored image; while ensuring the rationality and authenticity of the restoration content, the edge parts and texture details are better restored, achieving detail differences without losing the overall structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a structural diagram of the GAN model provided in Example 1;

[0051] Figure 2 This is a structural diagram of the image restoration model and discriminator provided in Example 1. DETAILED DESCRIPTION

[0052] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0053] Example 1

[0054] like Figures 1 to 2 As shown, this embodiment provides an image edge and local consistency restoration method based on prior semantics, including:

[0055] Methods for training GAN models include:

[0056] The GAN model is divided into generator G1 and generator G2; the vector z randomly initialized by generator G1 n , N feature maps T′ formed after generator G1 n The methods include:

[0057] The vector z randomly initialized by generator G1 n , N feature maps T formed after generator G1 n , for the feature map T n Add channel weights to get feature map T′ n , the expression formula is:

[0058] T′ n =G1(z n )⊙α n

[0059] In the formula, ⊙ represents the Hadamard operation; α n Expressed as channel weight.

[0060] is the feature graph T′ n After assigning adaptive weights, feature fusion is performed to obtain the feature map x gt The calculation formula is:

[0061]

[0062] In the formula, β n Represented as feature map T′ n Adaptive weight of; N is represented by the feature map T′ n The number of

[0063] Calculate feature map x gt and the fitting loss of the corrupted image The methods include:

[0064] Calculate the feature map x using mean square error MSE gt and the pixel-level reconstruction error between the damaged image; the feature map x gt The distance between the perceptual features extracted from the damaged image input to the VGG16 network and the perceptual error is calculated;

[0065] The fitting loss is obtained by summing the pixel-level reconstruction error and the perceptual error The expression formula is:

[0066]

[0067] In the formula, Expressed as pixel-level reconstruction error; is the perceptual error; x is the damaged image, Represented as VGG16 network function.

[0068] According to the fitting loss Optimize the parameters of the GAN model and repeat the iteration until the set number of iterations A is reached to output the trained GAN model. The expression formula is:

[0069]

[0070] Among them, β represents the feature map T′ n The adaptive weight set of θ is represented by the parameters of the GAN model; z n , β, α n , θ are optimized respectively β * 、 and θ * .

[0071] The methods for training the discriminator first include:

[0072] The parameters of the image restoration model are fixed; a real image is obtained and input into the image restoration model, and a virtual image is generated by the image restoration model; the discriminator is supervised and trained by the virtual image and the real image, and the discriminator includes a local discriminator and a global discriminator. The local discriminator is used to discriminate the local restoration result of the restoration simulation image; the global discriminator is used to discriminate the overall restoration result of the restoration simulation image; the discriminant loss L of the discriminator is calculated by the binary cross entropy loss function s , according to the discriminant loss L s The parameters of the discriminator are optimized and iterated repeatedly until the set number of iterations B is reached to output the trained discriminator.

[0073] Then the method of training the image restoration model includes:

[0074] Fix the pre-trained discriminator parameters, multiply the real image with the binary mask representing the damaged area to obtain the damaged simulated image, and construct it as the training data set;

[0075] The method of training an image restoration model using a training data set and inputting a damaged simulated image into the image restoration model to obtain a restored simulated image includes:

[0076] Inputting the damaged simulated image into an image restoration model, wherein the image restoration model adopts a U-Net structure; the image restoration model includes a feature extraction network, an image reconstruction network, a dilated convolution and an attention module;

[0077] The damaged simulated image is input into the feature extraction network, and the feature map U is obtained by the kth extraction layer of the feature extraction network. k ;

[0078] The feature map U k Input into the dilated convolution and image reconstruction network in sequence, and the kth upsampling layer of the image reconstruction network outputs the feature map U′ k ;

[0079] The feature map U of the same sizek and feature map U′ k At the same time, it is input into the kth attention module for feature fusion to obtain the feature map H k , fuse the features to obtain the feature map H k Input to the k+1th upsampling layer of the image reconstruction network to obtain the feature map U′ k+1 , repeat feature fusion until the simulated image is restored by the output of the last upsampling layer in the image reconstruction network.

[0080] Calculate the reconstruction loss L based on the repaired simulated image and the damaged simulated image g , the expression formula is:

[0081] L g =λ1||I gt ⊙MI out ⊙M||2+λ2||I gt ⊙(1-M)-I out ⊙(1-M)||2

[0082] I out =I gt ⊙M+G(I gt ⊙M)⊙(1-M)

[0083] In the formula, I gt represents the real image, I out It is represented as the repair simulation image, M is the binary mask map, G is the repair network function of the image repair model; λ1 and λ2 are the set influence coefficients.

[0084] The pre-trained discriminator is used to judge the repaired simulated image and the adversarial loss L is calculated based on the judgment result. adv ; For reconstruction loss L g and adversarial loss L adv Perform weighted summation to obtain the restoration loss L; use the Adam optimizer to optimize the image restoration model according to the restoration loss L, and repeat the iteration until the set number of iterations C is reached to output the trained image restoration model.

[0085] A fitting image of the damaged image is generated through the pre-trained GAN model, and the fitting image is input into the pre-trained image restoration model to obtain the restored image; while ensuring the rationality and authenticity of the restoration content, the edge parts and texture details are better restored, achieving detail differences without losing the overall structure.

[0086] Example 2

[0087] This embodiment provides an image edge and local consistency restoration system based on prior semantics. The system in this embodiment can be applied to the method described in Example 1. The system includes:

[0088] The repair module is used to generate a fitting image of the damaged image through the pre-trained GAN model, and input the fitting image into the pre-trained image repair model to obtain the repaired image;

[0089] A construction module is used to fix the pre-trained discriminator parameters, obtain the real image, and multiply it with the binary mask representing the damaged area to obtain the damaged simulated image, which is constructed as the training data set;

[0090] The loss calculation module is used to train the image restoration model using the training data set, input the damaged simulated image into the image restoration model to obtain the restored simulated image, and calculate the reconstruction loss L based on the restored simulated image and the damaged simulated image. g ; The discriminant result of the pre-trained discriminator on the repaired simulated image is used to calculate the adversarial loss L based on the discriminant result adv ; For reconstruction loss L g and adversarial loss L adv Perform weighted summation to obtain the restoration loss L;

[0091] The optimization module is used to optimize the image restoration model according to the restoration loss L using the Adam optimizer, and repeat the iteration until the set number of iterations C is reached to output the trained image restoration model.

[0092] Specifically, the discriminator includes a local discriminator and a global discriminator. The local discriminator is used to discriminate the local restoration result of the restoration simulation image, thereby enhancing the local consistency of the restoration simulation image; the overall discriminator is used to discriminate the overall restoration result of the restoration simulation image.

[0093] Example 3

[0094] This embodiment provides a computer-readable storage medium, characterized in that a computer program is stored thereon, and when the program is executed by a processor, the steps of the target posture positioning method described in Example 1 are implemented.

[0095] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0096] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0097] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0098] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0099] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for image edge and local consistency restoration based on prior semantics, characterized in that: include: Generate a fitting image of the damaged image through the pre-trained GAN model, and input the fitting image into the pre-trained image restoration model to obtain the restoration image; The training process of the image restoration model includes: Fix the pre-trained discriminator parameters, obtain the real image and multiply it with the binary mask representing the damaged area to obtain the damaged simulated image, which is constructed as the training data set; The image restoration model is trained using the training data set, and the damaged simulated image is input into the image restoration model to obtain the restored simulated image, specifically including: Inputting the damaged simulated image into an image restoration model, wherein the image restoration model adopts a U-Net structure; the image restoration model includes a feature extraction network, an image reconstruction network, a dilated convolution and an attention module; The damaged simulated image is input into the feature extraction network, and the feature map is obtained by the kth extraction layer of the feature extraction network. ; The feature map Input into the dilated convolution and image reconstruction network in sequence, and the kth upsampling layer of the image reconstruction network outputs the feature map ; The feature maps of the same size and feature maps At the same time, it is input into the kth attention module for feature fusion to obtain the feature map , the features are fused to obtain the feature map Input to the k+1th upsampling layer of the image reconstruction network to obtain the feature map , repeat feature fusion until the simulated image is restored by the output of the last upsampling layer in the image reconstruction network; Calculate the reconstruction loss based on the repaired simulated image and the damaged simulated image ; The pre-trained discriminator is used to judge the repaired simulated image and the adversarial loss is calculated based on the judgment result. ; Reconstruction losses and combat losses Perform weighted summation to obtain the restoration loss L; use the Adam optimizer to optimize the image restoration model according to the restoration loss L, and repeat the iteration until the set number of iterations C is reached to output the trained image restoration model.

2. The image edge and local consistency restoration method based on prior semantics according to claim 1, characterized in that: The training process of the GAN model includes: Splitting the GAN model into generators and generators ; by the generator Randomly initialized vector , and through the generator The feature map formed later ; The feature map Input to the generator In the middle, it is the feature map After assigning adaptive weights, feature fusion is performed to obtain feature maps ; Calculate feature map and the fitting loss of the corrupted image ; According to the fitting loss The parameters of the GAN model are optimized and iterated repeatedly until the set number of iterations A is reached to output the trained GAN model.

3. The image edge and local consistency restoration method based on prior semantics according to claim 1, characterized in that: The training process of the discriminator includes: Fix the parameters of the image restoration model; input the real image into the image restoration model, and the image restoration model generates a virtual image; supervise the discriminator through virtual and real images, and calculate the discriminant loss of the discriminator through the binary cross entropy loss function , according to the discriminant loss The parameters of the discriminator are optimized and iterated repeatedly until the set number of iterations B is reached to output the trained discriminator.

4. The image edge and local consistency restoration method based on prior semantics according to claim 2, characterized in that: By the generator Randomly initialized vector , through the generator The N feature maps formed later The methods include: By the generator Randomly initialized vector , through the generator The N feature maps formed later , for the feature map Add channel weights to get feature maps , the expression formula is: ； In the formula, It is expressed as Hadamard operation; Expressed as channel weight.

5. The image edge and local consistency restoration method based on prior semantics according to claim 4, characterized in that: is the feature map After assigning adaptive weights, feature fusion is performed to obtain feature maps The calculation formula is: ； In the formula, Represented as a feature map Adaptive weight; N represents the feature map The number of 6. The image edge and local consistency restoration method based on prior semantics according to claim 2, characterized in that: Calculating feature maps and the fitting loss of the corrupted image The methods include: Calculate feature maps using mean square error MSE and the pixel-level reconstruction error between the damaged image; the feature map The distance between the perceptual features extracted from the damaged image input to the VGG16 network and the perceptual error is calculated; The fitting loss is obtained by summing the pixel-level reconstruction error and the perceptual error , the expression formula is: ； In the formula, Expressed as pixel-level reconstruction error; is the perceptual error; x is the damaged image, Represented as VGG16 network function.

7. The image edge and local consistency restoration method based on prior semantics according to claim 6, characterized in that: According to the fitting loss Methods for optimizing the parameters of the GAN model include: ； in, Represented as a feature map The adaptive weight set of Represented as parameters of the GAN model; 、 、 、 After optimization, they are 、 、 and ; Expressed as channel weight.

8. The image edge and local consistency restoration method based on prior semantics according to claim 1, characterized in that: Calculate the reconstruction loss based on the repaired simulated image and the damaged simulated image , the expression formula is: ； ； In the formula, represents the real image, Represented as the repaired simulated image, is a binary mask image, The restoration network function of the image restoration model; , is the influence coefficient of the setting; Represented as a Hadamard operation.

9. The application system of the image edge and local consistency restoration method based on prior semantics according to any one of claims 1 to 8, characterized in that: include: The repair module is used to generate a fitting image of the damaged image through the pre-trained GAN model, and input the fitting image into the pre-trained image repair model to obtain the repaired image; A construction module is used to fix the pre-trained discriminator parameters, obtain the real image, and multiply it with the binary mask representing the damaged area to obtain the damaged simulated image, which is constructed as the training data set; The loss calculation module is used to train the image restoration model using the training data set, input the damaged simulated image into the image restoration model to obtain the restored simulated image, and calculate the reconstruction loss based on the restored simulated image and the damaged simulated image. ; The pre-trained discriminator is used to judge the repaired simulated image and the adversarial loss is calculated based on the judgment result. ; Reconstruction losses and combat losses Perform weighted summation to obtain the restoration loss L; The optimization module is used to optimize the image restoration model using the Adam optimizer according to the restoration loss L, and repeat the iteration until the set number of iterations C is reached to output the trained image restoration model; The loss calculation module inputs the damaged simulated image into the image restoration model to obtain the restored simulated image, specifically including: Inputting the damaged simulated image into an image restoration model, wherein the image restoration model adopts a U-Net structure; The image restoration model includes a feature extraction network, an image reconstruction network, a dilated convolution and an attention module; The damaged simulated image is input into the feature extraction network, and the feature map is obtained by the kth extraction layer of the feature extraction network. ; The feature map Input into the dilated convolution and image reconstruction network in sequence, and the kth upsampling layer of the image reconstruction network outputs the feature map ; The feature maps of the same size and feature maps At the same time, it is input into the kth attention module for feature fusion to obtain the feature map , the features are fused to obtain the feature map Input to the k+1th upsampling layer of the image reconstruction network to obtain the feature map , repeat feature fusion until the simulated image is restored by the output of the last upsampling layer in the image reconstruction network.

Images