Image Inpainting Method Based on Multi-Fusion Attention Module and Deep Image Prior

Abstract

The present disclosure relates to an image restoration method, apparatus, medium and electronic device, including: generating a binary image of the same type as the image to be restored according to the damaged area and the undamaged area in the image to be restored, wherein the damaged area has the same value in the binary image, and the undamaged area has the same value in the binary image and is different from the value of the damaged area; generating a normal random noise image of the same type as the image to be restored, and calculating the Hadamard product of the binary image and the image to be restored to obtain a damaged image; iteratively adjusting the parameters of the image restoration model according to the normal random noise image, the binary image and the damaged image, and taking the iterative image output when the image restoration model finishes iteration as the target restored image. This method does not require sample annotation and model training, etc., reduces the cost of image restoration, and can improve the accuracy of image restoration.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image data processing, and specifically, to an image restoration method based on a multi-fusion attention module and a deep image prior. Background Art

[0002] Image restoration can supplement damaged or missing areas in an image. In traditional algorithms, by calculating the similarity between the missing area and other areas, the similar information of the image is obtained, and then the damaged or missing area is restored according to the similar information. However, if there is no similar or corresponding other similar image domain for the damaged or missing area of the image, the restoration cannot be completed.

[0003] In related technologies, an image restoration method based on channel learning has emerged. This method performs a composite linear fitting on the data of the image through the neuron parameters in the neural network, and then calculates the data content of the damaged or missing area with the composite function after fitting. However, this method requires using a carefully crafted paired dataset to train the image restoration model. There is a one-to-one correspondence between a damaged (or missing) image and an original image in the paired dataset. First, the cost of model training is high. Second, the cost of making the paired dataset is high. It not only requires collecting images but also pairing the data. In addition, the number of images is limited and usually cannot cover all the prior information in the real world. Another important point is that the image restoration model trained with a specific dataset is more inclined to cater to the prior information of the existing dataset when restoring unfamiliar images, rather than focusing on obtaining the prior information of the damaged image itself. When restoring images in a specific professional field, there is a problem of low accuracy of the restored images. Summary of the Invention

[0004] The purpose of the present disclosure is to provide an image restoration method based on a multi-fusion attention module and a deep image prior, aiming to reduce the cost of image restoration and improve the accuracy of image restoration.

[0005] To achieve the above purpose, in the first aspect of the embodiments of the present disclosure, an image restoration method is provided, and the method includes:

[0006] Generate a binary image of the same type as the image to be restored according to the damaged area and the undamaged area in the image to be restored, where the damaged area has the same value in the binary image, and the undamaged area has the same value in the binary image and is different from the value of the damaged area;

[0007] Generate a normal random noise image of the same type as the image to be restored, and calculate the Hadamard product of the binary image and the image to be restored to obtain a damaged image;

[0008] Iteratively adjust the parameters of the image restoration model according to the normal random noise image, the binary image, and the damaged image, and use the iterative image output when the iteration of the image restoration model is completed as the target restored image.

[0009] Optionally, the step of iteratively adjusting the parameters of the image restoration model according to the normal random noise image, the binary image, and the damaged image, and using the iterative image output when the iteration of the image restoration model is completed as the target restored image includes:

[0010] Input the normal random noise image into the image restoration model to obtain the iterative image output by the image restoration model;

[0011] Determine the loss value of the current iteration according to the iterative image, the binary image, and the damaged image;

[0012] Backpropagate the loss value to adjust the parameters of the image restoration model;

[0013] Iteratively input the normal random noise image into the image restoration model after the current parameter adjustment, and determine the loss value of the current iteration according to the binary image, the damaged image, and the iterative image output by the image restoration model after the current parameter adjustment. Backpropagate the loss value of the current iteration to adjust the parameters of the image restoration model until the iteration is completed, and use the iterative image output when the iteration of the image restoration model is completed as the target restored image.

[0014] Optionally, the image restoration model outputs the iterative image in the following manner:

[0015] Perform downsampling convolution on the normal random noise image according to a preset convolution kernel and a preset convolution stride to obtain a downsampled noise image;

[0016] Perform multi-level multi-perceptual attention three-dimensional convolution calculation on the downsampled noise image in the encoder to obtain an encoded image, where each level of multi-perceptual attention three-dimensional convolution includes one multi-perceptual attention calculation and one three-dimensional convolution calculation, and each multi-perceptual attention calculation includes spatial attention feature extraction, channel attention feature extraction, and pooling-atrous convolution calculation;

[0017] Perform multi-level multi-perceptual attention two-dimensional convolution calculation on the encoded image in the decoder to obtain a decoded image, where each level of multi-perceptual attention two-dimensional convolution includes one multi-perceptual attention calculation and one two-dimensional convolution calculation;

[0018] In the upsampling layer, perform multi-level upsampling convolution calculations on the decoded image to obtain an iterative image of the same type as the normal random noise image, where each level of upsampling convolution includes alternating calculations of upsampling and convolution, and finally multiple convolution calculations are performed.

[0019] Optionally, the step of performing multi-level multi-sensory attention three-dimensional convolution calculations on the downsampled noise image in the encoder to obtain an encoded image includes:

[0020] Perform spatial attention feature extraction, channel attention feature extraction, and pooling-atrous convolution calculations on the output of the previous-level multi-sensory attention three-dimensional convolution calculation respectively to obtain the spatial attention feature image, channel attention feature image, and pooling-atrous convolution image corresponding to this level;

[0021] Perform weighted summation on the spatial attention feature image, the channel attention feature image, and the pooling-atrous convolution image to obtain a multi-sensory attention image;

[0022] Perform three-dimensional convolution calculation on the multi-sensory attention image to obtain the encoded image.

[0023] Optionally, the channel attention feature image is obtained by performing channel attention feature extraction on the input image in the following manner:

[0024] Perform deformation processing on the input image in the channel direction of the input image to obtain a planar deformed image, and perform transpose processing on the planar deformed image to obtain a planar transposed image;

[0025] Calculate the product of the planar deformed image and the planar transposed image to obtain a first backup image;

[0026] Determine the maximum value of each row in the first backup image row by row, and assign other values in that row to the maximum value corresponding to that row to obtain a maximum value matrix feature image;

[0027] Calculate the difference between the maximum value matrix feature image and the first backup image to obtain a second backup image;

[0028] Calculate the product of the planar deformed image and the second backup image to obtain a third backup image;

[0029] Perform deformation processing on the third backup image again to obtain a target image of the same type as the input image;

[0030] Sum the input image and the target image to obtain the channel attention feature image.

[0031] Optionally, the pooling-atrous convolution image is obtained by performing pooling-atrous convolution calculation on the input image as follows:

[0032] Perform mean pooling calculation, max pooling calculation, and atrous convolution calculations with the same convolutional kernels but different dilation rates on the input image to obtain a mean pooling image, a max pooling image, and atrous convolution images corresponding to different dilation rates;

[0033] Perform convolution calculation and upsampling calculation on the mean pooling image and the max pooling image in sequence to obtain an upsampled mean pooling image and an upsampled max pooling image;

[0034] Sum the upsampled mean pooling image and the upsampled max pooling image to obtain a fourth alternative image;

[0035] Stitch the fourth alternative image and the atrous convolution images corresponding to different dilation rates to obtain the pooling-atrous convolution image.

[0036] Optionally, the step of determining the loss value of the current iteration according to the iterative image, the binary image, and the damaged image includes:

[0037] Calculate the Hadamard product of the iterative image and the binary image to obtain a Hadamard matrix image.

[0038] Determine the loss value of the current iteration according to the damaged image and the Hadamard matrix image.

[0039] In a second aspect of the embodiments of the present disclosure, an image restoration device is provided, and the device includes:

[0040] A generation module configured to generate a binary image of the same type as the image to be restored according to the damaged area and the undamaged area in the image to be restored, wherein the damaged area has the same value in the binary image, and the undamaged area has the same value in the binary image and is different from the value of the damaged area;

[0041] A calculation module configured to generate a normal random noise image of the same type as the image to be restored, and calculate the Hadamard product of the binary image and the image to be restored to obtain a damaged image;

[0042] An iteration module configured to iteratively adjust the parameters of the image restoration model according to the normal random noise image, the binary image, and the damaged image, and use the iterative image output when the image restoration model is iteratively completed as the target restored image.

[0043] Optionally, the iteration module includes:

[0044] An input sub-module, configured to input the normal random noise image into the image inpainting model to obtain an iterative image output by the image inpainting model;

[0045] A determination sub-module, configured to determine the loss value of the current iteration according to the iterative image, the binary image, and the damaged image;

[0046] A backpropagation sub-module, configured to backpropagate the loss value to adjust the parameters of the image inpainting model;

[0047] An iteration sub-module, configured to iteratively input the normal random noise image into the image inpainting model after the current parameter adjustment, determine the loss value of the current iteration according to the binary image, the damaged image, and the iterative image output by the image inpainting model after the current parameter adjustment, backpropagate the loss value of the current iteration, and adjust the parameters of the image inpainting model until the iteration is completed, and use the iterative image output when the image inpainting model iteration is completed as the target inpainted image.

[0048] Optionally, the image inpainting model is configured to output an iterative image in the following manner:

[0049] Perform downsampling convolution on the normal random noise image according to a preset convolution kernel and a preset convolution stride to obtain a downsampled noise image;

[0050] Perform multi-level multi-perceptual attention three-dimensional convolution calculation on the downsampled noise image in an encoder to obtain an encoded image, where each level of multi-perceptual attention three-dimensional convolution includes one multi-perceptual attention calculation and one three-dimensional convolution calculation, and each multi-perceptual attention calculation includes spatial attention feature extraction, channel attention feature extraction, and pooling-atrous convolution calculation;

[0051] Perform multi-level multi-perceptual attention two-dimensional convolution calculation on the encoded image in a decoder to obtain a decoded image, where each level of multi-perceptual attention two-dimensional convolution includes one multi-perceptual attention calculation and one two-dimensional convolution calculation;

[0052] Perform multi-level upsampling convolution calculation on the decoded image in an upsampling layer to obtain an iterative image of the same type as the normal random noise image, where each level of upsampling convolution includes alternating calculation of upsampling and convolution, and finally passes through multiple convolution calculations.

[0053] Optionally, the image inpainting model is configured to:

[0054] Perform spatial attention feature extraction, channel attention feature extraction, and pooling-atrous convolution calculation on the outputs of the three-dimensional convolution calculation of the upper-level multi-sensory attention respectively, to obtain the spatial attention feature image, channel attention feature image, and pooling-atrous convolution image corresponding to this level respectively;

[0055] Perform weighted summation on the spatial attention feature image, the channel attention feature image, and the pooling-atrous convolution image to obtain a multi-sensory attention image;

[0056] Perform three-dimensional convolution calculation on the multi-sensory attention image to obtain the encoded image.

[0057] Optionally, the image inpainting model is configured to perform channel attention feature extraction on the input image to obtain the channel attention feature image in the following manner:

[0058] Perform deformation processing on the input image in the channel direction of the input image to obtain a planar deformed image, and perform transpose processing on the planar deformed image to obtain a planar transposed image;

[0059] Calculate the product of the planar deformed image and the planar transposed image to obtain a first standby image;

[0060] Determine the maximum value of each row in the first standby image row by row, and assign all other values in that row to the maximum value corresponding to that row to obtain a maximum value matrix feature image;

[0061] Calculate the difference between the maximum value matrix feature image and the first standby image to obtain a second standby image;

[0062] Calculate the product of the planar deformed image and the second standby image to obtain a third standby image;

[0063] Perform deformation processing on the third standby image again to obtain a target image of the same type as the input image;

[0064] Sum the input image and the target image to obtain the channel attention feature image.

[0065] Optionally, the image inpainting model is configured to perform pooling-atrous convolution calculation on the input image to obtain the pooling-atrous convolution image in the following manner:

[0066] Perform average pooling calculation, max pooling calculation, and atrous convolution calculations with the same convolution kernels but different dilation rates on the input image respectively, to obtain an average pooling image, a max pooling image, and atrous convolution images corresponding to different dilation rates;

[0067] Perform convolutional calculations and upsampling calculations on the mean-pooled image and the max-pooled image in sequence to obtain an upsampled mean-pooled image and an upsampled max-pooled image;

[0068] Sum the upsampled mean-pooled image and the upsampled max-pooled image to obtain a fourth backup image;

[0069] Stitch the fourth backup image and the atrous convolution images corresponding to different atrous rates to obtain the pooled-atrous convolution image.

[0070] Optionally, the determining sub-module is configured to:

[0071] Calculate the Hadamard product of the iterative image and the binary image to obtain a Hadamard matrix image.

[0072] Determine the loss value of the current iteration according to the damaged image and the Hadamard matrix image.

[0073] In a third aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the image restoration method described in any item of the first aspect are implemented.

[0074] In a fourth aspect of the embodiments of the present disclosure, there is provided an electronic device, including:

[0075] A memory, on which a computer program is stored;

[0076] A processor, configured to execute the computer program in the memory to implement the steps of the image restoration method described in any item of the first aspect.

[0077] Through the above technical solutions, at least the following beneficial effects can be achieved:

[0078] Perform separate learning and restoration on each image to be restored, without the need to collect and label training samples, nor to perform model training, reducing the cost of image restoration. Moreover, since the model does not depend on a large amount of data sets, no additional image prior information will be introduced. When restoring an image, by obtaining the prior information of a single degraded image to restore the damaged part of the image, the ethical issues faced are much smaller than those of a model trained relying on big data. It not only improves the accuracy and restoration ability of image restoration, but also has better ethical interpretability.

[0079] Other features and advantages of the present disclosure will be described in detail in the subsequent specific implementation part. Description of the Drawings

[0080] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure, but do not constitute a limitation to the present disclosure. In the accompanying drawings:

[0081] Figure 1 is a flowchart of an image inpainting method shown according to an exemplary embodiment.

[0082] Figure 2 is an implementation shown according to an exemplary embodiment Figure 1 of the flowchart of step S13 in

[0083] Figure 3 is a flowchart of obtaining an iterative image by an image inpainting model shown according to an exemplary embodiment.

[0084] Figure 4 is a schematic diagram of obtaining an iterative image by an image inpainting model shown according to an exemplary embodiment.

[0085] Figure 5 is an implementation shown according to an exemplary embodiment Figure 3 of the flowchart of step S32 in

[0086] Figure 6 is a schematic diagram of a multi-sensory attention module shown according to an exemplary embodiment.

[0087] Figure 7 is a schematic diagram of determining a maximum matrix feature image shown according to an exemplary embodiment.

[0088] Figure 8 is a comparison schematic diagram of a to-be-inpainted image and a target inpainted image shown according to an exemplary embodiment.

[0089] Figure 9 is a comparison schematic diagram of a to-be-inpainted image and a target inpainted image shown according to an exemplary embodiment.

[0090] Figure 10 is a comparison schematic diagram of a to-be-inpainted image and a target inpainted image shown according to an exemplary embodiment.

[0091] Figure 11 is a comparison schematic diagram of a to-be-inpainted image and a target inpainted image shown according to an exemplary embodiment.

[0092] Figure 12 is a comparison schematic diagram of a to-be-inpainted image and a target inpainted image shown according to an exemplary embodiment.

[0093] Figure 13It is a comparison schematic diagram of a to-be-restored image and a target restored image shown according to an exemplary embodiment.

[0094] Figure 14 It is a comparison schematic diagram of a to-be-restored image and a target restored image shown according to an exemplary embodiment.

[0095] Figure 15 It is a comparison schematic diagram of a to-be-restored image and a target restored image shown according to an exemplary embodiment.

[0096] Figure 16 It is a block diagram of an image restoration device shown according to an exemplary embodiment.

[0097] Figure 17 It is a block diagram of an electronic device for image restoration shown according to an exemplary embodiment.

[0098] Figure 18 It is a block diagram of an electronic device for image restoration shown according to an exemplary embodiment. Detailed implementation manners

[0099] The following will describe the detailed implementation manners of the present disclosure with reference to the accompanying drawings. It should be understood that the specific implementation manners described herein are only used to illustrate and explain the present disclosure, and are not used to limit the present disclosure.

[0100] It should be noted that all actions of obtaining signals, information, or data in the present disclosure are carried out on the premise of complying with the corresponding data protection regulations and policies of the country where it is located and obtaining authorization from the owner of the corresponding device.

[0101] The image restoration method provided by the embodiments of the present disclosure can be applied to a mobile terminal, such as a smart phone or a digital camera, and can also be applied to a cloud server. Refer to Figure 1 As shown, the method includes:

[0102] In step S11, according to the damaged area and the undamaged area in the to-be-restored image, a binary image of the same type as the to-be-restored image is generated.

[0103] Among them, the damaged areas have the same value in the binary image, and the undamaged areas have the same value in the binary image and are different from the value of the damaged areas.

[0104] It can be explained that the to-be-restored image in the present disclosure is an image obtained by converting a grayscale image or an RGB image into a matrix. The conversion methods are all prior arts and will not be elaborated herein. Therefore, except for the finally obtained target restored image, all other images mentioned in the present disclosure hereinafter are numerical matrix images.

[0105] It is understandable that matrix isomorphism means that the number of rows and columns of a matrix is the same. That is to say, a binary image is actually a matrix, and its number of rows and columns is the same as that of the image to be repaired. In the embodiments of the present disclosure, a mask matrix of the same type as the image to be repaired can be constructed, and then the mask matrix is overlapped with the image to be repaired. The same value is assigned to the row and column positions in the mask matrix corresponding to the damaged area of the image to be repaired, and the same another value is assigned to the row and column positions in the mask matrix corresponding to the undamaged area of the image to be repaired. Then, a binary image is generated according to the values of the rows and columns. In this way, the damaged part and the undamaged part are distinguished by different values of the mask. For example, the part with a mask of 1 in the binary image represents the undamaged area, and the part with a mask of 0 represents the damaged area.

[0106] In step S12, a normal random noise image of the same type as the image to be repaired is generated, and the Hadamard product of the binary image and the image to be repaired is calculated to obtain a damaged image.

[0107] It is easy to understand that the normal random noise image is a matrix that conforms to the normal distribution and has the same number of rows and columns as the image to be repaired. The Hadamard product is to multiply the values at the same positions of different matrices, and the new matrix obtained is the damaged image.

[0108] In step S13, according to the normal random noise image, the binary image, and the damaged image, the parameters of the image restoration model are iteratively adjusted, and the iterative image output when the image restoration model finishes iteration is used as the target restored image.

[0109] Among them, the completion of iteration can be either that the number of times of adjusting the parameters of the image restoration model reaches a preset number of times, or that the parameters change stably after multiple adjustments before and after.

[0110] The above technical solution performs separate learning and restoration on each image to be repaired, does not require collection and annotation of training samples, nor model training, reducing the cost of image restoration. And since the model does not depend on a large amount of data sets, no additional image prior information is introduced. When repairing an image, by obtaining the prior information of a single degraded image to repair the damaged part of the image, the ethical issues it faces are much smaller than those of a model trained relying on big data. It not only improves the accuracy and restoration ability of image restoration, but also has better ethical interpretability.

[0111] Optionally, as shown in Figure 2 In step S13, the step of iteratively adjusting the parameters of the image restoration model according to the normal random noise image, the binary image, and the damaged image, and using the iterative image output when the image restoration model finishes iteration as the target restored image includes:

[0112] In step S131, the normal random noise image is input into the image inpainting model to obtain the iterative image output by the image inpainting model.

[0113] In step S132, based on the iterative image, the binary image, and the damaged image, the loss value of this iteration is determined.

[0114] In this step, the Hadamard product of the iterative image and the binary image is calculated to obtain the Hadamard matrix image, and then based on the damaged image and the Hadamard matrix image, the loss value of this iteration is determined.

[0115] It can be understood that the loss function corresponding to the image inpainting model calculates the difference value between the Hadamard matrix image and the damaged image to obtain the loss value of this iteration.

[0116] In step S133, the loss value is backpropagated to adjust the parameters of the image inpainting model.

[0117] In step S134, iteratively input the normal random noise image into the image inpainting model after this parameter adjustment, and based on the binary image, the damaged image, and the iterative image output by the image inpainting model after this parameter adjustment, determine the loss value of this iteration. Backpropagate the loss value of this iteration to adjust the parameters of the image inpainting model until the iteration is completed. Take the iterative image output when the image inpainting model iteration is completed as the target inpainted image.

[0118] It can be understood that according to the loss value, the parameters of the encoder and decoder in the image inpainting model are adjusted based on the gradient. And after each adjustment of the parameters of the image inpainting model, the normal random noise image is input again to obtain the iterative image of this iteration.

[0119] Similarly, in this step, calculate the Hadamard product of the iterative image output by the image inpainting model after this parameter adjustment and the binary image to obtain the Hadamard matrix image corresponding to this iteration. Then, based on the damaged image and the Hadamard matrix image corresponding to this iteration, determine the loss value of this iteration.

[0120] Optionally, as shown in Figure 3 The image inpainting model obtains and outputs the iterative image through the following steps.

[0121] In step S31, according to the preset convolution kernel and the preset convolution stride, perform downsampling convolution on the normal random noise image to obtain the downsampled noise image.

[0122] For example, the normal random noise image is , where , where a represents the number of rows of the matrix of the normal random noise image, b represents the number of columns of the matrix of the normal random noise image, and c represents the number of channels of the matrix of the normal random noise image. In the case where the image to be repaired is a grayscale image, c can be 1, that is, downsampling convolution is performed only in two dimensions. In the case where the image to be repaired is an RGB image, c can be an integer greater than 1, and then downsampling convolution is performed in three dimensions. For example, downsampling convolution is performed on the normal random noise image according to a convolution kernel of size 3×3 and a convolution stride of 4 to obtain a downsampled noise image .

[0123] In step S32, multi-level multi-perceptual attention three-dimensional convolution calculation is performed on the downsampled noise image in the encoder to obtain an encoded image.

[0124] Among them, each level of multi-perceptual attention three-dimensional convolution includes one multi-perceptual attention calculation and one three-dimensional convolution calculation, and each multi-perceptual attention calculation includes spatial attention feature extraction, channel attention feature extraction, and pooling-atrous convolution calculation.

[0125] In the embodiment of the present disclosure, referring to Figure 4 as shown, the encoder can be set to multiple layers. The first-level multi-perceptual attention (Multi-Receptive Attention Module, abbreviated as MRAM) calculation is performed in the first layer of the encoder, and the first-level three-dimensional convolution is performed in the second layer of the encoder. Then, the remaining multi-level multi-perceptual attention three-dimensional convolution calculations are respectively performed in other layers of the encoder in sequence. Exemplarily, the downsampled noise image is input into the first layer of the encoder, and one multi-perceptual attention calculation is performed to obtain a perceptual attention image , in the second layer of the encoder, a three-dimensional convolution calculation is performed on the perceptual attention image , where the convolution stride of this time can be the same as the preset convolution stride for downsampling convolution of the normal random noise image, and the output result of the second layer is obtained as .

[0126] Furthermore, one-level multi-perceptual attention three-dimensional convolution calculation is respectively performed in each layer of the encoder to obtain an image after three-dimensional convolution , for example, referring to Figure 4 as shown, one-level multi-perceptual attention three-dimensional convolution calculations are respectively performed in the 3rd, 4th, 5th, and 6th layers of the encoder, and an atrous convolution calculation is performed on the image output by the three-dimensional convolution in the 6th layer to obtain an encoded image, where the convolution stride of the three-dimensional convolution in each level is the same as the convolution stride of the atrous convolution.

[0127] In one embodiment, referring to Figure 4As shown, except for the first - stage multi - perception attention three - dimensional convolution calculation, after each stage of three - dimensional convolution, a three - dimensional convolution is performed on the image output at this stage to obtain a spare convolution image. This spare convolution image can be spliced with the image after two - dimensional convolution in the decoder as the decoded image at this stage. Among them, the spare convolution image corresponding to the second - stage multi - perception attention three - dimensional convolution is spliced with the decoded image of the last - stage multi - perception attention two - dimensional convolution, and the spare convolution image corresponding to the third - stage multi - perception attention three - dimensional convolution is spliced with the decoded image of the penultimate - stage multi - perception attention two - dimensional convolution.

[0128] In step S33, in the decoder, a multi - stage multi - perception attention two - dimensional convolution calculation is performed on the encoded image to obtain a decoded image. Among them, each stage of multi - perception attention two - dimensional convolution includes a multi - perception attention calculation and a two - dimensional convolution calculation.

[0129] Continue to refer to Figure 4 As shown, in each layer of the decoder, a multi - perception attention two - dimensional convolution is performed respectively. In the decoder, after a multi - perception attention calculation is performed first, a two - dimensional convolution calculation is then performed. For example, refer to Figure 4 As shown, the decoder can be set to 3 layers, and the convolution kernel size of each two - dimensional convolution can be 1×1, obtaining a multi - stage multi - perception attention two - dimensional convolution calculation after the decoder performs convolution, and obtaining a decoded image.

[0130] In step S34, in the up - sampling layer, a multi - stage up - sampling convolution calculation is performed on the decoded image to obtain an iterative image of the same type as the normal random noise image. Among them, each stage of up - sampling convolution includes alternating calculations of up - sampling and convolution, and finally undergoes multiple convolution calculations.

[0131] Continue to refer to Figure 4 As shown, taking two - stage up - sampling convolution calculation as an example, in the first - stage up - sampling convolution, after the first up - sampling, a three - dimensional convolution calculation is performed, and then the image after the three - dimensional convolution calculation is subjected to a second up - sampling, and a three - dimensional convolution is performed on the image after the second up - sampling to obtain an up - sampling convolution image , after two - stage up - sampling convolution calculation, the up - sampled image is of the same type as the down - sampled noise image. Further, a three - dimensional convolution and a two - dimensional convolution are performed on the up - sampling convolution image again to obtain the calculation result of the up - sampling convolution at this stage. Finally, the image output by the up - sampling layer is calculated through the sigmoid function to obtain an iterative image.

[0132] Optionally, refer to Figure 5 As shown, in step S32, the step of performing a multi - stage multi - perception attention three - dimensional convolution calculation on the down - sampled noise image in the encoder to obtain an encoded image includes:

[0133] In step S321, spatial attention feature extraction, channel attention feature extraction, and pooling-atrous convolution calculation are respectively performed on the output of the upper-level multi-sensory attention three-dimensional convolution calculation to obtain the spatial attention feature image, channel attention feature image, and pooling-atrous convolution image corresponding to this level.

[0134] Among them, the spatial attention feature image mainly shows the feature correlation between planar vectors in the image and does not show the feature correlation between RGB channels. See Figure 6 As shown, spatial attention feature extraction is performed on the output of the upper-level multi-sensory attention three-dimensional convolution calculation through the spatial attention module. Among them, the spatial attention module can perform Q convolution, K convolution, and V convolution on the output of the upper-level multi-sensory attention three-dimensional convolution calculation respectively. Then, the result of the Q convolution is deformed and transposed, the result after the K convolution is deformed, and the result after the V convolution is also deformed. Then, the product of the matrix image after the deformation and transposition of the Q convolution and the matrix image after the deformation of the K convolution is calculated. For the matrix image obtained after the product, based on the normalized exponential function and transposition, an exponential transposed image is obtained, and the product of the exponential transposed image and the matrix image after the deformation of the V convolution is calculated, and the image obtained after this product is deformed. Then, according to the learning rate of the spatial attention module and the output image of the upper-level multi-sensory attention three-dimensional convolution calculation, the spatial attention feature image is obtained. For example, first calculate the product of the learning rate of the spatial attention module and the deformed image, and then sum it with the output image of the upper-level multi-sensory attention three-dimensional convolution calculation to obtain the spatial attention feature image.

[0135] Among them, different from the spatial attention feature image mainly showing the feature correlation between planar vectors in the image, the channel attention feature image shows the feature correlation between RGB channels. Continue to see Figure 6 As shown, the channel attention feature image is obtained by performing channel attention feature extraction on the input image in the following way:

[0136] For the channel direction of the input image, the input image is deformed to obtain a planar deformed image, and the planar deformed image is transposed to obtain a planar transposed image.

[0137] As Figure 6 shown, the original input image of H×W×8C is deformed to obtain a planar deformed image of 8C×(W×H), and then the planar deformed image of 8C×(W×H) is transposed to obtain a planar transposed image of (W×H)×8C.

[0138] Calculate the product of the planar deformed image and the planar transposed image to obtain a first backup image.

[0139] Determine the maximum value of each row in the first backup image row by row, and assign all other values in that row to the maximum value corresponding to that row to obtain the maximum value matrix feature image.

[0140] See Figure 7 As shown, the maximum value of the first row is 50, so all values in the first row are assigned 50, and the maximum value of the last row is 99, so all values in the last row are assigned 99 to obtain the maximum value matrix feature image.

[0141] Calculate the difference between the maximum value matrix feature image and the first backup image to obtain the second backup image.

[0142] It is easy to understand that based on the normalized exponential function and the matrix subtraction operation rule, the values at the same positions of the maximum value matrix feature image and the first backup image are subtracted to obtain the second backup image.

[0143] Calculate the product of the planar deformation image and the second backup image to obtain the third backup image.

[0144] Perform deformation processing on the third backup image again to obtain a target image of the same type as the input image.

[0145] Sum the input image and the target image to obtain the channel attention feature image.

[0146] In one implementation, after performing deformation processing on the third backup image again, calculate the product of the learning rate of the channel attention module and the deformed image to obtain a target image of the same type as the input image.

[0147] The above technical solution enables the model to focus on features of different dimensions, thereby avoiding feature omission and improving the accuracy of the target repaired image.

[0148] Optionally, the input image is subjected to pooling-atrous convolution calculation to obtain the pooling-atrous convolution image in the following manner:

[0149] Perform average pooling calculation, max pooling calculation, and atrous convolution calculations with the same convolutional kernels but different dilation rates on the input image respectively to obtain the average pooling image, max pooling image, and atrous convolution images corresponding to different dilation rates.

[0150] Among them, perform convolution calculation and upsampling calculation on the average pooling image and the max pooling image in sequence to obtain the upsampled average pooling image and the upsampled max pooling image.

[0151] See Figure 6 As shown, both the average pooling calculation and the max pooling calculation are performed by a 1×1 two-dimensional convolutional kernel for convolution calculation, and one upsampling is performed after the convolution calculation to obtain the average pooling image and the max pooling image.

[0152] Sum the upsampled mean-pooled image and the upsampled max-pooled image to obtain a fourth backup image;

[0153] Stitch the fourth backup image and the dilated convolution images corresponding to different dilation rates to obtain a pooled-dilated convolution image.

[0154] Among them, as shown in Figure 6 In the max-pooling-dilated convolution module, four dilated convolution modules with different dilation rates are set. For example, the dilation rates d of the four dilated convolution modules are 1, 4, 6, and 8 respectively. Then, the images obtained by dilated convolution calculations with the same number of convolution kernels but different dilation rates are stitched together.

[0155] In the embodiments of the present disclosure, after stitching the fourth backup image and the dilated convolution images corresponding to different dilation rates, convolution calculation is performed through a 1×1 two-dimensional convolution kernel to obtain a pooled-dilated convolution image.

[0156] The above technical solution can obtain comprehensive and accurate features by performing feature extraction in different ways through mean pooling and max pooling, and then combining the results of dilated convolution with different dilation rates. This ensures the accuracy of the final target repaired image.

[0157] In step S322, perform weighted summation on the spatial attention feature image, the channel attention feature image, and the pooled-dilated convolution image to obtain a multi-perceptual attention image.

[0158] In step S323, perform three-dimensional convolution calculation on the multi-perceptual attention image to obtain the encoded image.

[0159] As shown in Figures 8 to 15 The figure shows the target repaired images obtained by using the image repair method provided by the present disclosure to repair different images.

[0160] The embodiments of the present disclosure further provide an image repair device. As shown in Figure 16 The device 190 includes:

[0161] A generation module 191 configured to generate a binary image of the same type as the image to be repaired according to the damaged area and the undamaged area in the image to be repaired, where the damaged areas have the same value in the binary image, and the undamaged areas have the same value in the binary image and are different from the value of the damaged areas;

[0162] A calculation module 192 configured to generate a normal random noise image of the same type as the image to be repaired, and calculate the Hadamard product of the binary image and the image to be repaired to obtain a damaged image;

[0163] The iterative module 193 is configured to iteratively adjust the parameters of the image restoration model according to the normal random noise image, the binary image, and the damaged image, and use the iterative image output when the image restoration model is iteratively completed as the target restored image.

[0164] The above device performs separate learning and restoration on each image to be restored, without the need for training sample collection, annotation, or model training, reducing the cost of image restoration. Moreover, since the model does not rely on a large dataset, no additional image prior information is introduced. When restoring an image, by obtaining the prior information of a single degraded image to restore the lost part of the image, the ethical issues faced are much smaller than those of a model trained relying on big data. It not only improves the accuracy and restoration ability of image restoration but also has better ethical interpretability.

[0165] Optionally, the iterative module 193 includes:

[0166] An input sub-module configured to input the normal random noise image into the image restoration model to obtain the iterative image output by the image restoration model;

[0167] A determination sub-module configured to determine the loss value of this iteration according to the iterative image, the binary image, and the damaged image;

[0168] A backpropagation sub-module configured to backpropagate the loss value to adjust the parameters of the image restoration model;

[0169] An iteration sub-module configured to iteratively input the normal random noise image into the image restoration model after this parameter adjustment, and determine the loss value of this iteration according to the binary image, the damaged image, and the iterative image output by the image restoration model after this parameter adjustment, backpropagate the loss value of this iteration, and adjust the parameters of the image restoration model until the iteration is completed, and use the iterative image output when the image restoration model is iteratively completed as the target restored image.

[0170] Optionally, the image restoration model is configured to output the iterative image in the following manner:

[0171] Perform downsampling convolution on the normal random noise image according to a preset convolution kernel and a preset convolution stride to obtain a downsampled noise image;

[0172] Perform multi-level multi-sensory attention three-dimensional convolution calculation on the downsampled noise image in the encoder to obtain an encoded image, where each level of multi-sensory attention three-dimensional convolution includes one multi-sensory attention calculation and one three-dimensional convolution calculation, and each multi-sensory attention calculation includes spatial attention feature extraction, channel attention feature extraction, and pooling-atrous convolution calculation;

[0173] Perform multi-level multi-sensory attention two-dimensional convolution calculation on the encoded image in the decoder to obtain a decoded image, where each level of multi-sensory attention two-dimensional convolution includes one multi-sensory attention calculation and one two-dimensional convolution calculation;

[0174] Perform multi-level upsampling convolution calculation on the decoded image in the upsampling layer to obtain an iterative image of the same type as the normal random noise image, where each level of upsampling convolution includes alternating calculation of upsampling and convolution, and finally passes through multiple convolution calculations.

[0175] Optionally, the image inpainting model is configured to:

[0176] Perform spatial attention feature extraction, channel attention feature extraction, and pooling-atrous convolution calculation on the output of the previous-level multi-sensory attention three-dimensional convolution calculation, respectively, to obtain the spatial attention feature image, channel attention feature image, and pooling-atrous convolution image corresponding to this level;

[0177] Perform weighted summation on the spatial attention feature image, the channel attention feature image, and the pooling-atrous convolution image to obtain a multi-sensory attention image;

[0178] Perform three-dimensional convolution calculation on the multi-sensory attention image to obtain the encoded image.

[0179] Optionally, the image inpainting model is configured to extract channel attention feature images from the input image in the following manner:

[0180] Perform deformation processing on the input image in the channel direction of the input image to obtain a planar deformed image, and perform transpose processing on the planar deformed image to obtain a planar transposed image;

[0181] Calculate the product of the planar deformed image and the planar transposed image to obtain a first backup image;

[0182] Determine the maximum value of each row in the first backup image row by row, and assign all other values in that row to the maximum value corresponding to that row to obtain a maximum value matrix feature image;

[0183] Calculate the difference between the maximum value matrix feature image and the first backup image to obtain a second backup image;

[0184] Calculate the product of the planar deformed image and the second backup image to obtain a third backup image;

[0185] Perform deformation processing on the third backup image again to obtain a target image of the same type as the input image;

[0186] Sum the input image and the target image to obtain a channel attention feature image.

[0187] Optionally, the image inpainting model is configured to perform pooling-atrous convolution calculation on the input image to obtain a pooling-atrous convolution image in the following manner:

[0188] Perform average pooling calculation, max pooling calculation, and atrous convolution calculations with the same convolutional kernels but different atrous rates on the input image respectively to obtain an average pooling image, a max pooling image, and atrous convolution images corresponding to different atrous rates;

[0189] Perform convolution calculation and upsampling calculation on the average pooling image and the max pooling image in sequence to obtain an upsampled average pooling image and an upsampled max pooling image;

[0190] Sum the upsampled average pooling image and the upsampled max pooling image to obtain a fourth alternative image;

[0191] Stitch the fourth alternative image and the atrous convolution images corresponding to different atrous rates together to obtain a pooling-atrous convolution image.

[0192] Optionally, the determination sub-module is configured to:

[0193] Calculate the Hadamard product of the iterative image and the binary image to obtain a Hadamard matrix image.

[0194] Determine the loss value of this iteration according to the damaged image and the Hadamard matrix image.

[0195] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0196] Those skilled in the art should understand that the device embodiments described above are merely illustrative. For example, the division of modules is only a logical function division, and there may be other division methods in actual implementation. For example, multiple modules may be combined or integrated into one module. In addition, the modules described as separate components may or may not be physically separated. For example, the calculation module 192 and the iteration module 193 may be the same module or different modules physically, and each module may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. When implemented using hardware, it may be implemented in whole or in part in the form of an integrated circuit or chip.

[0197] The embodiments of the present disclosure also provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the image inpainting method described in any one of the foregoing embodiments are implemented.

[0198] An embodiment of the present disclosure also provides an electronic device, including:

[0199] A memory storing a computer program thereon;

[0200] A processor configured to execute the computer program in the memory to implement the steps of the image restoration method according to any one of the foregoing embodiments.

[0201] Figure 17 is a block diagram of an electronic device 700 shown according to an exemplary embodiment. As Figure 17 shown, the electronic device 700 may include: a processor 701, a memory 702. The electronic device 700 may further include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.

[0202] Among them, the processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the above image repair method. The memory 702 is used to store various types of data to support the operation of the electronic device 700. These data may include, for example, instructions for any application or method operating on the electronic device 700, as well as application-related data, such as contact data, sent and received messages, pictures, audio, video, and so on. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disc. The multimedia component 703 may include a screen and an audio component. Among them, the screen can be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal can be further stored in the memory 702 or sent through the communication component 705. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 704 provides an interface between the processor 701 and other interface modules. The above other interface modules can be a keyboard, a mouse, buttons, etc. These buttons can be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, correspondingly, the communication component 705 may include: a Wi-Fi module, a Bluetooth module, an NFC module, and so on.

[0203] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components, and is used to execute the above-mentioned image restoration method.

[0204] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When the program instructions are executed by a processor, the steps of the above-mentioned image restoration method are implemented. For example, the computer-readable storage medium may be the above-mentioned memory 702 including program instructions, and the above-mentioned program instructions may be executed by the processor 701 of the electronic device 700 to complete the above-mentioned image restoration method.

[0205] Figure 18 is a block diagram of an electronic device 1900 shown according to an exemplary embodiment. For example, the electronic device 1900 may be provided as a server. Referring to Figure 18 , the electronic device 1900 includes a processor 1922, the number of which may be one or more, and a memory 1932 for storing computer programs executable by the processor 1922. The computer programs stored in the memory 1932 may include one or more modules each corresponding to a set of instructions. In addition, the processor 1922 may be configured to execute the computer program to execute the above-mentioned image restoration method.

[0206] In addition, the electronic device 1900 may further include a power supply component 1926 and a communication component 1950. The power supply component 1926 may be configured to perform power management of the electronic device 1900, and the communication component 1950 may be configured to implement communication of the electronic device 1900, for example, wired or wireless communication. In addition, the electronic device 1900 may further include an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932.

[0207] In another exemplary embodiment, there is also provided a computer-readable storage medium including program instructions that, when executed by a processor, implement the steps of the above-described image restoration method. For example, the computer-readable storage medium may be the above-described memory 1932 including program instructions, and the above program instructions may be executed by the processor 1922 of the electronic device 1900 to complete the above-described image restoration method.

[0208] In another exemplary embodiment, there is also provided a computer program product that includes a computer program executable by a programmable device, and the computer program has a code portion for executing the above-described image restoration method when executed by the programmable device.

[0209] The preferred embodiments of the present disclosure have been described in detail above in conjunction with the accompanying drawings. However, the present disclosure is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0210] In addition, it should be noted that, in the above specific embodiments, the various specific technical features described can be combined in any suitable manner without conflict. To avoid unnecessary repetition, the present disclosure will not separately describe various possible combination methods.

[0211] Furthermore, any combination can be made between various different embodiments of the present disclosure as long as it does not violate the idea of the present disclosure, and it should also be regarded as the content disclosed by the present disclosure.

Claims

1. An image inpainting method, characterized in that, The method includes: Generating a binary image of the same type as the image to be restored according to the damaged area and the undamaged area in the image to be restored, wherein the damaged area has the same value in the binary image, and the undamaged area has the same value in the binary image and is different from the value of the damaged area; Generating a normal random noise image of the same type as the image to be restored, and calculating the Hadamard product of the binary image and the image to be restored to obtain a damaged image; Iteratively adjusting the parameters of the image restoration model according to the normal random noise image, the binary image, and the damaged image, and taking the iterative image output when the image restoration model finishes iteration as the target restored image; Among them, the step of iteratively adjusting the parameters of the image restoration model according to the normal random noise image, the binary image, and the damaged image, and taking the iterative image output when the image restoration model finishes iteration as the target restored image includes: Inputting the normal random noise image into the image restoration model to obtain an iterative image output by the image restoration model; Determining the loss value of this iteration according to the iterative image, the binary image, and the damaged image; Backpropagating the loss value to adjust the parameters of the image restoration model; Iteratively inputting the normal random noise image into the image restoration model after this parameter adjustment, and determining the loss value of this iteration according to the binary image, the damaged image, and the iterative image output by the image restoration model after this parameter adjustment, and backpropagating the loss value of this iteration to adjust the parameters of the image restoration model until the iteration is completed, and taking the iterative image output when the image restoration model finishes iteration as the target restored image.

2. The method according to claim 1, wherein The image restoration model outputs an iterative image in the following manner: Performing downsampling convolution on the normal random noise image according to a preset convolution kernel and a preset convolution stride to obtain a downsampled noise image; Performing multi-level multi-perceptual attention three-dimensional convolution calculation on the downsampled noise image in the encoder to obtain an encoded image, wherein each level of multi-perceptual attention three-dimensional convolution includes one multi-perceptual attention calculation and one three-dimensional convolution calculation, and each multi-perceptual attention calculation includes spatial attention feature extraction, channel attention feature extraction, and pooling-atrous convolution calculation; Performing multi-level multi-perceptual attention two-dimensional convolution calculation on the encoded image in the decoder to obtain a decoded image, wherein each level of multi-perceptual attention two-dimensional convolution includes one multi-perceptual attention calculation and one two-dimensional convolution calculation; Performing multi-level upsampling convolution calculation on the decoded image in the upsampling layer to obtain an iterative image of the same type as the normal random noise image, wherein each level of upsampling convolution includes alternating calculation of upsampling and convolution, and finally passing through multiple convolution calculations.

3. The method according to claim 2, wherein The step of performing multi-level multi-perceptual attention three-dimensional convolution calculation on the downsampled noise image in the encoder to obtain an encoded image includes: Perform spatial attention feature extraction, channel attention feature extraction, and pooling-atrous convolution calculation on the outputs of the three-dimensional convolution calculation of the upper-level multi-sensory attention respectively, and obtain the spatial attention feature image, channel attention feature image, and pooling-atrous convolution image corresponding to this level respectively; Perform weighted summation on the spatial attention feature image, the channel attention feature image, and the pooling-atrous convolution image to obtain a multi-sensory attention image; Perform three-dimensional convolution calculation on the multi-sensory attention image to obtain the encoded image.

4. The method according to claim 3, characterized in that, The channel attention feature image is obtained by performing channel attention feature extraction on the input image in the following manner: For the channel direction of the input image, perform deformation processing on the input image to obtain a plane deformation image, and perform transpose processing on the plane deformation image to obtain a plane transpose image; Calculate the product of the plane deformation image and the plane transpose image to obtain a first backup image; Determine the maximum value of each row in the first backup image row by row, and assign all other values in that row to the maximum value corresponding to that row to obtain a maximum value matrix feature image; Calculate the difference between the maximum value matrix feature image and the first backup image to obtain a second backup image; Calculate the product of the plane deformation image and the second backup image to obtain a third backup image; Perform deformation processing on the third backup image again to obtain a target image of the same type as the input image; Sum the input image and the target image to obtain the channel attention feature image.

5. The method according to claim 3, characterized in that The pooling-atrous convolution image is obtained by performing pooling-atrous convolution calculation on the input image in the following manner: Perform average pooling calculation, max pooling calculation, and atrous convolution calculations with the same convolution kernels but different dilation rates on the input image respectively to obtain an average pooling image, a max pooling image, and atrous convolution images corresponding to different dilation rates; Perform convolution calculation and upsampling calculation on the average pooling image and the max pooling image in sequence respectively to obtain an upsampled average pooling image and an upsampled max pooling image; Sum the upsampled average pooling image and the upsampled max pooling image to obtain a fourth backup image; Stitch the fourth backup image and the atrous convolution images corresponding to different dilation rates to obtain the pooling-atrous convolution image.

6. The method according to any one of claims 1-5, characterized in that, The step of determining the loss value of this iteration according to the iterative image, the binary image, and the damaged image includes: Calculate the Hadamard product of the iterative image and the binary image to obtain a Hadamard matrix image; Determine the loss value of this iteration according to the damaged image and the Hadamard matrix image.

7. An image restoration device, characterized in that, The device includes: A generation module configured to generate a binary image of the same type as the image to be repaired according to the damaged area and the undamaged area in the image to be repaired, wherein the damaged area has the same value in the binary image, and the undamaged area has the same value in the binary image and is different from the value of the damaged area; A calculation module, configured to generate a normal random noise image of the same type as the image to be repaired, and calculate the Hadamard product of the binary image and the image to be repaired to obtain a damaged image; An iteration module, configured to iteratively adjust the parameters of the image repair model according to the normal random noise image, the binary image, and the damaged image, and use the iteration image output when the image repair model iteration is completed as the target repaired image; Wherein, the iteration module includes: An input sub-module, configured to input the normal random noise image into the image repair model to obtain the iteration image output by the image repair model; A determination sub-module, configured to determine the loss value of the current iteration according to the iteration image, the binary image, and the damaged image; A backpropagation sub-module, configured to backpropagate the loss value to adjust the parameters of the image repair model; An iteration sub-module, configured to iteratively input the normal random noise image into the image repair model after the current parameter adjustment, and determine the loss value of the current iteration according to the binary image, the damaged image, and the iteration image output by the image repair model after the current parameter adjustment, and backpropagate the loss value of the current iteration to adjust the parameters of the image repair model until the iteration is completed, and use the iteration image output when the image repair model iteration is completed as the target repaired image.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method according to any one of claims 1-6.

9. An electronic device, characterized in that, Including: A memory, on which a computer program is stored; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1-6.

Images