Image Inpainting Method, Apparatus, Device, and Storage Medium

Abstract

The present invention discloses an image restoration method, device, equipment and storage medium, belonging to the field of picture processing. The image restoration method of the present invention can obtain a picture to be restored and a user instruction; perform pixel conversion on the picture to be restored according to the user instruction to obtain a mask map file; perform encoding processing on the mask map file and the picture to be restored to obtain a compiled file; restore the compiled file through a preset image restoration model to obtain a restored file; perform an addition process on the compiled file and the restored file to obtain a target file; perform decoding processing on the target file through a preset decoding algorithm to obtain a target picture, where the target picture is a restored picture, thereby simply and efficiently realizing global restoration of the picture.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image restoration method, apparatus, device, and storage medium. Background Art

[0002] Currently, when restoring images, restoration methods based on CNN or GAN are often used. These two processing methods cannot simply and efficiently restore the entire image. Therefore, how to provide an image restoration method that can efficiently restore the entire image has become an urgent problem to be solved. Summary of the Invention

[0003] The present invention aims to at least solve one of the technical problems existing in the prior art. For this purpose, the present invention provides an image restoration method that can efficiently restore globally distorted images.

[0004] The present invention also provides an image restoration apparatus having the above image restoration method.

[0005] The present invention also provides an image restoration device having the above image restoration method.

[0006] The present invention also provides a computer-readable storage medium.

[0007] The image restoration method according to the first aspect embodiment of the present invention includes:

[0008] Obtain the image to be restored and a user instruction;

[0009] Perform pixel conversion on the image to be restored according to the user instruction to obtain a mask map file;

[0010] Encode the mask map file and the image to be restored to obtain a compiled file;

[0011] Restore the compiled file through a preset image restoration model to obtain a restored file;

[0012] Add the compiled file and the restored file to obtain a target file;

[0013] Decode the target file through a preset decoding algorithm to obtain a target image, where the target image is the restored image.

[0014] The image restoration method according to an embodiment of the present invention has at least the following beneficial effects: The image restoration method provided by the present invention can obtain a to-be-restored picture and a user instruction input by a user, and perform pixel conversion on the to-be-restored picture according to the user instruction to obtain a mask map file containing information on the area to be restored; by encoding the mask map file and the to-be-restored picture, the method can generate a compiled file that can be recognized by an image restoration model, and input this compiled file into the image restoration model for restoration operations, thereby generating a corresponding restored file; by adding the compiled file and the restored file, the present invention can obtain a target file containing the restored picture data after compilation, and perform decoding processing on the target file through a preset decoding algorithm, thereby obtaining a target picture containing the final result, and using this target picture as the restored picture. Through this image restoration method, the present invention can simply and efficiently perform corresponding restoration on any specified area of the image to be restored according to the user's needs, and generate the restored picture required by the user; through encoding and decoding of data, the method can effectively ensure the integrity of the picture during transmission, thereby ensuring stability during picture restoration.

[0015] According to some embodiments of the present invention, performing pixel conversion on the to-be-restored picture according to the user instruction to obtain a mask map file includes:

[0016] Changing the pixel values of the to-be-restored picture to 0 to obtain a modified file;

[0017] According to the user instruction, changing the pixel values of a preset area in the modified file to 255 to obtain the mask map file.

[0018] According to some embodiments of the present invention, encoding the mask map file and the to-be-restored picture to obtain a compiled file includes:

[0019] Obtaining a preset encoding model;

[0020] Encoding the mask map file and the to-be-restored picture through the encoding model to obtain the compiled file.

[0021] According to some embodiments of the present invention, restoring the compiled file through a preset image restoration model to obtain a restored file includes:

[0022] Performing feature extraction on the compiled file to obtain first feature data and second feature data;

[0023] Performing a convolution operation on the first feature data to obtain first processed data, and performing a convolution operation and a Fourier transform operation on the second feature data to obtain second processed data;

[0024] Perform an activation operation on the first processed data through a preset activation function to obtain first extraction data, and perform an activation operation on the second processed data through the activation function to obtain second extraction data;

[0025] Obtain the repaired file according to the first extraction data and the second extraction data.

[0026] According to some embodiments of the present invention, the performing a convolution operation on the first feature data to obtain first processed data, and performing a convolution operation and a Fourier transform operation on the second feature data to obtain second processed data includes:

[0027] Input the first feature data into two identical convolutional layers respectively to obtain first convolutional data and second convolutional data;

[0028] Input the second feature data into the convolutional layer to obtain third convolutional data;

[0029] Perform a Fourier frequency domain conversion on the second feature data to generate fourth convolutional data;

[0030] Perform an addition process on the first convolutional data and the third convolutional data to obtain first addition data, and perform an activation process on the first addition data through a preset activation function to obtain the first processed data;

[0031] Perform an addition process on the second convolutional data and the fourth convolutional data to obtain second addition data, and perform an activation process on the second addition data through the activation function to obtain the second processed data.

[0032] According to some embodiments of the present invention, the performing a Fourier frequency domain conversion on the second feature data to generate fourth convolutional data includes:

[0033] Perform a discrete Fourier transform calculation on the second feature data to obtain frequency domain data;

[0034] Perform a convolution operation on the frequency domain data to obtain intermediate data;

[0035] Perform an inverse discrete Fourier transform on the intermediate data to obtain the fourth convolutional data.

[0036] According to the picture repair device of the second aspect embodiment of the present invention, it includes:

[0037] A collection module for acquiring a picture to be repaired and a user instruction;

[0038] A conversion module for performing pixel conversion on the picture to be repaired according to the user instruction to obtain a mask map file;

[0039] An encoding module, configured to encode the mask image file and the image to be repaired to obtain a compiled file;

[0040] A repair module, configured to repair the compiled file through a preset image repair model to obtain a repaired file;

[0041] A calculation module, configured to perform an addition operation on the compiled file and the repaired file to obtain a target file;

[0042] A decoding module, configured to decode the target file through a preset decoding method to obtain a repaired image.

[0043] The image repair device according to an embodiment of the present invention has at least the following beneficial effects: The image repair device provided by the present invention can obtain the image to be repaired and the user instruction input by the user through the conversion module, and perform pixel conversion on the image to be repaired according to the user instruction to obtain a mask image file containing information on the area to be repaired; by encoding the mask image file and the image to be repaired through the encoding module, the device can generate a compiled file that can be recognized by the image repair model, and input this compiled file into the repair module, so as to perform a repair operation through the image repair model, thereby generating a corresponding repaired file; the calculation module can perform an addition operation on the compiled file and the repaired file, and the present invention can obtain a target file containing the repaired image data after compilation, and perform decoding processing on the target file through a preset decoding algorithm in the decoding module, thereby obtaining a target image containing the final target image, and using this target image as the repaired image. Through this image repair device, it is possible to simply and efficiently repair any specified area of the image to be repaired according to the user's needs, and generate the repaired image required by the user; by encoding and decoding the data, it is possible to effectively ensure the integrity of the image during transmission, thereby stably realizing the repair of the image.

[0044] According to some embodiments of the present invention, the repair module includes:

[0045] An extraction unit, configured to extract features from the compiled file to obtain first feature data and second feature data;

[0046] A processing unit, configured to perform convolution operations and Fourier transform operations on the first feature data and the second feature data respectively to obtain first processed data and second processed data;

[0047] An activation unit, configured to perform activation operations on the first processed data and the second processed data respectively through a preset activation function to obtain first extraction data and second extraction data;

[0048] A generating unit, configured to obtain the repaired file according to the first extracted data and the second extracted data;

[0049] The picture repair device according to the third aspect embodiment of the present invention includes: at least one processor, and

[0050] a memory communicatively connected to the at least one memory, wherein

[0051] instructions are stored in the memory, and the at least one processor can execute the instructions, so that when the at least one processor executes the instructions, the image repair method described in the first aspect embodiment of the present invention can be implemented.

[0052] The picture repair device according to the embodiment of the present invention has at least the following beneficial effects:

[0053] In the picture repair device provided by the present invention, there is at least one memory. Among them, the memory stores instructions. Through at least one processor communicatively connected to the memory, the device provided by the present invention can execute the instructions through the processor, so as to obtain the picture to be repaired and the user instruction input by the user, and perform pixel conversion on the picture to be repaired according to the user instruction to obtain a mask map file containing the information of the area to be repaired; by encoding the mask map file and the picture to be repaired, this method can generate a compiled file that can be recognized by the image repair model, and input this compiled file into the image repair model for repair operation, so as to generate a corresponding repaired file; by adding the compiled file and the repaired file, the present invention can obtain a target file containing the repaired picture data after compilation, and perform decoding processing on the target file through a preset decoding algorithm, so as to obtain a target picture containing the final target picture, and use this target picture as the repaired picture. Through this image repair method, the present invention can simply and efficiently repair any specified area of the image to be repaired according to the user's needs, and generate the repaired picture required by the user; by encoding and decoding the data, the integrity of the picture during transmission can be effectively guaranteed, thereby ensuring the stability during picture repair.

[0054] The computer-readable storage medium according to the fourth aspect embodiment of the present invention stores computer-executable instructions, and the computer can implement the image repair method described in the first aspect embodiment of the present invention by executing the computer-executable instructions.

[0055] The computer-readable storage medium according to the embodiment of the present invention has at least the following beneficial effects: By a computer executing computer-executable instructions in the computer-readable storage medium, the present invention can enable the computer to implement the image repair method according to the embodiment of the first aspect of the present invention, so as to obtain a picture to be repaired and a user instruction input by the user, and perform pixel conversion on the picture to be repaired according to the user instruction to obtain a mask map file containing information on the area to be repaired; by encoding the mask map file and the picture to be repaired, the present method can generate a compiled file that can be recognized by the image repair model, and input this compiled file into the image repair model for repair operations, so as to generate a corresponding repair file; by adding the compiled file and the repair file, the present invention can obtain a target file containing the repaired picture data after compilation, and perform decoding processing on the target file through a preset decoding algorithm, so as to obtain a target picture containing the final target picture, and use this target picture as the repaired picture. Through this image repair method, the present invention can simply and efficiently repair any specified area of the image to be repaired according to the user's needs, and generate the repaired picture required by the user; through encoding and decoding of data, the integrity of the picture during transmission can be effectively guaranteed, thus ensuring stability during picture repair.

[0056] Additional aspects and advantages of the present invention will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The following further describes the present invention with reference to the drawings and embodiments, where:

[0058] Figure 1 is a flowchart of the image repair method according to the embodiment of the present invention.

[0059] Figure 2 is Figure 1 a specific flowchart of step S120 in

[0060] Figure 3 is Figure 1 a specific flowchart of step S130 in

[0061] Figure 4 is Figure 1 a specific flowchart of step S140 in

[0062] Figure 5 is Figure 4 a specific flowchart of step S420 in

[0063] Figure 6 is Figure 5 a specific flowchart of step S530 in

[0064] Figure 7 This is the structural diagram of the picture repair device according to the embodiments of the present invention.

[0065] Reference numerals: 100, collection module; 200, conversion module; 300, encoding module; 400, repair module; 500, calculation module; 600, decoding module; 610, extraction unit; 620, processing unit; 630, activation unit; 640, generation unit. Detailed implementation manners

[0066] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals denote the same or similar elements or elements with the same or similar functions from beginning to end. The embodiments described below with reference to the drawings are exemplary and are only used to explain the present invention, and should not be construed as a limitation to the present invention.

[0067] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by terms such as up, down, front, back, left, right, etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention.

[0068] In the description of the present invention, the meaning of "several" is more than one, the meaning of "multiple" is more than two, and understandings such as "greater than", "less than", "exceeding", etc. do not include the present number, and understandings such as "above", "below", "within", etc. include the present number. If there is a description of "first" and "second", it is only for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features or implicitly indicating the sequence relationship of the indicated technical features.

[0069] In the description of the present invention, unless otherwise clearly defined, terms such as "set", "installed", "connected", etc. should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above terms in the present invention in combination with the specific content of the technical solution.

[0070] In the description of the present invention, the description referring to terms such as "one embodiment", "some embodiments", "schematic embodiments", "examples", "specific examples", or "some examples", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0071] In a first aspect, referring to Figure 1 , the present invention provides an image restoration method, including:

[0072] S100, obtaining the picture to be restored and a user instruction;

[0073] S200, performing pixel conversion on the picture to be restored according to the user instruction to obtain a mask map file;

[0074] S300, performing encoding processing on the mask map file and the picture to be restored to obtain a compiled file;

[0075] S400, restoring the compiled file through a preset image restoration model to obtain a restored file;

[0076] S500, performing an addition operation on the compiled file and the restored file to obtain a target file;

[0077] S600, performing decoding processing on the target file through a preset decoding algorithm to obtain a target picture, where the target picture is the restored picture.

[0078] The image restoration method provided by the present invention can obtain the picture to be restored input by the user and the user instruction, and perform pixel conversion on the picture to be restored according to the user instruction to obtain a mask map file containing information on the area to be restored; through encoding processing on the mask map file and the picture to be restored, this method can generate a compiled file that can be recognized by the image restoration model, and input this compiled file into the image restoration model for restoration operations, thereby generating a corresponding restored file; by performing an addition operation on the compiled file and the restored file, the present invention can obtain a target file containing the restored picture data after compilation, and perform decoding processing on the target file through a preset decoding algorithm, thereby obtaining a target picture containing the final result, and using this target picture as the restored picture. Through this image restoration method, the present invention can simply and efficiently perform corresponding restoration on any specified area of the image to be restored according to the user's needs, and generate the restored picture required by the user; through encoding and decoding of the data, the integrity of the picture during transmission can be effectively guaranteed, thereby ensuring the stability during picture restoration.

[0079] Referring to Figure 2 , in some embodiments, step S120 includes:

[0080] S210, changing the pixel values of the picture to be restored to 0 to obtain a modified file;

[0081] S220, according to the user instruction, changing the pixel values of a preset area in the modified file to 255 to obtain a mask map file.

[0082] The image restoration method provided by the present invention can select a target area according to a user instruction input by the user for the area to be restored. Specifically, this method can perform pixel conversion on the selected area by the user, that is, change the pixel values of the corresponding area of the picture to be restored to 0, and use this modified file as a modified file, and then change the pixel values of the unselected areas in the modified file to 255 according to the user instruction, so as to obtain a mask map file containing the information of the selected area in the user instruction, thereby realizing the restoration of the area selected by the user according to this file.

[0083] In some specific embodiments, the user can select the corresponding restoration area by touching on some interactive devices. This method can control the interactive device and use the touched area as the area to be restored of the picture to be restored, or it can be other methods, without limitation.

[0084] Refer to Figure 3 , in some embodiments, step S130 includes:

[0085] S310, obtain a preset encoding model;

[0086] S320, perform encoding processing on the mask map file and the picture to be restored through the encoding model to obtain a compiled file.

[0087] The image restoration method provided by the present invention can encode the mask map file and the file to be restored according to a preset encoding model, so as to generate a compiled file that can be recognized by the image restoration model, thereby ensuring the stability of the data transmission process in this method. Through this method, the present invention can effectively avoid data loss caused by network factors such as network packet loss by transmitting encoded data, and effectively ensure the stability of the data transmission process.

[0088] Refer to Figure 4 , in some embodiments, step S140 includes:

[0089] S410, perform feature extraction on the compiled file to obtain first feature data and second feature data;

[0090] S420, perform a convolution operation on the first feature data to obtain first processed data, and perform a convolution operation and a Fourier transform operation on the second feature data to obtain second processed data;

[0091] S430, perform an activation operation on the first processed data through a preset activation function to obtain first extracted data, and perform an activation operation on the second processed data through the activation function to obtain second extracted data;

[0092] S440, obtain a restoration file according to the first extracted data and the second extracted data.

[0093] The image repair method provided by the present invention can repair the compilation file through a preset image repair model. Specifically, the present invention can extract the feature data in the compilation file to obtain first feature data and second feature data. Among them, the first feature data can be a local information block, and this local information block can be a matrix data block including local feature information; the second feature data can be a global information block, and this global information block can be a matrix data block including global feature information. This method can also perform a convolution operation on the first feature data to obtain first processed data, and perform a convolution operation and a Fourier transform operation on the second feature data respectively to obtain second processed data. This method can also perform activation operations on the first processed data and the second processed data respectively through a preset activation function to obtain first extraction data and second extraction data in sequence, so as to synthesize a repair file according to the first extraction data and the second extraction data.

[0094] In some specific embodiments, the present invention can also input the repair file obtained through steps S410 to S440 into the image repair model again to improve the accuracy of the image repair method provided by the present invention. Specifically, the present invention can use the generated repair file as update information to update the compilation file, so that this repair file is used as the input for the next picture repair, and then perform picture repair on the repair file again through steps S410 to S440, thereby improving the repair accuracy of the image repair method. In some embodiments, the present invention repeats steps S410 to S440 at least nine times to implement the image repair method provided by the present invention.

[0095] Refer to Figure 5 , in some embodiments, step S420 includes:

[0096] S510, input the first feature data into two identical convolutional layers respectively to obtain first convolutional data and second convolutional data;

[0097] S520, input the second feature data into a convolutional layer to obtain third convolutional data;

[0098] S530, perform a Fourier frequency domain conversion on the second feature data to generate fourth convolutional data;

[0099] S540, perform an addition process on the first convolutional data and the third convolutional data to obtain first addition data, and perform an activation process on the first addition data through a preset activation function to obtain first processed data;

[0100] For S550, the second convolution data and the fourth convolution data are added to obtain second addition data, and the second addition data is activated by an activation function to obtain second processed data.

[0101] The image inpainting method provided by the present invention can input first feature data into a preset convolution layer to implement a convolution operation. Specifically, the convolution layer adopted by the present invention is based on fast Fourier convolution operation, which can achieve the technical effects of obtaining global information and expanding the receptive field, thereby providing a technical basis for the method to simply and efficiently implement global image inpainting. Further, the method first inputs the first feature data into two identical convolution layers to obtain first convolution data and second convolution data, and inputs the second feature data into the same convolution layer to obtain third convolution data. Through Fourier frequency domain transformation, the method can transform the second feature data into fourth convolution data, and the first addition data generated by adding the first convolution data and the third convolution data is activated by a preset activation function to obtain first processed data; at the same time, the method can also activate the second convolution data and the fourth convolution data by a preset activation function to obtain second processed data, thereby effectively performing global inpainting on the image.

[0102] In some specific embodiments, the size of the convolution layer is 3*3, the size of the generated fourth convolution data is 1*1, and the Relu function is used to activate the first addition data and the second addition data, thereby avoiding overfitting in the data activation process and enhancing the convergence of this image inpainting model.

[0103] Refer to Figure 6 In some embodiments, step S530 includes:

[0104] S610, perform a discrete Fourier transform calculation on the second feature data to obtain frequency domain data;

[0105] S620, perform a convolution operation on the frequency domain data to obtain intermediate data;

[0106] S630, perform an inverse discrete Fourier transform on the intermediate data to obtain fourth convolution data.

[0107] The image restoration method provided by the present invention can perform a discrete Fourier transform on the second feature data to obtain frequency-domain data, thereby obtaining the image features in the frequency domain corresponding to the image data, and performing a convolution calculation on this frequency-domain data to obtain intermediate data, so as to perform an inverse discrete Fourier transform on the intermediate data to obtain the fourth convolution data that includes all image features in space. In some specific embodiments, after performing a convolution calculation on the frequency-domain data, this method also uses a preset activation function to perform an activation process on the calculation result to obtain intermediate data with a higher fitting degree. Through this method, the restoration of image data can be effectively achieved.

[0108] In a second aspect, the present invention provides a picture restoration device, including:

[0109] A collection module 100, configured to obtain a picture to be restored and a user instruction;

[0110] A conversion module 200, configured to perform pixel conversion on the picture to be restored according to the user instruction to obtain a mask map file;

[0111] An encoding module 300, configured to encode the mask map file and the picture to be restored to obtain a compiled file;

[0112] A restoration module 400, configured to restore the compiled file through a preset image restoration model to obtain a restored file;

[0113] A calculation module 500, configured to perform an addition operation on the compiled file and the restored file to obtain a target file;

[0114] A decoding module 600, configured to decode the target file through a preset decoding method to obtain a restored picture.

[0115] The picture restoration device according to the embodiments of the present invention has at least the following beneficial effects: The picture restoration device provided by the present invention can obtain the picture to be restored and the user instruction input by the user through the conversion module 200, and perform pixel conversion on the picture to be restored according to the user instruction to obtain a mask map file containing the information of the area to be restored; by encoding the mask map file and the picture to be restored through the encoding module 300, the device can generate a compiled file that can be recognized by the image restoration model, and input this compiled file into the restoration module 400, so as to perform a restoration operation through the image restoration model, thereby generating a corresponding restoration file; the calculation module 500 can perform an addition operation on the compiled file and the restoration file, and the present invention can obtain a target file containing the restored picture data after compilation, and perform decoding processing on the target file through the preset decoding algorithm in the decoding module 600, so as to obtain the final target picture, and use this target picture as the restored picture. Through this picture restoration device, it is possible to simply and efficiently perform corresponding restoration on any specified area of the image to be restored according to the user's needs, and generate the restored picture required by the user; through the encoding and decoding of data, the integrity of the picture during transmission can be effectively guaranteed, so as to stably realize the picture restoration function.

[0116] In some embodiments, the restoration module 400 includes:

[0117] An extraction unit 610, configured to extract features from the compiled file to obtain first feature data and second feature data;

[0118] A processing unit 620, configured to perform convolution operations and Fourier transform operations on the first feature data and the second feature data respectively to obtain first processed data and second processed data;

[0119] An activation unit 630, configured to perform activation operations on the first processed data and the second processed data respectively through a preset activation function to obtain first extraction data and second extraction data;

[0120] A generation unit 640, configured to obtain a restoration file according to the first extraction data and the second extraction data.

[0121] In a third aspect, the present invention provides a picture restoration device, including:

[0122] At least one processor, and

[0123] A memory communicatively connected to the at least one memory, wherein

[0124] Instructions are stored in the memory, and the at least one processor can execute the instructions so that when the at least one processor executes the instructions, it can implement the image restoration method according to the embodiments of the first aspect of the present invention.

[0125] In the picture restoration device provided by the present invention, there is at least one memory. The memory stores instructions. Through at least one processor communicatively connected to the memory, the device provided by the present invention can execute the instructions through the processor, so as to obtain the picture to be restored and user instructions input by the user, and perform pixel conversion on the picture to be restored according to the user instructions to obtain a mask map file containing information on the area to be restored; by encoding the mask map file and the picture to be restored, the present method can generate a compiled file that can be recognized by an image restoration model, and input this compiled file into the image restoration model for restoration operations, so as to generate a corresponding restored file; by adding the compiled file and the restored file, the present invention can obtain a target file containing the restored picture data after compilation, and perform decoding processing on the target file through a preset decoding algorithm, so as to obtain a target picture containing the final result, and use this target picture as the restored picture. Through this image restoration method, the present invention can simply and efficiently perform corresponding restoration on any specified area of the image to be restored according to the user's needs, and generate the restored picture required by the user; through encoding and decoding of data, the integrity of the picture during transmission can be effectively guaranteed, so as to ensure the stability during picture restoration.

[0126] In a fourth aspect, the present invention provides a computer-readable storage medium, in which computer-executable instructions are stored. By executing the computer-executable instructions, a computer can implement the image restoration method as in the first aspect embodiment of the present invention.

[0127] By executing the computer-executable instructions in the computer-readable storage medium, the present invention can enable a computer to implement the image restoration method as in the first aspect embodiment of the present invention, so as to obtain the picture to be restored and user instructions input by the user, and perform pixel conversion on the picture to be restored according to the user instructions to obtain a mask map file containing information on the area to be restored; by encoding the mask map file and the picture to be restored, the present method can generate a compiled file that can be recognized by an image restoration model, and input this compiled file into the image restoration model for restoration operations, so as to generate a corresponding restored file; by adding the compiled file and the restored file, the present invention can obtain a target file containing the restored picture data after compilation, and perform decoding processing on the target file through a preset decoding algorithm, so as to obtain a target picture containing the final result, and use this target picture as the restored picture. Through this image restoration method, the present invention can simply and efficiently perform corresponding restoration on any specified area of the image to be restored according to the user's needs, and generate the restored picture required by the user; through encoding and decoding of data, the integrity of the picture during transmission can be effectively guaranteed, so as to ensure the stability during picture restoration.

[0128] Additional aspects and advantages of the present invention will be given in part in the following description, become apparent in part from the following description, or be learned by practice of the present invention.

[0129] The embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the above embodiments, and various changes can be made without departing from the spirit of the present invention within the scope of knowledge possessed by those of ordinary skill in the art. In addition, the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

Claims

1. A method for image restoration, characterized in that, Including: Obtain the picture to be repaired and the user instruction; Perform pixel conversion on the picture to be repaired according to the user instruction to obtain a mask map file; Perform encoding processing on the mask map file and the picture to be repaired to obtain a compiled file; Repair the compiled file through a preset image repair model to obtain a repaired file; Perform addition processing on the compiled file and the repaired file to obtain a target file; Perform decoding processing on the target file through a preset decoding algorithm to obtain a target picture, where the target picture is the repaired picture; The repairing the compiled file through a preset image repair model to obtain a repaired file includes: Extract features from the compiled file to obtain first feature data and second feature data; Perform a convolution operation on the first feature data to obtain first processed data, and perform a convolution operation and a Fourier transform operation on the second feature data to obtain second processed data; Perform an activation operation on the first processed data through a preset activation function to obtain first extracted data, and perform an activation operation on the second processed data through the activation function to obtain second extracted data; Obtain the repaired file according to the first extracted data and the second extracted data; The performing a convolution operation on the first feature data to obtain first processed data, and performing a convolution operation and a Fourier transform operation on the second feature data to obtain second processed data includes: Input the first feature data into two identical convolutional layers respectively to obtain first convolutional data and second convolutional data; Input the second feature data into the convolutional layer to obtain third convolutional data; Perform a Fourier frequency domain conversion on the second feature data to generate fourth convolutional data; Perform addition processing on the first convolutional data and the third convolutional data to obtain first added data, and perform activation processing on the first added data through a preset activation function to obtain the first processed data; Perform addition processing on the second convolutional data and the fourth convolutional data to obtain second added data, and perform activation processing on the second added data through the activation function to obtain the second processed data; The performing a Fourier frequency domain conversion on the second feature data to generate fourth convolutional data includes: Perform a discrete Fourier transform calculation on the second feature data to obtain frequency domain data; Perform a convolution operation on the frequency domain data to obtain intermediate data; Perform an inverse discrete Fourier transform on the intermediate data to obtain the fourth convolutional data.

2. The image restoration method according to claim 1, wherein The performing pixel conversion on the picture to be repaired according to the user instruction to obtain a mask map file includes: Change the pixel values of the picture to be repaired to 0 to obtain a modified file; According to the user instruction, change the pixel values of a preset area in the modified file to 255 to obtain the mask map file.

3. The picture restoration method according to claim 1 or 2, characterized in that The performing encoding processing on the mask map file and the picture to be repaired to obtain a compiled file includes: Obtain a preset encoding model; Perform encoding processing on the mask map file and the picture to be repaired through the encoding model to obtain the compiled file.

4. An image restoration device, characterized in that, Including: A collection module for obtaining the picture to be repaired and user instructions; A conversion module for performing pixel conversion on the picture to be repaired according to the user instructions to obtain a mask map file; An encoding module for encoding the mask map file and the picture to be repaired to obtain a compiled file; A repair module for repairing the compiled file through a preset image repair model to obtain a repaired file; A calculation module for performing an addition operation on the compiled file and the repaired file to obtain a target file; A decoding module for decoding the target file through a preset decoding method to obtain a repaired picture; The repair module includes: An extraction unit for extracting features from the compiled file to obtain first feature data and second feature data; A processing unit for performing convolution operations and Fourier transform operations on the first feature data and the second feature data respectively to obtain first processed data and second processed data; An activation unit for performing activation operations on the first processed data and the second processed data respectively through a preset activation function to obtain first extracted data and second extracted data; A generation unit for obtaining the repaired file according to the first extracted data and the second extracted data; The processing unit includes: Inputting the first feature data into two identical convolutional layers respectively to obtain first convolutional data and second convolutional data; Inputting the second feature data into the convolutional layer to obtain third convolutional data; Performing Fourier frequency domain conversion on the second feature data to generate fourth convolutional data; Performing an addition process on the first convolutional data and the third convolutional data to obtain first added data, and performing activation processing on the first added data through a preset activation function to obtain the first processed data; Performing an addition process on the second convolutional data and the fourth convolutional data to obtain second added data, and performing activation processing on the second added data through the activation function to obtain the second processed data; The processing unit further includes: Performing discrete Fourier transform calculation on the second feature data to obtain frequency domain data; Performing a convolution operation on the frequency domain data to obtain intermediate data; Performing an inverse discrete Fourier transform on the intermediate data to obtain the fourth convolutional data.

5. An image restoration device, characterized in that, Including: At least one processor, and A memory communicatively connected to the at least one memory, where Instructions are stored in the memory, and the at least one processor can execute the instructions so that when the at least one processor executes the instructions, it can implement the picture repair method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, Computer-executable instructions are stored in the computer-readable storage medium, and the computer can implement the picture repair method according to any one of claims 1 to 3 by executing the computer-executable instructions.

Images