High-quality face restoration method, device, equipment, and storage medium
Through the combination of pre-trained repair model and BFRControlNet and U-Net models, low-quality face images are repaired, which solves the problem of limited utilization of low-quality images in fields such as identity verification, and realizes the repair of high-quality images and Maintaining identity characteristics.
Patent Information
- Application Number
- CN202411204107.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-08-30
AI Technical Summary
The prior art is difficult to effectively repair low-quality face images, resulting in limited image utilization in the fields of identity verification, public security monitoring, etc.
Using a pre-trained repair model, repairing repair materials through a general or identity-specific repair model, image repair is performed using a stable diffusion model combined with BFRControlNet and U-Net models, and sample identity information is injected through the decoupling of cross attention mechanism. To improve the repair effect.
It significantly improves the quality of low-quality face images, enhances the practicality and reliability of images, and can effectively maintain identity characteristics and improve visual quality.
Smart Images

Figure CN118710559B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a high-quality face restoration method, device, equipment, and storage medium. Background Art
[0002] With the development of computer vision technology and the Internet, images have become an indispensable information carrier in people's social life. However, in the process of image acquisition, storage, and transmission, there are various environmental factors that cause various image degradations such as noise pollution, blur effects, compression artifacts, etc., which make it difficult to use images. Image restoration technology is a technology that repairs image degradation to improve image quality.
[0003] Blind face restoration is an important branch of image restoration. Face images are widely used in modern society. They play a key role in personal identity authentication, public security monitoring, social networking platforms, etc. However, due to technical or environmental limitations, poor image quality is often encountered. Blind face restoration (BFR) technology aims to solve this problem and improve the practicality and reliability of low-quality face images. Summary of the invention
[0004] In order to solve one of the above-mentioned technical defects, the present application provides a high-quality face restoration method, device, equipment, and storage medium.
[0005] In a first aspect, the present application provides a high-quality face restoration method, the method comprising:
[0006] Acquire a material to be restored; wherein the material to be restored is a low-quality image to be restored, or the material to be restored is a low-quality image to be restored and a high-quality reference image, and the image to be restored and the reference image correspond to the same identity information;
[0007] The material to be repaired is repaired through the pre-trained repair model to obtain the repaired image;
[0008] If the material to be restored is a low-quality image to be restored, the restoration model is a universal restoration model; if the material to be restored is a low-quality image to be restored and a high-quality reference image, the restoration model is a restoration model for a specific identity;
[0009] The general restoration model is trained based on real sample images and corresponding low-quality sample images; the identity-specific restoration model is trained based on real sample images, corresponding low-quality sample images and corresponding sample identity information.
[0010] Optionally, the repair model is a stable diffusion model including a blind repair control network BFRControlNet, the stable diffusion model includes a U-Net model, the U-Net model includes an encoder and an intermediate block, and BFRControlNet replicates the encoder and the intermediate block;
[0011] Repair model training process, including:
[0012] Use real sample images and corresponding low-quality sample images to train BFRControlNet to obtain a general repair model;
[0013] Inject the corresponding sample identity information into the general repair model;
[0014] Based on the real sample images, the corresponding low-quality sample images and the corresponding sample identity information, the injected general restoration model is trained to obtain the identity-specific restoration model;
[0015] Among them, in each layer of the U-Net model of the specific identity restoration model, the corresponding low-quality sample image and the corresponding sample identity information are involved in the calculation.
[0016] Optionally, in the general inpainting model, BFRControlNet concatenates the channels of the corresponding low-quality image samples with the channels of the noise variables of the stable diffusion model.
[0017] Optionally, the loss function of BFRControlNet ;
[0018] in, is the loss function of BFRControlNet, For the The latent space variables of the denoised image. For text prompt conditions, is the number of denoising steps, For noise, For low quality images, is the noise prediction parameter, For the The prediction noise of the step, is the expected function.
[0019] Optionally, inject the corresponding sample identity information into the general repair model, including:
[0020] By decoupling the cross-attention mechanism, the corresponding sample identity information is injected into the general repair model;
[0021] Among them, the decoupled cross attention mechanism is implemented using the following formula:
[0022] ;
[0023] in, is the cross attention mechanism, For query, A key for a generic repair model, is the value of the general repair model, is a key based on the corresponding sample identity information, is the value based on the corresponding sample identity information, is the normalization function, for The dimension of for The transpose of for The transpose of .
[0024] Optionally, if the material to be restored is a low-quality image to be restored and a high-quality reference image, the material to be restored is restored by using a pre-trained restoration model to obtain a restored image, including:
[0025] Obtaining identity information based on a reference image;
[0026] Inputting the image to be repaired, the reference image and the identity information into the restoration model of a specific identity to obtain the restored image;
[0027] Among them, the loss function of the restoration model of a specific identity when obtaining the restored image is ;
[0028] in, The loss function for obtaining the restored image for the restoration model of a specific identity, For the The latent space variables of the denoised image. For text prompt conditions, is the number of denoising steps, For noise, is the image to be repaired, is the reference image, is the noise prediction parameter, For the The prediction noise of the step, is the expected function.
[0029] Optionally, obtaining identity information based on a reference image includes:
[0030] Project the reference image through a multi-layer perceptron MLP to obtain the projected identity features;
[0031] The projected identity features are used as the basis for querying the transformer Q-Former to query the image encoder CLIP image features in the reference image;
[0032] The projected identity features are combined with the CLIP image features using a shortcut structure to obtain identity information.
[0033] In a second aspect of the present application, a high-quality face restoration device is provided, the device comprising:
[0034] An acquisition module is used to acquire a material to be restored; wherein the material to be restored is a low-quality image to be restored, or the material to be restored is a low-quality image to be restored and a high-quality reference image, and the image to be restored and the reference image correspond to the same identity information;
[0035] The restoration module is used to restore the material to be restored obtained by the acquisition module through a pre-trained restoration model to obtain a restored image;
[0036] If the material to be restored is a low-quality image to be restored, the restoration model is a universal restoration model; if the material to be restored is a low-quality image to be restored and a high-quality reference image, the restoration model is a restoration model for a specific identity;
[0037] The general restoration model is trained based on real sample images and corresponding low-quality sample images; the identity-specific restoration model is trained based on real sample images, corresponding low-quality sample images and corresponding sample identity information.
[0038] In a third aspect of the present application, an electronic device is provided, including:
[0039] Memory;
[0040] Processor; and
[0041] Computer programs;
[0042] The computer program is stored in the memory and is configured to be executed by the processor to implement the method as described in the first aspect above.
[0043] According to a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored; the computer program is executed by a processor to implement the method described in the first aspect above.
[0044] The present application provides a high-quality face restoration method, device, equipment, and storage medium. The method includes: obtaining a material to be restored; wherein the material to be restored is a low-quality image to be restored, or the material to be restored is a low-quality image to be restored and a high-quality reference image, and the image to be restored and the reference image correspond to the same identity information; repairing the material to be restored by a pre-trained restoration model to obtain a restored image; wherein, if the material to be restored is a low-quality image to be restored, the restoration model is a universal restoration model; if the material to be restored is a low-quality image to be restored and a high-quality reference image, the restoration model is a restoration model for a specific identity; the universal restoration model is trained based on a real sample image and a corresponding low-quality sample image; the identity-specific restoration model is trained based on a real sample image, a corresponding low-quality sample image, and a corresponding sample identity information. The method provided by the present application enhances image quality. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0046] Figure 1 A flowchart of a high-quality face restoration method provided in an embodiment of the present application;
[0047] Figure 2 A schematic diagram of an implementation architecture of a high-quality face restoration method provided in an embodiment of the present application;
[0048] Figure 3 A schematic diagram of the restoration effect of a high-quality face restoration method provided in an embodiment of the present application;
[0049] Figure 4 A schematic diagram of the structure of a high-quality face restoration device provided in an embodiment of the present application;
[0050] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0051] In order to make the technical solutions and advantages in the embodiments of the present application more clearly understood, the exemplary embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.
[0052] This embodiment provides a high-quality face restoration method. Figure 1The implementation process of this method is as follows:
[0053] 101. Obtain the material to be restored.
[0054] The material to be restored is a low-quality image to be restored, or the material to be restored is a low-quality image to be restored and a high-quality reference image, and the image to be restored and the reference image correspond to the same identity information.
[0055] The image to be restored and the reference image correspond to the same identity information. That is, the image to be restored and the reference image are images of the same face, but with different resolutions. This is like two images of a person at different times and / or in different locations, and / or two images of different resolutions at the same time and location.
[0056] 102. The material to be restored is restored by using a pre-trained restoration model to obtain a restored image.
[0057] If the material to be restored is a low-quality image to be restored, the restoration model is a universal restoration model. If the material to be restored is a low-quality image to be restored and a high-quality reference image, the restoration model is a restoration model for a specific identity.
[0058] The general restoration model is trained based on real sample images and corresponding low-quality sample images. The identity-specific restoration model is trained based on real sample images, corresponding low-quality sample images and corresponding sample identity information.
[0059] Whether it is a general repair model or a repair model for a specific identity, it is a repair model. The repair model of this application adopts the structure and parameters of a stable diffusion model (such as Stable Diffusion v1.5) to ensure the generation capability. Therefore, a trainable blind repair control network (BFRControlNet) is created, which replicates the encoder and intermediate blocks of the denoising U-Net model in the stable diffusion model.
[0060] That is to say, the repair model is a stable diffusion model including BFRControlNet, which includes the U-Net model, which includes an encoder and an intermediate block, and BFRControlNet copies the encoder and the intermediate ware. That is, the general repair model is a stable diffusion model including BFRControlNet, which includes the U-Net model, which includes an encoder and an intermediate block, and BFRControlNet copies the encoder and the intermediate ware. The repair model of a specific identity is also a stable diffusion model including BFRControlNet, which includes the U-Net model, which includes an encoder and an intermediate block, and BFRControlNet copies the encoder and the intermediate ware. The general repair model and the repair model of a specific identity have the same structure.
[0061] In practical applications, the general repair model and the identity-specific repair model are the products of two stages obtained by training the stable diffusion model including the blind repair control network BFRControlNet. When training the stable diffusion model including the blind repair control network BFRControlNet, the general repair model is first obtained, and then the general repair model is trained to obtain the identity-specific repair model.
[0062] Specifically, the repair model training process includes:
[0063] 1. Use real sample images and corresponding low-quality sample images to train BFRControlNet to obtain a general repair model.
[0064] In the general inpainting model, BFRControlNet concatenates the channels of the corresponding low-quality image samples with the channels of the noise variables of the stable diffusion model.
[0065] That is to say, BFRControlNet can splice the low-quality image to be repaired in the pixel domain with the noise latent variable of the original input in the stable diffusion model on the channel axis. For example, the number of channels of the noise latent variable of the original input is 4, and the number of channels of the low-quality image to be repaired in the pixel domain is 3. After splicing, the number of input channels of BFRControlNet increases to 7. Therefore, the number of channels of the BFRControlNet input convolutional layer is also 7 to adapt to the spliced input.
[0066] In addition, the loss function of BFRControlNet .
[0067] in, is the loss function of BFRControlNet, For the The latent space variables of the denoised image. For text prompt conditions, is the number of denoising steps, For noise, For low quality images, is the noise prediction parameter, For the The prediction noise of the step, is the expected function.
[0068] During the training process, is a low-quality sample image; when repairing based on the trained general repair model, The low-quality image to be restored.
[0069] In the specific implementation, it can be based on Figure 2 The content injection module shown implements the general repair model training process, which can also be based on Figure 2 The content injection module shown realizes the repair of the material to be repaired.
[0070] For example, during training, the content injection module injects the corresponding low-quality sample images into the stable diffusion model including BFRControlNet to obtain a universal repair model. When repairing the material to be repaired based on the trained universal repair model, the content injection module injects the low-quality image to be repaired into the universal repair model. Whether in the training stage or the repair stage of the material to be repaired, the universal repair model does not save specific identity information, but only repairs based on low-quality images (LQ).
[0071] The universal restoration model can make full use of the spatial structure and text information in the image to be restored during the face restoration process, avoid serious color shift problems, and the pixel-level stitching can avoid information loss when encoding the image to be restored into the latent space.
[0072] In the training of BFRControlNet, it can be initialized from the pre-trained stable diffusion model, and the parameters added by channel splicing are initialized to zero.
[0073] 2. Inject the corresponding sample identity information into the general repair model.
[0074] In specific implementation, it can be Figure 2 The identity injection module shown implements the injection of identity information.
[0075] For example, by decoupling the cross-attention mechanism, the corresponding sample identity information is injected into the general repair model. That is, the decoupling cross-attention mechanism is used to inject the sample identity information into the denoising U-Net model of the general repair model.
[0076] Among them, the decoupled cross attention mechanism is implemented using the following formula:
[0077] .
[0078] in, is the cross attention mechanism, For query, A key for a generic repair model, is the value of the general repair model, is a key based on the corresponding sample identity information, is the value based on the corresponding sample identity information, is the normalization function, for The dimension of for The transpose of for The transpose of .
[0079] 3. Based on the real sample images, the corresponding low-quality sample images and the corresponding sample identity information, the general repair model after training injection is obtained to obtain a repair model for a specific identity.
[0080] Among them, in each layer of the U-Net model of the specific identity restoration model, the corresponding low-quality sample image and the corresponding sample identity information are involved in the calculation.
[0081] After obtaining the trained general restoration model and the identity-specific restoration model, if the material to be restored is a low-quality image to be restored, the material to be restored is restored by using the pre-trained restoration model to obtain a restored image, including:
[0082] The low-quality image to be repaired is input into a general repair model to obtain the repaired image.
[0083] This process does not save specific identity information, but only performs inpainting based on the low-quality image (LQ).
[0084] If the material to be repaired is a low-quality image to be repaired and a high-quality reference image, the material to be repaired is repaired by a pre-trained repair model to obtain a repaired image, including:
[0085] 1. Obtain identity information based on the reference image.
[0086] For example, 1) Project the reference image through MLP (Multilayer Perceptron) to obtain the projected identity features. 2) Use the projected identity features as the basis of Q-Former (Query Transformer) to query the CLIP (Contrastive Language–Image Pre-training) image features in the reference image. 3) Use the shortcut structure to combine the projected identity features with the CLIP image features to obtain the identity information.
[0087] 2. Input the image to be repaired, the reference image and the identity information into the restoration model of a specific identity to obtain the restored image.
[0088] For example, by Figure 2 The identity injection module shown injects identity information, and obtains a restored image through a restoration model of a specific identity based on the image to be restored, the reference image and the identity information.
[0089] Among them, the loss function of the restoration model of a specific identity when obtaining the restored image is .
[0090] in, The loss function for obtaining the restored image for the restoration model of a specific identity, For the The latent space variables of the denoised image. For text prompt conditions, is the number of denoising steps, For noise, is the image to be repaired, is the reference image, is the noise prediction parameter, For the The prediction noise of the step, is the expected function.
[0091] By injecting identity information, the blindly repaired identity features can be maintained. When obtaining identity information based on the reference image, CLIP (image encoder) and Q-Former are integrated to extract identity information from high-quality reference images. In practice, the identity features are projected through a multi-layer perceptron (MLP) as a query basis for Q-Former to extract identity-related CLIP image features. At the same time, a shortcut structure is used to combine the projected identity features with the identity-related CLIP image features to form the final identity features. This combination ensures that identity information can be more fully extracted from the reference facial image than relying solely on CLIP image features.
[0092] The identity-specific inpainting models not only fully exploit the contents in the low-quality image (i.e., the image to be inpainted), but also extract sufficient identity information, and their direct collaboration is usually able to maintain faithful identity and visual quality simultaneously.
[0093] However, the general inpainting model has only been trained on the image to be inpainted and the corresponding high-resolution image pair, and has no ability to extract information from the reference image. If the identity information in the reference image is directly injected into the U-Net model, it cannot correctly generate a high-resolution face with identity preservation. In order to fully and accurately utilize the identity information in the reference image, the following loss function is adopted when the inpainting model of a specific identity obtains the inpainted image .
[0094] The following experimental process is used to verify the effect of the method provided in this embodiment (the scenario implemented based on the restoration model of a specific identity):
[0095] Verification process: For a damaged face image, there is a high-quality image as an identity reference. The high-quality face restoration method provided in this embodiment is introduced to restore it to a face image that matches the identity of the high-quality image.
[0096] Specifically:
[0097] (1) Face image collection. The data used in the experiment comes from the FFHQ dataset and the CelebRef-HQ dataset.
[0098] FFHQ dataset This dataset contains 70,000 high-quality images with a resolution of 512×512.
[0099] CelebRef-HQ dataset This dataset contains 1,005 identities and a total of 10,555 high-quality images. Each identity contains 3 to 21 face images with a resolution of 512×512.
[0100] During validation, 905 identities and 9,453 images are randomly selected as the dataset. For real-world cases, corresponding reference images are selected from the CelebRef-HQ test set, and a subset of the LFW dataset is extracted, called LFW-Ref-Test. LFW-Ref-Test contains a total of 16 identities and 76 images.
[0101] (2) Training the blind restoration control network (BFRControlNet). The FFHQ training set is degraded online to construct low-quality data (i.e., the image to be restored in this embodiment). The data pairs of low-quality images and high-quality images (i.e., the low-quality image to be restored and the high-quality reference image in this embodiment) are input into the network. When inputting, the low-quality image is adjusted to 64×64 size through bicubic upsampling to match the size of the input latent variable of the denoising U-Net model.
[0102] During training, a total of 87,500 iterations were performed with a batch size of 8 to improve the quality of the images.
[0103] (3) For each identity in the CelebRef-HQ dataset, one high-quality image is retained as a reference image. The other images are constructed and degraded online. The image pairs of low-quality image, high-quality reference image, and high-quality image are input. 99,540 iterations are performed with a batch size of 4 to ensure identity consistency during the restoration process.
[0104] (4) The low-quality image is first processed by the pre-trained SwinIR model to remove part of the degradation. The processed image and a high-quality reference image of the same identity are input into the restoration model of this method. UniPC sampling is used for 20 steps to obtain the restored image result.
[0105] Figure 3 A schematic diagram of the restoration effect of the high-quality face restoration method provided in this embodiment, wherein the first column is the input low-quality image, the second and third columns are other methods for comparison, the fourth column is the image restored by the high-quality face restoration method provided in this embodiment, the fifth column is the reference image, and the sixth column is the original image.
[0106] In addition, in this embodiment, when repairing a low-quality image to be repaired, the first Image after denoising The determination process is:
[0107] 201, obtain the current processed image.
[0108] 202, convert the tensor of the current processed image from [H, W, C] to [C, H, W]. Normalize the values of the converted tensor in each dimension to [0, 1]. Normalize the values of the normalized tensor in each dimension to have a mean of 0 and a variance of 1. Use the normalized image as the final processed image .
[0109] Among them, H is the height, W is the width, and C is the number of channels.
[0110] 203, for the final processed image , starting from step 1, perform forward multi-step noise processing to obtain the Step Image .
[0111] in, is the target number of steps for adding noise to the final processed image, for example .
[0112] like , then in step 203, the final processed image will be denoised for 100 steps, and the image after the 100th step of denoising That is Step Image .
[0113] In addition, in each step of the noise addition process for the repaired image, The image after noise processing in this step is obtained.
[0114] in, is the number of noise adding steps for the repaired image. , For the Step image, is the noise drawn from a standard Gaussian distribution, For the The noise processing parameters of the step, , is the step number indicator, For the The noise factor of the step, , For the Diffusion speed when adding noise.
[0115] like , then in step 203, the repaired image will be denoised for 100 steps, and the image after the 100th step of denoising That is Step Image .
[0116] 204, based on low-pass filter ,right From Step 1 is used to perform reverse multi-step denoising, and each After the denoising process, the low-frequency components of the denoised image are exchanged with The low-frequency component of Step Image .
[0117] in, are the parameters of the low-pass filter, The initial value of is determined according to the degree of preservation of face identity information, such as The initial value of is 2. is the reverse denoising step number identifier, , are the parameters of the low-pass filter, The initial value of is determined according to the degree of preservation of face identity information. For The denoising target number of steps, .
[0118] In addition, From Starting from the step, after each step of the reverse denoising process, Exchange the low-frequency components of the denoised image with low frequency components.
[0119] in, For the After the denoising process, for The low-frequency component of for The low-frequency component of For the Step image.
[0120] The low-pass filter can be used to filter the input A filtering operation that downsamples by a factor of 1 and then upsamples.
[0121] like , , then in step 102, it will start from step 1 and perform 100 steps of noise addition forward to obtain the image after the 100th step of noise addition. (Right now ). In step 103, starting from step 100, multiple steps of denoising and swapping are performed backward to obtain an image denoised and swapped to step 50. (Right now ).
[0122] 205, according to Predict the first image .
[0123] Among them, the first image is the predicted 0-step output image, .
[0124] in, For the The noise processing parameters of the step, , is the step number indicator, For the The diffusion speed when adding noise, is the noise drawn from a standard Gaussian distribution.
[0125] By executing steps 203 to 205, the final processed image will be subjected to the first denoising and denoising process, wherein the number of denoising steps is greater than the number of denoising steps, and the 0-step output image is predicted based on the denoised image. .
[0126] 206, yes , starting from step 1, perform forward multi-step noise processing to obtain the Step Image .
[0127] in, For The target number of noise steps is .
[0128] The denoising process in this step is similar to the denoising process in step 203, except that the denoising object and the number of denoising steps when the denoising is finally stopped are different. The denoising object in step 203 is the final processed image. The noise object in this step is the one predicted in step 205. Step 203 adds noise Stop after step 1, and finally get Step Image , step 206 adds noise Stop after step 1, and finally get Step Image .
[0129] In addition, in In each step of the noise addition process, Get the image of this step.
[0130] in, For The number of noise adding steps is marked, , For the Step image, For the The noise processing parameters of the step, , is the step number indicator, For the The noise factor of the step, , For the Diffusion speed when adding noise.
[0131] It should be noted that The value can be The same value as The values of are different, and this embodiment and subsequent embodiments do not limit The value of The relationship between the values.
[0132] like 250, then in step 105 Add noise for 250 steps to get the image after adding noise at the 250th step .
[0133] 207, based on low-pass filter ,right From Step 1 is used to perform reverse multi-step denoising. After the denoising process, the low-frequency components of the denoised image are exchanged with The low-frequency component of Step Image .
[0134] in, is the reverse denoising step number identifier, , For The denoising target number of steps, .
[0135] The denoising process in this step is similar to the denoising process in step 204, except that the denoising object and the number of denoising steps when the denoising is finally stopped are different. The denoising object in step 204 is the denoising object obtained in step 203. The denoising object in this step is the one obtained in step 206. Step 204: After adding noise in the first step, the image is denoised and exchanged in multiple steps. After the denoising and exchange are completed, stop and get the Step Image , step 207 from the Step 1, after multiple steps of denoising, in step After the denoising and exchange are completed, stop and get the Step Image .
[0136] In addition, From Starting from the step, after each step of the reverse denoising process, Exchange the low-frequency components of the denoised image with low frequency components.
[0137] in, For the After the denoising process, for The low-frequency component of for The low-frequency component of For the Step image.
[0138] It should be noted that The value can be The same value as The values of are different, and this embodiment and subsequent embodiments do not limit The value of The relationship between the values.
[0139] like , , this step will Denoising from step 250 to step 150, we get .
[0140] The second denoising process is completed through step 206 , and the first stage of the second denoising process is performed through step 207 .
[0141] 208, according to the current Relationship to preconditions, and The mean square error between the low-frequency component of the final processed image is obtained Image after denoising .
[0142] If the current The preset conditions are not met and the current If the mean square error between the low-frequency component of the image to be repaired is greater than the preset threshold, the image is enlarged. , repeatedly performing a low-pass filter based ,right From Step 1 is used to perform reverse multi-step denoising. After the denoising process, the low-frequency components of the denoised image are exchanged with The low-frequency component of Step Image and subsequent steps (i.e., repeatedly executing steps 207 and 208).
[0143] Among them, the expansion Can be .
[0144] like The initial value of is 2, so the preset condition can be , the preset threshold is .
[0145] That is, if the current The preset conditions are not met and the current If the mean square error between the low-frequency components of the final processed image is greater than a preset threshold, the first stage of the second denoising process will be repeated.
[0146] If the current Satisfy the preset conditions, or the current If the mean square error between the low-frequency component of the final processed image and the low-frequency component of the final processed image is not greater than the preset threshold, the low-pass filter is used to filter the image. ,right From Step 1 is used to perform reverse multi-step denoising. After the denoising process, the low-frequency components of the denoised image are exchanged with The low-frequency component of Step Image , That is, the denoised image .
[0147] in, is the reverse denoising step number identifier, , For The denoising target number of steps, .
[0148] In addition, From Starting from the step, after each step of the reverse denoising process, Exchange the low-frequency components of the denoised image with low frequency components.
[0149] in, For the After the denoising process, for The low-frequency component of for The low-frequency component of For the Step image.
[0150] At this point, the second stage of the second denoising process is completed, and the entire second denoising process is also completed.
[0151] Below , , The initial value of is 2. , , ,expand for , the default condition is , the preset threshold is For example, in step 203, the final processed image Add noise for 100 steps and get the image of the 100th step (That is, the image after adding noise for 100 steps). In step 204, based on the low-pass filter ,right Starting from step 100, reverse multi-step denoising is performed, and after each step of denoising, the low-frequency components of the denoised image are exchanged with The low-frequency component of the image at step 50 is obtained (i.e., the image after noise reduction and exchange in step 50). In step 205, according to Predicted Image In step 206, Add noise for 250 steps and get the image at step 250 (That is, the image after 250 steps of noise addition). In step 207, based on the low-pass filter ,right Starting from step 250, reverse multi-step denoising is performed, and after each step of denoising, the low-frequency components of the denoised image are exchanged with The low-frequency component of the image at step 150 is obtained (That is, the image after denoising and swapping at step 150).
[0152] At this time, the low-pass filter of , the preset conditions are not met.
[0153] A. If The final processed image obtained in step 202 The mean square error between the low-frequency components is no greater than , then the second stage of the second denoising process is performed.
[0154] B. If The final processed image obtained in step 202 The mean square error between the low-frequency components is greater than , then Updated to 4 (i.e. ), and re-execute step 106.
[0155] When step 207 is re-executed, Therefore, after executing step 207 Still 4, which does not meet the preset conditions.
[0156] b1, if step 207 is executed again The final processed image obtained in step 202 The mean square error between the low-frequency components is no greater than , then the second stage of the second denoising process is performed.
[0157] b2, if step 207 is executed again The final processed image obtained in step 202 The mean square error between the low-frequency components is greater than , then Updated to 8 (i.e. ), re-execute step 207.
[0158] When step 207 is executed again, , and so on, until After 16, the second stage of the second denoising process is performed.
[0159] In the second stage of the second denoising process, a low-pass filter is used to , for the current Starting from step 150, reverse multi-step denoising is performed, and after each step of denoising, the low-frequency components of the denoised image are exchanged with The low-frequency component of the image at step 50 is obtained ; Right now .
[0160] In addition, in order to improve the detail generation capability of the method provided in this embodiment, the U-Net model is used to predict noise in the process of generating higher resolution images. The receptive field of the model is improved by using dilated convolutions on the convolutional layers of the U-Net intermediate layers of the model, thereby further improving the image generation details while ensuring the original prior of the generation process.
[0161] Specifically, the U-Net model is implemented using the following convolution formula:
[0162] ( .
[0163] in, is the feature map, is the preset expansion rate, is the convolution kernel, is the convolution parameter, is the spatial position in the feature map, is the spatial position within the convolution kernel.
[0164] The images generated at higher resolutions appear blurry and semantically biased, which can be attributed to the limited receptive field of the U-Net layer. The convolution kernel of the U-Net denoiser used in the method provided in this embodiment is a dilated convolution with a specific dilation rate δ. This convolution allows the high-resolution image generation method provided in this embodiment to cover a wider range of features without additional adjustment, while retaining the original prior at a higher resolution to sample nearby features to enhance local details, thereby conveniently combining comprehensive information in the generation process. The convolution process diagram is shown in FIG. Figure 2 shown.
[0165] In addition, the same noise level has different degrees of damage to images of different resolutions, which leads to the need to modify the noise level at the junction of different resolutions to improve the quality of image generation. In order to add different degrees of noise to images of different resolutions to improve the quality and fidelity of the generated images, the U-Net model controls the noise amplitude of each denoising through the following equation:
[0166] .
[0167] in, Adjustment coefficient, is the signal-to-noise ratio of the low-resolution image before denoising, is the signal-to-noise ratio of the high-resolution image after denoising.
[0168] The signal-to-noise ratio of any image , is the denoising step number indicator, For the The noise processing parameters of the step, , is the step number indicator, For the The noise factor of the step, , For the Diffusion speed when adding noise.
[0169] By adjusting By adjusting the value of , a new noise coefficient can be obtained under different resolution image generation tasks to adapt to the generation capability of the original model and improve the quality and fidelity of image generation.
[0170] The present embodiment provides a high-quality face restoration method, obtaining a material to be restored; wherein the material to be restored is a low-quality image to be restored, or the material to be restored is a low-quality image to be restored and a high-quality reference image, and the image to be restored and the reference image correspond to the same identity information; the material to be restored is restored by a pre-trained restoration model to obtain a restored image; wherein, if the material to be restored is a low-quality image to be restored, the restoration model is a universal restoration model; if the material to be restored is a low-quality image to be restored and a high-quality reference image, the restoration model is a restoration model for a specific identity; the universal restoration model is trained based on a real sample image and a corresponding low-quality sample image; the restoration model for a specific identity is trained based on a real sample image, a corresponding low-quality sample image and a corresponding sample identity information. The method provided in the present embodiment enhances image quality.
[0171] Based on the same inventive concept of a high-quality face restoration method, this embodiment provides a high-quality face restoration device, see Figure 4 , the device comprises:
[0172] The acquisition module 401 is used to acquire the material to be restored, wherein the material to be restored is a low-quality image to be restored, or the material to be restored is a low-quality image to be restored and a high-quality reference image, and the image to be restored and the reference image correspond to the same identity information.
[0173] The restoration module 402 is used to restore the material to be restored obtained by the acquisition module 401 by using a pre-trained restoration model to obtain a restored image.
[0174] If the material to be restored is a low-quality image to be restored, the restoration model is a universal restoration model. If the material to be restored is a low-quality image to be restored and a high-quality reference image, the restoration model is a restoration model for a specific identity.
[0175] The general restoration model is trained based on real sample images and corresponding low-quality sample images. The identity-specific restoration model is trained based on real sample images, corresponding low-quality sample images and corresponding sample identity information.
[0176] Among them, the repair model is a stable diffusion model including a blind repair control network BFRControlNet, the stable diffusion model includes a U-Net model, the U-Net model includes an encoder and an intermediate block, and BFRControlNet replicates the encoder and the middleware.
[0177] The device also includes: a training module, which is used to train BFRControlNet using real sample images and corresponding low-quality sample images to obtain a universal repair model. The corresponding sample identity information is injected into the universal repair model. The universal repair model injected is trained based on the real sample images, the corresponding low-quality sample images and the corresponding sample identity information to obtain a repair model for a specific identity.
[0178] Among them, in each layer of the U-Net model of the specific identity restoration model, the corresponding low-quality sample image and the corresponding sample identity information are involved in the calculation.
[0179] Among them, in the general restoration model, BFRControlNet splices the channels of the corresponding low-quality image samples with the channels of the noise variables of the stable diffusion model.
[0180] Among them, the loss function of BFRControlNet .
[0181] in, is the loss function of BFRControlNet, For the The latent space variables of the denoised image. For text prompt conditions, is the number of denoising steps, For noise, For low quality images, is the noise prediction parameter, For the The prediction noise of the step, is the expected function.
[0182] Among them, the training module is used to inject the corresponding sample identity information into the general repair model by decoupling the cross-attention mechanism.
[0183] Among them, the decoupled cross attention mechanism is implemented using the following formula:
[0184] .
[0185] in, is the cross attention mechanism, For query, A key for a generic repair model, is the value of the general repair model, is a key based on the corresponding sample identity information, is the value based on the corresponding sample identity information, is the normalization function, for The dimension of for The transpose of for The transpose of .
[0186] If the material to be restored is a low-quality image to be restored and a high-quality reference image, the restoration module 402 is used to obtain identity information based on the reference image. The image to be restored, the reference image and the identity information are input into a restoration model of a specific identity to obtain a restored image.
[0187] Among them, the loss function of the restoration model of a specific identity when obtaining the restored image is .
[0188] in, The loss function for obtaining the restored image for the restoration model of a specific identity, For the The latent space variables of the denoised image. For text prompt conditions, is the number of denoising steps, For noise, is the image to be repaired, is the reference image, is the noise prediction parameter, For the The prediction noise of the step, is the expected function.
[0189] Among them, the restoration module 402 is used to project the reference image through a multi-layer perceptron MLP to obtain the projected identity features. The projected identity features are used as the basis for querying the converter Q-Former to query the image encoder CLIP image features in the reference image. The projected identity features are combined with the CLIP image features using a shortcut structure to obtain identity information.
[0190] The device provided in this embodiment enhances image quality.
[0191] Based on the same inventive concept of a high-quality face restoration method, this embodiment provides an electronic device, such as Figure 5 As shown, it includes: a memory 501, a processor 502, and a computer program.
[0192] The computer program is stored in the memory 501 and is configured to be executed by the processor 502 to implement the above-mentioned high-quality face restoration method.
[0193] Specifically,
[0194] The material to be restored is obtained, wherein the material to be restored is a low-quality image to be restored, or the material to be restored is a low-quality image to be restored and a high-quality reference image, and the image to be restored and the reference image correspond to the same identity information.
[0195] The material to be repaired is repaired through the pre-trained restoration model to obtain the restored image.
[0196] If the material to be restored is a low-quality image to be restored, the restoration model is a universal restoration model. If the material to be restored is a low-quality image to be restored and a high-quality reference image, the restoration model is a restoration model for a specific identity.
[0197] The general restoration model is trained based on real sample images and corresponding low-quality sample images. The identity-specific restoration model is trained based on real sample images, corresponding low-quality sample images and corresponding sample identity information.
[0198] Optionally, the repair model is a stable diffusion model including a blind repair control network BFRControlNet, the stable diffusion model includes a U-Net model, the U-Net model includes an encoder and an intermediate block, and BFRControlNet replicates the encoder and the intermediate block.
[0199] Repair model training process, including:
[0200] Use real sample images and corresponding low-quality sample images to train BFRControlNet and obtain a general repair model.
[0201] Inject the corresponding sample identity information into the general repair model.
[0202] Based on the real sample images, the corresponding low-quality sample images and the corresponding sample identity information, the injected general restoration model is trained to obtain the identity-specific restoration model.
[0203] Among them, in each layer of the U-Net model of the specific identity restoration model, the corresponding low-quality sample image and the corresponding sample identity information are involved in the calculation.
[0204] Optionally, in the general inpainting model, BFRControlNet concatenates the channels of the corresponding low-quality image samples with the channels of the noise variables of the stable diffusion model.
[0205] Optionally, the loss function of BFRControlNet .
[0206] in, is the loss function of BFRControlNet, For the The latent space variables of the denoised image. For text prompt conditions, is the number of denoising steps, For noise, For low quality images, is the noise prediction parameter, For the The prediction noise of the step, is the expected function.
[0207] Optionally, inject the corresponding sample identity information into the general repair model, including:
[0208] By decoupling the cross-attention mechanism, the corresponding sample identity information is injected into the general inpainting model.
[0209] Among them, the decoupled cross attention mechanism is implemented using the following formula:
[0210] .
[0211] in, is the cross attention mechanism, For query, A key for a generic repair model, is the value of the general repair model, is a key based on the corresponding sample identity information, is the value based on the corresponding sample identity information, is the normalization function, for The dimension of for The transpose of for The transpose of .
[0212] Optionally, if the material to be restored is a low-quality image to be restored and a high-quality reference image, the material to be restored is restored by using a pre-trained restoration model to obtain a restored image, including:
[0213] The identity information is obtained based on the reference image.
[0214] The image to be repaired, the reference image and the identity information are input into the restoration model of a specific identity to obtain the restored image.
[0215] Among them, the loss function of the restoration model of a specific identity when obtaining the restored image is .
[0216] in, The loss function for obtaining the restored image for the restoration model of a specific identity, For the The latent space variables of the denoised image. For text prompt conditions, is the number of denoising steps, For noise, is the image to be repaired, is the reference image, is the noise prediction parameter, For the The prediction noise of the step, is the expected function.
[0217] Optionally, obtaining identity information based on a reference image includes:
[0218] The reference image is projected through a multi-layer perceptron MLP to obtain the projected identity features.
[0219] The projected identity features are used as a basis for querying the transformer Q-Former to query the image encoder CLIP image features in the reference image.
[0220] The projected identity features are combined with the CLIP image features using a shortcut structure to obtain identity information.
[0221] This embodiment provides an electronic device on which a computer program is executed by a processor to enhance image quality.
[0222] Based on the same inventive concept of a high-quality face restoration method, this embodiment provides a computer-readable storage medium, and a computer program is stored thereon. The computer program is executed by a processor to implement the high-quality face restoration method.
[0223] Specifically,
[0224] The material to be restored is obtained, wherein the material to be restored is a low-quality image to be restored, or the material to be restored is a low-quality image to be restored and a high-quality reference image, and the image to be restored and the reference image correspond to the same identity information.
[0225] The material to be repaired is repaired through the pre-trained restoration model to obtain the restored image.
[0226] If the material to be restored is a low-quality image to be restored, the restoration model is a universal restoration model. If the material to be restored is a low-quality image to be restored and a high-quality reference image, the restoration model is a restoration model for a specific identity.
[0227] The general restoration model is trained based on real sample images and corresponding low-quality sample images. The identity-specific restoration model is trained based on real sample images, corresponding low-quality sample images and corresponding sample identity information.
[0228] Optionally, the repair model is a stable diffusion model including a blind repair control network BFRControlNet, the stable diffusion model includes a U-Net model, the U-Net model includes an encoder and an intermediate block, and BFRControlNet replicates the encoder and the intermediate block.
[0229] Repair model training process, including:
[0230] Use real sample images and corresponding low-quality sample images to train BFRControlNet and obtain a general repair model.
[0231] Inject the corresponding sample identity information into the general repair model.
[0232] Based on the real sample images, the corresponding low-quality sample images and the corresponding sample identity information, the injected general restoration model is trained to obtain the identity-specific restoration model.
[0233] Among them, in each layer of the U-Net model of the specific identity restoration model, the corresponding low-quality sample image and the corresponding sample identity information are involved in the calculation.
[0234] Optionally, in the general inpainting model, BFRControlNet concatenates the channels of the corresponding low-quality image samples with the channels of the noise variables of the stable diffusion model.
[0235] Optionally, the loss function of BFRControlNet .
[0236] in, is the loss function of BFRControlNet, For the The latent space variables of the denoised image. For text prompt conditions, is the number of denoising steps, For noise, For low quality images, is the noise prediction parameter, For the The prediction noise of the step, is the expected function.
[0237] Optionally, inject the corresponding sample identity information into the general repair model, including:
[0238] By decoupling the cross-attention mechanism, the corresponding sample identity information is injected into the general inpainting model.
[0239] Among them, the decoupled cross attention mechanism is implemented using the following formula:
[0240] .
[0241] in, is the cross attention mechanism, For query, A key for a generic repair model, is the value of the general repair model, is a key based on the corresponding sample identity information, is the value based on the corresponding sample identity information, is the normalization function, for The dimension of for The transpose of for The transpose of .
[0242] Optionally, if the material to be restored is a low-quality image to be restored and a high-quality reference image, the material to be restored is restored by using a pre-trained restoration model to obtain a restored image, including:
[0243] The identity information is obtained based on the reference image.
[0244] The image to be repaired, the reference image and the identity information are input into the restoration model of a specific identity to obtain the restored image.
[0245] Among them, the loss function of the restoration model of a specific identity when obtaining the restored image is .
[0246] in, The loss function for obtaining the restored image for the restoration model of a specific identity, For the The latent space variables of the denoised image. For text prompt conditions, is the number of denoising steps, For noise, is the image to be repaired, is the reference image, is the noise prediction parameter, For the The prediction noise of the step, is the expected function.
[0247] Optionally, obtaining identity information based on a reference image includes:
[0248] The reference image is projected through a multi-layer perceptron MLP to obtain the projected identity features.
[0249] The projected identity features are used as a basis for querying the transformer Q-Former to query the image encoder CLIP image features in the reference image.
[0250] The projected identity features are combined with the CLIP image features using a shortcut structure to obtain identity information.
[0251] This embodiment provides a computer-readable storage medium, on which a computer program is executed by a processor to enhance image quality.
[0252] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present application may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.
[0253] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0254] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0255] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0256] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.
[0257] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A facial image restoration method, characterized in that: The method comprises: Acquire a material to be restored; wherein the material to be restored is a low-quality image to be restored, or the material to be restored is a low-quality image to be restored and a high-quality reference image, and the image to be restored and the reference image correspond to the same identity information; Repairing the material to be repaired by using a pre-trained repair model to obtain a repaired image; Wherein, if the material to be repaired is a low-quality image to be repaired, the repair model is a general repair model; if the material to be repaired is a low-quality image to be repaired and a high-quality reference image, the repair model is a repair model for a specific identity; wherein the repair model is a stable diffusion model including a blind repair control network BFRControlNet, the stable diffusion model includes a U-Net model, the U-Net model includes an encoder and an intermediate block, and the BFRControlNet replicates the encoder and the intermediate block; The general inpainting model is trained based on real sample images and corresponding low-quality sample images; the identity-specific inpainting model is trained based on real sample images, corresponding low-quality sample images and corresponding sample identity information; for the general inpainting module, the BFRControlNet is trained using real sample images and corresponding low-quality sample images to obtain a general inpainting model; in the general inpainting model, the BFRControlNet concatenates the channels of the corresponding low-quality image samples with the channels of the noise variables of the stable diffusion model; For the restoration model of a specific identity, injecting the corresponding sample identity information into the general restoration model includes: injecting the corresponding sample identity information into the general restoration model through a decoupled cross attention mechanism; wherein the decoupled cross attention mechanism is implemented using the following formula: ;in, is a cross attention mechanism, Q is a query, K is a key of the universal repair model, V is a value of the universal repair model, is a key based on the corresponding sample identity information, V' is a value based on the corresponding sample identity information, is the normalization function, d is the dimension of K, is the transpose of K, for The transpose of Based on the real sample images, the corresponding low-quality sample images and the corresponding sample identity information, the injected general restoration model is trained to obtain the identity-specific restoration model; Among them, in each layer of the U-Net model of the specific identity restoration model, the corresponding low-quality sample image and the corresponding sample identity information are involved in the calculation.
2. The method according to claim 1, characterized in that Loss function of BFRControlNet ; in, is the loss function of BFRControlNet, is the latent space variable of the image after denoising at step t, c is the text prompt condition, t is the number of denoising steps, For noise, For low quality images, is the noise prediction parameter, is the prediction noise at step t, is the expected function.
3. The method according to claim 1, characterized in that If the material to be repaired is a low-quality image to be repaired and a high-quality reference image, repairing the material to be repaired by using a pre-trained repair model to obtain a repaired image includes: obtaining identity information based on the reference image; Inputting the image to be repaired, the reference image and the identity information into a repair model of a specific identity to obtain a repaired image; Among them, the loss function of the restoration model of the specific identity when obtaining the restored image is ; in, The loss function for obtaining the restored image for the restoration model of a specific identity, is the latent space variable of the image after denoising at step t, c is the text prompt condition, t is the number of denoising steps, For noise, is the image to be repaired, is the reference image, is the noise prediction parameter, is the prediction noise at step t, is the expected function.
4. The method according to claim 3, characterized in that The obtaining of identity information based on the reference image includes: Projecting the reference image through a multi-layer perceptron MLP to obtain a projected identity feature; Using the projected identity feature as a basis for querying the converter Q-Former to query the image encoder CLIP image feature in the reference image; The projected identity feature is combined with the CLIP image feature using a shortcut structure to obtain identity information.
5. A facial image restoration device, characterized in that: The device comprises: An acquisition module is used to acquire a material to be restored; wherein the material to be restored is a low-quality image to be restored, or the material to be restored is a low-quality image to be restored and a high-quality reference image, and the image to be restored and the reference image correspond to the same identity information; A restoration module, used to restore the material to be restored obtained by the acquisition module through a pre-trained restoration model to obtain a restored image; Wherein, if the material to be repaired is a low-quality image to be repaired, the repair model is a general repair model; if the material to be repaired is a low-quality image to be repaired and a high-quality reference image, the repair model is a repair model for a specific identity; wherein the repair model is a stable diffusion model including a blind repair control network BFRControlNet, the stable diffusion model includes a U-Net model, the U-Net model includes an encoder and an intermediate block, and the BFRControlNet replicates the encoder and the intermediate block; The general inpainting model is trained based on real sample images and corresponding low-quality sample images; the identity-specific inpainting model is trained based on real sample images, corresponding low-quality sample images and corresponding sample identity information; for the general inpainting module, the BFRControlNet is trained using real sample images and corresponding low-quality sample images to obtain a general inpainting model; in the general inpainting model, the BFRControlNet concatenates the channels of the corresponding low-quality image samples with the channels of the noise variables of the stable diffusion model; For the restoration model of a specific identity, injecting the corresponding sample identity information into the general restoration model includes: injecting the corresponding sample identity information into the general restoration model through a decoupled cross attention mechanism; wherein the decoupled cross attention mechanism is implemented using the following formula: ;in, is a cross attention mechanism, Q is a query, K is a key of the universal repair model, V is a value of the universal repair model, is a key based on the corresponding sample identity information, is the value based on the corresponding sample identity information, is the normalization function, for The dimension of for The transpose of for The transpose of Based on the real sample images, the corresponding low-quality sample images and the corresponding sample identity information, the injected general restoration model is trained to obtain the identity-specific restoration model; Among them, in each layer of the U-Net model of the specific identity restoration model, the corresponding low-quality sample image and the corresponding sample identity information are involved in the calculation.
6. An electronic device, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 4.
7. A computer-readable storage medium, characterized in that: A computer program is stored thereon; the computer program is executed by a processor to implement the method according to any one of claims 1 to 4.
Citation Information
Patent Citations
Face image restoration method and system
CN114219728A
Face image diversified restoration method based on sample guidance
CN114549341A
Gait sequence repairing and monitoring audience identity recognition method based on priori knowledge
CN115761906A
Face blind restoration method and device, equipment and medium
CN118537265A
Face inpainting using piece-wise affine warping and sparse coding
WO2016050729A1
