A blind face recovery method based on domain-aligned GAN priors

By employing a blind face restoration method based on domain-aligned GAN priors, and utilizing the loss functions of the generator and discriminator for training, generative face prior features are fused to solve the problem of restoring low-quality face images and achieve effective restoration of high-quality face images.

CN116362991BActive Publication Date: 2025-10-28TSINGHUA SHENZHEN INTERNATIONAL GRADUATE SCHOOL
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310061184.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-01-17
Publication Date
2025-10-28
Estimated Expiration
2043-01-17

AI Technical Summary

Technical Problem

Existing technologies struggle to effectively recover high-quality facial images with rich details, high fidelity, and good clarity from low-quality facial images, especially when high-quality facial references are lacking in real-world scenarios.

Method used

A blind face restoration method based on domain-aligned GAN priors is adopted. By constructing a network including a domain-aligned GAN inversion branch and a feature extraction-fusion branch, a pre-trained generator and discriminator are used to train loss functions in the image domain and the latent space domain. Generative face prior features and low-resolution image features are fused to achieve high-quality face image restoration.

Benefits of technology

It has enabled the restoration of high-quality facial images with rich details, high fidelity, and good clarity from blurry faces, expanding the application scope of facial image restoration and improving the restoration effect of low-resolution facial images in the real world.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116362991B_ABST
    Figure CN116362991B_ABST
Patent Text Reader

Abstract

A method for blind face reconstruction based on domain-aligned GAN priors includes the following steps: acquiring a low-resolution face image training set; constructing a network, wherein the network includes a domain-aligned GAN inversion branch and a feature extraction-fusion branch; the domain-aligned GAN inversion branch includes an encoder E w The pre-trained generator G, the image domain discriminator D, and the latent spatial domain discriminator Di w The feature extraction-fusion branch includes an encoder and a decoder, with the decoder further containing a feature fusion module. The training set is input into the network for training, where loss functions are defined from the image domain and the latent space domain of the GAN network, respectively, to constrain the network training. The low-resolution face image to be restored is input into the trained network for recovery, obtaining a restored high-resolution face image. This invention can restore face images with rich details, high fidelity, and excellent clarity from blurry faces.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of face restoration technology, and in particular to a blind face restoration method based on domain-aligned GAN (Generative Adversarial Network) priors. Background Technology

[0002] With the development of computers and artificial intelligence, applications such as ultra-high-definition video, face editing, and face recognition are emerging in large numbers, increasing the demand for and reliance on high-quality facial images. However, in the real world, due to objective factors such as lighting, weather, equipment, and compression sampling during storage, facial images often suffer from complex and diverse unknown degradation, including low resolution, noise, blurring, and compression, thus failing to meet the needs of a range of application scenarios. Therefore, the restoration of low-quality facial images is an urgent challenge that needs to be addressed.

[0003] Blind face restoration is an image processing technique that reconstructs high-quality faces with realistic details from low-quality face images with unknown degradation. In recent years, due to the enormous potential of face restoration technology in practical applications, many blind face restoration algorithms have been proposed. Early works mainly utilized inherent facial attributes of face images, such as facial resolution maps, facial landmarks, and facial heatmaps, as prior knowledge to help restore the geometric features and facial details of the face. However, these methods often fail to achieve satisfactory results due to inaccurate geometric priors extracted from low-quality images and a lack of rich texture details. Other methods use high-quality faces as references to guide the restoration of low-quality faces, but ignore the fact that high-quality face references are unavailable in real-world scenes, thus limiting the practicality of the algorithms.

[0004] It should be noted that the information disclosed in the background section above is only for understanding the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The main objective of this invention is to overcome the shortcomings of the aforementioned background technology and provide a blind face recovery method based on domain-aligned GAN priors, so as to restore a face image with richer details, higher fidelity, and better clarity from a blurred face.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In the first aspect, a blind face recovery method based on domain-aligned GAN priors includes the following steps:

[0008] A1. Obtain a low-resolution training set of human face images;

[0009] A2. Construct a network, which includes a domain-aligned GAN inversion branch and a feature extraction-fusion branch; the domain-aligned GAN inversion branch includes an encoder E w The pre-trained generator G, the image domain discriminator D, and the latent spatial domain discriminator Di w The encoder E w The low-resolution input image is inverted into the latent space of the GAN network to obtain a latent encoding. The pre-trained generator G is used to obtain generative face prior features from the corresponding latent encoding. The image domain discriminator D is used to distinguish between the reconstructed image and the real image. The latent space domain discriminator D... w This is used to align the domain of the inverted latent encoding with the original latent encoding trained by the GAN network; the feature extraction-fusion branch includes an encoder and a decoder, wherein the decoder further includes a feature fusion module;

[0010] A3. Input the training set into the network for training, wherein loss functions are defined from the image domain and the latent space domain of the GAN network respectively to constrain the training of the network;

[0011] A4. Input the low-resolution face image that needs to be restored into the trained network for restoration to obtain the restored high-resolution face image.

[0012] In a second aspect, a computer-readable storage medium stores a computer program that, when executed by a processor, implements the aforementioned blind face recovery method based on domain-aligned GAN priors.

[0013] The present invention has the following beneficial effects:

[0014] This invention proposes a blind face restoration method based on domain-aligned GAN priors. It utilizes generative face prior information, such as color and texture, contained in a pre-trained GAN network (e.g., the StyleGAN face generator) to assist in restoring faithful details during face reconstruction. Unlike previous GAN-based face restoration methods, this invention employs a domain-aligned GAN inversion branch, enabling the network to more effectively utilize generative face priors. Furthermore, it achieves a balance between detail fidelity and accuracy in the restored face through feature fusion. This method can effectively reconstruct more detailed, higher-fidelity, and clearer face images from blurred faces, thereby expanding the applicability of other face image-related technologies.

[0015] The advantages of the embodiments of the present invention include:

[0016] 1. Add the discriminator D wTrain a GAN inversion branch to align the inverted latent encoding with the real latent space, thereby improving the quality of generative face priors.

[0017] 2. The feature extraction-fusion branch of the present invention integrates multi-resolution convolutional features extracted from the low-resolution input image and generative face prior features extracted from the generator G through the feature fusion module. This can better balance the restoration of face regions and image background, fully restore the faithful details of the original low-resolution image, and maintain face fidelity.

[0018] 3. The embodiments of the present invention can restore complex and degraded low-resolution facial images in the real world, and have better effects and practicality compared with existing methods. Attached Figure Description

[0019] Figure 1 This is a flowchart of a blind face restoration method based on domain-aligned GAN priors according to an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of the network structure according to an embodiment of the present invention.

[0021] Figure 3 This is a flowchart illustrating the network training process according to an embodiment of the present invention. Detailed Implementation

[0022] The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary and not intended to limit the scope and application of the present invention.

[0023] See Figure 1 This invention provides a method for blind face recovery based on domain-aligned GAN priors, comprising the following steps:

[0024] A1. Obtain a low-resolution training set of human face images;

[0025] A2. Construct a network, which includes a domain-aligned GAN inversion branch and a feature extraction-fusion branch; the domain-aligned GAN inversion branch includes an encoder E w The pre-trained generator G, the image domain discriminator D, and the latent spatial domain discriminator Di w The encoder E w The low-resolution input image is inverted into the latent space of the GAN network to obtain a latent encoding. The pre-trained generator G is used to obtain generative face prior features from the corresponding latent encoding. The image domain discriminator D is used to distinguish between the reconstructed image and the real image. The latent space domain discriminator D... wThis is used to align the domain of the inverted latent encoding with the original latent encoding trained by the GAN network; the feature extraction-fusion branch includes an encoder and a decoder, wherein the decoder further includes a feature fusion module;

[0026] A3. Input the training set into the network for training, wherein loss functions are defined from the image domain and the latent space domain of the GAN network respectively to constrain the training of the network;

[0027] A4. Input the low-resolution face image that needs to be restored into the trained network for restoration to obtain the restored high-resolution face image.

[0028] The following describes specific embodiments of the present invention.

[0029] See Figures 2 to 3 In some embodiments, the blind face recovery method based on domain-aligned GAN priors may include the following steps:

[0030] S1: Training dataset construction: Obtain N high-resolution face images from an open-source high-resolution face image database. HQ The corresponding low-resolution face image I is obtained after degradation processing. LQ This constitutes the training set;

[0031] S2: Network Construction: Building the network structure required for training;

[0032] S3: Define the loss function: Define the loss function from the image domain and the latent space domain of the GAN network (such as the face generation network StyleGAN) to restrict the training of the network;

[0033] S4: Network Training: Input the training set into the network for training;

[0034] S5: Low-resolution face image restoration: Input the low-resolution face image to be restored into the trained network for restoration to obtain the restored high-resolution face image.

[0035] The open-source high-definition face image database can use the FFHQ face dataset, which contains 70,000 high-definition face images.

[0036] The degradation process can be implemented in the following ways:

[0037] High-resolution facial images undergo a series of degradation processes with random parameters, including blurring, downsampling, adding Gaussian noise, and JPEG compression. The mathematical formula for this process can be expressed as:

[0038]

[0039] where k σ n δ Represents the fuzzy kernel and Gaussian noise, operator These represent 2D convolution, downsampling, and JPEG compression, respectively. The parameters σ, s, δ, and q take values ​​in the ranges {0, 2:10}, {1:8}, {0:15}, and {60:100}, respectively.

[0040] Preferably, the network structure constructed in step S2 includes the following structure:

[0041] The domain-aligned GAN inversion branch is generated by an encoder E. w A pre-trained generator G, an image domain discriminator D, and a latent spatial domain discriminator Di w composition.

[0042] The feature extraction-fusion branch consists of an encoder and a decoder, with the decoder also containing the designed feature fusion module.

[0043] Preferably, the loss function defined in step S3 includes:

[0044] (1) Image reconstruction loss, including L2 loss in pixel space and perceptual loss in feature space for reconstructed image and input low-resolution image.

[0045] (2) Adversarial loss in image space can generate realistic textures more effectively during image restoration.

[0046] (3) Adversarial loss of the latent space of the GAN network aligns the latent code obtained by the encoder from the reverse low-resolution image with the latent code during the training of the original GAN ​​network.

[0047] See Figures 2 to 3 In some embodiments, a blind face recovery method based on domain-aligned GAN priors may specifically include the following steps:

[0048] S1, Training Dataset Construction

[0049] 70,000 high-resolution face images were downloaded from the FFHQ open-source face dataset and randomly divided into training and test sets at a ratio of 9:1. All images were processed using a randomized hybrid degradation model to construct a low-resolution-high-resolution paired dataset for training.

[0050] Preferably, the stochastic hybrid degradation model can be expressed as:

[0051]

[0052] where k σ nδ Represents the fuzzy kernel and Gaussian noise, operator These represent 2D convolution, downsampling, and JPEG compression, respectively. The parameters σ, s, δ, and q take values ​​in the ranges {0, 2:10}, {1:8}, {0:15}, and {60:100}, respectively.

[0053] S2, Network Construction

[0054] like Figure 2 As shown, the domain-aligned GAN inversion branch consists of the following parts: encoder E w A generator G is used to invert the low-resolution input image into the latent space of the GAN network to obtain the latent encoding; a pre-trained generator G is used to obtain generative face prior features from the corresponding latent encoding; a frequency discriminator D is used to distinguish the reconstructed image from the real image; frequency discriminator D w This is used to align the domain of the inverted latent encoding with the original latent encoding trained on the GAN network.

[0055] The feature extraction-fusion branch consists of an encoder and a decoder. The encoder is composed of multiple stacked resblocks used to extract convolutional features from the degraded placement image. We progressively halve the resolution of the feature map to 4. 2 Finally, the multi-resolution convolutional features are input into the decoder for image reconstruction. The decoder consists of multiple stacked resblocks, each followed by a feature fusion module to fuse two different features until the final result is generated. Furthermore, the decoder and encoder use skip connections to bridge the fusion of shallow and deep layers, providing more semantic information during the decoding process.

[0056] The feature fusion module adaptively integrates local features and global dependencies through a concatenated global attention module and local attention module. First, the global attention module selectively emphasizes certain feature maps across all channels. Then, it is concatenated with the local attention module to selectively aggregate features at each location, resulting in the final feature expression.

[0057] S3, Define the loss function

[0058] Typical image restoration problems aim to minimize the root mean square error (MSE) between the restored image and the original low-resolution image. However, this often results in a blurry restored image. Therefore, we incorporate adversarial loss from Generative Adversarial Networks (GANs) during model training to achieve clearer and more natural texture boundaries in the restored image. Furthermore, during the training of the E... wTo address this, we propose incorporating adversarial loss in the latent space domain, ensuring that the inverted latent encoding falls within a high-dimensional space with the same data distribution as the latent encoding trained on the original GAN ​​network. The modified loss function is as follows:

[0059] The loss function for training the domain-aligned GAN inversion branch is as follows:

[0060]

[0061] Among them I lq and I hq It consists of a degraded low-resolution input image and a reference high-resolution image. Ew is the encoder, G is the generator, and λ is the reference image. L2 and λ per φ(·) represents the weight hyperparameters of pixel and perceptual loss, respectively, and φ(·) represents the VGG feature extraction model;

[0062]

[0063] Where D is the latent spatial domain discriminator. These are the weight hyperparameters for image domain adversarial loss;

[0064]

[0065] Where D w It is a latent spatial domain discriminator. It is a potential spatial domain resistance loss;

[0066] Finally, when training the domain-aligned GAN inversion branch, the loss function is the reconstruction loss described above. Image domain adversarial loss Combating losses in potential spatial domains composition:

[0067]

[0068] The loss function for training the feature extraction-fusion branch is as follows:

[0069]

[0070] Among them I lq I hg and I rec λ represents the degraded low-resolution input image, the reference high-resolution image, and the final reconstructed image, respectively. L2 and λ per These represent the weight hyperparameters for pixel and perceptual loss, respectively;

[0071]

[0072] Where λadv The weighted hyperparameters representing the resistance to loss;

[0073] Finally, the loss function during training of the feature extraction-fusion branch is the reconstruction loss described above. and combat losses composition:

[0074]

[0075] S4: Network Training: Input the training set into the network for training;

[0076] Further, such as Figure 3 As shown, the network training includes at least:

[0077] S401: Train the domain-aligned GAN inversion branch. First train the E... w This process involves inverting the input low-resolution image into the latent space of a pre-trained GAN network to obtain a latent encoding, which is then input into the pre-trained generator G to obtain a reconstructed image. By limiting the reconstruction loss function between the reconstructed and high-resolution images, a model based on the PSNR metric is obtained, serving as the E... w Initialization;

[0078] S402: Add the D to the domain-aligned GAN inversion branch. w Along with D, adversarial losses in the image space and latent space are added for iterative training until the output layer error reaches the preset accuracy requirement or the training count reaches the maximum number of iterations, resulting in the trained E. w D w and D;

[0079] Wherein, the encoder E w and the discriminator D w The learning rates for D are 1e-5 and 1e-4, respectively, and are halved after every 3e4 iterations.

[0080] S403: E w D w With the network parameters of D fixed, the feature extraction-fusion branch is trained until the output layer error reaches the preset accuracy requirement or the training number reaches the maximum number of iterations. The current model network structure and parameters are saved to obtain the optimized model.

[0081] The learning rates of the Encoder and the Decoder are 1e-4 and 2e-4, respectively.

[0082] S5: Low-resolution face image restoration: Input the low-resolution face image to be restored into the trained network for restoration to obtain the restored high-resolution face image.

[0083] The blind face restoration method based on domain-aligned GAN priors in this invention utilizes generative face prior information, such as color and texture, contained in a pre-trained GAN network (e.g., the StyleGAN face generator), to assist in restoring faithful details during face restoration. Unlike previous face restoration methods based on GAN priors, this invention uses a domain-aligned GAN inversion branch to enable the network to more effectively utilize generative face priors, and achieves a balance between detail fidelity and accuracy in the restored face through feature fusion. This method can effectively restore face images with richer details, higher fidelity, and better clarity from blurred faces, thereby expanding the applicability of other face image application-related technologies.

[0084] The advantages of the embodiments of the present invention include:

[0085] Add the discriminator D w Train a GAN inversion branch to align the inverted latent encoding with the real latent space, thereby improving the quality of generative face priors.

[0086] The feature extraction-fusion branch of this invention integrates multi-resolution convolutional features extracted from the low-resolution input image and generative face prior features extracted from the generator G through the feature fusion module. This better balances the restoration of face regions and image background, fully restores the faithful details of the original low-resolution image, and maintains face fidelity.

[0087] The embodiments of the present invention can restore complex and degraded low-resolution facial images in the real world, and have better effects and practicality compared with existing methods.

[0088] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0089] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0090] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0092] The background section of this invention may include background information about the problems or environment in which the invention is being developed, and is not necessarily a description of prior art. Therefore, the content included in the background section does not constitute an admission of prior art by the applicant.

[0093] The above description provides a further detailed explanation of the present invention in conjunction with specific / preferred embodiments, and it should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various substitutions or modifications can be made to these described embodiments without departing from the concept of the present invention, and all such substitutions or modifications should be considered within the scope of protection of the present invention. In the description of this specification, the reference to terms such as "an embodiment," "some embodiments," "preferred embodiment," "example," "specific example," or "some examples," etc., indicates that the specific features, structures, materials, or characteristics described in connection with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples. Although the embodiments of the present invention and their advantages have been described in detail, it should be understood that various changes, substitutions, and modifications can be made herein without departing from the scope of protection of the patent application.

Claims

1. A blind face recovery method based on domain-aligned GAN priors, characterized in that, Includes the following steps: A1. Obtain a low-resolution training set of human face images; A2. Construct a network, which includes a domain-aligned GAN inversion branch and a feature extraction-fusion branch; the domain-aligned GAN inversion branch includes an encoder E w The pre-trained generator G, the image domain discriminator D, and the latent spatial domain discriminator Di w The encoder E w The low-resolution input image is inverted into the latent space of the GAN network to obtain a latent encoding. The pre-trained generator G is used to obtain generative face prior features from the corresponding latent encoding. The image domain discriminator D is used to distinguish between the reconstructed image and the real image. The latent space domain discriminator D... w This is used to align the domain of the inverted latent code and the original latent code trained by the GAN network; the feature extraction-fusion branch includes an encoder and a decoder, wherein the decoder further includes a feature fusion module, which fuses multi-resolution convolutional features extracted from the low-resolution input image and generative face prior features extracted from the pre-trained generator G. A3. Input the training set into the network for training, wherein loss functions are defined from the image domain and the latent space domain of the GAN network respectively to constrain the training of the network; A4. Input the low-resolution face image that needs to be restored into the trained network for restoration to obtain the restored high-resolution face image.

2. The blind face recovery method based on domain-aligned GAN prior as described in claim 1, characterized in that, In step A1, acquire N high-resolution face images I HQ The corresponding low-resolution face image I is obtained after degradation processing. LQ This constitutes the training set; The degradation process is performed in the following manner: The high-definition face image is subjected to randomized blurring, downsampling, Gaussian noise addition, and JPEG compression degradation processing, the mathematical formula of which is expressed as follows: ; in , Represents the fuzzy kernel and Gaussian noise, operator These represent 2D convolution, downsampling, and JPEG compression, respectively, with parameters... The values ​​of are {0, 2:10}, {1:8}, {0:15}, and {60:100}.

3. The blind face recovery method based on domain-aligned GAN prior as described in any one of claims 1 to 2, characterized in that, The feature fusion module adaptively integrates local features and global dependencies through a series of global attention modules and local attention modules. Specifically, the global attention module selectively emphasizes specific feature maps of all channels, and the local attention module is connected in series to selectively aggregate features at each position to obtain the final feature expression.

4. The blind face recovery method based on domain-aligned GAN prior as described in any one of claims 1 to 2, characterized in that, The loss function includes image reconstruction loss, image space adversarial loss, and GAN network latent space adversarial loss. The image reconstruction loss includes L2 loss in pixel space and perceptual loss in feature space for the reconstructed image and the input low-resolution image. The image space adversarial loss is used to generate realistic textures during image restoration. The GAN network latent space adversarial loss is used to align the latent encoding obtained by the encoder from the inverted low-resolution image with the latent encoding during the training of the original GAN ​​network.

5. The blind face recovery method based on domain-aligned GAN prior as described in any one of claims 1 to 2, characterized in that, The loss function for training the domain-aligned GAN inversion branch is as follows: ; in and It consists of a degraded low-resolution input image and a reference high-resolution image. Ew is the encoder, and G is the generator. and These represent the weight hyperparameters for pixel and perceptual loss, respectively. This represents the VGG feature extraction model; ; Where D is the latent spatial domain discriminator. These are the weight hyperparameters for image domain adversarial loss; ; Where D w It is a latent spatial domain discriminator. It is a potential spatial domain resistance loss; Finally, when training the domain-aligned GAN inversion branch, the loss function is the reconstruction loss described above. Image domain adversarial loss Combating losses in potential spatial domains composition: ; The loss function for training the feature extraction-fusion branch is as follows: ; in , and These represent the degraded low-resolution input image, the reference high-resolution image, and the final reconstructed image, respectively. and These represent the weight hyperparameters for pixel and perceptual loss, respectively; ; in The weighted hyperparameters representing the resistance to loss; Finally, the loss function during training of the feature extraction-fusion branch is the reconstruction loss described above. and combat losses composition: 。 6. The blind face recovery method based on domain-aligned GAN prior as described in any one of claims 1 to 2, characterized in that, In step A3, the training specifically includes the following steps: A31: Training the domain-aligned GAN inversion branch: First train the encoder E w This process involves inverting the input low-resolution image into the latent space of a pre-trained GAN network to obtain a latent encoding, which is then input into the pre-trained generator G to obtain a reconstructed image. A PSNR-based model is obtained by limiting the reconstruction loss function between the reconstructed and high-resolution images, serving as the encoder E. w Initialization; A32: Add the image domain discriminator D and the latent spatial domain discriminator D to the domain-aligned GAN inversion branch. w The image space and latent space adversarial losses are added for iterative training until the output layer error reaches the preset accuracy requirement or the training count reaches the maximum number of iterations, resulting in the trained encoder E. w The image domain discriminator D and the latent spatial domain discriminator D w ; A33: The encoder E w The image domain discriminator D and the latent spatial domain discriminator D w With the network parameters fixed, the feature extraction-fusion branch is trained until the output layer error reaches the preset accuracy requirement or the training number reaches the maximum number of iterations. The current model network structure and parameters are then saved to obtain the optimized model.

7. The blind face recovery method based on domain-aligned GAN prior as described in claim 6, characterized in that, The encoder E is set in stages A31 and A32. w and the latent spatial domain discriminator D w The learning rates of the image domain discriminator D are 1e-5 and 1e-4, respectively, and are halved after every 3e4 iterations; in step A33, the learning rates of the encoder and the decoder are set to 1e-4 and 2e-4, respectively.

8. The blind face recovery method based on domain-aligned GAN prior as described in any one of claims 1 to 2, characterized in that, The encoder consists of multiple stacked resblocks that extract convolutional features from the degraded placement image, gradually halving the resolution of the feature map to 4. 2 Finally, the multi-resolution convolutional features are input into the decoder for image reconstruction. The decoder consists of multiple stacked resblocks, each followed by a feature fusion module to fuse two different features until the final result is generated.

9. The blind face recovery method based on domain-aligned GAN prior as described in any one of claims 1 to 2, characterized in that, The decoder and encoder bridge the fusion of shallow and deep layers using a skip connection.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the blind face recovery method based on domain-aligned GAN prior as described in any one of claims 1 to 9.

Citation Information

Patent Citations

  • Blind person face image restoration method based on generative adversarial network prior

    CN114511466A

  • Face image restoration method and system

    CN114943656A