Face image restoration method based on inverse mapping of generative adversarial network

Abstract

The invention provides a face image restoration method based on generative adversarial network inverse mapping, and belongs to the technical field of image restoration and computer vision. According to the method, an encoder-decoder architecture is adopted, an encoder is a ResNet combined with a channel attention mechanism and can encode a to-be-restored face image (containing block-shaped shielding) to a potential space of a StyleGAN, and a W + vector of 18 * 512 dimensions is obtained; and the decoder is a pre-trained StyleGAN generator, and can convert the potential vector into a complete face image to realize restoration. In the training process, through a weighted combination constraint model of L2 loss, perception loss and face identity loss, it is ensured that a restoration result is consistent with an original image in pixel, feature and identity levels. Meanwhile, by means of the decoupling characteristic of the StyleGAN potential space, the repaired face features (such as smile and age) can be edited. According to the method, complex preprocessing is not needed, large-area missing face images can be efficiently repaired, identity consistency is kept, and the method is suitable for monitoring image enhancement, old photo repair and other scenes.

Description

Technical Field

[0001] This invention belongs to the field of image restoration and computer vision technology, and in particular relates to a method for face image restoration using inverse mapping of generative adversarial networks (GAN). It is suitable for processing face images with distortion problems such as block occlusion, especially for scenarios with large-area missing or key feature loss, and also supports editing of the restored face features. Background Technology

[0002] With the rapid development of electronic and computer technologies, images, as an important carrier of information, play an irreplaceable role in fields such as public safety, film production, and mobile phone photography. However, during the acquisition, transmission, and processing of images, partial loss or missing parts are easily caused by factors such as weather interference, equipment failure, and human obstruction. Examples include motion blur in surveillance images, mold and damage in old photos, and accidental obstruction in the captured image. These problems seriously affect the usability of images.

[0003] Image inpainting technology is based on the effective information from incomplete images and prior knowledge of the original images. Through algorithmic methods, it corrects regional losses and reconstructs a high-quality, semantically similar complete image. In the digital age, image inpainting has become a research hotspot in computer vision, with its application scenarios constantly expanding. For example, in public safety, inpainting technology improves the quality of surveillance images to assist in crime detection; in the film industry, it is used to remove objects from scenes that do not fit the context; and in mobile applications, it serves as an auxiliary means of photo processing to enhance user experience.

[0004] Face image restoration, as an important subtask of image restoration, presents unique challenges. The human face has a complex structure, containing multiple unique organs such as the eyes, nose, and mouth, and exhibits overall symmetry. The human eye has an extremely strong ability to recognize facial features. Even minor deviations in texture detail, structural balance, and symmetry in the restored face image are easily detected by the human eye. Therefore, face image restoration not only requires visual coherence between the restored area and its surrounding environment but also must ensure the authenticity of facial features and the consistency of identity information, placing higher demands on restoration algorithms.

[0005] Early image inpainting algorithms were mainly divided into two categories: methods based on partial differential equations (such as the BSCB and CDD algorithms), which describe the physical and mathematical features of an image by establishing a partial differential equation model and using boundary information and pixel relationships for inpainting. These methods can handle small-area defects well, but they do not consider the contextual semantic information of the image, resulting in poor repair performance for large-area defects and difficulty in restoring texture details. Methods based on texture synthesis (such as the Criminisi algorithm) fill missing areas using the texture similarity of undamaged areas in the image. While this method can supplement some texture information, it also lacks an understanding of global semantics, and is prone to structural distortions in highly complex images such as faces.

[0006] With the development of deep learning, image inpainting methods based on neural networks have gradually become mainstream, mainly including: (1) Recurrent Neural Network (RNN) based methods (such as PixelRNN), which treat the image pixel distribution as a time series and predict missing pixels through a temporal network. However, this method requires traversing all pixels, resulting in low computational efficiency and limited inpainting effect on high-resolution face images. (2) Encoder-decoder architecture based on Convolutional Neural Network (CNN), which extracts image features through the encoder and reconstructs the complete image through the decoder. For example, the Context Encoders method uses reconstruction loss and adversarial loss to constrain the inpainting results, but it only focuses on local information, which can easily lead to global semantic inconsistency. (3) Generative Adversarial Network (GAN) based methods, which use adversarial training between the generator and the discriminator to generate realistic inpainting results. Generative face completion methods introduce semantic parsing loss to improve the inpainting quality, but when faced with large-area occlusion, there are still problems such as blurry generated results and distorted identity information.

[0007] Existing methods for face image restoration generally have the following limitations: for cases with large-area missing areas or complete loss of key features (such as eyes and mouth), the restoration results are prone to structural imbalance and texture distortion; most methods rely on precise annotation or edge information of the missing areas, which increases the user's operational burden; and it is difficult to support flexible facial feature editing while restoring, which limits the practical application scenarios of the technology.

[0008] This invention constructs an encoder-decoder method based on Generative Adversarial Networks (GANs) and their inversion technique, specifically for face image inpainting. The encoder employs a ResNet combined with a channel attention mechanism, enabling the inversion of the face image to be inpainted into the GAN's latent space to obtain the corresponding latent vectors. The decoder uses a pre-trained StyleGAN generator, an advanced GAN model with powerful face generation capabilities, and its latent space (e.g., the W+ space) exhibits decoupling characteristics. Through this architecture, this invention cleverly transforms the face image inpainting task into a latent vector-based generation task: the latent vectors output by the encoder are input into the StyleGAN generator, which can reconstruct a complete face image highly similar to the input image to be inpainted, thus completing the missing regions. Simultaneously, leveraging the decoupling characteristics of the StyleGAN latent space, precise control over the restored facial features (such as expression and age) can be achieved by adjusting the latent vectors, thereby improving both the inpainting quality and the flexibility of the technology's application. Summary of the Invention

[0009] To address the problems of existing methods, this invention provides a face image inpainting method based on generative adversarial network (GAN) inverse mapping. It employs an encoder-decoder architecture and leverages GAN inverse mapping to transform the face image inpainting task into an image generation task, achieving high-quality restoration of missing face images while supporting facial feature editing. The specific technical solution is as follows:

[0010] 1. A face image inpainting method based on generative adversarial network inverse mapping, characterized by employing an encoder-decoder architecture. The encoder uses a ResNet combined with a channel attention mechanism, and the decoder uses a pre-trained StyleGAN generator. The method encodes the face image to be inpainted into the latent vector space of StyleGAN, utilizes the generative capabilities of StyleGAN to achieve face image inpainting, and leverages the editing capabilities of StyleGAN to modify facial features. The method includes the following steps:

[0011] S1, Construct a ResNet encoder based on a combined channel attention mechanism: The encoder adopts a modified ResNet structure, adding an SE module at the end of each residual module to implement the channel attention mechanism. The PReLU activation function is used instead of the ReLU activation function in the residual module, and a 3*3 max pooling layer with a stride of 2 is added to ensure that the input of the residual module matches the channel and spatial dimensions of the output of the previous layer.

[0012] S2 uses a pre-trained StyleGAN generator as the decoder: The StyleGAN generator has a progressive pyramid structure, which gradually generates high-resolution face images from low resolution. It takes the 18*512 dimension W+ vector output by the encoder as input, where the W vectors at different levels control different style features of the generated images.

[0013] S3, Model Training and Fine-tuning. First, the encoder is trained on the FFHQ dataset to obtain a pre-trained model. Then, faces in the FFHQ dataset are occluded and fed back into the pre-trained model for fine-tuning.

[0014] S4 enables face restoration and feature editing. By inputting an occluded image into the finely tuned model, a high-quality, complete image is obtained. Furthermore, features such as facial expression and age can be edited.

[0015] 2. The method as described in claim 1, wherein the construction of the ResNet encoder based on a channel attention mechanism in step S1 includes the following steps:

[0016] S11, Basic Network Structure Design. A modified ResNet architecture is adopted as the main encoder, containing multiple residual modules. Each residual module consists of convolutional layers, batch normalization (BatchNorm) layers, and activation functions. The convolutional layer configuration uses 3×3 kernels, with stride and padding set to 1 or 2 depending on the feature map size requirements. The output feature map size is calculated using the following formula:

[0017]

[0018] Activation function improvement: PReLU (parameterized ReLU) is used instead of traditional ReLU. This part can be expressed as:

[0019]

[0020] Here, α is a learnable parameter to prevent neuron death.

[0021] S12, Channel Attention Integration. An SE (Squeeze-and-Excitation) module is added to the end of each residual module to achieve adaptive feature enhancement between channels. This includes the following steps:

[0022] Compression involves performing global average pooling on the input feature map, compressing the two-dimensional features of each channel into one-dimensional real numbers:

[0023]

[0024] Excitation operation: Channel weights are learned through two fully connected layers (or 1×1 convolutions), and normalized weights are obtained after sigmoid activation.

[0025] s c =σ(W2δ(W1z) c (4)

[0026] Where σ is the Sigmoid function, δ is the PReLU activation function, and W1 and W2 are the parameters of the fully connected layer.

[0027] Reweighting: Multiplying the weights with the original feature map channel by channel to enhance the features of key channels.

[0028] u c =s c ·u c (5)

[0029] S13, Feature mapping to the latent space. The multi-layer feature maps extracted by the encoder are transformed into latent vectors required by StyleGAN through the map2style structure, and finally output a W+ vector of 18×512 dimensions (adapted for 1024×1024 image generation).

[0030] 3. The method as described in claim 1, characterized in that, step S2, employing a pre-trained StyleGAN as the decoder, includes the following steps:

[0031] S21, Decoder Structure Design. A pre-trained StyleGAN generator is used as the decoder, which has a progressive pyramid structure, gradually increasing the resolution from low resolution (4×4) to 1024×1024. Each layer injects a style vector through adaptive instance normalization (AdaIN).

[0032]

[0033] Where x i Let μ and σ be the input features, and μ and σ be the feature mean and standard deviation, respectively. s,i ,y b,i The scaling and bias parameters are obtained by affine transformation of the style vector y.

[0034] S22, Latent Vector Hierarchical Control. The encoder outputs 18×512 W+ vectors, corresponding to the 18 levels of StyleGAN, controlling features at different granularities:

[0035] Low layer (4×4, 8×8): Affects global features such as facial structure and hair shape;

[0036] Middle layer (16×16, 32×32): Controls detailed features such as facial expressions and appearance;

[0037] High-resolution (64×64 and above): Adjusts fine-grained features such as skin color and pupil color.

[0038] 4. The method as described in claim 1, wherein the model training and fine-tuning in step S3 includes the following steps:

[0039] S31, Loss Function Design. It includes three types of losses:

[0040] Pixel-level loss L2 is used to constrain pixel-level differences:

[0041] L2(x)=||xf(x)||2 (7)

[0042] Perceived loss L LPIPS Features are extracted using a pre-trained VGG network, and feature consistency is constrained:

[0043] L LPIPS =||F(x)-F(f(x))||2 (8)

[0044] Where F stands for VGG network.

[0045] Identity loss L ID Based on the ArcFace network, facial features are extracted, and cosine similarity is used to constrain identity consistency.

[0046] L ID (x)=1-<R(x),R(f(x))> (9)

[0047] The final loss function is a weighted sum of the three loss functions:

[0048] L=λ1L2+λ2L LPIPS +λ3L ID (10)

[0049] Where λ1=1, λ2=0.8, and λ3=0.2 are weight parameters.

[0050] S32, Inverse mapping pre-training (complete image reconstruction), the specific process is as follows:

[0051] Input image: a complete human face image x;

[0052] Output: The encoder encodes x into a W+ vector, which is then processed by StyleGAN to generate the reconstructed image f(x);

[0053] Training involves taking x as input to obtain an f(x) graph, calculating the loss function and gradient using the S31 method, updating the encoder parameters after backpropagation, and repeating this process several times to obtain the initialized face reconstruction encoder.

[0054] S33 involves fine-tuning the model for the face restoration task. Based on S32, the following training steps are performed:

[0055] Input: an image of a face with occlusion x input

[0056] Output: The image f(x) restored by the model.

[0057] Target: The corresponding complete image x real

[0058] Training: The S31 loss function is used again, but the calculation objects are changed to the output image f(x) and the complete image x. real After a few iterations, the final repair model was obtained.

[0059] 5. The method as described in claim 1, characterized in that the face image restoration and editing in step S4 includes the following steps:

[0060] S41, Image Restoration Process. The image to be restored is input into the finely tuned model, and the encoder outputs the W+ vector. The W+ vector is then input into StyleGAN to generate a complete restored image. Missing areas are filled in using the GAN's generation capabilities, while maintaining texture and identity consistency.

[0061] S42, Facial Feature Editing. Facial features are adjusted by modifying the W+ vector, such as smile editing.

[0062] Calculate the control vector: Subtract the W+ vector from the "smiling" and "not smiling" images of the same person to obtain the smile control vector W. smile ;

[0063] Edit the original vector according to the following formula:

[0064] W new =W + α·W smile (11)

[0065] Here, α is the weight, used to control the intensity of editing.

[0066] Edited vector W new The input can be made to make the input face smile.

[0067] Beneficial Effects: Compared with existing technologies, this invention provides a face image restoration method based on generative adversarial network inverse mapping, which can achieve high-quality image restoration and editing, and produces the following beneficial effects:

[0068] 1. Robust restoration results, adaptable to complex missing scenes: For facial images with large-area occlusion (such as half-face missing) or complete loss of key features (eyes, mouth), traditional methods are prone to structural distortion and blurred textures. This invention transforms restoration into a generation task through GAN inverse mapping. Leveraging the generative capabilities of StyleGAN, even with minimal input information, it can still generate complete images that conform to the structural rules of the face and have natural textures. For example, when the eyes are completely occluded, the model can generate eye details consistent with the overall style based on features such as face shape and hairstyle, resulting in a result that conforms to human visual perception.

[0069] 2. Simple operation, lowering the user threshold: Most existing restoration methods require manual annotation of missing areas or extraction of edge features, which is complex. This invention directly extracts features and encodes latent vectors from damaged images through an encoder, without requiring users to provide additional masks or prior information. Only the image to be restored needs to be input to output the results, greatly simplifying the process. It is suitable for practical scenarios such as mobile photo restoration and surveillance image enhancement.

[0070] 3. Maintaining identity consistency and avoiding distortion: Traditional methods often lead to identity confusion due to over-reliance on local texture filling. This invention introduces a face identity loss based on the ArcFace network. By constraining the feature similarity between the generated image and the original image, it ensures that the restoration result is consistent with the original face in key dimensions such as expression, skin color, and face shape. This is especially suitable for scenarios with high requirements for identity accuracy, such as public security investigation and identity verification.

[0071] 4. Supports feature editing, expanding application scenarios: Leveraging the decoupling characteristics of the StyleGAN latent space, this invention enables precise control of facial features based on restoration. For example, by modifying the dimensions corresponding to "smile" and "age" in the latent vector, restoration results for different expressions or age states can be generated. This integrated "restoration + editing" capability can be extended to fields such as film and television post-production and virtual makeup try-on, enhancing the practicality and commercial value of the technology. Attached Figure Description

[0072] Figure 1 This is a flowchart of a face image restoration method based on generative adversarial network inverse mapping.

[0073] Figure 2 The complete training process for this invention.

[0074] Figure 3 This invention describes the process of fine-tuning the model to achieve image restoration.

[0075] Figure 4 This is the result of the present invention in repairing eye and mouth obstructions.

[0076] Figure 5 This is a flowchart illustrating the image editing process of this invention.

[0077] Figure 6 This is a diagram illustrating the effect of image editing achieved by the present invention. Detailed Implementation

[0078] The present invention provides a face image inpainting method based on generative adversarial network inverse mapping. This method constructs an encoder-decoder architecture to encode damaged face images into the latent space of StyleGAN and generate inpainted results, while also supporting feature editing of the face images. The invention is further illustrated below with specific embodiments.

[0079] Example 1: Image Restoration for Occluded Faces

[0080] Data preparation: The FFHQ dataset with a resolution of 256×256 was used. Half-face occlusion (e.g., obscuring the right cheek and eye) was applied to the training set images using randomly generated masks, creating paired training data of "damaged image - complete image". The test set consisted of 100 real-world face images (covering different genders, ages, and skin tones) containing half-face occlusion.

[0081] Model Training: Inverse Mapping Pre-training: The complete face image is input into the encoder (a ResNet combined with channel attention mechanism), outputting an 18×512 W+ vector, which is then used by the pre-trained StyleGAN to generate the reconstructed image. A multi-objective loss function is used, with Adam as the optimizer, and training is performed for 500 epochs until the reconstruction loss converges. Repair Fine-tuning: Using the occluded damaged image as input and the corresponding complete image as the target, the model is fine-tuned for 100 epochs based on the pre-trained model to adapt the encoder to the extraction of missing features.

[0082] Repair process: The half-face occlusion image to be repaired is input into the finely tuned encoder, which outputs the W+ vector; StyleGAN generates a complete image based on this vector. The occluded area (right cheek and eye) is completed by the generator, and its texture and skin color are consistent with the unoccluded area (left face and hairstyle); Output result: The repaired face structure is symmetrical, the details of the right eye (such as pupil color and eyelashes) are coordinated with the left eye, and the overall face conforms to the original face's identity features (such as face shape and eyebrow shape).

[0083] Implementation Example 2: Editing a Face's Smile

[0084] like Figure 6 As shown, images of the same person smiling and not smiling are encoded separately to obtain W. smile and W normal Calculate the smile control vector ΔW = W smile -W normal .

[0085] For any input face image to be edited, first encode it to obtain a vector W, then calculate the edited vector W. new =W + α·ΔW (α = 0.8), where W new Inputting the image into StyleGAN will produce the final edited image.

[0086] Figure 1 The flowchart for image inpainting according to this invention includes an encoder and a decoder. The encoder uses a ResNet incorporating channel attention, and the decoder is a StyleGAN decoder.

[0087] Figure 2 This describes the complete training process for this invention. The dataset comes from FFHQ, and both the input and output are complete images, updating the encoder parameters.

[0088] Figure 3 This invention describes the process of fine-tuning the model to achieve image inpainting. The input is an occluded image; the generated image and the complete image are subjected to loss calculations, gradient backpropagation is performed, and then the encoding parameters are fine-tuned.

[0089] Figure 4 This invention presents the repair results under conditions of eye occlusion, mouth occlusion, and large-area loss. It can be seen that the invention achieves equally ideal repair results in three scenarios: symmetrical eye occlusion, asymmetrical mouth occlusion, and large-area occlusion.

[0090] Figure 5 This is a flowchart illustrating the image editing process of this invention. First, a smile vector is obtained, and then it is proportionally added to the W+ vector of the input image to achieve smile editing.

[0091] Figure 6 This is a diagram illustrating the effect of image editing achieved by the present invention. As can be seen, the present invention can edit smiles and ages, and obtain high-quality, realistic images.

[0092] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0093] While the specific embodiments of the present invention have been described above, they are not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A face image inpainting method based on generative adversarial network inverse mapping, characterized in that, An encoder-decoder architecture is adopted, with the encoder using ResNet combined with a channel attention mechanism and the decoder using a pre-trained StyleGAN generator. The face image to be restored is encoded into the latent vector space of StyleGAN, and the generative capabilities of StyleGAN are used to achieve face image restoration. Furthermore, the editing capabilities of StyleGAN are used to modify facial features, including the following steps: S1, Construct a ResNet encoder based on a combined channel attention mechanism: The encoder adopts a modified ResNet structure, adding an SE module at the end of each residual module to implement the channel attention mechanism. The PReLU activation function is used instead of the ReLU activation function in the residual module, and a 3*3 max pooling layer with a stride of 2 is added to ensure that the input of the residual module matches the channel and spatial dimensions of the output of the previous layer. S2 uses a pre-trained StyleGAN generator as the decoder: The StyleGAN generator has a progressive pyramid structure, which gradually generates high-resolution face images from low resolution. It takes the 18*512 dimension W+ vector output by the encoder as input, where the W vectors at different levels control different style features of the generated images. S3, Model Training and Fine-tuning. First, the encoder is trained on the FFHQ dataset to obtain a pre-trained model. Then, faces in the FFHQ dataset are occluded and fed back into the pre-trained model for fine-tuning. S4 enables face restoration and feature editing. By inputting an occluded image into the finely tuned model, a high-quality, complete image is obtained. Furthermore, features such as facial expression and age can be edited.

2. The method as described in claim 1, characterized in that, The construction of the ResNet encoder in step S1, which incorporates a channel attention mechanism, includes the following steps: S11, Basic Network Structure Design. A modified ResNet architecture is adopted as the main encoder, containing multiple residual modules. Each residual module consists of convolutional layers, batch normalization (BatchNorm) layers, and activation functions. The convolutional layer configuration uses 3×3 kernels, with stride and padding set to 1 or 2 depending on the feature map size requirements. The output feature map size is calculated using the following formula: Activation function improvement: PReLU (parameterized ReLU) is used instead of traditional ReLU. This part can be expressed as: Here, α is a learnable parameter to prevent neuron death. S12, Channel Attention Integration. An SE (Squeeze-and-Excitation) module is added to the end of each residual module to achieve adaptive feature enhancement between channels. This includes the following steps: Compression involves performing global average pooling on the input feature map, compressing the two-dimensional features of each channel into one-dimensional real numbers: Excitation operation: Channel weights are learned through two fully connected layers (or 1×1 convolutions), and normalized weights are obtained after sigmoid activation. s c =σ(W2δ(W1z c )) (4) Where σ is the Sigmoid function, δ is the PReLU activation function, and W1 and W2 are the parameters of the fully connected layer. Reweighting: Multiplying the weights with the original feature map channel by channel to enhance the features of key channels. in c =with c ·in c (5) S13, Feature mapping to the latent space. The multi-layer feature maps extracted by the encoder are transformed into latent vectors required by StyleGAN through the map2style structure, and finally output a W+ vector of 18×512 dimensions (adapted for 1024×1024 image generation).

3. The method as described in claim 1, characterized in that, Step S2, which uses a pre-trained StyleGAN as the decoder, includes the following steps: S21, Decoder Structure Design. A pre-trained StyleGAN generator is used as the decoder, which has a progressive pyramid structure, gradually increasing the resolution from low resolution (4×4) to 1024×1024. Each layer injects a style vector through adaptive instance normalization (AdaIN). Where x i Let μ and σ be the input features, and μ and σ be the feature mean and standard deviation, respectively. s,i ,y b,i The scaling and bias parameters are obtained by affine transformation of the style vector y. S22, Latent Vector Hierarchical Control. The encoder outputs 18×512 W+ vectors, corresponding to the 18 levels of StyleGAN, controlling features at different granularities: Low layer (4×4, 8×8): Affects global features such as facial structure and hair shape; Middle layer (16×16, 32×32): Controls detailed features such as facial expressions and appearance; High-resolution (64×64 and above): Adjusts fine-grained features such as skin color and pupil color.

4. The method as described in claim 1, characterized in that, The model training and fine-tuning in step S3 includes the following steps: S31, Loss Function Design. It includes three types of losses: Pixel-level loss L2 is used to constrain pixel-level differences: L2(x)=||xf(x)||2 (7) Perceived loss L LPIPS Features are extracted using a pre-trained VGG network, and feature consistency is constrained: L LPIPS =||F(x)-F(f(x))||2 (8) Where F stands for VGG network. Identity loss L ID Facial features are extracted based on the ArcFace network, and identity consistency is constrained by cosine similarity. L ID (x)=1-<R(x),R(f(x))> (9) The final loss function is a weighted sum of the three loss functions: L=λ1L2+λ2L LPIPS +λ3L ID (10) Where λ1=1, λ2=0.8, and λ3=0.2 are weight parameters. S32, Inverse mapping pre-training (complete image reconstruction), the specific process is as follows: Input image: a complete human face image x; Output: The encoder encodes x into a W+ vector, which is then processed by StyleGAN to generate the reconstructed image f(x); Training involves taking x as input to obtain an f(x) graph, calculating the loss function and gradient using the S31 method, updating the encoder parameters after backpropagation, and repeating this process several times to obtain the initialized face reconstruction encoder. S33 involves fine-tuning the model for the face restoration task. Based on S32, the following training steps are performed: Input: an image of a face with occlusion x input Output: The image f(x) restored by the model. Target: The corresponding complete image x real Training: The S31 loss function is used again, but the calculation objects are changed to the output image f(x) and the complete image x. real After a few iterations, the final repair model was obtained.

5. The method as described in claim 1, characterized in that, The face image restoration and editing in step S4 includes the following steps: S41, Image Restoration Process. The image to be restored is input into the finely tuned model, and the encoder outputs the W+ vector. The W+ vector is then input into StyleGAN to generate a complete restored image. Missing areas are filled in using the GAN's generation capabilities, while maintaining texture and identity consistency. S42, Facial Feature Editing. Facial features are adjusted by modifying the W+ vector, such as smile editing. Calculate the control vector: Subtract the W+ vector from the "smiling" and "not smiling" images of the same person to obtain the smile control vector W. smile ; Edit the original vector according to the following formula: W new =W+α·W smile (11) Here, α is the weight, used to control the intensity of editing. Edited vector W new The input can be made to make the input face smile.

Images