Face super-resolution method based on identity-style decoupling guidance, storage medium and equipment

Abstract

The invention discloses a face super-resolution method based on identity-style decoupling guidance, a storage medium and equipment, and the method comprises the steps: firstly collecting face image data, and constructing a training data pair; loading a pre-trained implicit diffusion model and freezing parameters; extracting geometric embedding and text embedding from the style reference face by using a style control network, and injecting the geometric embedding and the text embedding into a diffusion model through double-path cross attention to realize style guidance; the method comprises the following steps: preprocessing an input face by using an identity protection network, respectively extracting implicit facial features and identity embedding, and guiding a diffusion model in a multi-scale feature space; jointly training the model by using denoising loss and quality perception ternary identity loss, and updating model parameters; and finally, initializing a Gaussian noise map, using the optimized style control network and the identity protection network to guide the denoising network to carry out continuous iteration denoising, using a hidden space decoder to map a denoising result to a pixel space, and reconstructing a high-quality face image.

Description

Technical Field

[0001] This invention belongs to the technical field of face super-resolution, and mainly relates to a face super-resolution method, storage medium and device based on identity-style decoupling guidance. Background Technology

[0002] In recent years, with the rapid increase in the number of public transportation hubs, facial recognition technology has been widely used in traffic monitoring. Unlike controlled facial capture environments, such as facial recognition gates and financial payment systems, the imaging distance of facial images in traffic monitoring scenarios is difficult to control. Long-distance acquisition leads to a decrease in facial image resolution, resulting in low-resolution facial images. Furthermore, image quality is not only affected by imaging distance but is also often accompanied by complex lighting and changes in facial pose, leading to low-resolution facial images with non-frontal or complex lighting conditions. Facial distortion caused by non-frontal poses and color distortion caused by complex lighting are further exacerbated under low-resolution conditions, forming a complex and difficult-to-decouple composite degradation pattern. This composite degradation problem not only affects human visual observation but also significantly reduces the performance of downstream intelligent algorithms. Therefore, reconstructing high-quality facial images from composite-degraded low-resolution facial images is of great significance for the practical application of facial recognition algorithms.

[0003] Face super-resolution technology aims to improve the detail and clarity of facial images, transforming low-resolution face images into high-resolution ones. It is widely used in security monitoring, video communication, digital entertainment, and medical imaging. In recent years, with the rise of deep learning, neural network-based face super-resolution methods have made significant progress, achieving more accurate face super-resolution by learning the mapping relationships between a large number of low-resolution and high-resolution image pairs. Early face super-resolution methods mainly used convolutional neural networks and generative adversarial networks, which achieved significant results on low-resolution face images in standard scenes. However, when processing low-resolution non-face images with complex degradation, traditional face super-resolution methods can only improve the clarity of details without the ability to correct pose and restore illumination. Currently, a simple approach is to assume that degradation is independent and use different cascaded models to handle different degradation factors. However, due to the complex degradation coupling effect and the nonlinear enhancement and error accumulation caused by the huge difference between the simulated degradation and the actual degradation, this approach often results in facial artifacts and identity distortion. Although some scholars have attempted to combine other degradation problems with low-resolution problems, these methods either only focus on complex lighting or only on non-frontal poses, leaving a gap in research on low-resolution problems under multiple complex conditions.

[0004] Recently, conditional diffusion models have achieved significant breakthroughs in face super-resolution. Leveraging prior knowledge learned from large-scale real-world raw image data, they possess powerful face generation capabilities, providing a new approach to complex facial degradation problems in the real world. Face super-resolution methods based on diffusion models have achieved remarkable performance in processing general low-resolution face images. These methods typically consist of two stages: a diffusion stage and a denoising stage. The diffusion stage continuously adds Gaussian noise to the high-resolution image to disrupt the image's data distribution, ultimately degrading the high-resolution image into a pure Gaussian noise image. The denoising stage uses a denoising network to iteratively denoise the Gaussian noise image at various noise levels and adjusts it according to the input low-resolution face, ultimately reconstructing a clear high-resolution face image. To generate deterministic targets, researchers have designed multiple conditional control mechanisms to enhance the controllability of generation. However, existing methods generally rely solely on the degraded input as a guiding condition. Since the features of composite degraded low-resolution faces are blurry and ambiguous, using only this as a guiding condition significantly reduces the quality of the generated face. Although some researchers have attempted to use high-quality facial images as external cues, they require high-quality facial images of the same person, which are not available in this task. Summary of the Invention

[0005] This invention addresses the problem of existing technologies struggling to reconstruct high-resolution, well-lit, frontal facial images from low-resolution facial images with non-frontal or complex lighting. It proposes a face super-resolution method, storage medium, and device based on identity-style decoupling guidance. First, face image data is collected to construct training data pairs. A pre-trained implicit diffusion model is loaded and its parameters are frozen. High-quality faces with different identities are randomly selected as style reference faces. A style control network extracts geometric and text embeddings from these reference faces, injecting them into the diffusion model through dual-path cross-attention for style guidance. Next, an identity protection network preprocesses the input face, extracting implicit facial features and identity embeddings from the preprocessed face and guiding the diffusion model in a multi-scale feature space. The style control network and identity protection network are jointly trained using denoising loss and quality-aware ternary identity loss, updating the model parameters. Finally, a Gaussian noise map is initialized, and the optimized style control network and identity protection network guide the denoising network to iteratively denoise. A latent space decoder maps the denoising results to pixel space, ultimately reconstructing a high-quality face image. This invention can simultaneously achieve super-resolution facial reconstruction, pose correction, and illumination restoration, restoring a high-resolution, frontal facial image with uniform illumination from a composite degraded low-resolution facial image.

[0006] To achieve the above objectives, the technical solution adopted by this invention is: a face super-resolution method based on identity-style decoupling guidance, comprising the following steps:

[0007] S1. Data collection and processing: Collect face images and construct training data pairs, which include low-resolution face images and high-resolution face images; wherein, the low-resolution face image is a face image obtained after 8x downsampling under non-frontal pose and complex lighting conditions; the high-resolution face image is a face image obtained after cropping and alignment under frontal pose and good lighting conditions.

[0008] S2. Load the pre-trained implicit diffusion model and freeze its parameters: The implicit diffusion model includes a denoising network, a latent space encoder / decoder, and a text encoder, which perform the diffusion and denoising processes in the latent space; The denoising network, latent space encoder / decoder, and text encoder are frozen during model training.

[0009] S3. Style Guidance: Randomly select high-quality face images of other identities as style reference face images. Use a style control network to extract geometric embeddings and textual embeddings from the style reference faces, and inject these embeddings into the pre-trained diffusion model in step S2 through dual-path cross-attention to achieve style guidance. The style control network includes a geometric perception module, a dual semantic prompting module, and a dual-path cross-attention module.

[0010] The geometric embedding is generated by the geometric perception module, based on the output of the geometric face encoded by the geometric encoder;

[0011] The text embedding is generated by the dual semantic prompting module, based on the output of the text prompting encoded by the text encoder;

[0012] S4. Identity Guidance: The low-quality input face image is preprocessed using an identity protection network. Implicit facial features and identity embeddings are extracted from the preprocessed face image. Identity guidance is then applied to the diffusion model in a multi-scale space. The identity protection network includes a preprocessing module, an identity encoder, and a face guidance module. The identity encoder is used to extract the identity embedding, and the face guidance module uses a parallel branch that shares the encoder structure with the denoising network to extract multi-scale face features.

[0013] S5. Model Training: Train the model using denoising loss and quality-aware ternary identity loss, and update the model parameters.

[0014] S6. Face image output: Initialize the Gaussian noise map, use the optimized style control network and identity protection network to guide the denoising network to continuously denoise, and use the latent space decoder to map the denoising result to the pixel space to reconstruct a high-resolution face image.

[0015] As an improvement of the present invention, in step S2, the implicit diffusion model adds Gaussian noise to the face image during the diffusion process to obtain a noisy latent representation:

[0016]

[0017] In the formula, For clean facial images, , , , It is a step over time Changing hyperparameters;

[0018] In the denoising process, a denoising network is used to predict the noise added at random sampling time steps. A clean face image is obtained through iterative denoising and mapped to the pixel space using a latent space decoder.

[0019] As an improvement of the present invention, step S3 involves using a three-dimensional face reconstruction algorithm to predict the parameterized face shape in the geometric perception module. Albedo map Spherical harmonic illumination coefficient and camera parameters Using face rendering algorithm Rendered as a geometric face And use a geometric encoder to encode the geometric face into a geometric embedding:

[0020] ;

[0021] ;

[0022] in, Geometric encoder.

[0023] As another improvement of the present invention, in step S3, the basic text template is defined as follows in the dual semantic prompt module: In this context, "id" serves as a placeholder for identity injection. GPT-4o is used to describe the pose, lighting conditions, and image quality of the style reference face, forming a token sequence. The output phrases are added to the basic text template to obtain style semantic information, and the identity embedding of the input face is used to replace " "Tokens are used to construct identity semantic information, using a text encoder." Encode text prompts as text embeddings :

[0024] .

[0025] As another improvement of the present invention, in step S3, the dual-path cross-attention module constructs attention weights for each path, for text embedding. It directly reuses the pre-trained cross-attention layer in the denoising network; for geometric embeddings An additional cross-attention layer is introduced into each denoising module; the outputs of the two branches are fused by element-wise summation, given the denoising query features. The calculation formula is as follows:

[0026] ,

[0027] In the formula, , , , , , , , , , These are the corresponding linear projection weight matrices. and These are the training parameters.

[0028] As another improvement of the present invention, in the face guidance module of step S4, the encoder parameters of the pre-trained denoising network are used for initialization. Its decoder consists of a 1×1 convolutional layer and skip connections. The preprocessed low-quality face image is encoded into the latent space using a latent space encoder. The noise is concatenated with the latent space and then input into the face guidance module. Identity embedding is continuously injected through cross-attention. The multi-scale face features extracted by the face guidance module are represented as follows:

[0029]

[0030] In the formula, For facial guidance module. It is Gaussian noise. The low-quality face image after preprocessing. Embedded identity.

[0031] As a further improvement of the present invention, the noise reduction loss in step S5 adopts... The loss approximates the distance between the predicted noise and the actual noise, as detailed below:

[0032] ,

[0033] In the formula, Indicates a time step. This represents a denoising network. For noise labeling, Features extracted by the style control network Features extracted for identity protection networks;

[0034] The specific loss of the quality-perceived three-dimensional identity is as follows:

[0035]

[0036] In the formula, Indicates a tagged face. Facial images representing other identities, For cosine similarity, For quality perception decision boundaries, For identity encoder, For single-step reconstruction of the face.

[0037] To achieve the above objectives, the present invention also adopts the following technical solution: a non-transitory machine-readable storage medium storing executable code thereon, wherein when the executable code is executed by a processor of an electronic device, the processor executes the identity-style decoupling guided face super-resolution method as described in any one of claims 1-7.

[0038] To achieve the above objectives, the present invention also adopts the following technical solution: a computer device, comprising:

[0039] Memory, on which executable code is stored;

[0040] A processor is configured to execute the executable code, causing the computer device to perform operations of the identity-style decoupling guided face super-resolution method as described in any one of claims 1-7.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] (1) The method of the present invention utilizes an innovative diffusion model to simultaneously achieve face super-resolution, face frontalization, and face illumination restoration, decouples facial identity information and style information, and uses the decoupled identity and style information to collaboratively guide the generation of the diffusion model.

[0043] (2) The method of the present invention uses a style control network to construct fine-grained geometric priors and rich textual semantic cues from style reference faces, effectively transferring the required facial style from high-quality face images.

[0044] (3) The method of the present invention uses an identity protection network, which effectively constrains facial identity information by utilizing multi-scale latent space features and robust identity embedding. It can still show good identity fidelity for extremely degraded input faces.

[0045] (4) The method of the present invention uses quality-aware ternary identity loss, which further improves the consistency and robustness of the generated face identity. Attached Figure Description

[0046] Figure 1 This is a diagram illustrating the process of facial degeneration.

[0047] Figure 2 This is an example image of a low-resolution face image with composite degradation used in the method of this invention;

[0048] Figure 3 This is a schematic diagram illustrating the working principle of the face super-resolution device based on identity-style decoupling guided by the present invention.

[0049] Figure 4 This is a schematic diagram of the dual-path cross-attention structure in the method of the present invention;

[0050] Figure 5 This is a diagram illustrating the quality-perceived ternary identity loss process in the method of this invention.

[0051] Figure 6 This is a schematic diagram comparing the reconstruction results of different models in the test examples of this invention;

[0052] Figure 7 This is a flowchart of the steps of the face super-resolution method based on identity-style decoupling guided by the present invention. Detailed Implementation

[0053] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0054] Example 1

[0055] In this embodiment, the input image is a low-resolution face image with composite degradation, and the degradation process is as follows: Figure 1 As shown, when the sampling distance increases, such as Figure 1 In (S2), the image resolution will decrease; when external lighting and facial pose change, such as Figure 1 In step (S3), the image degrades into a composite degraded low-resolution face image containing three degradation factors. To simulate this degradation process, this method acquires a non-frontal face image with complex lighting (512×512 pixels), and obtains a composite degraded low-resolution face image (64×64 pixels) after 8x downsampling, as shown below. Figure 2 As shown. Figure 2As can be seen, the face images in the dataset are of poor quality, exhibiting three degradation problems: low resolution, non-frontal poses, and complex lighting. Facial distortion caused by non-frontal poses and color distortion caused by complex lighting are further exacerbated by the low resolution conditions, forming a complex and difficult-to-decouple composite degradation pattern. This composite low-resolution degradation problem significantly reduces the performance of downstream face recognition algorithms.

[0056] This embodiment uses a similar approach. Figure 2 The image distribution shown contains 50,000 face images from 250 subjects, with the first 200 subjects used for training and the remaining 50 subjects used for testing. This invention uses a face super-resolution device guided by identity-style decoupling, and its working principle is as follows... Figure 3 As shown, a pre-trained implicit diffusion model is used as the backbone network. The style guidance network and the identity protection network are used to decouple the style and identity prior information respectively, and jointly guide the generation of the diffusion model, thereby effectively transferring high-quality facial styles and ensuring the fidelity of identity.

[0057] Face super-resolution methods guided by identity-style decoupling, such as Figure 7 As shown, the specific steps include:

[0058] Step S1: Collect face images under different facial poses and lighting conditions. High-resolution faces with no offset angle pose and uniform lighting are used as high-quality labeled faces. Face images with offset angles and complex lighting are defined as composite degenerate face images. The collected faces are cropped and aligned. The composite degenerate faces are downsampled to obtain low-resolution composite degenerate face images. The high-quality face images corresponding to the identities are used as labeled images to construct training data pairs.

[0059] Face images (512×512 pixels) under different facial poses and complex lighting conditions were acquired, including 13 different angles (-90º, -75º, -60º, -45º, -30º, -15º, 0º, +15º, +30º, +45º, +60º, +75º, +90º) and 9 different lighting conditions (good lighting, two types of global and local shadows, and two types of global and local bright lights). Face images with offset poses (not 0º) were defined as non-frontal face images. Face images with 0º poses were defined as frontal face images. The acquired faces were cropped and aligned. Non-frontal face images under complex lighting conditions were downsampled by 8 times to obtain low-resolution face images (64×64 pixels) under non-frontal and complex lighting conditions, which were then labeled as... High-resolution, well-lit, frontal facial images corresponding to the individual's identity are used as label images and marked as such. .

[0060] Step S2: Load the pre-trained implicit diffusion model and freeze its parameters. The implicit diffusion model includes a denoising network, a latent space encoder / decoder, and a text encoder. The implicit diffusion model is used as the generation backbone, leveraging its rich prior knowledge learned on a large-scale raw image dataset to ensure the quality of the generated face images. The pre-trained parameters of the implicit diffusion model are obtained by training on a large-scale raw image dataset, and its denoising network, latent space encoder / decoder, and text encoder are frozen during training; the denoising network is a Unet network with an encoder-decoder structure.

[0061] In this embodiment, the implicit diffusion model used is Stable Diffusion v2.1, which includes three main modules: a denoising network. Hidden space encoder / decoder Text encoder During training, the parameters of these modules are frozen to preserve their large-scale visual diffusion priors. The implicit diffusion model comprises a diffusion process and a denoising process, both performed in the latent space. During the diffusion process, Gaussian noise is progressively added to the face image to obtain a noisy latent representation, as follows:

[0062] ,

[0063] In the formula, For clean facial images, , , , It is a step over time Changing hyperparameters.

[0064] During the denoising process, the U-Net denoising network is used to predict the noise added at random sampling time steps. A clean face image is obtained through iterative denoising and mapped to the pixel space using a latent space decoder.

[0065] Step S3: Randomly select high-quality facial images of other identities as style reference facial images and label them as... Geometric embeddings are extracted from style reference faces using a style control network. and text embedding The style control network incorporates both geometric perception, dual semantic cueing, and dual-path cross-attention into a pre-trained diffusion model to achieve style guidance. The network consists of a geometric perception module, a dual semantic cueing module, and a dual-path cross-attention module.

[0066] The geometric embedding is generated by a geometric perception module. Given a style reference face, the geometric perception module uses a 3D face reconstruction algorithm to predict parameterized face shape, albedo map, spherical harmonic illumination coefficient, and camera parameters. A face rendering algorithm is then used to render these parameters into a geometric face, which is finally encoded into a geometric embedding using a geometric encoder. The geometric encoder comprises a residual network layer and a fully connected layer.

[0067] A geometry-aware module is used to extract geometric embeddings to provide fine-grained facial structure priors. Given The 3D face reconstruction method EMOCA is used to predict parameterized facial shapes. Albedo map spherical harmonic illumination coefficient and camera parameters ,as follows:

[0068]

[0069] Then, the rendering function is used. Render the rough shape into a geometric face. The facial features and details are depicted in great detail, as follows:

[0070]

[0071] Next, a lightweight geometry encoder is used. right Encode as follows:

[0072]

[0073] in, It consists of two residual blocks and one fully connected layer.

[0074] The text embedding is generated by the dual semantic prompt module. First, a customized text template is set: "An identity is..." The text begins by describing a face image, then using GPT4o to describe the pose, lighting, and image quality of a style reference face. The output phrases are then added to the template text to obtain style semantic information, and the identity of the input face is used to replace the provided text. "The token is used to construct identity semantic information, and finally the text prompt is encoded into a text embedding using a text encoder."

[0075] A dual semantic prompting module is used to extract text embeddings to provide high-quality style semantic supervision. First, the basic text template is defined as follows: The "id" is used as a placeholder for identity injection. Then, GPT-40 is used in a visual question-answering manner. Perform three queries: facial pose, lighting conditions, and image quality. The returned text phrases are appended to the base template to form blended text, as follows:

[0076] ,

[0077] In the formula , , These are descriptive phrases related to attitude, lighting, and quality, respectively.

[0078] then, The tokens are segmented into a sequence of tokens. ={ The token corresponding to "id" is replaced with an identity token, which is obtained from... The learnable identity embeddings extracted from it are labeled as . Updated to ={ Finally, through a pre-trained text encoder... Extracting text embeddings ,as follows:

[0079] .

[0080] The dual-path cross-attention method establishes attention weights for geometric embedding and text embedding respectively. For text embedding, the cross-attention layer in the pre-trained denoising network is directly reused; for geometric embedding, an additional cross-attention layer is introduced in each denoising block; finally, the two results are fused and injected into the denoising network.

[0081] By leveraging dual-path cross-attention, geometric embeddings and text embeddings are injected into a pre-trained denoising network, such as... Figure 4 As shown. Dual-path cross-attention constructs attention weights for each path, for text embeddings... It directly reuses the pre-trained cross-attention layer in the denoising network; for geometric embeddings An additional cross-attention layer is introduced into each denoising module. Finally, the outputs of the two branches are fused element-wise. Given denoising query features... The calculation formula is as follows:

[0082] ,

[0083] In the formula, , , , , , , , , , These are the corresponding linear projection weight matrices. Since the text cross-attention parameters are reused pre-trained parameters and frozen during training, only a limited number of parameters are trainable. and .

[0084] Step S4: The low-quality input face image is preprocessed using an identity protection network. Then, implicit facial features and identity embeddings are extracted from the preprocessed face. The face guidance module guides the diffusion model in a multi-scale space to improve identity. The identity protection network mainly consists of a preprocessing module, an identity encoder, and a face guidance module.

[0085] A preprocessing module performs initial degradation removal on the input face, resulting in a preprocessed face. This preprocessed face is then mapped into the latent space using a latent space encoder and concatenated with Gaussian noise before being input into the face guidance module. The preprocessing module employs an encoder-decoder network structure to perform pixel-level learning on the degraded face, directly addressing the degradation issues in complex, low-resolution faces. In this embodiment, a SwinIR network is used. The loss function is used to train the preprocessing module as follows:

[0086] ,

[0087] In the formula, For preprocessing modules, For high-quality tagged face images.

[0088] The identity embedding is obtained by an identity encoder and injected into the face guidance module through cross-attention. The identity encoder is a pre-trained quality-aware face recognition model used to extract image quality-robust identity embeddings. The identity encoder is used to extract identity embeddings to constrain the consistency of faces during the generation process. To cope with complex changes in face image quality, this embodiment uses the pre-trained quality-aware face recognition model AdaFace as the identity encoder. A mapping network is then used to map the encoding results to the required dimensions to construct robust identity embeddings, as shown below:

[0089] ,

[0090] In the formula, For low-quality faces after preprocessing, For identity encoder, The mapping network consists of one linear layer and one normalization layer.

[0091] The face guidance module is defined as extracting multi-scale facial features using a parallel branch that shares the encoder structure with the denoising network. It is initialized using the encoder parameters of a pre-trained denoising network, and its decoder consists of 1×1 convolutional layers and skip connections to ensure spatial alignment with the denoising network. A latent space encoder is used to... The encoded data is stored in the latent space, concatenated with noise, and then input into the face guidance module. Identity embedding is continuously injected through cross-attention. Finally, the multi-scale facial features extracted by the face guidance module are represented as follows:

[0092]

[0093] In the formula, For facial guidance module. It is Gaussian noise.

[0094] Step S5: Train the model jointly using denoising loss and quality-aware ternary identity loss, and update the model parameters.

[0095] The model is jointly trained using denoising loss and quality-aware ternary identity loss to update the parameters of the wind control network and the identity protection network. Specifically:

[0096] Noise reduction loss is adopted The distance between the predicted noise and the actual noise is approximated by the loss function as follows:

[0097] ,

[0098] In the formula, Indicates a time step. This represents a denoising network. For noise labeling, Features extracted by the style control network Features extracted for identity protection networks.

[0099] like Figure 5 As shown, the quality-aware ternary identity loss function uses the face reconstructed step by step during the denoising process to supervise the identity, making it close to the labeled face in the embedding space and far away from the style reference face, and adaptively adjusting the decision boundary according to the image quality to adapt to complex facial degradation; the updated model parameters only include the style control network and the identity protection network.

[0100] The quality-aware ternary identity loss constrains the distance between the single-step predicted face and the labeled face in the embedding space, and adaptively optimizes the discrimination boundary based on the quality of the input face by incorporating the quality-aware angular boundary. The single-step reconstructed face in the latent space is estimated by inverse reasoning as follows:

[0101]

[0102] Subsequently, the spatial decoder decodes the data to pixel space, yielding the single-step reconstruction result in pixel space:

[0103]

[0104] The complete quality perception three-dimensional identity loss is as follows:

[0105]

[0106] In the formula, Indicates a tagged face. Facial images representing other identities, including stylized facial images. For cosine similarity, The quality-perceived decision boundary is calculated using the following formula:

[0107] ,

[0108] In the formula, The default value for the hyperparameter is 0.5. It is the characteristic norm.

[0109] Finally, the total loss function is as follows:

[0110] ,

[0111] In the formula, The hyperparameter used to balance the loss is set to 0.5 by default.

[0112] To clarify the implementation process, the overall network training process is summarized below: First, the preprocessing module is used to update the parameters, and preprocessed face images are obtained. The training parameters are as follows: 24 epochs are trained using the Adam optimizer with a batch size of 8 and a learning rate of 1e-4. Then, the pre-trained diffusion model, the pre-trained identity encoder, and the optimized preprocessing module are loaded. The style control network and identity protection network are trained, and the network parameters are updated. The training parameters are as follows: 12 epochs are trained using the Adam optimizer with a batch size of 8 and a learning rate of 5e-5.

[0113] Step S6: Initialize Gaussian noise map The pre-trained diffusion model, along with the optimized style control network and identity protection network, guides the denoising network to iterate and denoise continuously. The latent space decoder is used to map the denoising result to the pixel space, and finally reconstructs a high-resolution face image with frontal and uniform illumination.

[0114] The model reasoning process can be summarized as follows:

[0115] First, a preprocessing module is loaded to obtain the preprocessed face. Then, a pre-trained diffusion model and identity encoder are loaded, and an optimized style control network and identity protection network are used to collaboratively guide the denoising network to predict noise. Finally, the DDIM sampling formula is used for iterative denoising, extracting noise from the noisy image. Estimating noise-free face images .as follows:

[0116]

[0117] in, It is noise sampled from a standard normal distribution. These are hyperparameters that vary over time. It is the time step. This represents a denoising network. Features extracted by the style control network Features extracted for identity protection networks This represents the variance, which defaults to 0.

[0118] The sampling formula was used to perform 50 iterations to obtain the results. Ultimately, through the latent space decoder Will Mapping to pixel space yields high-quality face images.

[0119] Test case

[0120] The proposed method was compared with other methods on composite degraded low-resolution faces. The comparison methods included three categories: (1) model cascading, i.e., executing face super-resolution models (RealESRGAN, SR3, StableSR), face frontalization models (Optimized-CRGAN, abbreviated as FT), and illumination restoration models (Retinexformer, abbreviated as IN) in sequence, with the order adjustable; (2) face super-resolution models under complex conditions: IC-FSRNet and HiFiDiff; (3) blind face restoration models: BFRffusion, GFPGAN, and DiffBir. To ensure the fairness of the comparison, the official configuration parameters corresponding to these methods were used to retrain on the training dataset, and the trained models were used for inference on the same degraded faces.

[0121] Specifically, such as Figure 6As shown. The first type of method: When using FT and IN for pose correction and illumination restoration before face super-resolution, obvious artifacts and distortions appear. Conversely, if face super-resolution is performed first, followed by pose correction and illumination restoration, local blurring and a lack of realism occur in the facial area. Therefore, simply cascading models sequentially cannot effectively recover accurate facial information. The second type of method: IC-FSRNet can effectively adjust overall brightness, but performs poorly in handling local deformations, leading to loss of detail. HiFiDiff can generate high-fidelity, high-resolution frontal faces, but the illumination distribution is unnatural. The third type of method, GFPGAN, BFR-Fusion, and DiffBIR, although generating visually appealing results, suffer from low identity fidelity due to a lack of effective identity guidance.

[0122] In comparison, the reconstructed image produced by the method in this case is clearer, more accurate, and more effective. This invention can accurately restore a high-resolution face image with good lighting from a composite degraded low-resolution face image, and it far surpasses other methods in terms of fidelity and realism.

[0123] In summary, the method of this invention extracts style information from high-quality faces with other identities and extracts identity information from low-quality input face images, thereby achieving identity-style display decoupling. It also utilizes these two elements to guide a pre-trained diffusion model, ensuring both fidelity and realism.

[0124] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. A face super-resolution method based on identity-style decoupled guidance, characterized in that, The method comprises the following steps: S1, collecting data and processing: collecting face images and constructing training data pairs, wherein the data pairs comprise low-resolution face images and high-resolution face images; the low-resolution face images are obtained by 8 times down-sampling processing of face images under non-frontal pose and complex illumination; and the high-resolution face images are obtained by cropping and aligning face images under front pose and good illumination; S2, loading a pre-trained implicit diffusion model and freezing parameters: the implicit diffusion model comprises a denoising network, an implicit space encoder / decoder, and a text encoder, and a diffusion process and a denoising process are performed in the implicit space; the denoising network, the implicit space encoder / decoder, and the text encoder are frozen in the model training process; S3, style guidance: a high-quality face image of another identity is randomly selected as a style reference face image, geometric embedding and text embedding are extracted from the style reference face image by using a style control network, and the geometric embedding and the text embedding are injected into the diffusion model pre-trained in step S2 by a double-path cross attention to realize style guidance; the style control network comprises a geometric perception module, a bilingual semantic prompt module, and a double-path cross attention module, wherein the geometric embedding is generated by the geometric perception module based on the output of the geometric encoder after encoding the geometric face; the text embedding is generated by the bilingual semantic prompt module based on the output of the text encoder after encoding the text prompt; S4, identity guidance: an identity protection network is used to pre-process the input low-quality face image, and implicit facial features and identity embedding are extracted from the pre-processed face image, and the diffusion model is guided in a multi-scale space; the identity protection network comprises a pre-processing module, an identity encoder, and a face guidance module, wherein the identity encoder is used to extract the identity embedding, and the face guidance module uses a parallel branch sharing the encoder structure of the denoising network to extract multi-scale face features; S5, model training: the model is trained using denoising loss and quality-aware tri-identity loss, and the model parameters are updated; S6, face image output: initializing a Gaussian noise map, using the optimized style control network and identity protection network to guide the denoising network to iteratively denoise, using the implicit space decoder to map the denoising result to the pixel space, and reconstructing a high-resolution face image.

2. The face super-resolution method based on identity-style decoupled guidance according to claim 1, wherein: In step S2, in the diffusion process, Gaussian noise is added to the face image to obtain a noisy latent representation: ； In the formula, is a clean face image, , , , is a hyperparameter that changes over time steps ; In the denoising process, the denoising network is used to predict the noise added at a random sampling time step, a clean face image is obtained by iterative denoising, and the implicit space decoder is used to map to the pixel space.

3. The face super-resolution method based on identity-style decoupled guidance according to claim 1, wherein: The step S3 predicts a parametric face shape in a geometry perception module using a three-dimensional face reconstruction algorithm , an albedo map , spherical harmonic lighting coefficients , and camera parameters , employs a face rendering algorithm to render into a geometric face and encodes the geometric face into a geometry embedding using a geometry encoder: ； ； wherein, Geometric encoder.

4. The face super-resolution method based on identity-style decoupled guidance according to claim 1, wherein: The step S3 defines the base text template as where "id" is the placeholder for identity injection, and GPT-4o is used to describe the pose, lighting conditions, and image quality of the style reference face to form a token sequence The output phrase is supplemented in the base text template to obtain style semantic information, and the identity embedding of the input face is used to replace the token to construct identity semantic information, and a text encoder is used to encode the text prompt into a text embedding : 。 5. The face super-resolution method based on identity-style decoupled guidance according to claim 3 or 4, characterized in that: In the step S3, the dual-path cross-attention module constructs attention weights for each path. For text embedding , the pre-trained cross-attention layer in the denoising network is directly reused; for geometric embedding , an additional cross-attention layer is introduced in each denoising module; the outputs of the two branches are fused by element-wise addition, and given the denoising query feature , the calculation formula is as follows: ； wherein, , , , , , , , , , are the corresponding linear projection weight matrices, and are the training parameters.

6. The face super-resolution method based on identity-style decoupled guidance according to claim 1, wherein: In the face guidance module of step S4, the encoder parameters of the pre-trained denoising network are used for initialization, the decoder thereof is composed of a 1x1 convolution layer and a skip connection, the pre-processed low-quality face image is encoded into the implicit space by using the implicit space encoder, the noise is spliced with the pre-processed low-quality face image and input into the face guidance module, and the identity embedding is continuously injected by cross attention, and the multi-scale face feature extracted by the face guidance module is represented as: ； wherein, is a face guidance module, is a Gaussian noise, is a pre-processed low-quality face image, is an identity embedding.

7. The face super-resolution method based on identity-style decoupled guidance according to claim 1, wherein: The de-noising loss of the step S5 adopts The distance between the loss approximation prediction noise and the real noise is as follows: ； wherein, denotes a time step, denotes a denoising network, is a noise label, is a feature extracted by the style control network, is a feature extracted by the identity protection network; The quality-aware tri-identity loss is specifically: ； wherein, denotes a labeled face, denotes a face image of other identity, is a cosine similarity, is a quality-aware decision boundary, is an identity encoder, is a single-step reconstructed face.

8. A non-transitory machine-readable storage medium, characterized in that: The computer device has a memory and a processor.

9. A computer device, characterized by: The memory has executable code stored thereon. The processor is configured to execute the executable code, so that the computer device performs the operations of the face super-resolution method based on identity-style decoupling guidance according to any one of claims 1-7. The memory has executable code stored thereon. The processor is configured to execute the executable code, so that the computer device performs the operations of the face super-resolution method based on identity-style decoupling guidance according to any one of claims 1-7.

Images