Facial modification and reduction method based on Diffusion-Mama cooperation

Abstract

The invention discloses a face modification and restoration method based on Diffusion-Mama cooperation, and the method comprises the following steps: carrying out the down-sampling of a to-be-restored face image, and obtaining a low-resolution face image; detecting whether the face image is modified or not, and obtaining a modified image; constructing a DeltaFRR model, wherein the DeltaFRR model comprises a face difference image restorer and a face detail generator; making a difference between the to-be-restored face image and the modification image to obtain a difference image; splicing the difference image and the modification image added with the noise, and inputting the spliced image into a face difference image restorer for training; outputting a low-resolution restored image based on the trained face difference image restorer; and inputting the low-resolution restored image into a face detail generator to extract shallow layer features and deep layer features, and reconstructing based on the shallow layer features and the deep layer features to obtain a high-resolution image. Image details can be effectively enhanced, and meanwhile the calculation efficiency is kept.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a facial retouching and restoration method based on Diffusion-Mamba collaboration. Background Technology

[0002] With the widespread use of social media and the convenience of photo editing tools, more and more people are beautifying their facial photos. This poses a challenge to fields that require authenticity (such as identity verification and social media). Users can easily generate deceptive content by modifying facial images, leading to the spread of false information and potentially damaging the reliability of identity verification systems and social media. Therefore, research on the problem of facial retouching and restoration (FRR) has become very urgent and important.

[0003] Despite the urgent need for Face Retouching Detection (FRR), differences remain between FRR and existing image restoration tasks. The most relevant research direction is Face Retouching Detection, a binary classification problem used to detect whether facial images have been beautified. Ying et al. created a large-scale dataset (RetouchingFFHQ) and proposed a novel method for fine-grained detection of different types and degrees of face retouching. However, these methods cannot reverse the effects of retouching filters, fail to reveal the true facial image, and cannot provide legal evidence.

[0004] Diffusion models have demonstrated great potential in various visual tasks, including image super-resolution, image inpainting, image translation, image editing, semantic segmentation, video generation, and anomaly detection. These successes demonstrate the versatility of diffusion models and their superiority over traditional methods in complex visual tasks. However, diffusion models that directly process high-resolution images require significant computational resources. Summary of the Invention

[0005] To overcome the shortcomings and deficiencies of existing technologies, this invention provides a facial retouching and restoration method based on Diffusion-Mamba collaboration. It reverses various retouching operations and restores the image to its original state based on the DeltaFRR model. The DeltaFRR model consists of a coarse-to-fine two-stage network. In the first stage, a facial difference map restorer (FDR) based on a diffusion model is constructed to generate the basic outline of a low-resolution face. In the second stage, a facial detail generator based on Mamba technology is constructed to generate high-resolution facial details. This coarse-to-fine approach taps into the potential of the conditional diffusion model. Delta-Map Diffusion is used to solve the task of facial beautification and removal. Through Diffusion-Mamba collaboration, a high-resolution facial removal effect image is obtained, improving the facial details and resolution of low-resolution images, effectively enhancing image details while maintaining computational efficiency.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] This invention provides a facial retouching and restoration method based on Diffusion-Mamba collaboration, comprising the following steps:

[0008] The face image to be restored is downsampled to obtain a low-resolution face image;

[0009] Detect whether a face image has been modified and obtain the modified image;

[0010] Construct a DeltaFRR model, including a facial difference map restorer and a facial detail generator;

[0011] The difference between the face image to be restored and the retouched image is calculated to obtain the difference image;

[0012] Noise is added to the modified image, and the difference image and the modified image with added noise are stitched together and then input into the facial difference image restorer for training.

[0013] The face difference map restorer outputs a low-resolution restored image based on the trained face difference map;

[0014] The low-resolution restored image is input into the facial detail generator to extract shallow and deep features. Based on the extracted shallow and deep features, a high-resolution image is reconstructed.

[0015] As a preferred technical solution, the difference map and the noise-added modified map are stitched together and then input into the facial difference map restorer for training, specifically including:

[0016] The difference image and the noise-added modified image are concatenated as input, and the modified image is used as a supervised condition for training.

[0017] The facial difference map restorer is trained using a denoising diffusion probability model, which includes a forward noise addition stage and a backward denoising stage.

[0018] During the forward noise addition phase, noise is added at an increasing diffusion rate, generating multiple noisy images that converge to a standard Gaussian distribution.

[0019] The introduced noise is gradually removed in the backward denoising stage to obtain the target image.

[0020] As a preferred technical solution, a low-resolution restored image is input into a facial detail generator to extract shallow and deep features. A high-resolution image is then reconstructed based on the extracted shallow and deep features, specifically including:

[0021] The low-resolution reconstruction image generates shallow features f through convolutional layers. s ;

[0022] Shallow features f s Deep features f are extracted sequentially using multiple Mamba-based facial detail enhancers. D ;

[0023] Shallow features f s With deep features f D By combining elements one by one, the resulting output is used as the input for reconstruction, and finally, a high-resolution image is output.

[0024] As a preferred technical solution, shallow feature f s Deep features f are extracted sequentially using multiple Mamba-based facial detail enhancers. D Specifically, it includes:

[0025] Shallow features f s Through layer normalization, we obtain:

[0026] x N =LN(f s )

[0027] Feature x N The processing is carried out through two branches. One branch consists of a linear layer and SiLU, which expands the feature channels to λC. The other branch includes a linear layer, depthwise convolution (DWConv), SiLU, a two-dimensional state space model, and a layer normalization layer, which expands the feature channels to λC, where λ is a predefined channel expansion factor and C represents the current number of channels.

[0028] The features from the two branches are aggregated using the Hadamard product, specifically as follows:

[0029] x M =LN(2D-SSM(SiLU(DWConv(Linear(x N )))))⊙SiLU(Linear(x N ))

[0030] Where DWConv represents depthwise convolution, SiLU represents the activation function, and ⊙ represents the Hadamard product;

[0031] Feature x M Through a linear layer, and with weighted shallow features f s Add them together to generate the output x. 2d , is represented as:

[0032] x 2D =Linear(x M )+Scale×f s

[0033] Feature x2D After layer normalization, it is passed through a convolutional layer and a channel attention module, and then combined with the weighted feature x. 2D The sums are then passed to another convolutional layer, represented as:

[0034] x CA =CA(Conv(LN(x) 2D )))+Scale×x 2D

[0035] The features are processed sequentially through N facial detail enhancers. After all facial detail enhancers have been processed, a convolution operation is performed to optimize the extracted features and obtain deep features.

[0036] As a preferred technical solution, the facial detail enhancer based on Mamba technology also includes a two-dimensional selective scanning module. The two-dimensional selective scanning module flattens the two-dimensional image features into a one-dimensional sequence and scans along four different directions: from the top left to the bottom right, from the bottom right to the top left, from the top right to the bottom left, and from the bottom left to the top right.

[0037] Discrete state-space equations are used to capture the long-range dependencies of each sequence.

[0038] The present invention also provides a facial retouching and restoration system based on Diffusion-Mamba collaboration, comprising: a downsampling module, a retouching map acquisition module, a DeltaFRR model construction module, a difference map acquisition module, a facial difference map restorer training module, a low-resolution restoration map output module, and an image reconstruction module;

[0039] The downsampling module is used to downsample the face image to be restored to obtain a low-resolution face image;

[0040] The modified image acquisition module is used to detect whether a face image has been modified and to acquire a modified image;

[0041] The DeltaFRR model building module is used to build a DeltaFRR model, including a facial difference map restorer and a facial detail generator;

[0042] The difference map acquisition module is used to subtract the face image to be restored from the modified image to obtain the difference map.

[0043] The facial difference map restorer training module is used to stitch together the difference map and the noise-added modified map and input them into the facial difference map restorer for training;

[0044] The low-resolution restored image output module outputs a low-resolution restored image based on the trained facial difference map restorer.

[0045] The image reconstruction module is used to input the low-resolution restored image into the facial detail generator to extract shallow and deep features, and to reconstruct a high-resolution image based on the extracted shallow and deep features.

[0046] As a preferred technical solution, the facial difference map restorer training module is used to stitch together the difference map and the noise-added modified map and input them into the facial difference map restorer for training, specifically including:

[0047] The difference image and the noise-added modified image are concatenated as input, and the modified image is used as a supervised condition for training.

[0048] The facial difference map restorer is trained using a denoising diffusion probability model, which includes a forward noise addition stage and a backward denoising stage.

[0049] During the forward noise addition phase, noise is added at an increasing diffusion rate, generating multiple noisy images that converge to a standard Gaussian distribution.

[0050] The introduced noise is gradually removed in the backward denoising stage to obtain the target image.

[0051] As a preferred technical solution, the image reconstruction module is used to input the low-resolution restored image into a facial detail generator to extract shallow and deep features, and reconstruct a high-resolution image based on the extracted shallow and deep features, specifically including:

[0052] The low-resolution reconstruction image generates shallow features f through convolutional layers. s ;

[0053] Shallow features f s Deep features f are extracted sequentially using multiple Mamba-based facial detail enhancers. D ;

[0054] Shallow features f s With deep features f D By combining elements one by one, the resulting output is used as the input for reconstruction, and finally, a high-resolution image is output.

[0055] As a preferred technical solution, shallow feature f s Deep features f are extracted sequentially using multiple Mamba-based facial detail enhancers. D Specifically, it includes:

[0056] Shallow features f s Through layer normalization, we obtain:

[0057] x N =LN(f s )

[0058] Feature xN The processing is carried out through two branches. One branch consists of a linear layer and SiLU, which expands the feature channels to λC. The other branch includes a linear layer, depthwise convolution (DWConv), SiLU, a two-dimensional state space model, and a layer normalization layer, which expands the feature channels to λC, where λ is a predefined channel expansion factor and C represents the current number of channels.

[0059] The features from the two branches are aggregated using the Hadamard product, specifically as follows:

[0060] x M =LN(2D-SSM(SiLU(DWConv(Linear(x N )))))⊙SiLU(Linear(x N ))

[0061] Where DWConv represents depthwise convolution, SiLU represents the activation function, and ⊙ represents the Hadamard product;

[0062] Feature x M Through a linear layer, and with weighted shallow features f s Add them together to generate the output x. 2D , is represented as:

[0063] x 2D =Linear(x M )+Scale×f s

[0064] Feature x 2D After layer normalization, it is passed through a convolutional layer and a channel attention module, and then combined with the weighted feature x. 2D The sums are then passed to another convolutional layer, represented as:

[0065] x CA =CA(Conv(LN(x) 2D )))+Scale×x 2D

[0066] The features are processed sequentially through N facial detail enhancers. After all facial detail enhancers have been processed, a convolution operation is performed to optimize the extracted features and obtain deep features.

[0067] As a preferred technical solution, the facial detail enhancer based on Mamba technology also includes a two-dimensional selective scanning module. The two-dimensional selective scanning module flattens the two-dimensional image features into a one-dimensional sequence and scans along four different directions: from the top left to the bottom right, from the bottom right to the top left, from the top right to the bottom left, and from the bottom left to the top right.

[0068] Discrete state-space equations are used to capture the long-range dependencies of each sequence.

[0069] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0070] This invention reverses various retouching operations and restores images to their original state based on the DeltaFRR model. The DeltaFRR model consists of a coarse-to-fine two-stage network. In the first stage, a Facial Difference Map Restorer (FDR) based on a diffusion model is constructed to generate the basic outline of a low-resolution face. In the second stage, a facial detail generator based on Mamba technology is constructed to generate high-resolution facial details. It adopts a technical solution based on Delta-Map Diffusion and Mamba collaboration to solve the technical problem of face removal and beautification, achieving a technical effect that is very close to the original face. When tested on existing datasets, the DeltaFRR model significantly outperforms other existing methods on four widely used quantitative metrics. These results fully verify its effectiveness. Attached Figure Description

[0071] Figure 1 This is a schematic diagram of the overall implementation architecture of the facial retouching and restoration method based on Diffusion-Mamba collaboration of the present invention;

[0072] Figure 2 This is a schematic diagram illustrating the generation process of the difference map in this invention;

[0073] Figure 3 This is a schematic diagram of the implementation architecture of the two-dimensional selective scanning module (2D-SSM) of the present invention;

[0074] Figure 4 This is a schematic diagram illustrating the density distribution of cosine similarity between the original and beautified / unbeautified facial image features of this invention.

[0075] Figure 5 A schematic diagram illustrating the qualitative comparison of different FRR methods;

[0076] Figure 6 This is a qualitative schematic diagram illustrating the effect of the FRRffusion method of the present invention on FRR;

[0077] Figure 7 This is a schematic diagram illustrating the qualitative cross-validation on different datasets in this invention. Detailed Implementation

[0078] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0079] Example

[0080] To maintain consistency with standard notation, this embodiment uses bold lowercase letters to represent (column) vectors and bold or special uppercase letters to represent matrices;

[0081] like Figure 1 As shown, this embodiment provides a facial retouching and restoration method based on Diffusion-Mamba collaboration. It employs Delta-Map Diffusion to solve the task of facial beautification and removal, and utilizes a two-level framework. Through Diffusion-Mamba collaboration, it obtains a high-resolution facial removal effect image. Specifically, it includes the following steps:

[0082] S1: Obtain the face image to be restored, and downsample the face image to obtain a low-resolution image so that this image can be processed in the Diffusion step;

[0083] S2: Detect whether an image has been edited (beautified) based on the DenseNet121 pre-trained model. If it is determined that the image has been edited, proceed to the next step:

[0084] In this embodiment, image retouching achieves the desired aesthetic effect by modifying specific or all facial features of the original image (such as skin smoothing, facial whitening, face lifting, and eye enlargement). These modifications typically do not affect the image background.

[0085] S3: Construct the DeltaFRR model, including the Facial Delta-Map Restorer (FDR) and the Facial Detail Generator (FDG);

[0086] The DeltaFRR model in this embodiment is a framework based on a diffusion model and Mamba technology. This framework includes a two-stage network. In the first stage, the modified facial image and the difference map are downsampled to obtain a low-resolution image. A basic facial contour is generated based on the facial difference map restorer in the diffusion model, capturing the overall structure and shape of the face. In the second stage, a Mamba-driven facial detail generator uses Mamba technology to generate a high-quality, high-resolution facial image from the low-resolution facial image generated in the first stage. Specifically, this includes:

[0087] S31: Begin processing the image to be restored using the Facial Delta-Map Restorer (FDR);

[0088] Specifically, the difference between the original image and the modified image is calculated to obtain a difference image. Noise is added to the modified image. The difference image and the modified image with added noise are concatenated. After encoding and decoding, the noise added to the modified image is calculated. Then, the loss is obtained, and backpropagation is performed to train the network.

[0089] like Figure 2 As shown, a difference map is generated by calculating the difference between the original image and the modified image. Compared with the original image, the difference map has a simpler structure, which helps the diffusion model learn more efficiently. The white part in the difference map is the difference between the original image and the modified image, and the black part is the same part between the original image and the modified image.

[0090] In this embodiment, the face difference map restorer is a diffusion model that captures the difference between the original image and the modified image using the difference map. Specifically, it adopts the Denoising Diffusion Probabilistic Model (DDPM), which uses the concatenation of the difference map and the modified image as input and the modified image as a supervision signal for training. This simplifies the learning process and makes image restoration more efficient.

[0091] In this embodiment, the denoising diffusion probability model DDPM training process includes two main stages: forward noise addition and backward denoising. In the forward noise addition stage, noise diffuses at an increasing rate β. t ,β t ∈[0.0001,0.02], t∈[1,T], where T represents the total number of diffusion steps, systematically added to the original sample x0. This incremental addition of noise produces a series of noisy images x0. 1:T It eventually converges to a standard Gaussian distribution.

[0092] Specifically, the forward diffusion process can be represented as a Markov chain, as follows:

[0093]

[0094] Here, the current time step of the diffusion process is denoted as t∈[1,T];

[0095] To simplify the notation, α is defined. t :=1-β t and Based on these definitions, the probability distribution q(x) can be... t |x0) is represented as:

[0096]

[0097] For a given original image x0, the sampled value at time step t is xt It can be represented as:

[0098]

[0099] Based on x t Calculations show that as the number of diffusion steps T increases, The value of x approaches zero. Therefore, x t It will converge to ∈.

[0100] In the denoising process, the posterior distribution p(x) is solved. t-q |x t This presents a certain challenge. To address this issue, a neural network θ is used to approximate the distribution. The predicted distribution is denoted as p. θ (x t-q |x t Assume p θ (x t-1 |x t The mean and variance of ) are μ θ (x t ,t) and σ θ (x t If ,t), then p can be θ (x t-1 |x t ) is represented as:

[0101]

[0102] In the forward diffusion process, it can be deduced that given x0 and x... t In the case of distribution p(x) t-1 |x t (x0) can be represented as:

[0103]

[0104] in, By solving for x0 from equation (4) and substituting it into μ t (x t In x0), we can obtain the following expression:

[0105]

[0106] Given p(x) t-1 |x t The variance σ of x0) t It is a constant, predicting p(x) t-1 |x t x0) is equivalent to estimating μ t (x t Therefore, the network model θ can be used to adjust μ.θ (x t Parameterization can be performed on ,t). Specifically, this can be achieved in the following ways:

[0107]

[0108] Where ∈ θ (x t Let p0(x,t) represent the noise value predicted at time step t. During training, the goal is to minimize the prediction distribution p0(x,t). t-1 |x t ) and posterior distribution p(x) t-1 |x t The difference between t, x0) and μ can be minimized. t (x t ,t) and μ θ (x t This is achieved through the expected squared difference between (t) and (t).

[0109] A simple and effective loss function can be expressed by the following formula:

[0110]

[0111] Once the network model (θ) is sufficiently trained, it can predict the noise introduced at each time step t. The time step t starts at 1 and gradually increases until it reaches a maximum value T. When T is sufficiently large, the variable x at time step T... T It will follow a standard Gaussian distribution. During the sampling process, a pure noise term (denoted as n) is sampled from the standard Gaussian distribution. T The noise term is then fed into the trained diffusion model. Given an input of n... t With time step t, the noise term n after denoising within one step t-1 It can be represented as:

[0112]

[0113] According to equation (10), the noise introduced during image degradation can be derived from n t Remove gradually until the target image (n0) is obtained.

[0114] In this embodiment, a paired dataset is used. Training is performed with i = 0, 1, ..., S, where S represents the size of the dataset. This represents the i-th modified face image. This represents the corresponding facial difference image. For simplicity, the common training sample is denoted as (x0, y0).

[0115] Throughout the training process, not only at each time step t, but also at the input of a noisy facial difference image x t Furthermore, the modified face image y0 is used as a supervisory input. This condition serves as a reference, effectively guiding the diffusion model to generate FRR images.

[0116] When the conditional input y0 is introduced, the FRR diffusion model θ can be defined. FRR The posterior probability is:

[0117]

[0118] Accordingly, the mean of the prediction noise can be expressed as:

[0119]

[0120] At the same time, a loss function can be defined. for:

[0121]

[0122] In this embodiment, the FRR diffusion model θ is trained. FRR The process specifically includes:

[0123] Repeat the following steps:

[0124] (1) From the data distribution Sample a pair of initial images and conditions (x0, y0);

[0125] (2) Uniformly sample a time step t from {1,…,T};

[0126] (3) From the standard normal distribution Sampling noise ∈;

[0127] (4) Perform gradient descent to minimize the following objective:

[0128]

[0129] (5) Continue until convergence;

[0130] In this embodiment, the sampling process under the given condition y0 specifically includes:

[0131] (1) From the standard normal distribution Initial noise of medium sampling x T And prepare condition y0;

[0132] (2) For t = T, ..., 1, perform the following steps:

[0133] (3) If t>1, then from Sample the noise z; otherwise, set z = 0.

[0134] (4) Update x according to the following formula t-1

[0135]

[0136] (5) End the loop;

[0137] (6) Return the final sample x0.

[0138] S32: After T-step training, the network tends to converge, and a low-resolution restored image is obtained based on the trained network and the modified image.

[0139] S33: The low-resolution restored image is input into a facial detail generator based on Mamba technology for enhancement, which consists of three stages: shallow feature extraction, deep feature extraction and reconstruction stage;

[0140] For a given low-resolution input image x0, a shallow feature f is first generated using a 3×3 convolutional layer. s That is, f s =Conv(x0). Shallow features f s Capture low-level features, such as edges, textures, and simple patterns, followed by shallow features f s Entering the deep feature extraction stage, deep features f are obtained. D This stage consists of multiple stacked Mamba Face Detail Enhancers (MFDEs). Furthermore, an additional convolutional layer is introduced at the end of the multiple Mamba MFDEs to refine the features extracted from them. Finally, element-wise summation is used to obtain the input F for the high-quality reconstruction stage. R =f s +f D This allows for the reconstruction of a high-quality (HQ) output image I. HQ ;

[0141] Building upon Mamba's success in long-range modeling with linear complexity, it effectively addresses the challenge of whole-graph interactions based on Transformer networks, proposing a State Space Block (SSB) for face image inpainting reversal. The SSB captures long-range dependencies through state-space equations. The complete SSB processing flow is as follows:

[0142] x N =LN(f s (14)

[0143] Layer normalization (LN) is applied to the input f.s Then pass it to the subsequent module. Next, input x. N The process is divided into two branches. One branch consists of a linear layer and SiLU, which expands the feature channels to λC. The other branch includes a linear layer, depthwise convolution (DWConv), SiLU, a two-dimensional state-space model (2D-SSM), and another LN layer, which also expands the feature channels to λC, where λ is a predefined channel expansion factor and C represents the current number of channels.

[0144] Subsequently, the features from the two branches are aggregated using the Hadamard product. This process can be represented by the following mathematical expression:

[0145] x M =LN(2D-SSM(SiLU(DWConv(Linear(x N )))))⊙SiLU(Linear(x N (15)

[0146] Where DWConv represents depthwise convolution, SiLU represents the activation function, and ⊙ represents the Hadamard product.

[0147] Enter x M This will be passed to a linear layer, and then combined with a weighted f. s Add them together to generate the output x. 2D Finally, the number of channels will be reduced to C to generate an output x with the same shape as the input. M This process can be represented by the following mathematical expression:

[0148] x 2D =Linear(x M )+Scale×f s (16)

[0149] Output x 2D It will undergo layer normalization (LN) processing, followed by a convolutional layer and a channel attention module. Then it will be combined with a weighted x. 2D The results are added together and passed to another convolutional layer. This process can be represented by the following mathematical expression:

[0150] x CA =CA(Conv(LN(x) 2D )))+Scale×x 2D (17)

[0151] This completes one SSB processing step. After N SSBs are completed, x... CAThe extracted features are optimized through convolution operations. This process can be represented by the following mathematical expression:

[0152] x CA =Conv(x CA (18)

[0153] At this point, one MFDE processing step is complete. The feature is processed sequentially through N Mamba FaceDetail Enhancers (MFDEs). After all MFDEs have processed the feature, a convolution operation is performed to optimize the extracted features. In this stage, the deep features f... D The extraction is complete. Then, f... D With shallow features f s By combining elements one by one, the output is used as the input for reconstruction, thereby generating a high-resolution image I. HQ .

[0154] In this embodiment, shallow and deep features are combined to reconstruct the image, improving facial details and resolution of the low-resolution image, thereby obtaining a higher quality FRR image. Mamba is not only good at efficiently handling long-range dependencies and capturing global context, but also has a lightweight design that enables it to effectively enhance image details while maintaining computational efficiency.

[0155] Because the standard Mamba model can only capture information from the scanned portion of the data, it presents significant challenges when processing two-dimensional spatial information, such as... Figure 3 As shown, this embodiment introduces a two-dimensional selective scanning module (2D-SSM), which flattens two-dimensional image features into a one-dimensional sequence and scans along four different directions: from top left to bottom right, from bottom right to top left, from top right to bottom left, and from bottom left to top right. Subsequently, the long-range dependencies of each sequence are captured using discrete state-space equations;

[0156] Advances in structured state-space sequence models have been largely inspired by continuous linear time-invariant (LTI) systems. These systems utilize implicit latent states. One-dimensional function or sequence Mapped to This mapping can be formally represented as a linear ordinary differential equation (ODE):

[0157]

[0158] in as well as N represents the state dimension. Typically, most state-space models (SSMs) omit the second term in the observation equation, i.e., Dx(t) = 0, which can be viewed as a skip connection in deep learning models.

[0159] Subsequently, the discretization process is typically used to transform equation (19) into practical applications in deep learning algorithms. Specifically, let Δ be the time scale parameter, used to convert continuous parameters A and B into discrete parameters. A commonly used discretization method is the zero-order preserve (ZOH) rule, defined as follows:

[0160]

[0161] After discretization, equation (19) with step size Δ can be rewritten in the following recurrent neural network (RNN) form:

[0162]

[0163] More specifically, under the conditions In this case, the discrete state-space model (SSM) can independently compute the output at each time step, as shown below:

[0164]

[0165] Furthermore, equations (21) and (22) can also be mathematically transformed into the following convolutional neural network (CNN) form:

[0166]

[0167] Where L is the length of the input sequence. This represents a convolution operation, x = [x0, x1, ..., x...]. k ], It is a structured convolution kernel.

[0168] The advanced state-space model Mamba further improves upon B, C, and Δ, enabling them to dynamically adjust based on the input and achieve richer feature representations. Mamba's intuition regarding FRR builds upon the advantages of structured state-space sequence models. Specifically, Mamba retains the same recursive form as equation (19), allowing the model to memorize very long sequences and activate more pixels for FRR. Furthermore, the parallel scan algorithm allows Mamba to leverage the advantages of parallel processing, as shown in equation (23), thereby facilitating efficient training.

[0169] In this embodiment, four datasets were used: the deepFRR dataset, and the RetouchingFFHQR-1, RetouchingFFHQR-2, and RetouchingFFHQR-3 datasets extracted from the comprehensive RetouchingFFHQR dataset. The deepFRR dataset serves as the baseline dataset for the FRR task, employing six beautification operations, each using the highest level of beautification intensity. The RetouchingFFHQR-1, RetouchingFFHQR-2, and RetouchingFFHQR-3 datasets were used for generalization testing. These three subsets correspond to the Alibaba API, Tencent API, and Megvii API operations, respectively, with only one of the four beautification operations applied (i.e., skin smoothing, face whitening, face lifting, and eye enlargement). Specifically, RetouchingFFHQR-1, RetouchingFFHQR-2, and RetouchingFFHQR-3 contain 36,000, 192,000, and 57,304 retouched facial images, respectively, with the degree of retouching divided into three levels (mild: 30, moderate: 60, severe: 90). To ensure comprehensive comparative evaluation, several high-performing image generation methods were selected, including GP-UNIT, StableDiffusion, FRRffusion, and DiT.

[0170] The network in this embodiment is built on the PyTorch framework and trained using six Tesla 4090 GPUs. In the first stage, T is set to 1000. The downsampled image size input to the diffusion model is 128×128, and the network exhibits good convergence after 1,500,000 iterations. In the second stage, data augmentation is performed by applying horizontal flipping and random rotations of 90°, 180°, and 270°. Furthermore, during training, the original images are cropped into 64×64 blocks for FRR. For FRR, the model weights are initialized using pre-trained weights from the ×4 model, while the learning rate and total number of training iterations are halved to reduce training time.

[0171] To ensure fair comparison, this embodiment adjusts the training batch size of FRR to 32. The Adam optimizer is used to train the DeltaFRR model, with parameters set to β1 = 0.9 and β1 = 0.999. The initial learning rate is set to 2 × 10⁻⁶. -4 The learning rate is halved when training iterations reach a certain milestone. Finally, the DeltaFRR model has 125 + 16.7 MB of parameters and a computational cost of 88 + 439 GFLOPS.

[0172] To evaluate the performance of the proposed DeltaFRR, this embodiment employs four common metrics: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), VGG Score (VGGS), and CLIP Score (CLIPS). SSIM measures the structural similarity of an image by considering brightness, contrast, and structural information. PSNR quantifies image reconstruction quality by measuring the mean squared error (MSE) between the original and reconstructed images, making it particularly suitable for objective pixel-level distortion assessment. VGGS and CLIP evaluate the perceptual similarity between the original face image and the de-beautified image. VGGS is computed using a VGG network trained on the VGGface2 dataset, providing efficient similarity quantification focused on local face regions. CLIP, on the other hand, is computed using a CLIP network trained on a large-scale image-text pairing dataset, enabling better similarity assessment of the overall image.

[0173] In this embodiment, the performance of the DeltaFRR method and its comparative methods was evaluated on four datasets. It is important to note that in the experiments, all images in the depth FRR dataset were resized to 512×512 for computational convenience. The experimental results are shown in Tables 1 and 2. The following are some observations:

[0174] Table 1 Comparison of SSIM and PSNR performance on a single dataset

[0175]

[0176] Table 2 compares the performance of VGGS and CLIIPS on a single dataset.

[0177]

[0178] Where DeltaFRR_stage 1 and DeltaFRR_stage 2 represent the 128×128 and 512×512 images output by DeltaFRR in the first and second stages, respectively. Raw and Retouched represent the corresponding values ​​calculated from the original image and the beautified image, respectively;

[0179] First, the DeltaFRR method in this embodiment demonstrates significant superiority across all four evaluation metrics (SSIM, PSNR, VGGS, and CLIPS) on all four datasets. Compared to the baseline method FRRffusion, DeltaFRR achieves significant improvements of 3.13%, 5.79%, 1.47%, and 3.72% in average SSIM, PSNR, CLIPS, and VGGS, respectively. These improvements are attributed to the optimization of DeltaFRR, including models with Delta-map and FDG components specifically designed for FRR tasks, which not only exhibit superior performance but also achieve significant gains, with the most significant improvements reaching 32.94% in SSIM and 13.92% in PSNR. These significant advancements are primarily due to the use of VAE encoding in other methods (such as DiT and Stable Diffusion) during processing, leading to the loss of some image information and thus reducing network performance. In contrast, DeltaFRR outperforms GP-UNIT on all objective metrics. It is worth noting that although GP-UNIT performs well in style transfer applications, it remains insufficient for face colorization tasks.

[0180] Secondly, compared to the Raw and Retouched baselines, DeltaFRR's SSIM, PSNR, VGGS, and CLIPS metrics gradually approach—and in some cases even surpass—Raw and Retouched. Specifically, DeltaFRR outperforms Raw and Retouched on most datasets in VGGS and CLIPS metrics. However, in terms of SSIM and PSNR, only the Deep FRR dataset outperforms Raw and Retouched. This difference can be attributed to the fact that VGGS and CLIPS primarily evaluate the visual similarity between two images and are less sensitive to pixel-level details, which are more critical in SSIM and PSNR evaluations. Furthermore, the Deep FRR dataset undergoes extensive modification involving four types of operations with a maximum intensity of 100, while the other three datasets involve only one type of modification with a lower intensity. This greater degree of modification in the Deep FRR dataset may contribute to its relatively lower performance in SSIM and PSNR metrics.

[0181] Finally, from DeltaFRR_stage 1 to DeltaFRR_stage 2, various metrics (including SSIM, PSNR, VGGS, and CLIPS) showed varying degrees of decline. Specifically, the average decline across the datasets was as follows: SSIM decreased by 7.55%, PSNR by 7.15%, VGGS by 0.24%, and CLIPS by 1.14%. This transition was accompanied by an increase in image resolution from 128×128 to 512×512, and the restoration of facial details. With the increased resolution, the model must more effectively capture and reconstruct subtle details, which not only increases computational complexity but may also lead to insufficient detail processing. This poses a significant challenge to deep networks, especially at the pixel level, where the model's ability to restore details faces higher demands. Compared to the Transformer-based method in FRRffusion, the Mamba-based FDG module (i.e., the Facial Detail Generator FDG) proposed in this embodiment demonstrates superior performance.

[0182] In this embodiment, cross-validation experiments were conducted to evaluate the generalization performance of the proposed DeltaFRR model. The results are shown in Table 3 below. The percentages and arrows represent the percentage changes when comparing single-item tests with cross-dataset tests. The following observations can be made:

[0183] Table 3. Experimental results of the cross-validation experiment.

[0184]

[0185] Compared to performance on a single dataset, the cross-dataset test results (indicated by blue percentages and arrows) show a slight decline, but still demonstrate satisfactory generalization ability across all four objective metrics. In particular, they outperform SSIM and PSNR on VGGS and CLIPS metrics. This is because, compared to SSIM and PSNR, which focus on pixel-level distortion measurements, VGGS and CLIPS metrics better reflect the perceptual similarity between different face images of the same identity from a face recognition perspective. Furthermore, the Deep FRR and Retouching FFHQ-3 datasets outperform other cross-validations. This can be explained by the fact that both datasets underwent similar retouching algorithms, namely the Megvii Face Retouching API.

[0186] Compared to FRRffusion, all four metrics across the dataset show improvement. This significant improvement in generalization can be attributed to two main reasons. First, in the initial stage, a Delta-map module was specifically designed to emphasize retouching, which facilitated better training of the neural network. Second, in subsequent stages, the FDG module effectively captured complex facial details, successfully achieving detail reconstruction and super-resolution of facial images. This improvement stems in part from the fact that Transformer-based recovery networks typically divide the input into smaller blocks or employ moving window attention to maintain efficiency, which limits interaction at the overall image level.

[0187] In this embodiment, to fully verify the effectiveness of the proposed DeltaFRR in the similarity recognition task, four state-of-the-art face recognition models—VGG-Face, FaceNet, OpenFace, and ArcFace—were used to evaluate the recognition similarity between the original image and the modified / unmodified image. The results are as follows: Figure 4 As shown in the figure, the Gaussian peak of DeltaFRR is closest to the reference value of 1 among all contrast methods, indicating that DeltaFRR achieves the best performance in the demodification task. Furthermore, the peak of GP-UNIT is similar to the baseline of both the original and retouched images (i.e.,...). Figure 4 The peak value of Stable Diffusion is slightly further away from the reference value compared to Retouching, and the peak value of Stable Diffusion is significantly far away from the reference value, further confirming its limitations in processing FRR tasks.

[0188] It is worth noting that in the FRR mission, VGG-Face ( Figure 4 (a) and ArcFace Figure 4 (d) in this context is superior to FaceNet in clearly displaying the de-embellishment effect. Figure 4 (b) and OpenFace Figure 4 (c)). This difference may be due to the close clustering of Gaussian peaks in the FaceNet and OpenFace results, leading to insufficient differentiation between different methods. In contrast, VGG-Face and ArcFace show more obvious separation in the distribution of Gaussian peaks, making the evaluation of different de-modification strategies more intuitive and clear. This difference may be attributed to changes in model architecture and training data, which affect the model's ability to analyze facial details and residual features after modification.

[0189] In this embodiment, numerous subjective results and analyses are provided to demonstrate the effectiveness of the proposed DeltaFRR. A set of samples is randomly selected from the deepFRR dataset, such as... Figure 5As shown, the corresponding images restored by the comparison method are displayed. By magnifying these images, it can be observed that the facial images processed by DeltaFRR have a higher similarity to their original images compared to the retouched facial images. This higher similarity is reflected in several aspects, including skin tone, face size, face shape, eye size, facial details, and eyebrow shape. This indicates that the method achieves very good results in FRR.

[0190] The observation that the similarity between the facial images restored by Stable Diffusion and the original faces is lower than that of the retouched facial images indicates that this powerful image generation method (Stable Diffusion) cannot meet the requirements of the FRR task. This finding confirms the conclusions drawn in the quantitative results above. GP-UNIT exhibits a counterintuitive phenomenon. That is, visually, most facial images generated by GP-UNIT are more similar to their original images than to the retouched facial images. However, when reviewing the previous quantitative evaluation, it was found that the results of the four objective metrics obtained by the GP-UNIT method are lower than those of the original and retouched baselines. This shows an inconsistency between subjective and objective measurements, reflecting that existing methods cannot adequately address the FRR problem. Encouragingly, the proposed DeltaFRR fills this gap, achieving consistency between subjective and objective metrics.

[0191] Finally, the generalization ability of the method is demonstrated from a visual perspective, such as... Figure 6 As shown in each row, the facial image restored by DeltaFRR after a single retouching operation is highly similar to the original image. This demonstrates that DeltaFRR can handle single types of retouching operations, including facial smoothing, facial lifting, eye enlargement, and facial whitening. Furthermore, in Figure 7 The diagram demonstrates cross-API performance, showing that DeltaFRR was trained on datasets with different API modifiers and tested on the same deepFRR dataset. It can be observed that the images in each column look almost identical. These two results demonstrate that DeltaFRR exhibits excellent generalization ability in both single-modifier and cross-API modifier operations.

[0192] To comprehensively evaluate the impact of input image size on the overall performance of the FDR module, the input image resolution was adjusted to 64×64, 128×128, and 256×256. The magnification factor in the FDG module was also adjusted accordingly due to the change in input size. The experimental results for each resolution setting are shown in Table 4. The results show that increasing the input image size has a positive impact on network performance. Specifically, when the input image size increased from 64×64 to 128×128 and 256×256, the PSNR values ​​improved by 10% and 6%, respectively. However, while higher resolution input images significantly enhance network performance, they also require more computational resources. Due to limited computational resources, the potential benefits of larger input sizes (such as 512×512) could not be further explored. With lower input resolution, the FDR module (i.e., the Face Difference Map Restorer FDR) outputs low-resolution images, which then require large-scale super-resolution reconstruction by the FDG module. During this process, the PSNR and SSIM values ​​decrease significantly, leading to a substantial drop in overall network performance.

[0193] A comprehensive evaluation of the impact of the FDG module on overall performance was conducted, exploring five different types of super-resolution networks: GAN-based Real-ESRGAN, Transformer-based HAT-B and DAEFR, diffusion-based BFRffusion, and the Mamba-based FDG module. Test results for each network are summarized in Table 5. Experimental results show that the Mamba-based FDG module outperforms the PSNR, SSIM, VGGS, and CLIPS metrics. This significant performance is attributed to Mamba's unique advantages in facial super-resolution tasks, particularly its powerful ability to recover subtle local details and accurately capture facial features. Furthermore, Mamba's efficient multi-scale information fusion enhances image reconstruction while reducing noise interference. Considering the balance between model complexity, computational cost, and performance, the Mamba-based FDG module was ultimately selected as the super-resolution backbone of DeltaFRR to ensure optimal performance in complex real-world scenes.

[0194] Table 4 Ablation studies with different downsampling sizes

[0195]

[0196] Table 5 Ablation studies using different super-resolution algorithms

[0197]

[0198] In this embodiment, a new deepFRR dataset was created, which is the first FRR dataset. Based on the deepFRR dataset, an innovative solution was proposed, and the DeltaFRR network was designed, which contains two key components: a face difference map restorer (FDR) based on a diffusion model and a face detail generator based on Mamba technology. Extensive experimental results validate the feasibility and effectiveness of the method in solving the FRR problem. Furthermore, ablation experiments were conducted to gain a deeper understanding of the parameter selection process of DeltaFRR, providing valuable guidance for further exploration of the FRR problem. It is worth noting that because current face retouching APIs operate within local facial regions using sophisticated algorithms, neither traditional manual image restoration methods nor existing deep methods (such as Stable Diffusion and GP-UNIT) can generate satisfactory results close to the original image. The DeltaFRR in this embodiment explores the potential of the conditional diffusion model in a coarse-to-fine manner, providing a new paradigm for handling the FRR problem.

[0199] Example 2

[0200] This embodiment provides a facial retouching and restoration system based on Diffusion-Mamba collaboration, used to implement the facial retouching and restoration method based on Diffusion-Mamba collaboration in Embodiment 1 above. It includes: a downsampling module, a retouching map acquisition module, a DeltaFRR model construction module, a difference map acquisition module, a facial difference map restorer training module, a low-resolution restoration map output module, and an image reconstruction module.

[0201] In this embodiment, the downsampling module is used to downsample the face image to be restored to obtain a low-resolution face image;

[0202] In this embodiment, the modified image acquisition module is used to detect whether the face image has been modified and to acquire the modified image;

[0203] In this embodiment, the DeltaFRR model building module is used to build a DeltaFRR model, including a facial difference map restorer and a facial detail generator;

[0204] In this embodiment, the difference map acquisition module is used to subtract the face image to be restored from the modified image to obtain the difference map;

[0205] In this embodiment, the facial difference map restorer training module is used to stitch together the difference map and the noise-added modified map and input them into the facial difference map restorer for training;

[0206] In this embodiment, the low-resolution reconstruction image output module outputs a low-resolution reconstruction image based on the trained facial difference image restorer.

[0207] In this embodiment, the image reconstruction module is used to input the low-resolution restored image into the facial detail generator to extract shallow and deep features, and to reconstruct a high-resolution image based on the extracted shallow and deep features.

[0208] In this embodiment, the facial difference map restorer training module is used to stitch the difference map and the noise-added modified map together and input them into the facial difference map restorer for training, specifically including:

[0209] The difference image and the noise-added modified image are concatenated as input, and the modified image is used as a supervised condition for training.

[0210] The facial difference map restorer is trained using a denoising diffusion probability model, which includes a forward noise addition stage and a backward denoising stage.

[0211] During the forward noise addition phase, noise is added at an increasing diffusion rate, generating multiple noisy images that converge to a standard Gaussian distribution.

[0212] The introduced noise is gradually removed in the backward denoising stage to obtain the target image.

[0213] In this embodiment, the image reconstruction module is used to input the low-resolution restored image into the facial detail generator to extract shallow and deep features, and reconstruct a high-resolution image based on the extracted shallow and deep features, specifically including:

[0214] The low-resolution reconstruction image generates shallow features f through convolutional layers. s ;

[0215] Shallow features f s Deep features f are extracted sequentially using multiple Mamba-based facial detail enhancers. D ;

[0216] Shallow features f s With deep features f D By combining elements one by one, the resulting output is used as the input for reconstruction, and finally, a high-resolution image is output.

[0217] In this embodiment, shallow feature f s Deep features f are extracted sequentially using multiple Mamba-based facial detail enhancers. D Specifically, it includes:

[0218] Shallow features f s Through layer normalization, we obtain:

[0219] x N =LN(f s )

[0220] Feature x N The processing is carried out through two branches. One branch consists of a linear layer and SiLU, which expands the feature channels to λC. The other branch includes a linear layer, depthwise convolution (DWConv), SiLU, a two-dimensional state space model, and a layer normalization layer, which expands the feature channels to λC, where λ is a predefined channel expansion factor and C represents the current number of channels.

[0221] The features from the two branches are aggregated using the Hadamard product, specifically as follows:

[0222] x M =LN(2D-SSM(SiLU(DWConv(Linear(x N )))))⊙SiLU(Linear(x N ))

[0223] Where DWConv represents depthwise convolution, SiLU represents the activation function, and ⊙ represents the Hadamard product;

[0224] Feature x M Through a linear layer, and with weighted shallow features f s Add them together to generate the output x. 2D , is represented as:

[0225] x 2D =Linear(x M )+Scale×f s

[0226] Feature x 2D After layer normalization, it is passed through a convolutional layer and a channel attention module, and then combined with the weighted feature x. 2D The sums are then passed to another convolutional layer, represented as:

[0227] x CA =CA(Conv(LN(x) 2D )))+Scale×x 2D

[0228] The features are processed sequentially through N facial detail enhancers. After all facial detail enhancers have been processed, a convolution operation is performed to optimize the extracted features and obtain deep features.

[0229] In this embodiment, the facial detail enhancer based on Mamba technology is further provided with a two-dimensional selective scanning module. The two-dimensional selective scanning module flattens the two-dimensional image features into a one-dimensional sequence and scans along four different directions: from the top left to the bottom right, from the bottom right to the top left, from the top right to the bottom left, and from the bottom left to the top right.

[0230] Discrete state-space equations are used to capture the long-range dependencies of each sequence.

[0231] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A facial retouching and restoration method based on Diffusion-Mamba collaboration, characterized in that, Includes the following steps: The face image to be restored is downsampled to obtain a low-resolution face image; Detect whether a face image has been modified and obtain the modified image; Construct a DeltaFRR model, including a facial difference map restorer and a facial detail generator; The difference between the face image to be restored and the retouched image is calculated to obtain the difference image; Noise is added to the modified image, and the difference image and the modified image with added noise are stitched together and then input into the facial difference image restorer for training. The face difference map restorer outputs a low-resolution restored image based on the trained face difference map; The low-resolution restored image is input into the facial detail generator to extract shallow and deep features. Based on the extracted shallow and deep features, a high-resolution image is reconstructed.

2. The facial retouching and restoration method based on Diffusion-Mamba collaboration according to claim 1, characterized in that, The difference map and the noise-added modified map are concatenated and then input into the facial difference map restorer for training. Specifically, this includes: The difference image and the noise-added modified image are concatenated as input, and the modified image is used as a supervised condition for training. The facial difference map restorer is trained using a denoising diffusion probability model, which includes a forward noise addition stage and a backward denoising stage. During the forward noise addition phase, noise is added at an increasing diffusion rate, generating multiple noisy images that converge to a standard Gaussian distribution. The introduced noise is gradually removed in the backward denoising stage to obtain the target image.

3. The facial retouching and restoration method based on Diffusion-Mamba collaboration according to claim 1, characterized in that, The low-resolution reconstructed image is input into a facial detail generator to extract shallow and deep features. Based on the extracted shallow and deep features, a high-resolution image is reconstructed, specifically including: The low-resolution reconstruction image generates shallow features f through convolutional layers. s ; Shallow features f s Deep features f are extracted sequentially using multiple Mamba-based facial detail enhancers. D ; Shallow features f s With deep features f D By combining elements one by one, the resulting output is used as the input for reconstruction, and finally, a high-resolution image is output.

4. The facial retouching and restoration method based on Diffusion-Mamba collaboration according to claim 3, characterized in that, Shallow features f s Deep features f are extracted sequentially using multiple Mamba-based facial detail enhancers. D Specifically, it includes: Shallow features f s Through layer normalization, we obtain: x N =LN(f s ) Feature x N The processing is carried out through two branches. One branch consists of a linear layer and SiLU, which expands the feature channels to λC. The other branch includes a linear layer, depthwise convolution (DWConv), SiLU, a two-dimensional state space model, and a layer normalization layer, which expands the feature channels to λC, where λ is a predefined channel expansion factor and C represents the current number of channels. The features from the two branches are aggregated using the Hadamard product, specifically as follows: x M =LN(2D-SSM(SiLU(DWConv(Linear(x N )))))⊙SiLU(Linear(x N )) Where DWConv represents depthwise convolution, SiLU represents the activation function, and ⊙ represents the Hadamard product; Feature x M Through a linear layer, and with weighted shallow features f s Add them together to generate the output x. 2D , represented as: x 2D =Linear(x M )+Scale×f s Feature x 2D After layer normalization, it is passed through a convolutional layer and a channel attention module, and then combined with the weighted feature x. 2D The sums are then passed to another convolutional layer, represented as: x CA =CA(Conv(LN(x 2D )))+Scale×x 2D The features are processed sequentially through N facial detail enhancers. After all facial detail enhancers have been processed, a convolution operation is performed to optimize the extracted features and obtain deep features.

5. The facial retouching and restoration method based on Diffusion-Mamba collaboration according to claim 3, characterized in that, The facial detail enhancer based on Mamba technology also features a two-dimensional selective scanning module, which flattens two-dimensional image features into a one-dimensional sequence and scans them in four different directions: from top left to bottom right, from bottom right to top left, from top right to bottom left, and from bottom left to top right. Discrete state-space equations are used to capture the long-range dependencies of each sequence.

6. A facial retouching and restoration system based on Diffusion-Mamba collaboration, characterized in that, include: The module includes a downsampling module, a modification map acquisition module, a DeltaFRR model construction module, a difference map acquisition module, a facial difference map restorer training module, a low-resolution restored map output module, and an image reconstruction module. The downsampling module is used to downsample the face image to be restored to obtain a low-resolution face image; The modified image acquisition module is used to detect whether a face image has been modified and to acquire a modified image; The DeltaFRR model building module is used to build a DeltaFRR model, including a facial difference map restorer and a facial detail generator; The difference map acquisition module is used to subtract the face image to be restored from the modified image to obtain the difference map. The facial difference map restorer training module is used to stitch together the difference map and the noise-added modified map and input them into the facial difference map restorer for training; The low-resolution restored image output module outputs a low-resolution restored image based on the trained facial difference map restorer. The image reconstruction module is used to input the low-resolution restored image into the facial detail generator to extract shallow and deep features, and to reconstruct a high-resolution image based on the extracted shallow and deep features.

7. The facial retouching and restoration system based on Diffusion-Mamba collaboration according to claim 6, characterized in that, The facial difference map restorer training module is used to stitch together the difference map and the noise-added modified map and input them into the facial difference map restorer for training. Specifically, it includes: The difference image and the noise-added modified image are concatenated as input, and the modified image is used as a supervised condition for training. The facial difference map restorer is trained using a denoising diffusion probability model, which includes a forward noise addition stage and a backward denoising stage. During the forward noise addition phase, noise is added at an increasing diffusion rate, generating multiple noisy images that converge to a standard Gaussian distribution. The introduced noise is gradually removed in the backward denoising stage to obtain the target image.

8. The facial retouching and restoration system based on Diffusion-Mamba collaboration according to claim 6, characterized in that, The image reconstruction module is used to input the low-resolution restored image into the facial detail generator to extract shallow and deep features, and reconstruct a high-resolution image based on the extracted shallow and deep features, specifically including: The low-resolution reconstruction image generates shallow features f through convolutional layers. s ; Shallow features f s Deep features f are extracted sequentially using multiple Mamba-based facial detail enhancers. D ; Shallow features f s With deep features f D By combining elements one by one, the resulting output is used as the input for reconstruction, and finally, a high-resolution image is output.

9. The facial retouching and restoration system based on Diffusion-Mamba collaboration according to claim 8, characterized in that, Shallow features f s Deep features f are extracted sequentially using multiple Mamba-based facial detail enhancers. D Specifically, it includes: Shallow features f s Through layer normalization, we obtain: x N =LN(f s ) Feature x N The processing is carried out through two branches. One branch consists of a linear layer and SiLU, which expands the feature channels to λC. The other branch includes a linear layer, depthwise convolution (DWConv), SiLU, a two-dimensional state space model, and a layer normalization layer, which expands the feature channels to λC, where λ is a predefined channel expansion factor and C represents the current number of channels. The features from the two branches are aggregated using the Hadamard product, specifically as follows: x M =LN(2D-SSM(SiLU(DWConv(Linear(x N )))))⊙SiLU(Linear(x N )) Where DWconv represents depthwise convolution, SiLU represents the activation function, and ⊙ represents the Hadamard product; Feature x M Through a linear layer, and with weighted shallow features f s Add them together to generate the output x. 2D , represented as: x 2D =linear(x M )+Scale×f s Feature x 2D After layer normalization, it is passed through a convolutional layer and a channel attention module, and then combined with the weighted feature x. 2D The sums are then passed to another convolutional layer, represented as: x CA =CA(Conv(LN(x 2D )))+Scale×x 2D The features are processed sequentially through N facial detail enhancers. After all facial detail enhancers have been processed, a convolution operation is performed to optimize the extracted features and obtain deep features.

10. The facial retouching and restoration system based on Diffusion-Mamba collaboration according to claim 8, characterized in that, The facial detail enhancer based on Mamba technology also features a two-dimensional selective scanning module, which flattens two-dimensional image features into a one-dimensional sequence and scans them in four different directions: from top left to bottom right, from bottom right to top left, from top right to bottom left, and from bottom left to top right. Discrete state-space equations are used to capture the long-range dependencies of each sequence.

Images