Identity figure image enhancement system based on diffusion model

Abstract

The invention discloses an identity figure image enhancement system based on a diffusion model, which relates to the technical field of figure image enhancement and comprises an identity embedding extraction module, a saliency map generation module, a noise scheduling construction module, a denoising guide generation module, a diffusion denoising execution module and an attention modulation module. The identity embedding and extracting module is used for performing identity feature extraction on an input low-quality figure image by adopting a pre-trained face recognition model to obtain an identity embedding vector; the saliency map generation module is used for carrying out pixel-level region division on an input low-quality figure image by adopting a lightweight semantic segmentation network to generate a face saliency thermodynamic diagram, and the face saliency thermodynamic diagram is used for representing spatial distribution of face key regions; and the noise scheduling construction module is used for carrying out spatial weighting adjustment on a basic noise scheduling coefficient according to the facial saliency thermodynamic diagram, and constructing a pixel-level adaptive local noise variance field.

Description

Technical Field

[0001] This invention relates to the field of image enhancement technology, and in particular to an identity image enhancement system based on a diffusion model. Background Technology

[0002] Image enhancement technology for people refers to the use of computer vision and deep learning techniques to process low-quality input images of people in order to improve their clarity, remove noise, repair damaged parts, and restore or enhance key features of the person in the image as much as possible, thereby obtaining higher-quality images. Therefore, how to improve the intelligence and security of image enhancement using advanced technologies has become one of the urgent problems to be solved.

[0003] In the field of portrait image enhancement, existing enhancement algorithms struggle to effectively restore image clarity and detail for portrait images under complex conditions, resulting in unsatisfactory enhancement effects. Furthermore, traditional diffusion models fail to fully consider the spatial distribution characteristics of key facial regions during the denoising process, which can easily lead to distortion or loss of important facial features, affecting the quality of the final generated image. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides an identity image enhancement system based on a diffusion model to solve the problem that existing enhancement algorithms are unable to effectively restore the clarity and details of images under complex conditions, resulting in unsatisfactory enhancement effects. Furthermore, traditional diffusion models fail to fully consider the spatial distribution characteristics of key facial regions during the denoising process, which can easily cause distortion or loss of important facial features and affect the quality of the final generated image.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides an identity and person image enhancement system based on a diffusion model, comprising:

[0008] The module includes an identity embedding extraction module, a saliency map generation module, a noise scheduling construction module, a denoising guidance generation module, a diffusion denoising execution module, and an attention modulation module.

[0009] The identity embedding extraction module is used to extract identity features from the input low-quality human image using a pre-trained face recognition model to obtain an identity embedding vector.

[0010] The saliency map generation module is used to perform pixel-level region division on the input low-quality human image using a lightweight semantic segmentation network to generate a facial saliency heatmap, which is used to characterize the spatial distribution of key facial regions.

[0011] The noise scheduling construction module is used to spatially weight and adjust the basic noise scheduling coefficients according to the facial saliency heatmap to construct a pixel-level adaptive local noise variance field. The local noise variance field is used to differentiate the noise injection intensity in different regions during the diffusion process.

[0012] The denoising guidance generation module is used to calculate the gradient direction of identity consistency loss based on the identity embedding vector and the intermediate image in the current denoising state, and generate an identity gradient guidance term. The identity gradient guidance term is used to dynamically constrain the image evolution path during the back diffusion process to maintain identity features.

[0013] The diffusion denoising execution module is used to perform denoising operations step by step within the total time step, starting from the initial noisy image. The denoising update at each time step is based on the superposition result of the predicted noise and the identity gradient guiding term, and the state transition is performed in combination with the local noise variance field to obtain a progressively clearer intermediate image sequence and output an enhanced image.

[0014] The attention modulation module is used to map the identity embedding vector into a key-value vector in the attention layer of the U-Net architecture adopted by the diffusion denoising execution module, and to achieve dynamic modulation of the feature attention weights by identity semantic information through the fusion of learnable weights and standard attention mechanism. The modulation result is used to improve the feature reconstruction accuracy of key facial regions.

[0015] As a preferred embodiment of the identity enhancement system based on the diffusion model described in this invention, the identity embedding extraction module uses a pre-trained face recognition model to extract features from the input low-quality person image to obtain an identity embedding vector, specifically including:

[0016] A face recognition model with ResNet-34 as the backbone network is used to perform multi-layer convolution and pooling operations on the input low-quality human images to extract deep semantic features;

[0017] The last feature map is input into a global average pooling layer to obtain a high-dimensional feature vector;

[0018] The high-dimensional feature vector is subjected to L2 normalization to obtain a unit-length identity embedding vector;

[0019] The identity embedding vector serves as the conditional input signal for subsequent modules.

[0020] As a preferred embodiment of the identity image enhancement system based on the diffusion model described in this invention, the saliency map generation module employs a lightweight semantic segmentation network to perform pixel-level classification on the input low-quality person image, generating a facial saliency heatmap, specifically including:

[0021] A semantic segmentation network with an encoder-decoder structure is used to encode features of the input low-quality human images and extract multi-scale contextual information.

[0022] During the decoding stage, spatial resolution is restored through upsampling and skip connections, and a pixel-level prediction map with the same size as the input image is output.

[0023] The regions in the predicted image that belong to the eyes, eyebrows, nose, mouth, and cheeks are merged into a mask for key identity regions;

[0024] Gaussian smoothing is applied to the mask using the following kernel function:

[0025] ;

[0026] Obtain facial saliency thermogram .

[0027] As a preferred embodiment of the identity image enhancement system based on the diffusion model described in this invention, the noise scheduling construction module uses a facial saliency heatmap to spatially weight and adjust the basic noise scheduling coefficients to construct a local noise variance field, specifically including:

[0028] The basic noise scheduling sequence is obtained by the following expression:

[0029] ;

[0030] scalar Expanded into a matrix form of the same size as the image;

[0031] Facial saliency heatmap as weighted map For each spatial location The modulation operation is performed, and the expression is:

[0032] ;

[0033] in These are modulation coefficients used to control the noise suppression strength of the identity region;

[0034] The local noise variance field corresponding to each time step is obtained through the above calculations. It exhibits lower noise variance in saliency regions and maintains higher noise variance in non-saliency regions.

[0035] The local noise variance field Spatial differential control used in diffusion denoising processes.

[0036] As a preferred embodiment of the identity image enhancement system based on the diffusion model described in this invention, the denoising-guided generation module generates a gradient guidance term using the feature difference between the identity embedding vector and the intermediate image, specifically including:

[0037] When the diffusion denoising execution module reaches a time step, the current intermediate state image is acquired;

[0038] Will Input a face recognition model with the same structure as the identity embedding and extraction module to obtain the current identity features. ;

[0039] The loss for identity consistency is calculated as follows:

[0040] ;

[0041] Regarding the loss Regarding the intermediate image Find the gradient, and get ;

[0042] Multiply the gradient by the time-varying guiding coefficient The identity gradient guiding term is obtained. The expression is:

[0043] ;

[0044] in With time step Monotonically decreasing;

[0045] The identity gradient guidance item Used to correct the direction of noise reduction and maintain identity consistency.

[0046] As a preferred embodiment of the identity and person image enhancement system based on the diffusion model described in this invention, the diffusion denoising execution module employs a modified denoising term and a local noise variance field to jointly drive the reverse diffusion process, specifically including:

[0047] Initialize noisy image ;

[0048] For each time step from Decrease to 1, then perform the following steps:

[0049] A denoising network with a U-Net structure is used to... and identity embedding vector Given the input, predict the noise component. ;

[0050] Predict noise With identity gradient guidance Adding them together yields the corrected noise term. ;

[0051] Based on the diffusion model transfer rule and combined with the local noise variance field, the image state at the next time step is calculated, and the expression is:

[0052] ;

[0053] in, The residual noise variance;

[0054] Repeat the above process until Output enhanced image .

[0055] As a preferred embodiment of the identity enhancement system based on the diffusion model described in this invention, the attention modulation module generates an auxiliary attention branch using an identity embedding vector and fuses it with the main attention mechanism, specifically including:

[0056] The identity embedding vector is mapped to a key vector through a linear transformation, and then mapped to a value vector through another linear transformation.

[0057] In the attention layer of the U-Net decoder, the attention weights between the query vector and the standard key matrix are calculated;

[0058] Simultaneously calculate the attention weights of the query vector and the identity key vector;

[0059] The two attention weights are weighted and fused, as expressed by:

[0060] ;

[0061] Integrate attention weights Acting on the standard value matrix The modulated features are output from the identity value vector, and the expression is:

[0062] ;

[0063] in The learnable fusion coefficient;

[0064] The modulated features As the output of the current layer, it is used for subsequent feature propagation.

[0065] As a preferred embodiment of the identity enhancement system based on the diffusion model described in this invention, the model parameters of the identity embedding extraction module and the saliency map generation module, on which the diffusion denoising execution module depends, remain frozen during the system training phase, specifically including:

[0066] During the end-to-end training phase of the system, the weight parameters of the face recognition model and the semantic segmentation network are fixed, and only the learnable parameters and guidance coefficients in the U-Net backbone network and the attention modulation module are optimized.

[0067] Training is performed using a joint loss function, expressed as follows:

[0068] ;

[0069] in, To reduce noise and reconstruct losses, For loss of identity consistency, For attention distribution alignment loss, , This is the balance coefficient.

[0070] In a second aspect, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the diffusion model-based identity person image enhancement system as described in the first aspect of the present invention.

[0071] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the diffusion model-based identity person image enhancement system as described in the first aspect of the present invention.

[0072] The beneficial effects of this invention are as follows: A lightweight semantic segmentation network is used to perform pixel-level region division of the input image, generating a facial saliency heatmap. This achieves accurate representation of the spatial distribution of key facial regions, emphasizing attention to important feature regions such as eyes and eyebrows, allowing for more targeted optimization in subsequent operations. By spatially weighting the basic noise scheduling coefficients of different facial regions based on the facial saliency heatmap, a local noise variance field is constructed, enabling precise control of noise injection intensity for different regions during diffusion denoising. Based on the identity embedding vector and the intermediate image in the current denoising state, the gradient direction of the identity consistency loss is calculated, generating an identity gradient guiding term. This achieves the function of dynamically constraining the image evolution path to maintain identity features during reverse diffusion. Attached Figure Description

[0073] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0074] Figure 1 This is a flowchart of the identity and person image enhancement system based on the diffusion model in Example 1. Detailed Implementation

[0075] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0076] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0077] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0078] Example 1, referring to Figure 1 This is the first embodiment of the present invention, which provides an identity person image enhancement system based on a diffusion model, including:

[0079] The module includes an identity embedding extraction module, a saliency map generation module, a noise scheduling construction module, a denoising guidance generation module, a diffusion denoising execution module, and an attention modulation module.

[0080] The identity embedding extraction module is used to extract identity features from low-quality human images input by using a pre-trained face recognition model, and obtain an identity embedding vector.

[0081] Furthermore, a face recognition model with ResNet-34 as the backbone network is used to perform multi-layer convolution and pooling operations on the input low-quality human images to extract deep semantic features.

[0082] The last feature map is input into a global average pooling layer to obtain a high-dimensional feature vector;

[0083] The high-dimensional feature vector is subjected to L2 normalization to obtain a unit-length identity embedding vector;

[0084] The identity embedding vector serves as the conditional input signal for subsequent modules;

[0085] It should be noted that the identity embedding extraction module uses a pre-trained and fixed-structure ResNet-34 face recognition model, which can stably extract high-level identity semantic features from the input image without introducing additional training perturbations. Its output identity embedding vector has good cross-image comparability and robustness, and is especially suitable for identity feature expression under complex conditions such as low light, blur or partial occlusion, providing reliable identity prior information for subsequent modules.

[0086] The saliency map generation module is used to perform pixel-level region division on the input low-quality human image using a lightweight semantic segmentation network, and generate a facial saliency heatmap. The facial saliency heatmap is used to characterize the spatial distribution of key facial regions.

[0087] Furthermore, a semantic segmentation network with an encoder-decoder structure is used to encode features of the input low-quality human images and extract multi-scale contextual information.

[0088] During the decoding stage, spatial resolution is restored through upsampling and skip connections, and a pixel-level prediction map with the same size as the input image is output.

[0089] The regions in the predicted image that belong to the eyes, eyebrows, nose, mouth, and cheeks are merged into a mask for key identity regions;

[0090] Gaussian smoothing is applied to the mask using the following kernel function:

[0091] ;

[0092] Obtain facial saliency thermogram ;

[0093] It should be noted that the lightweight semantic segmentation network (BiSeNet series) used in the saliency map generation module significantly reduces computational overhead while ensuring high segmentation accuracy, making it suitable for embedding into end-to-end diffusion model processes. By merging key facial regions and performing Gaussian smoothing, the generated heatmap not only preserves the continuity of spatial structure but also effectively alleviates the jagged segmentation boundary problem, providing a high-quality attention guidance map for subsequent spatial adaptive noise control.

[0094] The noise scheduling construction module is used to spatially weight and adjust the basic noise scheduling coefficients based on the facial saliency heatmap to construct a pixel-level adaptive local noise variance field. The local noise variance field is used to differentiate the noise injection intensity in different regions during the diffusion process.

[0095] Furthermore, the basic noise scheduling sequence is obtained, expressed as:

[0096] ;

[0097] scalar Expanded into a matrix form of the same size as the image;

[0098] Facial saliency heatmap as weighted map For each spatial location The modulation operation is performed, and the expression is:

[0099] ;

[0100] in These are modulation coefficients used to control the noise suppression strength of the identity region;

[0101] The local noise variance field corresponding to each time step is obtained through the above calculations. It exhibits lower noise variance in saliency regions and maintains higher noise variance in non-saliency regions.

[0102] Local noise variance field Spatial differential control used in diffusion denoising process;

[0103] It should be noted that the noise scheduling construction module uses facial saliency heatmaps as spatial modulation weights to implement a differentiated noise injection strategy during the diffusion process, which "adds less noise to key areas and adds normal noise to non-key areas". The mechanism can retain the original structural information of key identity areas in the forward diffusion stage, thereby reducing the information reconstruction burden in the reverse denoising stage and improving denoising efficiency and the fidelity of facial details.

[0104] The denoising-guided generation module is used to calculate the gradient direction of identity consistency loss based on the identity embedding vector and the intermediate image in the current denoising state, and generate the identity gradient guidance term. The identity gradient guidance term is used to dynamically constrain the image evolution path to maintain identity features during the back diffusion process.

[0105] Furthermore, when the diffusion denoising execution module reaches a time step, it acquires the current intermediate state image;

[0106] Will Input a face recognition model with the same structure as the identity embedding and extraction module to obtain the current identity features. ;

[0107] The loss for identity consistency is calculated as follows:

[0108] ;

[0109] Regarding the loss Regarding the intermediate image Find the gradient, and get ;

[0110] Multiply the gradient by the time-varying guiding coefficient The identity gradient guiding term is obtained. The expression is:

[0111] ;

[0112] in With time step Monotonically decreasing;

[0113] Identity gradient guidance item Used to correct the direction of noise reduction and maintain identity consistency;

[0114] It should be noted that the denoising-guided generation module dynamically calculates the identity consistency gradient during the reverse diffusion process, and implements a progressive identity preservation strategy with strong guidance in the early stage and weak constraints in the later stage through the time-varying guidance coefficient t. This avoids the problem of image distortion or mode collapse caused by excessive gradient in traditional guidance methods, and effectively balances the contradiction between image clarity and identity feature stability.

[0115] The diffusion denoising execution module is used to perform denoising operations step by step within the total time step, starting from the initial noisy image. The denoising update at each time step is based on the superposition result of the predicted noise and the identity gradient guiding term, and the state transition is combined with the local noise variance field to obtain a progressively clearer intermediate image sequence and output the enhanced image.

[0116] Furthermore, initialize the noisy image. ;

[0117] For each time step from Decrease to 1, then perform the following steps:

[0118] A denoising network with a U-Net structure is used to... and identity embedding vector As input, predict noise components ;

[0119] Predict noise With identity gradient guidance Adding them together yields the corrected noise term. ;

[0120] Based on the diffusion model transfer rule and combined with the local noise variance field, the image state at the next time step is calculated, and the expression is:

[0121] ;

[0122] in, The residual noise variance;

[0123] Repeat the above process until Output enhanced image ;

[0124] During the execution of the diffusion denoising module, the model parameters of the identity embedding extraction module and the saliency map generation module, on which it depends, remain frozen during the system training phase. Specifically, this includes:

[0125] During the end-to-end training phase of the system, the weight parameters of the face recognition model and the semantic segmentation network are fixed, and only the learnable parameters and guidance coefficients in the U-Net backbone network and the attention modulation module are optimized.

[0126] Training is performed using a joint loss function, expressed as follows:

[0127] ;

[0128] in, To reduce noise and reconstruct losses, For loss of identity consistency, For attention distribution alignment loss, , This is the balance coefficient;

[0129] It should be noted that the diffusion denoising module freezes the parameters of the identity embedding extraction module and the saliency map generation module during the training phase. This prevents semantic drift of the pre-trained semantic model due to gradient backpropagation during end-to-end training, ensuring the stability of identity feature and saliency map extraction. At the same time, it combines the joint loss function to perform multi-objective optimization of reconstruction quality, identity consistency and attention distribution, which significantly improves the model convergence and the overall quality of the generated image.

[0130] The attention modulation module is used to map the identity embedding vector into a key-value vector in the attention layer of the U-Net architecture used by the diffusion denoising execution module. It also integrates the identity semantic information with the standard attention mechanism through learnable weights to achieve dynamic modulation of the feature attention weights. The modulation result is used to improve the feature reconstruction accuracy of key facial regions.

[0131] Furthermore, the identity embedding vector is mapped to a key vector through a linear transformation, and then mapped to a value vector through another linear transformation.

[0132] In the attention layer of the U-Net decoder, the attention weights between the query vector and the standard key matrix are calculated;

[0133] Simultaneously calculate the attention weights of the query vector and the identity key vector;

[0134] The two attention weights are weighted and fused, as expressed by:

[0135] ;

[0136] Integrate attention weights Acting on the standard value matrix The modulated features are output from the identity value vector, and the expression is:

[0137] ;

[0138] in The learnable fusion coefficient;

[0139] Modulated features As the output of the current layer, it is used for subsequent feature transfer;

[0140] It should be noted that the attention modulation module constructs an independent identity semantic auxiliary path by introducing the identity embedding vector into the U-Net attention layer. Its learnable fusion with the main attention mechanism can dynamically enhance the feature response of key facial regions, especially in deep networks, effectively alleviating the problem of long-distance dependency decay. This allows for the accurate reconstruction of highly recognizable features such as eye wrinkles and lip lines during the detail restoration stage, significantly improving the identity recognizability and visual naturalness of the enhanced results.

[0141] This embodiment also provides a computer device suitable for an identity and person image enhancement system based on a diffusion model, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the identity and person image enhancement system based on a diffusion model as proposed in the above embodiment.

[0142] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0143] This embodiment also provides a storage medium on which a computer program is stored. When executed by a processor, the program implements the identity and person image enhancement system based on the diffusion model as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0144] In summary, this invention uses a lightweight semantic segmentation network to perform pixel-level region division on the input image, generating a facial saliency heatmap. This achieves accurate representation of the spatial distribution of key facial regions, emphasizing attention to important feature regions such as eyes and eyebrows, allowing for more targeted optimization in subsequent operations. By spatially weighting the basic noise scheduling coefficients of different facial regions based on the facial saliency heatmap, a local noise variance field is constructed, enabling precise control of noise injection intensity for different regions during diffusion denoising. Based on the identity embedding vector and the intermediate image in the current denoising state, the gradient direction of the identity consistency loss is calculated, generating an identity gradient guiding term. This achieves the function of dynamically constraining the image evolution path to maintain identity features during back diffusion.

[0145] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An identity and person image enhancement system based on a diffusion model, characterized in that: include: The module includes an identity embedding extraction module, a saliency map generation module, a noise scheduling construction module, a denoising guidance generation module, a diffusion denoising execution module, and an attention modulation module. The identity embedding extraction module is used to extract identity features from the input low-quality human image using a pre-trained face recognition model to obtain an identity embedding vector. The saliency map generation module is used to perform pixel-level region division on the input low-quality human image using a lightweight semantic segmentation network to generate a facial saliency heatmap, which is used to characterize the spatial distribution of key facial regions. The noise scheduling construction module is used to spatially weight and adjust the basic noise scheduling coefficients according to the facial saliency heatmap to construct a pixel-level adaptive local noise variance field. The local noise variance field is used to differentiate the noise injection intensity in different regions during the diffusion process. The denoising guidance generation module is used to calculate the gradient direction of identity consistency loss based on the identity embedding vector and the intermediate image in the current denoising state, and generate an identity gradient guidance term. The identity gradient guidance term is used to dynamically constrain the image evolution path during the back diffusion process to maintain identity features. The diffusion denoising execution module is used to perform denoising operations step by step within the total time step, starting from the initial noisy image. The denoising update at each time step is based on the superposition result of the predicted noise and the identity gradient guiding term, and the state transition is performed in combination with the local noise variance field to obtain a progressively clearer intermediate image sequence and output an enhanced image. The attention modulation module is used to map the identity embedding vector into a key-value vector in the attention layer of the U-Net architecture adopted by the diffusion denoising execution module, and to achieve dynamic modulation of the feature attention weights by identity semantic information through the fusion of learnable weights and standard attention mechanism. The modulation result is used to improve the feature reconstruction accuracy of key facial regions.

2. The identity and person image enhancement system based on the diffusion model as described in claim 1, characterized in that: The identity embedding extraction module uses a pre-trained face recognition model to extract features from the input low-quality person image to obtain an identity embedding vector, specifically including: A face recognition model with ResNet-34 as the backbone network is used to perform multi-layer convolution and pooling operations on the input low-quality human images to extract deep semantic features; The last feature map is input into a global average pooling layer to obtain a high-dimensional feature vector; The high-dimensional feature vector is subjected to L2 normalization to obtain a unit-length identity embedding vector; The identity embedding vector serves as the conditional input signal for subsequent modules.

3. The identity and person image enhancement system based on the diffusion model as described in claim 2, characterized in that: The saliency map generation module uses a lightweight semantic segmentation network to perform pixel-level classification on the input low-quality human image, generating a facial saliency heatmap, specifically including: A semantic segmentation network with an encoder-decoder structure is used to encode features of the input low-quality human images and extract multi-scale contextual information. During the decoding stage, spatial resolution is restored through upsampling and skip connections, and a pixel-level prediction map with the same size as the input image is output. The regions in the predicted image that belong to the eyes, eyebrows, nose, mouth, and cheeks are merged into a mask for key identity regions; Gaussian smoothing is applied to the mask using the following kernel function: A facial saliency heatmap M was obtained.

4. The identity and person image enhancement system based on the diffusion model as described in claim 3, characterized in that: The noise scheduling construction module uses a facial saliency heatmap to spatially weight and adjust the basic noise scheduling coefficients to construct a local noise variance field, specifically including: The basic noise scheduling sequence is obtained by the following expression: scalar β t Expanded into a matrix form of the same size as the image; Using the facial saliency heatmap as the weight map M, a modulation operation is performed on each spatial location (i,j), expressed as: Where δ is the modulation coefficient, used to control the noise suppression strength of the identity region; The local noise variance field corresponding to each time step is obtained through the above calculations. It exhibits lower noise variance in saliency regions and maintains higher noise variance in non-saliency regions; The local noise variance field Spatial differential control used in diffusion denoising processes.

5. The identity and person image enhancement system based on the diffusion model as described in claim 4, characterized in that: The denoising guidance generation module generates a gradient guidance term using the feature difference between the identity embedding vector and the intermediate image, specifically including: When the diffusion denoising execution module reaches a time step, the current intermediate state image is acquired; x t Input a face recognition model with the same structure as the identity embedding and extraction module to obtain the current identity feature φ(x). t ); The loss for identity consistency is calculated as follows: L id =||φ(x t )-z|| 2 ; For loss L id Regarding the intermediate image x t Find the gradient, and get Multiply the gradient by the time-varying guiding coefficient η t We obtain the identity gradient guiding term g. t The expression is: Where η t It decreases monotonically with time step t; The identity gradient guiding term g t Used to correct the direction of noise reduction and maintain identity consistency.

6. The identity and person image enhancement system based on the diffusion model as described in claim 5, characterized in that: The diffusion denoising execution module employs a modified denoising term and a local noise variance field to jointly drive the reverse diffusion process, specifically including: Initialize noisy image x t , t=T, For each time step t, which decreases from T to 1, perform the following steps: A denoising network using the U-Net structure, with x t t and identity embedding vector z are inputs, and the predicted noise component ∈ θ (x t ,t,z); Predict noise ∈ θ With identity gradient guiding term g t Adding them together yields the corrected noise term. Based on the diffusion model transfer rule and combined with the local noise variance field, the image state at the next time step is calculated, and the expression is: Where, γ t The residual noise variance; Repeat the above process until t-1, and output the enhanced image x0.

7. The identity and person image enhancement system based on the diffusion model as described in claim 6, characterized in that: The attention modulation module uses identity embedding vectors to generate auxiliary attention branches and integrates them with the main attention mechanism, specifically including: The identity embedding vector is mapped to a key vector through a linear transformation, and then mapped to a value vector through another linear transformation. In the attention layer of the U-Net decoder, the attention weights between the query vector and the standard key matrix are calculated; Simultaneously calculate the attention weights of the query vector and the identity key vector; The two attention weights are weighted and fused, as expressed by: Applying the fusion attention weight A to the standard value matrix γ and the identity value vector, the modulated feature is output as follows: h=OFF+γOff id ; Where γ is the learnable fusion coefficient; The modulated feature h is used as the output of the current layer for subsequent feature transfer.

8. The identity and person image enhancement system based on the diffusion model as described in claim 7, characterized in that: During the operation of the diffusion denoising execution module, the model parameters of the identity embedding extraction module and the saliency map generation module, on which it depends, remain frozen during the system training phase, specifically including: During the end-to-end training phase of the system, the weight parameters of the face recognition model and the semantic segmentation network are fixed, and only the learnable parameters and guidance coefficients in the U-Net backbone network and the attention modulation module are optimized. Training is performed using a joint loss function, expressed as follows: L=L rec +αL id +βL att ; Among them, L rec To reduce noise and reconstruct loss, L i For identity consistency loss, L att The loss is the attention distribution alignment loss, and α and β are the balance coefficients.

9. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the identity and person image enhancement system based on the diffusion model as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the identity and person image enhancement system based on the diffusion model as described in any one of claims 1 to 8.

Images