High-quality portrait generation method and system based on facial semantic priori

Abstract

The invention discloses a high-quality portrait generation method and system based on facial semantic priori. The method comprises the following steps: constructing a diffusion model taking a facial semantic segmentation map as a condition, embedding semantic constraints of a facial component into a generation process through a cross attention mechanism, and fusing global identity features by using a self-attention mechanism to ensure identity consistency; the semantic segmentation map correction module ensures the accuracy of semantic priori; enhancing expression mixed shape data to improve the generalization ability of the model for complex and unseen expressions; and a selective training strategy driven by local semantic loss is adopted, so that the detail fidelity of the generated image is remarkably improved. The method effectively solves the problems of inaccurate expression control, poor identity consistency, lack of local details, insufficient generalization ability for new expressions and the like in the existing portrait generation technology, and can generate digital portraits with high fidelity, accurate expressions and consistent identities.

Description

Technical Field

[0001] This invention belongs to the field of human image generation technology in computer vision, and in particular, it is a high-quality human image generation method and system based on facial semantic prior. Background Technology

[0002] Digital portrait generation technology is a computer image processing technique that can generate portraits of people with different identities and expressions. With the rise of virtual reality, digital humans, film and television special effects, high-quality, high-fidelity digital portrait generation technology has shown great application potential. Existing digital portrait generation methods can be mainly divided into methods based on 3D modeling and methods based on 2D generation models.

[0003] 3D model-based methods, such as 3D deformable models and their subsequent improvements, generate faces by controlling the linear combination of facial shape and texture through parameters. However, these methods are limited by the scale and quality of the training data, making it difficult to capture subtle high-frequency details (such as wrinkles and pores), and they lack the ability to represent internal structures such as the eyes and mouth, often resulting in coarse or distorted results. Although new technologies such as Neural Radiation Field (NeRF) and Dynamic 3D Gaussian have improved reconstruction accuracy, they are still prone to mesh distortion or loss of local details when dealing with complex micro-expressions.

[0004] Methods based on 2D generative models, such as Generative Adversarial Networks (GANs) and Diffusion Models, have made significant progress in improving image quality. Diffusion models, in particular, have become the mainstream paradigm for current image generation. However, existing methods of this type have the following limitations: (1) Diffusion models are optimized for diversity, which is inherently in conflict with the identity consistency requirements for human portrait generation; (2) For complex and extreme expressions not present in the training data, the generation quality drops significantly; (3) Because the 3D deformable models on which they are relied upon lack precise definitions of the eye and mouth regions, the generation results for these key regions remain poor.

[0005] Therefore, there is an urgent need in this field for a new method for human portrait generation that can simultaneously guarantee high generation quality, precise expression control, strict identity consistency, and strong generalization ability. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings and needs of the existing technologies by providing a high-quality portrait generation method based on facial semantic priors. This method effectively solves problems such as inaccurate facial expressions, missing details, inconsistent identities, and weak generalization ability for new expressions by introducing facial semantic segmentation maps as strong structure priors and combining them with innovative data augmentation and training strategies.

[0007] The technical solution to achieve the objective of this invention is as follows: On the one hand, a high-quality portrait generation method based on facial semantic prior is provided, the method comprising the following steps:

[0008] Step 1: Construct the input dataset, in which each sample includes the source portrait image and its corresponding physical parameters and facial semantic segmentation map;

[0009] Step 2: Build and train the first diffusion model, while freezing the parameters of the trained model.

[0010] Step 3: Perform data augmentation on the input dataset based on the expression blending shape;

[0011] Step 4: Correct the facial semantic segmentation map to ensure that the facial component categories are complete and the positions are accurate;

[0012] Step 5: Based on the first diffusion model, construct and train the main model;

[0013] Step 6: Input the source portrait image to be processed into the trained main model and output the corresponding high-quality portrait image.

[0014] Furthermore, the method for obtaining the facial semantic segmentation map in step 1 is as follows: a pre-trained BiSeNet model is used to generate a corresponding facial semantic segmentation map for each source portrait image.

[0015] Furthermore, step 2 involves constructing and training the first diffusion model, specifically including: training the basic diffusion model using the existing face dataset to generate the first diffusion model.

[0016] Furthermore, step 3 involves data augmentation of the input dataset based on the expression blending shape, specifically including:

[0017] Step 3-1: Construct a model of diverse composite expressions, and generate diverse composite expression data related to identity based on this model; wherein, the model of diverse composite expressions is:

[0018]

[0019] In the formula, For models of diverse and complex facial expressions, A model for neutral facial expressions. A hybrid shape model based on basic facial expressions. The offset of the k-th basic expression, i.e. and The difference, for The corresponding mixing coefficient;

[0020] Step 3-2: Render the diverse composite expression data to generate portrait images with new composite expressions;

[0021] Step 3-3: Obtain the physical parameters and facial semantic segmentation map corresponding to the portrait image generated in Step 3-2;

[0022] Step 3-4: Add the data from Step 3-3 to the input dataset to augment the input dataset.

[0023] Furthermore, in step 3-2, the diverse composite facial expression data is rendered using linear blending skinning calculation and 3D Gaussian splashing.

[0024] Furthermore, step 4 refines the facial semantic segmentation map to ensure complete facial component categories and accurate positioning, specifically including:

[0025] Step 4-1: Detect whether the facial semantic segmentation map contains all predefined facial component categories and whether there are any facial component regions that are misclassified. If both are not true, proceed to step 5; otherwise, extract the regions where facial components are missing or misclassified.

[0026] Step 4-2: For areas where facial components are missing or misclassified, use a facial landmark detection algorithm to obtain the contour coordinates of the facial components that should be present in that area.

[0027] Step 4-3, based on the contour coordinates, corrects different facial components, specifically including:

[0028] For the eyebrow and nose components, polygon filling is performed using contour coordinates to correct the components;

[0029] For the eye and mouth components, execute:

[0030] (1) Calculate the aspect ratio EAR of the component;

[0031] (2) By comparing the aspect ratio EAR with the preset EAR threshold, the open and closed state of the component is determined: if the EAR is lower than the preset EAR threshold, the component is determined to be in the closed state, and then (3) is executed; otherwise, the component is determined to be in the open state, and then (4) is executed.

[0032] (3) Calculate the shrinkage ratio of the component key point coordinates dynamically based on the EAR value, adjust the key point coordinates according to the shrinkage ratio, and then perform polygon filling to correct the component.

[0033] (4) Use the convex hull algorithm to correct the component shape;

[0034] Step 4-4: By replacing pixels, the area containing the corrected facial components is integrated back into the original facial semantic segmentation map to obtain the corrected facial semantic segmentation map.

[0035] Furthermore, the adjustment of the keypoint coordinates according to this shrinkage ratio is specifically calculated using the following formula:

[0036]

[0037] In the formula, and These represent the original keypoint coordinates and the adjusted keypoint coordinates, respectively. Indicates the geometric center of the component. Represents the dynamic proportional contraction function:

[0038]

[0039] In the formula, and These are the minimum and maximum shrinkage rates, respectively. The preset EAR threshold.

[0040] Furthermore, in step 5, based on the first diffusion model, a main model is constructed. The specific process includes:

[0041] Step 5-1: Construct the second diffusion model in the same manner as in Step 2;

[0042] Step 5-2: Combine the parameters of the first diffusion model from Step 2 with the parameters of the second diffusion model to form the initial main model;

[0043] Step 5-3: Input the semantic segmentation map corrected in Step 3 as a control condition into the initial main model;

[0044] Step 5-4: Add a cross-attention mechanism to the initial subject model's Unet to make the generated portrait image conform to the semantic layout defined by the facial semantic segmentation map.

[0045] Step 5-5: Extract the global identity code of the source portrait image and enhance it through a self-attention mechanism. Then, inject the enhanced identity code into the subject model formed in step 5-4 to form the final subject model.

[0046] Furthermore, step 5 involves training the main model, specifically including the following process:

[0047] The source portrait image is input into the subject model, and the semantic loss between the image generated by the subject model and the source portrait image in predefined key regions is calculated.

[0048] During backpropagation, the gradients of non-critical regions are frozen, and only the model parameters related to critical regions are updated, or the semantic loss of critical regions is given higher weight.

[0049] On the other hand, a high-quality portrait generation system based on facial semantic priors is provided, the system comprising:

[0050] The first module is used to construct the input dataset, in which each sample includes the source human image and its corresponding physical parameters and facial semantic segmentation map;

[0051] The second module is used to: build and train the first diffusion model, while freezing the parameters of the trained model;

[0052] The third module is used to perform data augmentation on the input dataset based on the facial expression blending shape.

[0053] The fourth module is used to correct the facial semantic segmentation map to ensure that the facial component categories are complete and the positions are accurate.

[0054] The fifth module is used to: construct and train the main model based on the first diffusion model;

[0055] The sixth module is used to input the source portrait image to be processed into the trained main model and output the corresponding high-quality portrait image.

[0056] Compared with the prior art, the significant advantages of this invention are:

[0057] (1) By introducing prior and global identity encoding of facial semantic segmentation map, the generated image has higher generation quality and stronger identity consistency.

[0058] (2) By using facial expression hybrid shape-driven data augmentation and combining it with selective training strategies, the model can accurately reproduce complex and extreme facial expressions and has good generalization ability for uncommon facial expressions.

[0059] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0060] Figure 1 This is a schematic diagram of a high-quality portrait generation method based on facial semantic priors in one embodiment.

[0061] Figure 2 This is an overall framework diagram of a high-quality portrait generation method based on facial semantic priors in one embodiment.

[0062] Figure 3 This is a schematic diagram of the principle framework for hybrid shape data augmentation in one embodiment.

[0063] Figure 4 This is a schematic diagram of the semantic segmentation graph correction principle in one embodiment.

[0064] Figure 5This is a framework diagram of selective training in one embodiment.

[0065] Figure 6 This is a comparison chart of the experimental results of the present invention with existing advanced methods in one embodiment. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0067] It should be noted that if the embodiments of the present invention involve descriptions such as "first" and "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include at least one of those features. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0068] In one embodiment, combined Figure 1 and Figure 2 This paper presents a high-quality portrait generation method based on facial semantic priors. The method includes: constructing a diffusion model conditioned on facial semantic segmentation maps; embedding semantic constraints of facial components into the generation process through a cross-attention mechanism; and fusing global identity features using a self-attention mechanism to ensure identity consistency. A semantic segmentation map correction module ensures the accuracy of semantic priors. Expression mixing shape data enhancement is used to improve the model's generalization ability to complex and unseen expressions. A selective training strategy driven by local semantic loss is adopted to significantly improve the detail fidelity of the generated images.

[0069] The method includes the following steps:

[0070] Step 1: Construct the input dataset, in which each sample includes the source portrait image and its corresponding physical parameters and facial semantic segmentation map;

[0071] Step 2: Build and train the first diffusion model, while freezing the parameters of the trained model.

[0072] Step 3: Perform data augmentation on the input dataset based on the expression blending shape;

[0073] Step 4: Correct the facial semantic segmentation map to ensure that the facial component categories are complete and the positions are accurate;

[0074] Step 5: Based on the first diffusion model, construct and train the main model;

[0075] Step 6: Input the source portrait image to be processed into the trained main model and output the corresponding high-quality portrait image.

[0076] Furthermore, in one embodiment, the facial semantic segmentation map is obtained in step 1 by generating a corresponding facial semantic segmentation map for each source portrait image using a pre-trained BiSeNet model. Here, the input data of the BiSeNet model consists of some source portrait images and their corresponding facial semantic segmentation maps.

[0077] Furthermore, in one embodiment, step 2, which involves building and training a first diffusion model, specifically includes: training a basic diffusion model using an existing face dataset to generate the first diffusion model.

[0078] Preferably, but not limited to, the FFHQ face dataset is used here.

[0079] Furthermore, in one embodiment, combined with Figure 3 Step 3 involves data augmentation of the input dataset based on the expression blending shape, specifically including:

[0080] Step 3-1: Construct a model of diverse composite expressions, and generate diverse composite expression data related to identity based on this model; wherein, the model of diverse composite expressions is:

[0081]

[0082] In the formula, For models of diverse and complex facial expressions, A model for neutral facial expressions. A hybrid shape model based on basic facial expressions. The offset of the k-th basic expression, i.e. and The difference, for The corresponding mixing coefficient;

[0083] here, , , All of these are models based on Gaussian distribution representation, containing all the basic properties of Gaussian distribution;

[0084] Step 3-2: Render the diverse composite expression data to generate portrait images with new composite expressions;

[0085] Step 3-3: Obtain the physical parameters and facial semantic segmentation map corresponding to the portrait image generated in Step 3-2;

[0086] Step 3-4: Add the data from Step 3-3 to the input dataset to augment the input dataset.

[0087] Preferably, in some embodiments, in step 3-2, the diverse composite facial expression data is rendered by linear blending skinning calculation and 3D Gaussian splashing.

[0088] Furthermore, in one embodiment, combined with Figure 4 Step 4 involves refining the facial semantic segmentation map to ensure complete facial component categories and accurate positioning. Specifically, this includes:

[0089] Step 4-1: Detect whether the facial semantic segmentation map contains all predefined facial component categories and whether there are any facial component regions that are misclassified. If both are not true, proceed to step 5; otherwise, extract the regions where facial components are missing or misclassified.

[0090] Step 4-2: For areas where facial components are missing or misclassified, use a facial landmark detection algorithm to obtain the contour coordinates of the facial components that should be present in that area.

[0091] Step 4-3, based on the contour coordinates, corrects different facial components, specifically including:

[0092] For the eyebrow and nose components, polygon filling is performed using contour coordinates to correct the components;

[0093] For the eye and mouth components, execute:

[0094] (1) Calculate the aspect ratio EAR of the component;

[0095] (2) By comparing the aspect ratio EAR with the preset EAR threshold, the open and closed state of the component is determined: if the EAR is lower than the preset EAR threshold, the component is determined to be in the closed state, and then (3) is executed; otherwise, the component is determined to be in the open state, and then (4) is executed.

[0096] (3) Calculate the shrinkage ratio of the component key point coordinates dynamically based on the EAR value, adjust the key point coordinates according to the shrinkage ratio, and then perform polygon filling to correct the component.

[0097] (4) Use the convex hull algorithm to correct the component shape;

[0098] Step 4-4: By replacing pixels, the area containing the corrected facial components is integrated back into the original facial semantic segmentation map to obtain the corrected facial semantic segmentation map.

[0099] Preferably, in some embodiments, the adjustment of the key point coordinates according to the shrinkage ratio is specifically calculated using the following formula:

[0100]

[0101] In the formula, and These represent the original keypoint coordinates and the adjusted keypoint coordinates, respectively. Indicates the geometric center of the component. Represents the dynamic proportional contraction function:

[0102]

[0103] In the formula, and These are the minimum and maximum shrinkage rates, respectively. The preset EAR threshold.

[0104] Furthermore, in step 5, based on the first diffusion model, a main model is constructed. The specific process includes:

[0105] Step 5-1: Construct the second diffusion model in the same manner as in Step 2;

[0106] Step 5-2: Combine the parameters of the first diffusion model from Step 2 with the parameters of the second diffusion model to form the initial main model;

[0107] Step 5-3: Input the semantic segmentation map corrected in Step 3 as a control condition into the initial main model;

[0108] Step 5-4: Add a cross-attention mechanism to the initial subject model's Unet (encode the facial semantic segmentation map as keys (K) and values ​​(V), use the noise latent features of the intermediate layer of the subject model as queries (Q), compute the cross-attention map and apply it to the values ​​(V) to map the semantic space constraints to the generation space) so that the generated portrait image conforms to the semantic layout defined by the facial semantic segmentation map;

[0109] Step 5-5: Extract the global identity code of the source portrait image and enhance it through a self-attention mechanism. Then, inject the enhanced identity code into the main body model formed in step 5-4 to ensure the consistency of the identity of the generated portrait and form the final main body model.

[0110] Furthermore, step 5 involves training the main model, specifically including the following process:

[0111] The source portrait image is input into the subject model, and the semantic loss of the image generated by the subject model compared with the source portrait image in predefined key regions (including eyes and mouth) is calculated.

[0112] Combination Figure 5 During backpropagation, images generated by the main model with lower EE scores are selected, the gradients of non-critical regions are frozen, and only the model parameters related to the critical regions are updated (reducing the model's updates to these regions, randomly selecting several regions for the critical parts, reassigning their semantic categories, and guiding the model to focus on training the critical regions), or the semantic loss of the critical regions is given higher weights to force the model to focus on learning the detailed features of the critical regions.

[0113] Here, to quantify the accuracy of facial expressions in the generated images, Expression Error (EE) is introduced as an evaluation metric. This metric is obtained by extracting facial expression coefficients from both the generated image and the corresponding real image using the DECA model, and then calculating the difference between the two sets of coefficients. The lower the EE value, the closer the generated image's expression is to the target expression.

[0114] In one embodiment, a high-quality portrait generation system based on facial semantic priors is provided, the system comprising:

[0115] The first module is used to construct the input dataset, in which each sample includes the source human image and its corresponding physical parameters and facial semantic segmentation map;

[0116] The second module is used to: build and train the first diffusion model, while freezing the parameters of the trained model;

[0117] The third module is used to perform data augmentation on the input dataset based on the facial expression blending shape.

[0118] The fourth module is used to correct the facial semantic segmentation map to ensure that the facial component categories are complete and the positions are accurate.

[0119] The fifth module is used to: construct and train the main model based on the first diffusion model;

[0120] The sixth module is used to input the source portrait image to be processed into the trained main model and output the corresponding high-quality portrait image.

[0121] Specific limitations regarding the high-quality portrait generation system based on facial semantic priors can be found in the limitations of the high-quality portrait generation method based on facial semantic priors mentioned above, and will not be repeated here. Each module in the aforementioned high-quality portrait generation system based on facial semantic priors can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0122] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements:

[0123] Step 1: Construct the input dataset, in which each sample includes the source portrait image and its corresponding physical parameters and facial semantic segmentation map;

[0124] Step 2: Build and train the first diffusion model, while freezing the parameters of the trained model.

[0125] Step 3: Perform data augmentation on the input dataset based on the expression blending shape;

[0126] Step 4: Correct the facial semantic segmentation map to ensure that the facial component categories are complete and the positions are accurate;

[0127] Step 5: Based on the first diffusion model, construct and train the main model;

[0128] Step 6: Input the source portrait image to be processed into the trained main model and output the corresponding high-quality portrait image.

[0129] For specific limitations on each step, please refer to the limitations on high-quality portrait generation methods based on facial semantic priors mentioned above, which will not be repeated here.

[0130] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program being implemented when executed by a processor:

[0131] Step 1: Construct the input dataset, in which each sample includes the source portrait image and its corresponding physical parameters and facial semantic segmentation map;

[0132] Step 2: Build and train the first diffusion model, while freezing the parameters of the trained model.

[0133] Step 3: Perform data augmentation on the input dataset based on the expression blending shape;

[0134] Step 4: Correct the facial semantic segmentation map to ensure that the facial component categories are complete and the positions are accurate;

[0135] Step 5: Based on the first diffusion model, construct and train the main model;

[0136] Step 6: Input the source portrait image to be processed into the trained main model and output the corresponding high-quality portrait image.

[0137] For specific limitations on each step, please refer to the limitations on high-quality portrait generation methods based on facial semantic priors mentioned above, which will not be repeated here.

[0138] As a specific example, qualitative and quantitative experiments were conducted to verify the effectiveness of the invention. The experimental results fully demonstrate that the present invention achieves better results in the field of portrait generation compared to existing generation methods. Qualitative generation results are compared to... Figure 6 As shown, the first column represents real images, the second column represents images generated by this invention, and columns 3 to 5 represent images generated by other current advanced methods. It can be seen that the portraits generated by this invention are superior in terms of facial expression accuracy and detail richness (especially in the eyes and the inside of the mouth).

[0139] The quantitative results are shown in Table 1. The dataset used was the average of the results from publicly available datasets in the IMAvatar, INSTA, and NeRF hybrid shapes. A total of six evaluation metrics were set, of which L2 loss, PSNR, SSIM, and LPIPS were used to evaluate the quality of the generated images. and These represent the errors in expression coefficients between the generated results and the real images for standard and complex expressions, respectively, and are used to evaluate the accuracy of the generated portrait's expression. (Symbols) The higher the value, the better the generated result; conversely, the lower the value, the better the generated result.

[0140] Table 1 Comparison of evaluation results for portrait generation using different methods

[0141]

[0142] The results show that the four image quality indicators generated by the present invention are all higher than those of existing advanced methods, while having a smaller difference in expression error.

[0143] In summary, this invention effectively solves the problems of imprecise expression control, poor identity consistency, lack of local details, and insufficient generalization ability for new expressions in existing portrait generation technologies, and can generate high-fidelity digital portraits with accurate expressions and consistent identities.

[0144] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention without departing from its spirit and scope should be included within the protection scope of the present invention.

Claims

1. A high-quality portrait generation method based on facial semantic prior, characterized in that, The method comprises the following steps: Step 1, constructing an input data set, each sample in the data set comprising a source portrait image and corresponding physical parameters and a face semantic segmentation map; Step 2, constructing and training a first diffusion model while freezing the parameters of the trained model; Step 3, performing data enhancement on the input data set based on expression mixed shapes; Step 4, correcting the face semantic segmentation map to make the face component classes complete and the positions accurate; Step 5, based on the first diffusion model, constructing and training a main model; Step 6, inputting a source portrait image to be processed into the trained main model to output a corresponding high-quality portrait image.

2. The method of claim 1, wherein, In step 1, the face semantic segmentation map is obtained by using a pre-trained BiSeNet model to generate a corresponding face semantic segmentation map for each source portrait image.

3. The method of claim 1, wherein, In step 2, the first diffusion model is constructed and trained, specifically including: using existing face data sets to train a basic diffusion model to generate the first diffusion model.

4. The method for generating high-quality portrait according to facial semantic priori of claim 1, characterized in that, In step 3, the input data set is enhanced based on expression mixed shapes, specifically including: Step 3-1, constructing a model of diversified composite expressions, and generating diversified composite expression data related to identity based on the model; wherein the model of diversified composite expressions is: ； wherein, is a model of a diversified composite expression, is a model of a neutral expression, is a mixed shape model of a base expression, is an offset amount of the kth base expression, i.e. is a difference value of is a corresponding mixing coefficient;​ Step 3-2, rendering the diversified composite expression data to generate portrait images with new composite expressions; Step 3-3, obtaining the physical parameters and face semantic segmentation map corresponding to the portrait images generated in step 3-2; Step 3-4, adding the data of step 3-3 to the input data set to achieve data enhancement of the input data set.

5. The method for generating high-quality portrait according to facial semantic priori based on claim 4, characterized in that, In step 3-2, the diversified composite expression data is rendered by linearly mixing the skin and calculating 3D Gaussian splashes.

6. The method for high-quality portrait generation based on facial semantic prior according to claim 1, characterized in that, In step 4, the face semantic segmentation map is corrected to make the face component classes complete and the positions accurate, specifically including: Step 4-1, detecting whether the face semantic segmentation map contains all predefined face component classes and whether there is a classification error in the face component region, if both are no, then skip to step 5, otherwise, extract the region where the face component is missing or the classification is wrong; Step 4-2, for the region where the face component is missing or the classification is wrong, using a face key point detection algorithm to obtain the contour coordinates of the face component that should be in the region; Step 4-3, based on the contour coordinates, correcting different face components, specifically including: For eyebrow and nose components, using contour coordinates for polygon filling to correct the components; For eye and mouth components, performing: (1) calculating the aspect ratio EAR of the component; (2) comparing the aspect ratio EAR with a preset EAR threshold to determine the opening and closing state of the component: if the EAR is lower than the preset EAR threshold, it is determined that the component is in a closed state, and then step (3) is performed, otherwise, it is determined that the component is in an open state, and then step (4) is performed; (3) dynamically calculating the shrinkage ratio of the key point coordinates of the component according to the EAR value, adjusting the key point coordinates according to the shrinkage ratio, and then performing polygon filling to correct the component; (4) using a convex hull algorithm to correct the shape of the component; Step 4-4, the corrected face component region is integrated back to the original face semantic segmentation map by pixel replacement to obtain a corrected face semantic segmentation map.

7. The method of claim 6, wherein, The key point coordinates are adjusted according to the shrinkage ratio, and the specific formula is: ； wherein, and denote the original keypoint coordinates and the adjusted keypoint coordinates, respectively, denotes the component geometric center, denotes a dynamic scale shrink function: ； wherein and are the minimum and maximum shrinkage, respectively, is the preset EAR threshold.

8. The method for high-quality portrait generation based on facial semantic prior according to claim 1, characterized in that, In step 5, the main model is constructed based on the first diffusion model, and the specific process includes: Step 5-1, a second diffusion model is constructed in the manner of step 2; Step 5-2, the parameters of the first diffusion model of step 2 and the parameters of the second diffusion model are spliced to form an initial main model; Step 5-3, the corrected semantic segmentation map of step 3 is input into the initial main model as a control condition; Step 5-4, cross-attention mechanism is added to the Unet of the initial main model to make the generated portrait image conform to the semantic layout defined by the face semantic segmentation map; Step 5-5, the global identity code of the source portrait image is extracted and strengthened through self-attention mechanism, and then the strengthened identity code is injected into the main model formed in step 5-4 to form a final main model.

9. The method for generating high-quality portrait according to facial semantic priori based on claim 8, characterized in that, In step 5, the main model is trained, and the specific process includes: The source portrait image is input into the main model, and the semantic loss of the image generated by the main model and the source portrait image in the pre-defined key area is calculated; During back propagation, the gradient of the non-key area is frozen, only the model parameters related to the key area are updated, or the key area semantic loss is given a higher weight.

10. A high-quality portrait generation system based on facial semantic priors based on the method of any one of claims 1 to 9, characterized in that, The system includes: A first module for implementing: constructing an input data set, each sample in the data set including a source portrait image and its corresponding physical parameters and face semantic segmentation map; A second module for implementing: constructing and training a first diffusion model, while freezing the parameters of the trained model; A third module for implementing: data augmentation on the input data set based on expression mixed shape; A fourth module for implementing: correcting the face semantic segmentation map to make the face component class complete and the position accurate; A fifth module for implementing: constructing and training a main model based on the first diffusion model; A sixth module for implementing: inputting the source portrait image to be processed into the trained main model to output a corresponding high-quality portrait image.

Images