Face fusion preprocessing method and system based on hair repair and lightening skin beautification

Abstract

The present application provides a kind of face fusion pre-processing method and system based on hair repair and light weight beautifying face, it is related to face image pre-processing and computer vision technical field;Core method is first to the original image is repaired to hair cover, obtains repair image and then is light weight beautifying and obtains beautifying image;Risk assessment and intervention closed loop, system calculates three key indicators: skin area non-natural smoothness index;Hair repair boundary structure distortion index;Skin color non-physiological deviation index;These indexes are combined with the risk degree of weight model solution;Based on the measurement, warning threshold and critical threshold, determine dynamic fusion coefficient, finally to repair image and beautifying image are weighted and fused to generate final image;The method introduces boundary structure distortion index, solves the problem of lack of collaborative control between repair and beautification module, can dynamically adjust beautifying intensity, effectively prevent artifact superposition, improve the sense of reality and naturalness of output image.

Description

Technical Field

[0001] This invention relates to the fields of facial image preprocessing and computer vision technology, specifically to a facial fusion preprocessing method and system based on hair restoration and lightweight skin beautification. Background Technology

[0002] This technical solution involves the image preprocessing stage before applications such as face fusion or virtual clothing swapping. Its core objective is to enhance the aesthetic quality of the image while ensuring the realism, naturalness, and cultural appropriateness of the output image. Existing technologies often treat hair-covered area repair and facial beautification as isolated steps in a series process. This processing mechanism has the following key technical defects and challenges:

[0003] The lack of coordinated control between the restoration and enhancement modules: Lightweight skin beautification directly affects the restored image, easily causing secondary damage to the previously accurately reconstructed hairline boundary structure, such as blurring or distortion, leading to artifact superposition and making it difficult to ensure the coordination and consistency between the two modules; Lack of quantification and control over the risk of over-enhancement: If enhancement processing is unrestrained, it is easy to introduce unnatural distortions, such as excessive loss of skin texture details, resulting in unnatural smoothness of skin areas and excessive skin color shift, exceeding the natural range acceptable to human physiology, causing potential unrealistic or culturally uncomfortable feelings; Lack of adaptive safety degradation logic: Existing processing solutions mostly use fixed parameters, making it difficult to dynamically and finely adjust the intensity of the enhancement effect according to the potential restoration and enhancement risks of the image; Once the risk is too high, the quality and naturalness of the processing result will be significantly reduced, affecting the robustness of the system;

[0004] Therefore, how to construct a closed-loop mechanism for risk assessment and dynamic intervention that includes multi-dimensional indicators, proactively identify and quantify risks that may be introduced by beautification processing, such as unnatural smoothing, boundary structure distortion, and skin color deviation, and dynamically adjust the intensity of beautification effect based on this risk metric, so as to ensure that the output image is both beautiful and realistic while achieving efficient aesthetic enhancement, has become an urgent technical problem to be solved. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a face fusion preprocessing method and system based on hair repair and lightweight skin beautification. Specifically, the technical solution of this invention is as follows:

[0006] A face fusion preprocessing method based on hair repair and lightweight skin beautification includes:

[0007] The original input image is processed to repair the hair-occluded areas, and the repaired image is obtained.

[0008] Lightweight skin smoothing is applied to the repaired image to obtain a beautified image;

[0009] Based on the variance between the beautified image and the preset reference, the unnatural smoothness index of the skin region is determined.

[0010] Based on the hairline boundary region of the restored and beautified images, the hair restoration boundary structure distortion index is determined;

[0011] Based on the color histogram of the beautified image and the preset spatial reference probability distribution of natural skin color, the non-physiological deviation index of skin color is determined.

[0012] By combining the indicators of unnatural smoothness of skin regions, distortion of hair repair boundary structure, and non-physiological deviation of skin color, a risk measure is calculated through a pre-set weighted model.

[0013] The dynamic fusion coefficient is determined based on risk measurement, preset warning threshold, and preset critical threshold.

[0014] Based on the dynamic fusion coefficient, the repaired image and the beautified image are weighted and fused to generate the final image.

[0015] Preferably, determining the dynamic fusion coefficient includes:

[0016] When the risk metric is less than the preset warning threshold, the dynamic fusion coefficient is set to the preset maximum value;

[0017] When the risk metric is greater than or equal to the preset warning threshold and less than or equal to the preset critical threshold, the dynamic fusion coefficient is linearly reduced from the preset maximum value to zero based on the risk metric.

[0018] When the risk metric exceeds the preset critical threshold, the dynamic fusion coefficient is set to zero.

[0019] Preferably, the hair-covered area repair process is achieved by using a pre-defined generative adversarial network;

[0020] Among them, pixel loss and edge loss are combined to form a composite loss function, which is used to optimize and constrain the generative adversarial network.

[0021] The preferred lightweight skincare is achieved by using a pre-defined lightweight student network;

[0022] Among them, knowledge distillation technology is used to guide the training of lightweight student networks using a pre-set teacher network.

[0023] Preferably, the knowledge distillation technique utilizes soft label loss and feature map alignment loss for constraint.

[0024] Preferably, the indicators for determining the unnatural smoothness of skin areas include:

[0025] Calculate the texture variance of the skin region in the beautified image and compare it with a preset reference variance.

[0026] A face fusion preprocessing system based on hair restoration and lightweight skin beautification includes:

[0027] The hair-occluded area repair module is used to repair the hair-occluded areas of the original input image and obtain the repaired image.

[0028] The lightweight skin-smoothing module is used to perform lightweight skin-smoothing on repaired images to obtain beautified images;

[0029] The smoothness index determination module is used to determine the unnatural smoothness index of the skin area based on the beautified image and the preset reference variance.

[0030] The distortion index determination module is used to determine the distortion index of the hair restoration boundary structure based on the hairline boundary region between the restored image and the beautified image.

[0031] The deviation index determination module is used to determine non-physiological deviation indexes of skin color based on the color histogram of the beautified image and the preset spatial reference probability distribution of natural skin color.

[0032] The risk measurement and calculation module is used to combine the skin region non-natural smoothness index, hair repair boundary structure distortion index, and skin color non-physiological deviation index, and calculate the risk measurement through a preset weighted model.

[0033] The coefficient generation module is used to determine the dynamic fusion coefficient based on risk measurement, preset warning threshold, and preset critical threshold.

[0034] The image synthesis module is used to perform weighted fusion of the repaired image and the beautified image based on dynamic fusion coefficients to generate the final image.

[0035] Preferably, the coefficient generation module is configured as follows:

[0036] When the risk metric is less than the preset warning threshold, the dynamic fusion coefficient is set to the preset maximum value;

[0037] When the risk metric is greater than or equal to the preset warning threshold and less than or equal to the preset critical threshold, the dynamic fusion coefficient is linearly reduced from the preset maximum value to zero based on the risk metric.

[0038] When the risk metric exceeds the preset critical threshold, the dynamic fusion coefficient is set to zero.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. By introducing a hair restoration boundary structure distortion index, this method solves the technical problem of lack of coordinated control between the restoration and beautification modules. This index can actively assess the degree of secondary damage caused by the skin beautification process to the previously accurately reconstructed hairline boundary structure. Based on this assessment result, the system can dynamically adjust the intensity of the beautification effect, effectively prevent artifact superposition, and ensure that the restored hairline area and the beautified skin area are consistent, greatly improving the realism and naturalness of the output image.

[0041] 2. This method constructs a multi-dimensional risk assessment mechanism that includes indicators of unnatural skin smoothness and non-physiological deviation of skin color. By solving these three indicators into a unified risk metric, and designing a segmented dynamic fusion coefficient based on this metric, this adaptive intervention loop can smoothly and gradually reduce the beautification effect as the risk metric increases. It avoids the one-size-fits-all approach with fixed parameters, maximizes the aesthetic effect under the premise of controllable risk, and decisively disables beautification when the risk is too high, ensuring the bottom-line safety and robustness of the output results.

[0042] 3. By introducing a non-physiological deviation index for skin color and calculating it based on the natural skin color spatial reference probability distribution, this method can effectively ensure that the beautified skin color remains within the range of natural skin color that is physiologically acceptable to humans. This mechanism actively avoids unrealistic or strange tones that may result from beautification processing, thereby eliminating the risk of images causing potential cultural discomfort. This is not only precise control in terms of technology, but also improves the universality and user acceptance of the product at the application level.

[0043] 4. In terms of specific implementation, this solution uses generative adversarial networks to repair hair-occluded areas with high quality, ensuring the fine depiction of key structures such as the hairline; at the same time, lightweight skin smoothing adopts a lightweight student network based on knowledge distillation technology, which significantly reduces the amount of computation while achieving skin smoothing effects similar to complex models; these optimizations significantly improve the efficiency and accuracy of the entire preprocessing process and the deployment advantages on computing-restricted devices. Attached Figure Description

[0044] The present invention will be further explained below with reference to the accompanying drawings and embodiments:

[0045] Figure 1 This is a flowchart of the method of the present invention.

[0046] Figure 2 This is a structural diagram of the system of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0048] Example 1:

[0049] Please see Figure 1 A face fusion preprocessing method based on hair repair and lightweight skin beautification includes:

[0050] The original input image is processed to repair the hair-occluded areas, and the repaired image is obtained.

[0051] Lightweight skin smoothing is applied to the repaired image to obtain a beautified image;

[0052] Based on the variance between the beautified image and the preset reference, the unnatural smoothness index of the skin region is determined.

[0053] Based on the hairline boundary region of the restored and beautified images, the hair restoration boundary structure distortion index is determined;

[0054] Based on the color histogram of the beautified image and the preset spatial reference probability distribution of natural skin color, the non-physiological deviation index of skin color is determined.

[0055] By combining the indicators of unnatural smoothness of skin regions, distortion of hair repair boundary structure, and non-physiological deviation of skin color, a risk measure is calculated through a pre-set weighted model.

[0056] The dynamic fusion coefficient is determined based on risk measurement, preset warning threshold, and preset critical threshold.

[0057] Based on the dynamic fusion coefficient, the repaired image and the beautified image are weighted and fused to generate the final image.

[0058] This embodiment provides a face fusion preprocessing method based on hair restoration and lightweight skin beautification. The method aims to perform a series of automated processing on the input original image before face fusion or virtual dressing applications, repair the information loss caused by hair occlusion, and appropriately improve the aesthetic quality of the image. Its core lies in introducing a risk assessment and dynamic intervention mechanism to ensure that the final generated image is both beautiful and realistic and culturally appropriate, avoiding the generation of unnatural or offensive artifacts.

[0059] In a specific implementation scenario, the method includes the following steps:

[0060] The original input image that is the object of processing Perform hair-covered areas repair to obtain repaired images. The purpose of this step is to accurately reconstruct facial areas, such as the forehead, that are obscured by hair, especially bangs; to repair the image. The areas previously obscured by hair in the central region have been restored, revealing a complete and natural facial structure;

[0061] Based on the restored image Perform lightweight skin beautification to obtain enhanced images. The purpose of this step is to provide a lightweight aesthetic enhancement to the repaired image without introducing significant distortion, such as smoothing skin and evening out skin tone; thus beautifying the image. While maintaining the characteristics of the characters, it has a better visual presentation;

[0062] The system enters the risk assessment phase, which quantifies the potential distortion risk introduced by the beautification process by calculating three key indicators.

[0063] Based on the variance between the beautified image and a preset reference, an index for the unnatural smoothness of the skin region is determined. The physical meaning of this indicator lies in quantifying whether excessive light-weight skin beautification leads to the loss of skin texture details, resulting in an unnatural appearance; its calculation is based on the beautified image. The texture variance of the mid-skin region is compared to a baseline value;

[0064] Based on the hairline boundary region of the restored and beautified images, the structural distortion index of the hair restoration boundary was determined. The physical meaning of this indicator is to assess whether the skin beautification process has caused secondary damage to the previously repaired hairline area, such as blurring or distortion, in order to evaluate the synergistic effect between the repair and beautification modules and prevent artifact superposition.

[0065] Based on the color histogram of the beautified image and the preset spatial reference probability distribution of natural skin tone, non-physiological deviation indicators of skin tone are determined. The physical significance of this indicator is to ensure that the beautified skin tone remains within the natural range acceptable to human physiology, avoiding unrealistic or strange tones, and thus controlling the color authenticity of the generated image.

[0066] Hair repair boundary structure distortion index The specific calculation method is as follows:

[0067] Determine the hairline boundary region; using facial landmark detection algorithms, such as deep learning-based landmark detection networks, locate the key points of the hairline, and extract the hairline contour upwards with a width of [missing information]. pixels, downward extraction width is The regions of pixels, respectively from the restored image and beautify images Extract the corresponding boundary region, denoted as . and ;

[0068] Edge extraction is performed on the boundary region; the Canny edge detection algorithm or the Sobel operator is used. and Perform edge detection to obtain an edge map. and In this embodiment, the Canny algorithm is used, with the low threshold set to 50 and the high threshold set to 150.

[0069] Calculate edge structure similarity; evaluate using the Structural Similarity Index (SSIM). and The structural consistency between them is calculated using the following formula:

[0070] ;

[0071] in, and They are respectively and The mean, and Their standard deviations are respectively. For covariance, and In this embodiment, the stability constant is used. , ;

[0072] Then, the distortion index is calculated; the distortion index of the hair repair boundary structure. Defined as:

[0073] ;

[0074] The value range of is [0, 1], where 0 indicates that the boundary structure is completely consistent and there is no distortion; 1 indicates that the boundary structure is completely different and there is severe distortion; when , When the value is close to 0, it indicates that the skin rejuvenation treatment did not damage the hairline boundary structure; when When the hairline is enlarged, it indicates that the skin rejuvenation treatment has caused secondary damage such as blurring and distortion.

[0075] Non-physiological deviation indicators of skin color The specific calculation method is as follows:

[0076] Color space conversion and histogram extraction; image enhancement The color space is converted from RGB to LAB, which better reflects human visual perception. A two-dimensional joint histogram of the a channel (red-green hue) and b channel (yellow-blue hue) of the skin region is extracted, denoted as... The histogram resolution is set to 32×32 bins;

[0077] Construct a spatial reference probability distribution for natural skin color; this reference distribution This data was obtained from 100,000 randomly selected natural face images from the CelebA public face dataset. Specifically, skin region segmentation was performed on each image, and the a and b channel values ​​in the LAB space were extracted. Their probability distribution over 32×32 bins was statistically analyzed. Finally, the distribution of the 100,000 images was averaged and normalized to obtain the final result. This reference distribution covers a wide range of natural skin colors across different races, genders, and ages, and is broadly representative.

[0078] Calculate the distribution deviation; use KL divergence to standardize the skin color distribution in the beautified image. Relative to the natural reference distribution The degree of deviation is calculated using the following formula:

[0079] ;

[0080] in, Represents the KL divergence, used to measure the distribution of skin tones in a beautified image. Natural skin color reference distribution The degree of deviation between them; and These are the bins indices for channel a and channel b, respectively. and These are the probability values ​​for the corresponding bins; to avoid numerical instability in logarithmic operations, when or When the value is 0, this term is not included in the summation;

[0081] Normalization to a distortion index; in order to make The range of values ​​for is convenient for threshold setting, and the following normalization formula is adopted:

[0082] ;

[0083] in, The normalization coefficient is the 99th percentile value of the KL divergence statistically obtained from the validation dataset. In this embodiment... After normalization, The value range of is approximately [0, 1], where 0 indicates that the skin color distribution is completely consistent with the natural distribution, and the larger the value, the more severe the deviation; when , A score exceeding 1 indicates that skin color has significantly deviated from the physiologically acceptable range for humans.

[0084] To establish a unified risk assessment, an index of non-natural smoothness in skin regions was incorporated. Hair repair boundary structure distortion index and non-physiological deviation indicators of skin color The risk metric is calculated using a pre-defined weighted model. Risk measurement It is a comprehensive reflection of image beautification The dimensionless scalar of overall distortion risk ranges from [0, +∞), with a larger value indicating a higher risk; this single scalar provides a clear decision-making basis for subsequent dynamic interventions.

[0085] The preset weighted model uses a linear weighted summation form, and the specific calculation formula is as follows:

[0086] ;

[0087] in, , and The weight coefficients for the three indicators are non-negative real numbers, obtained through multiple linear regression training on a training set containing 1000 manually labeled images. During training, each image was subjectively rated by five review experts on a distortion risk score of 0-10, and the average value was used as the supervision label. The optimal weight coefficients were then fitted using the least squares method. In this embodiment, the typical value of the weight coefficients determined through the above training process is... , , The relative magnitudes of the weighting coefficients reflect the degree to which different types of distortion affect user perception, among which... A larger value indicates that hairline boundary structure distortion has the most significant impact on overall visual quality. In other embodiments, the weighting coefficient can be adjusted according to the specific application scenario and the preferences of the target user group. For example, for applications that place more emphasis on skin tone realism, the weighting coefficient can be appropriately increased. The value of .

[0088] Based on the calculated risk metric Preset warning thresholds and preset critical threshold Determine the dynamic fusion coefficient The purpose of this step is to generate an adjustment coefficient to control the intensity of the beautification effect based on the comprehensive risk score calculated in the previous steps; dynamic fusion coefficient. It is a floating-point number whose value ranges from 0 to a preset maximum value; preset warning threshold. With preset critical threshold These are two key risk demarcation points used to divide the safe zone, warning zone, and danger zone; their determination method is as follows:

[0089] Construct a calibration dataset; select 500 restored and beautified face images, covering different risk metrics. Level, ranging from 0.1 to 2.0; 50 test users were invited to rate the naturalness and acceptability of each image. The rating criteria were: 9-10 points indicate completely natural and acceptable (safe zone), 6-8 points indicate minor flaws but acceptable (warning zone), and 0-5 points indicate obvious distortion and unacceptable (danger zone).

[0090] Statistical analysis was used to determine the threshold; the average subjective score was calculated for each image, and the subjective scores were plotted against the risk metric. A scatter plot; analysis revealed that when At that time, 95% of the images scored above 9 points, placing them in the safe zone; when At that time, the image score gradually decreased and entered the warning zone; when At that time, 80% of the images scored below 6 points, placing them in the danger zone;

[0091] Based on the above statistical analysis, in this embodiment, a preset warning threshold is used. Set to 0.3, preset critical threshold. The threshold is set to 0.7; in other embodiments, the threshold can be adjusted according to the quality requirements of the target application. For example, for application scenarios with more stringent requirements, the threshold can be... Reduce to 0.2, Reduce to 0.5; for scenarios where a higher level of enhancement can be tolerated, it can be... Increase to 0.4, Increased to 0.9.

[0092] To generate the final result, based on the dynamic fusion coefficient For image restoration and beautify images Weighted fusion is performed to generate the final image. The purpose of this step is to adaptively apply enhancement effects to the repaired image based on the results of the risk assessment; final image By safely restoring images And potentially risky beautified images The fusion ratio is obtained by linear interpolation and is determined by the dynamic fusion coefficient. Precise control; final image The specific calculation formula is as follows:

[0093] ;

[0094] in, This is the dynamic fusion coefficient, and its value range is [0, ...]. ];when hour, This means the output is a completely retouched image, without any enhancement effects. This carries the lowest risk but offers zero aesthetic improvement. When the beautification effect reaches its preset maximum intensity, the aesthetic enhancement is maximized, but it is necessary to ensure that the risks are controllable; when At that time, the output image is between the repaired image and the beautified image, achieving a balance between aesthetic enhancement and risk control; this weighted fusion strategy ensures a smooth transition of the final image at the pixel level, avoiding the abrupt changes in effect caused by directly using beautified or repaired images;

[0095] This invention constructs a complete processing-evaluation-intervention closed loop by linking hair restoration, lightweight skin beautification, and a risk assessment mechanism containing multi-dimensional indicators. Compared to existing technologies that treat restoration and beautification as isolated steps, this invention can proactively identify and quantify risks that may be introduced during the beautification process, such as unnatural smoothing, boundary structure distortion, and skin tone deviation, and dynamically adjust the intensity of the beautification effect based on this risk metric. This not only solves the problem of reconstructing areas obscured by hair and achieves efficient aesthetic enhancement, but more importantly, by introducing a safety degradation logic, it ensures the authenticity and naturalness of the output image, effectively avoiding artifacts and potential cultural discomfort caused by over-processing, and significantly improving the robustness and user acceptance of face fusion applications.

[0096] Example 2:

[0097] Determine the dynamic fusion coefficient, including setting the dynamic fusion coefficient to the preset maximum value when the risk metric is less than the preset warning threshold;

[0098] When the risk metric is greater than or equal to the preset warning threshold and less than or equal to the preset critical threshold, the dynamic fusion coefficient is linearly reduced from the preset maximum value to zero based on the risk metric.

[0099] When the risk metric exceeds the preset critical threshold, the dynamic fusion coefficient is set to zero.

[0100] This embodiment is a specific implementation of the step of determining the dynamic fusion coefficient in Embodiment 1; the core purpose of this design is to establish a clear, smooth, and responsive risk-effect transformation function, i.e., security degradation logic, to ensure that the system can adapt to risk measurement. Adjust the size and intensity of the enhancement accordingly;

[0101] This process is defined by a piecewise function:

[0102] When risk measurement Less than the preset warning threshold At that time, the system determines that the image has been beautified. The quality is very high, the risk of distortion is extremely low, and it is in a safe zone. At this point, the dynamic fusion coefficient will be... Set to the preset maximum value ; It is a business-preset scalar hyperparameter whose function is to define the maximum enhancement intensity that the system is allowed to apply. Its source is set according to the aesthetic requirements of the product and the preferences of the target user group, and its value range is constrained to [0, 1]. This is intended to provide the best visual enhancement effect under the premise of controllable risk.

[0103] When risk measurement Greater than or equal to the preset warning threshold And less than or equal to the preset critical threshold At that time, the system determines that the image has been beautified. Perceptible but still acceptable flaws have begun to appear, placing the area in the warning zone; to ensure a smooth transition and avoid abrupt changes in effects, the dynamic fusion coefficient is adjusted. The linear decay strategy is adopted, and its calculation formula is as follows:

[0104] ;

[0105] In this formula, The risk measure calculated from the preceding steps is a dimensionless scalar data type. The preset warning threshold is a dimensionless scalar data type, and its source is determined by manually calibrating the risk level; The preset critical threshold is a dimensionless scalar data type, and its source is also determined by manually calibrating the risk level. The maximum value is preset according to business needs, and its data type is a dimensionless scalar.

[0106] The applicable scope of this formula is: Within this range, as from Increase to , Monotonically decreasing from The value decreases to 0; to ensure numerical stability, boundary constraints can be added in practical implementations. This constraint ensures Always in [0, Within the specified range, avoid outliers caused by numerical errors;

[0107] When risk measurement Greater than the preset critical threshold At that time, the system determines that the image has been beautified. Severe distortion or artifacts exist, indicating a dangerous zone that may cause user discomfort; in this case, the system adopts the most conservative strategy, adjusting the dynamic fusion coefficients... Forced to be set to 0;

[0108] This segmented dynamic coefficient determination method provides the system with a refined and adaptive safety control mechanism. It avoids a one-size-fits-all processing logic and achieves smooth and gradual intervention of the beautification effect by establishing safe zones, warning zones, and danger zones. When the image quality is excellent, it can maximize the aesthetic effect; when risks occur, it can reduce the effect proportionally instead of shutting it down directly; when the risk is too high, it can decisively disable the beautification to ensure the bottom-line safety of the output result. This graceful degradation capability greatly improves the stability of the system and the user experience.

[0109] Example 3:

[0110] The hair-covered area repair is achieved by using a pre-defined generative adversarial network;

[0111] Among them, pixel loss and edge loss are combined to form a composite loss function, which is used to optimize and constrain the generative adversarial network.

[0112] This embodiment is a specific implementation of the hair-occluded area repair processing step in Embodiment 1; its purpose is to use a deep learning model to generate areas such as the forehead that are occluded by hair with high fidelity, so that they are seamlessly connected with other parts of the image in terms of texture, lighting and structure.

[0113] In this embodiment, the processing is implemented by using a pre-defined generative adversarial network (GAN); specifically, the GAN in this embodiment is a generator and a corresponding discriminator based on the U-Net architecture, and its function is to generate highly realistic image content through adversarial training between the generator and the discriminator.

[0114] The generator's specific structure is as follows: It adopts the standard U-Net architecture, consisting of a 4-layer encoder and a 4-layer decoder; each layer of the encoder contains two convolutional layers (3×3 kernel size, stride 1, padding 1) and one max-pooling layer (stride 2), with 64, 128, 256, and 512 channels respectively; the decoder adopts a symmetrical structure, with each layer containing an upsampling layer (stride 2) and two convolutional layers, and the feature maps of the corresponding layers of the encoder are concatenated with the feature maps of the decoder through skip connections; the input of the generator is the original image containing occlusion and its corresponding binary occlusion mask, and the output is the repaired image;

[0115] The discriminator's specific structure is as follows: It adopts the PatchGAN architecture, containing 5 convolutional layers with a kernel size of 4×4, and the stride is alternately set to 2 and 1. The number of channels is 64, 128, 256, 512, and 1, respectively. The discriminator uses a receptive field of 70×70 to distinguish between real and fake local regions of the image, and the final output is an N×N discriminant map, where each element represents the probability of real or fake for the corresponding region. Compared with a global discriminator, this design can more effectively preserve the texture details of the repaired region.

[0116] Training strategy: Standard adversarial training is used, with the generator and discriminator updated alternately; the learning rate is set to 0.0002, and the Adam optimizer is employed. , The training iterations are 50,000, and the batch size is 16. During training, the model performance is evaluated on the validation set every 1,000 iterations, and the model with the lowest validation loss is selected as the final model.

[0117] To effectively optimize and constrain the generative adversarial network so that its generated results are not only accurate in content but also have clear edges, this embodiment combines pixel loss and edge loss to form a composite loss function. The innovative consideration of this design is that it recognizes that individual pixel-level matching is prone to blurring, and structural constraints must be introduced to ensure the authenticity of details.

[0118] The composite loss function The calculation method is as follows:

[0119] ;

[0120] In the formula, To repair the total loss of the network, it is used as a scalar to guide network training; The pixel loss, in its physical sense, is to ensure the overall color and brightness consistency of the repaired area. Its source is the L1 norm, commonly used in image-to-image translation tasks, and its calculation method is as follows: ,in A true reference image without any occlusions; The edge loss, in its physical sense, forces the network to generate realistic hairline edges in terms of structure. It originates from a structural similarity metric based on image gradients and is calculated as follows: ,in It is a preset Laplacian operator used to extract high-frequency edge information from an image; and The scalar weight coefficients are derived by conducting multiple experiments on the standard validation set to select the weight combination that best performs in terms of peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) of the restored image.

[0121] In this embodiment, to ensure pixel loss and edge loss The two loss terms are comparable on a numerical scale and are normalized separately; specifically... Normalize based on the total number of pixels in the image. Normalization is performed based on the effective pixel count of the edge map; after normalization, the numerical range of both loss terms is approximately [0, 255], therefore, the weight coefficients can be directly used. and A linear combination is performed; the values ​​of the weighting coefficients reflect the relative importance of pixel consistency and edge sharpness. In this embodiment, , This configuration allows the network to maintain overall color consistency while appropriately emphasizing the structural clarity of the hairline.

[0122] By employing a generative adversarial network and designing a composite loss function that combines pixel loss and edge loss, this embodiment can achieve high-quality hair occlusion repair. Pixel loss ensures the macroscopic consistency of the generated content, while edge loss focuses on the fine depiction of key structures such as the hairline. The combination of the two effectively overcomes the defects of traditional methods that are prone to producing blurring and distortion artifacts, and significantly improves the realism and clarity of the repaired area.

[0123] Example 4:

[0124] Lightweight skincare is achieved through a pre-designed lightweight student network.

[0125] Among them, knowledge distillation technology is used to guide the training of lightweight student networks using a pre-set teacher network.

[0126] Knowledge distillation techniques utilize soft label loss and feature map alignment loss for constraints.

[0127] This embodiment is a specific implementation of the lightweight skin beautification steps in Embodiment 1, and elaborates on its core training technology in detail. The purpose of this design is to significantly reduce the computational complexity of the model while ensuring a powerful skin beautification effect, so that it can run efficiently on devices with limited computing resources.

[0128] This lightweight skin beautification is achieved by using a pre-defined lightweight student network. This lightweight student network is a neural network with a relatively simple model structure, few parameters, and fast computation speed. Its role is to serve as the model for performing skin beautification tasks during actual deployment.

[0129] To enable this lightweight network to achieve performance comparable to complex models, this embodiment employs knowledge distillation, utilizing a pre-set teacher network to guide the training of the lightweight student network. Knowledge distillation is a model compression method whose core idea is to allow a high-performance but structurally complex teacher network to transfer its learned knowledge to the lightweight student network. The teacher network is a pre-trained, large neural network with top-tier performance.

[0130] To achieve deeper and more effective knowledge transfer, the knowledge distillation technique in this embodiment uses soft label loss and feature map alignment loss for constraint; this dual constraint design requires students not only to imitate the teacher's final answer, but also to imitate the teacher's feature extraction logic.

[0131] The total distillation loss function of this technology for In the formula, This is the total distillation loss scalar used to guide students' online training; It is a scalar weight hyperparameter, which is obtained through external experimentation and is used to balance the importance of the final output alignment and the intermediate feature alignment. The soft label loss aims to enable the student network to learn the complete probability distribution of the teacher network's output, rather than just the final prediction result. In this embodiment, it is derived from the KL divergence obtained by fitting the probability distribution of the teacher network's output. ,in and These are the final outputs of the student and teacher networks, respectively. The feature map alignment loss is used to enable the student network to learn the feature representation capabilities of the corresponding layers of the teacher network in its intermediate layers. In this embodiment, it is derived from the feature-level knowledge distillation method and is calculated using the L2 norm. ,in and The student and teacher networks are respectively located in the pre-designed intermediate layer. The extracted feature map, It is a collection of multiple intermediate layer indices, which are derived from the network layers containing rich texture and structural information selected after hierarchical analysis of the teacher network.

[0132] This invention realizes a highly efficient and high-quality lightweight skin-beautifying method. It adopts knowledge distillation technology, especially through the dual constraints of soft label loss and feature map alignment loss, which enables the lightweight student network to deeply simulate and inherit the powerful capabilities of complex teacher networks. While achieving skin-beautifying effects similar to top-tier large-scale models, it significantly reduces computational load and memory usage, allowing this advanced skin-beautifying function to run smoothly on mobile devices and other devices, which has extremely high practical value and deployment advantages.

[0133] Example 5:

[0134] Determine indicators of unnatural smoothness in skin areas, including:

[0135] Calculate the texture variance of the skin region in the beautified image and compare it with a preset reference variance.

[0136] This embodiment is a specific implementation of the step in Example 1 to determine the unnatural smoothness index of skin areas; its purpose is to provide an objective and quantitative method to detect whether lightweight skin beautification is excessive, thereby causing the skin to lose its natural texture.

[0137] In this embodiment, the process of determining the index includes calculating the texture variance of the skin region in the beautified image and comparing it with a preset reference variance. The technical principle behind this design is that there are subtle texture variations on the natural skin surface, which will be reflected as the variance of pixel values ​​in the high-pass filtered image. Excessive lightweight skin beautification will smooth out these details, resulting in a significant reduction in variance.

[0138] The specific calculation formula is as follows In the formula, This is an index of the unnatural smoothness of skin areas. Its data type is a dimensionless scalar, and the larger the value, the more unnatural the smoothness. The preset reference variance is a benchmark for providing a natural skin texture. It is derived from a large and diverse database of natural facial skin textures. The variance of skin regions after the same high-pass filtering is statistically analyzed, and the average variance value is obtained. This ensures the universality and objectivity of the benchmark. This is a variance calculation operation used to calculate the variance of pixel values ​​in the input image region; This is a high-pass filter, such as a Laplacian operator, which is used to extract high-frequency texture details in an image; To enhance the skin area in an image, the source is obtained through facial landmark localization or semantic segmentation techniques. Automatic extraction from Chinese;

[0139] The texture variance comparison method used in this embodiment provides a reference-free and highly sensitive technique for evaluating skin smoothness. It innovatively compares the texture features of the processed image with a natural texture benchmark statistically derived from large-scale real data, thereby accurately quantifying the degree of detail loss caused by excessive skin smoothing. Compared with traditional methods that rely on subjective judgment or reference images, this method has a higher level of objectivity and automation, providing reliable data support for the accurate decision-making of risk control models.

[0140] Example 6:

[0141] Please see Figure 2 The hair-occluded area repair module is used to repair the hair-occluded area of ​​the original input image and obtain the repaired image.

[0142] The lightweight skin-smoothing module is used to perform lightweight skin-smoothing on repaired images to obtain beautified images;

[0143] The smoothness index determination module is used to determine the unnatural smoothness index of the skin area based on the beautified image and the preset reference variance.

[0144] The distortion index determination module is used to determine the distortion index of the hair restoration boundary structure based on the hairline boundary region between the restored image and the beautified image.

[0145] The deviation index determination module is used to determine non-physiological deviation indexes of skin color based on the color histogram of the beautified image and the preset spatial reference probability distribution of natural skin color.

[0146] The risk measurement and calculation module is used to combine the skin region non-natural smoothness index, hair repair boundary structure distortion index, and skin color non-physiological deviation index, and calculate the risk measurement through a preset weighted model.

[0147] The coefficient generation module is used to determine the dynamic fusion coefficient based on risk measurement, preset warning threshold, and preset critical threshold.

[0148] The image synthesis module is used to perform weighted fusion of the repaired image and the beautified image based on dynamic fusion coefficients to generate the final image.

[0149] This embodiment provides a face fusion preprocessing system based on hair restoration and lightweight skin beautification. The system is designed to perform the methods of any of the foregoing embodiments. The purpose of the system is to provide an integrated and automated solution to achieve high-quality and secure face image preprocessing.

[0150] The system consists of the following collaborative modules: a hair occlusion repair module, used to repair hair occlusion areas in the original input image to obtain a repaired image; a lightweight skin smoothing module, used to perform lightweight skin smoothing on the repaired image to obtain a beautified image; a smoothness index determination module, used to determine the unnatural smoothness index of the skin region based on the beautified image and a preset reference variance; a distortion index determination module, used to determine the hair repair boundary structure distortion index based on the hairline boundary region of the repaired image and the beautified image; a deviation index determination module, used to determine the non-physiological deviation index of skin color based on the color histogram of the beautified image and a preset natural skin color spatial reference probability distribution; a risk measurement calculation module, used to combine the above three indices and calculate the risk measurement through a preset weighted model; a coefficient generation module, used to determine the dynamic fusion coefficient based on the risk measurement, a preset warning threshold, and a preset critical threshold; and an image synthesis module, used to perform weighted fusion of the repaired image and the beautified image according to the dynamic fusion coefficient to generate the final image.

[0151] The system implements the complete process of the method in Example 1 through a clear modular design; each module has a clear responsibility and interface, forming an automated pipeline from image input, repair, beautification, multi-dimensional risk assessment to adaptive image synthesis; this system architecture not only ensures the feasibility of the technical solution, but also makes it easy to maintain, upgrade and expand, laying a solid system foundation for providing stable, efficient and reliable face image preprocessing services.

[0152] Example 7:

[0153] The coefficient generation module is configured as follows:

[0154] When the risk metric is less than the preset warning threshold, the dynamic fusion coefficient is set to the preset maximum value;

[0155] When the risk metric is greater than or equal to the preset warning threshold and less than or equal to the preset critical threshold, the dynamic fusion coefficient is linearly reduced from the preset maximum value to zero based on the risk metric.

[0156] When the risk metric exceeds the preset critical threshold, the dynamic fusion coefficient is set to zero.

[0157] The specific configuration method of the coefficient generation module in the system; the purpose of this configuration is to solidify the security degradation logic described in Example 2 into the internal working mechanism of the module, so that it can automatically and accurately execute risk response;

[0158] In this embodiment, the coefficient generation module is configured to operate according to the following logic: when the risk metric it receives is less than a preset warning threshold, the dynamic fusion coefficient is set to a preset maximum value; when the risk metric is greater than or equal to the preset warning threshold and less than or equal to a preset critical threshold, the dynamic fusion coefficient is linearly reduced from the preset maximum value to zero based on the risk metric; when the risk metric is greater than the preset critical threshold, the dynamic fusion coefficient is set to zero.

[0159] By configuring the coefficient generation module in such a clear three-stage logic, the entire system possesses intelligent and smooth risk avoidance capabilities. This configuration encapsulates complex decision-making logic within a single module, ensuring that risk assessment results can be accurately and flawlessly translated into refined control over the beautification effect. This not only enhances the system's automation level but also, through a predictable and reasonable response mechanism, guarantees that the system can output high-quality and reliable images at any risk level, thereby greatly enhancing the system's robustness and end-user trust.

[0160] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A face fusion preprocessing method based on hair repair and lightweight skin beautification, characterized in that, include: The original input image is processed to repair the hair-occluded areas, and the repaired image is obtained. Lightweight skin smoothing is applied to the repaired image to obtain a beautified image; Based on the variance between the beautified image and the preset reference, the unnatural smoothness index of the skin region is determined. Based on the hairline boundary region of the restored and beautified images, the hair restoration boundary structure distortion index is determined; Based on the color histogram of the beautified image and the preset spatial reference probability distribution of natural skin color, the non-physiological deviation index of skin color is determined. By combining the indicators of unnatural smoothness of skin regions, distortion of hair repair boundary structure, and non-physiological deviation of skin color, a risk measure is calculated through a pre-set weighted model. The dynamic fusion coefficient is determined based on risk measurement, preset warning threshold, and preset critical threshold. Based on the dynamic fusion coefficient, the restored image and the beautified image are weighted and fused to generate the final image; The preset reference variance, in its physical sense, provides a benchmark for natural skin texture. It is derived from a large, diverse database of natural facial skin textures, by statistically analyzing the variance of skin regions after the same high-pass filtering process, and finally obtaining the average variance value. Final image The specific calculation formula is as follows: in, This is the dynamic fusion coefficient, and its value range is [0, ...]. ];when hour, This means the output is a completely retouched image, without any enhancement effects. This carries the lowest risk but offers zero aesthetic improvement. When the beautification effect reaches its preset maximum intensity, the aesthetic enhancement is maximized, but it is necessary to ensure that the risks are controllable; when At this time, the output image falls between the repaired image and the beautified image, achieving a balance between aesthetic enhancement and risk control.

2. The face fusion preprocessing method based on hair repair and lightweight skin beautification according to claim 1, characterized in that, Determine the dynamic fusion coefficients, including: When the risk metric is less than the preset warning threshold, the dynamic fusion coefficient is set to the preset maximum value; When the risk metric is greater than or equal to the preset warning threshold and less than or equal to the preset critical threshold, the dynamic fusion coefficient is linearly reduced from the preset maximum value to zero based on the risk metric. When the risk metric exceeds the preset critical threshold, the dynamic fusion coefficient is set to zero.

3. The face fusion preprocessing method based on hair repair and lightweight skin beautification according to claim 1, characterized in that, The hair-covered area repair is achieved by using a pre-defined generative adversarial network; Among them, pixel loss and edge loss are combined to form a composite loss function, which is used to optimize and constrain the generative adversarial network.

4. The face fusion preprocessing method based on hair repair and lightweight skin beautification according to claim 1, characterized in that, Lightweight skincare is achieved through a pre-designed lightweight student network. Among them, knowledge distillation technology is used to guide the training of lightweight student networks using a pre-set teacher network.

5. The face fusion preprocessing method based on hair repair and lightweight skin beautification according to claim 4, characterized in that, Knowledge distillation techniques utilize soft label loss and feature map alignment loss for constraints.

6. The face fusion preprocessing method based on hair repair and lightweight skin beautification according to claim 1, characterized in that, Determine indicators of unnatural smoothness in skin areas, including: Calculate the texture variance of the skin region in the beautified image and compare it with a preset reference variance.

7. A face fusion preprocessing system based on hair repair and lightweight skin beautification, based on the face fusion preprocessing method based on hair repair and lightweight skin beautification as described in any one of claims 1-6, characterized in that, include: The hair-occluded area repair module is used to repair the hair-occluded areas of the original input image and obtain the repaired image. The lightweight skin-smoothing module is used to perform lightweight skin-smoothing on repaired images to obtain beautified images; The smoothness index determination module is used to determine the unnatural smoothness index of the skin area based on the beautified image and the preset reference variance. The distortion index determination module is used to determine the distortion index of the hair restoration boundary structure based on the hairline boundary region between the restored image and the beautified image. The deviation index determination module is used to determine non-physiological deviation indexes of skin color based on the color histogram of the beautified image and the preset spatial reference probability distribution of natural skin color. The risk measurement and calculation module is used to combine the non-natural smoothness index of skin area, the distortion index of hair repair boundary structure, and the non-physiological deviation index of skin color, and calculate the risk measurement through a preset weighted model. The coefficient generation module is used to determine the dynamic fusion coefficient based on risk measurement, preset warning threshold, and preset critical threshold. The image synthesis module is used to perform weighted fusion of the repaired image and the beautified image based on dynamic fusion coefficients to generate the final image.

8. The face fusion preprocessing system based on hair repair and lightweight skin beautification according to claim 7, characterized in that, The coefficient generation module is configured as follows: When the risk metric is less than the preset warning threshold, the dynamic fusion coefficient is set to the preset maximum value; When the risk metric is greater than or equal to the preset warning threshold and less than or equal to the preset critical threshold, the dynamic fusion coefficient is linearly reduced from the preset maximum value to zero based on the risk metric. When the risk metric exceeds the preset critical threshold, the dynamic fusion coefficient is set to zero.

Images