A beautification method based on blemish-texture binary recognition and multi-source condition guidance
By combining multi-scale convolutional neural networks and conditional coding networks, we have achieved accurate differentiation between blemishes and textures and environmentally adaptive beautification, solving the problems of unclear differentiation between blemishes and textures and insufficient environmental adaptation in existing technologies, and achieving natural and realistic beautification effects.
Patent Information
- Application Number
- CN202610670394.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-05-15
- Publication Date
- 2026-07-31
AI Technical Summary
Existing technology cannot effectively distinguish between skin imperfections and natural textures, resulting in a loss of recognizability and realism during the beautification process. At the same time, it cannot adapt to different shooting environments, leading to unnatural beautification effects.
A multi-scale convolutional neural network is used for blemish-texture binary recognition. Combined with a conditional coding network and an adaptive instance normalization module, blemish differentiation and texture protection are achieved, and fine repair and reconstruction are performed based on ambient light and skin texture parameters.
It achieves pixel-level precise differentiation of blemishes and textures, maintaining the original skin texture, while the beautification effect is natural and the lighting is natural in different environments, improving recognizability and robustness.
Smart Images
Figure CN122492513A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of image processing and computer vision technology, and to a real-time image beautification method for mobile devices, particularly a beautification method based on blemish-texture binary recognition and multi-source condition guidance. Background Technology
[0002] With the development of mobile imaging technology and social media, users' demands for mobile phone camera beautification functions have shifted from the early crude "excessive skin smoothing" to the pursuit of "ultimate naturalness" and "texture restoration". Users hope to remove facial blemishes (such as acne scars and freckles) while fully preserving or even enhancing the skin's natural texture (such as pores and fine skin lines), that is, to achieve a "natural, personalized, and original skin" beautification effect.
[0003] Existing camera apps, such as BeautyCam and LightCam, generally suffer from the following problems with their beautification functions: 1. Homogenization and Distortion of Technology: Existing technologies mostly rely on overall skin smoothing or feature extraction based on a single scale, which cannot effectively distinguish between skin blemishes (such as acne scars and spots) and natural skin textures (such as pores and skin texture). This often leads to the removal of natural textures such as pores when processing blemishes, resulting in a "plastic" or "internet celebrity face" style of distortion, severely losing the original recognizability and realism of the human face.
[0004] 2. Poor scene adaptation: Most beautification algorithms use fixed parameters and cannot dynamically adapt to complex and changing shooting environments, such as strong light, low light, and backlight. This leads to problems such as uneven skin tone, overexposure, or loss of detail in beautification effects under different lighting conditions, resulting in poor integration with the overall ambient light and shadow and an abrupt appearance.
[0005] Therefore, how to provide an AI beautification method that can accurately preserve the user's original skin texture, effectively remove blemishes, and adapt to various shooting scenarios is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0006] To address the aforementioned shortcomings of existing technologies, this invention provides a beautification method based on defect-texture binary recognition and multi-source condition guidance.
[0007] This invention proposes a beautification method based on blemish-texture binary recognition and multi-source condition guidance, comprising the following steps: Step a: Acquire real-time facial images captured by the user. ,in Represents the set of real numbers. Indicates the image height value. This represents the image width value; and the synchronously acquired ambient light parameters. and the user's skin type parameters ,in, Indicates the brightness value. Indicates the color temperature coefficient. Represents the tensor dimension; Step b: Extract multi-scale features of the face image using a multi-scale convolutional neural network. , The number of feature tensors is used to perform binary recognition based on the multi-scale features, distinguishing between blemish features and skin texture features in the face image, and generating corresponding blemish masks. and texture mask ; Step c: Based on the blemish mask and texture mask, perform blemish removal and texture protection processing on the face image to obtain the initial retouched image. ; Step d: The light parameters Skin type parameters The input conditional coding network is mapped to conditional feature vectors in a high-dimensional semantic space. ;as well as Step e: By using the concatenation and adaptive instance normalization (AdaIN) module, the feature conditional vector is injected into the generator network to perform fine repair, texture restoration and skin color reconstruction on the features of the initial image, thereby achieving deep coupling between environment and skin texture features.
[0008] The present invention has the following beneficial effects: 1. Achieve precise beautification with "imperfection removal while preserving natural texture": Through an innovative multi-scale "imperfection-texture" binary recognition network, it achieves precise differentiation between imperfections and textures at the pixel level for the first time, fundamentally solving the problem of "skin retouching damaging the skin" in traditional technologies. While efficiently removing imperfections, it can preserve the user's original pores and skin texture 1:1, making the beautification effect natural and realistic, and significantly improving the user's facial recognition.
[0009] 2. Enhanced beautification effects across multiple scenarios: The introduction of conditional coding and feature fusion modules creates a strong semantic correlation between beautification effects and shooting lighting and user skin texture. The model can perform pixel-level reconstruction based on real-time light and shadow semantics, achieving "lossless preservation" and even "targeted enhancement" of microscopic details such as pores and skin texture. This results in optimized images with uniform skin tone, natural lighting, and a seamless appearance, significantly improving the algorithm's robustness and consistency across different environments. Attached Figure Description
[0010] Figure 1 This is a flowchart of a beautification method based on defect-texture binary recognition and multi-source condition guidance according to a preferred embodiment of the present invention.
[0011] Figure 2 This is a schematic diagram of the mapping principle of the conditional encoding module according to a preferred embodiment of the present invention.
[0012] Figure 3 This is a schematic diagram of the architecture of the end-to-end generator and feature fusion mechanism according to a preferred embodiment of the present invention. Detailed Implementation
[0013] The present invention will be further illustrated below through examples, the purpose of which is only to better understand the research content of the present invention and not to limit the scope of protection of the present invention.
[0014] like Figure 1 As shown, a preferred embodiment of the beautification method based on blemish-texture binary recognition and multi-source condition guidance of the present invention includes the following steps a~f.
[0015] Step a: Acquire real-time facial images captured by the user. ,in Represents the set of real numbers. Indicates the image height value. This represents the image width value and the synchronously acquired ambient light parameters. and the user's skin type parameters ,in, Indicates the brightness value. Indicates the color temperature coefficient. This represents the tensor dimension. Here, ambient light parameters include brightness and color temperature, which can be captured by the current device's sensors. Skin texture parameters include skin color, whether the skin is oily, brightness, etc., which can be obtained from the user's device.
[0016] Step b: Extract multi-scale features of the face image using a multi-scale convolutional neural network. , The number of feature tensors is used to perform binary recognition based on the multi-scale features, distinguishing between blemish features and skin texture features in the face image, and generating corresponding blemish masks and texture masks. Here, blemish features include acne scars, spots, and fine lines. Skin texture features include pores and skin texture. Preferably, the face image is preprocessed before being input into the multi-scale convolutional neural network.
[0017] Preferably, in this step, the multi-scale convolutional neural network includes a pre-trained backbone network. And two independent branch networks. The backbone network is used for multi-scale feature extraction, represented as... It can capture features at different scales, from macroscopic (such as spots) to microscopic (such as pores). Subsequently, these features are fused to obtain ,in , This indicates a channel-level concatenation operation. Then, features are fused. They are fed in parallel into two independent branch networks: a defect detection branch. and texture recognition branch Specifically, by performing forward inference on the defect recognition branch network, the defect response map is obtained: And then through The operator maps it to a pixel-level probabilistic mask (flaw mask). =sigmoid( This is used to accurately mark the areas that need repair; similarly, by performing forward inference on the texture recognition branch network, the texture response result map is obtained: Then through The operator maps it to a pixel-level probabilistic mask (texture mask). This accurately identifies areas of skin texture that need to be preserved or enhanced. The backbone network, blemish recognition branch, and texture recognition branch all employ lightweight convolutional neural networks.
[0018] Step c: Based on the blemish mask and texture mask, perform blemish removal and texture protection processing on the face image to obtain the initial retouched image. ,in This represents a mapping function for neighborhood sampling inpainting and texture preservation based on dual-mask constraints. Here, image inpainting algorithms are used to eliminate blemishes in blemished areas while preserving skin details through texture. Specifically, this step employs a neighborhood pixel sampling inpainting algorithm. Based on the blemish mask, areas such as acne scars, spots, and localized redness awaiting repair are identified. Candidate pixels are sampled from surrounding normal skin areas that are not covered by the blemish mask and have similar skin tone and brightness. The color and texture information of these candidate pixels are then used to fill the blemish areas. Simultaneously, the smoothing intensity of pores and normal skin texture areas is reduced based on the texture mask to prevent these natural skin details from being mistakenly eliminated during the repair process. Through the above processing, blemish areas can be naturally replaced by surrounding normal skin, while textured areas can retain their original texture, thus preserving the user's original skin texture while removing blemishes. The initial retouched image obtained in this step has already undergone precise processing of blemishes and skin texture, but its lighting and skin texture blending are still in a state of refinement.
[0019] Step d: The light parameters Skin type parameters The input conditional coding network (conditional embedding network) is mapped to conditional feature vectors in a high-dimensional semantic space. In this step, non-image-related ray parameters are... With skin type parameters ,in, This refers to a conditional coding network, used to convert environmental and physiological parameters into conditional feature vectors that can be processed by a neural network. ,in Represents the set of real numbers. This represents the preset vector dimension. Conditional feature vectors are also called conditional vectors or conditional tensors. Specifically, such as... Figure 2 As shown, the light parameters are converted into conditional feature vectors through the perceptual coding layer of the conditional coding network, and the skin texture parameters are converted into conditional feature vectors through the discrete embedding layer of the conditional coding network.
[0020] Step e: The conditional feature vector is injected into the generator network through feature concatenation and adaptive instance normalization (AdaIN) modules to perform fine-tuning, texture restoration, and skin color reconstruction on the features of the initial image, achieving deep coupling between environment and skin texture features. In this step, the AdaIN module is represented as... ,in These are feature maps of the generator network. It is a conditional tensor. and Generated by a conditional mapping network, It is a scaling tensor. It is a bias tensor. and These are the mean and standard deviation of the feature tensor, respectively.
[0021] Preferably, step e further includes steps e1-e5, which are described below in conjunction with... Figure 3 Detailed explanation follows. Step e1: The initial retouched image obtained in step c... Input to generator network Preferably, the generator network is an encoder-decoder structure or a fully convolutional generator network. The generator network first extracts features through convolutional residual blocks to obtain multiple layers of intermediate feature maps. Where l represents the layer number. Step e2, the condition vector obtained in step d... The input is a conditional mapping network, which generates intermediate feature maps from the generator. corresponding scaling tensor and bias tensor This step allows ambient lighting parameters and user skin texture parameters to directly influence the feature reconstruction process of the generator network.
[0022] Step e3, for any layer feature map in the generator network The AdaIN module first calculates the mean and standard deviation of the feature map and then normalizes it to remove some distribution information related to brightness, color, and appearance statistics from the original features. Then, in step e4, the scaling tensor generated from the conditional vector is used... and bias tensor The normalized feature maps are recalibrated to incorporate the current shooting environment and user skin texture into the intermediate feature maps of the generator network. The scaling tensor and bias tensor can be further expressed as: in, and This represents the weight matrix corresponding to the l-th layer in the conditional mapping network. and This indicates the corresponding bias term.
[0023] Step e5: The features modulated by AdaIN are then fed into the generator network. The subsequent convolutional residual blocks and reconstruction modules yield the final output graph. .
[0024] As shown above, the generator network accurately repairs facial regions through multi-layer residual connections and feature reconstruction, while meticulously preserving texture areas, outputting high-fidelity original skin beautified images. Preferably, the present invention also employs multi-constraint loss optimization for model training. Specifically, during the training phase, L1 pixel loss, adversarial loss (enhancing the realism of details), and specific texture preservation constraints are used to ensure that the output image, while removing imperfections, presents a highly realistic "original skin" texture.
[0025] Preferably, step f is included after step e: determining whether the generated beautified image conforms to the real data distribution. Here, during training, this is determined by the discriminator network. It is responsible for judging whether the generated beautified image conforms to the real data distribution based on the data distribution of the real flawless image, thereby guiding the generator to generate a beautified result image that better meets the expected goal.
[0026] Obviously, those skilled in the art should recognize that the above embodiments are only used to illustrate the present invention and are not intended to limit the present invention. Any changes or modifications to the above embodiments that are within the essential spirit of the present invention will fall within the scope of the claims of the present invention.
Claims
1. A blemish-texture binary recognition and multi-source condition guided beauty method, characterized in that, Includes the following steps: Step a: obtaining a real-time photographed face image of a user wherein denotes a real number set, denotes an image height value, denotes an image width value; and synchronously collecting ambient light parameters and skin quality parameters of the user wherein, denotes a lightness value, denotes a color temperature coefficient, denotes a tensor dimension; Step b: extracting multi-scale features of the face image by a multi-scale convolutional neural network , representing the number of feature tensors, performing binary recognition based on the multi-scale features, distinguishing between flaw features and skin texture features in the face image, and generating corresponding flaw masks and texture masks ; Step c: performing blemish removal and texture protection processing on the face image based on the blemish mask and the texture mask to obtain a preliminary repaired image ; Step d: inputting the light parameter and skin parameter into a condition encoding network to map the condition feature vector into a high-dimensional semantic space ; and Step e: By using the concatenation and adaptive instance normalization (AdaIN) module, the conditional feature vector is injected into the generator network to perform fine repair, texture restoration and skin color reconstruction on the features of the initial image, thereby achieving deep coupling between environment and skin texture features.
2. The method according to claim 1, characterized in that, In step b, the multi-scale convolutional neural network comprises a pre-trained backbone network and two independent branch networks, the backbone network is used for multi-scale feature extraction, and the two branch networks comprise a flaw recognition branch and a texture recognition branch .
3. The method according to claim 2, characterized in that, In step b, the multi-scale features include features at different scales, from macroscopic to microscopic.
4. The method according to claim 1, characterized in that, In step c, a neighborhood pixel sampling repair algorithm is used. Based on the area to be repaired determined by the blemish mask, candidate pixels are sampled from the surrounding normal skin area that is not covered by the blemish mask, and the color and texture information of the candidate pixels are filled into the blemish area.
5. The method according to claim 1, characterized in that, In step d, the AdaIN module is represented as ,in These are feature maps of the generator network. It is a conditional tensor. and Generated by a conditional mapping network, It is a scaling tensor. It is a bias tensor. and These are the mean and standard deviation of the feature tensor, respectively.
6. The method according to claim 5, characterized in that, Step e further includes the following steps: Step e1: The initial retouched image obtained in step c Input to generator network The generator network extracts features through convolutional residual blocks to obtain multi-layer intermediate feature maps. , where l represents the number of layers; Step e2: The conditional feature vector obtained in step d... The input is a conditional mapping network, which generates intermediate feature maps from the generator. corresponding scaling tensor and bias tensor ; Step e3: For any layer feature map in the generator network The AdaIN module first calculates the mean of the feature map. and standard deviation The feature map is then normalized. Step e4: Using the scaling tensor generated from the condition vector and bias tensor The normalized feature maps are recalibrated to incorporate the current shooting environment and user skin texture into the intermediate feature maps of the generator network. middle; as well as Step e5: The features modulated by AdaIN are then fed into the generator network. The subsequent residual module and reconstruction module yield the final output graph. .
7. The method according to claim 6, characterized in that, The scaling tensor and bias tensor can be further expressed as: in, and This represents the weight matrix corresponding to the l-th layer in the conditional mapping network. and This indicates the corresponding bias term.
8. The method according to claim 1, characterized in that, Step e is followed by step f: determining whether the generated beautified image conforms to the real data distribution.