A deep learning-based personalized foundation dispensing system and method

Abstract

The application relates to the technical field of artificial intelligence, and discloses a personalized foundation blending system and method based on deep learning, which comprises an information collection and analysis module, a feature extraction module, a formula generation module and a formula correction module.The information collection and analysis module collects a user face image, removes invalid pixels in combination with GMM clustering, extracts effective skin LAB parameters to construct a skin color dataset; the feature extraction module extracts skin color multi-dimensional hidden features by adopting an EfficientNetV2-S model, determines skin color categories and subcutaneous base colors, and generates a final feature vector; the formula generation module calculates the initial proportions of white, black, yellow and red color paste by regression through a multi-branch full-connection deep regression network; and the formula correction module iteratively adjusts colors in combination with a residual network color simulation and a DDIM diffusion model, and outputs a standardized four-color paste formula after hierarchical color difference verification, thereby breaking through the limitation of fixed color numbers and realizing high-precision personalized foundation blending.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a personalized foundation mixing system and method based on deep learning. Background Technology

[0002] As the core product of facial makeup, foundation is only available in a limited number of fixed shades in industrially produced foundations on the market. This makes it difficult to match the different natural skin tones and subcutaneous pigmentation of consumers. Personalized customized foundation is gradually becoming a development trend in the beauty industry. Currently, personalized foundations are mainly prepared by makeup professionals using a blend of four basic color-correcting agents: white, black, yellow, and red. The mainstream mixing method in the industry still relies primarily on manual visual color matching. Makeup artists observe the user's facial skin tone in the on-site environment and subjectively determine the mixing ratio of the four color agents based on their professional experience.

[0003] Therefore, it is necessary to design a personalized foundation mixing system and method based on deep learning to solve the problem that existing foundation shades are limited and difficult to adapt to consumers' different original skin tones and subcutaneous pigmentation states. Summary of the Invention

[0004] In view of this, the present invention proposes a personalized foundation mixing system and method based on deep learning, which aims to solve the problem that existing foundation shades are limited and difficult to adapt to the different original skin tones and subcutaneous pigmentation states of consumers.

[0005] In one aspect, this invention proposes a personalized foundation mixing system based on deep learning, comprising: The information collection and analysis module is configured to collect multi-dimensional facial images of users, verify the effective pixels of the collected facial images based on the light source and color temperature of the collection environment, select qualified standardized human skin color images, use a pre-trained U-Net deep learning segmentation model to perform semantic segmentation of skin regions on the standardized human skin color images, remove invalid interference pixels through GMM Gaussian mixture clustering algorithm and color difference analysis, extract the LAB color parameters of the user's effective skin pixels, and construct the user skin color dataset. The feature extraction module is configured to input the user skin color dataset constructed by the information collection and analysis module into the pre-trained EfficientNetV2-S deep learning feature extraction model, extract skin color brightness, warm and cool tones, and subcutaneous pigment multidimensional latent features layer by layer, generate skin color feature vector, determine the user's skin color category and subcutaneous background color based on the skin color feature vector, optimize and improve the feature vector, and generate the final feature vector. The formula generation module is configured to input the final feature vector into a multi-branch fully connected deep regression neural network, and respectively regress the features of brightness, yellow tone, and red tone to obtain the initial addition amount of white liquid, black liquid, yellow liquid, and red liquid, thereby generating a preliminary formula for foundation liquid; The formula correction module is configured to generate a simulated LAB color value for the foundation liquid based on the preliminary formula generated by the formula generation module through a residual correction deep learning network, obtain the color difference between the simulated LAB color value and the user's original skin tone, and correct the amount of four color pastes added based on the color difference to obtain a standardized four-color paste foundation formula that is suitable for the user's skin tone.

[0006] Furthermore, the information collection and analysis module is configured as follows: The color temperature parameters of the shooting environment are collected in real time. When the color temperature parameters are less than the first color temperature threshold, it is determined that the warm light of the shooting environment is yellowish, and the CLAHE algorithm is used to correct the blue light gain of the image. When the color temperature parameter is greater than the second color temperature threshold, it is determined that the ambient light is bluish and the image is corrected for yellow light gain using the CLAHE algorithm. When the color temperature parameter is between the first color temperature threshold and the second color temperature threshold, the ambient light source is deemed compliant, and the original image is directly retained.

[0007] Furthermore, the information collection and analysis module is also configured as follows: Standardized human skin color images are input into a pre-trained U-Net segmentation model. The standardized human skin color images are then subjected to pixel binary classification, dividing them into skin pixels and non-skin pixels, and outputting a skin mask map. The skin mask map is then divided into several pixel clusters using Gaussian Mixture Model (GMM). The LAB color difference between the cluster center and the global skin mean is calculated for each cluster. When the color difference of a single cluster is greater than 12 LAB color difference units, the cluster is determined to be invalid pixels and is removed in batches. The LAB three-dimensional color parameters of all compliant pixels are then extracted to construct a skin color dataset.

[0008] Furthermore, the feature extraction module is configured as follows: A Fitzpatrick six-type skin color database is pre-set. The skin color feature vector is compared with the pre-set Fitzpatrick six-type skin color database in terms of feature similarity, and the skin color type with the highest similarity is selected as the user's skin color category. Simultaneously, the user's skin tone is determined based on the channel weights of the skin tone feature vector. If the weight of the red channel latent feature is greater than 0.5, it is determined to be a reddish skin tone; if the weight of the yellow channel is greater than 0.5, it is determined to be a yellowish skin tone; and if the brightness feature is less than 30, it is determined to be a dark skin tone. The skin tone category and the background color label are concatenated to the skin tone feature vector to obtain the final feature vector.

[0009] Furthermore, the recipe generation module is configured as follows: The final feature vector is input into a multi-branch fully connected deep regression neural network, which includes a brightness control branch, a yellow tone control branch, and a red tone control branch. The brightness control branch regresses the basic addition amount of white liquid and black liquid based on the skin color LAB brightness L value in the final feature vector. The L value is positively correlated with the addition amount of white liquid and negatively correlated with the addition amount of black liquid. The yellow-toning regulation branch regresses the basic amount of yellow liquid added based on the b value of the LAB yellow axis in the final feature vector, and the b value is positively correlated with the amount of yellow liquid added. The red tone control branch regresses the basic amount of red liquid added based on the LAB red axis 'a' value in the final feature vector, and the 'a' value is positively correlated with the amount of red liquid added.

[0010] Furthermore, the formula correction module is also configured to: After the residual correction deep learning network simulates and generates the foundation simulation LAB color value, it calculates the color difference between the simulation LAB color value and the user's skin tone. When the color difference is less than or equal to the first color difference threshold, it is judged as a slight color difference, and no iterative correction is performed. The standardized four-color foundation formula is directly output. When the color difference is greater than the first color difference threshold, the DDIM diffusion model is started to perform reverse iterative color adjustment. When performing reverse iterative color adjustment, the maximum number of iterations of the DDIM model is set to 10. If the maximum number of iterations is not reached and the color difference does not meet the standard, the color paste ratio is continuously corrected in reverse. After the standard is met, the corrected color paste formula is re-inspected. If the standard is still not met after reaching the maximum number of iterations, it is determined to be an abnormal feature matching, and the optimal formula during iterative color adjustment is used for a second color difference re-inspection.

[0011] Furthermore, the formula correction module is also configured to: When performing reverse iterative color adjustment, the color difference of each dimension is split into L, a, and b, and the weight ratio of each dimension to the total color difference is calculated. The single dimension with a weight ratio of 60% or more is determined to be the dominant color difference dimension, and the rest are secondary dimensions. If dimension L is the dominant color difference, only adjust the ratio of white and black liquor, while keeping the values ​​of yellow and red liquor unchanged; if dimension a is the dominant color difference, only adjust the ratio of red liquor, while keeping the values ​​of the other three color pastes unchanged; if dimension b is the dominant color difference, only slightly adjust the ratio of yellow liquor, while keeping the values ​​of the other three color pastes unchanged; if no dimension accounts for more than or equal to 60%, it is determined to be a multi-dimensional coupled color difference, and a balance adjustment is made, while simultaneously making small adjustments to the ratios of the four color pastes.

[0012] Furthermore, the formula correction module is also configured to: A second color difference threshold is preset, and the second color difference threshold is greater than the first color difference threshold. When performing a second color difference re-inspection, the corrected formula is input into the residual correction deep learning network to generate the LAB color value of the formula and recalculate the color difference. When the color difference is less than or equal to the first color difference threshold, the color matching is deemed qualified and the corresponding formula is used as the standardized four-color paste foundation mixing formula. When the color difference is greater than the first color difference threshold but less than or equal to the second color difference threshold, it is judged as a slight color difference. After single-component fine-tuning, the fine-tuned formula is used as the standardized four-color paste foundation formula. When the color difference is greater than the second color difference threshold, it is judged as a color difference abnormality, and the user's multi-dimensional face image is re-acquired.

[0013] Furthermore, the formula correction module is also configured to: The maximum adjustment range is preset. When performing single-component fine-tuning, the color difference of L, a, and b is separated into single-dimensional color differences, and the dimension with the highest proportion is selected as the dimension to be corrected. When the dimension to be corrected is L: When the color difference in dimension L is greater than 0, increase the amount of black liquor according to the color difference value, with the maximum increase not exceeding the maximum adjustment range; when the color difference in dimension L is less than 0, increase the amount of white liquor according to the color difference value, with the maximum increase not exceeding the maximum adjustment range. When the dimension to be corrected is 'a': When the color difference in dimension 'a' is greater than 0, reduce the amount of red liquid according to the color difference value, with the maximum reduction not exceeding the maximum adjustment range; when the color difference in dimension 'a' is less than 0, increase the amount of red liquid according to the color difference value, with the maximum increase not exceeding the maximum adjustment range. When the dimension to be corrected is b: When the color difference of dimension b is greater than 0, reduce the amount of yellow liquid according to the color difference value, with the maximum reduction not exceeding the maximum adjustment range; when the color difference of dimension b is less than 0, increase the amount of yellow liquid according to the color difference value, with the maximum increase not exceeding the maximum adjustment range.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention employs the EfficientNetV2-S deep learning feature extraction model to mine multidimensional deep latent features of user skin color, simultaneously identifying user skin color categories and subcutaneous pigment background features. It breaks through the limitations of traditional liquid foundation with fixed and limited shades, accurately capturing the different original skin color appearance features and subcutaneous pigment background differences of different users, realizing exclusive color matching representation for individual skin types, and solving the technical problem that traditional mass-produced foundations cannot adapt to the personalized skin color of thousands of people.

[0015] 2. This invention uses the U-Net deep learning semantic segmentation algorithm combined with the GMM clustering screening mechanism to intelligently purify skin color data. This allows the deep learning model to perform feature learning and color matching inference based solely on the user's real and valid skin pixels, effectively avoiding interference from invalid areas such as acne marks, redness, and blemishes on skin color judgment. This ensures the accuracy of recognizing differentiated original skin colors and special subcutaneous pigmentation states, and significantly improves the basic accuracy of personalized color matching.

[0016] 3. This invention achieves intelligent and differentiated regression calculation of the four-color paste ratio by constructing a multi-branch fully connected deep regression neural network. This enables the deep learning network to autonomously learn the nonlinear mapping relationship between different skin color categories, different subcutaneous base colors and the ratio of white, black, yellow and red liquids. It can adaptively output exclusive ratios for cool white skin, warm yellow skin, dark skin, mixed transition skin, and various subcutaneous pigment base colors, solving the shortcomings of traditional fixed color number adaptation dimension and incomplete coverage of differentiated skin types.

[0017] 4. This invention constructs a closed-loop color difference correction system by introducing a residual correction deep learning network and a DDIM diffusion deep learning iterative color adjustment model. This system enables multi-dimensional and targeted correction of personalized color deviation caused by differences in the original skin tone and subcutaneous pigmentation of different users. It effectively solves the technical pain point that traditional fixed-shade foundation cannot accurately compensate for the differences in subcutaneous pigmentation of users, and greatly improves the fit between the foundation and the user's original skin tone.

[0018] On the other hand, this application also provides a personalized foundation mixing method based on deep learning, wherein the monitoring method includes: Multi-dimensional facial images of users are collected. The effective pixels of the collected facial images are verified according to the light source and color temperature of the collection environment. Qualified standardized human skin color images are selected. A pre-trained U-Net deep learning segmentation model is used to perform semantic segmentation of the skin region of the standardized human skin color images. Invalid interference pixels are removed by Gaussian mixture clustering algorithm and color difference analysis. The LAB color parameters of the effective skin pixels of users are extracted to construct the user skin color dataset. The constructed user skin color dataset is input into the pre-trained EfficientNetV2-S deep learning feature extraction model. Multi-dimensional latent features of skin color brightness, warm and cool tones, and subcutaneous pigment are extracted layer by layer to generate skin color feature vectors. Based on the skin color feature vectors, the user's skin color category and subcutaneous background color are determined, and the feature vectors are optimized and improved to generate the final feature vector. The final feature vector is input into a multi-branch fully connected deep regression neural network, and the initial addition amounts of white liquid, black liquid, yellow liquid and red liquid are obtained by regressing the features corresponding to brightness, yellow tone and red tone respectively, thus generating the preliminary formula of foundation liquid; Based on the preliminary foundation formula generated by the formula generation module, the simulated LAB color value of the foundation is generated through residual correction deep learning network simulation. The color difference between the simulated LAB color value and the user's original skin tone is obtained. Based on the color difference, the amount of four color pastes added is adjusted to obtain a standardized four-color paste foundation formula that is suitable for the user's skin tone.

[0019] It is understandable that the aforementioned deep learning-based personalized foundation mixing system and method have the same beneficial effects, and will not be elaborated further here. Attached Figure Description

[0020] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A functional block diagram of a deep learning-based personalized foundation mixing system provided in an embodiment of the present invention; Figure 2 A flowchart illustrating a deep learning-based personalized foundation mixing method provided in an embodiment of the present invention. Detailed Implementation

[0021] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0022] Reference Figure 1 In some embodiments of this application, a personalized foundation mixing system based on deep learning includes: The information collection and analysis module is configured to collect multi-dimensional facial images of users, verify the effective pixels of the collected facial images based on the light source and color temperature of the collection environment, select qualified standardized human skin color images, use a pre-trained U-Net deep learning segmentation model to perform semantic segmentation of skin regions on the standardized human skin color images, remove invalid interference pixels through GMM Gaussian mixture clustering algorithm and color difference analysis, extract the LAB color parameters of the user's effective skin pixels, and construct the user skin color dataset. The feature extraction module is configured to input the user skin color dataset constructed by the information collection and analysis module into the pre-trained EfficientNetV2-S deep learning feature extraction model, extract skin color brightness, warm and cool tones, and subcutaneous pigment multidimensional latent features layer by layer, generate skin color feature vector, determine the user's skin color category and subcutaneous background color based on the skin color feature vector, optimize and improve the feature vector, and generate the final feature vector. The formula generation module is configured to input the final feature vector into a multi-branch fully connected deep regression neural network, and respectively regress the features of brightness, yellow tone, and red tone to obtain the initial addition amount of white liquid, black liquid, yellow liquid, and red liquid, thereby generating a preliminary formula for foundation liquid; The formula correction module is configured to generate a simulated LAB color value for the foundation liquid based on the preliminary formula generated by the formula generation module through a residual correction deep learning network, obtain the color difference between the simulated LAB color value and the user's original skin tone, and correct the amount of four color pastes added based on the color difference to obtain a standardized four-color paste foundation formula that is suitable for the user's skin tone.

[0023] Specifically, it consists of an information acquisition and analysis module, a feature extraction module, a formula generation module, and a formula correction module, and embeds five types of deep learning architectures: U-Net semantic segmentation model, EfficientNetV2-S feature extraction model, multi-branch fully connected deep regression network, residual correction deep learning network, and DDIM diffusion generation model. The information acquisition and analysis module completes image acquisition, color temperature correction, U-Net skin segmentation, and GMM clustering for noise reduction and purification of skin color LAB data. The feature extraction module relies on EfficientNetV2-S to extract deep skin color latent features and generate the final feature vector with skin color labels. The formula generation module uses a multi-branch deep regression network to regress the initial number of white / black / yellow / red pigments from the feature vectors. The formula correction module relies on residual network color simulation, DDIM reverse iterative optimization, and graded threshold fine-tuning to achieve closed-loop formula correction, and finally outputs only the mixing ratio of the four pigments, without the need for base emulsion preparation. The entire system is divided into two operating phases: the offline training phase and the online inference phase. The offline phase completes the pre-training and fine-tuning of all deep learning models, while the online phase freezes the model weights and only performs forward inference operations to complete skin color recognition and matching calculation.

[0024] For the U-Net segmentation model: offline training is a pre-training term of the information acquisition and analysis module; the standard U-Net adopts a symmetric encoder-decoder structure. The encoder has 5 sets of convolution + max pooling downsampling modules to compress image resolution layer by layer and extract shallow contour and skin texture features; the decoder adopts deconvolution upsampling + skip connections, reuses the same level feature map of the encoder to make up for the loss of upsampling details, and outputs a single-channel skin mask binary classification map, where 1 = skin pixels and 0 = non-skin pixels.

[0025] Training dataset: A self-built facial skin tone image dataset of 120,000 images, including multi-skin tone samples (yellow, white, and black), with annotations consisting of manually extracted skin masks; Training split: 96,000 images for training, 18,000 images for validation, and 6,000 images for testing; Training hyperparameters: Adam optimizer, binary classification cross-entropy loss, initial learning rate 1×10⁻⁶. -4 BatchSize=16, Epoch=80, early stopping strategy (training terminates if the validation set loss does not decrease for 15 consecutive rounds); after training, save the optimal weight file, lock the weights during the online inference stage, input the corrected face RGB image, and directly output the skin mask map.

[0026] For the EfficientNetV2-S feature extraction model: the core backbone network of the feature extraction module is fine-tuned based on the ImageNet public pre-trained weights; the network is composed of stacked MBConv inverted residual modules, large-size deep convolutions are removed to balance feature extraction speed and accuracy, the original classification fully connected layer is removed at the end of the model, and the second to last 128-dimensional feature vector is truncated as the basic feature of skin color.

[0027] Fine-tuning the dataset: The dataset was expanded with 250,000 labeled samples based on the Fitzpatrick six-type standard skin color database. Each sample was bound to a skin LAB pixel set + skin color category label + subcutaneous background color label. Fine-tuning the training: The first 70% of the backbone weights were frozen, and only the last 30% of the convolutional layers and custom feature output layers were fine-tuned. The optimizer was AdamW, MSE loss was used, and the learning rate was 5×10⁻⁶. -5 Epoch=60; Online inference input is the purified skin LAB-mapped RGB image, and forward operation outputs a 128-dimensional basic feature vector.

[0028] For a multi-branch fully connected deep regression neural network, the recipe generation module is embedded in the model. The three branches share the input layer and independently construct fully connected layers: the input layer has a dimension of 129 (128-dimensional basic features + skin color category encoding + background color encoding); the brightness branch has 4 fully connected layers and outputs two regression values ​​for white and black liquid; the yellow tone branch has 3 fully connected layers and outputs a single value for yellow liquid; the red tone branch has 3 fully connected layers and outputs a single value for red liquid; all hidden layers use ReLU activation function, and the output layer has no activation (linear regression).

[0029] Training dataset: 320,000 labeled data points; input: 129-dimensional skin color features; labels: number of manually calibrated qualified four-color pigment samples; training uses MSE mean squared loss, Adam optimizer, and a learning rate of 2×10⁻⁶. -4 BatchSize=32, Epoch=120; after offline convergence, the weights are frozen, the final feature vector is received online, and the initial addition amount of the four pigments is obtained through parallel three-branch forward inference.

[0030] For the residual correction deep learning network, a color simulation network built into the formula correction module is used, employing a lightweight 8-layer ResNet residual structure. The input consists of four proportions of white / black / yellow / red, and the output is the simulated LAB three-dimensional color value. The training dataset contains 280,000 sets of data, with the input being the four color proportions and the labels being the actual measured LAB values ​​after mixing. The L1 loss function and Adam optimizer are used. The formula parameters are input online, and the color value of the finished product is simulated forward.

[0031] Specifically, the information collection and analysis module first collects multi-dimensional facial images of the user's front and left / right sides. After verifying the light source and pixels, qualified standardized images are selected. Effective skin color pixels are purified through AI segmentation and clustering to construct a precise skin color dataset. Subsequently, the feature extraction module performs multi-dimensional feature extraction and skin color attribute determination based on this dataset, generating the final feature vector. Then, the formula generation module uses a multi-branch deep learning network to calculate the initial ratio of the four color pastes and generate a preliminary formula. Finally, the formula correction module optimizes the formula accuracy and outputs the final standardized formula ratio through deep learning simulation and color difference iterative correction. By leveraging a deep learning model across the entire process, the system completely abandons the outdated model of traditional manual color identification and fixed color matching. It can accurately adapt to the different skin tones, redness, yellowing, and dullness of different users, achieving personalized foundation mixing for each individual. This fundamentally solves the technical pain points of traditional foundations, such as limited shades, poor personalization, large errors in manual color matching, and obvious color differences in finished products. It significantly improves the fit between the foundation and the user's original skin tone. At the same time, the fully automated operation greatly enhances the efficiency and stability of personalized foundation mixing.

[0032] Specifically, the white liquor in the four-color paste is composed of titanium dioxide, other auxiliary reagents, and deionized water; the black liquor is composed of iron oxide black, other auxiliary reagents, and deionized water; the yellow liquor is composed of iron oxide yellow, other auxiliary reagents, and deionized water; and the red liquor is composed of iron oxide red, other auxiliary reagents, and deionized water.

[0033] Specifically, the present invention also includes a Bluetooth module. The system of the present invention can wirelessly connect with existing display devices such as mobile phones and televisions via the Bluetooth module to display the output standardized four-color foundation formula, user skin tone category and base color.

[0034] Understandably, this invention employs the EfficientNetV2-S deep learning feature extraction model to mine multidimensional deep latent features of user skin color, simultaneously identifying user skin color categories and subcutaneous pigment background features. This breaks through the limitations of traditional liquid foundation with fixed and limited shades, accurately capturing the differentiated original skin color appearance features and subcutaneous pigment background differences of different users, achieving exclusive color matching representation for individual skin types, and solving the technical problem that traditional mass-produced foundations cannot adapt to the personalized skin color of thousands of people.

[0035] Understandably, this invention uses the U-Net deep learning semantic segmentation algorithm combined with the GMM clustering screening mechanism to intelligently purify skin color data. This allows the deep learning model to perform feature learning and color matching inference based solely on the user's real and valid skin pixels, effectively avoiding interference from invalid areas such as acne marks, redness, and blemishes on skin color judgment. This ensures the accuracy of identifying differentiated original skin colors and special subcutaneous pigmentation states, and significantly improves the basic accuracy of personalized color matching.

[0036] Understandably, this invention achieves intelligent and differentiated regression calculation of the four-color paste ratio by constructing a multi-branch fully connected deep regression neural network. This enables the deep learning network to autonomously learn the nonlinear mapping relationship between different skin tone categories, different subcutaneous base colors, and the ratios of white, black, yellow, and red liquids. It can adaptively output exclusive ratios for cool white skin, warm yellow skin, dark skin, mixed transition skin, and various subcutaneous pigment base colors, solving the shortcomings of traditional fixed color number adaptation dimension and incomplete coverage of differentiated skin types.

[0037] It is understandable that this invention constructs a closed-loop color difference correction system by introducing a residual correction deep learning network and a DDIM diffusion deep learning iterative color adjustment model. This enables multi-dimensional and targeted correction of personalized color deviation caused by differences in the original skin tone and subcutaneous pigmentation of different users. It effectively solves the technical pain point that traditional fixed-shade foundations cannot accurately compensate for the differences in subcutaneous pigmentation of users, and greatly improves the fit between the foundation and the user's original skin tone.

[0038] In some specific embodiments of the present invention, the information collection and analysis module is configured as follows: The color temperature parameters of the shooting environment are collected in real time. When the color temperature parameters are less than the first color temperature threshold, it is determined that the warm light of the shooting environment is yellowish, and the CLAHE algorithm is used to correct the blue light gain of the image. When the color temperature parameter is greater than the second color temperature threshold, it is determined that the ambient light is bluish and the image is corrected for yellow light gain using the CLAHE algorithm. When the color temperature parameter is between the first color temperature threshold and the second color temperature threshold, the ambient light source is deemed compliant, and the original image is directly retained.

[0039] Specifically, a first color temperature threshold of 5000K and a second color temperature threshold of 7000K are preset. The information acquisition and analysis module reads the values ​​collected by the hardware color temperature sensor in real time. If the color temperature is <5000K, it is determined that the warm light is yellowish and CLAHE blue light gain correction is performed. If the color temperature is >7000K, it is determined that the cool light is bluish and CLAHE yellow light gain correction is performed. If the color temperature is 5000K≤color temperature≤7000K, it is a D65 standard light source and the original image is directly saved. The CLAHE processing flow is as follows: the image is divided into 8×8 local pixel blocks, and the contrast cropping limit factor is 2.0; for warm light < 5000K scenes: the overall pixel gain of the B channel (blue channel) is 5%~12% based on CLAHE equalization; for cool light > 7000K scenes: the overall gain of the G+R channel (yellow tones) is 4%~10%; the gain amplitude is linearly linked to the difference in color temperature from the threshold. After calibration, the image size is uniformly scaled to the fixed input size of the U-Net model, 512×512×3 (RGB), to meet the U-Net network input tensor specification and avoid inference errors caused by size mismatch.

[0040] Understandably, this invention, through fixed threshold hierarchical correction and CLAHE adaptive preprocessing, unifies the color of images input to the U-Net deep learning model to the D65 standard light source benchmark. This eliminates the drawback of incorrect skin color feature extraction caused by ambient light color cast, significantly reduces the systematic offset of LAB data after U-Net segmentation, and ensures the consistency of data benchmarks for subsequent EfficientNetV2-S feature extraction and deep regression matching calculations from the source. This further improves the environmental robustness of the entire deep learning color matching system under different usage scenarios and avoids the defect of inconsistent formulas for users with the same skin color due to differences in shooting environment.

[0041] In some specific embodiments of the present invention, the information collection and analysis module is further configured as follows: Standardized human skin color images are input into a pre-trained U-Net segmentation model. The standardized human skin color images are then subjected to pixel binary classification, dividing them into skin pixels and non-skin pixels, and outputting a skin mask map. The skin mask map is then divided into several pixel clusters using Gaussian Mixture Model (GMM). The LAB color difference between the cluster center and the global skin mean is calculated for each cluster. When the color difference of a single cluster is greater than 12 LAB color difference units, the cluster is determined to be invalid pixels and is removed in batches. The LAB three-dimensional color parameters of all compliant pixels are then extracted to construct a skin color dataset.

[0042] Specifically, the information acquisition and analysis module inputs standardized human skin color images into the pre-trained U-Net segmentation model. The pre-trained U-Net segmentation model performs online inference of details. After inputting a 512×512 RGB corrected image, the encoder downsamples layer by layer to extract multi-scale features, the decoder skips connections to fuse features, and the Sigmoid activation outputs a single-channel mask image. Mask pixels > 0.5 are judged as valid skin areas, and the rest are discarded. Then, GMM clustering is performed: all RGB pixels of the skin corresponding to the mask are extracted, converted to the LAB color space, and then input into EM to iterate GMM, automatically clustering and dividing the data until the change in cluster centers is <10. -6 Stop; calculate the Euclidean color difference ΔE between the cluster center LAB and the average LAB of all skin pixels for each cluster. If ΔE > 12 LAB, the entire cluster is marked as invalid pixels such as acne marks, blemishes, and local redness and deleted; after the removal is completed, extract the L, a, b three-dimensional values ​​of the remaining pixels one by one and summarize them to generate a dataset.

[0043] Understandably, this invention relies on the U-Net deep learning model trained on a massive number of flawed samples to accurately segment skin regions. Combined with GMM clustering with a fixed color difference threshold to filter out flawed pixels, the dataset fed into the EfficientNetV2-S deep learning feature extraction network is free from interfering data such as acne marks, scars, and makeup residue. This avoids flawed pixels causing the deep learning model to incorrectly learn local abnormal skin color features, ensuring that the feature vector closely matches the user's true original skin color. This significantly reduces the matching prediction error of the subsequent multi-branch regression network, effectively solving the pain points of traditional single-point skin measurement devices being unable to remove flaws and having distorted sampling data.

[0044] In some specific embodiments of the present invention, the feature extraction module is configured as follows: A Fitzpatrick six-type skin color database is pre-set. The skin color feature vector is compared with the pre-set Fitzpatrick six-type skin color database in terms of feature similarity, and the skin color type with the highest similarity is selected as the user's skin color category. Simultaneously, the user's skin tone is determined based on the channel weights of the skin tone feature vector. If the weight of the red channel latent feature is greater than 0.5, it is determined to be a reddish skin tone; if the weight of the yellow channel is greater than 0.5, it is determined to be a yellowish skin tone; and if the brightness feature is less than 30, it is determined to be a dark skin tone. The skin tone category and the background color label are concatenated to the skin tone feature vector to obtain the final feature vector.

[0045] Specifically, the feature extraction module maps the purified LAB dataset to corresponding skin color images and feeds them into the pre-trained EfficientNetV2-S deep learning model to extract a 128-dimensional basic feature vector. The features are matched with the Fitzpatrick six-type skin color database using cosine similarity to select the optimal skin color category. The underlying skin color is determined by the feature channel weight threshold (red / yellow channel 0.5, brightness 30), and the labels are encoded and concatenated to the feature vector to obtain the final 129-dimensional feature vector. EfficientNetV2-S uses transfer learning to complete offline fine-tuning and outputs features online only through forward inference.

[0046] The EfficientNetV2-S deep learning model of this invention consists of 7 MBConv modules (Stages 1-7) cascaded together, featuring a lightweight convolutional structure, removal of the original classification head, and the addition of a global average pooling layer and a custom fully connected layer at the end of the network to output a 128-dimensional floating-point feature vector. In transfer learning training, the base weights are taken from ImageNet1K pre-trained weights, the convolutional weights of the first 5 stages are frozen, and fine-tuning is performed using a self-built Fitzpatrick expanded skin color dataset (250,000 samples, each bound to a skin color category label, subcutaneous background color label, and corresponding LAB data). The training objective is a combination of feature space clustering loss and classification joint loss, with the AdamW optimizer and a weight decay of 1×10⁻⁶. -5 Learning rate cosine annealing decay; Epoch=60, fine-tuning terminated when validation set classification accuracy > 95%; During online inference, the normalized image corresponding to the user's skin color dataset is fed into the network, and forward propagation yields 128-dimensional basic features; cosine similarity calculation: the feature vector is cosine similar to each of the six standard features of Fitzpatrick. The maximum value corresponds to the skin color category; the channel weight is the mean of the output feature channels of the intermediate layer of the model. The red channel weight > 0.5 = reddish background, the yellow channel weight > 0.5 = yellowish background, and the brightness feature < 30 = dark skin color. The skin color category and the background color label are concatenated at the end of the 128-dimensional vector after one-hot encoding to generate the final 129-dimensional feature vector.

[0047] Understandably, the EfficientNetV2-S deep learning model, optimized through transfer learning, can uncover deep latent features of subcutaneous pigmentation that cannot be represented by the surface LAB. By combining similarity matching and weighted threshold classification, it achieves dual-dimensional labeling of skin color category and subcutaneous background color. The labels are incorporated into the feature vector to expand the input dimension, enabling the downstream multi-branch deep regression neural network to learn the mapping rules of different ratios corresponding to different subcutaneous background colors of the same LAB. This completely breaks through the limitations of traditional devices that rely solely on fixed color matching of the surface LAB. It automatically outputs differentiated pigment ratios for users with similar skin tones but different subcutaneous pigments, achieving a personalized color matching foundation for each individual from the feature input end.

[0048] In some specific embodiments of the present invention, the recipe generation module is configured as follows: The final feature vector is input into a multi-branch fully connected deep regression neural network, which includes a brightness control branch, a yellow tone control branch, and a red tone control branch. The brightness control branch regresses the basic addition amount of white liquid and black liquid based on the skin color LAB brightness L value in the final feature vector. The L value is positively correlated with the addition amount of white liquid and negatively correlated with the addition amount of black liquid. The yellow-toning regulation branch regresses the basic amount of yellow liquid added based on the b value of the LAB yellow axis in the final feature vector, and the b value is positively correlated with the amount of yellow liquid added. The red tone control branch regresses the basic amount of red liquid added based on the LAB red axis 'a' value in the final feature vector, and the 'a' value is positively correlated with the amount of red liquid added.

[0049] Specifically, the multi-branch fully connected deep regression neural network of the formula generation module receives a 129-dimensional final feature vector, which is split into three independent fully connected branches: brightness control, yellow tone control, and red tone control. The brightness branch regresses the amount of white and black liquor (L is positively correlated with white and negatively correlated with black). The yellow tone branch regresses the yellow liquor from the b value (positively correlated). The red tone branch regresses the red liquor from the a value (positively correlated), and outputs the initial number of four-color pastes. This network is a dedicated regression deep learning model trained by supervised learning, which completes convergence offline and then performs fixed-weight inference online.

[0050] The network topology of this multi-branch fully connected deep regression neural network is as follows: Shared input layer (129-dimensional floating-point vector); Brightness branch: FC(129→64)→ReLU→FC(64→32)→ReLU→FC(32→2), output: number of white liquor components, number of black liquor components; Yellow tone branch: FC(129→48)→ReLU→FC(48→16)→ReLU→FC(16→1), output: number of yellow liquid portions; Red branch: FC(129→48)→ReLU→FC(48→16)→ReLU→FC(16→1), Output: number of red liquid portions; The supervised training process is as follows: The training set contains 320,000 entries; input: 129-dimensional labeled features; labels: the optimal four-color pulp content determined through repeated manual trials; the loss function is multi-output joint MSE loss. ; Optimizer Adam, initial learning rate 2×10 -4 With BatchSize=32 and Epoch=120, training stops when the validation set loss does not decrease for 20 consecutive rounds. The training process automatically learns the positive and negative correlation mapping between Lab and four-color paste, without the need for manual input of fixed calculation formulas. The online inference process is as follows: 129-dimensional features are fed into three branches in parallel for one forward operation, and four initial quality scores are directly output. The model internally relies on the training weights to naturally achieve the association constraints that the higher L is, the higher the white and the lower the black, the higher b is, the higher the yellow and the higher a is, the higher the red.

[0051] Understandably, a three-branch decoupled deep regression deep learning architecture is adopted, which separates the brightness color adjustment and red-yellow color adjustment into independent sub-networks for fitting, avoiding multi-variable coupling interference from a single fully connected network. After supervised training with massive labeled samples, the network autonomously learns the non-linear relationship between skin color features and the amount of colorant used, replacing manual experience and fixed empirical formulas. The output initial formula matches the user's basic skin color, greatly reducing the correction range of subsequent DDIM iteration correction, and balancing the accuracy of the initial formula preparation and computational efficiency.

[0052] In some specific embodiments of the present invention, the formula correction module is further configured as follows: After the residual correction deep learning network simulates and generates the foundation simulation LAB color value, it calculates the color difference between the simulation LAB color value and the user's skin tone. When the color difference is less than or equal to the first color difference threshold, it is judged as a slight color difference, and no iterative correction is performed. The standardized four-color foundation formula is directly output. When the color difference is greater than the first color difference threshold, the DDIM diffusion model is started to perform reverse iterative color adjustment. When performing reverse iterative color adjustment, the maximum number of iterations of the DDIM model is set to 10. If the maximum number of iterations is not reached and the color difference does not meet the standard, the color paste ratio is continuously corrected in reverse. After the standard is met, the corrected color paste formula is re-inspected. If the standard is still not met after reaching the maximum number of iterations, it is determined to be an abnormal feature matching, and the optimal formula during iterative color adjustment is used for a second color difference re-inspection.

[0053] Specifically, the first color difference threshold is preset to 1.2. The formula correction module simulates the formula LAB color value through the residual correction deep learning network. If ΔE ≤ 1.2 (first color difference threshold), the formula is directly finalized. If ΔE > 1.2, the DDIM diffusion deep learning model is started to iterate and adjust the color in reverse. The maximum iteration limit of DDIM is 10 times. If the color difference does not meet the standard after 10 iterations, the iteration stops. If the formula is still unqualified after 10 iterations, the optimal formula is selected for a second review. The residual network is responsible for color simulation, and DDIM is responsible for reverse optimization of the ratio. Both types of models are trained offline under supervision.

[0054] The training and inference process of the residual correction deep learning network is as follows: input 4-dimensional data (white, black, yellow, and red values), output 3D simulation LAB; the network consists of 4 sets of residual blocks connected in series, each residual block containing two fully connected layers and shorted residual links; the training dataset contains 280,000 sets, with the input being the actual measured proportions and the labels being the actual LAB measured by the spectrometer after mixing; the loss is L1 loss, the Adam optimizer is used, and the learning rate is 3×10. -4 The initial ratio is input online, and the simulation LAB is output through forward inference. The simulation LAB is then compared with the user's native LAB to calculate ΔE. The training and inference process of the DDIM discrete diffusion model is as follows: Total diffusion steps T=50, forward diffusion: Gaussian noise is gradually added to the standard ratio LAB to generate noisy samples; backward inference limits the maximum sampling steps to 10 (the maximum iteration to match the claims is 10); the training dataset has 220,000 sets, and the sample format is {initial ratio, target LAB, optimal corrected ratio}; the training objective is to learn the noise reduction rule of reconstructing the optimal ratio from the deviation ratio using color difference deviation as a guiding condition; after offline training is completed, in the online stage: when ΔE>1.2, DDIM starts backward sampling to optimize the four color paste values ​​with the user's target LAB as a constraint, and completes one round of ratio correction in a single iteration. The residual network synchronously recalculates the simulated color difference; the iteration counter increments automatically, and the iteration is forcibly terminated when the count reaches 10. The iteration history is traversed and the ratio with the smallest ΔE is taken as the optimal formula.

[0055] Understandably, residual deep learning networks are trained on massive amounts of real mixed samples to accurately simulate the color development rules of toner mixing, eliminating the need for physical sample testing; DDIM generative deep learning relies on the global optimization capability of the diffusion model to reverse the ratio. Compared with the traditional fine-tuning method of small-scale random adjustments in one direction, the iterative convergence speed is improved by 60%, and the upper limit constraint of 10 iterations avoids infinite loop calculations. While ensuring the accuracy of formula optimization, it also constrains the system's computing power consumption, further reducing the color difference between the formula and the original skin tone.

[0056] In some specific embodiments of the present invention, the formula correction module is further configured as follows: When performing reverse iterative color adjustment, the color difference of each dimension is split into L, a, and b, and the weight ratio of each dimension to the total color difference is calculated. The single dimension with a weight ratio of 60% or more is determined to be the dominant color difference dimension, and the rest are secondary dimensions. If dimension L is the dominant color difference, only adjust the ratio of white and black liquor, while keeping the values ​​of yellow and red liquor unchanged; if dimension a is the dominant color difference, only adjust the ratio of red liquor, while keeping the values ​​of the other three color pastes unchanged; if dimension b is the dominant color difference, only slightly adjust the ratio of yellow liquor, while keeping the values ​​of the other three color pastes unchanged; if no dimension accounts for more than or equal to 60%, it is determined to be a multi-dimensional coupled color difference, and a balance adjustment is made, while simultaneously making small adjustments to the ratios of the four color pastes.

[0057] Specifically, during each DDIM iteration, the color difference is separated into three independent dimensions: L, a, and b. A color difference in each dimension accounting for ≥60% of the total ΔE weight is considered the dominant dimension. If L is dominant, only black and white liquids are adjusted; if a is dominant, only red liquids are adjusted; and if b is dominant, only yellow liquids are adjusted. If no dimension is ≥60%, all four colors are simultaneously balanced and fine-tuned. Before each DDIM iteration, the system separates the total color difference. Single-dimensional weight W a W b Similarly, after comparing the weights with the 60% threshold, constraint instructions are generated and sent to the DDIM backsampling module. W L When the DDIM optimization space is ≥60%, only the adjustment range of the parameters of white liquor and black liquor is open, while the parameters of yellow liquor and red liquor are fixed and do not participate in gradient update. W a When the concentration is ≥60%, only the red liquor parameter optimization is enabled, while the parameters for the other three color pastes are frozen. W b When the concentration is ≥60%, only the yellow liquid parameter optimization is enabled; When all weights are less than 60%, all parameters of the four-color paste participate in gradient optimization and are simultaneously slightly corrected.

[0058] This constraint only applies to online inference. There are no dimensional constraints during the offline training phase of DDIM, ensuring that the model's basic correction rules are learned completely.

[0059] Understandably, by constraining the parameter optimization range of the DDIM deep learning model through color difference weight threshold, targeted correction of color deviation can be achieved. This abandons the traditional color adjustment logic of random adjustment of all components without difference, effectively avoiding the problem of causing new color deviations in other dimensions when correcting a single dimension. This makes DDIM iterative correction more targeted, and the color difference reduction is increased under the same number of iterations, further shortening the formula convergence cycle.

[0060] In some specific embodiments of the present invention, the formula correction module is further configured as follows: A second color difference threshold is preset, and the second color difference threshold is greater than the first color difference threshold. When performing a second color difference re-inspection, the corrected formula is input into the residual correction deep learning network to generate the LAB color value of the formula and recalculate the color difference. When the color difference is less than or equal to the first color difference threshold, the color matching is deemed qualified and the corresponding formula is used as the standardized four-color paste foundation mixing formula. When the color difference is greater than the first color difference threshold but less than or equal to the second color difference threshold, it is judged as a slight color difference. After single-component fine-tuning, the fine-tuned formula is used as the standardized four-color paste foundation formula. When the color difference is greater than the second color difference threshold, it is judged as a color difference abnormality, and the user's multi-dimensional face image is re-acquired.

[0061] Specifically, the optimized ratio after DDIM iteration is used again as the input tensor of the residual correction network. The model freezes the weights and outputs a new version of the simulation LAB. The system recalculates ΔE: when ΔE≤1.2, the formula is qualified and archived; when 1.2<ΔE≤1.5, it jumps to the micro-adjustment program; when ΔE>1.5, it returns to the front-end image acquisition module to restart the entire process.

[0062] Understandably, the standardized secondary simulation verification is completed by relying on a residual deep learning model trained on massive amounts of data. The three-level processing logic is divided with a fixed color difference threshold, replacing manual visual inspection. The verification standard is unified and there is no subjective error. The graded handling mechanism takes into account both formula accuracy and mixing efficiency. Minor color differences are fine-tuned and repaired, and severe color differences are re-sampling. The process blocks the output of unqualified formulas and further ensures that the final color matches the user's original skin tone.

[0063] In some specific embodiments of the present invention, the formula correction module is further configured as follows: The maximum adjustment range is preset. When performing single-component fine-tuning, the color difference of L, a, and b is separated into single-dimensional color differences, and the dimension with the highest proportion is selected as the dimension to be corrected. When the dimension to be corrected is L: When the color difference in dimension L is greater than 0, increase the amount of black liquor according to the color difference value, with the maximum increase not exceeding the maximum adjustment range; when the color difference in dimension L is less than 0, increase the amount of white liquor according to the color difference value, with the maximum increase not exceeding the maximum adjustment range. When the dimension to be corrected is 'a': When the color difference in dimension 'a' is greater than 0, reduce the amount of red liquid according to the color difference value, with the maximum reduction not exceeding the maximum adjustment range; when the color difference in dimension 'a' is less than 0, increase the amount of red liquid according to the color difference value, with the maximum increase not exceeding the maximum adjustment range. When the dimension to be corrected is b: When the color difference of dimension b is greater than 0, reduce the amount of yellow liquid according to the color difference value, with the maximum reduction not exceeding the maximum adjustment range; when the color difference of dimension b is less than 0, increase the amount of yellow liquid according to the color difference value, with the maximum increase not exceeding the maximum adjustment range.

[0064] Specifically, the maximum single adjustment range in the single-component fine-tuning stage is preset to 0.3 parts by mass; the L / a / b color difference is split and the dimension with the highest weight is selected: L is whiter and blacker, L is darker and whiter; a is too high and red is reduced, a is too low and red is added; b is too high and yellow is reduced, b is too low and yellow is added. All adjustment ranges are ≤0.3 parts, and the final version is directly made after fine-tuning.

[0065] Understandably, by using a standardized fine-tuning rule with an upper limit of 0.3 parts, small color differences can be accurately corrected without altering the core proportions output by the deep learning model. This avoids excessive fine-tuning from disrupting the basic color matching rules learned by the deep learning model, achieving final accuracy optimization at low cost and high efficiency, and further reducing the color difference between the finished product and the user's skin tone.

[0066] like Figure 2 As shown, in some embodiments of the present invention, a personalized foundation mixing method based on deep learning is also provided, including the following steps: Multi-dimensional facial images of users are collected. The effective pixels of the collected facial images are verified according to the light source and color temperature of the collection environment. Qualified standardized human skin color images are selected. A pre-trained U-Net deep learning segmentation model is used to perform semantic segmentation of the skin region of the standardized human skin color images. Invalid interference pixels are removed by Gaussian mixture clustering algorithm and color difference analysis. The LAB color parameters of the effective skin pixels of users are extracted to construct the user skin color dataset. The constructed user skin color dataset is input into the pre-trained EfficientNetV2-S deep learning feature extraction model. Multi-dimensional latent features of skin color brightness, warm and cool tones, and subcutaneous pigment are extracted layer by layer to generate skin color feature vectors. Based on the skin color feature vectors, the user's skin color category and subcutaneous background color are determined, and the feature vectors are optimized and improved to generate the final feature vector. The final feature vector is input into a multi-branch fully connected deep regression neural network, and the initial addition amounts of white liquid, black liquid, yellow liquid and red liquid are obtained by regressing the features corresponding to brightness, yellow tone and red tone respectively, thus generating the preliminary formula of foundation liquid; Based on the preliminary foundation formula generated by the formula generation module, the simulated LAB color value of the foundation is generated through residual correction deep learning network simulation. The color difference between the simulated LAB color value and the user's original skin tone is obtained. Based on the color difference, the amount of four color pastes added is adjusted to obtain a standardized four-color paste foundation formula that is suitable for the user's skin tone.

[0067] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program goods. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program goods embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0068] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program goods according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0069] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0070] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A personalized foundation mixing system based on deep learning, characterized in that, include: The information collection and analysis module is configured to collect multi-dimensional facial images of users, verify the effective pixels of the collected facial images based on the light source and color temperature of the collection environment, select qualified standardized human skin color images, use a pre-trained U-Net deep learning segmentation model to perform semantic segmentation of skin regions on the standardized human skin color images, remove invalid interference pixels through GMM Gaussian mixture clustering algorithm and color difference analysis, extract the LAB color parameters of the user's effective skin pixels, and construct the user skin color dataset. The feature extraction module is configured to input the user skin color dataset constructed by the information collection and analysis module into the pre-trained EfficientNetV2-S deep learning feature extraction model, extract skin color brightness, warm and cool tones, and subcutaneous pigment multidimensional latent features layer by layer, generate skin color feature vector, determine the user's skin color category and subcutaneous background color based on the skin color feature vector, optimize and improve the feature vector, and generate the final feature vector. The formula generation module is configured to input the final feature vector into a multi-branch fully connected deep regression neural network, and respectively regress the features of brightness, yellow tone, and red tone to obtain the initial addition amount of white liquid, black liquid, yellow liquid, and red liquid, thereby generating a preliminary formula for foundation liquid; The formula correction module is configured to generate a simulated LAB color value for the foundation liquid based on the preliminary formula generated by the formula generation module through a residual correction deep learning network, obtain the color difference between the simulated LAB color value and the user's original skin tone, and correct the amount of four color pastes added based on the color difference to obtain a standardized four-color paste foundation formula that is suitable for the user's skin tone.

2. The personalized foundation mixing system based on deep learning according to claim 1, characterized in that, The information collection and analysis module is configured as follows: The color temperature parameters of the shooting environment are collected in real time. When the color temperature parameters are less than the first color temperature threshold, it is determined that the warm light of the shooting environment is yellowish, and the CLAHE algorithm is used to correct the blue light gain of the image. When the color temperature parameter is greater than the second color temperature threshold, it is determined that the ambient light is bluish and the image is corrected for yellow light gain using the CLAHE algorithm. When the color temperature parameter is between the first color temperature threshold and the second color temperature threshold, the ambient light source is deemed compliant, and the original image is directly retained.

3. The personalized foundation mixing system based on deep learning according to claim 2, characterized in that, The information collection and analysis module is also configured to: Standardized human skin color images are input into a pre-trained U-Net segmentation model. The standardized human skin color images are then subjected to pixel binary classification, dividing them into skin pixels and non-skin pixels, and outputting a skin mask map. The skin mask map is then divided into several pixel clusters using Gaussian Mixture Model (GMM). The LAB color difference between the cluster center and the global skin mean is calculated for each cluster. When the color difference of a single cluster is greater than 12 LAB color difference units, the cluster is determined to be invalid pixels and is removed in batches. The LAB three-dimensional color parameters of all compliant pixels are then extracted to construct a skin color dataset.

4. The personalized foundation mixing system based on deep learning according to claim 3, characterized in that, The feature extraction module is configured as follows: A Fitzpatrick six-type skin color database is pre-set. The skin color feature vector is compared with the pre-set Fitzpatrick six-type skin color database in terms of feature similarity, and the skin color type with the highest similarity is selected as the user's skin color category. Simultaneously, the user's skin tone is determined based on the channel weights of the skin tone feature vector. If the weight of the red channel latent feature is greater than 0.5, it is determined to be a reddish skin tone; if the weight of the yellow channel is greater than 0.5, it is determined to be a yellowish skin tone; and if the brightness feature is less than 30, it is determined to be a dark skin tone. The skin tone category and the background color label are concatenated to the skin tone feature vector to obtain the final feature vector.

5. The personalized foundation mixing system based on deep learning according to claim 4, characterized in that, The recipe generation module is configured as follows: The final feature vector is input into a multi-branch fully connected deep regression neural network, which includes a brightness control branch, a yellow tone control branch, and a red tone control branch. The brightness control branch regresses the basic addition amount of white liquid and black liquid based on the skin color LAB brightness L value in the final feature vector. The L value is positively correlated with the addition amount of white liquid and negatively correlated with the addition amount of black liquid. The yellow-toning regulation branch regresses the basic amount of yellow liquid added based on the b value of the LAB yellow axis in the final feature vector, and the b value is positively correlated with the amount of yellow liquid added. The red tone control branch regresses the basic amount of red liquid added based on the LAB red axis 'a' value in the final feature vector, and the 'a' value is positively correlated with the amount of red liquid added.

6. The personalized foundation mixing system based on deep learning according to claim 5, characterized in that, The formula correction module is also configured to: After the residual correction deep learning network simulates and generates the foundation simulation LAB color value, it calculates the color difference between the simulation LAB color value and the user's skin tone. When the color difference is less than or equal to the first color difference threshold, it is judged as a slight color difference, and no iterative correction is performed. The standardized four-color foundation formula is directly output. When the color difference is greater than the first color difference threshold, the DDIM diffusion model is started to perform reverse iterative color adjustment. When performing reverse iterative color adjustment, the maximum number of iterations of the DDIM model is set to 10. If the maximum number of iterations is not reached and the color difference does not meet the standard, the color paste ratio is continuously corrected in reverse. After the standard is met, the corrected color paste formula is re-inspected. If the maximum number of attempts is not reached and the target is still not met, it is determined to be an abnormal feature matching, and the optimal formula used in the iterative color adjustment is used for a second color difference re-inspection.

7. The personalized foundation mixing system based on deep learning according to claim 6, characterized in that, The formula correction module is also configured to: When performing reverse iterative color adjustment, the color difference of each dimension is split into L, a, and b, and the weight ratio of each dimension to the total color difference is calculated. The single dimension with a weight ratio of 60% or more is determined to be the dominant color difference dimension, and the rest are secondary dimensions. If dimension L is the dominant color difference, only adjust the ratio of white and black liquor, while keeping the values ​​of yellow and red liquor unchanged; if dimension a is the dominant color difference, only adjust the ratio of red liquor, while keeping the values ​​of the other three color pastes unchanged; if dimension b is the dominant color difference, only slightly adjust the ratio of yellow liquor, while keeping the values ​​of the other three color pastes unchanged; if no dimension accounts for more than or equal to 60%, it is determined to be a multi-dimensional coupled color difference, and a balance adjustment is made, while simultaneously making small adjustments to the ratios of the four color pastes.

8. The personalized foundation mixing system based on deep learning according to claim 7, characterized in that, The formula correction module is also configured to: A second color difference threshold is preset, and the second color difference threshold is greater than the first color difference threshold. When performing a second color difference re-inspection, the corrected formula is input into the residual correction deep learning network to generate the LAB color value of the formula and recalculate the color difference. When the color difference is less than or equal to the first color difference threshold, the color matching is deemed qualified and the corresponding formula is used as the standardized four-color paste foundation mixing formula. When the color difference is greater than the first color difference threshold but less than or equal to the second color difference threshold, it is judged as a slight color difference. After single-component fine-tuning, the fine-tuned formula is used as the standardized four-color paste foundation formula. When the color difference is greater than the second color difference threshold, it is judged as a color difference abnormality, and the user's multi-dimensional face image is re-acquired.

9. The personalized foundation mixing system based on deep learning according to claim 8, characterized in that, The formula correction module is also configured to: The maximum adjustment range is preset. When performing single-component fine-tuning, the color difference of L, a, and b is separated into single-dimensional color differences, and the dimension with the highest proportion is selected as the dimension to be corrected. When the dimension to be corrected is L: When the color difference in dimension L is greater than 0, increase the amount of black liquor according to the color difference value, with the maximum increase not exceeding the maximum adjustment range; when the color difference in dimension L is less than 0, increase the amount of white liquor according to the color difference value, with the maximum increase not exceeding the maximum adjustment range. When the dimension to be corrected is 'a': When the color difference in dimension 'a' is greater than 0, reduce the amount of red liquid according to the color difference value, with the maximum reduction not exceeding the maximum adjustment range; when the color difference in dimension 'a' is less than 0, increase the amount of red liquid according to the color difference value, with the maximum increase not exceeding the maximum adjustment range. When the dimension to be corrected is b: When the color difference of dimension b is greater than 0, reduce the amount of yellow liquid according to the color difference value, with the maximum reduction not exceeding the maximum adjustment range; when the color difference of dimension b is less than 0, increase the amount of yellow liquid according to the color difference value, with the maximum increase not exceeding the maximum adjustment range.

10. A personalized foundation mixing method based on deep learning, characterized in that, The personalized foundation mixing system based on deep learning as described in any one of claims 1-9 includes the following steps: Multi-dimensional facial images of users are collected. The effective pixels of the collected facial images are verified according to the light source and color temperature of the collection environment. Qualified standardized human skin color images are selected. A pre-trained U-Net deep learning segmentation model is used to perform semantic segmentation of the skin region of the standardized human skin color images. Invalid interference pixels are removed by Gaussian mixture clustering algorithm and color difference analysis. The LAB color parameters of the effective skin pixels of users are extracted to construct the user skin color dataset. The constructed user skin color dataset is input into the pre-trained EfficientNetV2-S deep learning feature extraction model. Multi-dimensional latent features of skin color brightness, warm and cool tones, and subcutaneous pigment are extracted layer by layer to generate skin color feature vectors. Based on the skin color feature vectors, the user's skin color category and subcutaneous background color are determined, and the feature vectors are optimized and improved to generate the final feature vector. The final feature vector is input into a multi-branch fully connected deep regression neural network, and the initial addition amounts of white liquid, black liquid, yellow liquid and red liquid are obtained by regressing the features corresponding to brightness, yellow tone and red tone respectively, thus generating the preliminary formula of foundation liquid; Based on the preliminary foundation formula generated by the formula generation module, the simulated LAB color value of the foundation is generated through residual correction deep learning network simulation. The color difference between the simulated LAB color value and the user's original skin tone is obtained. Based on the color difference, the amount of four color pastes added is adjusted to obtain a standardized four-color paste foundation formula that is suitable for the user's skin tone.

Images