An Automatic Evaluation Method and System for the Efficacy of Cosmetics Based on Facial Spectrum Sensing

Abstract

The present invention relates to the field of artificial intelligence technology for cosmetics, and specifically relates to a method and system for automatically evaluating the efficacy of cosmetics based on facial spectrum perception, which are as follows: Collect the facial spectra of the subjects before and after using cosmetics, and conduct real scoring on the two facial spectra, and construct a dataset with skin negative sample spectra; Perform affine transformation on the data in the dataset to obtain the skin spectra before and after using cosmetics respectively, extract the features of the negative sample spectra in the dataset to obtain the negative sample spectrum features; Construct a cosmetics efficacy evaluation model, input the skin spectra before and after using cosmetics and the negative sample spectrum features into the cosmetics efficacy evaluation model to obtain the predicted score of the facial spectrum after using cosmetics; After optimizing the model, input the facial spectrum to be evaluated into the model for evaluation. The present invention can meet the industry's demand for fast and accurate evaluation tools, and can also provide important support for personalized skin care and cosmetics research and development.

Description

Technical Field

[0001] The present invention relates to the technical field of cosmetics artificial intelligence, and particularly to an automatic evaluation method and system for cosmetics efficacy based on facial spectrum perception. Background Art

[0002] In recent years, with the rapid development of the cosmetics industry and the continuous improvement of consumers' demand for personalized skin care, how to scientifically and accurately evaluate the efficacy of cosmetics has become an important direction for cosmetics R & D and consumer selection. Traditional methods for evaluating cosmetics efficacy mainly rely on laboratory tests and manual subjective evaluations, such as judging by skin experts' observations or consumers' usage feedback. However, these methods are usually time-consuming, costly, and affected by personal subjective factors, resulting in certain uncertainties and limitations in the evaluation results.

[0003] With the continuous progress of artificial intelligence technology and spectral analysis technology, skin state analysis based on spectral perception has gradually become a hot research direction in the field of cosmetics efficacy evaluation. Spectral perception technology can non-invasively and quickly obtain multi-dimensional information of the skin, such as moisture content, oil distribution, pigmentation, pore state, etc., by capturing the optical characteristics of the skin surface and deep layers. These data can provide a more scientific and objective basis for cosmetics efficacy evaluation. At the same time, the addition of artificial intelligence methods makes it possible to process and analyze a large amount of skin spectral data. By constructing a data-driven evaluation model, the accuracy and efficiency of the evaluation can be greatly improved.

[0004] However, at present, the evaluation of cosmetics efficacy based on spectral perception still faces many challenges in practical applications. For example, facial skin spectral data has complexity and diversity, and differences in skin characteristics of different individuals, environmental lighting conditions, and measurement devices will all affect the evaluation results. In addition, how to extract key features from multi-source and multi-dimensional spectral data and construct an evaluation model with both robustness and generality is also an urgent problem to be solved.

[0005] Therefore, the present invention proposes an automatic evaluation method and system for cosmetics efficacy based on facial spectrum perception to solve the above problems. Summary of the Invention

[0006] Aiming at the deficiencies of the prior art, the present invention develops an automatic evaluation method and system for cosmetics efficacy based on facial spectrum perception. The present invention can meet the industry's demand for fast and accurate evaluation tools, and can also provide important support for personalized skin care and cosmetics R & D.

[0007] The technical solution for the present invention to solve the technical problem is an automatic evaluation method for cosmetics efficacy based on facial spectrum perception, including the following steps:

[0008] S1. Construct Dataset: Collect facial spectra of subjects before and after using cosmetics. The facial spectra include the facial spectra before using cosmetics and the facial spectra after using cosmetics. Then, score the two types of facial spectra separately, and construct the facial spectra, the obtained true scores, and typical skin negative sample spectra on the Internet Dataset; S2. For Preprocess the dataset: For In the dataset, perform affine transformation on the facial spectra by their facial feature points to obtain the skin spectra before using cosmetics And the skin spectra after using cosmetics , for Extract features from the negative sample spectra in the dataset to obtain negative sample spectral features ;

[0009] S3. Construct a cosmetic efficacy evaluation model. The model includes a benchmark spectral feature extractor, a negative sample encoder, a negative sample weight calculator, spectral perception attention, a changed spectral feature extractor, a skin tone evaluator, a fineness evaluator, and a glossiness evaluator. Input the skin spectra before using cosmetics , the skin spectra after using cosmetics And the negative sample spectral features Into the cosmetic efficacy evaluation model to obtain the predicted scores of the facial spectra after using cosmetics;

[0010] S4. Calculate the loss between the true scores and the predicted scores through a loss function, and iterate the parameters in the cosmetic efficacy evaluation model through an Adam optimizer to obtain a trained cosmetic efficacy evaluation model;

[0011] S5. Input the facial spectra after using cosmetics to be evaluated into the trained cosmetic efficacy evaluation model to obtain the final predicted scores.

[0012] S1 is specifically as follows:

[0013] S1.1. The subjects are volunteers of different ages, genders, and skin types recruited. The skin types of the subjects include oily, dry, combination, and sensitive, and the subjects have no obvious skin diseases or facial wounds;

[0014] S1.2. Collect facial spectra through a high-precision multispectral imaging device. The device supports multi-band collection in the range of 400nm - 1000nm, with a resolution of 1920×1080 pixels and a spectral band interval of no more than 10nm;

[0015] S1.3. When collecting facial spectra, use a standard D65 light source, set a pure black background, and the indoor temperature is 22 2°C, relative humidity is 50 10%, after the subjects wash their faces, they wait for 30 minutes to ensure that the facial skin is in a natural state. Two spectral data are collected, respectively before using cosmetics and 30 days after continuous use of cosmetics. The specific cosmetics used are skin care products in cosmetics;

[0016] S1.4. The scores for the facial spectrum are skin color score, fineness score and glossiness score. Here, the score is the true score of the facial spectrum;

[0017] Among them, the skin color score is based on the L value in the Lab color space, calculating the pink uniformity and brightness, and the scoring range is 1 - 100; the fineness score is based on the standard deviation of facial texture features, extracting skin roughness parameters through image processing methods, and the scoring range is 1 - 100; the glossiness score is based on the light reflectance in the 500nm - 600nm band of the spectrogram; using a mathematical model to calculate the facial glossiness, and the scoring range is 1 - 100; different experts score the subjects, and the average value is taken as the true score after removing the highest and lowest scores;

[0018] S1.5. Collect multiple publicly available skin negative sample data on the Internet, including severe pigmentation, obvious enlarged pores, and dull and dry skin problems, and standardize all skin negative sample spectra.

[0019] S2 is specifically as follows:

[0020] S2.1. For the facial spectra in the dataset, use the 68 - point model of the open - source Dlib library to extract the key features of the subjects' faces, covering positions including the corners of the eyes, the tip of the nose, the corners of the mouth, and the facial boundary positions. Use affine transformation to transform the facial spectra. Specifically, complete the affine transformation through the affine function to output the aligned image, and store the spectral data of it as vectors and , represents the skin spectrum before using cosmetics, represents the skin spectrum after using cosmetics;

[0021] S2.2. For the skin negative sample spectra in the dataset, use the trained auto - encoder model to extract their features and obtain the negative sample spectral features .

[0022] S3 is specifically as follows:

[0023] The cosmetic efficacy evaluation model includes a benchmark spectral feature extractor, a negative sample encoder, a negative sample weight calculator, spectral perception attention, a variable spectral feature extractor, a skin color evaluator, a fineness evaluator, and a glossiness evaluator;

[0024] S3.1. The benchmark spectral feature extractor sequentially includes a first convolutional module, a second convolutional module, a third convolutional module, and an average pooling layer. The first convolutional module sequentially includes a convolutional layer with a convolution kernel of 3×3 and a stride of 1, a batch normalization layer, and an activation function layer; the second convolutional module sequentially includes a convolutional layer with a convolution kernel of 1×1 and a stride of 1; the third convolutional module sequentially includes a convolutional layer with a convolution kernel of 5×5 and a stride of 2, a batch normalization layer, and an activation function layer;

[0025] Input into the benchmark spectral feature extractor, and obtain feature through the first convolutional block. Then input into the second convolutional block to obtain feature . Then input and into the third convolutional block to obtain feature . Add and and input the sum into the average pooling layer to obtain the benchmark spectral feature ;

[0026] S3.2. Normalize the negative sample spectral feature , and the calculation process is as follows:

[0027] ,

[0028] where and respectively represent the mean and standard deviation of the dimensions of the negative sample spectral feature , and represents the normalized negative sample spectral feature;

[0029] S3.3. The negative sample encoder is essentially a fully connected neural network, including a first hidden layer, a second hidden layer, and a third hidden layer. Among them, the first hidden layer includes a fully connected layer with an output dimension of 128 and an activation function of ; the second hidden layer includes a fully connected layer with an output dimension of 32 and an activation function of ; the third hidden layer includes a fully connected layer with an output dimension of 16 and no activation function;

[0030] Input Input into the negative sample encoder, passing through the first hidden layer, the second hidden layer, and the third hidden layer in sequence to obtain the negative sample encoded features , , denotes the dimension of the negative sample encoded features . denotes the -th dimensional feature in the negative sample encoded features ;

[0031] S3.4. Input the features of different dimensions in the negative sample encoded features into the negative sample weight calculator to obtain the negative sample weight features , and the calculation formula is as follows:

[0032] ,

[0033] ,

[0034] ,

[0035] wherein, denotes the activation function, denotes the -th dimensional feature and the -th dimensional feature in the negative sample encoded features , , and are both indices of , denotes the linear weight of the -th dimensional feature , denotes the covariance between the feature and the feature , denotes the product of the standard deviations of the feature and the feature , denotes the variance of the feature , denotes the variance of the feature

[0036] S3.5. Input the reference spectrum features and the negative sample weight features The spectral perception features are calculated through the spectral perception attention formula , and the calculation formula is as follows:

[0037] ,

[0038] wherein, represents the first-layer weight matrix in the spectral perception attention formula, represents the second-layer weight matrix in the spectral perception attention formula, represents the bias term of the first layer in the spectral perception attention formula, represents the bias term of the second layer in the spectral perception attention formula;

[0039] S3.6. The structure of the variable spectral feature extractor is the same as that of the reference spectral feature extractor, and their weights are updated independently. Input into the variable spectral feature extractor to obtain the variable spectral features .

[0040] S3.7. The skin color evaluator, the fineness evaluator, and the glossiness evaluator have the same structure, all of which include a first fully connected layer, a second fully connected layer, and a third fully connected layer. The weights of the three evaluators are updated independently;

[0041] wherein, the output dimension of the first fully connected layer is 64, and the ReLU activation function is used; the output dimension of the second fully connected layer is 8, and the ReLU activation function is used; the output dimension of the third fully connected layer is 1, and no activation function is used;

[0042] Add the spectral perception features and the variable spectral features and input them into the skin color evaluator, the fineness evaluator, and the glossiness evaluator respectively to obtain the predicted skin color score , the predicted fineness score , and the predicted glossiness score .

[0043] S4 is specifically as follows:

[0044] Calculate the loss function of the predicted score and the true score , and the calculation formula is as follows:

[0045] ,

[0046] wherein, , represents the total number of samples participating in the training, represents index;

[0047] Set the learning rate of the Adam optimizer to 0.01, the first-order momentum decay coefficient to 0.9, and the second-order momentum decay coefficient to 0.999.

[0048] The present invention also provides an automatic evaluation system for the efficacy of cosmetics based on facial spectral perception, which executes an automatic evaluation method for the efficacy of cosmetics based on facial spectral perception.

[0049] The effects provided in the invention content are only the effects of the embodiments, rather than all the effects of the invention. The above technical solutions have the following advantages or beneficial effects:

[0050] The present invention discloses an automatic evaluation method and system for the efficacy of cosmetics based on facial spectral perception. By using spectral data and a deep learning model to quantitatively evaluate the efficacy of cosmetics, accurate prediction of skin color, fineness, and glossiness can be achieved. This method combines spectral feature extraction and an attention mechanism to construct an efficient evaluation process for the efficacy of cosmetics;

[0051] By collecting the facial spectral data of subjects before and after using cosmetics, as well as the real skin color scores, fineness scores, and glossiness scores, a CoSK dataset containing spectral data at different times, real scores, and typical skin negative sample spectra is constructed. Constructing a CoSK dataset containing spectral data at different times, real scores, and typical skin negative samples can provide rich data sources, enhance the generalization ability of the model, and improve the prediction accuracy through real scores. Introducing negative samples helps to accurately distinguish different skin types, and the analysis in the time dimension makes it possible to evaluate the immediate and long-term effects of cosmetics. At the same time, by preprocessing the facial spectra, data consistency is ensured, thus realizing efficient and automated evaluation of the efficacy of cosmetics;

[0052] Perform face feature point alignment and affine transformation on the facial spectral image to achieve data preprocessing, and obtain the skin spectra before and after using cosmetics respectively. Performing face feature point alignment and affine transformation on the facial spectral image can effectively ensure data consistency, reduce interference caused by facial posture, expression, or lighting changes, and thus improve the accuracy of subsequent analysis. This preprocessing method ensures the spatial comparability of the skin spectra before and after using cosmetics, enabling the model to more accurately capture the real impact of cosmetics on the skin condition;

[0053] By using the existing feature extraction model to extract negative sample spectral features from the skin negative sample spectrum and inputting them into the negative sample encoder, and further extracting negative sample encoding features, the model's ability to understand and identify different skin types can be effectively enhanced by using the existing feature extraction model to extract features from the skin negative sample spectrum and further encode them. This process helps to mine the potential information of skin negative samples, thereby improving the model's ability to distinguish, so that it can more accurately reflect the impact of different skin conditions on the effect when evaluating the efficacy of cosmetics;

[0054] On this basis, the negative sample weight features are calculated through the negative sample weight calculator to quantify the impact of negative samples on the model. The negative sample weight features can be calculated through the negative sample weight calculator to quantify the impact of negative samples on the model, so that the model can pay more attention to important negative samples during the training process. This mechanism not only improves the model's adaptability to complex skin types, but also effectively reduces the risk of misclassification and enhances its accuracy and reliability in cosmetic efficacy evaluation;

[0055] In order to enhance the perception of spectral data, the skin spectrum before using cosmetics is input into the baseline spectral feature extractor to extract the baseline spectral features, and combined with the negative sample weight features, the spectral perception features are generated through the spectral perception attention mechanism; at the same time, the skin spectrum after using cosmetics is input into the change spectral feature extractor to extract the change spectral features, and after adding the spectral perception features to the change spectral features, the results are input into the skin color evaluator, fineness evaluator and glossiness evaluator respectively to obtain the corresponding prediction scores. The skin spectrum after using cosmetics is input into the change spectral feature extractor and added with the spectral perception features, which can comprehensively consider the actual changes and perceived effects of cosmetics on skin condition, thereby providing more comprehensive and accurate information for skin color, fineness and glossiness evaluation. This integration method improves the accuracy of the model prediction score, so that it better reflects the true effectiveness of cosmetics.

[0056] The error between the predicted score and the real score of the model is calculated by the designed loss function, and the model parameters are iterated using the Adam optimizer. After training, the model can automatically predict the skin color score, fineness score, and glossiness score based on the facial spectrum data to be tested, thereby achieving an accurate quantitative evaluation of the efficacy of cosmetics. The error between the predicted score and the real score is calculated by the designed loss function, and the Adam optimizer is used for parameter iteration, which can effectively improve the learning efficiency and stability of the model, allowing it to converge quickly and optimize performance during the training process. This process not only ensures that the model can automatically and accurately predict the skin color, fineness, and glossiness scores, but also achieves an accurate quantitative evaluation of the efficacy of cosmetics, providing users with a reliable reference.

[0057] In summary, the present invention has the characteristics of high precision, automation, and wide adaptability, providing a new technical solution for the evaluation of cosmetic efficacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention, and do not constitute a limitation to the present invention.

[0059] Figure 1 It is a schematic structural diagram of the cosmetic efficacy evaluation model in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0060] In order to clearly illustrate the technical features of the present solution, the present invention will be described in detail below through specific embodiments and in conjunction with its accompanying drawings. The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. To simplify the disclosure of the present invention, the components and settings of specific examples are described below.

[0061] Embodiment 1

[0062] An automatic evaluation method for cosmetic efficacy based on facial spectrum perception, comprising the following steps:

[0063] S1. Construct a dataset: Collect the facial spectra of the subjects before and after using cosmetics. The facial spectra include the facial spectra before using cosmetics and the facial spectra after using cosmetics. Then, score the two types of facial spectra respectively, and construct a dataset with the facial spectra, the obtained true scores, and the typical skin negative sample spectra on the Internet;

[0064] S2. Preprocess the dataset: Affine transform the facial spectra in the dataset through their facial feature points to obtain the skin spectra before using cosmetics and the skin spectra after using cosmetics , and extract the features of the negative sample spectra in the dataset to obtain the negative sample spectrum features ;

[0065] S3. Construct a cosmetic efficacy evaluation model, which includes a reference spectrum feature extractor, a negative sample encoder, a negative sample weight calculator, a spectrum perception attention, a changed spectrum feature extractor, a skin color evaluator, a fineness evaluator, and a glossiness evaluator. Input the skin spectra before using cosmetics , the skin spectra after using cosmetics and the negative sample spectrum features Input it into the cosmetic efficacy evaluation model to obtain the predicted score of the facial spectrum after using the cosmetic;

[0066] S4. Calculate the loss between the true score and the predicted score through the loss function, and iterate the parameters in the cosmetic efficacy evaluation model through the Adam optimizer to obtain the trained cosmetic efficacy evaluation model;

[0067] S5. Input the facial spectrum after using the cosmetic to be evaluated into the trained cosmetic efficacy evaluation model to obtain the final predicted score.

[0068] S1 is specifically as follows:

[0069] S1.1. The subjects are volunteers recruited with different ages, genders, and skin types. The skin types of the subjects include oily, dry, combination, and sensitive, and the subjects have no obvious skin diseases or facial wounds;

[0070] S1.2. Collect facial spectra through a high-precision multispectral imaging device. The device supports multi-band collection in the range of 400nm - 1000nm, with a resolution of 1920×1080 pixels and a spectral band interval of no more than 10nm;

[0071] S1.3. When collecting facial spectra, use a standard D65 light source, set a pure black background, the indoor temperature is 22 2°C, and the relative humidity is 50 10%. After the subjects wash their faces, wait for 30 minutes to ensure that the facial skin is in a natural state, and collect spectral data twice, before using the cosmetic and 30 days after continuous use of the cosmetic. The cosmetic used is specifically skin care products in the cosmetics;

[0072] S1.4. The scores for the facial spectrum are skin color score, fineness score, and glossiness score. Here, the scores are the true scores of the facial spectrum;

[0073] Among them, the skin color score is based on the L value in the Lab color space, calculates the pink uniformity and brightness, and the score range is 1 - 100; the fineness score is based on the standard deviation of facial texture features, extracts skin roughness parameters through image processing methods, and the score range is 1 - 100; the glossiness score is based on the light reflectance in the 500nm - 600nm band of the spectral graph; uses a mathematical model to calculate facial glossiness, and the score range is 1 - 100; different experts score the subjects, and the average value is taken as the true score after removing the highest and lowest scores;

[0074] S1.5. Collect multiple publicly available skin negative sample data on the Internet, including severe pigmentation, obvious enlarged pores, and dull and dry skin problems, and standardize all skin negative sample spectra.

[0075] S2 is as follows:

[0076] S2.1. For the facial spectra in the dataset, use the 68-point model of the open-source Dlib library to extract the key features of the subject's face, covering positions including the corners of the eyes, the tip of the nose, the corners of the mouth, and the facial boundary positions. Use affine transformation to transform the facial spectra. Specifically, by adopting the affine function to complete the affine transformation, and output the aligned image. Store the spectral data of it as vectors and , represents the skin spectrum before using cosmetics, represents the skin spectrum after using cosmetics;

[0077] S2.2. For the skin negative sample spectra in the dataset, use the trained autoencoder model to extract features from it, and obtain the negative sample spectral features .

[0078] S3 is as follows:

[0079] The cosmetic efficacy evaluation model includes a benchmark spectral feature extractor, a negative sample encoder, a negative sample weight calculator, spectral perception attention, a changed spectral feature extractor, a skin color evaluator, a fineness evaluator, and a glossiness evaluator;

[0080] The benchmark spectral feature extractor sequentially includes a first convolution module, a second convolution module, a third convolution module, and an average pooling layer. The first convolution module sequentially includes a convolution layer with a convolution kernel of 3×3 and a stride of 1, a batch normalization layer, and an activation function layer; the second convolution module sequentially includes a convolution layer with a convolution kernel of 1×1 and a stride of 1; the third convolution module sequentially includes a convolution layer with a convolution kernel of 5×5 and a stride of 2, a batch normalization layer, and an activation function layer;

[0081] Input into the benchmark spectral feature extractor. After passing through the first convolution block, obtain the feature , input into the second convolution block again to obtain the feature , and then input and into the third convolution block to obtain the feature , input and add them together and input into the average pooling layer to obtain the benchmark spectral feature ;

[0082] S3.2. Normalize the spectral features of negative samples, and the calculation process is as follows: The calculation process is as follows:

[0083] ,

[0084] where and represent the mean and standard deviation of the spectral features of negative samples in each dimension, respectively, and represents the normalized spectral features of negative samples;

[0085] S3.3. The negative sample encoder is essentially a fully connected neural network, including a first hidden layer, a second hidden layer, and a third hidden layer. Among them, the first hidden layer includes a fully connected layer with an output dimension of 128, and the activation function is ; the second hidden layer includes a fully connected layer with an output dimension of 32, and the activation function is ; the third hidden layer includes a fully connected layer with an output dimension of 16 and no activation function;

[0086] Input into the negative sample encoder, and successively pass through the first hidden layer, the second hidden layer, and the third hidden layer to obtain the negative sample encoded features , , represents the dimension of the negative sample encoded features , represents the -th dimensional feature in the negative sample encoded features ;

[0087] S3.4. Input the features of different dimensions in the negative sample encoded features into the negative sample weight calculator to obtain the negative sample weight features , and the calculation formula is as follows:

[0088] ,

[0089] ,

[0090] ,

[0091] where represents the activation function, represents the -th dimensional feature and the -th The interaction weight, , , , and are all indices of indicating the linear weight of the d - dimensional feature, indicating the bias term, set , indicating the covariance between feature and feature indicating the product of the standard deviation of feature and feature indicating the variance of feature ;

[0092] S3.5. Calculate the spectral perception feature using the benchmark spectral feature and the negative - sample weight feature through the spectral perception attention formula. The calculation formula is as follows:

[0093] ,

[0094] where represents the first - layer weight matrix in the spectral perception attention formula, represents the second - layer weight matrix in the spectral perception attention formula, represents the bias term of the first layer in the spectral perception attention formula, represents the bias term of the second layer in the spectral perception attention formula;

[0095] S3.6. Keep the structure of the variable - spectrum feature extractor the same as that of the benchmark spectral feature extractor, and update their weights independently. Input into the variable - spectrum feature extractor to obtain the variable - spectrum feature .

[0096] S3.7. The skin - color evaluator, fineness evaluator, and glossiness evaluator have the same structure, all including a first fully - connected layer, a second fully - connected layer, and a third fully - connected layer. The weights of the three evaluators are updated independently;

[0097] Among them, the output dimension of the first fully - connected layer is 64, using the ReLU activation function; the output dimension of the second fully - connected layer is 8, using the ReLU activation function; the output dimension of the third fully - connected layer is 1, without using an activation function;

[0098] Add the spectral perception features and the variable spectral features and input them into the skin color evaluator, fineness evaluator, and glossiness evaluator respectively after addition to obtain the predicted skin color score , predicted fineness score , and predicted glossiness score .

[0099] S4 is specifically as follows:

[0100] Calculate the loss function of the predicted score and the true score , and the calculation formula is as follows:

[0101] ,

[0102] wherein, , represents the total number of samples participating in the training, represents index;

[0103] Set the learning rate of the Adam optimizer to 0.01, the first-order momentum decay coefficient to 0.9, and the second-order momentum decay coefficient to 0.999.

[0104] Example 2

[0105] An automatic evaluation system for cosmetic efficacy based on facial spectral perception, which executes an automatic evaluation method for cosmetic efficacy based on facial spectral perception, including the following modules:

[0106] Data collection module: Collect facial spectra, where the facial spectra include the facial spectra before using cosmetics and the facial spectra after using cosmetics, and construct a dataset by combining the true scores obtained from the facial spectra and the typical skin negative sample spectra on the Internet;

[0107] Data preprocessing module: Preprocess the facial spectra before using cosmetics, the facial spectra after using cosmetics, and the skin negative sample spectra in the dataset;

[0108] Cosmetic efficacy evaluation module: Include a reference spectral feature extractor, a negative sample encoder, a negative sample weight calculator, spectral perception attention, a variable spectral feature extractor, a skin color evaluator, a fineness evaluator, and a glossiness evaluator, and input the output of the data preprocessing module into the cosmetic efficacy evaluation module for skin color, fineness, and glossiness evaluation;

[0109] Optimization module: Optimize the parameters in the cosmetic efficacy evaluation module through the loss function and the Adam optimizer, and optimize the cosmetic efficacy evaluation module.

[0110] Example 3

[0111] To better demonstrate the technical effects of the present invention, the method in the present invention is used for experiments, specifically as follows:

[0112] Recruit 120 volunteers, covering different age groups (18 - 25 years old, 26 - 35 years old, 36 - 45 years old, over 46 years old) and different genders to ensure data diversity. Determine the skin type according to the Fitzpatrick skin classification standard to ensure that the subjects cover common types such as oily, dry, combination, and sensitive skin. The screening conditions ensure that the subjects have no obvious skin diseases or facial wounds to avoid affecting the spectral data collection;

[0113] Use a high-precision multi-spectral imaging device that supports multi-band acquisition in the range of 400nm - 1000nm. The device resolution needs to reach 1920×1080 pixels, and the spectral band interval does not exceed 10nm. Use a standard D65 light source (6500K, color rendering index > 95) to avoid ambient light interference; set a pure black background to prevent reflection interference; keep the indoor temperature at 22±2°C and the relative humidity controlled at 50±10%; after the subjects wash their faces, wait for 30 minutes to ensure that the facial skin is in a natural state, and collect spectral data twice, before using the cosmetic and 30 days after using the cosmetic;

[0114] Ask experts in the relevant field to score the facial spectrum. This score is the real score, which is used to calculate the loss with the predicted score obtained through the cosmetic efficacy evaluation model to adjust the parameters of the cosmetic efficacy evaluation model and realize the training of the cosmetic efficacy evaluation model;

[0115] Collect 36 publicly available skin negative sample data on the Internet, including problem skins such as severe pigmentation, obvious enlarged pores, and dry and dull skin. The spectral data needs to completely cover the range of 400nm - 1000nm to ensure high reference. Standardize all negative sample data, and unify the spectral reflectance range to 0 - 1;

[0116] Construct a dataset according to the above data, and then divide the dataset into a training set and a test set according to the ratio of 8:2.

[0117] For After processing the data in the dataset, the data in the training set is input into the cosmetic efficacy evaluation model for training, then the cosmetic efficacy evaluation model is optimized, and finally the data in the test set is input into the optimized model to obtain the final evaluation result. The final evaluation result is shown in Table 1. At the same time, two other methods are used for evaluation and comparison;

[0118] Table 1 Comparison of evaluation results between the method of the present invention and existing methods

[0119]

[0120] The baseline model A is the SGS cosmetic evaluation model, and the baseline model B is the 3D melanin epidermis model. In terms of skin color scoring, the performance of the model of the present invention on the test set is significantly better than that of the baseline model. Especially when dealing with complex skin color samples, it shows higher accuracy and consistency; the result of the fineness scoring indicates that the model of this patent can better identify the subtle changes in skin texture, which is due to its effective learning of negative sample features, while the baseline model is weak in this aspect; the improvement of the glossiness scoring further verifies the advantage of the model of this patent in capturing the skin glossiness, especially in the evaluation of the after-use effect, showing an obvious leading advantage compared with the baseline model.

[0121] Although the specific implementation manners of the invention are described above in conjunction with the accompanying drawings, it is not a limitation to the protection scope of the present invention. Based on the technical solutions of the present invention, various modifications or deformations that can be made by those skilled in the art without creative efforts are still within the protection scope of the present invention.

Claims

1. A method for automatic evaluation of cosmetic efficacy based on facial spectrum perception, characterized in that: The following steps are involved: S1. Construction Dataset: Collect facial spectra of subjects before and after using cosmetics. The facial spectra include the facial spectra before and after using cosmetics. Then score the two types of facial spectra respectively, and construct a facial spectrum with the facial spectra, the real scores and the typical skin negative sample spectra on the Internet. Datasets; S2, yes Preprocess the data set: The facial spectrum in the dataset is obtained by performing affine transformation on its facial feature points to obtain the skin spectrum before using cosmetics. and skin spectrum after using cosmetics ,right The negative sample spectra in the data set are feature extracted to obtain the negative sample spectral features ; S3. Build a cosmetics efficacy evaluation model, which includes a baseline spectral feature extractor, a negative sample encoder, a negative sample weight calculator, spectral awareness attention, a change spectral feature extractor, a skin color evaluator, a fineness evaluator, and a glossiness evaluator. The skin spectrum before applying cosmetics is used. , Skin spectrum after using cosmetics And negative sample spectral features Input into the cosmetic efficacy evaluation model to obtain the predicted score of the facial spectrum after using cosmetics; The cosmetic efficacy evaluation model includes a baseline spectral feature extractor, a negative sample encoder, a negative sample weight calculator, spectral-aware attention, a variation spectral feature extractor, a skin color evaluator, a fineness evaluator, and a glossiness evaluator; S3.1, the benchmark spectral feature extractor includes a first convolution module, a second convolution module, a third convolution module and an average pooling layer in sequence. The first convolution module includes a convolution layer with a convolution kernel of 3×3 and a step size of 1, a batch normalization layer and a The second convolution module includes a convolution layer with a convolution kernel of 1×1 and a stride of 1; the third convolution module includes a convolution layer with a convolution kernel of 5×5 and a stride of 2, a batch normalization layer and a Activation function layer; Will Input into the benchmark spectral feature extractor, After the first convolution block, the features are obtained ,Will Then input it into the second convolution block to get the features , and then and Input to the third convolution block to get features ,Will and Add the input to the average pooling layer to obtain the baseline spectral features ; S3.

2. Negative sample spectral features After normalization, the calculation process is as follows: ， in, and Represent the spectral features of negative samples The mean and standard deviation of the dimension, Represents the normalized spectral features of negative samples; S3.3, the negative sample encoder is actually a fully connected neural network, including the first hidden layer, the second hidden layer and the third hidden layer, where the first hidden layer includes a fully connected layer, the output dimension is 128, and the activation function is ; The second hidden layer includes a fully connected layer with an output dimension of 32 and an activation function of ; The third hidden layer includes a fully connected layer with an output dimension of 16 and no activation function; Will Input into the negative sample encoder, pass through the first hidden layer, the second hidden layer and the third hidden layer in turn, and obtain the negative sample encoding feature , , Represents the negative sample encoding feature The dimension of Represents the negative sample encoding feature Middle Dimensional features, ; S3.

4. Encoding features of negative samples The features of different dimensions in are input into the negative sample weight calculator to obtain the negative sample weight features , the calculation formula is as follows: ， ， ， in, express Activation function, Represents the negative sample encoding feature Middle Dimensional Features and Dimensional Features The interaction weights, , , , and All for The index of Indicates Dimensional Features The linear weight of Indicates the bias term, set , Representation characteristics and Features The covariance of Representation characteristics and Features The product of the standard deviations, Representation characteristics The variance of Representation characteristics The variance of S3.

5. Baseline spectral features and negative sample weight features The spectral perception features are calculated by the spectral perception attention formula , the calculation formula is as follows: ， in, represents the first layer weight matrix in the spectral perception attention formula, represents the second-layer weight matrix of the spectral perception attention formula, represents the bias term of the first layer in the spectral-aware attention formula, represents the bias term of the second layer in the spectral-aware attention formula; S3.6, the structure of the variation spectrum feature extractor is the same as the baseline spectrum feature extractor, and the weights of the two are updated independently. Input into the variation spectrum feature extractor to obtain the variation spectrum feature ; S3.7, the skin color evaluator, the fineness evaluator and the glossiness evaluator have the same structure, all of which include the first fully connected layer, the second fully connected layer and the third fully connected layer. The weights of the three evaluators are updated independently; Among them, the output dimension of the first fully connected layer is 64, using the ReLU activation function; the output dimension of the second fully connected layer is 8, using the ReLU activation function; the output dimension of the third fully connected layer is 1, and no activation function is used; Spectral sensing features and changing spectral characteristics After addition, they are input into the skin color evaluator, fineness evaluator and glossiness evaluator respectively to obtain the predicted skin color scores , prediction fineness score , predict gloss score ; S4. Calculate the loss of the real score and the predicted score through the loss function, and iterate the parameters in the cosmetics efficacy evaluation model through the Adam optimizer to obtain the trained cosmetics efficacy evaluation model; S5. Input the facial spectrum after using the cosmetics to be evaluated into the trained cosmetics efficacy evaluation model to obtain the final prediction score.

2. The method for automatic evaluation of cosmetic efficacy based on facial spectrum perception according to claim 1, characterized in that: S1 is as follows: S1.

1. The subjects are volunteers of different ages, genders and skin types, including oily, dry, mixed and sensitive skin types, and they have no obvious skin diseases or facial wounds. S1.

2. Use high-precision multi-spectral imaging equipment to collect facial spectra. The equipment supports multi-band collection in the range of 400nm-1000nm, with a resolution of 1920×1080 pixels and a spectral band interval of no more than 10nm. S1.3, when collecting facial spectra, use standard D65 light source, set a pure black background, and set the indoor temperature to 22 2℃, relative humidity 50 10%, the subjects washed their faces and waited for 30 minutes to ensure that the facial skin was in a natural state. Spectral data were collected twice, before using cosmetics and after using cosmetics for 30 consecutive days. The cosmetics used were specifically skin care products. S1.

4. The facial spectrum is scored for skin color, fineness and glossiness. The score here is the real score of the facial spectrum; The skin color score is based on the L value of the Lab color space, and the pink uniformity and brightness are calculated, with a score range of 1-100. The fineness score is based on the standard deviation of facial texture features, and the skin roughness parameters are extracted through image processing methods, with a score range of 1-100. The glossiness score is based on the light reflectance of the 500nm-600nm band in the spectrum. A mathematical model is used to calculate the facial glossiness, with a score range of 1-100. Different experts score the subjects, and the average value is taken as the actual score after removing the highest and lowest scores. S1.

5. Collect multiple skin negative sample data publicly available on the Internet, including severe pigmentation, obviously enlarged pores, dry and dull skin problems, and standardize all skin negative sample spectra.

3. The method for automatically evaluating the efficacy of cosmetics based on facial spectrum perception according to claim 2, wherein S2 The details are as follows: S2.

1. For The facial spectra in the dataset use the 68-point model of the open source Dlib library to extract the key features of the subject's face, covering the corners of the eyes, the tip of the nose, the corners of the mouth, and the facial boundaries. The facial spectra are transformed using affine transformation. Specifically, The affine function completes the affine transformation, outputs the aligned image, and stores its spectral data as vectors and , Indicates the skin spectrum before using cosmetics, Indicates the spectrum of skin after using cosmetics; S2.2, Yes The skin negative sample spectra in the dataset are trained by The autoencoder model extracts features from it and obtains the negative sample spectral features .

4. The method for automatic evaluation of cosmetic efficacy based on facial spectrum perception according to claim 3, characterized in that: S4 is as follows: Calculating prediction scores and real rating The loss function , the calculation formula is as follows: ， in, , Represents the total number of samples involved in training, express The index of The learning rate of the Adam optimizer is set to 0.01, the first-order momentum decay coefficient is set to 0.9, and the second-order momentum decay coefficient is set to 0.

999.

5. An automatic cosmetic efficacy evaluation system based on facial spectrum perception, characterized by: Execute the method for automatic evaluation of cosmetic efficacy based on facial spectrum perception as described in any one of claims 1-4.

Images