Cosmetic analysis method, apparatus, device, and storage medium

Abstract

The application discloses a kind of cosmetic analysis method, device, equipment and storage medium, belong to automation field, the first face feature of user before makeup is obtained in the application, the second face feature of user selected first makeup template corresponding face after makeup, and the third face feature of first makeup template corresponding face before makeup;The first face feature, second face feature and third face feature are input to preset effect prediction model, analysis is carried out based on the preset effect prediction model, and the predicted makeup effect that preset makeup robot is based on the first makeup template is reached to the user for makeup, wherein, the preset effect prediction model is based on the training sample with first makeup effect label, and the preset to be trained model is iteratively trained to obtain. The satisfaction of user to makeup effect is improved, to improve the use experience of user.

Description

Technical Field

[0001] This application relates to the field of automation technology, and in particular to a cosmetic analysis method, apparatus, device, and storage medium. Background Technology

[0002] In related technologies, when using makeup equipment to automatically apply makeup to a user, the makeup is usually applied automatically based on the makeup template selected by the user.

[0003] However, if makeup is applied solely based on a user-selected makeup template, the user's satisfaction with the final result cannot be guaranteed, as the template may not suit the user. This could lead to a poor user experience.

[0004] Therefore, there are technical issues with the related technologies, such as poor user experience for automatic makeup application. Summary of the Invention

[0005] The main purpose of this application is to provide a makeup analysis method, apparatus, device, and storage medium, which aims to solve the technical problem of poor user experience with automated makeup.

[0006] To achieve the above objectives, this application provides a cosmetic analysis method, which includes the following steps:

[0007] The system obtains the user's first facial features before makeup, the second facial features of the user's selected first makeup template after makeup, and the third facial features of the user's selected first makeup template before makeup.

[0008] The first facial feature, the second facial feature, and the third facial feature are input into a preset effect prediction model. Based on the preset effect prediction model, the model is analyzed to obtain the predicted makeup effect achieved by the preset makeup robot when applying makeup to the user based on the first makeup template. This allows the user to determine a satisfactory target makeup template based on the predicted makeup effect, and the preset makeup robot applies makeup to the user based on the target makeup template.

[0009] The preset effect prediction model is obtained by acquiring training samples, determining the first makeup effect label corresponding to the training samples, and iteratively training the preset model to be trained based on the training samples and the corresponding first makeup effect label. The training samples include the fourth face feature of the user before makeup, the fifth face feature of the face corresponding to the second makeup template selected by the user after makeup, and the sixth face feature of the face corresponding to the second makeup template before makeup. The makeup of the user is obtained by a preset makeup robot applying makeup based on the second makeup template. The first makeup effect label is the similarity between the makeup of the user and the makeup of the second makeup template calculated based on a preset calculation formula.

[0010] In one possible implementation of this application, prior to the step of obtaining the user's first facial features before makeup application, the method includes:

[0011] Obtain the seventh facial feature of a user who has applied makeup;

[0012] Based on the sixth and fifth facial features, the user's ideal makeup difference before and after makeup is determined. Based on the fourth and seventh facial features, the user's actual makeup difference before makeup and after makeup application based on the second makeup template is determined.

[0013] Based on the differences between the ideal makeup and the actual makeup, the similarity between the makeup of the user who has applied makeup and the makeup of the second makeup template is determined, and the similarity is used as the first makeup effect label corresponding to the training sample.

[0014] The preset calculation formula is as follows:

[0015]

[0016] Where S is the similarity, A is the difference in ideal makeup, B is the difference in actual makeup, n is the number of users who have applied makeup, and i is the i-th user who has applied makeup.

[0017] In one possible implementation of this application, after the step of using the similarity as the first makeup effect label corresponding to the training sample, the method further includes:

[0018] Obtain the second makeup effect label corresponding to the training sample. The second makeup effect label is obtained by the user who has applied makeup by evaluating the similarity between their own makeup and the makeup of the second makeup template.

[0019] Based on preset weights, the first makeup effect label and the second makeup effect label are weighted to obtain the target makeup effect label. Based on the training samples and the corresponding target makeup effect label, the preset training model is iteratively trained to obtain an effect prediction model that meets the accuracy conditions.

[0020] In one possible implementation of this application, the step of obtaining the user's first facial features before makeup, the second facial features of the face corresponding to the first makeup template selected by the user after makeup, and the third facial features of the face corresponding to the first makeup template before makeup includes:

[0021] Obtain the user's initial facial features and facial structure information before makeup application;

[0022] The user's facial structure information is matched with the facial structure information of each facial model in the preset bare face model library. Based on the degree of matching of the corresponding facial structure information, multiple matching facial models are selected from high to low, so that the user can select the first makeup template from multiple makeup templates corresponding to each matching facial model.

[0023] Feature extraction is performed on the first facial information, the second facial information of the face corresponding to the first makeup template, and the third facial information of the face corresponding to the first makeup template before makeup, respectively, to obtain the first facial features, the second facial features, and the third facial features.

[0024] In one possible implementation of this application, after the step of analyzing based on the preset effect prediction model to obtain the predicted makeup effect achieved by the preset makeup robot applying makeup to the user based on the first makeup template, the method further includes:

[0025] Based on a preset makeup transfer model, the makeup corresponding to the first makeup template is transferred to the user's 3D face model to obtain a simulated makeup, so that the user can determine a satisfactory target makeup template based on the 3D face model after makeup and the predicted makeup effect.

[0026] The three-dimensional face model is obtained by three-dimensional modeling based on the user's facial structure information.

[0027] In one possible implementation of this application, after the step of transferring the makeup corresponding to the first makeup template to the user's 3D face model based on a preset makeup transfer model to obtain a 3D face model after makeup application, the method further includes:

[0028] Based on a preset makeup evaluation model, the simulated makeup is evaluated to obtain a makeup evaluation of the simulated makeup. This evaluation allows users to determine a satisfactory target makeup template based on the 3D face model after makeup application, the predicted makeup effect, and the makeup evaluation of the simulated makeup.

[0029] In one possible implementation of this application, the first facial feature is characterized by including at least one of facial texture features, facial features, and facial features.

[0030] This application also provides a cosmetic analysis device, the device comprising:

[0031] The feature acquisition module is used to acquire the user's first facial features before makeup, the second facial features of the user's selected first makeup template after makeup, and the third facial features of the user's selected first makeup template before makeup.

[0032] The effect prediction module is used to input the first face feature, the second face feature, and the third face feature into a preset effect prediction model, analyze the preset effect prediction model, and obtain the predicted makeup effect achieved by the preset makeup robot applying makeup to the user based on the first makeup template, so that the user can determine a satisfactory target makeup template based on the predicted makeup effect, and the preset makeup robot applies makeup to the user based on the target makeup template.

[0033] The preset effect prediction model is obtained by acquiring training samples, determining the first makeup effect label corresponding to the training samples, and iteratively training the preset model to be trained based on the training samples and the corresponding first makeup effect label. The training samples include the fourth face feature of the user before makeup, the fifth face feature of the face corresponding to the second makeup template selected by the user after makeup, and the sixth face feature of the face corresponding to the second makeup template before makeup. The makeup of the user is obtained by a preset makeup robot applying makeup based on the second makeup template. The first makeup effect label is the similarity between the makeup of the user and the makeup of the second makeup template calculated based on a preset calculation formula.

[0034] This application also provides a cosmetic analysis device, the device comprising: a memory, a processor, and a cosmetic analysis program stored in the memory and executable on the processor, the cosmetic analysis program being configured to implement the steps of the cosmetic analysis method as described in any of the preceding claims.

[0035] This application also provides a storage medium storing a makeup analysis program, which, when executed by a processor, implements the steps of the makeup analysis method as described in any of the preceding claims.

[0036] This application provides a makeup analysis method. Compared to related technologies that simply apply makeup to a user based on a user-selected makeup template, which may not suit the user and therefore cannot guarantee user satisfaction with the makeup effect, this application obtains the user's first facial features before makeup, the second facial features of the user's face corresponding to the first makeup template after makeup, and the third facial features of the user's face corresponding to the first makeup template before makeup. The first, second, and third facial features are input into a preset effect prediction model. Based on the analysis of the preset effect prediction model, a predicted makeup effect achieved by a preset makeup robot applying makeup to the user based on the first makeup template is obtained. The preset effect prediction model is established by acquiring training samples and determining the first makeup effect label corresponding to the training samples. The training samples are obtained by iteratively training a preset model based on the training samples and the corresponding first makeup effect label. The training samples include the fourth face feature of the user before makeup, the fifth face feature of the face corresponding to the second makeup template selected by the user after makeup, and the sixth face feature of the face corresponding to the second makeup template before makeup. The makeup of the user is obtained by a preset makeup robot based on the second makeup template. The first makeup effect label is the similarity between the makeup of the user and the makeup of the second makeup template calculated based on a preset calculation formula.

[0037] In other words, when a user selects a first makeup template, the makeup application isn't directly based on that template. Instead, it uses the user's first facial features before makeup, the second facial features of the corresponding face after makeup, and the third facial features of the corresponding face before makeup. A preset effect prediction model is obtained by iteratively training a preset model based on training samples with a first makeup effect label. This model predicts the makeup effect the preset makeup robot will achieve when applying makeup to the user based on the first makeup template. This provides a reference for the user, allowing them to determine a satisfactory target makeup template based on the predicted effect. The preset makeup robot then applies makeup to the user based on this target template. This avoids a large discrepancy between the predicted makeup effect achieved by the first makeup template and the preset makeup robot's application of makeup to the user based on the first makeup template, improving user satisfaction with the makeup effect and thus enhancing the user experience. Furthermore, the preset model is iteratively trained based on the training samples and the first makeup effect label calculated using a preset formula to obtain an effect prediction model that meets the accuracy requirements. This enables the effect prediction model to predict makeup effects more accurately. Based on the preset effect prediction model, the effect prediction can provide users with a more objective, accurate and quick reference for selecting makeup templates. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating the first embodiment of a cosmetic analysis method according to this application;

[0039] Figure 2 This is a first logical architecture diagram of the cosmetic analysis method according to the first embodiment of this application;

[0040] Figure 3 This is a second logical architecture diagram of the cosmetic analysis method according to the first embodiment of this application;

[0041] Figure 4 This is a schematic diagram of the structure of the cosmetic analysis device in the hardware operating environment involved in the embodiments of this application;

[0042] Figure 5 This is a schematic diagram of the cosmetic analysis device according to the first embodiment of this application.

[0043] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0044] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application. Although the terms first, second, third, etc., may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information.

[0045] This application provides a cosmetic analysis method, referring to... Figure 1 and Figure 2 In this embodiment, the cosmetic analysis method includes:

[0046] Step S10: Obtain the user's first facial features before makeup, the second facial features of the user's selected first makeup template after makeup, and the third facial features of the user's selected first makeup template before makeup.

[0047] Step S20: Input the first face feature, the second face feature, and the third face feature into the preset effect prediction model. Analyze the preset effect prediction model to obtain the predicted makeup effect achieved by the preset makeup robot applying makeup to the user based on the first makeup template. This allows the user to determine a satisfactory target makeup template based on the predicted makeup effect. The preset makeup robot applies makeup to the user based on the target makeup template. The preset effect prediction model is obtained by acquiring training samples, determining the first makeup effect label corresponding to the training samples, and iteratively training the preset model to be trained based on the training samples and the corresponding first makeup effect label. The training samples include the fourth face feature of the user before makeup, the fifth face feature of the face corresponding to the second makeup template selected by the user after makeup, and the sixth face feature of the face corresponding to the second makeup template before makeup. The makeup of the user is obtained by the preset makeup robot applying makeup based on the second makeup template. The first makeup effect label is the similarity between the makeup of the user and the makeup of the second makeup template, calculated based on a preset calculation formula.

[0048] As an example, the cosmetic analysis method is applied to a cosmetic analysis device, which is subordinate to a cosmetic analysis equipment.

[0049] As an example, the cosmetic analysis device may be a cosmetic robot, or an intelligent device used in conjunction with a cosmetic robot, etc., and is not limited thereto.

[0050] The application scenario for this embodiment is:

[0051] Makeup has gradually become an indispensable skill for many users. However, mastering makeup skills or quickly completing a makeup routine is challenging, generally requiring guidance from acquaintances or learning from videos. This learning process is time-consuming and involves constant experimentation. Therefore, makeup assistance or automated makeup tools are crucial for users, saving them time before going out or allowing them to experiment with various makeup looks. Makeup can be applied at home, in business, or in education.

[0052] However, current methods of automating makeup application using smart devices typically rely on user-selected makeup templates. However, if makeup is applied solely based on a template, user satisfaction cannot be guaranteed, as the chosen template may not suit the individual user. This could lead to a poor user experience.

[0053] This embodiment aims to provide users with a reference so that they can determine a satisfactory target makeup template based on the predicted makeup effect. A preset makeup robot then applies makeup to the user based on this target template. This avoids user disappointment due to significant differences between the initial makeup template and the actual makeup effect achieved by the preset makeup robot, thereby increasing user satisfaction with the final makeup result and enhancing the user experience.

[0054] The specific steps are as follows:

[0055] Step S10: Obtain the user's first facial features before makeup, the second facial features of the user's selected first makeup template after makeup, and the third facial features of the user's selected first makeup template before makeup.

[0056] As an example, generally, when a user uses a makeup analysis device, the device collects the user's facial information and, based on this information, applies makeup to the user or controls a makeup robot to apply makeup. Before applying makeup, the user needs to select a first makeup template, which represents the desired makeup effect. In this embodiment, to ensure user satisfaction with the makeup effect, after the user selects a first makeup template, makeup is not applied directly based on that template. Instead, it provides users with the expected makeup effect after the makeup robot applies makeup to them based on the first makeup template. The reasons are as follows: First, the face corresponding to the first makeup template is a filtered, relatively perfect, and standard face that has undergone makeup application. However, a user's appearance is random; different users have different facial features, face shape, hairstyle, and temperament, but each user wants to try different styles. Therefore, the first makeup template chosen by the user may not be suitable, meaning the user may not be satisfied with the makeup effect. Second, when the makeup robot applies makeup to the user based on the first makeup template, the robot's ability to transform the template into the user's makeup varies. It is necessary to predict the effect the makeup robot will achieve based on the first makeup template so that the user can choose a makeup template knowing the desired effect, thus improving the user's satisfaction with the final chosen template.

[0057] As an example, to achieve the above objective, one can first obtain the user's first facial features before makeup, the second facial features of the user's selected first makeup template after makeup, and the third facial features of the user's selected first makeup template before makeup.

[0058] As an example, "before makeup" refers to the user's state before using the makeup analysis device, or the user's state after removing makeup, or the user's state after applying makeup (which needs to be modified by the makeup analysis device), etc., without being limited here.

[0059] As an example, the first makeup template can be selected by the user from a preset makeup template library, or it can be recommended by a preset makeup recommendation model based on the user's first facial features; there is no limitation here.

[0060] In this embodiment, the first facial feature includes at least one of facial texture features, facial features, and facial features.

[0061] As an example, the facial texture features are two-dimensional texture features of a human face, which can be extracted based on models such as LBP (Local Binary Pattern) operator or Gabor (wavelet) operator.

[0062] As an example, the facial features can be extracted using a preset face recognition model.

[0063] As an example, the facial features can be the characteristics of individual facial features such as eyes and nose, or they can be the image type obtained by combining all facial features. This image type includes sweet type, handsome type, gentle type, etc., and is not limited here.

[0064] As an example, the makeup assessment can be obtained by evaluating the user's facial image based on a preset makeup assessment model.

[0065] As an example, the first facial feature may also include gender features, hairstyle features, hair color features, face shape features, etc., without limitation.

[0066] As an example, the first makeup template corresponds to a face image in the makeup template, which can be a two-dimensional face image or a three-dimensional face image, etc.

[0067] As an example, the second facial features of the face corresponding to the first makeup template after makeup include at least one of facial texture features, facial features, and facial features.

[0068] As an example, the third facial feature is the facial feature of the face corresponding to the first makeup template before makeup. Specifically, the preset makeup template library includes not only multiple makeup templates, but also the facial image of each makeup template before makeup.

[0069] As an example, the third facial feature includes at least one of facial texture features, facial features, and facial features.

[0070] As an example, due to the difference between before and after makeup, the second and third facial features also differ. For instance, facial texture features in the second facial feature may include features such as many pimples and wrinkles, while features in the third facial feature may include features such as no pimples and a smooth face.

[0071] As an example, in the process of predicting the makeup effect of users, using the rich facial features that can be extracted as described above can increase the accuracy of the makeup effect prediction.

[0072] Step S20: Input the first face feature, the second face feature, and the third face feature into the preset effect prediction model. Analyze the preset effect prediction model to obtain the predicted makeup effect achieved by the preset makeup robot applying makeup to the user based on the first makeup template. This allows the user to determine a satisfactory target makeup template based on the predicted makeup effect. The preset makeup robot applies makeup to the user based on the target makeup template. The preset effect prediction model is obtained by acquiring training samples, determining the first makeup effect label corresponding to the training samples, and iteratively training the preset model to be trained based on the training samples and the corresponding first makeup effect label. The training samples include the fourth face feature of the user before makeup, the fifth face feature of the face corresponding to the second makeup template selected by the user after makeup, and the sixth face feature of the face corresponding to the second makeup template before makeup. The makeup of the user is obtained by the preset makeup robot applying makeup based on the second makeup template. The first makeup effect label is the similarity between the makeup of the user and the makeup of the second makeup template, calculated based on a preset calculation formula.

[0073] As an example, makeup effect prediction refers to the process where, after a user selects a first makeup template, the preset effect prediction model first predicts the effect that a preset makeup robot will achieve when applying makeup to the user based on the first makeup template. After the user determines a satisfactory target makeup template based on the predicted makeup effect, the preset makeup robot then applies makeup to the user based on the target makeup template.

[0074] As an example, the preset effect prediction model is obtained by iteratively training a preset model based on training samples labeled with the first makeup effect. The first, second, and third facial features are input into the preset effect prediction model, and analysis is performed based on this model. This allows for an objective and accurate prediction of the makeup effect achieved by the preset makeup robot when applying makeup to the user based on the first makeup template. In other words, it provides users with a relatively objective and accurate reference for template selection.

[0075] Step S50: Based on the training samples and the corresponding first makeup effect label, iteratively train the preset model to be trained to obtain an effect prediction model that meets the accuracy conditions.

[0076] As an example, based on the training samples and the first makeup effect label calculated using a preset calculation formula, the preset training model is iteratively trained to obtain an effect prediction model that meets the accuracy requirements. This enables the effect prediction model to predict makeup effects (such as...) Figure 3 The Y shown is more accurate. Based on the preset effect prediction model, the effect prediction can provide users with a more objective, accurate and fast reference for choosing makeup templates.

[0077] In this embodiment, the preset effect prediction model is obtained by acquiring training samples, determining the first makeup effect label corresponding to the training samples, and iteratively training the preset model to be trained based on the training samples and the corresponding first makeup effect label.

[0078] As an example, the training samples include the fourth facial features of the user before makeup, the fifth facial features of the face corresponding to the second makeup template selected by the user after makeup, and the sixth facial features of the face corresponding to the second makeup template before makeup. The makeup of the user is obtained by a preset makeup robot based on the second makeup template.

[0079] As an example, "users who have applied makeup" refers to historical users during the process of acquiring training samples.

[0080] As an example, a user who has already applied makeup refers to a user whose makeup has been applied by a preset makeup robot based on the second makeup template, wherein the second makeup template is the makeup template previously selected by the user who has already applied makeup.

[0081] As an example, users who have already applied makeup before applying makeup (such as...) Figure 3 The fourth facial feature described in the user's bare face information description a) refers to the facial features of the user before makeup is applied by the preset makeup robot based on the second makeup template.

[0082] It should be noted that, before obtaining the training samples, the authorization of the user who has applied makeup has been obtained, and the user has been informed that the obtained training samples are used to predict the makeup effect of the user who has applied makeup, and the obtained training samples will not be used in other scenarios unrelated to this application.

[0083] As an example, a first makeup effect label is determined for the training sample. The first makeup effect label is the similarity between the makeup of the user with makeup applied and the makeup of the second makeup template, calculated based on a preset calculation formula.

[0084] As an example, since the similarity judgment may not be accurate due to the subjectivity of human observation if the training samples are labeled entirely by humans, this embodiment calculates the similarity between the makeup of the user who has applied makeup and the makeup of the second makeup template based on a preset calculation formula, and uses the calculated similarity as the first makeup effect label.

[0085] As an example, the preset calculation formula can be the formula for calculating cosine similarity, the formula for calculating Tanmoto (Tanimoto Coefficient), etc., and is not limited here.

[0086] In this embodiment, when a user selects a first makeup template, the makeup application is not directly based on that template. Instead, it uses the user's first facial features before makeup, the second facial features of the corresponding face after makeup, and the third facial features of the corresponding face before makeup. A preset effect prediction model is obtained by iteratively training a preset model using training samples labeled with a first makeup effect. This model predicts the makeup effect the preset makeup robot will achieve when applying makeup to the user based on the first makeup template. This provides a reference for the user, allowing them to determine a satisfactory target makeup template based on the predicted effect. The preset makeup robot then applies makeup to the user based on this target template. This avoids user disappointment due to a large difference between the first makeup template and the actual makeup effect achieved by the preset makeup robot, increasing user satisfaction with the makeup effect and thus improving the user experience.

[0087] Furthermore, based on the above embodiments of this application, another embodiment of this application is provided, with reference to... Figure 2 In this embodiment, prior to the step of obtaining the user's first facial features before makeup application, the method includes:

[0088] Step A1: Obtain the seventh facial feature of the user after makeup application;

[0089] As an example, the seventh facial feature of a user after applying makeup (such as...) Figure 3 The description of the user's facial information after applying candidate makeup looks on the machine (a') refers to the facial features obtained by extracting features from the facial image of the user who has already applied makeup.

[0090] Step A2: Based on the sixth and fifth facial features, determine the user's ideal makeup difference before and after makeup application; based on the fourth and seventh facial features, determine the user's actual makeup difference before makeup application and after makeup application based on the second makeup template.

[0091] As an example, the extracted facial features can be either feature vectors obtained after vectorization or feature matrices obtained after matrixization.

[0092] As an example, the difference between the ideal makeup look before and after makeup refers to the difference in makeup appearance of the corresponding face of the first makeup template that the user wants to achieve before and after makeup.

[0093] As an example, the actual difference in makeup appearance before and after applying makeup based on the second makeup template refers to the difference in makeup appearance that a user can achieve before and after applying makeup using a makeup robot based on the second makeup template.

[0094] As an example, based on the sixth facial feature (such as...) Figure 3 The candidate makeup and bare-face information description (b) and the fifth facial features (such as...) are shown. Figure 3 The candidate makeup bare face information description b') shown can be used to determine the user's ideal makeup difference before and after makeup. The steps can be: subtract the fifth face feature and the sixth face feature that exist in the form of feature vector after makeup to obtain the user's ideal makeup difference before and after makeup, that is, to obtain the vector A = (a' - a).

[0095] As an example, the step of determining the actual makeup difference of a user before and after applying makeup based on the fourth and seventh face features can be as follows: subtract the fourth and seventh face features, which are in the form of feature vectors, to obtain the user's ideal makeup difference before and after applying makeup, that is, to obtain the vector B = (b' - b).

[0096] Step A3: Based on the differences between the ideal makeup and the actual makeup, determine the similarity between the makeup of the user who has applied makeup and the makeup of the second makeup template, and use the similarity as the first makeup effect label corresponding to the training sample;

[0097] As an example, based on the differences between the ideal makeup and the actual makeup, the similarity between the makeup of the user who has applied makeup and the makeup of the second makeup template can be determined, which can yield a relatively objective first makeup effect label (such as...). Figure 3 The formula shown is: X = cos((a′-a), (b′-b)).

[0098] As an example, the preset calculation formula can be the formula for calculating cosine similarity. Specifically, the preset calculation formula is:

[0099]

[0100] Where S is the similarity, A is the difference in ideal makeup, B is the difference in actual makeup, n is the number of users who have applied makeup, and i is the i-th user who has applied makeup.

[0101] As an example, the preset calculation formula can also be Tanmoto's calculation formula. Specifically, the preset calculation formula is:

[0102]

[0103] Wherein, S represents the similarity, A represents the difference in the ideal makeup look, and B represents the difference in the actual makeup look.

[0104] In this embodiment, the preset effect prediction model (such as...) is used. Figure 3 The machine learning model shown is obtained by iteratively training a preset model to be trained based on the training samples and the first makeup effect label calculated based on a preset calculation formula. This is based on a preset loss function (such as...). Figure 3 The formula shown is: Loss=(YX)*(YX)). Iterative training is performed on the preset model to be trained to obtain the effect prediction model that meets the accuracy condition. This makes the effect prediction model more accurate in predicting the makeup effect. After the effect prediction is performed based on the preset effect prediction model, it can provide users with a more objective, accurate and fast reference for selecting makeup templates.

[0105] Furthermore, based on the above embodiments of this application, another embodiment of this application is provided. In this embodiment, after the step of using the similarity as the first makeup effect label corresponding to the training sample, the method further includes:

[0106] Step S60: Obtain the second makeup effect label corresponding to the training sample. The second makeup effect label is obtained by the user who has applied makeup by evaluating the similarity between their own makeup and the makeup of the second makeup template.

[0107] As an example, since the predicted makeup effect obtained by the preset makeup robot based on the first makeup template is provided to the user for reference, the user's satisfaction with the predicted makeup effect is subjective. If the similarity between the user's makeup and the second makeup template is calculated solely based on the preset formula, the user may miss the makeup template they actually want. Therefore, this embodiment obtains the second makeup effect label obtained by the user evaluating the similarity between their own makeup and the second makeup template.

[0108] Step S70: Based on preset weights, the first makeup effect label and the second makeup effect label are weighted to obtain the target makeup effect label. Based on the training samples and the corresponding target makeup effect label, the preset training model is iteratively trained to obtain an effect prediction model that meets the accuracy conditions.

[0109] As an example, the target makeup effect label can be an effect score (point) or an effect rating (good, medium, poor, etc.).

[0110] As an example, to obtain a more accurate makeup prediction effect, this embodiment weights the first makeup effect label and the second makeup effect label based on preset weights to obtain the target makeup effect label. The preset weights can be determined based on historical experience data.

[0111] As an example, this embodiment weights the first makeup effect label and the second makeup effect label based on preset weights to obtain the target makeup effect label. Based on the weighted target makeup effect label and training samples, the preset training model is iteratively trained. The resulting effect prediction model that meets the accuracy conditions can combine the subjectivity of manual annotation and the objectivity of calculation based on preset calculation formulas to obtain a more comprehensive and more suitable predicted makeup effect for users.

[0112] Furthermore, based on the above embodiments of this application, another embodiment of this application is provided. In this embodiment, the step of obtaining the user's first facial features before makeup, the second facial features of the face corresponding to the first makeup template selected by the user after makeup, and the third facial features of the face corresponding to the first makeup template before makeup includes:

[0113] Step B1: Obtain the user's initial facial information and facial structure information before makeup application;

[0114] As an example, during the process of a user applying makeup using a makeup analysis device, the device collects the user's facial information and, based on that information, applies makeup to the user or controls a makeup robot to apply makeup to the user.

[0115] Specifically, makeup analysis devices acquire the user's initial facial information and facial structure information before applying makeup.

[0116] As an example, the first facial information reflects two-dimensional information about the user's face, including information such as facial features and facial skin.

[0117] As an example, facial structure information reflects the three-dimensional structure of a user's face, including information such as bone structure and the distribution of facial features.

[0118] Step B2: Match the user's facial structure information with the facial structure information of each facial model in the preset bare-face facial model library. Based on the degree of matching of the corresponding facial structure information, filter from high to low to obtain multiple matching facial models, so that the user can select the first makeup template from multiple makeup templates corresponding to each matching facial model.

[0119] As an example, in order to initially select suitable makeup templates for users, it is necessary to first filter facial models whose structures are similar to the user's facial structure. The makeup template corresponding to the facial model is more likely to be suitable for the user. Therefore, in this embodiment, the user's facial structure information is matched with the facial structure information of each facial model in the preset bare-face facial model library (including facial models of celebrities or models, etc.). Based on the degree of matching of the corresponding facial structure information, multiple matching facial models are selected from high to low for the user to choose from.

[0120] As an example, multiple matching face models are filtered from highest to lowest based on the degree of matching. This could involve selecting the top 3 or 5 highly matched face models, which are then displayed on the makeup analysis device's screen. Users can rotate the face models by touching the screen, facilitating selection.

[0121] As an example, each face model corresponds to multiple makeup templates, and multiple makeup templates corresponding to each matched face model can be displayed at the same time for users to choose from.

[0122] As an example, the matching process can be to extract features from the user's facial structure information and the facial structure information of each facial model in the preset plain face model library, and then compare the extracted features (to calculate the similarity of the extracted features in vector form and obtain the degree of matching).

[0123] Step B3: Extract features from the first facial information, the second facial information of the face corresponding to the first makeup template, and the third facial information of the face corresponding to the first makeup template before makeup, respectively, to obtain the first facial features, the second facial features, and the third facial features.

[0124] As an example, the second facial information corresponding to the first makeup template reflects the two-dimensional information of the face.

[0125] As an example, the third facial information of the first makeup template before makeup reflects the two-dimensional information of the face before makeup.

[0126] In this embodiment, after the step of analyzing based on the preset effect prediction model to obtain the predicted makeup effect achieved by the preset makeup robot applying makeup to the user based on the first makeup template, the method further includes:

[0127] Step S80: Based on the preset makeup transfer model, the makeup corresponding to the first makeup template is transferred to the user's three-dimensional face model to obtain a simulated makeup, so that the user can determine a satisfactory target makeup template based on the three-dimensional face model after makeup and the predicted makeup effect. The three-dimensional face model is obtained by three-dimensional modeling based on the user's facial structure information.

[0128] As an example, by inputting the first, second, and third facial features extracted above into the effect prediction model, the predicted makeup effect achieved by the preset makeup robot applying makeup to the user based on the first makeup template can be obtained.

[0129] As an example, the preset makeup transfer model is obtained by training the makeup transfer model to be trained based on training samples with simulated makeup labels.

[0130] As an example, the three-dimensional face model is obtained by three-dimensional modeling based on the user's facial structure information.

[0131] As an example, if users choose a makeup template solely based on the predicted makeup effect, they may only be able to judge based on numerical values ​​or levels, which is not intuitive for them. Therefore, this embodiment uses a preset makeup transfer model to transfer the makeup corresponding to the first makeup template to the user's 3D face model, obtaining a simulated makeup. The user can then determine a satisfactory target makeup template based on the 3D face model after makeup application and the predicted makeup effect, thus improving the user experience.

[0132] In this embodiment, after the step of transferring the makeup corresponding to the first makeup template to the user's 3D face model based on a preset makeup transfer model to obtain the 3D face model after makeup application, the method further includes:

[0133] Step S90: Based on the preset makeup evaluation model, perform a makeup evaluation on the simulated makeup to obtain the makeup evaluation of the simulated makeup, so that the user can determine a satisfactory target makeup template based on the three-dimensional face model after makeup, the predicted makeup effect, and the makeup evaluation of the simulated makeup.

[0134] As an example, to further assist users in selecting makeup templates, this embodiment evaluates the simulated makeup based on a preset makeup evaluation model to obtain a makeup evaluation of the simulated makeup. This evaluation allows users to determine a satisfactory target makeup template based on the 3D face model after makeup application, the predicted makeup effect, and the makeup evaluation of the simulated makeup.

[0135] In this embodiment, after the user determines a satisfactory target makeup template, the makeup analysis device or the makeup analysis device controls the makeup robot to intelligently select the corresponding makeup materials according to the target makeup template selected by the user. Specifically, the makeup robot identifies the location of the makeup materials based on a preset material recognition model, picks up the materials, and applies makeup to the user.

[0136] In this embodiment, after the makeup robot performs the makeup on the user, the user's makeup is scored based on a preset makeup evaluation model for the user's reference, thus completing the fully automated makeup process.

[0137] As an example, a preset makeup evaluation model can be used to score a user's makeup before makeup application and calculate the difference between the user's makeup before and after makeup application (the degree of improvement in makeup) for the user's reference.

[0138] In this embodiment, a fully automated makeup process is provided, which includes facial information collection, facial model matching, makeup template selection based on the 3D facial model after makeup application, the predicted makeup effect, and the makeup evaluation of the simulated makeup, automatic makeup application, and makeup scoring, thereby improving the user experience.

[0139] Reference Figure 4 , Figure 4 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of this application.

[0140] like Figure 4 As shown, the cosmetic analysis device may include: a processor 1001, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to realize the connection and communication between the processor 1001 and the memory 1005.

[0141] Optionally, the cosmetic analysis device may also include a user interface, a network interface, a camera, RF (Radio Frequency) circuitry, sensors, a WiFi module, etc. The user interface may include a display screen and an input submodule such as a keyboard; optional user interfaces may also include standard wired or wireless interfaces. The network interface may include standard wired or wireless interfaces (such as a Wi-Fi interface).

[0142] Those skilled in the art will understand that Figure 4 The structure of the cosmetic analysis device shown does not constitute a limitation on the cosmetic analysis device. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0143] like Figure 4 As shown, the memory 1005, serving as a storage medium, may include an operating system, a network communication module, and a cosmetic analysis program. The operating system is a program that manages and controls the hardware and software resources of the cosmetic analysis device, supporting the operation of the cosmetic analysis program and other software and / or programs. The network communication module is used to enable communication between the various components within the memory 1005, as well as communication with other hardware and software in the cosmetic system.

[0144] exist Figure 4 In the cosmetic analysis device shown, the processor 1001 is used to execute the cosmetic analysis program stored in the memory 1005 to implement the steps of the cosmetic analysis method described above.

[0145] The specific implementation method of the cosmetic analysis device in this application is basically the same as the embodiments of the cosmetic analysis method described above, and will not be repeated here.

[0146] This application also provides a cosmetic analysis device, such as Figure 5 As shown, the device includes:

[0147] The feature acquisition module 10 is used to acquire the user's first facial features before makeup, the second facial features of the user's selected first makeup template after makeup, and the third facial features of the user's selected first makeup template before makeup.

[0148] The effect prediction module 20 is used to input the first face feature, the second face feature and the third face feature into the preset effect prediction model, analyze the preset effect prediction model, and obtain the predicted makeup effect achieved by the preset makeup robot applying makeup to the user based on the first makeup template, so that the user can determine a satisfactory target makeup template based on the predicted makeup effect, and the preset makeup robot applies makeup to the user based on the target makeup template.

[0149] The preset effect prediction model is obtained by acquiring training samples, determining the first makeup effect label corresponding to the training samples, and iteratively training the preset model to be trained based on the training samples and the corresponding first makeup effect label. The training samples include the fourth face feature of the user before makeup, the fifth face feature of the face corresponding to the second makeup template selected by the user after makeup, and the sixth face feature of the face corresponding to the second makeup template before makeup. The makeup of the user is obtained by a preset makeup robot based on the second makeup template. The first makeup effect label is the similarity between the makeup of the user and the makeup of the second makeup template calculated based on a preset calculation formula.

[0150] Optionally, in one possible embodiment of this application, prior to the step of acquiring the user's first facial features before makeup, the device includes:

[0151] The feature acquisition module is used to acquire the seventh facial features of users who have applied makeup after makeup.

[0152] The makeup difference calculation module is used to determine the user's ideal makeup difference before and after makeup based on the sixth and fifth facial features, and to determine the user's actual makeup difference before makeup and after makeup based on the fourth and seventh facial features.

[0153] The similarity calculation module is used to determine the similarity between the makeup of the user who has applied makeup and the makeup of the second makeup template based on the difference between the ideal makeup and the difference between the actual makeup, and to use the similarity as the first makeup effect label corresponding to the training sample.

[0154] The preset calculation formula is as follows:

[0155]

[0156] Where S is the similarity, A is the difference in ideal makeup, B is the difference in actual makeup, n is the number of users who have applied makeup, and i is the i-th user who has applied makeup.

[0157] Optionally, in one possible implementation of this application, after the step of using the similarity as the first makeup effect label corresponding to the training sample, the apparatus further includes:

[0158] The tag acquisition module is used to acquire the second makeup effect tag corresponding to the training sample. The second makeup effect tag is obtained by the user who has applied makeup by evaluating the similarity between their own makeup and the makeup of the second makeup template.

[0159] The second iterative training module is used to weight the first makeup effect label and the second makeup effect label based on preset weights to obtain the target makeup effect label. Based on the training samples and the corresponding target makeup effect label, the preset model to be trained is iteratively trained to obtain an effect prediction model that meets the accuracy conditions.

[0160] Optionally, in one possible implementation of this application, the feature acquisition module 10 includes:

[0161] The information acquisition unit is used to acquire the user's initial facial information and facial structure information before makeup application.

[0162] The information matching unit is used to match the user's facial structure information with the facial structure information of each facial model in the preset bare face model library. Based on the degree of matching of the corresponding facial structure information, multiple matching facial models are selected from high to low, so that the user can select the first makeup template from multiple makeup templates corresponding to each matching facial model.

[0163] The feature extraction unit is used to extract features from the first facial information, the second facial information of the face corresponding to the first makeup template, and the third facial information of the face corresponding to the first makeup template before makeup, respectively, to obtain the first facial features, the second facial features, and the third facial features.

[0164] Optionally, in one possible embodiment of this application, after the step of analyzing based on the preset effect prediction model to obtain the predicted makeup effect achieved by the preset makeup robot applying makeup to the user based on the first makeup template, the device further includes:

[0165] The makeup simulation module is used to transfer the makeup corresponding to the first makeup template to the user's three-dimensional face model based on a preset makeup transfer model to obtain a simulated makeup, so that the user can determine a satisfactory target makeup template based on the three-dimensional face model after makeup and the predicted makeup effect. The three-dimensional face model is obtained by three-dimensional modeling based on the user's facial structure information.

[0166] Optionally, in one possible implementation of this application, after the step of transferring the makeup corresponding to the first makeup template to the user's 3D face model based on a preset makeup transfer model to obtain a 3D face model after makeup application, the device further includes:

[0167] The template determination module is used to evaluate the simulated makeup based on a preset makeup evaluation model to obtain the makeup evaluation of the simulated makeup, so that the user can determine a satisfactory target makeup template based on the 3D face model after makeup, the predicted makeup effect, and the makeup evaluation of the simulated makeup.

[0168] Optionally, in one possible implementation of this application, the first facial feature includes at least one of facial texture features, facial features, and facial features.

[0169] The specific implementation of the cosmetic analysis device in this application is basically the same as the embodiments of the cosmetic analysis method described above, and will not be repeated here.

[0170] This application also provides a storage medium storing a makeup analysis program, which, when executed by a processor, implements the steps of the makeup analysis method as described in any of the preceding claims.

[0171] The specific implementation of the storage medium in this application is basically the same as the embodiments of the cosmetic analysis method described above, and will not be repeated here.

[0172] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0173] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0174] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0175] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A cosmetic analysis method, characterized in that, Includes the following steps: The system obtains the user's first facial features before makeup, the second facial features of the user's selected first makeup template after makeup, and the third facial features of the user's selected first makeup template before makeup. The first facial feature, the second facial feature, and the third facial feature are input into a preset effect prediction model. Based on the preset effect prediction model, the model is analyzed to obtain the predicted makeup effect achieved by the preset makeup robot when applying makeup to the user based on the first makeup template. This allows the user to determine a satisfactory target makeup template based on the predicted makeup effect, and the preset makeup robot applies makeup to the user based on the target makeup template. The preset effect prediction model is obtained by acquiring training samples, determining the first makeup effect label corresponding to the training samples, and iteratively training the preset model to be trained based on the training samples and the corresponding first makeup effect label. The training samples include the fourth face features of the user before makeup, the fifth face features of the face corresponding to the second makeup template selected by the user after makeup, and the sixth face features of the face corresponding to the second makeup template before makeup. The makeup of the user is obtained by a preset makeup robot applying makeup based on the second makeup template. The first makeup effect label is the similarity between the makeup of the user and the makeup of the second makeup template calculated based on a preset calculation formula. Prior to the step of obtaining the user's first facial features before makeup, the method includes: Obtain the seventh facial feature of a user who has applied makeup; Based on the sixth and fifth facial features, the user's ideal makeup difference before and after makeup is determined. Based on the fourth and seventh facial features, the user's actual makeup difference before makeup and after makeup application based on the second makeup template is determined. Based on the differences between the ideal makeup and the actual makeup, the similarity between the makeup of the user who has applied makeup and the makeup of the second makeup template is determined, and the similarity is used as the first makeup effect label corresponding to the training sample.

2. The cosmetic analysis method as described in claim 1, characterized in that, The preset calculation formula is: in, For the aforementioned similarity, For the differences in the ideal makeup look, To account for the differences in the actual makeup, The number of users who have applied makeup. For the first One user who has already applied makeup.

3. The cosmetic analysis method as described in claim 2, characterized in that, After the step of using the similarity as the first makeup effect label corresponding to the training sample, the method further includes: Obtain the second makeup effect label corresponding to the training sample. The second makeup effect label is obtained by the user who has applied makeup by evaluating the similarity between their own makeup and the makeup of the second makeup template. Based on preset weights, the first makeup effect label and the second makeup effect label are weighted to obtain the target makeup effect label. Based on the training samples and the corresponding target makeup effect label, the preset training model is iteratively trained to obtain an effect prediction model that meets the accuracy conditions.

4. The cosmetic analysis method as described in claim 1, characterized in that, The steps of obtaining the user's first facial features before makeup, the second facial features of the user's selected first makeup template corresponding to the user's second facial features after makeup, and the third facial features of the first makeup template corresponding to the user's third facial features before makeup include: Obtain the user's initial facial features and facial structure information before makeup application; The user's facial structure information is matched with the facial structure information of each facial model in the preset bare face model library. Based on the degree of matching of the corresponding facial structure information, multiple matching facial models are selected from high to low, so that the user can select the first makeup template from multiple makeup templates corresponding to each matching facial model. Feature extraction is performed on the first facial information, the second facial information of the face corresponding to the first makeup template, and the third facial information of the face corresponding to the first makeup template before makeup, respectively, to obtain the first facial features, the second facial features, and the third facial features.

5. The cosmetic analysis method as described in claim 4, characterized in that, After the step of analyzing based on the preset effect prediction model to obtain the predicted makeup effect achieved by the preset makeup robot applying makeup to the user based on the first makeup template, the method further includes: Based on the preset makeup transfer model, the makeup corresponding to the first makeup template is transferred to the user's three-dimensional face model to obtain a simulated makeup, so that the user can determine a satisfactory target makeup template based on the three-dimensional face model after makeup and the predicted makeup effect. The three-dimensional face model is obtained by three-dimensional modeling based on the user's facial structure information.

6. The cosmetic analysis method as described in claim 5, characterized in that, After the step of transferring the makeup corresponding to the first makeup template to the user's 3D face model based on a preset makeup transfer model to obtain the 3D face model after makeup application, the method further includes: Based on a preset makeup evaluation model, the simulated makeup is evaluated to obtain a makeup evaluation of the simulated makeup. This evaluation allows users to determine a satisfactory target makeup template based on the 3D face model after makeup application, the predicted makeup effect, and the makeup evaluation of the simulated makeup.

7. The cosmetic analysis method according to any one of claims 1-6, characterized in that, The first facial feature includes at least one of facial texture features, facial features, and facial features.

8. A cosmetic analysis device, characterized in that, The device includes: The feature acquisition module is used to acquire the user's first facial features before makeup, the second facial features of the user's selected first makeup template after makeup, and the third facial features of the user's selected first makeup template before makeup. The effect prediction module is used to input the first face feature, the second face feature, and the third face feature into a preset effect prediction model, analyze the preset effect prediction model, and obtain the predicted makeup effect achieved by the preset makeup robot applying makeup to the user based on the first makeup template, so that the user can determine a satisfactory target makeup template based on the predicted makeup effect, and the preset makeup robot applies makeup to the user based on the target makeup template. The preset effect prediction model is obtained by acquiring training samples, determining the first makeup effect label corresponding to the training samples, and iteratively training the preset model to be trained based on the training samples and the corresponding first makeup effect label. The training samples include the fourth face features of the user before makeup, the fifth face features of the face corresponding to the second makeup template selected by the user after makeup, and the sixth face features of the face corresponding to the second makeup template before makeup. The makeup of the user is obtained by a preset makeup robot applying makeup based on the second makeup template. The first makeup effect label is the similarity between the makeup of the user and the makeup of the second makeup template calculated based on a preset calculation formula. Prior to the step of acquiring the user's first facial features before makeup application, the device includes: The feature acquisition module is used to acquire the seventh facial features of users who have applied makeup after makeup. The makeup difference calculation module is used to determine the user's ideal makeup difference before and after makeup based on the sixth and fifth facial features, and to determine the user's actual makeup difference before makeup and after makeup based on the fourth and seventh facial features. The similarity calculation module is used to determine the similarity between the makeup of the user who has applied makeup and the makeup of the second makeup template based on the difference between the ideal makeup and the difference between the actual makeup, and to use the similarity as the first makeup effect label corresponding to the training sample.

9. A cosmetic analysis device, characterized in that, The device includes: a memory, a processor, and a makeup analysis program stored in the memory and executable on the processor, the makeup analysis program being configured to implement the steps of the makeup analysis method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a makeup analysis program, which, when executed by a processor, implements the steps of the makeup analysis method as described in any one of claims 1 to 7.

Images