Cosmetic analysis method and device, equipment and storage medium

Abstract

The invention discloses a makeup analysis method and device, equipment and a storage medium, and belongs to the technical field of automation, and the method comprises the steps: obtaining a first face feature of a user before makeup, a second face feature of a face corresponding to a first makeup template selected by the user after makeup, and a third face feature of the face corresponding to the first makeup template before makeup; and inputting the first face feature, the second face feature and the third face feature into a preset effect prediction model, and performing analysis based on the preset effect prediction model to obtain a predicted makeup effect achieved by a preset makeup robot for makeup on the user based on the first makeup template, the preset effect prediction model is obtained by performing iterative training on a preset to-be-trained model based on a training sample with a first makeup effect label. And the satisfaction degree of the user on the after-makeup effect is improved, so that the use experience of the user is improved.

Description

Technical Field

[0001] The present application relates to the field of automation technology, and in particular to a makeup analysis method, device, equipment and storage medium. Background Art

[0002] In the related art, when a makeup device is used to automatically apply makeup to a user, makeup is usually automatically applied to the user based on a makeup template selected by the user.

[0003] However, if the user is only given makeup based on the makeup template selected by the user, since the makeup template selected by the user may not be suitable for the user, the user's satisfaction with the makeup effect cannot be guaranteed, which may result in a poor user experience.

[0004] Therefore, there is a technical problem in the related art that users have a poor experience in automatic makeup. Summary of the invention

[0005] The main purpose of this application is to provide a makeup analysis method, device, equipment and storage medium, aiming to solve the technical problem that users have a poor experience with automatic makeup.

[0006] To achieve the above object, the present application provides a makeup analysis method, which comprises the following steps:

[0007] Acquire a first facial feature of the user before makeup, a second facial feature of the face corresponding to the first makeup template selected by the user after makeup, and a third facial feature of the face corresponding to the first makeup template before makeup;

[0008] Inputting the first facial feature, the second facial feature, and the third facial feature into a preset effect prediction model, performing analysis based on the preset effect prediction model, and obtaining a predicted makeup effect that a preset makeup robot will achieve by applying makeup to the user based on the first makeup template, so that the user can determine a satisfactory target makeup template based on the predicted makeup effect, and the preset makeup robot applies makeup to the user based on the target makeup template;

[0009] Among them, the preset effect prediction model is obtained by obtaining training samples, determining the first makeup effect label corresponding to the training samples, and iteratively training the preset model to be trained based on the training samples and the corresponding first makeup effect label, the training samples include the fourth facial features of the user who has put on makeup before makeup, the fifth facial features of the face corresponding to the second makeup template selected by the user who has put on makeup after makeup, and the sixth facial features of the face corresponding to the second makeup template before makeup, the makeup of the user who has put on makeup is obtained by applying makeup based on the second makeup template by a preset makeup robot, and the first makeup effect label is the similarity between the makeup of the user who has put on makeup and the makeup of the second makeup template calculated based on a preset calculation formula.

[0010] In a possible implementation manner of the present application, before the step of obtaining the first facial feature of the user before putting on makeup, the method includes:

[0011] Obtain the seventh facial feature of the user who has put on makeup;

[0012] Based on the sixth and fifth facial features, determining the difference between the user's ideal makeup before and after applying makeup, and based on the fourth and seventh facial features, determining the difference between the user's actual makeup before applying makeup and after applying makeup based on the second makeup template;

[0013] Based on the ideal makeup difference and the actual makeup difference, determine the similarity between the makeup of the user who has put on makeup and the makeup of the second makeup template, and use the similarity as the first makeup effect label corresponding to the training sample;

[0014] Wherein, the preset calculation formula is:

[0015]

[0016] Wherein, S is the similarity, A is the ideal makeup difference, B is the actual makeup difference, n is the number of users who have put on makeup, and i is the i-th user who has put on makeup.

[0017] In a possible implementation manner of the present application, after the step of using the similarity as the first makeup effect label corresponding to the training sample, the method further includes:

[0018] Obtaining a second makeup effect label corresponding to the training sample, where the second makeup effect label is obtained by a user who has put on makeup evaluating the similarity between his / her own makeup and the makeup of the second makeup template;

[0019] Based on the preset weight, the first makeup effect label and the second makeup effect label are weighted to obtain the target makeup effect label, and based on the training samples and the corresponding target makeup effect label, the preset model to be trained is iteratively trained to obtain an effect prediction model that meets the accuracy conditions.

[0020] In a possible implementation of the present application, the step of obtaining a first facial feature of the user before makeup, a second facial feature of the face corresponding to the first makeup template selected by the user after makeup, and a third facial feature of the face corresponding to the first makeup template before makeup includes:

[0021] Obtain the user's first facial information and facial structure information before makeup;

[0022] Matching the user's facial structure information with the facial structure information of each facial model in a preset bare-face facial model library, and filtering from high to low to obtain a plurality of matching facial models based on the matching degree of the corresponding facial structure information, so that the user can select the first makeup template from a plurality of makeup templates corresponding to each matching facial model;

[0023] Feature extraction is performed on the first facial information, the second facial information of the face corresponding to the first makeup template, and the third facial information of the face corresponding to the first makeup template before makeup, respectively, to obtain first facial features, second facial features, and third facial features accordingly.

[0024] In a possible implementation manner of the present application, after the step of analyzing based on the preset effect prediction model to obtain a predicted makeup effect achieved by the preset makeup robot applying makeup to the user based on the first makeup template, the method further includes:

[0025] Based on a preset makeup migration model, the makeup corresponding to the first makeup template is migrated to the three-dimensional face model of the user to obtain a simulated makeup, so that the user can determine a satisfactory target makeup template based on the three-dimensional face model after makeup and the predicted makeup effect;

[0026] The three-dimensional face model is obtained by three-dimensional modeling based on the face structure information of the user.

[0027] In a possible implementation of the present application, after the step of migrating the makeup corresponding to the first makeup template to the three-dimensional face model of the user based on the preset makeup migration model to obtain the three-dimensional face model after makeup, the method further includes:

[0028] Based on a preset makeup evaluation model, the simulated makeup is evaluated to obtain a makeup evaluation of the simulated makeup, so that the user can determine a satisfactory target makeup template based on the three-dimensional face model after makeup, the predicted makeup effect, and the makeup evaluation of the simulated makeup.

[0029] In a possible implementation manner of the present application, it is characterized in that the first facial feature includes at least one of facial texture features, facial features, and facial features.

[0030] The present application also provides a makeup analysis device, the device comprising:

[0031] A feature acquisition module, used to acquire a first facial feature of the user before makeup, a second facial feature of the face corresponding to the first makeup template selected by the user after makeup, and a third facial feature of the face corresponding to the first makeup template before makeup;

[0032] an effect prediction module, for inputting the first facial feature, the second facial feature, and the third facial feature into a preset effect prediction model, and performing analysis based on the preset effect prediction model to obtain a predicted makeup effect that a preset makeup robot will achieve by applying makeup to the user based on the first makeup template, so that the user can determine a satisfactory target makeup template based on the predicted makeup effect, and the preset makeup robot applies makeup to the user based on the target makeup template;

[0033] Among them, the preset effect prediction model is obtained by obtaining training samples, determining the first makeup effect label corresponding to the training samples, and iteratively training the preset model to be trained based on the training samples and the corresponding first makeup effect label, the training samples include the fourth facial features of the user who has put on makeup before makeup, the fifth facial features of the face corresponding to the second makeup template selected by the user who has put on makeup after makeup, and the sixth facial features of the face corresponding to the second makeup template before makeup, the makeup of the user who has put on makeup is obtained by applying makeup based on the second makeup template by a preset makeup robot, and the first makeup effect label is the similarity between the makeup of the user who has put on makeup and the makeup of the second makeup template calculated based on a preset calculation formula.

[0034] The present application also provides a makeup analysis device, which includes: a memory, a processor, and a makeup analysis program stored in the memory and executable on the processor, wherein the makeup analysis program is configured to implement the steps of the makeup analysis method as described in any one of the above items.

[0035] The present application also provides a storage medium, on which a makeup analysis program is stored. When the makeup analysis program is executed by a processor, the steps of the makeup analysis method as described in any one of the above items are implemented.

[0036] The present application provides a makeup analysis method. Compared with the related art, if makeup is only applied to the user based on the makeup template selected by the user, the user's satisfaction with the makeup effect cannot be guaranteed because the makeup template selected by the user may not be suitable for the user. The present application obtains the first facial features of the user before makeup, the second facial features of the face corresponding to the first makeup template selected by the user after makeup, and the third facial features of the face corresponding to the first makeup template before makeup; the first facial features, the second facial features, and the third facial features are input into a preset effect prediction model, and analysis is performed based on the preset effect prediction model to obtain the predicted makeup effect achieved by the preset makeup robot applying makeup to the user based on the first makeup template, wherein the preset effect prediction model is to obtain training samples, determine the first makeup effect label corresponding to the training samples, The makeup of the user who has put on makeup is obtained by iteratively training the preset model to be trained based on the training samples and the corresponding first makeup effect labels, the training samples include the fourth facial feature of the user who has put on makeup before makeup, the fifth facial feature of the face corresponding to the second makeup template selected by the user who has put on makeup after makeup, and the sixth facial feature of the face corresponding to the second makeup template before makeup, the makeup of the user who has put on makeup is obtained by applying makeup based on the second makeup template by a preset makeup robot, and the first makeup effect label is the similarity between the makeup of the user who has put on makeup and the makeup of the second makeup template calculated based on a preset calculation formula.

[0037] That is to say, when the user selects the first makeup template, the user is not directly made up based on the first makeup template. Instead, the preset effect prediction model is obtained by iteratively training the preset model to be trained based on the training sample with the first makeup effect label, and the predicted makeup effect achieved by the preset makeup robot when applying makeup to the user based on the first makeup template is predicted. A reference can be provided for the user to determine a satisfactory target makeup template based on the predicted makeup effect, and the preset makeup robot applies makeup to the user based on the target makeup template. Avoiding the difference between the predicted makeup effect achieved by the first makeup template and the preset makeup robot applying makeup to the user based on the first makeup template is too large, improving the user's satisfaction with the makeup effect, and thus improving the user's experience. Moreover, based on the training sample and the first makeup effect label calculated based on the preset calculation formula, the preset model to be trained is iteratively trained to obtain an effect prediction model that meets the accuracy condition. The makeup effect predicted by the effect prediction model can be made more accurate. After the effect prediction is performed based on the preset effect prediction model, a reference basis for selecting a makeup template can be provided to the user more objectively, accurately and quickly. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flow chart of a first embodiment of a makeup analysis method of the present application;

[0039] Figure 2 This is a first logical architecture diagram of the makeup analysis method of the first embodiment of the present application;

[0040] Figure 3 This is a second logical architecture diagram of the makeup analysis method of the first embodiment of the present application;

[0041] Figure 4 It is a structural diagram of a makeup analysis device in a hardware operating environment involved in an embodiment of the present application;

[0042] Figure 5 This is a schematic diagram of a makeup analysis device according to the first embodiment of the present application.

[0043] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0044] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. Although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information.

[0045] The present application embodiment provides a makeup analysis method, referring to Figure 1 and Figure 2 In this embodiment, the makeup analysis method includes:

[0046] Step S10: obtaining a first facial feature of the user before makeup, a second facial feature of the face corresponding to the first makeup template selected by the user after makeup, and a third facial feature of the face corresponding to the first makeup template before makeup;

[0047] Step S20: input the first facial feature, the second facial feature and the third facial feature into a preset effect prediction model, analyze based on the preset effect prediction model, obtain the predicted makeup effect achieved by the preset makeup robot applying makeup to the user based on the first makeup template, so that the user can determine a satisfactory target makeup template based on the predicted makeup effect, and the preset makeup robot applies makeup to the user based on the target makeup template, wherein the preset effect prediction model is obtained by obtaining training samples, determining the first makeup effect label corresponding to the training samples, and iteratively training the preset to-be-trained model based on the training samples and the corresponding first makeup effect label, the training samples include the fourth facial feature of the user who has applied makeup before makeup, the fifth facial feature of the face corresponding to the second makeup template selected by the user who has applied makeup after makeup, and the sixth facial feature of the face corresponding to the second makeup template before makeup, the makeup of the user who has applied makeup is obtained by the preset makeup robot applying makeup based on the second makeup template, and the first makeup effect label is the similarity between the makeup of the user who has applied makeup and the makeup of the second makeup template calculated based on a preset calculation formula.

[0048] As an example, the makeup analysis method is applied to a makeup analysis device, which is subordinate to a makeup analysis equipment.

[0049] As an example, the makeup analysis device may be a makeup robot, or a smart device used in conjunction with a makeup robot, etc., which is not limited here.

[0050] The application scenarios of this embodiment are:

[0051] Makeup has gradually become one of the indispensable skills for users. However, it is difficult to master makeup skills or complete makeup quickly. Generally, users need to learn based on the guidance of acquaintances or videos. However, the learning process of users takes a lot of time and requires constant attempts. Therefore, makeup methods such as assisted makeup or automatic makeup are very important for users, which can save users time when going out, or help users try a variety of makeup. Makeup can be used at home, in business, or in teaching.

[0052] However, currently, in the process of automatically applying makeup to users using smart devices, the makeup is usually applied automatically to the user based on the makeup template selected by the user. However, if the makeup is applied to the user based only on the makeup template selected by the user, the user's satisfaction with the makeup effect cannot be guaranteed because the makeup template selected by the user may not be suitable for the user. This may result in a poor user experience.

[0053] This embodiment aims to provide a reference for the user, so that the user can determine a satisfactory target makeup template based on the predicted makeup effect, and the preset makeup robot can apply makeup to the user based on the target makeup template. This prevents the user from being disappointed due to the large difference between the first makeup template and the actual makeup effect that can be completed by the preset makeup robot, improves the user's satisfaction with the makeup effect, and thus improves the user's experience.

[0054] The specific steps are as follows:

[0055] Step S10: obtaining a first facial feature of the user before makeup, a second facial feature of the face corresponding to the first makeup template selected by the user after makeup, and a third facial feature of the face corresponding to the first makeup template before makeup;

[0056] As an example, generally, when a user uses a makeup analysis device, the makeup analysis device will collect the user's facial information, and based on the user's facial information, apply makeup to the user or control a makeup robot to apply makeup to the user. Before applying makeup, the user needs to select a first makeup template, which is the makeup effect that the user hopes to achieve. In order to ensure the user's satisfaction with the makeup effect, this embodiment does not directly apply makeup to the user based on the first makeup template after the user selects it. Instead, it provides the user with the makeup effect that can be achieved by the makeup robot after applying makeup to the user based on the first makeup template. The reasons are: first, the human face corresponding to the first makeup template is a relatively perfect, standard and made-up human face obtained through screening, while the user's appearance is random. Different users have different facial features, face shapes, hairstyles, temperaments, etc., but each user wants to try different styles. Therefore, the first makeup template selected by the user is not necessarily suitable, that is, the user may not be satisfied with the makeup effect; second, when the makeup robot applies makeup to the user based on the first makeup template, the makeup robot has different abilities to convert the first makeup template into the user's makeup; it is necessary to predict the effect that the preset makeup robot will achieve when applying makeup to the user based on the first makeup template, so that the user can choose the makeup template on the premise of knowing the makeup effect, thereby improving the user's satisfaction with the template makeup template finally selected.

[0057] As an example, in order to achieve the above purpose, the first facial features of the user before makeup, the second facial features of the face corresponding to the first makeup template selected by the user after makeup, and the third facial features of the face corresponding to the first makeup template before makeup can be obtained first.

[0058] As an example, before makeup refers to the state of the user before using the makeup analysis device, or the state of the user after removing makeup, or the state of the user after applying makeup by themselves (needing to change the makeup through the makeup analysis device), etc., which is not limited here.

[0059] As an example, the first makeup template may be selected by the user from a preset makeup template library, or may be recommended by a preset makeup recommendation model based on the user's first facial feature, which is not limited here.

[0060] In this embodiment, the first facial feature includes at least one of facial texture features, facial features, and facial features.

[0061] As an example, the facial texture feature is a two-dimensional texture feature of a human face, and the facial texture feature can be extracted based on a model such as an LBP (Local Binary Pattern) operator or a Gabor (wavelet) operator.

[0062] As an example, the facial features may be extracted through a preset face recognition model.

[0063] As an example, the facial features may be the characteristics of a single facial feature such as eyes or nose, or may be an image type obtained by combining all facial features, including sweet type, handsome type, gentle type, etc., which are not limited here.

[0064] As an example, the makeup evaluation may be obtained by performing makeup evaluation on the user's facial image based on a preset makeup evaluation model.

[0065] As an example, the first facial feature may also include gender features, hairstyle features, hair color features, face shape features, etc., which are not limited here.

[0066] As an example, the face corresponding to the first makeup template is a face image in the makeup template, and the face image may be a two-dimensional face image or a three-dimensional face image.

[0067] As an example, the second facial feature of the face after makeup corresponding to the first makeup template includes at least one of facial texture features, facial features, and facial features.

[0068] As an example, the third facial feature is the facial feature of the face corresponding to the first makeup template before makeup. Specifically, the preset makeup template library includes not only multiple makeup templates, but also a facial image before makeup corresponding to each makeup template.

[0069] As an example, the third facial feature includes at least one of facial texture features, facial features, and facial features.

[0070] As an example, due to the difference before and after makeup, the second facial features and the third facial features are also different. For example, the facial texture features in the second facial features may include features such as many acne and wrinkles, while the third facial features may include features such as no acne and a smooth face.

[0071] As an example, in the process of predicting the makeup effect of a user, using the rich facial features that can be extracted as described above can increase the accuracy of the makeup effect prediction.

[0072] Step S20: input the first facial feature, the second facial feature and the third facial feature into a preset effect prediction model, analyze based on the preset effect prediction model, obtain the predicted makeup effect achieved by the preset makeup robot applying makeup to the user based on the first makeup template, so that the user can determine a satisfactory target makeup template based on the predicted makeup effect, and the preset makeup robot applies makeup to the user based on the target makeup template, wherein the preset effect prediction model is obtained by obtaining training samples, determining the first makeup effect label corresponding to the training samples, and iteratively training the preset to-be-trained model based on the training samples and the corresponding first makeup effect label, the training samples include the fourth facial feature of the user who has applied makeup before makeup, the fifth facial feature of the face corresponding to the second makeup template selected by the user who has applied makeup after makeup, and the sixth facial feature of the face corresponding to the second makeup template before makeup, the makeup of the user who has applied makeup is obtained by the preset makeup robot applying makeup based on the second makeup template, and the first makeup effect label is the similarity between the makeup of the user who has applied makeup and the makeup of the second makeup template calculated based on a preset calculation formula.

[0073] As an example, makeup effect prediction means that after the user selects the first makeup template, the preset makeup robot predicts the effect that will be achieved by applying makeup to the user based on the first makeup template based on a preset effect prediction model. After the user determines a satisfactory target makeup template based on the predicted makeup effect, the preset makeup robot then applies makeup to the user based on the target makeup template.

[0074] As an example, the preset effect prediction model is obtained by iteratively training the preset model to be trained based on the training sample with the first makeup effect label, and the first face feature, the second face feature and the third face feature are input into the preset effect prediction model, and the preset effect prediction model is analyzed to objectively and accurately obtain the predicted makeup effect that the preset makeup robot will achieve when applying makeup to the user based on the first makeup template. That is, it can provide users with a relatively objective and accurate reference for template selection.

[0075] Step S50: Based on the training sample and the corresponding first makeup effect label, iteratively train the preset model to be trained to obtain an effect prediction model that meets the accuracy condition.

[0076] As an example, based on the training sample and the first makeup effect label calculated based on the preset calculation formula, the preset model to be trained is iteratively trained to obtain an effect prediction model that meets the accuracy condition. Figure 3 The Y) shown is more accurate. After the effect prediction is performed based on the preset effect prediction model, it can provide users with a reference for selecting a makeup template more objectively, accurately and quickly.

[0077] In this embodiment, the preset effect prediction model is obtained by obtaining training samples, determining the first makeup effect label corresponding to the training samples, and iteratively training the preset model to be trained based on the training samples and the corresponding first makeup effect label.

[0078] As an example, the training sample includes a fourth facial feature of a user who has put on makeup before putting on makeup, a fifth facial feature of a face corresponding to a second makeup template selected by the user who has put on makeup after putting on makeup, and a sixth facial feature of a face corresponding to the second makeup template before putting on makeup, and the makeup of the user who has put on makeup is obtained by applying makeup by a preset makeup robot based on the second makeup template;

[0079] As an example, the makeup-wearing user refers to the historical user corresponding to the process of obtaining the training sample.

[0080] As an example, the user who has put on makeup refers to the user after the preset makeup robot has put on makeup for the user based on the second makeup template, wherein the second makeup template is the makeup template previously selected by the user who has put on makeup.

[0081] As an example, a user who has put on makeup before putting on makeup (such as Figure 3 The fourth facial feature of the user's bare face facial information description a) shown refers to: the facial feature of the user who has put on makeup before the preset makeup robot applies makeup to the user based on the second makeup template.

[0082] It should be noted that before obtaining the training samples, the authorization of the user who has put on makeup has been obtained, and the user who has put on makeup has been informed that the obtained training samples are used to predict the makeup effect of the user who has put on makeup, and the obtained training samples will not be used in other scenarios unrelated to this application.

[0083] As an example, a first makeup effect label corresponding to the training sample is determined, where the first makeup effect label is a similarity between a makeup of a user who has put on makeup and a makeup of a second makeup template calculated based on a preset calculation formula;

[0084] As an example, if the training samples are labeled entirely manually, the similarity judgment may not be accurate due to the subjectivity of manual observation. In this embodiment, the similarity between the makeup of the user who has put on makeup and the makeup of the second makeup template is calculated based on a preset calculation formula, and the calculated similarity is used as the first makeup effect label.

[0085] As an example, the preset calculation formula may be a calculation formula of cosine similarity, a calculation formula of Tanmoto (Tanimoto Coefficient), etc., which is not limited here.

[0086] In this embodiment, when the user selects the first makeup template, the user is not directly made up based on the first makeup template. Instead, the user is made up based on the first facial feature of the user before makeup, the second facial feature of the face corresponding to the first makeup template after makeup, and the third facial feature of the face corresponding to the first makeup template before makeup. The preset effect prediction model is obtained by iteratively training the preset to-be-trained model based on the training samples with the first makeup effect label, and the predicted makeup effect achieved by the preset makeup robot when applying makeup to the user based on the first makeup template is predicted. A reference can be provided for the user to determine a satisfactory target makeup template based on the predicted makeup effect, and the preset makeup robot applies makeup to the user based on the target makeup template. This avoids the user being disappointed due to the large difference between the first makeup template and the makeup effect that can be achieved by the actual preset makeup robot, improves the user's satisfaction with the makeup effect, and thus improves the user's user experience.

[0087] Further, based on the above-mentioned embodiment of the present application, another embodiment of the present application is provided, referring to Figure 2 In this embodiment, before the step of obtaining the first facial feature of the user before putting on makeup, the method includes:

[0088] Step A1: Acquire the seventh facial feature of the user who has put on makeup;

[0089] As an example, the seventh facial feature of the user who has put on makeup (such as Figure 3 The facial information description a′) of the user after the machine has applied the candidate makeup refers to the facial features obtained by extracting features from the facial image of the user who has applied the makeup.

[0090] Step A2: determining the difference between the user's ideal makeup before and after applying makeup based on the sixth and fifth facial features, and determining the difference between the user's actual makeup before applying makeup and after applying makeup based on the second makeup template based on the fourth and seventh facial features;

[0091] As an example, the extracted facial features may be feature vectors obtained after vectorization processing, or feature matrices obtained after matrix processing.

[0092] As an example, the ideal makeup difference before and after makeup refers to the makeup difference between the corresponding face of the first makeup template that the user wants to achieve before and after makeup.

[0093] As an example, the actual makeup difference of the user before applying makeup and after applying makeup based on the second makeup template refers to the makeup difference that the user can achieve before applying makeup and after applying makeup based on the second makeup template by the makeup robot.

[0094] As an example, based on the sixth facial feature (such as Figure 3 The candidate makeup and no makeup face information description b) and the fifth face feature (such as Figure 3 The candidate makeup-free face information description b') shown in the figure can determine the difference between the user's ideal makeup before and after applying makeup by subtracting the fifth facial feature after applying makeup from the sixth facial feature in the form of a feature vector to obtain the difference between the user's ideal makeup before and after applying makeup, that is, to obtain the vector A = (a'-a).

[0095] As an example, based on the fourth facial feature and the seventh facial feature, the step of determining the actual makeup difference of the user before applying makeup and after applying makeup based on the second makeup template can be: subtracting the fourth facial feature in the form of a feature vector from the seventh facial feature to obtain the ideal makeup difference of the user before and after applying makeup, that is, obtaining the vector B = (b'-b).

[0096] Step A3: determining the similarity between the makeup of the user who has put on makeup and the makeup of the second makeup template based on the ideal makeup difference and the actual makeup difference, and using the similarity as the first makeup effect label corresponding to the training sample;

[0097] As an example, based on the ideal makeup difference and the actual makeup difference, the similarity between the makeup of the user who has applied makeup and the makeup of the second makeup template is determined, so that a more objective first makeup effect label can be obtained (such as Figure 3 The formula shown is: X=cos((a′-a), (b′-b)).

[0098] As an example, the preset calculation formula may be a calculation formula for cosine similarity. Specifically, the preset calculation formula is:

[0099]

[0100] Wherein, S is the similarity, A is the ideal makeup difference, B is the actual makeup difference, n is the number of users who have put on makeup, and i is the i-th user who has put on makeup.

[0101] As an example, the preset calculation formula may also be Tanmoto's calculation formula. Specifically, the preset calculation formula is:

[0102]

[0103] Wherein, S is the similarity, A is the ideal makeup difference, and B is the actual makeup difference.

[0104] In this embodiment, the preset effect prediction model (such as Figure 3 The machine learning model shown in FIG. 1 is obtained by iteratively training the preset model to be trained based on the training sample and the first makeup effect label calculated based on the preset calculation formula. Figure 3 The formula shown is: Loss = (YX) * (YX)), the preset model to be trained is iteratively trained to obtain the effect prediction model that meets the accuracy conditions, which can make the effect prediction model predict the makeup effect more accurately. After the effect prediction is performed based on the preset effect prediction model, it can provide users with a reference for selecting a makeup template more objectively, accurately and quickly.

[0105] Furthermore, based on the above embodiment of the present application, another embodiment of the present application is provided. In this embodiment, after the step of using the similarity as the first makeup effect label corresponding to the training sample, the method further includes:

[0106] Step S60: obtaining a second makeup effect label corresponding to the training sample, where the second makeup effect label is obtained by the makeup-wearing user evaluating the similarity between his / her own makeup and the makeup of the second makeup template;

[0107] As an example, due to the analysis based on the preset effect prediction model, the predicted makeup effect achieved by the preset makeup robot on the user based on the first makeup template is provided to the user for reference, wherein whether the user is satisfied with the predicted makeup effect is subjective to a certain extent. If the similarity between the makeup of the user who has already put on makeup and the makeup of the second makeup template is calculated completely based on the preset calculation formula, it may cause the user to miss the makeup template that he really wants. Therefore, this embodiment obtains the second makeup effect label obtained by the user evaluating the similarity between his own makeup and the makeup of the second makeup template.

[0108] Step S70: Based on preset weights, weight the first makeup effect label and the second makeup effect label to obtain a target makeup effect label, and iteratively train the preset model to be trained based on the training samples and the corresponding target makeup effect label to obtain an effect prediction model that meets the accuracy conditions.

[0109] As an example, the target makeup effect label may be an effect score (score) or an effect rating (good, medium, poor, etc.).

[0110] As an example, in order to obtain a more accurate makeup prediction effect, this embodiment weights the first makeup effect label and the second makeup effect label based on a preset weight to obtain a target makeup effect label. The preset weight may be determined based on historical experience data.

[0111] As an example, this embodiment weights the first makeup effect label and the second makeup effect label based on preset weights to obtain a target makeup effect label, and iteratively trains the preset model to be trained based on the weighted target makeup effect label and training samples. The obtained effect prediction model that meets the accuracy conditions can combine the subjectivity of manual labeling and the objectivity calculated based on a preset calculation formula to obtain a more comprehensive predicted makeup effect that is more suitable for users.

[0112] Further, based on the above embodiment of the present application, another embodiment of the present application is provided. In this embodiment, the step of obtaining a first facial feature of the user before makeup, a second facial feature of the face corresponding to the first makeup template selected by the user after makeup, and a third facial feature of the face corresponding to the first makeup template before makeup includes:

[0113] Step B1: obtaining the first facial information and facial structure information of the user before makeup;

[0114] As an example, when a user uses a makeup analysis device to apply makeup, the makeup analysis device will collect the user's facial information, and based on the user's facial information, apply makeup to the user or control a makeup robot to apply makeup to the user.

[0115] Specifically, the makeup analysis device will obtain the user's first facial information and facial structure information before makeup.

[0116] As an example, the first facial information reflects two-dimensional information of the user's face, and the first facial information includes information such as facial features and facial skin.

[0117] As an example, the facial structure information reflects the three-dimensional structural information of the user's face, and the facial structure information includes information such as bone structure and distribution of facial features.

[0118] Step B2: matching the user's facial structure information with the facial structure information of each facial model in the preset bare-face facial model library, and based on the matching degree of the corresponding facial structure information, screening from high to low to obtain a plurality of matching facial models, so that the user can select the first makeup template from the plurality of makeup templates corresponding to each matching facial model;

[0119] As an example, in order to preliminarily screen a makeup template suitable for the user, it is necessary to first screen a face model whose structure is similar to the user's face structure. The makeup template corresponding to the face model is more likely to be suitable for the user. Therefore, this embodiment matches the user's face structure information with the face structure information of each face model in a preset bare face model library (including celebrity face models or model face models, etc.), and based on the degree of matching of the corresponding face structure information, multiple matching face models are screened from high to low for the user to select.

[0120] As an example, based on the degree of matching, multiple matching face models are screened from high to low. The top three or top five face models with higher matching degrees can be screened and displayed on the display screen of the makeup analysis device. The user can rotate the face model by touching the display screen, which is convenient for the user to select.

[0121] As an example, each face model corresponds to multiple makeup templates, and multiple makeup templates corresponding to each matching face model can be displayed simultaneously for user selection.

[0122] As an example, the matching process can be to extract features from the user's facial structure information and the facial structure information of each face model in a preset bare face model library, and then compare the extracted features (calculate the similarity of the extracted features in vector form to obtain the degree of matching).

[0123] Step B3: extract features of the first facial information, the second facial information of the face corresponding to the first makeup template, and the third facial information of the face corresponding to the first makeup template before makeup, and obtain first facial features, second facial features, and third facial features respectively.

[0124] As an example, the second facial information of the face corresponding to the first makeup template reflects the two-dimensional information of the face.

[0125] As an example, the third facial information of the face before makeup corresponding to the first makeup template reflects the two-dimensional facial information of the face before makeup.

[0126] In this embodiment, after the step of analyzing based on the preset effect prediction model to obtain the predicted makeup effect achieved by the preset makeup robot applying makeup to the user based on the first makeup template, the method further includes:

[0127] Step S80: Based on a preset makeup migration model, the makeup corresponding to the first makeup template is migrated to the three-dimensional face model of the user to obtain a simulated makeup, so that the user can determine a satisfactory target makeup template based on the three-dimensional face model after makeup and the predicted makeup effect, wherein the three-dimensional face model is obtained by three-dimensional modeling based on the facial structure information of the user.

[0128] As an example, the first facial features, the second facial features and the third facial features extracted above are input into the effect prediction model to obtain the predicted makeup effect that the preset makeup robot will achieve by applying makeup to the user based on the first makeup template.

[0129] As an example, the preset makeup transfer model is obtained by training the makeup transfer model to be trained based on training samples with simulated makeup labels.

[0130] As an example, the three-dimensional face model is obtained by three-dimensional modeling based on the face structure information of the user.

[0131] As an example, if the user only selects a makeup template based on the predicted makeup effect, the user may only be able to judge by numerical values ​​or levels, which is not intuitive for the user. Therefore, this embodiment migrates the makeup corresponding to the first makeup template to the user's three-dimensional face model based on a preset makeup migration model to obtain simulated makeup. The user can determine a satisfactory target makeup template based on the three-dimensional face model after makeup and the predicted makeup effect. This improves the user experience.

[0132] In this embodiment, after the step of migrating the makeup corresponding to the first makeup template to the three-dimensional face model of the user based on the preset makeup migration model to obtain the three-dimensional face model after makeup, the method further includes:

[0133] Step S90: Based on a preset makeup evaluation model, the simulated makeup is evaluated to obtain a makeup evaluation of the simulated makeup, so that the user can determine a satisfactory target makeup template based on the three-dimensional face model after makeup, the predicted makeup effect, and the makeup evaluation of the simulated makeup.

[0134] As an example, in order to further assist users in selecting a makeup template, this embodiment performs a makeup evaluation on the simulated makeup based on a preset makeup evaluation model to obtain a makeup evaluation of the simulated makeup, so that the user can determine a satisfactory target makeup template based on the three-dimensional face model after makeup, the predicted makeup effect, and the makeup evaluation of the simulated makeup.

[0135] In this embodiment, after the user determines a satisfactory target makeup template, the makeup analysis device or the makeup analysis device controls the makeup robot to intelligently select corresponding makeup materials based on the target makeup template selected by the user. Specifically, the makeup robot identifies the location of the makeup materials based on a preset material recognition model, retrieves the materials, and performs makeup on the user.

[0136] In this embodiment, after the makeup robot performs makeup on the user, the makeup of the user after makeup is scored based on a preset makeup evaluation model for the user's reference, thereby completing the fully automatic makeup process.

[0137] As an example, the user's makeup before makeup can also be scored based on a preset makeup evaluation model, and the makeup difference (makeup improvement) between the user's makeup before makeup and after makeup can be calculated for the user's reference.

[0138] In this embodiment, a fully automatic makeup process is provided, including facial information collection, facial model matching, makeup template selection based on the three-dimensional facial model after makeup, the predicted makeup effect and makeup evaluation of the simulated makeup, automatic makeup application and makeup scoring, so as to enhance the user experience.

[0139] Reference Figure 4 , Figure 4 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present application.

[0140] like Figure 4 As shown, the makeup analysis device may include: a processor 1001 , a memory 1005 , and a communication bus 1002 . The communication bus 1002 is used to realize the connection and communication between the processor 1001 and the memory 1005 .

[0141] Optionally, the makeup analysis device may further include a user interface, a network interface, a camera, an RF (Radio Frequency) circuit, a sensor, a WiFi module, etc. The user interface may include a display screen (Display), an input submodule such as a keyboard (Keyboard), and the optional user interface may also include a standard wired interface and a wireless interface. The network interface may include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0142] Those skilled in the art will understand that Figure 4 The makeup analysis device structure shown in the figure does not constitute a limitation of the makeup analysis device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0143] like Figure 4 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, and a makeup analysis program. The operating system is a program that manages and controls the hardware and software resources of the makeup analysis device, and supports the operation of the makeup analysis program and other software and / or programs. The network communication module is used to realize communication between the components inside the memory 1005, and communication with other hardware and software in the makeup system.

[0144] exist Figure 4 In the makeup analysis device shown, the processor 1001 is used to execute the makeup analysis program stored in the memory 1005 to implement the steps of any of the above-mentioned makeup analysis methods.

[0145] The specific implementation of the makeup analysis device of the present application is basically the same as the embodiments of the above-mentioned makeup analysis method, and will not be repeated here.

[0146] The present application also provides a makeup analysis device, such as Figure 5 As shown, the device comprises:

[0147] The feature acquisition module 10 is used to acquire a first facial feature of the user before makeup, a second facial feature of the face corresponding to the first makeup template selected by the user after makeup, and a third facial feature of the face corresponding to the first makeup template before makeup;

[0148] The effect prediction module 20 is used to input the first facial feature, the second facial feature and the third facial feature into a preset effect prediction model, and perform analysis based on the preset effect prediction model to obtain a predicted makeup effect that a preset makeup robot will achieve by applying makeup to the user based on the first makeup template, so that the user can determine a satisfactory target makeup template based on the predicted makeup effect, and the preset makeup robot applies makeup to the user based on the target makeup template;

[0149] The preset effect prediction model is obtained by obtaining training samples, determining the first makeup effect label corresponding to the training samples, and iteratively training the preset model to be trained based on the training samples and the corresponding first makeup effect label, the training samples including the fourth facial feature of the user who has put on makeup before makeup, the fifth facial feature of the face corresponding to the second makeup template selected by the user who has put on makeup after makeup, and the sixth facial feature of the face corresponding to the second makeup template before makeup, the makeup of the user who has put on makeup is obtained by applying makeup based on the second makeup template by a preset makeup robot, and the first makeup effect label is the similarity between the makeup of the user who has put on makeup and the makeup of the second makeup template calculated based on a preset calculation formula.

[0150] Optionally, in a possible implementation manner of the present application, before the step of obtaining the first facial feature of the user before putting on makeup, the device includes:

[0151] A feature acquisition module, used to acquire the seventh facial feature of the user who has put on makeup;

[0152] a makeup difference calculation module, configured to determine the difference between the user's ideal makeup before and after applying makeup based on the sixth facial feature and the fifth facial feature, and to determine the difference between the user's actual makeup before applying makeup and after applying makeup based on the second makeup template based on the fourth facial feature and the seventh facial feature;

[0153] a similarity calculation module, configured to determine the similarity between the makeup of the user who has applied makeup and the makeup of the second makeup template based on the ideal makeup difference and the actual makeup difference, and use the similarity as the first makeup effect label corresponding to the training sample;

[0154] Wherein, the preset calculation formula is:

[0155]

[0156] Wherein, S is the similarity, A is the ideal makeup difference, B is the actual makeup difference, n is the number of users who have put on makeup, and i is the i-th user who has put on makeup.

[0157] Optionally, in a possible implementation manner of the present application, after the step of using the similarity as the first makeup effect label corresponding to the training sample, the device further includes:

[0158] a label acquisition module, used to acquire a second makeup effect label corresponding to the training sample, where the second makeup effect label is obtained by the makeup user evaluating the similarity between his / her own makeup and the makeup of the second makeup template;

[0159] The second iterative training module is used to weight the first makeup effect label and the second makeup effect label based on preset weights to obtain a target makeup effect label, so as to iteratively train the preset model to be trained based on the training samples and the corresponding target makeup effect label to obtain an effect prediction model that meets the accuracy conditions.

[0160] Optionally, in a possible implementation manner of the present application, the feature acquisition module 10 includes:

[0161] An information acquisition unit, used to acquire the first facial information and facial structure information of the user before putting on makeup;

[0162] an information matching unit, configured to match the facial structure information of the user with the facial structure information of each facial model in a preset bare-face facial model library, and to screen and obtain a plurality of matching facial models from high to low based on the matching degree of the corresponding facial structure information, so that the user can select the first makeup template from a plurality of makeup templates corresponding to each matching facial model;

[0163] The feature extraction unit is used to extract features from the first facial information, the second facial information of the face corresponding to the first makeup template, and the third facial information of the face corresponding to the first makeup template before makeup, respectively, to obtain first facial features, second facial features, and third facial features accordingly.

[0164] Optionally, in a possible implementation manner of the present application, after the step of analyzing based on the preset effect prediction model to obtain a predicted makeup effect achieved by the preset makeup robot applying makeup to the user based on the first makeup template, the device further includes:

[0165] The makeup simulation module is used to transfer the makeup corresponding to the first makeup template to the user's three-dimensional face model based on a preset makeup migration model to obtain simulated makeup, so that the user can determine a satisfactory target makeup template based on the three-dimensional face model after makeup and the predicted makeup effect, wherein the three-dimensional face model is obtained by three-dimensional modeling based on the user's facial structure information.

[0166] Optionally, in a possible implementation manner of the present application, after the step of migrating the makeup corresponding to the first makeup template to the three-dimensional face model of the user based on a preset makeup migration model to obtain the three-dimensional face model after makeup, the device further includes:

[0167] The template determination module is used to perform makeup evaluation on the simulated makeup based on a preset makeup evaluation model to obtain a makeup evaluation of the simulated makeup, so that the user can determine a satisfactory target makeup template based on the three-dimensional face model after makeup, the predicted makeup effect, and the makeup evaluation of the simulated makeup.

[0168] Optionally, in a possible implementation manner of the present application, the first facial feature includes at least one of facial texture features, facial features, and facial features.

[0169] The specific implementation of the makeup analysis device of the present application is basically the same as the embodiments of the above-mentioned makeup analysis method, and will not be repeated here.

[0170] The present application also provides a storage medium, on which a makeup analysis program is stored. When the makeup analysis program is executed by a processor, the steps of the makeup analysis method as described in any one of the above items are implemented.

[0171] The specific implementation of the storage medium of the present application is basically the same as the embodiments of the above-mentioned makeup analysis method, and will not be repeated here.

[0172] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0173] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0174] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0175] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A makeup analysis method, characterized in that: The following steps are involved: Acquire a first facial feature of the user before makeup, a second facial feature of the face corresponding to the first makeup template selected by the user after makeup, and a third facial feature of the face corresponding to the first makeup template before makeup; Inputting the first facial feature, the second facial feature, and the third facial feature into a preset effect prediction model, performing analysis based on the preset effect prediction model, and obtaining a predicted makeup effect that a preset makeup robot will achieve by applying makeup to the user based on the first makeup template, so that the user can determine a satisfactory target makeup template based on the predicted makeup effect, and the preset makeup robot applies makeup to the user based on the target makeup template; Among them, the preset effect prediction model is obtained by obtaining training samples, determining the first makeup effect label corresponding to the training samples, and iteratively training the preset model to be trained based on the training samples and the corresponding first makeup effect label, the training samples include the fourth facial features of the user who has put on makeup before makeup, the fifth facial features of the face corresponding to the second makeup template selected by the user who has put on makeup after makeup, and the sixth facial features of the face corresponding to the second makeup template before makeup, the makeup of the user who has put on makeup is obtained by applying makeup based on the second makeup template by a preset makeup robot, and the first makeup effect label is the similarity between the makeup of the user who has put on makeup and the makeup of the second makeup template calculated based on a preset calculation formula.

2. The makeup analysis method according to claim 1, characterized in that: Before the step of obtaining the first facial feature of the user before putting on makeup, the method includes: Obtain the seventh facial feature of the user who has put on makeup; Based on the sixth and fifth facial features, determining the difference between the user's ideal makeup before and after applying makeup, and based on the fourth and seventh facial features, determining the difference between the user's actual makeup before applying makeup and after applying makeup based on the second makeup template; Based on the ideal makeup difference and the actual makeup difference, determine the similarity between the makeup of the user who has put on makeup and the makeup of the second makeup template, and use the similarity as the first makeup effect label corresponding to the training sample; Wherein, the preset calculation formula is: Wherein, S is the similarity, A is the ideal makeup difference, B is the actual makeup difference, n is the number of users who have put on makeup, and i is the i-th user who has put on makeup.

3. The makeup analysis method according to claim 2, characterized in that: After the step of using the similarity as the first makeup effect label corresponding to the training sample, the method further includes: Obtaining a second makeup effect label corresponding to the training sample, where the second makeup effect label is obtained by a user who has put on makeup evaluating the similarity between his / her own makeup and the makeup of the second makeup template; Based on the preset weight, the first makeup effect label and the second makeup effect label are weighted to obtain the target makeup effect label, and based on the training samples and the corresponding target makeup effect label, the preset model to be trained is iteratively trained to obtain an effect prediction model that meets the accuracy conditions.

4. The makeup analysis method according to claim 1, characterized in that: The step of obtaining a first facial feature of the user before makeup, a second facial feature of the face corresponding to the first makeup template selected by the user after makeup, and a third facial feature of the face corresponding to the first makeup template before makeup includes: Obtain the user's first facial information and facial structure information before makeup; Matching the user's facial structure information with the facial structure information of each facial model in a preset bare-face facial model library, and filtering from high to low to obtain a plurality of matching facial models based on the matching degree of the corresponding facial structure information, so that the user can select the first makeup template from a plurality of makeup templates corresponding to each matching facial model; Feature extraction is performed on the first facial information, the second facial information of the face corresponding to the first makeup template, and the third facial information of the face corresponding to the first makeup template before makeup, respectively, to obtain first facial features, second facial features, and third facial features accordingly.

5. The makeup analysis method according to claim 4, characterized in that: After the step of analyzing based on the preset effect prediction model to obtain the predicted makeup effect achieved by the preset makeup robot applying makeup to the user based on the first makeup template, the method further includes: Based on a preset makeup migration model, the makeup corresponding to the first makeup template is migrated to the three-dimensional face model of the user to obtain a simulated makeup, so that the user can determine a satisfactory target makeup template based on the three-dimensional face model after makeup and the predicted makeup effect; The three-dimensional face model is obtained by three-dimensional modeling based on the face structure information of the user.

6. The makeup analysis method according to claim 5, characterized in that: After the step of migrating the makeup corresponding to the first makeup template to the three-dimensional face model of the user based on the preset makeup migration model to obtain the three-dimensional face model after makeup, the method further includes: Based on a preset makeup evaluation model, the simulated makeup is evaluated to obtain a makeup evaluation of the simulated makeup, so that the user can determine a satisfactory target makeup template based on the three-dimensional face model after makeup, the predicted makeup effect, and the makeup evaluation of the simulated makeup.

7. The makeup analysis method according to any one of claims 1 to 6, characterized in that: The first facial feature includes at least one of facial texture features, facial features, and facial features.

8. A makeup analysis device, characterized in that: The device comprises: A feature acquisition module, used to acquire a first facial feature of the user before makeup, a second facial feature of the face corresponding to the first makeup template selected by the user after makeup, and a third facial feature of the face corresponding to the first makeup template before makeup; an effect prediction module, for inputting the first facial feature, the second facial feature, and the third facial feature into a preset effect prediction model, and performing analysis based on the preset effect prediction model to obtain a predicted makeup effect that a preset makeup robot will achieve by applying makeup to the user based on the first makeup template, so that the user can determine a satisfactory target makeup template based on the predicted makeup effect, and the preset makeup robot applies makeup to the user based on the target makeup template; The preset effect prediction model is obtained by iteratively training a preset model to be trained based on training samples with target makeup effect labels.

9. A makeup analysis device, characterized in that: The device comprises: a memory, a processor, and a makeup analysis program stored in the memory and executable on the processor, wherein the makeup analysis program is configured to implement the steps of the makeup analysis method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a makeup analysis program, which, when executed by a processor, implements the steps of the makeup analysis method according to any one of claims 1 to 7.

Images