Face recognition-based beauty makeup recommendation method and device, equipment and medium

Abstract

The invention discloses a face recognition-based beauty and makeup recommendation method, device and equipment and a medium, and relates to the technical field of artificial intelligence, and the face recognition-based beauty and makeup recommendation method comprises the steps of obtaining face information of a target user; the face information of the target user is input into a beauty makeup recommendation model, a face analysis result and beauty makeup suggestion information of the target user are obtained, and the beauty makeup suggestion information comprises recommended makeup and a weight value corresponding to the recommended makeup; performing preference scoring on the recommended makeup according to the face analysis result through a collaborative filtering recommendation algorithm to obtain a predicted preference scoring result of the target user on the recommended makeup; the weight value of the recommended makeup is adjusted according to the predicted preference degree scoring result in combination with the relation structure in the beauty makeup knowledge graph, the final recommended makeup for the target user is obtained, and the obtained final recommended makeup for the target user can improve the precision of beauty makeup recommendation.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a beauty recommendation method, device, equipment and medium based on face recognition. Background Art

[0002] Appropriate makeup can enhance personal confidence, shape personal image, and is a way of cultural expression and identity. In social occasions, appropriate makeup can show respect for the event and attention to other participants. In some cultures, specific social activities require participants to wear makeup to comply with etiquette. Therefore, appropriate makeup has become an indispensable part of personal life and social activities.

[0003] In the current beauty recommendation market, although there are a variety of beauty recommendation systems and methods, they still have many shortcomings in terms of accuracy. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to provide a beauty recommendation method, device, equipment and medium based on face recognition, aiming to improve the accuracy of beauty recommendations.

[0005] To achieve the above purpose, one aspect of an embodiment of the present application proposes a beauty makeup recommendation method based on face recognition, comprising the following steps:

[0006] Obtain the target user’s facial information;

[0007] Inputting the target user's facial information into a beauty recommendation model to obtain a facial analysis result and beauty recommendation information of the target user, wherein the beauty recommendation information includes a recommended makeup and a weight value corresponding to the recommended makeup, wherein the facial analysis result and the recommended makeup conform to the relational structure in the beauty knowledge graph, and the beauty recommendation model is generated by training a neural network through user portraits and combining the beauty knowledge graph;

[0008] Using a collaborative filtering recommendation algorithm to rate the recommended makeup according to the facial analysis result, and obtain a predicted rating result of the target user's preference for the recommended makeup;

[0009] The weight value of the recommended makeup is adjusted according to the predicted preference score result and the relationship structure in the beauty knowledge graph to obtain a final recommended makeup for the target user.

[0010] In some embodiments, before acquiring the facial information of the target user, the process includes:

[0011] Get physical elements in the beauty field;

[0012] According to the association relationship between the entity elements, a relationship structure in the beauty knowledge graph is established;

[0013] An initial weight value is set for the relationship structure in the beauty knowledge graph to obtain the beauty knowledge graph.

[0014] In some embodiments, training a neural network through user portraits and combining the beauty knowledge graph to generate the beauty recommendation model includes:

[0015] Acquire user data, wherein the user data includes user facial feature data and user behavior data, wherein the user behavior data is associated with beauty makeup;

[0016] Establishing a user portrait based on the user facial feature data and the user behavior data;

[0017] Using the label content in the user portrait as a data set to train the neural network;

[0018] The beauty recommendation model is generated based on the output result of the neural network and the beauty knowledge graph.

[0019] In some embodiments, using the label content in the user portrait as a data set to train the neural network includes:

[0020] Analyze the tag content in the user portrait to obtain data features of the tag content;

[0021] Performing characterization processing on the tag content according to the data feature to obtain characteristic data corresponding to the tag content;

[0022] The neural network is trained by using the feature data as a data set of the neural network.

[0023] In some embodiments, the characterization processing of the tag content according to the data feature includes:

[0024] If the data feature is a classification feature, the label content is labeled according to the corresponding classification standard;

[0025] If the data feature is a continuous feature, the label content is normalized;

[0026] If the data feature is a multi-category feature, the label content is represented by a vector through an embedding layer.

[0027] In some embodiments, the step of performing a preference rating on the recommended makeup according to the facial analysis result using a collaborative filtering recommendation algorithm to obtain a predicted preference rating result of the target user for the recommended makeup includes:

[0028] Determine a plurality of similar user profiles that are similar to the user profile of the target user;

[0029] Based on the multiple similar user portraits, the recommended makeup is scored for preference, to obtain multiple preference scoring results;

[0030] An average value of the plurality of preference rating results is calculated to obtain a predicted preference rating result of the target user for the recommended makeup.

[0031] In some embodiments, the likability rating of the recommended makeup based on the multiple similar user portraits to obtain multiple likability rating results includes:

[0032] Inputting the data of the plurality of similar user portraits into a prediction scoring formula to obtain a plurality of preference scoring results;

[0033] The prediction score formula is expressed as:

[0034]

[0035] Where score(u,p) represents the predicted score, u represents the user, p represents the recommended makeup, N(u) represents the set of multiple users most similar to user u, sim(u,v) represents the similarity between user u and user v, R v,p Indicates the rating or behavior intensity of user v on p.

[0036] To achieve the above-mentioned purpose, another aspect of the embodiment of the present application provides a beauty makeup recommendation device based on face recognition, and the beauty makeup recommendation device based on face recognition includes:

[0037] An acquisition module is used to obtain facial information of the target user;

[0038] An input module, used to input the facial information of the target user into a beauty recommendation model, to obtain a facial analysis result and beauty recommendation information of the target user, wherein the beauty recommendation information includes a recommended makeup and a weight value corresponding to the recommended makeup, wherein the facial analysis result and the recommended makeup conform to the relational structure in the beauty knowledge graph, and the beauty recommendation model is generated by training a neural network through user portraits and combining the beauty knowledge graph;

[0039] A scoring module, used to score the recommended makeup according to the facial analysis result by using a collaborative filtering recommendation algorithm, and obtain a predicted score result of the target user's preference for the recommended makeup;

[0040] The recommendation module is used to adjust the weight value of the recommended makeup according to the predicted preference score result combined with the relationship structure in the beauty knowledge graph to obtain the final recommended makeup for the target user.

[0041] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned beauty recommendation method based on face recognition when executing the computer program.

[0042] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes a computer-readable storage medium, on which a computer program is stored. When the above-mentioned computer program is executed by one or more processors, it can implement the steps of the above-mentioned beauty recommendation method based on face recognition.

[0043] The embodiments of the present application include at least the following beneficial effects:

[0044] The present application provides a beauty recommendation method, apparatus, device and medium based on face recognition. In an embodiment of the present application, first, facial information of a target user is obtained; the facial information of the target user is input into a beauty recommendation model to obtain a facial analysis result and beauty suggestion information of the target user, wherein the beauty suggestion information includes a recommended makeup and a weight value corresponding to the recommended makeup, the facial analysis result and the recommended makeup conform to the relational structure in a beauty knowledge graph, and the beauty recommendation model is generated by training a neural network through user portraits and combining the beauty knowledge graph; then, a preference score is performed on the recommended makeup according to the facial analysis result through a collaborative filtering recommendation algorithm to obtain a predicted preference score result of the target user for the recommended makeup; finally, the weight value of the recommended makeup is adjusted according to the predicted preference score result combined with the relational structure in the beauty knowledge graph to obtain a final recommended makeup for the target user. In the embodiment of the present application, facial information of a target user is input into a beauty recommendation model to obtain facial analysis results and beauty recommendation information of the target user. Since the beauty recommendation model is generated by training a neural network based on user portraits in combination with a beauty knowledge graph, targeted beauty recommendation information can be generated based on the characteristics of the target user, wherein the beauty recommendation information includes a recommended makeup and a weight value corresponding to the recommended makeup. Furthermore, a collaborative filtering recommendation algorithm is used to perform a preference score on the recommended makeup based on the facial analysis results and to adjust the weight value of the recommended makeup. The final recommended makeup for the target user obtained in this way can improve the accuracy of the beauty recommendation.

[0045] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0047] Figure 1 is a flowchart of a beauty makeup recommendation method based on face recognition provided by some embodiments of the present application;

[0048] Figure 2 is a schematic block diagram of modules of a beauty makeup recommendation device based on face recognition provided by some embodiments of the present application;

[0049] Figure 3 It is a schematic diagram of the hardware structure of an electronic device provided in some embodiments of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the reference to "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. The appearance of this phrase in various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0051] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0052] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0054] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0055] In order to make the inventive concept of the present application easy to understand, before describing the embodiments of the present application in detail, the English abbreviations (terms) / related concepts involved in the embodiments of the present application are first described. The English abbreviations (terms) / related concepts involved in the embodiments of the present application are subject to the following explanations.

[0056] Knowledge Graph: A knowledge graph is a structured semantic knowledge base that represents entities and various relationships between entities in the form of a graph. Entities in a knowledge graph refer to various objects in the real world, such as people, places, things, etc.; relationships in a knowledge graph refer to various associations between entities, such as "belongs to", "is located in", "creates", etc.; attributes in a knowledge graph are used to describe the characteristics of an entity, such as a person's age, a place's address, etc.

[0057] Beauty knowledge graph: The beauty knowledge graph is a knowledge graph specifically built for the beauty field. It organizes relevant knowledge such as beauty products, brands, ingredients, effects, usage methods, applicable populations, etc. in the form of a graph to provide users with more accurate and personalized beauty information and services.

[0058] User Profile: A user profile is an abstract representation created by collecting and analyzing various user data. It depicts a user's characteristics, behaviors, preferences, and needs.

[0059] Appropriate makeup can enhance personal confidence, shape personal image, and is a way of cultural expression and identity. In social occasions, appropriate makeup can show respect for the event and attention to other participants. In some cultures, specific social activities require participants to wear makeup to comply with etiquette. Therefore, appropriate makeup has become an indispensable part of personal life and social activities.

[0060] In the current beauty recommendation market, although there are a variety of beauty recommendation systems and methods, they still have many shortcomings in terms of accuracy, personalization and user experience.

[0061] When considering the personalized needs of users, existing beauty recommendation systems are often limited to basic information such as the user's age, gender, skin type, and facial features. However, they ignore more detailed factors such as the user's personal preferences (such as favorite makeup style and favorite color), scene requirements, and environmental factors (such as weather and season), resulting in a lack of personalization and differentiation in the recommended beauty solutions.

[0062] In the current makeup teaching field, a common problem is that teaching content often adopts a "one-size-fits-all" approach, that is, the same set of teaching content is applied to all users, lacking accurate and personalized teaching for different users' facial features, skill levels and personalized needs. This teaching method results in users often having only a vague understanding of key information such as the location of makeup application and the products to be used after watching the tutorial, making it difficult to achieve the expected learning effect.

[0063] Existing beauty recommendation systems have deficiencies in algorithm optimization and are often unable to dynamically adjust and optimize based on users' actual feedback and behavioral data, resulting in difficulty in continuously improving the accuracy and personalization of recommendation results.

[0064] In view of this, the present application proposes a beauty recommendation method, device, equipment and medium based on face recognition, which obtains the facial information of the target user; inputs the facial information of the target user into a beauty recommendation model to obtain the facial analysis result and beauty suggestion information of the target user, the beauty suggestion information includes a recommended makeup and a weight value corresponding to the recommended makeup, the facial analysis result and the recommended makeup conform to the relational structure in the beauty knowledge graph, and the beauty recommendation model is generated by training a neural network through user portraits and combining it with the beauty knowledge graph; then, the recommended makeup is scored for preference according to the facial analysis result through a collaborative filtering recommendation algorithm, and the predicted preference score result of the target user for the recommended makeup is obtained; finally, the weight value of the recommended makeup is adjusted according to the predicted preference score result combined with the relational structure in the beauty knowledge graph to obtain the final recommended makeup for the target user.

[0065] In the embodiment of the present application, facial information of a target user is input into a beauty recommendation model to obtain facial analysis results and beauty recommendation information of the target user. Since the beauty recommendation model is generated by training a neural network based on user portraits in combination with a beauty knowledge graph, targeted beauty recommendation information can be generated based on the characteristics of the target user, wherein the beauty recommendation information includes a recommended makeup and a weight value corresponding to the recommended makeup. Furthermore, a collaborative filtering recommendation algorithm is used to perform a preference score on the recommended makeup based on the facial analysis results and to adjust the weight value of the recommended makeup. The final recommended makeup for the target user obtained in this way can improve the accuracy of the beauty recommendation.

[0066] The method provided in the embodiment of the present application can be applied to the electronic device provided in the embodiment of the present application, wherein the electronic device can be a terminal or a server.

[0067] The terminal may be a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto.

[0068] The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, as well as big data and artificial intelligence platforms.

[0069] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0070] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0071] The following will describe in detail the implementation steps of a beauty recommendation method based on face recognition provided by an embodiment of the present application in conjunction with the accompanying drawings.

[0072] Please refer to Figure 1 , Figure 1 A flowchart of a beauty makeup recommendation method based on face recognition provided for some embodiments of the present application. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than that shown here.

[0073] The method of the embodiment of the present application comprises the following steps:

[0074] Step 101: Obtain facial information of a target user;

[0075] Step 102: Input the target user's facial information into the beauty recommendation model to obtain the target user's facial analysis results and beauty recommendation information, the beauty recommendation information including the recommended makeup and the weight value corresponding to the recommended makeup. The facial analysis results and the recommended makeup conform to the relationship structure in the beauty knowledge graph. The beauty recommendation model is generated by training the neural network through the user portrait and combining it with the beauty knowledge graph;

[0076] Step 103: Using a collaborative filtering recommendation algorithm to rate the recommended makeup according to the facial analysis results, and obtain a predicted rating result of the target user's preference for the recommended makeup;

[0077] Step 104: According to the predicted preference score result and the relationship structure in the beauty knowledge graph, the weight value of the recommended makeup is adjusted to obtain the final recommended makeup for the target user.

[0078] In steps 101 to 104 shown in the embodiment of the present application, facial information of the target user is input into a beauty recommendation model to obtain facial analysis results and beauty recommendation information of the target user. Since the beauty recommendation model is generated by training a neural network based on user portraits in combination with a beauty knowledge graph, it is possible to generate targeted beauty recommendation information based on the characteristics of the target user, wherein the beauty recommendation information includes a recommended makeup and a weight value corresponding to the recommended makeup. Furthermore, a collaborative filtering recommendation algorithm is used to perform a preference score on the recommended makeup based on the facial analysis results and to adjust the weight value of the recommended makeup. The final recommended makeup for the target user obtained in this way can improve the accuracy of the beauty recommendation.

[0079] The specific implementation methods of the above steps are introduced below.

[0080] In step 101, facial information of a target user is obtained.

[0081] The target user is the user targeted by the beauty recommendation. For makeup teaching, the target user is the model used to teach makeup; for personal makeup, the target user is the person who needs makeup and styling.

[0082] Facial information includes the user's facial features. The user's facial features can be obtained by taking a facial photo of the user with a camera, or obtaining a public facial photo from the user's social media account, etc. This application does not impose any restrictions on this.

[0083] The embodiment provided in the present application obtains the facial information of the target user, thereby providing a basis for subsequently recommending personalized and accurate makeup for the target user.

[0084] Before obtaining the facial information of the target user, some implementations of the present application may also include establishing a beauty knowledge graph.

[0085] Establishing a beauty knowledge graph may include obtaining entity elements in the beauty field; establishing a relational structure in the beauty knowledge graph based on the association between the entity elements; setting an initial weight value for the relational structure in the beauty knowledge graph to obtain the beauty knowledge graph.

[0086] Optionally, user data can be collected through a variety of channels, including but not limited to cameras (for capturing facial features), voice recognition (for obtaining user instructions), and behavioral data when users use the software. Specific types of data collected include user skin information (obtained through image analysis technology), facial features (such as face shape, facial features ratio, etc.), historical purchase records, browsing behavior (such as browsed beauty product categories, dwell time), and click behavior (clicked product links, advertisements, etc.). These data together form the basis for building personalized beauty recommendations and can serve as entity elements in the beauty field.

[0087] Exemplarily, entity elements in the field of beauty and cosmetics may include products: various beauty and cosmetics products, such as lipstick, foundation, eye shadow, etc.; brands: various beauty and cosmetics brands; ingredients: chemical or natural ingredients in the products, such as hyaluronic acid, vitamin C, etc.; effects: expected effects after using the products, such as moisturizing, whitening, concealing, etc.; usage: correct steps and techniques for using the products; applicable groups: user groups suitable for using specific products, such as dry skin, sensitive skin, etc.; skin type, facial features, makeup style, makeup techniques, makeup features, etc.

[0088] According to the association between entity elements, the relationship structure in the beauty knowledge graph is established. These relationship structures can be ingredient-inclusion relationships, such as the relationship between a product and its ingredients; brand-attribution relationships, such as the relationship between a product and its brand; applicable effect relationships, such as the relationship between a product and the effect it can achieve; usage suggestion relationships, such as the relationship between a product and its usage method; and skin-suitability relationships, such as the relationship between a product and its applicable population. The relationship structure can be expressed as "product-inclusion-ingredients", "product-suitable-skin-type", "skin-type-need-effects", etc.

[0089] After determining the relationship structure between entities and entities, initial weights can be set in each layer of relationship-entity. These weights reflect the importance of different relationships in the beauty recommendation process.

[0090] Through the above steps, a beauty knowledge graph for beauty can be established. By establishing a beauty knowledge graph, the acquired facial information of the target user can be associated with the corresponding entity as an entity and the relationship structure can be determined, providing a basis for beauty recommendations for the target user.

[0091] In step 102, the facial information of the target user is input into the beauty recommendation model to obtain the facial analysis results and beauty suggestion information of the target user. The beauty suggestion information includes recommended makeup and weight values ​​corresponding to the recommended makeup. The facial analysis results and recommended makeup conform to the relational structure in the beauty knowledge graph. The beauty recommendation model is generated by training a neural network through user portraits and combining it with the beauty knowledge graph.

[0092] It should be understood that the beauty recommendation model can be a trained model. When recommending beauty for a target user, it is only necessary to input the target user's facial information into the beauty recommendation model to obtain the target user's facial analysis results and beauty suggestion information.

[0093] The beauty recommendation model can be generated by training the neural network through user portraits and combining it with the beauty knowledge graph.

[0094] In some embodiments of the present application, user facial feature data and user behavior data can be obtained, wherein the user behavior data is associated with beauty makeup. Subsequently, a user portrait is established based on the user facial feature data and the user behavior data, and the label content in the user portrait is used as a data set to train a neural network. A beauty makeup recommendation model is generated based on the output results of the neural network combined with the beauty makeup knowledge graph.

[0095] Exemplarily, the user's facial feature data can be obtained by performing facial feature analysis on the user's photo. By performing facial feature analysis on the user's photo, the user's three parts and five eyes, inner contour, outer contour, face shape, eye shape, nose shape and lip shape can be obtained.

[0096] Exemplarily, user behavior data can be collected through the user's use of an application (APP), such as recording the user's unique identifier (user ID), behavior type (browsing beauty-related web pages, clicking on beauty-related links, purchasing beauty-related products, etc.), behavior time, associated beauty content ID, and other information.

[0097] After collecting the user's facial feature data and user behavior data, the corresponding user profile can be established based on these data. This includes extracting key information such as the user's favorite makeup style, learned makeup techniques, and skin characteristics based on the user behavior data. Then, the user profile is labeled based on the extracted features, such as user A, female, 19 years old, likes Korean makeup, and is a makeup novice with dry skin. The labels in the user profile can be entities in the knowledge graph.

[0098] The label content in the user portrait can be used as a data set to train the neural network. This process can be done by analyzing the label content in the user portrait to obtain the data features of the label content, then characterizing the label content based on the data features to obtain feature data corresponding to the label content, and then using the feature data as the data set of the neural network to train the neural network.

[0099] The neural network can be a feedforward neural network, a convolutional neural network, a recurrent neural network, a long short-term memory network, an adversarial network, an attention mechanism or a Transformer network, and this application does not limit this.

[0100] The training of neural networks can include steps such as data preprocessing, determining the neural network structure, selecting the loss function, selecting the optimization algorithm, training the network, and tuning the hyperparameters. Among them, data preprocessing is to clean the data and remove invalid or erroneous data; normalize / standardize the data and scale the data to a small specific range, such as between 0 and 1; increase the diversity of the data by rotating, scaling, cropping, etc.; and divide the data set into a training set, a validation set, and a test set. Determining the neural network structure includes selecting the model type, for example, using a convolutional neural network, and then determining the number of layers of the network, the neuron data and activation function of each layer, etc., and setting the initial values ​​for the network parameters. The loss function can be selected as mean square error or cross entropy loss, etc. Selecting the optimization algorithm is to determine which optimization algorithm to use to update the weights of the network, such as stochastic gradient descent (SGD), Adam, RMSprop, etc.

[0101] The label content in the user portrait is analyzed to obtain data features of the label content. The label content can include various types, each type corresponds to one or more data features, and these data features can be classification features, continuity features, or multi-category features.

[0102] In some embodiments, characterizing the label content according to the data features may include: when the data features are classification features, label encoding the label content according to the corresponding classification standard; when the data features are continuous features, normalizing the label content; when the data features are multi-category features, vectorizing the label content through an embedding layer.

[0103] For example, for classification features such as gender, skin quality, face shape, etc., these features usually exist in text form. In order to make them understandable to the machine learning model, label encoding (Label Encoding) numerical means can be used. For example, gender features can be encoded as: male = 0, female = 1; skin quality features may be encoded as: dry = 0, oily = 1, mixed = 2, etc. according to different classification standards.

[0104] For continuous features such as age, since the value range of such features is relatively wide, direct use may lead to gradient vanishing or gradient exploding problems, which in turn affects the performance of the model. Therefore, it is necessary to scale the range of the data to a specific interval (such as the interval range of 0 to 1 or -1 to 1) through normalization, so that features of different dimensions or ranges have similar weights in the model.

[0105] For features with a large number of categories and a relatively small number of samples in each category (such as the user's favorite style, favorite color, etc.), an embedding layer can be used as a vector representation. These vectors can capture the potential relationships and similarities between different categories in the embedding space, reduce the dimension and capture the potential relationships.

[0106] Through the above steps, the data set of the neural network can be obtained.

[0107] An example of a dataset can be found in Table 1.

[0108] Table 1 Examples of neural network datasets

[0109]

[0110] The generated data set is used as training data to train the neural network, and the content output after training is interacted with the beauty knowledge graph to obtain the corresponding entity content from the graph, and finally the facial analysis results and personalized beauty recommendations are output.

[0111] Exemplarily, the output facial analysis result may be:

[0112] Your face shape: round face, full and plump, like a cute apple, full of innocence and affinity;

[0113] Suitable eyebrow shape: willow leaf eyebrow;

[0114] Recommended makeup: oxygen-sense boiled water makeup, boiled water eye makeup, low-saturation, world-weary, tipsy makeup;

[0115] Results of the Three Courts and Five Eyes analysis:

[0116] The Third Chamber: The Chief of the Lower Chamber;

[0117] Ratio of three courts: 1:08:1.5;

[0118] Five-eye analysis: eyes are far apart;

[0119] The ratio of the five courts: 1:0.8:1.5:0.8:1.

[0120] Suitable eyebrow shapes and recommended makeup can be obtained through the beauty knowledge graph. For example, the interaction process with the beauty knowledge graph can be: obtain a specific face shape, obtain "round face", and input the data into the beauty knowledge graph to retrieve the relationship structure, that is, retrieve the triple relationship between face shape and eyebrow shape, and obtain the relationship structure of "round face-suitable-willow-eyebrow". It should be understood that the output analysis results of the three-part and five-eye can also have an impact on the recommended makeup. This part of the impact can be reflected through the operation of the three-part and five-eye, and then through the interaction with the beauty knowledge graph. The interaction process is similar to the above example and will not be repeated here.

[0121] For some makeup, there may be multiple possibilities for the data obtained through training. During the interaction of these data with the beauty knowledge graph, the initial average weight of content recommendation can be obtained. Taking eye shape as an example (entity relationship: eye shape-suitable-eye shadow), the obtained data results (beauty suggestion information) can be the corresponding relationship in Table 2.

[0122] Table 2 Example of beauty advice information

[0123] entity relation entity Weight Phoenix Eyes Suitable Forward Eyeshadow 25％ Phoenix Eyes Suitable Backward Eyes 25％ Phoenix Eyes Suitable Gradient eyeshadow 25％ Phoenix Eyes Suitable Layered eyeshadow 25％ … … … …

[0124] The weight values ​​in the beauty suggestion information output by the beauty recommendation model are usually initial weight values. It should be understood that the weight values ​​in Table 2 are only an example, and these initial weight values ​​can also be other weights, which is not limited in this application.

[0125] The embodiment of the present application obtains the target user's facial information by inputting it into the beauty recommendation model to obtain the target user's facial analysis results and beauty recommendation information. The beauty recommendation information includes the recommended makeup and the weight value corresponding to the recommended makeup. The facial analysis results and the recommended makeup conform to the relationship structure in the beauty knowledge graph. The beauty recommendation model is generated by training the neural network through user portraits and combining the beauty knowledge graph. The makeup recommended for the target user by the above method can conform to the user's own facial features, realizing personalized makeup recommendation.

[0126] In step 103, a preference score is performed on the recommended makeup according to the facial analysis result using a collaborative filtering recommendation algorithm to obtain a predicted preference score result of the target user for the recommended makeup.

[0127] Collaborative filtering recommendation algorithm is a method of making recommendations by analyzing the behaviors and preferences among users. Collaborative filtering recommendation algorithms can be user-based collaborative filtering (User-based) and item-based collaborative filtering (Item-based).

[0128] In some embodiments, multiple similar user portraits that are similar to the user portrait of the target user can be determined, and then the recommended makeup can be rated based on the multiple similar user portraits to obtain multiple preference rating results, and the average of the multiple preference rating results can be calculated to obtain the predicted preference rating result of the target user for the recommended makeup.

[0129] For example, based on the collected features of all users, the most suitable makeup or products for the same type of user portrait can be recommended to the user in combination with a collaborative filtering recommendation algorithm.

[0130] In some embodiments, the recommended makeup is scored based on multiple similar user portraits to obtain multiple preference score results. The multiple preference score results can be obtained by inputting data of the multiple similar user portraits into a predictive scoring formula.

[0131] Among them, the prediction score formula is expressed as:

[0132]

[0133] Where score(u,p) represents the predicted score, u represents the user, p represents the recommended makeup, N(u) represents the set of multiple users most similar to user u, sim(u,v) represents the similarity between user u and user v, R v,p Indicates the rating or behavior intensity of user v on p.

[0134] The predicted score can usually represent the potential interest or preference of user u for product p. The higher the score, the more likely user u is to be interested in product p. The higher the similarity between user u and user v, the closer the interests of the two users are. The similarity can be calculated using cosine similarity or Pearson correlation coefficient.

[0135] According to the above formula, the predicted scores of multiple users who are most similar to user u for all candidate products or makeup are calculated. These scores are sorted, and the products or makeup with the highest scores are selected as the recommendation results. And the average preference degree is calculated based on the preference degree of multiple similar users for makeup or products, and the predicted preference degree of the target user for makeup or product is obtained.

[0136] For example, there are two users similar to user u, user A and user B, and the predicted scores of user A and user B for eye shadow makeup can be shown in Table 3. The predicted scores (preferences) of user A and user B for eye shadow makeup are sorted, and the average of the preferences is calculated accordingly, so as to obtain the predicted preference (predicted score) of the target user for eye shadow makeup. The predicted preference of the target user can be shown in Table 4.

[0137] Table 3 Example of preference ratings for recommended makeup

[0138]

[0139] Table 4 Example of predicted preference of target users

[0140]

[0141] Through the above method, the predicted preference result of the target user is obtained, and the predicted preference can be used as an important basis for recommending makeup for the target user.

[0142] The embodiment of the present application uses a collaborative filtering recommendation algorithm to score the recommended makeup according to the facial analysis results, and obtains the target user's predicted preference score result for the recommended makeup, which provides an important basis for ultimately recommending beauty makeup to the target user.

[0143] In step 104, the weight value of the recommended makeup is adjusted according to the predicted preference score result and the relationship structure in the beauty knowledge graph to obtain the final recommended makeup for the target user.

[0144] For example, according to the predicted preference result of the target user obtained above, the weight value of the recommended makeup in the beauty knowledge graph can be adjusted by using the predicted preference result to obtain the final recommended makeup for the target user. It should be understood that the final recommended makeup should be the makeup corresponding to the highest weight value. The above correspondence can be shown in Table 5.

[0145] Table 5 Examples of candidate recommended makeup for target users

[0146] entity relation entity Weight Danfeng eyes (like) Suitable Backward Eyes 96％ Phoenix Eyes Suitable Gradient eyeshadow 91％ Phoenix Eyes Suitable Layered eyeshadow 86％ …… …… …… ……

[0147] As shown in Table 5, the makeup recommended for the target user should be back-shifted eyeshadow (back-shifted eyes). It should be understood that the above example only gives an example of eyeshadow makeup. According to the method provided in the embodiment of the present application, recommendations for lip makeup, contouring, eyebrow makeup, blush, etc. can also be given. The method provided in the embodiment of the present application can also provide recommendation information of makeup products suitable for the target user according to the needs of the user.

[0148] The embodiment provided in the present application adjusts the weight value of the recommended makeup according to the predicted preference score results combined with the relationship structure in the beauty knowledge graph, and recommends the makeup corresponding to the highest weight to the target user. This recommendation method is more in line with the user's expected beauty types and beauty products.

[0149] In some embodiments of the present application, the user's usage behavior, data and other content can be continuously collected during the user's use process to help iterate and optimize the above-mentioned beauty recommendation model and corresponding recommendation method, and update the recommendation content in real time to provide users with a better user experience.

[0150] By collecting behavioral data such as clicks, browsing, purchases, likes / dislikes, etc., these behaviors can instantly feedback users' preferences for recommended content, allowing models and recommendation methods to quickly adjust recommendation strategies and provide content that is more in line with users' interests, thereby significantly improving user experience.

[0151] Users' explicit feedback (such as ratings and likes) and implicit feedback (such as clicks and dwell time) are important data sources for optimizing recommendation algorithms. By collecting and analyzing these feedbacks and feeding them into the algorithm, the algorithm can continuously learn and understand the real interests of users, thereby optimizing the recommendation model and recommendation method, improving the accuracy and personalization of recommendations, and building a more refined user portrait to achieve more accurate personalized recommendations.

[0152] An example of an embodiment of the present application: User M wants to know the eye makeup that suits him / her. At this time, user M can input his / her own photo into the beauty makeup recommendation model. The beauty makeup recommendation model obtains the facial analysis result for user M, which is a round face and the corresponding three-part and five-eye information. The beauty makeup recommendations for user M include layered eye shadow and back-shifted eye shadow. The corresponding weights of layered eye shadow and back-shifted eye shadow are 65% and 35% respectively. Then, through the collaborative filtering method, the predicted preference scores of user M are 90% for layered eye shadow and 85% for back-shifted eye shadow respectively. Then, the weight values ​​of the recommended makeup are adjusted in combination with the relationship structure in the beauty makeup knowledge graph, and the eye makeup recommended for user M is layered eye shadow. If the user chooses layered eye shadow, the parameters in the beauty makeup recommendation model and the collaborative filtering method can be adjusted according to the user's choice, thereby optimizing the recommendation model and the recommendation method.

[0153] The above is an introduction to an embodiment of the beauty makeup recommendation method based on face recognition of the present application.

[0154] The implementation of the beauty makeup recommendation device based on face recognition provided by the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0155] For the beauty makeup recommendation method based on face recognition provided in the above embodiment, the present application embodiment also provides a beauty makeup recommendation device 200 based on face recognition, which is used to implement the above method. Figure 2 As shown, Figure 2 This is a schematic block diagram of modules of a beauty makeup recommendation device based on face recognition provided by some embodiments of the present application. The beauty makeup recommendation device based on face recognition includes:

[0156] An acquisition module 201 is used to acquire facial information of a target user;

[0157] An input module 202 is used to input the facial information of the target user into the beauty recommendation model to obtain the facial analysis result and beauty recommendation information of the target user, wherein the beauty recommendation information includes a recommended makeup and a weight value corresponding to the recommended makeup. The facial analysis result and the recommended makeup conform to the relationship structure in the beauty knowledge graph. The beauty recommendation model is generated by training a neural network through user portraits and combining the beauty knowledge graph;

[0158] Scoring module 203, used to score the recommended makeup according to the facial analysis result by collaborative filtering recommendation algorithm, and obtain the predicted score result of the target user's preference for the recommended makeup;

[0159] The recommendation module 204 is used to adjust the weight value of the recommended makeup according to the predicted preference score result combined with the relationship structure in the beauty knowledge graph to obtain the final recommended makeup for the target user.

[0160] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0161] like Figure 3 As shown, the embodiment of the present application further provides an electronic device 300, the electronic device 300 includes a memory 301, one or more processors 302 ( Figure 3 Only one is shown) and a computer program stored in the memory and executable on the processor. Among them: the memory 301 is used to store software programs and units, and the processor 302 executes various functional applications and data processing by running the software programs and units stored in the memory to obtain the resources corresponding to the above-mentioned preset events. Optionally, the processor 302 implements the above-mentioned beauty recommendation method based on face recognition by running the above-mentioned computer program stored in the memory 301.

[0162] The memory 301, as a non-transitory computer-readable medium, can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory 301 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 301 may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor 302 via a network.

[0163] It can be understood that the contents of the above method embodiments are all applicable to the electronic device embodiment, the functions specifically implemented by the electronic device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0164] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned beauty recommendation method based on face recognition.

[0165] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0166] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is executed by one or more processors, it can implement the steps of the above-mentioned beauty recommendation method based on face recognition.

[0167] It can be understood that the contents of the above method embodiments are all applicable to the present computer program product, the functions specifically implemented by the present computer program product embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0168] The beauty makeup recommendation method, device, electronic device, medium and computer program product based on face recognition provided by the embodiment of the present application obtains the facial information of the target user; inputs the facial information of the target user into the beauty makeup recommendation model to obtain the facial analysis result and beauty makeup suggestion information of the target user, the beauty makeup suggestion information includes the recommended makeup and the weight value corresponding to the recommended makeup, the facial analysis result and the recommended makeup conform to the relational structure in the beauty makeup knowledge graph, and the beauty makeup recommendation model is generated by training the neural network through user portraits and combining the beauty makeup knowledge graph; then, the recommended makeup is scored according to the facial analysis result through the collaborative filtering recommendation algorithm, and the predicted preference score result of the target user for the recommended makeup is obtained; finally, the weight value of the recommended makeup is adjusted according to the predicted preference score result combined with the relational structure in the beauty makeup knowledge graph, and the final recommended makeup for the target user is obtained. The embodiment of the present application can deeply explore the complex relationship of beauty makeup knowledge by constructing a knowledge graph of beauty makeup professional knowledge. By utilizing these relationships, we can more accurately understand users’ personalized needs and recommend beauty products that better meet their expectations. By dynamically adjusting the weights in the knowledge graph based on algorithmic rules through interaction and feedback mechanisms with users, we can capture changes in user interests in real time and adjust recommendation strategies accordingly to ensure the timeliness and personalization of recommended content.

[0169] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0170] Although specific embodiments are described herein, those of ordinary skill in the art will recognize that many other modifications or alternative embodiments are also within the scope of the present disclosure. For example, any of the functions and / or processing capabilities described in conjunction with a particular device or component may be performed by any other device or component. In addition, although various exemplary implementations and architectures have been described according to embodiments of the present disclosure, those of ordinary skill in the art will recognize that many other modifications to the exemplary implementations and architectures described herein are also within the scope of the present disclosure.

[0171] Some aspects of the present disclosure are described above with reference to the block diagram and flow chart of the system, method, system and / or computer program product according to the exemplary embodiments.It should be understood that the combination of one or more blocks in the block diagram and the flow chart and the blocks in the block diagram and the flow chart can be realized by executing computer executable program instructions respectively.Equally, according to some embodiments, some blocks in the block diagram and the flow chart may not need to be executed in the order shown, or may not need to be executed in full.In addition, additional components and / or operations beyond those components and / or operations shown in the blocks in the block diagram and the flow chart may be present in certain embodiments.

[0172] Therefore, the blocks in the block diagrams and flow charts support the combination of devices for performing the specified functions, the combination of elements or steps for performing the specified functions, and the program instruction devices for performing the specified functions. It should also be understood that each block in the block diagrams and flow charts and the combination of blocks in the block diagrams and flow charts can be implemented by a special-purpose hardware computer system or a combination of special-purpose hardware and computer instructions that performs a specific function, element or step.

[0173] The program modules, applications, etc. described herein may include one or more software components, including, for example, software objects, methods, data structures, etc. Each such software component may include computer-executable instructions that, in response to execution, cause at least a portion of the functionality described herein (e.g., one or more operations of the illustrative methods described herein) to be performed.

[0174] Software components can be encoded with any of various programming languages. An exemplary programming language can be a low-level programming language, such as an assembly language associated with a specific hardware architecture and / or an operating system platform. The software component comprising assembly language instructions may need to be converted to executable machine code by an assembler before being executed by hardware architecture and / or platform. Another exemplary programming language can be a higher-level programming language, which can be transplanted across multiple architectures. The software component comprising a higher-level programming language may need to be converted to an intermediate representation by an interpreter or a compiler before execution. Other examples of programming languages ​​include but are not limited to macro languages, shells or command languages, job control languages, scripting languages, database queries or search languages ​​or report writing languages. In one or more exemplary embodiments, the software component comprising the instruction of one of the above-mentioned programming language examples can be directly executed by an operating system or other software components, without first being converted into another form.

[0175] Software components may be stored as files or other data storage structures. Software components of similar type or related functions may be stored together, such as in a specific directory, folder, or library. Software components may be static (e.g., preset or fixed) or dynamic (e.g., created or modified at execution time).

[0176] The embodiments of the present application are described in detail above in conjunction with the accompanying drawings, but the present application is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in the relevant technical field without departing from the purpose of the present application.

Claims

1. A beauty makeup recommendation method based on face recognition, characterized in that: The following steps are involved: Obtain the target user’s facial information; Inputting the target user's facial information into a beauty recommendation model to obtain a facial analysis result and beauty recommendation information of the target user, wherein the beauty recommendation information includes a recommended makeup and a weight value corresponding to the recommended makeup, wherein the facial analysis result and the recommended makeup conform to the relational structure in the beauty knowledge graph, and the beauty recommendation model is generated by training a neural network through user portraits and combining the beauty knowledge graph; Using a collaborative filtering recommendation algorithm to rate the recommended makeup according to the facial analysis result, and obtain a predicted rating result of the target user's preference for the recommended makeup; The weight value of the recommended makeup is adjusted according to the predicted preference score result and the relationship structure in the beauty knowledge graph to obtain a final recommended makeup for the target user.

2. The method for beauty makeup recommendation based on face recognition according to claim 1, characterized in that: Before acquiring the facial information of the target user, the method further includes: Get physical elements in the beauty field; According to the association relationship between the entity elements, a relationship structure in the beauty knowledge graph is established; An initial weight value is set for the relationship structure in the beauty knowledge graph to obtain the beauty knowledge graph.

3. The method for beauty makeup recommendation based on face recognition according to claim 1, characterized in that: The neural network is trained through user portraits and combined with the beauty knowledge graph to generate the beauty recommendation model, including: Acquire user data, wherein the user data includes user facial feature data and user behavior data, wherein the user behavior data is associated with beauty makeup; Establishing a user portrait based on the user facial feature data and the user behavior data; Using the label content in the user portrait as a data set to train the neural network; The beauty recommendation model is generated based on the output result of the neural network and the beauty knowledge graph.

4. The method for beauty makeup recommendation based on face recognition according to claim 3, characterized in that: The using the label content in the user portrait as a data set to train the neural network includes: Analyze the tag content in the user portrait to obtain data features of the tag content; Performing characterization processing on the tag content according to the data feature to obtain characteristic data corresponding to the tag content; The neural network is trained by using the feature data as a data set of the neural network.

5. The method for beauty makeup recommendation based on face recognition according to claim 4, characterized in that: The characterization processing of the tag content according to the data feature includes: If the data feature is a classification feature, the label content is labeled according to the corresponding classification standard; If the data feature is a continuous feature, the label content is normalized; If the data feature is a multi-category feature, the label content is represented by a vector through an embedding layer.

6. The method for beauty makeup recommendation based on face recognition according to claim 1, characterized in that: The step of performing a preference rating on the recommended makeup according to the facial analysis result by using a collaborative filtering recommendation algorithm to obtain a predicted preference rating result of the target user for the recommended makeup includes: Determine a plurality of similar user profiles that are similar to the user profile of the target user; Based on the multiple similar user portraits, the recommended makeup is scored for preference, to obtain multiple preference scoring results; An average value of the plurality of preference rating results is calculated to obtain a predicted preference rating result of the target user for the recommended makeup.

7. The method for beauty makeup recommendation based on face recognition according to claim 6, characterized in that: The step of performing a preference rating on the recommended makeup based on the multiple similar user portraits to obtain multiple preference rating results includes: Inputting the data of the plurality of similar user portraits into a prediction scoring formula to obtain a plurality of preference scoring results; The prediction score formula is expressed as: Where score(u,p) represents the predicted score, u represents the user, p represents the recommended makeup, N(u) represents the set of multiple users most similar to user u, sim(u,v) represents the similarity between user u and user v, R v,p Indicates the rating or behavior intensity of user v on p.

8. A beauty makeup recommendation device based on face recognition, characterized in that: The device comprises: An acquisition module is used to obtain facial information of the target user; An input module, used to input the facial information of the target user into a beauty recommendation model, to obtain a facial analysis result and beauty recommendation information of the target user, wherein the beauty recommendation information includes a recommended makeup and a weight value corresponding to the recommended makeup, wherein the facial analysis result and the recommended makeup conform to the relational structure in the beauty knowledge graph, and the beauty recommendation model is generated by training a neural network through user portraits and combining the beauty knowledge graph; A scoring module, used to score the recommended makeup according to the facial analysis result by using a collaborative filtering recommendation algorithm, and obtain a predicted score result of the target user's preference for the recommended makeup; The recommendation module is used to adjust the weight value of the recommended makeup according to the predicted preference score result combined with the relationship structure in the beauty knowledge graph to obtain the final recommended makeup for the target user.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the beauty makeup recommendation method based on face recognition as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the beauty makeup recommendation method based on face recognition as described in any one of claims 1 to 7 is implemented.

Images