Facial feature-based beauty makeup recommendation method and device, equipment and medium

Abstract

The invention discloses a beauty makeup recommendation method and device based on facial features, equipment and a medium, and relates to the technical field of artificial intelligence, and the beauty makeup recommendation method based on the facial features comprises the steps: obtaining a face image of a target user; inputting the face image into the face feature extraction model to obtain the face feature information of the target user, recommending the target makeup to the target user through a comprehensive recommendation mechanism according to the face feature information, the comprehensive recommendation mechanism including an algorithm recommendation mechanism, an expert recommendation mechanism and a popular makeup recommendation mechanism, and finally, recommending the target makeup to the target user. According to feedback information of the target user on the target makeup, weights corresponding to the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism are adjusted respectively, updated weights of the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism are obtained, and the comprehensive recommendation mechanism recommends the target makeup for the target user through the updated weights. Through the method, the precision of beauty makeup recommendation can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and medium for beauty recommendation based on facial features. Background Art

[0002] Appropriate makeup can enhance personal confidence, shape personal image, and is a form of cultural expression and identity.

[0003] With traditional makeup, users apply makeup based on makeup instructional videos, following the makeup techniques and makeup looks in the videos. However, instructional videos cannot provide targeted guidance and instruction based on the user's own characteristics.

[0004] There are currently a variety of beauty advice products and services on the market, but they still have many shortcomings in terms of accuracy. Summary of the invention

[0005] The main purpose of the embodiments of the present application is to provide a method, device, equipment and medium for beauty recommendation based on facial features, aiming to improve the accuracy of beauty recommendation.

[0006] To achieve the above purpose, one aspect of an embodiment of the present application proposes a beauty makeup recommendation method based on facial features, comprising the following steps:

[0007] Obtain the target user's face image;

[0008] Inputting the face image into a face feature extraction model to obtain facial feature information of the target user, wherein the face feature extraction model is obtained by training a neural network with a labeled face image data set, and the facial feature information is used to characterize various data points of the face of the target user;

[0009] recommending a target makeup look for the target user through a comprehensive recommendation mechanism according to the facial feature information, wherein the comprehensive recommendation mechanism includes an algorithm recommendation mechanism, an expert recommendation mechanism, and a popular makeup look recommendation mechanism;

[0010] According to the target user's feedback information on the target makeup, the weights corresponding to the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism are adjusted respectively to obtain updated weights of the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism. The comprehensive recommendation mechanism recommends the target makeup to the target user through the updated weights.

[0011] In some embodiments, after acquiring the face image of the target user, the following steps are included:

[0012] Determining facial key points of the target user through the facial image;

[0013] Determining a rotation angle of the face image according to the position information of the facial key points;

[0014] Adjusting the facial image according to the rotation angle to obtain an updated facial image;

[0015] The facial key points of the target user are re-determined based on the updated facial image to obtain the facial key points of the frontal lens.

[0016] In some embodiments, the training of the neural network using a labeled face image dataset includes:

[0017] Initializing weights of the neural network based on the variance of the weights in the weight matrix;

[0018] The labeled face image data set is input into the neural network, and the neural network is trained through a forward propagation process, a loss calculation process and a back propagation process to obtain the face feature extraction model.

[0019] In some embodiments, inputting the facial image into a facial feature extraction model to obtain facial feature information of the target user includes:

[0020] Performing cropping, scaling and normalization processing on the face image to obtain a preprocessed face image;

[0021] Inputting the preprocessed face image into the face feature extraction model, and extracting a feature vector corresponding to the face image before the last convolutional layer or fully connected layer in the face feature extraction model;

[0022] The facial feature information corresponding to the face image is determined by the feature vector.

[0023] In some embodiments, the facial feature information includes facial features and face shape features, and determining the facial feature information corresponding to the face image through the feature vector includes:

[0024] Cut out a region containing specific facial features from the feature vector to obtain a target region;

[0025] Performing average pooling processing on the target area to obtain a feature description corresponding to the specific facial features, wherein the feature description is used to characterize the facial features;

[0026] Determining facial contour data corresponding to the facial image through the feature vector;

[0027] The facial features of the facial image are determined according to the facial contour data.

[0028] In some embodiments, adjusting the weights corresponding to the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism respectively according to the target user's feedback information on the target makeup includes:

[0029] Extracting scoring information through the feedback information;

[0030] Based on the rating information, calculating an average rating corresponding to the recommendation mechanism to obtain a recommendation mechanism rating;

[0031] Calculating the total average score of the algorithm recommendation mechanism, the expert recommendation mechanism, and the popular makeup recommendation mechanism to obtain a total average score;

[0032] Determine the impact factor corresponding to the recommendation mechanism by the difference between the recommendation mechanism score and the total average score;

[0033] Update the recommendation weight of the corresponding recommendation mechanism based on the impact factor;

[0034] The recommendation weights are normalized to obtain normalized weight information.

[0035] In some embodiments, after recommending a target makeup look to the target user through a comprehensive recommendation mechanism according to the facial feature information, the method includes:

[0036] Setting rendering conditions, the rendering conditions including configuring a texture unit, the texture unit being used to store an image of a cosmetic product;

[0037] According to the target makeup, mapping the image of the corresponding makeup product to the corresponding part of the face image;

[0038] Introduce the lighting model into the shader to adjust the direction and intensity of the lighting to obtain the corresponding visual effects.

[0039] To achieve the above-mentioned purpose, another aspect of the embodiment of the present application provides a beauty makeup recommendation device based on facial features, the beauty makeup recommendation device based on facial features comprising:

[0040] An acquisition module is used to acquire a face image of a target user;

[0041] An input module, used to input the face image into a face feature extraction model to obtain facial feature information of the target user, wherein the face feature extraction model is obtained by training a neural network with a labeled face image data set, and the facial feature information is used to characterize various data points of the face of the target user;

[0042] A recommendation module, configured to recommend a target makeup look to the target user through a comprehensive recommendation mechanism according to the facial feature information, wherein the comprehensive recommendation mechanism includes an algorithm recommendation mechanism, an expert recommendation mechanism, and a popular makeup look recommendation mechanism;

[0043] An adjustment module is used to adjust the weights corresponding to the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism according to the feedback information of the target user on the target makeup, so as to obtain updated weights of the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism. The comprehensive recommendation mechanism recommends the target makeup to the target user through the updated weights.

[0044] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned facial feature-based beauty recommendation method when executing the computer program.

[0045] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by one or more processors, the steps of the above-mentioned facial feature-based beauty recommendation method can be implemented.

[0046] The embodiments of the present application include at least the following beneficial effects:

[0047] The present application provides a method, apparatus, device and medium for beauty makeup recommendation based on facial features. In an embodiment of the present application, first, a facial image of a target user is obtained; then, the facial image is input into a facial feature extraction model to obtain facial feature information of the target user, the facial feature extraction model is obtained by training a neural network with a labeled facial image data set, and the facial feature information is used to characterize various data points on the face of the target user; then, a target makeup is recommended to the target user through a comprehensive recommendation mechanism based on the facial feature information, the comprehensive recommendation mechanism includes an algorithm recommendation mechanism, an expert recommendation mechanism and a popular makeup recommendation mechanism, and finally, the weights corresponding to the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism are adjusted respectively according to the feedback information of the target user on the target makeup, to obtain updated weights of the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism, and the comprehensive recommendation mechanism recommends the target makeup to the target user by updating the weights. The embodiment of the present application obtains the facial feature information of the target user by inputting the facial image into the facial feature extraction model. The facial features of the target user can reflect the facial characteristics of the target user, so that the recommended makeup is more targeted. Afterwards, the makeup is recommended to the target user based on the facial features of the target user through a comprehensive recommendation mechanism. Since the comprehensive recommendation mechanism combines multiple recommendation methods and can continuously update the weights based on user feedback, the accuracy of beauty recommendations can be improved through this method.

[0048] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0050] Figure 1 is a flowchart of a beauty makeup recommendation method based on facial features provided by some embodiments of the present application;

[0051] Figure 2 is another flow chart of a beauty makeup recommendation method based on facial features provided in some embodiments of the present application;

[0052] Figure 3 is a schematic block diagram of modules of a beauty makeup recommendation device based on facial features provided by some embodiments of the present application;

[0053] Figure 4 It is a schematic diagram of the hardware structure of an electronic device provided in some embodiments of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the reference to "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. The appearance of this phrase in various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0055] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0056] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0058] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0059] In order to make the inventive concept of the present application easy to understand, before describing the embodiments of the present application in detail, the English abbreviations (terms) / related concepts involved in the embodiments of the present application are first described. The English abbreviations (terms) / related concepts involved in the embodiments of the present application are subject to the following explanations.

[0060] FaceMesh: FaceMesh detects and tracks the geometry of the face in real time. It identifies 468 unique facial landmarks located at key locations such as eyes, nose, mouth, cheeks, etc. It runs in real time on a variety of devices, including smartphones and web browsers, supports multiple operating systems and hardware configurations, and is designed with user privacy in mind, without sending data to external servers for processing.

[0061] MediaPipe: MediaPipe is an open source cross-platform multimedia processing framework that is mainly used to build machine learning-based applications, especially in the fields of computer vision, audio processing, and pose estimation. MediaPipe provides a library of pre-trained machine learning models and tools that can help developers quickly build and deploy computer vision and audio processing applications.

[0062] OpenGL: OpenGL (Open Graphics Library) is a cross-language, cross-platform programming interface used primarily for rendering 2D and 3D vector graphics. It consists of nearly 350 different function calls that can be used to draw anything from simple graphics to complex three-dimensional scenes. OpenGL is widely used in CAD, virtual reality, scientific visualization programs, and video game development.

[0063] EfficientNet: EfficientNet is a convolutional neural network (CNN) architecture and scaling method that uniformly scales the depth, width, and resolution of the network by a compound factor. This approach differs from traditional practices, which typically scale these factors arbitrarily. The design of EfficientNet allows for more efficient use of additional computational resources by applying a fixed scaling factor to the width, depth, and resolution of the network and the input image.

[0064] Appropriate makeup can enhance personal confidence, shape personal image, and is a form of cultural expression and identity.

[0065] With traditional makeup, users apply makeup based on makeup instructional videos, following the makeup techniques and makeup looks in the videos. However, instructional videos cannot provide targeted guidance and instruction based on the user's own characteristics.

[0066] There are currently a variety of beauty advice products and services on the market, but most still have some shortcomings. First, many applications rely on users to manually select makeup styles or products, lacking personalized and intelligent suggestions. Second, although some image recognition-based beauty advice systems can provide a certain degree of personalized suggestions, they often have limitations in accuracy and user experience. In addition, the virtual makeup trial effect is not realistic and interactive enough, making it difficult for users to intuitively feel the effect of makeup, which affects the overall experience.

[0067] The applicant found that in terms of technical implementation, existing beauty recommendation algorithms often rely on a single image processing technology and lack the ability to integrate multiple algorithms and data sources. This leads to poor adaptability among users with different lighting conditions, different skin colors, and different face shapes. In addition, the existing technology is easily affected by noise when processing images, which affects the final recognition and recommendation quality.

[0068] To sum up, the beauty advice products and services currently on the market still have much room for improvement in terms of personalization, accuracy and user experience.

[0069] In view of this, the present application proposes a method, device, equipment and medium for beauty recommendation based on facial features. The scheme obtains a facial image of a target user; then, the facial image is input into a facial feature extraction model to obtain facial feature information of the target user, the facial feature extraction model is obtained by training a neural network through a labeled facial image data set, and the facial feature information is used to characterize various data points on the face of the target user; then, a target makeup is recommended to the target user through a comprehensive recommendation mechanism based on the facial feature information, and the comprehensive recommendation mechanism includes an algorithm recommendation mechanism, an expert recommendation mechanism and a popular makeup recommendation mechanism. Finally, according to the target user's feedback information on the target makeup, the weights corresponding to the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism are adjusted respectively to obtain updated weights of the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism, and the comprehensive recommendation mechanism recommends the target makeup to the target user by updating the weights. The embodiment of the present application obtains the facial feature information of the target user by inputting the facial image into the facial feature extraction model. The facial features of the target user can reflect the facial characteristics of the target user, so that the recommended makeup is more targeted. Afterwards, the makeup is recommended to the target user based on the facial features of the target user through a comprehensive recommendation mechanism. Since the comprehensive recommendation mechanism combines multiple recommendation methods and can continuously update the weights based on user feedback, the accuracy of beauty recommendations can be improved through this method.

[0070] The method provided in the embodiment of the present application can be applied to the electronic device provided in the embodiment of the present application, wherein the electronic device can be a terminal or a server.

[0071] The terminal may be a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto.

[0072] The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, as well as big data and artificial intelligence platforms.

[0073] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0074] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0075] The following will describe in detail the implementation steps of a beauty recommendation method based on facial features provided by an embodiment of the present application in conjunction with the accompanying drawings.

[0076] Please refer to Figure 1 , Figure 1A flowchart of a beauty makeup recommendation method based on facial features provided for some embodiments of the present application. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than that shown here.

[0077] The method of the embodiment of the present application comprises the following steps:

[0078] Step 101: Acquire a face image of a target user;

[0079] Step 102: input the face image into a face feature extraction model to obtain facial feature information of the target user. The face feature extraction model is obtained by training a neural network with a labeled face image dataset. The facial feature information is used to characterize various data points on the face of the target user.

[0080] Step 103: recommending a target makeup look for a target user through a comprehensive recommendation mechanism based on the facial feature information, wherein the comprehensive recommendation mechanism includes an algorithm recommendation mechanism, an expert recommendation mechanism, and a popular makeup look recommendation mechanism;

[0081] Step 104: According to the target user's feedback information on the target makeup, the weights corresponding to the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism are adjusted respectively to obtain updated weights of the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism. The comprehensive recommendation mechanism recommends the target makeup to the target user by updating the weights.

[0082] In steps 101 to 104 shown in the embodiment of the present application, facial feature information of the target user is obtained by inputting a facial image into a facial feature extraction model. The facial features of the target user can reflect the facial features of the target user, making the recommended makeup more targeted. Afterwards, makeup is recommended to the target user based on the facial features of the target user through a comprehensive recommendation mechanism. Since the comprehensive recommendation mechanism combines multiple recommendation methods and can continuously update weights based on user feedback, this method can improve the accuracy of beauty recommendations, continue to meet users' psychological expectations, and improve users' usage experience.

[0083] The specific implementation methods of the above steps are introduced below.

[0084] In step 101, a face image of a target user is obtained.

[0085] The target user is the user targeted by the beauty recommendation. For makeup teaching, the target user is the model used to teach makeup; for personal makeup, the target user is the person who needs makeup and styling.

[0086] The face image of the target user can be acquired through optical devices such as cameras, video cameras, etc. It should be understood that the acquired face image should ensure that the image quality is sufficient to support subsequent image processing and analysis work.

[0087] In actual use, facial images can be obtained through a front-end application applet, which has an intuitive and friendly interface that can guide users to correctly take or upload facial photos.

[0088] For example, the user's camera permission can be obtained first. After turning on the camera, the user can see the real-time picture and whether the light and position of the current camera picture meet the requirements (such as sufficient light, face centered, etc.), so that the user can adjust the posture and position. After the detected image meets the shooting requirements, the system automatically takes pictures. The shooting results are displayed for the user to confirm whether to reshoot, so as to obtain the user's face image.

[0089] The embodiment provided in the present application obtains the facial image of the target user, thereby providing a basis for subsequently recommending accurate and personalized makeup based on the target user's facial features, face shape and skin characteristics.

[0090] After obtaining the user's facial image, the user's facial key points can also be determined through the obtained facial image, and the rotation angle of the facial image can be determined based on the position information of the facial key points. The facial image is then adjusted according to the rotation angle to obtain an updated facial image, and the facial key points of the target user are re-determined based on the updated facial image to obtain the facial key points of the forward lens.

[0091] For example, after acquiring the user's facial image, necessary preprocessing may be performed on the facial image, such as denoising the image, marking key points, adjusting the facing direction of the facial image, and the like.

[0092] Specifically, some tools can be used to assist in completing the above steps. For example, you can use FaceMesh in the MediaPipe framework to accurately locate the key points on the face. FaceMesh can track and depict the face contour and important facial features such as eyes, nose, mouth, etc. in real time. It detects these key points by analyzing the pixels in the image and returns a series of coordinates that correspond to specific locations on the face.

[0093] After that, the center coordinates of the face can be calculated through the coordinate position returned by FaceMesh, and the direction of the face can be adjusted according to the center coordinates.

[0094] For example, the inner eye corner coordinates (x1, y1) (x2, y2) can be obtained through the coordinate position returned by FaceMesh, and then the center point coordinates (Xcenter, Ycenter) can be calculated based on the inner eye corner coordinates: center =(x1+x2) / 2, Y center =(y2+y1) / 2. Based on the inner eye corner coordinates and the center point coordinates, the rotation angle of the face image can be calculated through the function to adjust the face image. For example, the rotated_image function can be called, which will calculate the necessary rotation angle based on the position information of these points, thereby correcting the face in the original image to face forward.

[0095] After correcting the face image, you can use mediapipe again to locate the face key points and obtain the face point information of the front lens. Finally, store the corrected face key point information in the data set.

[0096] In order to ensure the privacy and security of user data, the SSL / TLS protocol can be used for encrypted communication when transmitting facial information and its corrected point information to the cloud for calculation. The client will establish a secure TLS connection with the server, confirm the identity of the server through certificate verification, and generate a symmetric key through the key exchange process. This key will be used to encrypt and decrypt the transmitted data. Facial information and related point data will be encrypted before sending, and only the cloud server with the corresponding decryption key can decrypt and process this data.

[0097] The embodiment of the present application can pre-process the facial image so that the acquired image data can better support subsequent image analysis and beauty recommendation methods.

[0098] In step 102, the face image is input into a face feature extraction model to obtain facial feature information of the target user. The face feature extraction model is obtained by training a neural network with a labeled face image data set. The facial feature information is used to characterize various data points of the target user's face.

[0099] It is understandable that the face image obtained in step 1 can be directly input into the feature extraction model, or the face key points located by mediapipe can be input into the face feature extraction model to obtain the facial feature information of the target user. The feature extraction model can adaptively adjust its processing method according to different types of input data.

[0100] Facial feature information is used to characterize various data points of the target user's face, which can be used to describe, identify or analyze the attributes and data of individual facial features. These data points can be used to describe the geometric features of the face: such as facial contour, eye position and size, nose shape and size, lip position and shape, cheekbone position and chin shape; they can also be used to describe the coordinates of feature points, such as the coordinate positions of key points such as the corners of the eyes, the center of the eyes, the tip of the nose, and the corners of the mouth; they can also be used to describe the texture features of the face, such as unique texture features such as skin texture, beard, moles, scars, etc.; they can also be used to describe color features, such as skin color, hair color, pupil color, etc.

[0101] The facial feature extraction model can use deep learning methods to extract key facial feature information, such as facial features, face shape, and skin color, from photos uploaded by users or input facial key point information. This feature information will be used to generate subsequent beauty recommendations.

[0102] The neural network can be a feedforward neural network, a convolutional neural network, a recurrent neural network, a long short-term memory network, an adversarial network, an attention mechanism or a Transformer network, and this application does not limit this.

[0103] Before training the neural network, you first need to prepare a data set. The data set can be generated by collecting a large amount of labeled face image data. The labels include the location of the face, the location of the key points of the facial features, etc. The data set should cover different ages, genders, races, and lighting conditions as much as possible to ensure the generalization ability of the neural network model.

[0104] After that, the data is preprocessed, including cropping and scaling of the image. For example, the face area obtained by mediapipe can be used to crop the part containing the face from the original image. This reduces the influence of irrelevant background and improves the efficiency of subsequent processing. After that, the cropped face image is scaled to a fixed size. Scaling can use methods such as bilinear interpolation.

[0105] The preprocessing operation also includes normalizing the image, that is, scaling the pixel values ​​of the image to between 0 and 1. Normalization can be done through a linear transformation formula, which can be expressed as:

[0106]

[0107] Among them, Original Pixel is the original pixel value, that is, the original grayscale value or color value of a pixel in the image. Min Value is the minimum pixel value in the data set, Max Value is the maximum pixel value in the data set, and NormalizedPixel is the normalized pixel value. This value is usually between 0 and 1.

[0108] Through the normalization formula, the pixel values ​​in the image can be scaled from the original range (usually 0 to 255) to between 0 and 1. The benefits of this are: 1. Data standardization, because different images may have different brightness or contrast, after normalizing the pixel values, the pixel values ​​between different images can be made comparable. 2. Numerical stability, the normalized data can better adapt to the training process of the neural network. 3. Accelerated convergence: Normalized data can accelerate the convergence of neural networks because they are more likely to fall within the working range of activation functions (such as ReLU).

[0109] After preparing the dataset, we can start to determine the neural network structure. For example, we can use EfficientNet as the neural network framework in the face feature extraction model. EfficientNet is an efficient convolutional neural network architecture that is widely used in image recognition tasks. This architecture can be used to perform deep learning on face information. The following is the process of using EfficientNet to calculate face feature information.

[0110] 1. Determine the depth, width, and resolution of the neural network. Compound scaling is an important innovation in the EfficientNet architecture, allowing the model to be scaled in three dimensions: width, depth, and resolution to adapt to different hardware and performance requirements. The specific formula is:

[0111] Scaled_width = α φ ×Base_width

[0112] Scaled_depth=β φ ×Base_depth

[0113] Scaled_resoulution = γ φ ×Base_resolution;

[0114] Where: α: width scaling factor. β: depth scaling factor. γ: resolution scaling factor. φ: scaling factor, usually ranging from 0 to 7, corresponding to EfficientNet-B0 to EfficientNet-B7. Base_width: base width. Base_depth: base depth. Base_resolution: base resolution.

[0115] 2. Initialize the network parameters. Perform He initialization on all parameters of the network to solve the problem of gradient disappearance or gradient explosion. Gradient disappearance refers to the fact that as the depth of the network increases, the gradient gradually decreases during the back propagation process, resulting in almost no update of the early layers of the network, which hinders the learning of the network. In contrast, gradient explosion refers to the fact that the gradient becomes too large, which may cause the weight update to be too large, making the training unstable. He initialization helps alleviate these problems by controlling the variance of the weights. Specifically, first, determine the input size of the layer (fan_in), that is, how many input units the layer receives signals from. Then calculate the variance var by the formula.

[0116] He initializes the variance of the weights to be:

[0117]

[0118] Then, a random number is drawn from a Gaussian distribution with a mean of 0 and a variance of var to initialize the weight matrix, ensuring that the nonlinear activation function can be activated at the beginning of training.

[0119] 3. Forward propagation. Forward propagation is the first step in the neural network training process. It involves feeding the input data into the network and passing it through a series of computational layers to finally get the predicted output. We pass the preprocessed image as input to EfficientNet. The features of the image are extracted through a series of operations such as convolution, pooling, and activation functions. Specifically, the multiple convolutional layers of EfficientNet will gradually extract the local features of the image, and finally form an output containing multiple feature maps. According to the face recognition task, add a fully connected layer with the number of nodes equal to the number of categories. Finally, add a Softmax layer after the output layer to get the probability distribution of each category.

[0120] 4. Calculate the loss. Calculating the loss is the process of evaluating the difference between the model's prediction results and the actual labels. In the face recognition task, the cross-entropy loss function can be used to measure the model's prediction error. The cross-entropy loss function can effectively quantify the difference between the predicted probability distribution and the true label. The cross-entropy loss function can be expressed as:

[0121]

[0122] Among them, y is the actual feature value of the face; is the predicted feature value; n is the feature type, and i is the variable.

[0123] By comparing the predicted probability of each category with the actual label, the cross entropy loss function can give a value indicating the prediction error of the model. The smaller this value is, the closer the model's prediction is to the true label.

[0124] 5. Back propagation. Back propagation is one of the core parts of neural network training. It is responsible for calculating the gradient of the loss function relative to each parameter in the network. Through the chain rule, the back propagation algorithm can start from the output layer and calculate the gradient layer by layer until the input layer. The obtained gradient information will be used to update the weight parameters of the network in order to reduce the value of the loss function. The gradient descent formula can be expressed as:

[0125]

[0126] Among them, w represents the weight; α represents the learning rate, which is a hyperparameter that determines the step size of the weight update; Represents the gradient of the loss function with respect to the weights.

[0127] Using the back-propagation algorithm, based on the calculated loss value, the gradient of each weight is calculated layer by layer starting from the output layer, and then the weights in the network are updated according to a certain learning rate using gradient descent.

[0128] By training the neural network, a facial feature extraction model can be obtained.

[0129] The face image obtained in step 101 is input into the face feature extraction model, in which the face image can be cropped, scaled and normalized to obtain a preprocessed face image. After that, the preprocessed face image is input into the trained neural network, and the feature vector corresponding to the face image is extracted before the last convolutional layer or the fully connected layer of the network; the facial feature information corresponding to the face image is determined through the feature vector.

[0130] The facial image obtained through step 101 can also be cropped, scaled and normalized to obtain a preprocessed facial image; the preprocessed facial image is input into the facial feature extraction model, and a feature vector corresponding to the facial image is extracted before the last convolutional layer or fully connected layer in the facial feature extraction model; and facial feature information corresponding to the facial image is determined through the feature vector.

[0131] Exemplarily, the preprocessed image is input into the EfficientNet network. The feature vector is obtained from the output before the last convolutional layer or the fully connected layer. This feature vector is usually a multi-dimensional vector that represents the high-level feature representation of the input image.

[0132] In some embodiments, facial feature information includes facial features and face shape features. The facial feature information corresponding to the face image is determined by the feature vector. The region containing specific facial features can be cropped out from the feature vector to obtain the target region, and average pooling is performed on the target region to obtain a feature description corresponding to the specific facial features. The feature description is used to characterize the facial features. The facial contour data corresponding to the face image is determined by the feature vector, and the face shape features of the face image are determined based on the facial contour data.

[0133] For example, a region containing specific facial features can be cropped from the feature map (feature extraction), and then global average pooling (GAP) is applied to the region to obtain a vector of fixed length. This vector represents the characteristic description of the facial features and can be used to distinguish the facial features of different individuals.

[0134] The global average pooling formula can be expressed as:

[0135]

[0136] Among them, F represents the feature map, H and W are the height and width of the feature map. By analyzing the feature vector, we can get the information and type of each part of the facial features.

[0137] The geometric properties of the facial contour, such as the length, width, and distance between specific points of the face, can be determined by analyzing the feature map. For example, by the formula:

[0138] D ij =|P i -P j |;

[0139] Determine the face shape information. Among them, P i and P j Respectively represent the positions of two key points on the facial contour, D ij represents the Euclidean distance between these two key points. Based on these geometric properties, the face shapes are classified into different categories, such as round, square, heart, etc.

[0140] Through the above steps, the key feature information of the face can be extracted from the image. This feature information provides an important basis for the subsequent generation of beauty suggestions. Through this feature information, the system can provide users with more accurate and personalized makeup suggestions, improving the user's makeup experience and satisfaction.

[0141] In step 103, a target makeup may be recommended to a target user through a comprehensive recommendation mechanism based on the facial feature information. The comprehensive recommendation mechanism includes an algorithm recommendation mechanism, an expert recommendation mechanism, and a popular makeup recommendation mechanism.

[0142] The facial feature information extracted from the face image can be used to recommend suitable makeup for the user.

[0143] The extracted facial feature information can be compared with the makeup images in the database to give makeup recommendations suitable for different face shapes and facial features. Specifically, the extracted facial feature information can be matched with the standard features in the database. The database includes classification information of various face shapes and facial features and the corresponding ideal makeup solutions. For example, for users with round faces, makeup styles that can lengthen the face and make the face look more three-dimensional will tend to be recommended; while for users with square faces, makeup with soft contours and weakened edges may be recommended. Ultimately, different customized solutions are provided based on the different feature information of users.

[0144] The comprehensive recommendation mechanism is a multi-dimensional recommendation method, including algorithm recommendation mechanism, expert recommendation mechanism and popular makeup recommendation mechanism, which aims to ensure the comprehensiveness and scientificity of the recommendation system.

[0145] Among them, the algorithm recommendation mechanism can automatically give the makeup suggestion that best suits the user's facial features based on the matching results of facial features and standard features in the database. This recommendation method focuses on the technical level and ensures the technicality and accuracy of the recommendation results.

[0146] The expert recommendation mechanism is a professional opinion provided by experienced makeup artists based on the specific situation of the user. Expert recommendations focus more on humanity and artistry, and can make up for the possible limitations of the algorithm.

[0147] The popular makeup recommendation mechanism analyzes the latest trends obtained from social media, fashion magazines, and people's outfits, and provides users with fashionable makeup recommendations. This method makes the recommendations more in line with the trend and meets the users' pursuit of fashion.

[0148] The above recommendation mechanism can be used to recommend makeup that the target user is interested in, so as to better cater to the user's preferences, improve the user experience and further improve the user's satisfaction.

[0149] In step 104, the weights corresponding to the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism are adjusted according to the target user's feedback information on the target makeup, so as to obtain updated weights of the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism. The comprehensive recommendation mechanism recommends the target makeup to the target user by updating the weights.

[0150] In some embodiments, scoring information can be extracted through user feedback information, and the average score corresponding to the recommendation mechanism can be calculated based on the scoring information to obtain the recommendation mechanism score, and the total average score of the algorithm recommendation mechanism, the expert recommendation mechanism, and the popular makeup recommendation mechanism can be calculated to obtain the total average score. Thereafter, the influence factor of the corresponding recommendation mechanism is determined by the difference between the recommendation mechanism score and the total average score, and the recommendation weight of the corresponding recommendation mechanism is updated based on the influence factor, and the recommendation weight is normalized to obtain normalized weight information.

[0151] For example, the comprehensive recommendation mechanism can dynamically adjust the number of recommendations of different recommendation mechanisms according to the user's rating. The weight ratio of the three recommendation mechanisms can be set to 1:1:1, and the user's rating of the recommended makeup will affect the weight of the corresponding recommendation mechanism. If the makeup recommended by a certain mechanism receives a high rating, the weight of the mechanism should be increased; otherwise, it should be reduced.

[0152] Users can comment or rate the recommended makeup. They can express their preference for the recommended makeup through a simple five-star rating system, and then convert the five-star rating into a corresponding score. After the user submits feedback, the system will adjust the weight of the recommendation source based on the user's rating. If the user is very satisfied with the recommended result, the system may increase the weight of the corresponding algorithm recommendation or expert recommendation; conversely, if the user does not like the recommended result, the corresponding weight may be reduced. The purpose of this is to enable the system to better predict and meet user preferences in the future.

[0153] The specific adjustment process can be:

[0154] 1. Calculate the score impact factor:

[0155] For each recommendation mechanism M i , normalize the scores of the recommended makeup and calculate their average score S i .

[0156] Afterwards, the average score S of all recommendation mechanisms is calculated.

[0157] Determine the score impact factor Δw i , Δw i =S i -S.

[0158] 2. Update weights

[0159] Update the weight of each recommendation mechanism according to the score impact factor, i is a variable:

[0160] w i =w i +Δw i .

[0161] Afterwards, normalize the weights:

[0162]

[0163] Calculate w j , that is, the updated weight, is the ratio of the sum of the three weights. i and j are variables, and their value range is the number of recommendation mechanisms.

[0164] 3. Dynamically adjust the number of recommendations for different recommendation mechanisms based on the recommendation weight.

[0165] As time goes by, the recommendation mechanism will continue to accumulate user feedback information and dynamically adjust the recommendation weight based on this information. This method helps the system gradually learn the long-term preferences of users, allowing the recommendation mechanism to continue to evolve and adapt to the personalized needs of users. This will achieve more intelligent services, provide users with more intimate and accurate makeup suggestions, and improve the overall user experience.

[0166] In some embodiments of the present application, the effects of the recommended makeup can also be displayed through visualization technology.

[0167] Visual display can be rendered in real time through OpenGL technology, which allows users to intuitively see the effect of makeup on their mobile phones or other mobile devices. This not only enhances the user experience, but also allows users to modify the makeup details according to their personal preferences until they are satisfied. The following is the specific implementation method of visual display:

[0168] 1. Initialize the OpenGL environment and set up rendering conditions. This includes loading the required shader programs, defining vertex array objects (VAOs) and vertex buffer objects (VBOs), and configuring texture units. Texture units are used to store images of cosmetic products, which will be mapped to key areas of the user's face in subsequent steps.

[0169] 2. In order to render the makeup effect on the user's face, the system needs to map the image of the makeup product to the corresponding facial area. Specifically, texture mapping technology is used to attach the image of the makeup product to the key parts of the user's face, such as the eyes, lips, etc. In this way, the user can intuitively see the effect of the makeup. The texture mapping process requires precise positioning of facial feature points to ensure the natural and harmonious makeup effect.

[0170] 3. In order to achieve a natural makeup effect, the system needs to adjust the rendering mode. OpenGL provides a variety of blending modes, such as GL_SRC_ALPHA and GL_ONE_MINUS_SRC_ALPHA, which determine how to blend the source color with the destination color. By setting the blending mode appropriately, you can ensure that the makeup effect blends perfectly with the user's original skin color, thus achieving a natural and harmonious effect.

[0171] 4. In order to make the makeup effect more realistic, the system also needs to simulate different lighting conditions. By introducing the lighting model in the shader, effects such as highlights and shadows can be added to make the makeup effect more three-dimensional and realistic. Users can adjust the direction and intensity of the lighting according to their personal preferences to obtain the best visual effect.

[0172] 5. When users are previewing makeup effects, the system should allow them to adjust makeup parameters, such as color depth, makeup intensity, etc., through sliders and other methods. These adjustments are actually real-time updates of rendering parameters. The system needs to respond to user input immediately and update the displayed results. This real-time adjustment function allows users to make fine adjustments based on their personal preferences until they find the most satisfactory makeup.

[0173] 6. In addition to the basic makeup effect display, the visualization display module should also provide rich user interaction functions. Users can freely choose different makeup templates, try a variety of styles, save their favorite makeup settings for quick application in the future, and share satisfactory makeup effects on social platforms to exchange experiences with friends.

[0174] Through the above steps, the visualization method can provide users with an efficient, convenient and personalized makeup experience. Users can preview the changes brought about by different makeup in real time in the application and adjust them according to their personal preferences until they find the one they like best. This module not only improves the user experience, but also provides a high-tech and intelligent platform for beauty brands, which helps to enhance the brand image and attract more users who pursue personalized and convenient services.

[0175] In some implementations of the present application, a data update mechanism may also be included.

[0176] The data update mechanism is responsible for the maintenance and update of the beauty database to ensure the timeliness and accuracy of the makeup suggestions and services provided. The design and implementation of the data update mechanism takes into account multiple factors, including but not limited to changes in beauty trends, analysis of user preferences, and user feedback.

[0177] Among them, beauty trend tracking involves regularly collecting and analyzing information from fashion magazines, social media platforms, beauty bloggers, professional beauty websites and other channels. The system uses web crawler technology to capture the content of these sources, and then uses natural language processing technology to extract key words about new makeup, popular colors, and popular products. A preliminary trend report is generated and reviewed by professional beauty consultants. Experts use their own experience and expertise to determine which trends are really worth paying attention to, and mark them in detail, including applicable population, seasonality, matching suggestions and other information.

[0178] User preference analysis is mainly achieved by analyzing the user's makeup records, search history and other data. For example, if a user is found to frequently view content about "nude makeup", the system will think that the user may be interested in natural and simple makeup styles. The system will increase the recommendation weight of related makeup and recommend more relevant beauty suggestions to the user in the future. In this way, the data update module can gradually learn and predict the user's preferences, and then provide more personalized services.

[0179] The beauty database is dynamically adjusted to include a large amount of makeup product information and various makeup matching suggestions. The data update mechanism can regularly update the database based on the results of beauty trend tracking and user preference learning. This includes not only adding currently popular makeup products and how to use them, but also removing outdated or no longer popular items. In addition, the system will also revise the content in the database based on user feedback to ensure the accuracy of the information.

[0180] The user feedback loop and data update mechanism will encourage users to share their views and feelings about makeup suggestions, and use these feedbacks as an important basis for optimizing the algorithm. Every time a user's evaluation is received, it will be analyzed to identify the key information, and then the algorithm parameters or database content will be adjusted accordingly. Through such a continuous learning process, the quality of makeup suggestions can be continuously improved to better meet user expectations.

[0181] Regular review and optimization can establish a regular review mechanism to ensure that all update operations are reasonable and effective. This is usually done by a team of professional beauty consultants and data scientists. They will regularly review the content in the database to check for incorrect information or outdated suggestions, and make corrections in a timely manner.

[0182] The data update mechanism is the key to the continuous evolution and vitality of beauty recommendation methods. By constantly tracking beauty trends, learning user preferences, updating beauty databases, and optimizing algorithms, the data update mechanism enables the entire system to keep pace with the times and provide users with more accurate and personalized makeup suggestions and services.

[0183] Please refer to Figure 2 , Figure 2 This is another flow chart of the beauty recommendation method based on facial features provided in some embodiments of the present application. Figure 2 In the process, the user's facial information is first obtained, and then the facial information is preprocessed. The preprocessed data is input into the EfficientNet network for image recognition to obtain the facial feature information, and the makeup recommendation results are given based on the facial feature information comprehensive algorithm recommendation, expert recommendation and popular makeup recommendation; the makeup recommendation results are simulated through the visualization display module to simulate the effect of the corresponding makeup after application, and the user gives feedback based on the makeup effect; the recommendation weight is adjusted through the feedback information to update the recommendation mechanism.

[0184] The above is an introduction to an embodiment of the beauty makeup recommendation method based on facial features of the present application.

[0185] The implementation of the beauty makeup recommendation device based on facial features provided by the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0186] For the beauty makeup recommendation method based on facial features provided in the above embodiment, the embodiment of the present application also provides a beauty makeup recommendation device based on facial features, which is used to implement the above method, such as Figure 3 As shown, Figure 3 This is a schematic block diagram of modules of a beauty makeup recommendation device based on facial features according to an embodiment of the present application. The beauty makeup recommendation device based on facial features 300 includes:

[0187] An acquisition module 301 is used to acquire a face image of a target user;

[0188] An input module 302 is used to input a face image into a face feature extraction model to obtain facial feature information of a target user. The face feature extraction model is obtained by training a neural network with a labeled face image data set. The facial feature information is used to characterize various data points of the face of the target user.

[0189] A recommendation module 303 is used to recommend a target makeup look for a target user through a comprehensive recommendation mechanism according to the facial feature information, wherein the comprehensive recommendation mechanism includes an algorithm recommendation mechanism, an expert recommendation mechanism, and a popular makeup look recommendation mechanism;

[0190] The adjustment module 304 is used to adjust the weights corresponding to the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism according to the feedback information of the target user on the target makeup, and obtain the updated weights of the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism. The comprehensive recommendation mechanism recommends the target makeup to the target user by updating the weights.

[0191] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0192] like Figure 4 As shown, the embodiment of the present application further provides an electronic device 400, the electronic device 400 includes a memory 401, one or more processors 402 ( Figure 4 Only one is shown in the figure) and a computer program stored in the memory 401 and executable on the processor 402. Among them: the memory 401 is used to store software programs and units, and the processor 402 executes various functional applications and data processing by running the software programs and units stored in the memory to obtain the resources corresponding to the above-mentioned preset events. Optionally, the processor 402 implements the above-mentioned beauty recommendation method based on facial features by running the above-mentioned computer program stored in the memory 401.

[0193] The memory 401, as a non-transitory computer-readable medium, can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory 401 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 401 may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor 402 via a network.

[0194] It can be understood that the contents of the above method embodiments are all applicable to the electronic device embodiment, the functions specifically implemented by the electronic device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0195] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned facial feature-based beauty recommendation method is implemented.

[0196] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0197] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is executed by one or more processors, it can implement the steps of the above-mentioned facial feature-based beauty recommendation method.

[0198] It can be understood that the contents of the above method embodiments are all applicable to the present computer program product, the functions specifically implemented by the present computer program product embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0199] The facial feature-based beauty recommendation method, device, electronic device, medium and computer program product provided in the embodiments of the present application can generate makeup suggestions that are more in line with the user's personal characteristics and preferences by comprehensively analyzing the user's facial features and current beauty trends, thereby improving the personalization of makeup suggestions; through an adaptive facial feature analysis algorithm and a personalized beauty database, it can meet the needs of users with different skin colors, face shapes and preferences, and provide more accurate and personalized makeup suggestions; using the mediapipe image processing library for facial detection and key point positioning, combined with deep learning technology, it can more accurately identify and locate facial key points, thereby improving the matching degree between makeup suggestions and the user's actual facial features; by superimposing makeup effects on the original image, the user can intuitively see what the makeup will look like after makeup. At the same time, users are allowed to adjust makeup effects, such as color depth, makeup intensity, etc., until they are satisfied, thereby improving user experience; based on user feedback, the system can automatically adjust algorithm parameters, optimize the accuracy of makeup suggestions and the realism of virtual makeup trial effects, thereby adapting to the needs of different users; by regularly updating the beauty database and learning from users' historical feedback and choices, the present invention can provide more personalized suggestions to ensure that users always obtain the latest beauty trend information; the embodiments of the present application can provide users with an efficient, convenient and personalized makeup experience by comprehensively using image recognition, deep learning, user interaction design and other technologies, helping users to easily master makeup skills that suit them and improve their overall image.

[0200] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0201] Although specific embodiments are described herein, those of ordinary skill in the art will recognize that many other modifications or alternative embodiments are also within the scope of the present disclosure. For example, any of the functions and / or processing capabilities described in conjunction with a particular device or component may be performed by any other device or component. In addition, although various exemplary implementations and architectures have been described according to embodiments of the present disclosure, those of ordinary skill in the art will recognize that many other modifications to the exemplary implementations and architectures described herein are also within the scope of the present disclosure.

[0202] Some aspects of the present disclosure are described above with reference to the block diagram and flow chart of the system, method, system and / or computer program product according to the exemplary embodiments.It should be understood that the combination of one or more blocks in the block diagram and the flow chart and the blocks in the block diagram and the flow chart can be realized by executing computer executable program instructions respectively.Equally, according to some embodiments, some blocks in the block diagram and the flow chart may not need to be executed in the order shown, or may not need to be executed in full.In addition, additional components and / or operations beyond those components and / or operations shown in the blocks in the block diagram and the flow chart may be present in certain embodiments.

[0203] Therefore, the blocks in the block diagrams and flow charts support the combination of devices for performing the specified functions, the combination of elements or steps for performing the specified functions, and the program instruction devices for performing the specified functions. It should also be understood that each block in the block diagrams and flow charts and the combination of blocks in the block diagrams and flow charts can be implemented by a special-purpose hardware computer system or a combination of special-purpose hardware and computer instructions that performs a specific function, element or step.

[0204] The program modules, applications, etc. described herein may include one or more software components, including, for example, software objects, methods, data structures, etc. Each such software component may include computer-executable instructions that, in response to execution, cause at least a portion of the functionality described herein (e.g., one or more operations of the illustrative methods described herein) to be performed.

[0205] Software components can be encoded with any of various programming languages. An exemplary programming language can be a low-level programming language, such as an assembly language associated with a specific hardware architecture and / or an operating system platform. The software component comprising assembly language instructions may need to be converted to executable machine code by an assembler before being executed by hardware architecture and / or platform. Another exemplary programming language can be a higher-level programming language, which can be transplanted across multiple architectures. The software component comprising a higher-level programming language may need to be converted to an intermediate representation by an interpreter or a compiler before execution. Other examples of programming languages ​​include but are not limited to macro languages, shells or command languages, job control languages, scripting languages, database queries or search languages ​​or report writing languages. In one or more exemplary embodiments, the software component comprising the instruction of one of the above-mentioned programming language examples can be directly executed by an operating system or other software components, without first being converted into another form.

[0206] Software components may be stored as files or other data storage structures. Software components of similar type or related functions may be stored together, such as in a specific directory, folder, or library. Software components may be static (e.g., preset or fixed) or dynamic (e.g., created or modified at execution time).

[0207] The embodiments of the present application are described in detail above in conjunction with the accompanying drawings, but the present application is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in the relevant technical field without departing from the purpose of the present application.

Claims

1. A beauty makeup recommendation method based on facial features, characterized in that: The following steps are involved: Obtain the target user's facial image; Inputting the face image into a face feature extraction model to obtain facial feature information of the target user, wherein the face feature extraction model is obtained by training a neural network with a labeled face image data set, and the facial feature information is used to characterize various data points of the face of the target user; recommending a target makeup look for the target user through a comprehensive recommendation mechanism according to the facial feature information, wherein the comprehensive recommendation mechanism includes an algorithm recommendation mechanism, an expert recommendation mechanism, and a popular makeup look recommendation mechanism; According to the target user's feedback information on the target makeup, the weights corresponding to the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism are adjusted respectively to obtain updated weights of the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism. The comprehensive recommendation mechanism recommends the target makeup to the target user through the updated weights.

2. The method for beauty makeup recommendation based on facial features according to claim 1, characterized in that: After acquiring the face image of the target user, the method further includes: Determining facial key points of the target user through the facial image; Determining a rotation angle of the face image according to the position information of the facial key points; Adjusting the facial image according to the rotation angle to obtain an updated facial image; The facial key points of the target user are re-determined based on the updated facial image to obtain the facial key points of the frontal lens.

3. The method for beauty makeup recommendation based on facial features according to claim 1, characterized in that: The step of training the neural network using a labeled face image dataset includes: Initializing weights of the neural network based on the variance of the weights in the weight matrix; The labeled face image data set is input into the neural network, and the neural network is trained through a forward propagation process, a loss calculation process and a back propagation process to obtain the face feature extraction model.

4. The method for beauty makeup recommendation based on facial features according to claim 1, characterized in that: The step of inputting the face image into a face feature extraction model to obtain facial feature information of the target user includes: Performing cropping, scaling and normalization processing on the face image to obtain a preprocessed face image; Inputting the preprocessed face image into the face feature extraction model, and extracting a feature vector corresponding to the face image before the last convolutional layer or fully connected layer in the face feature extraction model; The facial feature information corresponding to the face image is determined by the feature vector.

5. The method for beauty makeup recommendation based on facial features according to claim 4, characterized in that: The facial feature information includes facial features and face shape features, and determining the facial feature information corresponding to the face image through the feature vector includes: Cut out a region containing specific facial features from the feature vector to obtain a target region; Performing average pooling processing on the target area to obtain a feature description corresponding to the specific facial features, wherein the feature description is used to characterize the facial features; Determining facial contour data corresponding to the facial image through the feature vector; The facial features of the facial image are determined according to the facial contour data.

6. The method for beauty makeup recommendation based on facial features according to claim 1, characterized in that: The adjusting the weights corresponding to the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism respectively according to the feedback information of the target user on the target makeup includes: Extracting scoring information through the feedback information; Based on the rating information, calculating an average rating corresponding to the recommendation mechanism to obtain a recommendation mechanism rating; Calculating the total average score of the algorithm recommendation mechanism, the expert recommendation mechanism, and the popular makeup recommendation mechanism to obtain a total average score; Determine the impact factor corresponding to the recommendation mechanism by the difference between the recommendation mechanism score and the total average score; Update the recommendation weight of the corresponding recommendation mechanism based on the impact factor; The recommendation weights are normalized to obtain normalized weight information.

7. The method for beauty makeup recommendation based on facial features according to claim 1, characterized in that: After recommending a target makeup for the target user through a comprehensive recommendation mechanism according to the facial feature information, the method further includes: Setting rendering conditions, the rendering conditions including configuring a texture unit, the texture unit being used to store an image of a cosmetic product; According to the target makeup, mapping the image of the corresponding makeup product to the corresponding part of the face image; Introduce the lighting model into the shader to adjust the direction and intensity of the lighting to obtain the corresponding visual effects.

8. A beauty makeup recommendation device based on facial features, characterized in that: The device comprises: An acquisition module is used to acquire a face image of a target user; An input module, used to input the face image into a face feature extraction model to obtain facial feature information of the target user, wherein the face feature extraction model is obtained by training a neural network with a labeled face image data set, and the facial feature information is used to characterize various data points of the face of the target user; A recommendation module, configured to recommend a target makeup look to the target user through a comprehensive recommendation mechanism according to the facial feature information, wherein the comprehensive recommendation mechanism includes an algorithm recommendation mechanism, an expert recommendation mechanism, and a popular makeup look recommendation mechanism; An adjustment module is used to adjust the weights corresponding to the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism according to the feedback information of the target user on the target makeup, so as to obtain updated weights of the algorithm recommendation mechanism, the expert recommendation mechanism and / or the popular makeup recommendation mechanism. The comprehensive recommendation mechanism recommends the target makeup to the target user through the updated weights.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the facial feature-based beauty recommendation method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the facial feature-based beauty recommendation method according to any one of claims 1 to 7 is implemented.

Images