Virtual makeup trial method, device, electronic device and storage medium

Abstract

The present invention discloses a method, device, electronic device and storage medium for virtual makeup trial, and relates to the field of computer technology. A specific implementation of the method includes: acquiring a target image, the target image includes a real face of a user and an execution object that operates the real face, and identifying a target area and an operation strength of the execution object operating the real face; receiving a user instruction, the user instruction includes the characteristics of the makeup to be tried; and rendering and displaying the target area of ​​the real face in the target image according to the characteristics of the makeup to be tried and the operation strength. This implementation can solve the problem that for the same cosmetics, the makeup application method is single, and different makeup application methods cannot be provided to users, and the user needs cannot be met.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a virtual makeup trial method, device, electronic equipment and storage medium. Background Art

[0002] Makeup has become a lifestyle for all users, but with so many different types of cosmetics, how to quickly choose the right cosmetics for yourself has become an important need. With the development of intelligent technologies such as face recognition, virtual makeup trial technology has been widely used. Virtual makeup trial usually involves face recognition of the user, and then virtually displaying the cosmetics selected by the user on the recognized face in the corresponding makeup application method, thereby showing the effect of makeup, so that users can choose the right cosmetics based on these virtual makeup effects.

[0003] In the existing virtual makeup trial method, the makeup application method of each cosmetic is fixed, and the makeup application method for the same cosmetic is single, which cannot provide users with different makeup application methods and cannot meet user needs. Summary of the invention

[0004] In view of this, the embodiments of the present invention provide a virtual makeup trial method, device, electronic device and storage medium, which can solve the problem that the same cosmetics have a single makeup application method, cannot provide users with different makeup application methods, and cannot meet user needs.

[0005] To achieve the above objective, according to one aspect of an embodiment of the present invention, a virtual makeup trial method is provided.

[0006] A virtual makeup trial method according to an embodiment of the present invention includes: acquiring a target image, the target image including a real face of a user and an execution object that operates the real face, and identifying a target area and an operation force of the execution object operating the real face; receiving a user instruction, the user instruction including features of a makeup to be tried; and rendering and displaying a target area of ​​the real face in the target image according to the features of the makeup to be tried and the operation force.

[0007] In one embodiment, the identifying the intensity of the operation of the execution object on the real face includes:

[0008] Identify, according to the target image, operation features of the execution object operating the real human face;

[0009] The operation force with which the execution object operates the face is determined according to the correspondence between the operation feature and the operation force.

[0010] In yet another embodiment, identifying the target area of ​​the real face for the execution object to operate includes:

[0011] According to the target image, identifying the position information of the real human face corresponding to the preset key points and the operation position information of the execution object operating the real human face;

[0012] Determine, according to the operation position information and the position information of the preset key points, a set of key points that are closest to the operation position information;

[0013] The target area for the execution object to operate the real human face is determined according to the position information of the preset key points in the key point set.

[0014] In yet another embodiment, the operation feature comprises at least one of a deformation feature, a color distribution feature, and a shadow distribution feature of the real human face being operated on.

[0015] In yet another embodiment, the characteristics of the makeup to be tested include makeup type and display characteristics;

[0016] The step of rendering a target area of ​​a real face in the target image according to the features of the makeup to be tested and the operation strength includes:

[0017] Determining a makeup area to be tried corresponding to the makeup type according to the makeup type and a preset area template corresponding to the makeup type;

[0018] Determining whether the makeup area to be tried includes the target area;

[0019] If yes, determining a target makeup template corresponding to the makeup to be tested according to the makeup type, the display feature and the operation strength, and rendering the makeup area to be tested on the real face according to the target makeup template;

[0020] If not, a target makeup template corresponding to the makeup to be tested is determined according to the makeup type, the display feature and the operation strength, and a target area of ​​the real face is rendered according to the target makeup template.

[0021] In yet another embodiment, it is characterized in that determining the target makeup template corresponding to the makeup to be tested according to the makeup type, the display feature and the operation strength, and rendering the makeup area to be tested on the real face according to the target makeup template includes:

[0022] Determining a universal makeup template corresponding to the makeup to be tested according to the makeup type and the display characteristics;

[0023] Determining a target makeup template corresponding to the makeup to be tested according to the makeup type, the display feature, and the operation strength;

[0024] The target area of ​​the real face is rendered according to the target makeup template, and other areas of the to-be-tried makeup area of ​​the real face except the target area are rendered according to the target makeup template.

[0025] In yet another embodiment, the execution object includes a user's hand and / or a cosmetic tool.

[0026] To achieve the above objective, according to another aspect of an embodiment of the present invention, a virtual makeup trial device is provided.

[0027] A virtual makeup trial device according to an embodiment of the present invention includes: a recognition unit, used to capture a target image, the target image includes a real face of a user and an execution object that operates the real face, and to identify a target area and an operation force of the execution object operating the real face; a receiving unit, used to receive a user instruction, the user instruction includes features of a makeup to be tried; and a rendering unit, used to render and display a target area of ​​the real face in the target image according to the features of the makeup to be tried and the operation force.

[0028] In one embodiment, the identification unit is specifically used to:

[0029] Identify, according to the target image, operation features of the execution object operating the real human face;

[0030] The operation force with which the execution object operates the face is determined according to the correspondence between the operation feature and the operation force.

[0031] In yet another embodiment, the identification unit is specifically configured to:

[0032] According to the target image, identifying the position information of the real human face corresponding to the preset key points and the operation position information of the execution object operating the real human face;

[0033] Determine, according to the operation position information and the position information of the preset key points, a set of key points that are closest to the operation position information;

[0034] The target area for the execution object to operate the real human face is determined according to the position information of the preset key points in the key point set.

[0035] In yet another embodiment, the operation feature comprises at least one of a deformation feature, a color distribution feature, and a shadow distribution feature of the real human face being operated on.

[0036] In yet another embodiment, the characteristics of the makeup to be tested include makeup type and display characteristics;

[0037] The rendering unit is specifically used for:

[0038] Determining a makeup area to be tried corresponding to the makeup type according to the makeup type and a preset area template corresponding to the makeup type;

[0039] Determining whether the makeup area to be tried includes the target area;

[0040] If yes, determining a target makeup template corresponding to the makeup to be tested according to the makeup type, the display feature and the operation strength, and rendering the makeup area to be tested on the real face according to the target makeup template;

[0041] If not, a target makeup template corresponding to the makeup to be tested is determined according to the makeup type, the display feature and the operation strength, and a target area of ​​the real face is rendered according to the target makeup template.

[0042] In yet another embodiment, the rendering unit is specifically configured to:

[0043] Determining a universal makeup template corresponding to the makeup to be tested according to the makeup type and the display characteristics;

[0044] Determining a target makeup template corresponding to the makeup to be tested according to the makeup type, the display feature, and the operation strength;

[0045] The target area of ​​the real face is rendered according to the target makeup template, and other areas of the to-be-tried makeup area of ​​the real face except the target area are rendered according to the target makeup template.

[0046] In yet another embodiment, the execution object includes a user's hand and / or a cosmetic tool.

[0047] To achieve the above objective, according to another aspect of an embodiment of the present invention, an electronic device is provided.

[0048] An electronic device according to an embodiment of the present invention includes: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the virtual makeup trial method provided by the embodiment of the present invention.

[0049] To achieve the above objective, according to another aspect of an embodiment of the present invention, a computer readable medium is provided.

[0050] A computer-readable medium according to an embodiment of the present invention stores a computer program, which, when executed by a processor, implements the virtual makeup trial method provided by the embodiment of the present invention.

[0051] One embodiment of the above invention has the following advantages or beneficial effects: In the embodiment of the present invention, a target image is collected, and the target image includes the user's real face and the execution object that operates the real face, so the target image face recognition can be performed and the target area and operation force of the execution object operating the user's real face can be identified, and then after receiving the user's instruction, the target area in the target image can be rendered and displayed according to the characteristics of the makeup to be tried and the operation force. In this way, in the embodiment of the present invention, when the user virtually tries on makeup, the operation force and target area of ​​the execution object on the user's real face can be identified, so as to realize makeup rendering of the target area of ​​the target image based on the characteristics of the makeup to be tried and the operation force of the execution object, so that the same makeup item to be tried can be rendered with different depths of makeup according to the operation force, and any makeup trial on the face can be realized according to the target area operated by the execution object, so as to obtain different makeup trial effects, so as to provide users with more makeup application methods and meet the makeup trial needs of users.

[0052] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific implementation examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0054] Figure 1 is a schematic diagram of a main process of a virtual makeup trial method according to an embodiment of the present invention;

[0055] Figure 2 is a schematic diagram of another main process of the makeup rendering method according to an embodiment of the present invention;

[0056] Figure 3 is a schematic diagram of another main process of the virtual makeup trial method according to an embodiment of the present invention;

[0057] Figure 4 is a schematic diagram of main units of a virtual makeup trial device according to an embodiment of the present invention;

[0058] Figure 5 is another exemplary system architecture diagram to which the embodiments of the present invention can be applied;

[0059] Figure 6 It is a schematic diagram of the structure of a computer system suitable for implementing the embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0061] It should be pointed out that the embodiments of the present invention and the features in the embodiments may be combined with each other if there is no conflict.

[0062] An embodiment of the present invention provides a virtual makeup trial system, which can be used to provide a virtual makeup trial scenario for a user.

[0063] The embodiment of the present invention provides a virtual makeup trial method, which can be executed by a terminal device, such as Figure 1 As shown, the method includes the following steps.

[0064] S101: Acquire a target image, and identify a target area and an operation force of a real face operated by an execution object.

[0065] The target image includes the real face of the user and the execution object of the real face operation. The terminal device used to execute the embodiment of the present invention may include a shooting device, such as a camera, which can be used to collect the target image. The terminal device can periodically collect the target image through the shooting device, and the specific value of the period can be set according to the demand. In order to locate and identify the user's face in a timely and accurate manner, real-time collection is usually required, so the value of the period is usually set to a smaller value.

[0066] After the target image is collected, the target image can be identified, which usually includes face recognition and identification of the target area and operation intensity of the operation performed by the execution object on the real face.

[0067] After the terminal device captures the target image, it can use face recognition technology to identify the user's real face information. The real face information can specifically include the location information of each preset key point on the user's face, the location information of each part of the user's face, etc. When recognizing a face, it is usually necessary to first locate the preset characteristic parts of the user's face, and then accurately locate the positions of each part of the face by locating the preset key points corresponding to each characteristic part, and then recognize the real face, that is, the user's face. Face recognition can set more than 100 or even hundreds of key points for positioning, so that it can easily detect the user's wide-angle facial features, facial expressions, changes in facial light, etc., so that no matter how the user's muscles change, facial angles or expressions change, it can be accurately recognized.

[0068] It should be noted that, for the target image, if the real face and the execution object in the target image cannot be identified during recognition after acquisition, it means that there is no user who needs to try on makeup or the user has not tried on makeup. This target image can be ignored and the next target image acquired can be processed.

[0069] When trying on makeup, the user can adjust the makeup to be tried by operating the face. The terminal device collects the target image, so it can be determined whether the execution object has operated the user's real face based on the target image. When it is determined that the execution object has operated the user's real face, the target area and operation strength of the execution object's operation on the user's real face can be identified.

[0070] Specifically, in the embodiment of the present invention, the execution object may include hands and / or makeup tools, such as fingers or powder puffs and other tools used for makeup. The operation method of the execution object on the user's real face may include pressing, and the operation force is the pressing force. For the convenience of calculation, the value range of the operation force can be set in the embodiment of the present invention, and the operation force identified in this step is within the value range. For example, the value range of the operation force can be a decimal in the range of 0-1, where 0 represents that the execution object only fits the user's real face without operation, and 1 represents that the execution object operates the user's real face with all its strength, that is, the larger the value of the operation force, the greater the operation force of the execution object.

[0071] After the target image is collected and face recognition is performed, the operation of the execution object on the real face can be identified based on the target image, that is, the target area where the execution object operates the real face can be identified. The target area can be the area where the execution object contacts the real face, or the area where the execution object directly covers the real face. In this step, the operation characteristics of the execution object operating the real face can be identified based on the target image, and then the operation strength of the execution object operating the user's face can be determined based on the operation characteristics. Therefore, in the embodiment of the present invention, this step can be specifically performed as follows: the operation characteristics of the execution object operating the real face can be identified based on the target image; and the operation strength of the execution object operating the face can be determined based on the corresponding relationship between the operation characteristics and the operation strength. Among them, the operation characteristics include at least one of the deformation characteristics, color distribution characteristics, and shadow distribution characteristics of the real face being operated.

[0072] Specifically, in the embodiment of the present invention, the target image can be processed by an improved SSD (Lightweight Detection Model, Single Shot MultiBox Detector) model, and then the target area and operation strength of the real face performing the object operation can be identified.

[0073] Traditional SSD can identify the execution object in the target image. In the embodiment of the present invention, the traditional SSD can be improved. The improved SSD can include three branches: (1) a regression branch for obtaining the coordinates of the execution object frame (target area), and the area where the execution object operation is performed in the target image can be located according to the coordinates; (2) a classification branch for obtaining the confidence of whether the execution object frame is the execution object. If the confidence is greater than the set threshold (such as 0.5), it can be considered that the result obtained by the previous branch detection is a real execution object frame, otherwise it is considered to be a false detection; (3) a regression branch for obtaining the numerical value of the operation force, and the magnitude of the operation force of the execution object operation is a real face according to the numerical value of the operation force. Specifically, in the embodiment of the present invention, a fully connected layer can be added to the traditional SSD. After the feature layer in the SSD extracts the feature matrix, the fully connected layer can be input to obtain the magnitude of the operation force. At the same time, the SSD can extract the feature matrix according to the feature layer in the SSD to determine the target area of ​​the execution object operation real face.

[0074] In the embodiment of the present invention, the SSD model needs to be trained before using the SSD to perform this step. The execution object is a finger, and the operation on the real face is pressing.

[0075] During model training, images of fingers pressing on real faces can be collected in the dataset construction stage, and the correspondence between the value of the finger pressing force and the operation feature can be determined by observing the various operation features (such as facial deformation) in the collected image. The value of the finger pressing force can range from 0 to 1, and then the finger frame is annotated for each image, such as the horizontal and vertical coordinates of the upper left vertex and the lower right vertex of the finger frame in the image, and the value of the finger pressing force is annotated according to the preset correspondence, and then the annotated image is input into the SSD for training; in the training stage, the SSD can automatically extract various features from the image, such as operation features, etc., and learn their relationship with the position of the annotated finger frame and the value of the large pressure. For example, if the face is deeply sunken, the color of the pressed area is different from the surrounding area, and there is a more obvious shadow at the pressed area, it is considered that the pressing force is large, otherwise it is considered that the pressing force is small; SSD training is required to be tested. In the test phase, an image of a finger pressing a real face can be input into the trained SSD. The SSD outputs the recognition result according to the learned association, and specifically outputs the finger frame position coordinates, the confidence of whether the finger frame is a finger, and the value of the finger's pressing force. At the same time, the accuracy of the output result is determined based on the confidence. Specifically, if the confidence is less than the threshold, it is determined that the input image does not contain a finger, otherwise it can be determined that the output finger frame position coordinates and the value of the finger's pressing force are accurate.

[0076] It should be noted that the result obtained by direct recognition in SSD is usually the position of the execution object in the target image when operating the real face, that is, the operation position information of the execution object operating the real face in the target image, and there will be deviations between the positions of the specific features of the target image and the real face. In order to accurately determine the area where the execution object operates on the real face, the embodiment of the present invention can determine the area where the execution object operates on the real face based on the key points located during face recognition. Specifically, it can be performed as follows: according to the target image, identify the position information of the preset key points corresponding to the real face and the operation position information of the execution object operating the real face; according to the operation position information and the position information of the preset key points, determine the key point set closest to the operation position information; according to the position information of the preset key points in the key point set, determine the target area where the execution object operates the real face.

[0077] Among them, according to the target image recognition, the operation position information of the real face performing the object operation in the target image can be obtained according to SSD recognition, that is, the position information of the object frame, and then according to the operation position information and the position information of the preset key points, the distance between the operation position information and each key point is calculated, and a set of key points with a distance less than the preset distance threshold is obtained, that is, the set of key points closest to the operation position information, and then the target area can be determined by the position of the key point set at the user's face, that is, the target area of ​​the user's face where the object operation is performed, that is, the relative position of the position of the user's face where the object operation is performed and the user's face.

[0078] S102: Receive user instructions.

[0079] The user instruction includes the features of the makeup to be tested. The features of the makeup to be tested may include makeup type and display features. The makeup type may include various types used for makeup, such as lipstick, blush, foundation, etc. The display features refer to various features that affect the display effect of the makeup, such as color, texture, etc.

[0080] In an embodiment of the present invention, the terminal device may pre-store various makeup features. When a user tries on makeup, the terminal device may display various makeup features pre-stored for the user to select. The user enters a user instruction by clicking or the like, and the user instruction includes the features of the makeup that the user wants to try, that is, the features of the makeup to be tried. Since different makeup types correspond to different makeup features, for example, the display features corresponding to blush do not include black, usually, a correspondence between each makeup feature and the display feature can be pre-established. The user instruction may be entered twice. The first time the user instruction is entered, the terminal device queries the corresponding display features according to the makeup type to be tried and displays them for the user to select. The second time the user enters the user instruction, the display features of the makeup to be tried are included.

[0081] Alternatively, the terminal device may provide an input area for features of the makeup to be tried, and the user may enter a user instruction by inputting the features into the input area and then submitting the input. The user instruction includes the features of the makeup the user wants to try, i.e., the features of the makeup to be tried.

[0082] After receiving the user instruction, the terminal device can determine the characteristics of the makeup to be tried, and then determine the makeup type and display characteristics of the makeup to be tried. If the characteristics of the makeup to be tried included in the user instruction do not exist, a prompt message can be displayed to the user to prompt the user to re-enter the user instruction.

[0083] S103: Rendering and displaying a target area of ​​a real face in a target image according to the features of the makeup to be tested and the strength of the operation.

[0084] Among them, the operation force of the execution object on the real face represents the user's demand for the depth of makeup. Generally, the greater the operation force, the deeper the depth of makeup. In the embodiment of the present invention, the terminal device can pre-set the makeup templates corresponding to each makeup feature and operation force. Specifically, for the display features corresponding to each makeup type, makeup templates corresponding to different operation forces can be set. For example, for the makeup type of lipstick, it will correspond to multiple display features (such as color), and each display feature will display different effects for different operation forces. Therefore, in the embodiment of the present invention, the makeup templates corresponding to each operation force of each display feature of lipstick can be pre-set. Therefore, after determining the characteristics and operation force of the makeup to be tested, this step can query the target makeup template corresponding to the characteristics and operation force of the makeup to be tested, and render the target makeup template to the target area of ​​the target image, thereby obtaining the trial makeup effect, and displaying it so that the user can view the trial makeup effect.

[0085] In an embodiment of the present invention, when a user tries on makeup virtually, the operating force and target area of ​​the execution object on the user's real face can be identified, so as to render makeup on the target area of ​​the target image based on the characteristics of the makeup to be tried and the operating force of the execution object. This allows the same makeup item to be tried to be rendered with different shades of makeup according to the operating force, and any makeup trial on the face can be achieved according to the target area of ​​the execution object's operation, thereby obtaining different makeup trial effects. This can provide users with more makeup application methods to meet their makeup trial needs.

[0086] Combine the following Figure 1 The system architecture shown in FIG. 1 is used to specifically describe the makeup rendering method in step S103 of the embodiment of the present invention. Figure 2 As shown, the method includes the following steps.

[0087] S201: Determine a makeup area to be tried corresponding to the makeup type according to the makeup type and a preset area template.

[0088] In the beauty algorithm of virtual makeup trial, various templates are usually pre-set. Among them, each makeup type usually has a fixed makeup application position on the face. For example, if the makeup type is lipstick, its makeup application position is usually on the lips of the face. Therefore, based on the usual makeup application position of each makeup type, a corresponding preset area template can be set for each makeup type. After the face of each user is specifically identified, the preset area template of each makeup type can be deformed in combination with the identified face to make each preset area template more fit the user's real face. There are many deformation algorithms, such as affine transformation, IDW transformation, MLS transformation, RMLS transformation, etc.

[0089] Since the preset area template has been fitted to the user's real face, when the target area for performing the object operation is the same as or has a high degree of overlap with the area to be tried corresponding to the makeup type of the makeup to be tried, it can be said that the area the user wants to try makeup on is the area to be tried makeup, and makeup rendering can be performed according to the area to be tried makeup, and when the target area for performing the object operation is different from the area to be tried makeup corresponding to the makeup type of the makeup to be tried, it can be said that the area the user wants to try makeup on is not the area to be tried makeup, and makeup rendering can be performed according to the target area. Therefore, in the embodiment of the present invention, the area to be tried makeup corresponding to the makeup type of the makeup to be tried can be determined first according to the preset area template.

[0090] S202: Determine whether the makeup area to be tried includes the target area.

[0091] Among them, if the area to be tried on contains the target area, it can be said that the area the user wants to try on is the area to be tried on, and step S204 can be executed; if the area to be tried on does not contain the target area, it can be said that the area the user wants to try on is not the target area, and step S203 can be executed.

[0092] S203: If not, determining a target makeup template corresponding to the makeup to be tested according to the makeup type, display characteristics and operation strength, and rendering a target area of ​​the real face according to the target makeup template.

[0093] Among them, the terminal device is pre-set with makeup templates corresponding to each makeup type, display feature and operation strength, so in this step, the target makeup template corresponding to the makeup to be tested can be determined according to the makeup type, display feature and operation strength, and the target area of ​​the real face in the target image can be rendered according to the target makeup template.

[0094] S204: If yes, determine the target makeup template corresponding to the makeup to be tried according to the makeup type, display characteristics and operation strength, and render the makeup area to be tried in the real face according to the target makeup template.

[0095] Among them, the terminal device is pre-set with makeup templates corresponding to each makeup type, display feature and operation strength, so in this step, the target makeup template corresponding to the makeup to be tried can be determined according to the makeup type, display feature and operation strength, and the makeup area to be tried in the real face in the target image can be rendered according to the target makeup template.

[0096] In the embodiment of the present invention, when the makeup area to be tested includes the target area, it can also be considered that the user only adjusts the depth of the makeup in the target area, so the terminal device can pre-set the universal makeup template corresponding to each display feature of each makeup type, that is, the makeup template corresponding to each display feature of each makeup type when there is no object operation on the real face. Since the operation without the object is usually equivalent to the operation intensity of 0, the universal makeup template can be the makeup template corresponding to each display feature of each makeup type when the operation intensity is 0. In this way, in this step, the target area can be rendered according to the target makeup template, and the other areas of the makeup area to be tested on the real face except the target area can be rendered according to the universal makeup template. Specifically, this step can be performed as follows: according to the makeup type and the display characteristics, determine the universal makeup template corresponding to the makeup to be tested; according to the makeup type, the display characteristics and the operation intensity, determine the target makeup template corresponding to the makeup to be tested; render the target area of ​​the real face according to the target makeup template, and render the other areas of the makeup area to be tested on the real face except the target area according to the target makeup template.

[0097] In an embodiment of the present invention, when a user tries on virtual makeup, the target area operated by the executing object can be rendered based on the characteristics of the makeup to be tried and the strength of the executing object's operation on the real face, so as to achieve rendering of makeup of different depths for the characteristics of the same makeup to be tried, and any makeup trial can be achieved on the face according to the target area of ​​the operation, thereby obtaining different makeup trial effects. In this way, more makeup application methods can be provided to users to meet their makeup trial needs.

[0098] Combination Figure 1 and Figure 2 In the embodiment shown in the figure, in the embodiment of the present invention, the shooting device is a camera in the terminal device, and the execution object is a finger as an example to illustrate the technical solution. Figure 3 As shown, the specific process is as follows.

[0099] First, turn on the camera in the terminal device. The camera can periodically take images, and then process each frame of the image. The processing method is face recognition, and during face recognition, it is determined whether there are preset feature points (feature parts) in the image. If so, it means that the image includes a real face, so that the preset key points in the real face can be located according to the recognition result; if not, it means that the image does not include a real face, and the next frame of the image can be taken for processing. Then, after face recognition, it can be further identified whether the image includes a finger frame (the target area of ​​finger operation). If the finger frame is included, it means that the finger has operated on the real face, so the coordinates and operation strength of the finger frame can be identified and recorded; if the finger frame is not included, it means that the finger has not operated on the real face, and the next frame of the image can be taken for processing. After recording the coordinates and operation strength of the finger frame, makeup of different depths can be rendered according to the coordinates and operation strength of the finger frame, that is, the makeup corresponding to the depth of the operation strength is rendered at the position of the finger frame.

[0100] It should be noted that makeup rendering also needs to be based on the characteristics of the makeup to be tested. In the embodiment of the present invention, the characteristics of the makeup to be tested can be obtained in advance (this step is not included in Figure 3 ), and then the makeup rendering is performed in combination with the operation strength.

[0101] In an embodiment of the present invention, when a user tries on virtual makeup, the target area operated by the executing object can be rendered based on the characteristics of the makeup to be tried and the strength of the operating object on the real face, so as to achieve rendering of makeup of different depths for the characteristics of the same makeup to be tried, and any makeup trial can be achieved on the face according to the target area of ​​the operation, thereby obtaining different makeup trial effects. In this way, more makeup application methods can be provided to users to meet their makeup trial needs.

[0102] In order to solve the problems existing in the prior art, an embodiment of the present invention provides a virtual makeup trial device 400, such as Figure 4 As shown, the device 400 includes:

[0103] The recognition unit 401 is used to collect a target image, wherein the target image includes a real face of a user and an execution object that operates the real face, and recognize a target area and an operation strength of the execution object operating the real face;

[0104] A receiving unit 402 is used to receive a user instruction, wherein the user instruction includes a feature of a makeup to be tried;

[0105] The rendering unit 403 is used to render and display the target area of ​​the real face in the target image according to the characteristics of the makeup to be tested and the operation strength.

[0106] It should be understood that the manner of implementing the embodiments of the present invention is different from the Figure 1 The method of the illustrated embodiment is the same and will not be repeated here.

[0107] In an implementation manner of an embodiment of the present invention, in one embodiment, the identification unit 401 is specifically used to:

[0108] Identify, according to the target image, operation features of the execution object operating the real human face;

[0109] The operation force with which the execution object operates the face is determined according to the correspondence between the operation feature and the operation force.

[0110] In another implementation of the embodiment of the present invention, the identification unit 401 is specifically configured to:

[0111] According to the target image, identifying the position information of the real human face corresponding to the preset key points and the operation position information of the execution object operating the real human face;

[0112] Determine, according to the operation position information and the position information of the preset key points, a set of key points that are closest to the operation position information;

[0113] The target area for the execution object to operate the real human face is determined according to the position information of the preset key points in the key point set.

[0114] In another implementation of the embodiment of the present invention, the operation feature includes at least one of a deformation feature, a color distribution feature, and a shadow distribution feature of the real face being operated on.

[0115] In another implementation of the embodiment of the present invention, the features of the makeup to be tested include makeup type and display features;

[0116] The rendering unit 403 is specifically used for:

[0117] Determining a makeup area to be tried corresponding to the makeup type according to the makeup type and a preset area template corresponding to the makeup type;

[0118] Determining whether the makeup area to be tried includes the target area;

[0119] If yes, determining a target makeup template corresponding to the makeup to be tested according to the makeup type, the display feature and the operation strength, and rendering the makeup area to be tested on the real face according to the target makeup template;

[0120] If not, a target makeup template corresponding to the makeup to be tested is determined according to the makeup type, the display feature and the operation strength, and a target area of ​​the real face is rendered according to the target makeup template.

[0121] In another implementation of the embodiment of the present invention, the rendering unit 403 is specifically configured to:

[0122] Determining a universal makeup template corresponding to the makeup to be tested according to the makeup type and the display characteristics;

[0123] Determining a target makeup template corresponding to the makeup to be tested according to the makeup type, the display feature, and the operation strength;

[0124] The target area of ​​the real face is rendered according to the target makeup template, and other areas of the to-be-tried makeup area of ​​the real face except the target area are rendered according to the target makeup template.

[0125] In yet another implementation of the embodiment of the present invention, the execution object includes a user's hand and / or a makeup tool.

[0126] It should be understood that the manner of implementing the embodiments of the present invention is different from the Figure 1 , Figure 2 or Figure 3 The method of the illustrated embodiment is the same and will not be repeated here.

[0127] In an embodiment of the present invention, when a user tries on makeup virtually, the operating force and target area of ​​the execution object on the user's real face can be identified, so as to render makeup on the target area of ​​the target image based on the characteristics of the makeup to be tried and the operating force of the execution object. This allows the same makeup item to be tried to be rendered with different shades of makeup according to the operating force, and any makeup trial on the face can be achieved according to the target area of ​​the execution object's operation, thereby obtaining different makeup trial effects. This can provide users with more makeup application methods to meet their makeup trial needs.

[0128] According to an embodiment of the present invention, the embodiment of the present invention further provides an electronic device and a readable storage medium.

[0129] The electronic device of an embodiment of the present invention includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the data comparison method provided by the embodiment of the present invention.

[0130] Figure 5 An exemplary system architecture 500 is shown to which the virtual makeup trial method or virtual makeup trial apparatus according to the embodiments of the present invention may be applied.

[0131] like Figure 5As shown, system architecture 500 may include terminal devices 501, 502, 503, a network 504 and a server 505. Network 504 is used to provide a medium for communication links between terminal devices 501, 502, 503 and server 505. Network 504 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0132] The user can use the terminal devices 501, 502, 503 to interact with the server 505 through the network 504 to receive or send messages, etc. Various client applications can be installed on the terminal devices 501, 502, 503.

[0133] The terminal devices 501 , 502 , and 503 may be various electronic devices with a photographing device, including but not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, and the like.

[0134] The server 505 may be a server that provides various services. The server may be used to provide preset area templates or makeup templates, etc., to the terminal devices 501 , 502 , 503 .

[0135] It should be noted that the data comparison method provided in the embodiment of the present invention is generally executed by the server 505 , and accordingly, the data comparison device is generally set in the server 505 .

[0136] It should be understood that Figure 5 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0137] Reference below Figure 6 , which shows a schematic diagram of the structure of a computer system 600 suitable for implementing an embodiment of the present invention. Figure 6 The computer system shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0138] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0139] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0140] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present invention are executed.

[0141] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0142] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a unit, a program segment, or a part of a code, and the above-mentioned unit, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0143] The units involved in the embodiments of the present invention may be implemented by software or hardware. The units described may also be arranged in a processor, for example, it may be described as: a processor includes an identification unit, a receiving unit and a rendering unit. The names of these units do not constitute a limitation on the units themselves in some cases, for example, the receiving unit may also be described as a "unit of the function of the receiving unit".

[0144] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device executes the virtual makeup trial method provided by the present invention.

[0145] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A virtual makeup trial method, characterized in that: include: Capturing a target image, the target image including a real face of a user and an execution object that operates the real face, and identifying a target area and an operation strength of the execution object operating the real face; receiving a user instruction, wherein the user instruction includes characteristics of a makeup to be tried; Rendering and displaying a target area of ​​a real face in the target image according to the features of the makeup to be tested and the operation strength; The characteristics of the makeup to be tested include makeup type and display characteristics; The step of rendering a target area of ​​a real face in the target image according to the features of the makeup to be tested and the operation strength includes: Determining a makeup area to be tried corresponding to the makeup type according to the makeup type and a preset area template corresponding to the makeup type; Determining whether the makeup area to be tried includes the target area; If so, a target makeup template corresponding to the makeup to be tested is determined according to the makeup type, the display feature and the operation strength, and the makeup area to be tested on the real face is rendered according to the target makeup template.

2. The method according to claim 1, characterized in that The identifying the operation intensity of the execution object operating the real face includes: Identify, according to the target image, operation features of the execution object operating the real human face; The operation force with which the execution object operates the face is determined according to the correspondence between the operation feature and the operation force.

3. The method according to claim 2, characterized in that The identifying the target area of ​​the real face operated by the execution object includes: According to the target image, identifying the position information of the real human face corresponding to the preset key points and the operation position information of the execution object operating the real human face; Determine, according to the operation position information and the position information of the preset key points, a set of key points that are closest to the operation position information; The target area for the execution object to operate the real human face is determined according to the position information of the preset key points in the key point set.

4. The method according to claim 2, characterized in that: The operation feature includes at least one of a deformation feature, a color distribution feature, and a shadow distribution feature of the real human face being operated on.

5. The method according to claim 1, characterized in that Also includes: If not, a target makeup template corresponding to the makeup to be tested is determined according to the makeup type, the display feature and the operation strength, and a target area of ​​the real face is rendered according to the target makeup template.

6. The method according to claim 5, characterized in that The step of determining a target makeup template corresponding to the makeup to be tested according to the makeup type, the display feature, and the operation strength, and rendering the makeup area to be tested on the real face according to the target makeup template, includes: Determining a universal makeup template corresponding to the makeup to be tested according to the makeup type and the display characteristics; Determining a target makeup template corresponding to the makeup to be tested according to the makeup type, the display feature, and the operation strength; The target area of ​​the real face is rendered according to the target makeup template, and other areas of the to-be-tried makeup area of ​​the real face except the target area are rendered according to the target makeup template.

7. The method according to claim 1, characterized in that The execution object includes a user's hand and / or a cosmetic tool.

8. A virtual makeup trial device, characterized in that: include: an identification unit, configured to collect a target image, the target image including a real face of a user and an execution object that operates the real face, and identify a target area and an operation strength of the execution object operating the real face; A receiving unit, configured to receive a user instruction, wherein the user instruction includes a feature of a makeup to be tried; A rendering unit, configured to render and display a target area of ​​a real face in the target image according to the features of the makeup to be tested and the operation strength; The characteristics of the makeup to be tested include makeup type and display characteristics; The rendering unit is specifically used for: Determining a makeup area to be tried corresponding to the makeup type according to the makeup type and a preset area template corresponding to the makeup type; Determining whether the makeup area to be tried includes the target area; If so, a target makeup template corresponding to the makeup to be tested is determined according to the makeup type, the display feature and the operation strength, and the makeup area to be tested on the real face is rendered according to the target makeup template.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images