Virtual makeup method, device and equipment and computer readable storage medium

Abstract

The invention provides a virtual makeup method, apparatus and device, and a computer readable storage medium. The method comprises the steps of obtaining a target image including a human face; determining a face rotation angle and a plurality of face key points according to the target image; wherein the face rotation angle refers to an included angle formed in the horizontal direction when the face deflects with the front standard posture as the reference; adjusting each face key point based on the face rotation angle; and makeup is carried out on the target image based on each adjusted face key point and makeup material. According to the virtual makeup method, when the human face rotates from the front standard posture (namely, when the human face is converted into the side face), the key points of the human face are adjusted (or corrected) based on the human face rotation angle generated during rotation, then makeup processing is carried out based on the corrected key points of the human face, and makeup materials can be effectively prevented from being stretched. And makeup materials can be drawn at accurate positions of the human face, so that the makeup effect is further improved.

Description

[0001] This application is a divisional application of the application with the application number 202111661862.5, the application date of December 30, 2021, and the invention title of "Virtual Makeup Method, Device, Equipment and Computer Readable Storage Medium". Technical Field

[0002] This application relates to the field of image processing technology, and particularly relates to a virtual makeup method, device, equipment and computer readable storage medium. Background Art

[0003] Currently, in video processing scenarios (such as webcast scenarios, short video scenarios, etc.), there is often a need to automatically apply makeup to the user's face. Most of the common facial makeup methods currently are based on facial key points, that is, first identify the facial key points, and then draw the makeup based on the corresponding makeup materials at the positions of the facial key points to complete the makeup. This method has a better makeup effect on people's faces in a standard posture. When the face deflects, the facial makeup materials will be stretched, thus affecting the makeup effect. Summary of the Invention

[0004] In view of this, embodiments of the present application provide a virtual makeup method, device, equipment and computer readable storage medium.

[0005] In the first aspect, an embodiment of the present application provides a virtual makeup method, and the method includes:

[0006] Obtain a target image including a face;

[0007] Determine a face rotation angle and multiple facial key points based on the target image; wherein, the face rotation angle refers to the angle formed in the horizontal direction when the face deflects with respect to the frontal standard posture.

[0008] Adjust each of the facial key points based on the face rotation angle;

[0009] Apply makeup to the target image based on each adjusted facial key point and makeup materials.

[0010] In the second aspect, an embodiment of the present application provides a virtual makeup device, and the device includes:

[0011] A template image module, configured to obtain a target image including a face;

[0012] A key point and rotation angle determination module, configured to determine a face rotation angle and multiple facial key points based on the target image; wherein, the face rotation angle refers to the angle formed in the horizontal direction when the face deflects with respect to the frontal standard posture.

[0013] A key point adjustment module for adjusting each of the face key points based on the face rotation angle;

[0014] A makeup application module for applying makeup to the target image based on the adjusted face key points of each person and makeup materials.

[0015] In a third aspect, an embodiment of the present application provides a terminal device, including: a memory; one or more processors coupled to the memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the virtual makeup method provided in the first aspect above.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which program code is stored, and the program code can be called by a processor to execute the virtual makeup method provided in the first aspect above.

[0017] The virtual makeup method, device, equipment, and storage medium provided by the embodiments of the present application first obtain a target image including a face; then determine the face rotation angle and multiple face key points on the target image; wherein, the face rotation angle refers to the angle formed in the horizontal direction when the face deflects with the frontal standard posture as the reference; adjust each face key point based on the face rotation angle; and finally apply makeup to the target image based on the adjusted face key points of each person and makeup materials.

[0018] Since the face key points usually shift during the rotation of the face (i.e., from the frontal standard posture to the side face state), that is, there is a deviation between the actual face key points and the detected face key points. If makeup is applied based on the detected face key points, the makeup materials will be stretched. Therefore, in the virtual makeup method in this embodiment, when the face rotates from the frontal standard posture (i.e., turns into a side face), the face key points are adjusted (or corrected) based on the face rotation angle generated during the rotation, and then makeup application processing is performed based on the corrected face key points, which can effectively avoid the stretching of the makeup materials. And it can also be that the makeup materials are drawn at the accurate positions on the face, further improving the makeup effect. Description of the Drawings

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0020] Figure 1Schematic diagram of the application scenario of the virtual makeup method provided by the embodiment of the present application;

[0021] Figure 2 Schematic diagrams of the makeup of the front standard pose (i.e., the front face) and the side face provided by the embodiment of the present application;

[0022] Figure 3 Schematic diagrams of the triangular meshes of the front standard pose (i.e., the front face), the near-side human face before correction and after correction provided by an embodiment of the present application;

[0023] Figure 4 Schematic flowchart of the virtual makeup method provided by an embodiment of the present application;

[0024] Figure 5 Schematic diagrams of the triangular meshes of the front standard pose (i.e., the front face), the far-side human face before correction and after correction provided by an embodiment of the present application;

[0025] Figure 6 Schematic diagram of the human face mask provided by an embodiment of the present application;

[0026] Figure 7 Schematic flowchart of the virtual makeup method provided by another embodiment of the present application;

[0027] Figure 8 Structure diagram of the virtual makeup device provided by an embodiment of the present application;

[0028] Figure 9 Schematic diagram of the structure of the terminal device provided by an embodiment of the present application;

[0029] Figure 10 Schematic diagram of the structure of the computer-readable storage medium provided by an embodiment of the present application. Detailed implementation manners

[0030] The technical solutions in the embodiments of the present application will be clearly and completely described below. Apparently, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0031] To describe the present application in more detail, a virtual makeup method, device, terminal device, and computer storage medium provided by the present application will be specifically described below with reference to the accompanying drawings.

[0032] Please refer to Figure 1 , Figure 1FIG. 0 shows a schematic diagram of an application scenario of the virtual makeup method provided by an embodiment of the present application. This application scenario includes a terminal device 100 provided by an embodiment of the present application. The terminal device 100 can be various electronic devices with a display screen (such as the structural diagrams of 102, 104, 106, and 108), including but not limited to smart phones and computer devices, where the computer device can be at least one of devices such as a desktop computer, a portable computer, a laptop computer, and a tablet computer. First, a beauty application with a virtual makeup function is usually installed on the terminal device 100.

[0033] Optionally, the beauty application with a virtual makeup function can also be a live broadcast application with a beauty function. This live broadcast application can be used to perform virtual makeup (such as adding some makeup materials) on the face in the captured image (usually the live broadcast host), and then push the local video stream data to other live broadcast client devices through a live broadcast server. Further, the beauty application with a virtual makeup function can also be a short video or mini-video application with a beauty function. This short video or mini-video application is used to perform virtual makeup (such as adding some makeup materials) on the face during the process of shooting a short video, and publish the shot short video to a short video platform for other users to click and watch. The beauty application with a virtual makeup function in the embodiment of the present application can be one or more of the above-described beauty applications, and no specific limitation is made here.

[0034] Secondly, the terminal device 100 can generally refer to one of multiple terminal devices. Only the terminal device 100 is used as an example in this embodiment. Those skilled in the art can know that the number of the above terminal devices can be more or less. For example, the above terminal devices can be only a few, or the above terminal devices can be dozens or hundreds, or more. The embodiment of the present application does not limit the number and type of the terminal devices. The terminal device 100 can be used to execute a virtual makeup method provided by an embodiment of the present application.

[0035] Optionally, a GPU can be integrated in the terminal device 100. The GPU can be used to execute a virtual makeup method provided by an embodiment of the present application. Using the GPU can improve the makeup processing process.

[0036] In an optional implementation manner, in addition to the terminal device 100 provided by an embodiment of the present application, this application scenario can also include a server, where a network is provided between the server and the terminal device. The network is used as a medium to provide a communication link between the terminal device and the server. The network can include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0037] It should be understood that the numbers of the terminal devices, networks, and servers are merely illustrative. According to the implementation requirements, there can be any number of terminal devices, networks, and servers. For example, the server can be a server cluster composed of multiple servers, etc. Among them, the terminal device interacts with the server through the network to receive or send messages, etc. The server can be a server that provides various services. The server can be used to execute the steps of a virtual makeup method provided in the embodiments of the present application. In addition, when the terminal device executes a virtual makeup method provided in the embodiments of the present application, some steps can be executed on the terminal device, and some steps can be executed on the server, which is not limited here.

[0038] The virtual makeup method provided by the embodiments of the present application can be used to draw makeup on a human face, and it can be widely applied to fields such as online network live broadcasts and short videos. For the convenience of understanding, the following selects the online network live broadcast application scenario to illustrate this image processing method.

[0039] In addition, virtual makeup is usually based on face recognition key points, then a face mesh (such as a triangular mesh) is drawn according to the face, and then makeup is drawn. This method has a relatively good effect when the face is in a frontal standard pose and the face deflection angle is relatively small, but when the face deflection angle is relatively large, the facial makeup materials will be stretched. Please refer to Figure 2 As shown, when the face is in the front, the heart on the cheek is in a normal state. When the face rotates to the side face state, the heart on the cheek is stretched at this time, resulting in deformation. The main reason for this situation is that the face triangular mesh for drawing makeup is based on face key point recognition. The face key points (especially the edge key points) when the face deflection angle is relatively large are already points at different positions from the face key points when the face is in the frontal standard pose and the face deflection angle is relatively small. Please refer to Figure 3 As shown, the face key points will shift when the face is in the frontal standard pose and the side face state, so the generated triangular meshes are also different. Then, using the original face key points (that is, the face key points when the face is in the frontal standard pose and the face deflection angle is relatively small) to draw makeup will result in the stretching of the makeup materials.

[0040] Based on this, a virtual makeup method is provided in the embodiments of the present application. Please refer to Figure 4 , Figure 4 shows a schematic flowchart of a virtual makeup method provided by the embodiments of the present application. Taking the application of this method to the Figure 1 terminal device in as an example for illustration, it includes the following steps:

[0041] Step S110, obtain a target image containing a human face.

[0042] Among them, the target image refers to an image that needs to be processed for virtual makeup and includes human face data or information.

[0043] Virtual makeup means that through a beauty application, the user's face image can be processed, and corresponding makeup is drawn on the corresponding parts of the facial features or cheeks using makeup materials, so that the processed user's face image has the effect after makeup.

[0044] Makeup materials are usually some pictures or elements of makeup products, which can be covered on the corresponding parts of the face to produce a makeup effect. Makeup materials can include, but are not limited to, lipsticks, blushes, eyelashes, eyebrows, eyeliners, eyeshadows, and other stickers that can be pasted on the face (such as hearts, flags, etc.).

[0045] Step S120: Determine the face rotation angle and multiple face key points on the target image.

[0046] Among them, the face rotation angle refers to the included angle formed in the horizontal direction when the face deflects with the standard front pose as the reference.

[0047] When the face is completely frontal and the eyes are looking straight ahead, the formed pose is the standard front pose. When the face rotates to the right or left with the standard front pose as the reference, the left or right face is formed. Among them, the angle formed by the left or right face in the horizontal direction is recorded as the face rotation angle.

[0048] Step S130: Adjust each face key point based on the face rotation angle.

[0049] Specifically, after knowing the position or coordinates of the face key points and the face rotation angle, the position or coordinates of the face key points after the face rotation can be calculated, and then the face key points can be updated or adjusted.

[0050] Step S140: Apply makeup to the target image based on each adjusted face key point and makeup materials.

[0051] After adjusting the face key points, draw a triangular mesh based on the adjusted face key points, and then draw makeup based on the drawn triangular mesh and makeup materials, so as to complete the makeup processing on the target image.

[0052] Since the facial key points usually shift during the rotation of the face (i.e., from the frontal standard pose to the side face state), that is, there is a deviation between the actual facial key points and the detected facial key points. If makeup application is based on the detected facial key points, it will cause the makeup materials to be stretched. Therefore, in the virtual makeup method of this embodiment, when the face rotates from the frontal standard pose (i.e., turns into a side face), the facial key points are adjusted (or corrected) based on the facial rotation angle generated during the rotation, and then makeup application is performed based on the corrected facial key points, which can effectively avoid the stretching of makeup materials. And it can also be that the makeup materials are drawn at precise positions on the face, further improving the makeup effect.

[0053] In one embodiment, step S130 is executed to adjust each facial key point based on the facial rotation angle, including: determining the starting vector and ending vector of the vertical central axis of the face according to the target image; calculating the vector difference of the vertical central axis of the face according to the starting vector and the ending vector; and adjusting each facial key point according to the starting vector, the vector difference, and the facial rotation angle.

[0054] Specifically, the vertical central axis of the face refers to the central axis of the left and right faces. The starting vector refers to the vector formed from the coordinate origin (usually (0, 0)) to the starting point of the central axis; the ending vector refers to the vector formed from the coordinate origin (usually (0, 0)) to the ending point of the central axis. The starting point of the central axis can be the center point of the forehead or the center point of the two eyes. The ending point can be the center point of the chin.

[0055] In practical applications, on the one hand, the position of the forehead point is estimated, so the forehead point may not be very accurate; and there are large individual differences, and the positions of the forehead points of different users are not the same; on the other hand, pattern makeup is generally rarely used on the forehead, and the existence of the forehead point is mainly to serve as a transition buffer zone for makeup such as eyebrows. Based on this, in a preferred implementation, the starting point of the central axis is selected as the center point of the two eyes. The ending point can be the center point of the chin. After determining the starting vector and the ending vector, the vector difference of the vertical central axis of the face can be calculated. After obtaining the vector difference, each facial key point can be adjusted according to the starting vector, the vector difference, and the facial rotation angle.

[0056] Furthermore, several specific implementation methods for adjusting facial key points are given and described as follows:

[0057] The first implementation method:

[0058] In one embodiment, the facial key points include proximal facial key points. The proximal facial key points refer to the facial key points on the side face that is closer to the frontal standard pose when the face is deflected. Adjusting each facial key point according to the starting vector, the difference vector of the vertical central axis of the face, and the face rotation angle includes: correcting the vector difference based on the numerical value of the proximal key points; calculating the offset vector of each proximal facial key point according to the starting vector, the corrected vector difference, each proximal facial key point, and the face rotation angle; determining each new proximal facial key point according to each proximal facial key point and the offset vector corresponding to each proximal facial key point; and updating each proximal facial key point to the corresponding new proximal facial key point.

[0059] Specifically, the facial key points may include proximal facial key points and distal facial key points. The proximal facial key points refer to the key points of the proximal face, and the distal facial key points refer to the key points of the distal face. Among them, the proximal face refers to the side face that is closer to the frontal standard pose when the face is deflected, and the distal face refers to the side face that is farther from the frontal standard pose when the face is deflected. For example, when the face turns to the left, the right side face is the proximal face, and the left side face is the distal face; on the contrary, when the face turns to the right, the right side face is the distal face, and the left side face is the proximal face. When adjusting the proximal face, first correct the vector difference based on the numerical value of the proximal key points. The specific correction process may be to divide the vector difference by the numerical value of the proximal key points. Then calculate the offset vector of each proximal facial key point according to the starting vector, the corrected vector difference, each proximal facial key point, and the face rotation angle. After obtaining the offset vector of each proximal facial key point, perform a summation operation on each proximal facial key point and the offset vector corresponding to each proximal facial key point to determine each new proximal facial key point. Finally, update each proximal facial key point to the corresponding new proximal facial key point. Using this method can make the calculated new proximal facial key points more accurate.

[0060] It should be noted that when adjusting the proximal facial key points, the proximal facial key points are usually shrunk inward. In addition, when adjusting the proximal facial key points, the key points of the entire face can be adjusted, or only the key points of the entire cheek edge below the forehead can be adjusted. Since the offset of the key points on the forehead and the key points of the facial features is not obvious when forming a side face when the face is deflected, and there is less makeup material drawn in the forehead area, in order to improve the running speed and speed up the entire makeup process, it is possible to choose to only adjust the key points of the entire cheek edge below the forehead.

[0061] Furthermore, an implementation manner for calculating the offset vector of the proximal facial key points is given, and the specific description is as follows:

[0062] In one embodiment, calculating the offset vector for each proximal face key point based on the starting vector, the corrected vector difference, each proximal face key point, and the face rotation angle includes: calculating the vector sum of the starting vector and the corrected vector difference, denoted as the first vector; determining each proximal face key point vector based on each proximal face key point; calculating the vector difference between the first vector and each proximal face key point vector, denoted as each second vector; calculating the offset vector for each proximal face key point according to the first prediction vector coefficient, the face rotation angle, and each second vector. The vector of a proximal face key point refers to the vector formed from the origin of coordinates to the proximal face key point.

[0063] Specifically, let the proximal face key points (i.e., edge key points) of the face be P1, P2…Pn, where n represents the numerical value of the number of proximal face key points; the starting vector and the ending vector of the vertical central axis of the face are denoted as P FaceAxisStart , P FaceAxisEnd ; the face deflection angle θ, vector difference = P FaceAxisEnd - P FaceAxisStart ; the corrected vector difference P step = 1 / n(P FaceAxisEnd - P FaceAxisStart ); the first vector = P FaceAxisStart + i * P step , n represents the numerical value of the number of proximal face key points; the second vector = P FaceAxisStart + i * P step - P i ; the offset vector = k i * sin(θ)(P FaceAxisStart + i * P step - P i ). Then the calculation formula for the i-th new proximal face key point is as follows:

[0064] P′ i = P i + k i * sin(θ) (P FaceAxisStart + i * P step - P i )

[0065] Where, P′ i represents the coordinates of the i-th new proximal face key point; P i represents the coordinates of the i-th proximal face key point, 1 ≤ i ≤ n; k i represents the first prediction vector coefficient, which can also be called the fine-tuning coefficient of the i-th proximal face key point.

[0066] The second implementation method:

[0067] In one embodiment, the facial key points include distal facial key points. The distal facial key points refer to the facial key points on the side face that are away from the frontal standard posture when the face is deflected. Adjusting each facial key point according to the starting vector, the vector difference, and the face rotation angle includes: calculating the offset vector of each distal facial key point according to the starting vector, the corrected vector difference, each distal facial key point, and the face rotation angle; determining each new distal facial key point according to the starting vector and the offset vector of each distal facial key point; and updating each distal facial key point to the corresponding new distal facial key point.

[0068] When adjusting the distal face, first correct the vector difference based on the numerical value of the proximal key points. The specific correction process can be to divide the vector difference by the numerical value of the proximal key points. Then calculate the offset vector of each distal facial key point according to the starting vector, the corrected vector difference, each distal facial key point, and the face rotation angle. After obtaining the offset vector of each distal facial key point, perform a summation operation on the starting vector and the offset vector of each distal facial key point to determine each new distal facial key point. Finally, update each distal facial key point to the corresponding new distal facial key point. Using this method can make the calculated new distal facial key points more accurate.

[0069] It should be noted that when adjusting the distal facial key points, usually the distal facial key points are expanded outwards, and the result is as Figure 5 shown. In addition, when adjusting the distal facial key points, the key points of the entire face can be adjusted, or only the key points at the edge of the entire cheek below the forehead can be adjusted. Since the offset of the key points on the forehead and the key points of the facial features is not obvious when the face is deflected to form a side face, and there is less makeup material drawn in the forehead area, in order to improve the running speed and accelerate the entire makeup process, it is possible to choose to only adjust the key points at the edge of the entire cheek below the forehead.

[0070] Furthermore, an implementation manner for calculating the offset vector of the distal facial key points is given, and the specific description is as follows:

[0071] In one embodiment, calculating the offset vector of each distal facial key point according to the starting vector, the corrected vector difference, each distal facial key point, and the face rotation angle includes: determining each distal facial key point vector based on each distal facial key point; calculating the vector difference between each distal facial key point vector and the first vector, and recording it as each third vector; calculating each fourth vector according to the second prediction vector coefficient, the face rotation angle, and each third vector; and calculating the vector sum of the corrected vector difference and each fourth vector, and recording it as the offset vector of each distal facial key point. The vector of the distal facial key point refers to the vector formed from the coordinate origin to the distal facial key point.

[0072] Specifically, following the embodiment of the proximal face key points. The distal face key points (i.e., edge key points) can be set as P1, P2…Pm, where m represents the numerical value of the number of proximal face key points; the starting vector and ending vector of the vertical central axis of the face are respectively denoted as P FaceAxisStart , P FaceAxisEnd ; the face deflection angle θ, vector difference = P FaceAxisEnd - P FaceAxisStart ; the corrected vector difference P step = 1 / n(P FaceAxisEnd - P FaceAxisStart ); the third vector = P j - (P FaceAxisStart + j * P step ); the fourth vector = (1 + k j ) * sin(θ) ((P j - (P FaceAxisStart + j * P step )); the offset vector = j * P step + (1 + k j ) * sin(θ) ((P j - (P FaceAxisStart + j * P step ). Then the calculation formula for the j-th new distal face key point is as follows:

[0073] P′ j = P FaceAxisStart + j * P step + (1 + k j ) * sin(θ) ((P j - (P FaceAxisStart + j * P step )

[0074] Wherein, P′ j represents the coordinate of the j-th new distal face detection key point; P j represents the coordinate of the j-th proximal face key point, 1 ≤ j ≤ m; 1 + k j represents the second prediction vector coefficient. k j represents the fine-tuning coefficient of the j-th proximal face key point.

[0075] Next, an implementation method for applying makeup based on face key points and makeup materials is also given, which is described in detail as follows:

[0076] In one embodiment, applying makeup to the target image based on the adjusted facial key points and makeup materials includes: when the facial key points are the distal facial key points, drawing a facial mask template according to each facial key point before adjustment; applying makeup to the target image according to the facial mask template, each adjusted distal facial key point, and the makeup materials.

[0077] Since the adjustment of the distal facial key points is to expand the facial key points, then drawing a triangular mesh according to the adjusted distal facial key points, and then drawing the makeup according to the triangular mesh and the makeup materials, it is easy to cause the makeup to overflow, that is, the makeup materials are drawn to positions outside the facial area. To avoid this problem, a facial mask template (i.e., mask, as Figure 6 shown) can be drawn using the facial key points before adjustment, and then makeup is applied to the target image according to the facial mask template, each adjusted facial key point, and the makeup materials.

[0078] In one embodiment, applying makeup to the target image according to the facial mask template, each adjusted distal facial key point, and the makeup materials to form a final makeup image can be represented by the following formula:

[0079] P out (x,y) = P in (x,y) * (1 - P mask (x,y)) + P makeup (x,y) * P mask (x,y)

[0080] Wherein, P out (x, y) represents the final makeup image, P in (x, y) represents the triangular mesh formed according to each adjusted distal facial key point; P mask (x, y) represents the facial mask template (i.e., mask), P makeup (x, y) represents the makeup materials; x and y respectively represent the points of the facial image on the coordinate axes.

[0081] To facilitate understanding of the virtual makeup method of the present application, a detailed embodiment is given. Please refer to Figure 7, the specific process of the virtual makeup method is as follows: 1. Obtain the target image of the user; 2. Perform face point detection on the target image; 3. Calculate the face key points in the case of face deflection (i.e., side face), and then adjust or correct the face key points, and draw a face triangle mesh using the corrected face key points; 4. Draw a face triangle mesh using the face key points (i.e., the face key points before correction); 5. Draw a face mask area based on the face triangle mesh drawn using the face key points before correction; 6. Draw makeup using makeup materials and the face triangle mesh drawn based on the corrected face key points; 7. Intersect the drawn makeup and the face mask area to prevent the makeup from being drawn outside the face area. This step is applicable when applying makeup to a far-side face. If it is a near-side face, this step can be omitted; 8. Mix the target image with the makeup and output the final makeup image.

[0082] It should be understood that although Figure 2 and Figure 7 the steps in the flowcharts of Figure 2 and Figure 7 are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover

[0083] In the above embodiments disclosed in the present application, a virtual makeup method is described in detail. For the above method disclosed in the present application, it can be implemented by devices in various forms. Therefore, the present application also discloses a virtual makeup device corresponding to the above method, and specific embodiments are given below for detailed description.

[0084] Please refer to Figure 8 , a virtual makeup device disclosed in an embodiment of the present application, mainly includes:

[0085] A template image module 810, configured to obtain a target image including a face.

[0086] A key point and rotation angle determination module 820, configured to determine a face rotation angle and multiple face key points according to the target image; wherein, the face rotation angle refers to the included angle formed in the horizontal direction when the face deflects with respect to the front standard posture.

[0087] The key point adjustment module 830 is used to adjust each face key point based on the face rotation angle.

[0088] The makeup application module 840 is used to apply makeup to the target image based on each adjusted face key point and makeup materials.

[0089] In one embodiment, the key point adjustment module 830 is configured to determine a starting vector and an ending vector of the vertical central axis of the face according to the target image; calculate the vector difference of the vertical central axis of the face according to the starting vector and the ending vector; and adjust each face key point according to the starting vector, the vector difference, and the face rotation angle.

[0090] In one embodiment, the face key points include proximal face key points. The proximal face key points refer to the face key points on the side face that is close to the frontal standard pose when the face is deflected. The key point adjustment module 830 is configured to correct the vector difference based on the numerical value of the proximal key points; calculate the offset vector of each proximal face key point according to the starting vector, the corrected vector difference, each proximal face key point, and the face rotation angle; determine each new proximal face key point according to each proximal face key point and the offset vector corresponding to each proximal face key point; and update each proximal face key point to the corresponding new proximal face key point.

[0091] In one embodiment, the face key points include distal face key points. The distal face key points refer to the face key points on the side face that is far from the frontal standard pose when the face is deflected. The key point adjustment module 830 is configured to calculate the offset vector of each distal face key point according to the starting vector, the corrected vector difference, each distal face key point, and the face rotation angle; determine each new distal face key point according to the starting vector and the offset vector of each distal face key point; and update each distal face key point to the corresponding new distal face key point.

[0092] In one embodiment, the key point adjustment module 830 is configured to calculate the vector sum of the starting vector and the corrected vector difference, denoted as the first vector; determine each proximal face key point vector based on each proximal face key point; calculate the vector difference between the first vector and each proximal face key point vector, denoted as each second vector; and calculate the offset vector of each proximal face key point according to the first prediction vector coefficient, the face rotation angle, and each second vector.

[0093] In one embodiment, the key point adjustment module 830 is configured to determine each distal face key point vector based on each distal face key point; calculate the vector difference between each distal face key point vector and the first vector, denoted as each third vector; calculate each fourth vector according to the second prediction vector coefficient, the face rotation angle, and each third vector; calculate the vector sum of the corrected vector difference and each fourth vector, denoted as the offset vector of each distal face key point.

[0094] In one embodiment, the makeup application module 840 is configured to, when the face key point is a distal face key point, draw a face mask template according to each face key point before adjustment; apply makeup to the target image according to the face mask template, each adjusted distal face key point, and makeup materials.

[0095] For the specific limitations of the virtual makeup application device, reference can be made to the limitations of the method in the foregoing text, which will not be elaborated herein. Each module in the above device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in or independent of the processor in the terminal device in the form of hardware, or stored in the memory of the terminal device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0096] Please refer to Figure 9 , Figure 9 which shows a structural block diagram of a terminal device provided in an embodiment of the present application. The terminal device 90 may be a computer device. The terminal device 90 in the present application may include one or more of the following components: a processor 92, a memory 94, and one or more application programs, where one or more application programs may be stored in the memory 94 and configured to be executed by one or more processors 92, and one or more application programs are configured to execute the methods described in the embodiments of the above virtual makeup application method.

[0097] The processor 92 may include one or more processing cores. The processor 92 connects various parts within the entire terminal device 90 through various interfaces and lines, and executes various functions of the terminal device 90 and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 94, and by calling data stored in the memory 94. Optionally, the processor 92 may be implemented in at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 92 may integrate one or a combination of several of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU) for reporting and verifying buried point data, and a modem, etc. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing display content; the modem is used to process wireless communication. It can be understood that the above-mentioned modem may not be integrated into the processor 92 and may be implemented separately through a communication chip.

[0098] The memory 94 may include Random Access Memory (RAM), and may also include Read-Only Memory. The memory 94 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 94 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for implementing at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data created during the use of the terminal device 90.

[0099] Those skilled in the art can understand that Figure 9 the structure shown in

[0100] merely represents a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the terminal device to which the solution of the present application is applied. The specific terminal device may include more or fewer components than those shown in the figure, or combine certain components, or have a different component layout.

[0101] Please refer to Figure 10, which shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. Program code is stored in the computer-readable storage medium 1000, and the program code can be called by a processor to execute the method described in the above-mentioned embodiment of the virtual makeup method.

[0102] The computer-readable storage medium 1000 may be an electronic memory such as a flash memory, EEPROM (electrically erasable programmable read-only memory), EPROM, a hard disk, or a ROM. Optionally, the computer-readable storage medium 1000 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 1000 has a storage space for the program code 1002 that executes any method step in the above method. These program codes can be read out from or written into one or more computer program products. The program code 1002 may be compressed in a suitable form, for example.

[0103] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0104] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A virtual makeup method, characterized in that, The method includes: Obtaining a target image containing a human face; wherein, the target image refers to an image that needs to be subjected to virtual makeup processing and includes human face data or information; Determining a human face rotation angle and multiple human face key points based on the target image; wherein, the human face rotation angle refers to the included angle formed in the horizontal direction when the human face deflects with respect to the frontal standard posture; Adjusting each of the human face key points based on the human face rotation angle; including: determining a starting vector and an ending vector of the vertical central axis of the human face according to the target image; calculating a vector difference of the vertical central axis of the human face based on the starting vector and the ending vector; adjusting each of the human face key points based on the starting vector, the vector difference, and the human face rotation angle; Applying makeup to the target image based on each adjusted human face key point and makeup materials.

2. The method according to claim 1, characterized in that, The human face key points include proximal human face key points, and the proximal human face key points refer to the human face key points on the side face of the human face that is close to the frontal standard posture when the human face deflects; the adjusting each of the human face key points based on the starting vector, the vector difference of the vertical central axis of the human face, and the human face rotation angle includes: Correcting the vector difference based on the numerical value of the number of proximal key points; Calculating an offset vector for each of the proximal human face key points based on the starting vector, the corrected vector difference, each of the proximal human face key points, and the human face rotation angle; Determining each new proximal human face key point based on each of the proximal human face key points and the offset vector corresponding to each of the proximal human face key points; Updating each of the proximal human face key points to the corresponding new proximal human face key point.

3. The method according to claim 1, characterized in that The human face key points include distal human face key points, and the distal human face key points refer to the human face key points on the side face of the human face that is far from the frontal standard posture when the human face deflects; the adjusting each of the human face key points based on the starting vector, the vector difference, and the human face rotation angle includes: Calculating an offset vector for each of the distal human face key points based on the starting vector, the corrected vector difference, each of the distal human face key points, and the human face rotation angle; Determining each new distal human face key point based on the starting vector and the offset vector of each of the distal human face key points; Updating each of the distal human face key points to the corresponding new distal human face key point.

4. The method according to claim 2, characterized in that, The calculating an offset vector for each of the proximal human face key points based on the starting vector, the corrected vector difference, each of the proximal human face key points, and the human face rotation angle includes: Calculating the vector sum of the starting vector and the corrected vector difference, denoted as the first vector; Determining a vector of each proximal human face key point based on each proximal human face key point; Calculating the vector difference between the first vector and the vector of each proximal human face key point, denoted as each second vector; Calculating an offset vector for each of the proximal human face key points based on a first prediction vector coefficient, the human face rotation angle, and each of the second vectors.

5. The method according to claim 3, characterized in that Calculating an offset vector for each of the far facial key points according to the starting vector, the corrected vector difference, each of the far facial key points, and the facial rotation angle includes: Determining a far facial key point vector for each of the far facial key points; Calculating a vector difference between each far facial key point vector and the first vector, denoted as each third vector; Calculating each fourth vector according to the second prediction vector coefficient, the facial rotation angle, and each of the third vectors; Calculating a vector sum of the corrected vector difference and each of the fourth vectors, denoted as the offset vector for each of the far facial key points.

6. The method according to any one of claims 1-5, characterized in that, Applying makeup to the target image based on each adjusted facial key point and makeup material includes: When the facial key point is a far facial key point, drawing a facial mask template according to each facial key point before adjustment; Applying makeup to the target image according to the facial mask template, each adjusted far facial key point, and the makeup material.

7. A virtual makeup device, characterized in that, The device includes: A template image module, configured to obtain a target image including a face; wherein, the target image refers to an image that needs to be subjected to virtual makeup processing and includes facial data or information; A key point and rotation angle determination module, configured to determine a facial rotation angle and multiple facial key points according to the target image; wherein, the facial rotation angle refers to an angle formed in the horizontal direction when the face deflects with respect to the frontal standard pose; A key point adjustment module, configured to adjust each of the facial key points based on the facial rotation angle; including: determining a starting vector and an ending vector of the vertical central axis of the face according to the target image; calculating a vector difference of the vertical central axis of the face according to the starting vector and the ending vector; adjusting each of the facial key points according to the starting vector, the vector difference, and the facial rotation angle; A makeup application module, configured to apply makeup to the target image based on each adjusted facial key point and makeup material.

8. A terminal device, characterized in that, Including: A memory; One or more processors, coupled to the memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, Program code is stored in the computer-readable storage medium, and the program code can be called by the processor to execute the method according to any one of claims 1-6.

Images