Virtual human makeup redirection method and device, electronic equipment and storage medium

Abstract

The invention provides a virtual human makeup redirection method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a template face corresponding to a target face; and redirecting the template makeup of the template face based on the front views of the template face and the target face to obtain the target makeup of the target face. According to the technical scheme provided by the invention, the redirection processing of the template makeup is unbound with the face grid, the template makeup is not migrated according to the face grid any more, and the redirection of the template makeup is carried out based on the front view of the template face and the target face, so that the redirection of the template makeup is realized. And based on the front view, considering the face shape difference possibly existing between the target face and the template face, so that the display effect of the template makeup on the template face is the same as the display effect of the target makeup on the target face.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual digital humans, and in particular to a method, device, electronic device and storage medium for redirecting virtual human makeup. Background Art

[0002] With the continuous development of 3D virtual human technology, 3D video generation platforms allow users to create personalized 3D virtual humans as characters in 3D videos on the platform. Specifically, users can choose different hairstyles, face shapes, facial features, eyebrow shapes, makeup, and clothing for 3D virtual humans to create personalized 3D virtual humans.

[0003] In terms of technical implementation, different types of hairstyles, face shapes, facial features, eyebrow shapes, makeup, and clothing are pre-stored on the platform and deployed on the template face of the 3D virtual person for display and selection by users. When creating a personalized 3D virtual person, the hairstyle, face shape, facial features, eyebrow shape, makeup, clothing, etc. that match the template face of the 3D virtual person need to be migrated to the personalized 3D virtual person created by the user.

[0004] The existing makeup migration solution is to bind the makeup pattern to the face mesh of the 3D virtual person, and deploy and extend the makeup pattern along the wiring of the face mesh. However, due to the different face shapes of the template face and the target face, the face meshes are also different. Directly migrating the makeup based on the corresponding face mesh cannot maintain the display effect of the makeup. Summary of the invention

[0005] The present invention provides a method, device, electronic device and storage medium for redirecting virtual human makeup, which are used to solve the defects in the prior art and achieve the same display effect of template makeup on the template face as that of target makeup on the target face.

[0006] The present invention provides a method for redirecting virtual human makeup, comprising: Obtain a template face corresponding to the target face; The template makeup of the template face is redirected based on the front view of the template face and the target face to obtain the target makeup of the target face.

[0007] According to the present invention, a method for redirecting virtual human makeup is provided, wherein the template makeup of the template face is redirected based on the front view of the template face and the target face, comprising: Determining makeup areas in the front views of the template face and the target face respectively, and constructing a template positioning grid in the makeup area of ​​the template face; Redirecting the template positioning grid to obtain a target positioning grid corresponding to the target face; Based on the position offset of each vertex in the template positioning grid and the target positioning grid, the template makeup of the template face is redirected.

[0008] According to the present invention, a method for redirecting virtual human makeup is provided, wherein the step of constructing a template positioning grid in the makeup area of ​​the template face comprises: Calibrate at least one template reference positioning point in the makeup area of ​​the template face; the template reference positioning point includes an edge contour point of the makeup area of ​​the template face; The template positioning grid is constructed based on the template reference positioning points and the face mesh vertices of the template face.

[0009] According to the present invention, a method for redirecting virtual human makeup is provided, wherein the template positioning grid is redirected to obtain a target positioning grid corresponding to the target face, comprising: Calibrate at least one target reference positioning point in the makeup area of ​​the target face; the target reference positioning point corresponds to the template reference positioning point one by one; The template positioning grid is redirected based on the Laplace smoothness constraint to obtain the position information of each vertex in the target positioning grid; wherein each vertex in the template positioning grid corresponds to each other one by one.

[0010] According to the present invention, a method for redirecting virtual human makeup is provided, wherein the template makeup of the template face is redirected based on the position offset of each vertex in the template positioning grid and the target positioning grid, and comprises: Determine the position offset of each vertex in the template positioning grid and the target positioning grid in the two-dimensional texture space grid; Based on the position offset, the template makeup of the template face is translated and / or scaled.

[0011] According to the present invention, a method for redirecting virtual human makeup is provided, wherein determining the position offset of each vertex in the template positioning grid and the target positioning grid in the two-dimensional texture space grid comprises: Based on the position information of each vertex in the template positioning grid and the target positioning grid in the front view, calculating the centroid coordinates of each vertex in the face grid; Based on the correspondence between the two-dimensional texture space grid and the face grid, the position offset of each vertex in the two-dimensional texture space grid is calculated.

[0012] According to the present invention, a method for redirecting virtual human makeup is provided, wherein the template makeup includes template lip makeup and / or template eye makeup, and the vertices of the template positioning grid corresponding to the template eye makeup include key contour points of the template eye makeup.

[0013] According to the present invention, a method for redirecting virtual human makeup is provided, wherein the template positioning grid is redirected based on the Laplace smoothness constraint to obtain the position information of each vertex in the target positioning grid, including: Constructing the Laplace smooth constraint based on the position information of each vertex in the template positioning grid and the position information of each target reference positioning point in the target positioning grid, as well as the topological structure of the template positioning grid and the target positioning grid; constructing eye makeup structure constraints based on position information of key contour points of the template eye makeup; The Laplace smoothness constraint and the eye makeup structure constraint are jointly optimized to obtain position information of each vertex in the target positioning grid.

[0014] The present invention also provides a virtual human makeup redirection device, comprising: An acquisition module configured to acquire a template face corresponding to a target face; The redirection module is configured to redirect the template makeup of the template face based on the front view of the template face and the target face to obtain the target makeup of the target face.

[0015] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a method for redirecting a virtual human makeup as described in any one of the above is implemented.

[0016] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for redirecting the virtual human makeup as described in any one of the above is implemented.

[0017] The present invention also provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned methods for redirecting virtual human makeup.

[0018] The redirection method, device, electronic device and storage medium of virtual human makeup provided by the present invention obtain a template face corresponding to a target face, and then redirect the template makeup of the template face using a front view based on the template face and the target face to obtain a target makeup of the target face. The redirection method of virtual human makeup provided by the present invention unbinds the redirection processing of the template makeup from the face grid, and no longer migrates the template makeup based on the face grid, but redirects the template makeup based on the front view of the template face and the target face, and considers the possible face shape difference between the target face and the template face based on the front view, so that the display effect of the template makeup on the template face is the same as the display effect of the target makeup on the target face, and will not be affected by the face shape difference, and the template makeup can be adapted to target faces with different face shapes, which can meet user needs and improve user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 Schematic diagram of a face mesh of a 3D virtual human provided by an embodiment of the present invention.

[0021] Figure 2 This is an example diagram of a two-dimensional face texture image provided by the present invention.

[0022] Figure 3 It is a schematic diagram of using a two-dimensional face texture image to perform texture mapping on a 3D virtual human face model.

[0023] Figure 4 It is a schematic diagram of the effect of lip makeup migration in the prior art.

[0024] Figure 5 It is a schematic diagram of the effect of eye makeup migration in the prior art.

[0025] Figure 6 It is a flow chart of the redirection method of virtual human makeup provided by the present invention.

[0026] Figure 7 It is a schematic diagram of marking the lip line in the front view of the template face provided by the present invention.

[0027] Figure 8 is a schematic diagram of a lip segmentation area provided by an embodiment of the present invention.

[0028] Fig. 9 It is a schematic diagram of marking lip line contour points in a front view of a template face provided by the present invention.

[0029] Fig.10 It is a schematic diagram of a template positioning grid for template lip makeup provided by the present invention.

[0030] Fig.11 It is a schematic diagram of redirecting the template positioning grid of the template lip makeup from the template face to the target face in an embodiment of the present invention.

[0031] Fig.12 It is a schematic diagram of grid redirection based on two-dimensional texture space in an embodiment of the present invention.

[0032] Fig.13 This is a display effect diagram of adding the same template lip makeup to the template face and the target face in an embodiment of the present invention.

[0033] Fig.14 It is a schematic diagram of marking key contour points provided by the present invention.

[0034] Fig.15 It is a schematic diagram of a template positioning grid for template eye makeup provided by the present invention.

[0035] Fig.16 This is a display effect diagram of adding the same template eye makeup to the template face and the target face in an embodiment of the present invention.

[0036] Fig.17 It is a structural schematic diagram of the redirection device for virtual human makeup provided by the present invention.

[0037] Fig.18 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] The terms "first", "second", etc. in the present invention are used to distinguish similar objects, rather than to describe a specific order or precedence. It should be understood that the terms used in this way can be interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type.

[0040] like Figure 1 As shown, Figure 1 3D virtual human face model can be regarded as an irregular surface in three-dimensional space. In order to facilitate the processing of the irregular surface, the usual method is to first segment the irregular surface into a number of triangles or quadrilaterals, thereby forming a mesh of a 3D virtual human face. Figure 1 The face mesh shown in the figure is composed of face mesh lines and face mesh vertices. The face mesh is equivalent to adding longitude and latitude lines to the face. Based on the segmentation of the face by the face mesh, different positions on the face can be corresponded to different triangles or quadrilaterals. In other words, a coordinate system similar to "longitude and latitude" is built on the face through the face mesh, so that different positions on the face can be determined according to the face mesh.

[0041] In addition to an irregular surface formed in three-dimensional space, the surface of the 3D virtual human face model also needs to be textured, that is, the surface of the 3D virtual human face model is covered with an image to make the surface of the 3D virtual human face model have photo-level realism.

[0042] like Figure 2 and Figure 3 As shown, Figure 2 An example of a two-dimensional face texture image provided by the present invention, Figure 3 For use Figure 2 Schematic diagram of the effect of texture mapping of the 3D virtual human face model using a two-dimensional face texture image ( Figure 3 The face in the figure has been blurred). It can be seen that the face of a 3D virtual person is composed of a 3D face model and a corresponding 2D face texture image. The 3D face model mainly determines the face shape and facial features of the 3D virtual person, while the 2D face texture image mainly determines the face skin of the 3D virtual person.

[0043] To add makeup (such as eye makeup and lip makeup) to the above 3D virtual person, it is necessary to draw eye makeup effects such as eyeliner and eye bags around the eyes of the 3D virtual person, or draw lip makeup effects in the lip area. That is, draw eye makeup or lip makeup patterns in the area corresponding to the eyes or lips in the two-dimensional face texture image of the 3D virtual person.

[0044] It can be understood that to adjust the makeup of the 3D virtual person, it is only necessary to modify the makeup pattern drawn in the corresponding area of ​​the 2D face texture image of the 3D virtual person. For example, to adjust the lip makeup of the 3D virtual person from M lips to smiling lips, it is only necessary to use the smiling lip pattern instead of the M lip pattern to draw in the area corresponding to the lips of the 2D face texture image.

[0045] The face of the 3D virtual person and the corresponding makeup pattern have a very high match. When the face of the 3D virtual person remains unchanged, to adjust the makeup of the 3D virtual person, you only need to modify the makeup pattern. However, since the face shape of the personalized 3D virtual person created by the user is obviously different from that of the template face of the 3D virtual person, directly applying the makeup pattern corresponding to the template face of the 3D virtual person to the personalized 3D virtual person will cause a huge change in the display effect of the makeup, and it will not be consistent with the makeup of the template face of the 3D virtual person. Figure 4 and Figure 5 As shown ( Figure 4 The face in the image has been blurred. In the prior art, Figure 4 The lip makeup pattern that matches the template face of the 3D virtual person is directly applied to Figure 4 When the personalized 3D virtual person on the right is Figure 4 The lip makeup effect changes from round to sharp at the direction indicated by the arrow in . Figure 5 The eye makeup pattern adapted to the template face of the 3D virtual human is directly applied to Figure 5 When the personalized 3D virtual person on the right is Figure 5 Where the arrow points, the eye makeup effect changes from smooth to distorted.

[0046] Based on this, the present invention provides a method, device, electronic device and storage medium for redirecting virtual human makeup. Figure 6-Figure 16 The present invention describes a method for redirecting virtual human makeup.

[0047] Figure 6 FIG. 1 is a flow chart showing a method for redirecting virtual human makeup according to an exemplary embodiment. Figure 6 As shown, in an exemplary embodiment, the method for redirecting virtual human makeup includes steps 610 to 620, which are described in detail as follows.

[0048] Step 610: Acquire a template face corresponding to the target face.

[0049] In the embodiment of the present invention, the target face refers to the face of a personalized 3D virtual person created by a user. The user needs to generate a target makeup based on the face shape of the target face and draw the target makeup in the corresponding area of ​​the target face. The template face refers to the face of a 3D virtual person template adapted to the template makeup selected by the user for the target face. The template face is configured with the template makeup selected by the user so that the user can intuitively feel the display effects of different makeups and choose the makeup they like. The template face of a 3D virtual person is a pre-designed 3D face with a specific shape and features.

[0050] Step 620 , redirecting the template makeup of the template face based on the front view of the template face and the target face to obtain the target makeup of the target face.

[0051] In the embodiment of the present invention, the front view is a static view of the front face of the 3D virtual person. Since there will be a problem of makeup mismatch when the makeup of the template face is applied to the target face, it is necessary to redirect the template makeup of the template face to obtain the target makeup of the target face. After an in-depth study of the above technical issues, it is found that the main reason for the above problems is that the facial meshes are different due to different face shapes, so that the facial mesh is directly used as a reference system for makeup migration. Although the makeup after migration maintains the corresponding relationship with the facial mesh, the facial mesh itself has changed, which causes the display effect of the makeup to change.

[0052] The redirection method of virtual human makeup provided by the embodiment of the present invention first obtains a template face corresponding to a target face, and then redirects the template makeup of the template face using a front view based on the template face and the target face to obtain a target makeup of the target face. The method provided by the embodiment of the present invention unbinds the redirection processing of the template makeup from the face grid, and no longer migrates the template makeup based on the face grid, but redirects the template makeup based on the front view of the template face and the target face, introduces the front view in the redirection processing, and considers the possible face shape difference between the target face and the template face based on the front view, so that the display effect of the template makeup on the template face is the same as the display effect of the target makeup on the target face, and will not be affected by the face shape difference. The target makeup obtained after the template makeup is redirected can be adapted to target faces with different face shapes, which can meet user needs and improve user experience.

[0053] In an exemplary embodiment of the present invention, redirecting the template makeup of the template face based on the front view of the template face and the target face includes: Determine the makeup area in the front view of the template face and the target face respectively, and construct a template positioning grid in the makeup area of ​​the template face; Redirecting the template positioning grid to obtain a target positioning grid corresponding to the target face; Based on the position offset of each vertex in the template positioning grid and the target positioning grid, the template makeup of the template face is redirected.

[0054] In an embodiment of the present invention, a corresponding makeup area is determined in a front view of a template face, and a corresponding makeup area is determined in a front view of a target face. A template positioning grid is constructed in the makeup area of ​​the template face, and the constructed template positioning grid is used for redirection processing to obtain a target positioning grid corresponding to the target face. The makeup area includes both the area where makeup is required and the surrounding area of ​​the area where makeup is required. For example, when the template makeup is a template lip makeup, the makeup area includes the lip area where makeup is required and also includes an area corresponding to a distance extending outward along the lip area.

[0055] The target positioning grid in the embodiment of the present invention is generated after the template positioning grid is redirected. The redirection process mainly migrates the positions of each vertex in the template positioning grid. Therefore, the target positioning grid corresponds to the template positioning grid, that is, the vertices of the two grids and the lines between the vertices correspond to each other. Based on the position offset of each vertex in the template positioning grid and the target positioning grid, the template makeup of the template face can be redirected.

[0056] In an exemplary embodiment of the present invention, constructing a template positioning grid in a makeup area of ​​a template face includes: Calibrate at least one template reference positioning point in the makeup area of ​​the template face; the template reference positioning point includes an edge contour point of the makeup area of ​​the template face; A template positioning mesh is constructed based on the template reference positioning points and the face mesh vertices of the template face.

[0057] It can be understood that the target positioning grid in the embodiment of the present invention is generated based on the template positioning grid. Therefore, it is necessary to first construct the template positioning grid in the makeup area of ​​the template face. There are many ways to construct the template positioning grid, which will not be described here. In order to facilitate the subsequent redirection processing of the template positioning grid, the embodiment of the present invention uses the template reference positioning point as a framework and combines other vertices to construct the template positioning grid.

[0058] In the embodiment of the present invention, at least one template reference positioning point is determined in the makeup area of ​​the template face. This embodiment takes the template lip makeup as an example. In the front view of the template face without makeup (no makeup added), the inner and outer contour points of the lips and the lip line contour points are marked as the positioning reference points of the lips. Specifically, Figure 7 As shown in FIG. 1 , the lip line can be marked in the front view of the template face without makeup by manual marking, and the lip line can be used to generate the following Figure 8 The lip segmentation area (i.e., lip mask) shown in FIG. is marked at the preset position of the lip line based on the lip segmentation area, and the marked point is used as the lip line contour point. Fig. 9As shown in the figure, several lip line contour points are marked on the lip line of the template face through the above method as template reference positioning points, denoted as .

[0059] In the embodiment of the present invention, Fig.10 As shown in , in addition to the lip line contour points, several contour points can be selected on the edge contours around the lips of the template face (i.e., the lip outer contour) and the contour between the upper and lower lips (i.e., the lip inner contour) as template reference positioning points, such as Fig.10 The area enclosed by the outer contour points of the lips is the makeup area corresponding to the template lip makeup.

[0060] Based on the above description of the face mesh of a 3D virtual person, it can be seen that the face mesh is composed of face mesh lines and face mesh vertices, and the face mesh can be used to determine different positions on the face. Fig.10 As shown, in this embodiment, some vertices are selected from the face mesh vertices in the makeup area (i.e., the area within the outer contour of the lips) as template sampling points (i.e. Fig.10 The red dot in , the selected template reference positioning points and template sampling points are used as vertices of the template positioning grid to construct the template positioning grid, which is recorded as T. The selected template sampling points can sparsely represent the makeup area.

[0061] It should be noted that the above implementation method selects some vertices from the face mesh vertices of the template face as the vertices of the template positioning mesh, mainly because the vertices, grid lines and other related information of the face mesh of the template face are predetermined and can be used directly. However, after determining the template reference positioning point, several points in the makeup area can be randomly selected as template sampling points, and together with the template reference positioning point, they are used as the vertices of the template positioning mesh to construct the template positioning mesh.

[0062] In an exemplary embodiment of the present invention, redirecting the template positioning grid to obtain a target positioning grid corresponding to the target face includes: Calibrate at least one target reference positioning point in the makeup area of ​​the target face; the target reference positioning point corresponds to the template reference positioning point one by one; The template positioning grid is redirected based on the Laplace smoothness constraint to obtain the position information of each vertex in the target positioning grid; wherein each vertex in the template positioning grid corresponds to each vertex in the target positioning grid.

[0063] In an embodiment of the present invention, at least one target reference positioning point is calibrated in the makeup area of ​​the target face, and the selection of the target reference positioning point corresponds to the selection of the template reference positioning point. It can be understood that all vertices of the target positioning grid to be generated include target reference positioning points and target sampling points, the target reference positioning points correspond to the template reference positioning points, and the target sampling points correspond to the template sampling points. After the target reference positioning points are calibrated, the position information of the target reference positioning points is determined. In addition, the redirection processing of the template positioning grid still maintains the topological structure of the template positioning grid, that is, the adjacency relationship between vertices remains unchanged. Therefore, to generate the target positioning grid corresponding to the target face, it is only necessary to determine the position information of each target sampling point.

[0064] Laplace smoothness constraint is a smoothing technique commonly used in image processing and computer graphics, which can make the target makeup of the generated target face more natural and smooth visually. Based on the Laplace smoothness constraint, the template positioning grid is redirected to obtain the position information of each target sampling point in the target positioning grid. Combined with the position information of each target reference positioning point, the position information of each vertex in the target positioning grid can be obtained.

[0065] Specifically, select n template reference positioning points of the template face ( ) and m template sampling points ( ),in, ( ) represents the two-dimensional coordinate vector of the template reference positioning point of the template face in the front view of the template face, ( ) represents the two-dimensional coordinate vector of the template sampling point of the template face in the front view of the template face, n template reference positioning points and m template sampling points Together they form all the vertices of the template positioning mesh T .

[0066] Furthermore, based on the topological structure of the template positioning grid T of the makeup area of ​​the template lip makeup, the corresponding Laplacian matrix can be constructed, which is denoted as ,in, , , , Represent three matrices respectively. The number of rows and columns of these three matrices is the same, which is the number of all vertices of the template positioning grid T. , the relevant introduction of matrices D and A is as follows: is the degree matrix of the template positioning grid T. The diagonal elements of the degree matrix are used to represent the number of grid lines emitted by each vertex of the template positioning grid T, and the other elements are 0.

[0067] It is the adjacency matrix of the template positioning grid T. The diagonal elements of the adjacency matrix are all 0, and the other elements are used to represent the adjacency relationship between each vertex of the template positioning grid T. If the element is 0, it means no adjacency, and if the element is 1, it means adjacency.

[0068] Therefore, the Laplacian matrix The diagonal elements of are used to indicate the number of grid lines emitted by each vertex of the template positioning grid T, and the other elements are used to indicate the adjacency relationship between each vertex of the template positioning grid T. If the element is 0, it indicates no adjacency, and if the element is -1, it indicates adjacency.

[0069] The vertex coordinate matrix of the template positioning grid T can be expressed as .

[0070] Based on the above description, it can be seen that since the face shapes of the template face and the target face are different, the template positioning grid T is transferred from the template face to the target face. It is necessary to redirect the template positioning grid T to obtain the target positioning grid adapted to the target face. .

[0071] It should be noted that all elements in the Laplacian matrix in this embodiment are used to identify the topological structure of the template positioning grid T and the target positioning grid. , while the template positioning grid T and the target positioning grid The topological structure of the target positioning grid is exactly the same. The corresponding Laplace matrix is ​​also .

[0072] Similar to the template positioning grid T, the target positioning grid All vertices of By n target reference positioning points ( ) and m target sampling points ( )composition, ( ) represents the two-dimensional coordinate vector of the target reference positioning point of the target face in the frontal view of the target face, ( ) represents the two-dimensional coordinate vector of the target sampling point of the target face in the frontal view of the target face.

[0073] Positioning the grid for the target The vertex coordinate matrix can be expressed as .

[0074] It can be understood that the n target reference positioning points of the target face After the selection is completed, the two-dimensional coordinate vector in the front view is also determined. Therefore, to determine the target positioning grid All vertices of , m target sampling points need to be determined The 2D coordinate vector in the front view.

[0075] Establishing Laplace smoothness constraints , after substituting the two-dimensional coordinate vectors of each vertex into it, the Laplace smooth constraint can be transformed into: .

[0076] Based on the above Laplace smoothness constraints, m target sampling points can be calculated The two-dimensional coordinate vector in the front view. Fig.11 As shown, based on the calculated two-dimensional coordinate vector, the template positioning grid of the template lip makeup is redirected from the template face to the target face.

[0077] In an exemplary embodiment of the present invention, based on the position offset of each vertex in the template positioning grid and the target positioning grid, the template makeup of the template face is redirected, including: Determine the position offset of each vertex in the template positioning grid and the target positioning grid in the two-dimensional texture space grid; Based on the position offset, the template makeup of the template face is translated and / or scaled.

[0078] In the embodiment of the present invention, based on the two-dimensional coordinate vectors of each vertex of the template positioning grid and the target positioning grid obtained above, the position information of each vertex in the two-dimensional texture space grid can be determined, and then the position offset between each vertex can be determined. The two-dimensional texture space grid is a grid formed by each vertex in the positioning grid in the dimension of the two-dimensional face texture image.

[0079] Based on the above description, it can be known that the face of a 3D virtual person is composed of a 3D face model and a corresponding two-dimensional face texture image. The two-dimensional face texture image mainly determines the facial skin of the 3D virtual person, and the makeup acts on the skin. Therefore, based on the position offset of each vertex in the two-dimensional texture space grid, the template makeup is translated and / or scaled to maintain the makeup effect well.

[0080] Specifically, Fig.12As shown in the figure, the green grid in the figure is the two-dimensional texture space grid corresponding to the template positioning grid of the template makeup of the template face, and the red grid in the figure is the two-dimensional texture space grid corresponding to the target positioning grid of the target makeup of the target face. Based on the position offset of each vertex in the template positioning grid and the target positioning grid that corresponds one to one in the above two-dimensional texture space grid, the template lip makeup of the template face is deformed, that is, translated and / or scaled, to obtain the template lip makeup adapted to the target face. The position offset is the position deviation between two corresponding vertices in the template positioning grid and the target positioning grid.

[0081] like Fig.13 As shown, Fig.13 The display effect pictures are respectively showing the same template lip makeup added to the template face and the target face. Although there are differences in the lip shapes of the template face and the target face, the template lip makeup still maintains a similar display effect on different lip shapes.

[0082] In an exemplary embodiment of the present invention, determining the position offset of each vertex in the template positioning grid and the target positioning grid in the two-dimensional texture space grid includes: Based on the position information of each vertex in the template positioning grid and the target positioning grid in the front view, the centroid coordinates of each vertex in the face grid are calculated; Based on the correspondence between the two-dimensional texture space grid and the face grid, the position offset of each vertex in the two-dimensional texture space grid is calculated.

[0083] In the embodiment of the present invention, all vertices of the template positioning grid based on the makeup area in the front view of the template face are The two-dimensional coordinate vector of the template face is combined with the face mesh of the template face to determine the vertices in the template positioning mesh The correspondence between the two-dimensional texture space grid and the face grid is the mapping relationship between the three-dimensional space coordinate system and the two-dimensional texture coordinate system, that is, the mapping relationship between the two-dimensional face texture image and the face model mentioned above, which determines the vertices in the template positioning grid. The two-dimensional texture coordinates of .

[0084] Accordingly, all vertices of the target positioning mesh of the makeup area of ​​the target face in the front view obtained based on the above calculation are The two-dimensional coordinate vector of the target face is combined with the face mesh of the target face to determine the vertices The centroid coordinates in the face mesh. Then, based on the mapping relationship between the three-dimensional space coordinate system and the two-dimensional texture coordinate system, the vertices in the target positioning mesh are determined. The two-dimensional texture coordinates of the mesh. Positioning each vertex in the mesh based on the template and the corresponding target location grid vertices The two-dimensional texture coordinates can be used to calculate the position offset of each vertex in the two-dimensional texture space grid.

[0085] In an exemplary embodiment of the present invention, the template makeup includes template lip makeup and / or template eye makeup, and the vertices of the template positioning grid corresponding to the template eye makeup include key contour points of the template eye makeup.

[0086] In the embodiment of the present invention, the template makeup includes template lip makeup and / or template eye makeup. The relevant contents of the template lip makeup have been illustrated above and will not be repeated here. As for the template eye makeup, since there are differences between different template eye makeups, the template eye makeup needs to mark the eye makeup pattern in the front view compared to the template lip makeup. Fig.14 As shown, in Fig.14 The eye makeup pattern is marked in the front view of the template face with eye makeup on the left, and several key contour points of the eye makeup pattern are extracted , such as Fig.14 As shown in the image on the right of . And because the left and right eyes of a person are not exactly the same, it is necessary to mark the eye makeup pattern and extract the key contour points separately.

[0087] like Fig.15 As shown in the figure, unlike the makeup area of ​​the template lip makeup, the inner and outer contours and double eyelids in the makeup area of ​​the template eye makeup can be directly determined based on the face mesh of the template face without manual marking. That is, the face mesh vertex with a specified number can be directly used as the template reference positioning point corresponding to the template eye makeup (such as Fig.15 The yellow dots in the figure), the template reference positioning points of the template face in the front view are recorded as .

[0088] Similar to the template lip makeup, in the embodiment of the present invention, some vertices are selected from the face mesh vertices in the makeup area of ​​the template eye makeup (i.e., the area within the outer contour of the eye) as template sampling points (e.g. Fig.15 The red dot in , construct a template positioning grid for the makeup area of ​​the template eye makeup, recorded as template positioning grid G.

[0089] It should be noted that the eye makeup pattern is composed of multiple elements such as eyeliner and eye bags, and the structure of eyeliner and eye bags is a thin and long line. To transfer the eye makeup pattern from the template face to the target face, the eye makeup pattern needs to be redirected, which will make the display effect of the eye makeup pattern change from smooth to distorted. In order to keep the display effect of the eye makeup pattern unchanged before and after redirection, the embodiment of the present invention sets the template reference positioning point when constructing the template positioning grid G. ( Fig.15 Yellow dots in the figure), template sampling points ( Fig.15 The red dots in the middle) and the key contour points of the eye makeup template ( Fig.15 The blue dots in the figure are used as templates to locate the vertices of the mesh G for subsequent redirection processing.

[0090] In an exemplary embodiment of the present invention, the template positioning grid is redirected based on the Laplace smoothness constraint to obtain the position information of each vertex in the target positioning grid, including: Based on the position information of each vertex in the template positioning grid and the position information of each target reference positioning point in the target positioning grid, as well as the topological structure of the template positioning grid and the target positioning grid, a Laplace smooth constraint is constructed; Based on the position information of the key contour points of the template eye makeup, the eye makeup structure constraints are constructed; The Laplace smoothness constraint and the eye makeup structure constraint are jointly optimized to obtain the position information of each vertex in the target positioning mesh.

[0091] In the embodiment of the present invention, the x template reference positioning points of the template face ( ), y template sampling points ( ) and h key contour points ( ) together form all the vertices of the template positioning mesh G .

[0092] Similar to the template lip makeup, the topological structure of the template positioning grid G ​​based on the makeup area of ​​the template eye makeup can also construct the corresponding Laplacian matrix, which is recorded as , , , , The number of rows and columns of the three matrices is the same, which is the number of all vertices of the template positioning grid G. The description of the Laplace matrix will not be repeated here.

[0093] The vertex coordinate matrix of the template positioning grid G ​​can be expressed as .

[0094] The template positioning grid G ​​is transferred from the template face to the target face. The template positioning grid G ​​needs to be redirected to obtain the target positioning grid adapted to the target face. , target positioning grid All vertices of By x target reference positioning points ( ), y target sampling points ( ) and h target key contour points ( )composition.

[0095] Positioning the grid for the target The vertex coordinate matrix can be expressed as .

[0096] It can be understood that the x target reference positioning points of the target face The 2D coordinate vector in the frontal view is determined when selecting the vertices from the face mesh. All vertices of The two-dimensional coordinate vector of y target sampling points needs to be determined and h target key contour points The 2D coordinate vector in the front view.

[0097] Based on the position information of each vertex in the template positioning grid G ​​and the target positioning grid The position information of each target reference positioning point in the template positioning grid G, that is, the two-dimensional coordinate vector of each vertex in the front view of the template face and the target positioning grid G The two-dimensional coordinate vector of each target reference positioning point in the frontal view of the target face is constructed, and the Laplace smoothing constraint is constructed based on the topological structure of the template positioning grid and the target positioning grid. , the template positioning grid G ​​and the target positioning grid After substituting the two-dimensional coordinate vectors of each vertex into it, the Laplace smooth constraint can be transformed into: .

[0098] At the same time, according to the position information of the key contour points of the template eye makeup, the eye makeup structure constraints are established .

[0099] By performing joint optimization on the Laplace smoothness constraint and the eye makeup structure constraint, y target sampling points can be determined. and h target key contour points Based on the calculated two-dimensional coordinate vector in the front view of the target face, the template positioning grid of the template eye makeup is redirected from the template face to the target face.

[0100] In the embodiment of the present invention, all vertices of the template positioning grid based on the makeup area in the front view of the template face are The two-dimensional coordinate vector of the template face is combined with the face mesh of the template face to determine the vertices in the template positioning mesh The center of gravity coordinates in the face mesh. Then, based on the mapping relationship between the three-dimensional space coordinate system and the two-dimensional texture coordinate system, the vertices in the template positioning mesh are determined. The two-dimensional texture coordinates of .

[0101] Accordingly, all vertices of the target positioning mesh of the makeup area of ​​the target face in the front view obtained based on the above calculation are The two-dimensional coordinate vector of the target face is combined with the face mesh of the target face to determine the vertices The centroid coordinates in the face mesh. Then, based on the mapping relationship between the three-dimensional space coordinate system and the two-dimensional texture coordinate system, the vertices in the target positioning mesh are determined. The two-dimensional texture coordinates of the mesh. Positioning each vertex in the mesh based on the template and the corresponding target location grid vertices The position offset of each vertex in the two-dimensional texture space grid can be calculated based on the two-dimensional texture coordinates. Based on the position offset of each vertex in the two-dimensional texture space grid, the template eye makeup is translated and / or scaled, that is, each vertex in the two-dimensional texture space grid is translated and / or scaled independently, so that the makeup effect can be better maintained.

[0102] like Fig.16 As shown, Fig.16 The display effect diagrams are respectively shown as follows: adding the same template eye makeup to the template face and the target face. Although there are differences in the eye shapes of the template face and the target face, the template eye makeup still maintains a similar display effect on different eyes.

[0103] The redirection method of virtual human makeup provided in the embodiment of the present invention is implemented by a redirection device of virtual human makeup, which can be configured in a 3D virtual human creation platform or a 3D video generation platform, which can be installed in a local computer or in the cloud, and the local computer can be a computer, a tablet, etc., which is not specifically limited here. The target face of the acquired 3D virtual human can be used in various application scenarios, such as 3D video generation, virtual makeup trial, virtual anchor, game character, etc., which is not specifically limited in the embodiment of the present invention.

[0104] The following is a description of the redirection device for virtual human makeup provided by the present invention. The redirection device for virtual human makeup described below and the redirection method for virtual human makeup described above can be referred to each other. It should be noted that the device provided in the following embodiment and the method provided in the above embodiment belong to the same concept, and the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, which will not be repeated here.

[0105] In an exemplary embodiment of the present invention, see Fig.17 , Fig.17 A device for redirecting virtual human makeup according to an exemplary embodiment includes: An acquisition module 1710 is configured to acquire a template face corresponding to a target face; The redirection module 1720 is configured to perform redirection processing on the template makeup of the template face based on the front view of the template face and the target face to obtain the target makeup of the target face.

[0106] In an exemplary embodiment of the present invention, the redirection module 1720 includes: A construction submodule configured to determine makeup areas in the frontal view of the template face and the target face, respectively, and to construct a template positioning grid in the makeup area of ​​the template face; A first redirection processing submodule is configured to perform redirection processing on the template positioning grid to obtain a target positioning grid corresponding to the target face; The second redirection processing submodule is configured to perform redirection processing on the template makeup of the template face based on the position offset of each vertex in the template positioning grid and the target positioning grid.

[0107] In an exemplary embodiment of the present invention, a submodule is constructed, including: A first calibration unit is configured to calibrate at least one template reference positioning point in the makeup area of ​​the template face; the template reference positioning point includes an edge contour point of the makeup area of ​​the template face; The construction unit is configured to construct a template positioning grid based on the template reference positioning points and the face grid vertices of the template face.

[0108] In an exemplary embodiment of the present invention, the first redirection processing submodule includes: A second calibration unit is configured to calibrate at least one target reference positioning point in the makeup area of ​​the target face; the target reference positioning point corresponds to the template reference positioning point one by one; The redirection unit is configured to redirect the template positioning grid based on the Laplace smoothness constraint to obtain the position information of each vertex in the target positioning grid; wherein each vertex in the template positioning grid corresponds to each vertex in the target positioning grid.

[0109] In an exemplary embodiment of the present invention, the second redirection processing submodule includes: A determination unit configured to determine a position offset of each vertex in the template positioning grid and the target positioning grid in the two-dimensional texture space grid; The processing unit is configured to perform translation and / or scaling processing on the template makeup of the template face based on the position offset.

[0110] In an exemplary embodiment of the present invention, the determining unit includes: A first calculation subunit is configured to calculate the centroid coordinates of each vertex in the face mesh based on the position information of each vertex in the template positioning mesh and the target positioning mesh in the front view; The second calculation subunit is configured to calculate the position offset of each vertex in the two-dimensional texture space grid based on the corresponding relationship between the two-dimensional texture space grid and the face grid.

[0111] In an exemplary embodiment of the present invention, the template makeup includes template lip makeup and / or template eye makeup, and the vertices of the template positioning grid corresponding to the template eye makeup include key contour points of the template eye makeup.

[0112] In an exemplary embodiment of the present invention, the redirection unit includes: A first construction subunit is configured to construct a Laplace smooth constraint based on the position information of each vertex in the template positioning grid and the position information of each target reference positioning point in the target positioning grid, as well as the topological structure of the template positioning grid and the target positioning grid; A second construction subunit is configured to construct eye makeup structural constraints based on position information of key contour points of the template eye makeup; The joint optimization subunit is configured to perform joint optimization on the Laplace smoothness constraint and the eye makeup structure constraint to obtain the position information of each vertex in the target positioning grid.

[0113] Fig.18 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.18 As shown, the electronic device may include: a processor 1810, a communications interface 1820, a memory 1830 and a communication bus 1840, wherein the processor 1810, the communications interface 1820 and the memory 1830 communicate with each other via the communication bus 1840. The processor 1810 may call the logic instructions in the memory 1830 to execute the redirection method of the virtual human makeup, the method comprising: obtaining a template face corresponding to the target face; The template makeup of the template face is redirected based on the frontal view of the template face and the target face to obtain the target makeup of the target face.

[0114] In addition, the logic instructions in the above-mentioned memory 1830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0115] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the virtual human makeup redirection method provided by the above methods, the method includes: obtaining a template face corresponding to a target face; The template makeup of the template face is redirected based on the front view of the template face and the target face to obtain the target makeup of the target face.

[0116] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the virtual human makeup redirection method provided by the above methods, the method comprising: obtaining a template face corresponding to a target face; The template makeup of the template face is redirected based on the front view of the template face and the target face to obtain the target makeup of the target face.

[0117] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0118] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for redirecting virtual human makeup, characterized in that: include: Obtain a template face corresponding to the target face; The template makeup of the template face is redirected based on the front view of the template face and the target face to obtain the target makeup of the target face.

2. The method for redirecting virtual human makeup according to claim 1, characterized in that: The redirecting process of the template makeup of the template face based on the front view of the template face and the target face includes: Determining makeup areas in the front views of the template face and the target face respectively, and constructing a template positioning grid in the makeup area of ​​the template face; Redirecting the template positioning grid to obtain a target positioning grid corresponding to the target face; Based on the position offset of each vertex in the template positioning grid and the target positioning grid, the template makeup of the template face is redirected.

3. The method for redirecting virtual human makeup according to claim 2, characterized in that: The step of constructing a template positioning grid in the makeup area of ​​the template face includes: Calibrate at least one template reference positioning point in the makeup area of ​​the template face; the template reference positioning point includes an edge contour point of the makeup area of ​​the template face; The template positioning grid is constructed based on the template reference positioning points and the face mesh vertices of the template face.

4. The method for redirecting virtual human makeup according to claim 3, characterized in that: The redirecting of the template positioning grid to obtain a target positioning grid corresponding to the target face includes: Calibrate at least one target reference positioning point in the makeup area of ​​the target face; the target reference positioning point corresponds to the template reference positioning point one by one; The template positioning grid is redirected based on the Laplace smoothness constraint to obtain the position information of each vertex in the target positioning grid; wherein each vertex in the template positioning grid corresponds to each other one by one.

5. The method for redirecting virtual human makeup according to claim 4, characterized in that: The redirecting of the template makeup of the template face based on the position offset of each vertex in the template positioning grid and the target positioning grid includes: Determine the position offset of each vertex in the template positioning grid and the target positioning grid in the two-dimensional texture space grid; Based on the position offset, the template makeup of the template face is translated and / or scaled.

6. The method for redirecting virtual human makeup according to claim 5, characterized in that: The determining of the position offset of each vertex in the template positioning grid and the target positioning grid in the two-dimensional texture space grid comprises: Based on the position information of each vertex in the template positioning grid and the target positioning grid in the front view, calculating the centroid coordinates of each vertex in the face grid; Based on the correspondence between the two-dimensional texture space grid and the face grid, the position offset of each vertex in the two-dimensional texture space grid is calculated.

7. The method for redirecting virtual human makeup according to any one of claims 3 to 6, characterized in that: The template makeup includes template lip makeup and / or template eye makeup, and the vertices of the template positioning grid corresponding to the template eye makeup include key contour points of the template eye makeup.

8. The method for redirecting virtual human makeup according to claim 7, characterized in that: The redirecting process of the template positioning grid based on the Laplace smoothness constraint to obtain the position information of each vertex in the target positioning grid includes: Constructing the Laplace smooth constraint based on the position information of each vertex in the template positioning grid and the position information of each target reference positioning point in the target positioning grid, as well as the topological structure of the template positioning grid and the target positioning grid; constructing eye makeup structure constraints based on position information of key contour points of the template eye makeup; The Laplace smoothness constraint and the eye makeup structure constraint are jointly optimized to obtain position information of each vertex in the target positioning grid.

9. A virtual human makeup redirection device, characterized in that: include: An acquisition module configured to acquire a template face corresponding to a target face; The redirection module is configured to redirect the template makeup of the template face based on the front view of the template face and the target face to obtain the target makeup of the target face.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for redirecting the virtual human makeup as claimed in any one of claims 1 to 8 is implemented.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for redirecting virtual human makeup as claimed in any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for redirecting virtual human makeup as claimed in any one of claims 1 to 8 is implemented.

Images