A method for automatically generating a 3D face model with dynamic facial expressions

Abstract

The present invention provides a method for automatically generating a three-dimensional face model with dynamic facial expressions, comprising the following steps: making a standard three-dimensional face model with dynamic facial expressions, recording three-dimensional vertices P in the standard three-dimensional face model, m , and mark the first key point O in the standard 3D face model m ; Use the structured light 3D scanning system to obtain a static 3D face model and record the 3D vertices P in the static 3D face model n , and mark the second key point in the static three-dimensional face model; by comparing the second key point in the static three-dimensional face model with the first key point O in the standard three-dimensional face model m Corresponding to the overlap, the three-dimensional vertices P in the standard face model are transformed using the principal component analysis method. m Match the three-dimensional vertex P of the static three-dimensional face model n The method for automatically generating a three-dimensional face model with dynamic facial expressions provided by the present invention is easy to use and has high production efficiency.

Description

[Technical field]

[0001] The present invention relates to the technical field of computer vision, and in particular to a method for automatically generating a three-dimensional face model with dynamic expression movements. [Background technology]

[0002] With the development of computer graphics technology, 3D face modeling and 3D face models with facial expressions and movements have become a research hotspot in the field of computer graphics. 3D face modeling and 3D face models with facial expressions and movements have gradually been promoted and applied to many fields such as virtual reality, film and television production, face recognition, and game entertainment, and have great application value.

[0003] At present, the method for generating a 3D face model with facial expressions is as follows: a static face 3D model is obtained by a structured light 3D imaging device (specifically refer to CN106164979A), and then the static face model is manually bound to actions using animation software (such as Maya software from Autodesk, USA) to make it a 3D face with facial expressions. However, action binding using 3D animation software is only suitable for the production of 3D faces with single facial expressions; in the production of 3D faces with large quantities of facial expressions, such as personalized facial 3D expression production (specifically, the same static 3D model is bound to multiple sets of facial expressions), character switching and updating in personalized 3D games (specifically, different static models are bound to the same facial expressions), etc., there are quite a lot of repeated operations when using animation software to perform action binding, which takes a long time to produce and is inefficient.

[0004] In view of this, it is necessary to provide a new method for automatically generating a three-dimensional face model with dynamic facial expressions to overcome the above-mentioned defects. [Summary of the invention]

[0005] The purpose of the present invention is to provide a method for automatically binding expressions of a three-dimensional face model which is easy to use and has high production efficiency.

[0006] In order to achieve the above object, the present invention provides a method for automatically generating a three-dimensional face model with dynamic facial expressions, comprising the following steps:

[0007] S1, making a standard 3D face model with action expressions, recording the 3D vertices P in the standard 3D face model m , and mark the first key point O in the standard 3D face model m ;

[0008] S2, using the structured light 3D scanning system to obtain a static 3D face model, and record the 3D vertices P in the static 3D face model n , and mark the second key point in the static three-dimensional face model;

[0009] S3, by comparing the second key point in the static three-dimensional face model with the first key point O in the standard three-dimensional face model. m Corresponding to the overlap, the three-dimensional vertices P in the standard face model are transformed using the principal component analysis method. m Match the three-dimensional vertex P of the static three-dimensional face model n In the process, the static 3D face model and the standard 3D face model are fused.

[0010] In a preferred embodiment, the nose tip point of the standard three-dimensional face model is the first key point. m The nose tip point of the static three-dimensional face model is the second key point O n .

[0011] In a preferred embodiment, step S3 is performed by comparing the second key point in the static three-dimensional face model with the first key point in the standard three-dimensional face model. m The corresponding coincidence includes the following steps:

[0012] S31, respectively taking the first key point O of the standard 3D face model m is the center point and the second key point O of the static 3D face model n As the center point, take the three-dimensional vertex P of the standard three-dimensional face model and the static three-dimensional face model respectively. m , P n ;

[0013] S32, through the principal component analysis system, the three-dimensional vertices P of the standard three-dimensional face model and the static three-dimensional face model are respectively m , P n Reduced to two-dimensional vertex data; in two-dimensional distribution, the standard system of standard three-dimensional face model is O m -X m Y m Z m , the standard system of static 3D face model is O n -X n Y n Z n

[0014] S33, then the standard system of the static three-dimensional face model is set to O n -X n Y n Z n Through the rotation matrix R and the transfer matrix T transformation, O n -X n Y n Z n The standard system with the standard 3D face model is O m -X m Ym Z m coincide.

[0015] In a preferred embodiment, in the two-dimensional distribution described in step S32, the standard system of the standard three-dimensional face model is m -X m Y m Z m , the standard system of static 3D face model is O n -X n Y n Z n The specific process is:

[0016] The standard 3D face model is a standard 3D face model with the first key point O in the front view. m The symmetry line is Y m Axis, through the corresponding first key point O m And vertical Y m The straight line of the axis is X m Axis, respectively vertical Y m Axis, X m The straight line of the axis is Z m Axis, the standard system for establishing a standard three-dimensional face model is O m -X m Y m Z m ;

[0017] The static three-dimensional face model is a static three-dimensional face model with a second key point O in the front view. n The symmetry line is Y n Axis, through the second key point O n And vertical Y n The straight line of the axis is X n Axis, respectively vertical Y n Axis, X n The straight line of the axis is Z n Axis, the standard system for establishing a standard three-dimensional face model is O n -X n Y n Z n .

[0018] In a preferred embodiment, the principal component analysis method is used to transform the three-dimensional vertices P in the standard face model into m Match the three-dimensional vertex P of the static three-dimensional face model n The following steps are included:

[0019] S34, static 3D face model with the first key point O m is the center of the circle, r mCutting within the sphere with a radius of the sphere, cutting the static 3D face model into a valid area that can cover the main features of the face;

[0020] S35, each 3D vertex P of the standard 3D face model m Along the corresponding normal direction, search for the nearest 3D vertex P of the static 3D face model in the effective area corresponding to the static 3D face model that covers the main features of the face. n ;

[0021] S36, the three-dimensional vertices P of the static three-dimensional face model are n At the three-dimensional vertex P m The projection along the corresponding normal direction is taken as the corresponding point in the fused 3D face model.

[0022] In a preferred embodiment, the standard 3D face model includes a color 3D model file and an action file; the color 3D model file includes the 3D vertices P in the standard face model. m The action file includes the three-dimensional vertex P in the standard face model. m The binding relationship with the action and the direction of movement of the action.

[0023] In a preferred embodiment, the step S2 of acquiring a static 3D face model using a structured light 3D scanning system comprises the following steps:

[0024] S21, locating two-dimensional vertices in the static three-dimensional face model image;

[0025] S22, locating the three-dimensional vertex P corresponding to the two-dimensional vertex in step S21 in the static three-dimensional face model image n ;

[0026] S23, using the three-dimensional vertex P in S2 n Weighted calculation of shape reconstruction coefficients of a static three-dimensional face model and imaging model parameters;

[0027] S24, synthesizing the three-dimensional face shape by components using the reconstruction coefficient and the three-dimensional deformation model and performing fusion;

[0028] S25, extracting 3D facial texture using imaging model and facial symmetry.

[0029] The method for automatically generating a three-dimensional face model with dynamic facial expressions provided by the present invention is to automatically generate a three-dimensional face model with dynamic facial expressions by converting the three-dimensional vertices P in the standard three-dimensional face model into a m and the three-dimensional vertex P in the static three-dimensional face model n One-to-one correspondence fusion makes the three-dimensional vertex P in the static three-dimensional face model nInherit the 3D vertex P in the standard 3D face model m The color 3D model file and action file are then automatically generated to form a 3D face model with dynamic expressions. The method for automatically generating a 3D face model with dynamic expressions and actions provided by the invention is easy to use and has high production efficiency.

Brief Description of the Drawings

[0030] Figure 1 A schematic diagram of the method for automatically generating a three-dimensional face model with dynamic facial expressions provided by the present invention.

[0031] Figure 2 A schematic diagram of a standard three-dimensional face model provided by the present invention.

[0032] Figure 3 A schematic diagram of a static three-dimensional face model provided by the present invention.

[0033] Figure 4 This is a schematic diagram of the fusion of the standard 3D face model and the static 3D face model provided by the present invention.

[0034] Figure 5 for Figure 4 The schematic diagram of the fusion of the standard 3D face model and the static 3D face model is shown. [Specific implementation method]

[0035] In order to make the purpose, technical solution and beneficial technical effect of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be understood that the specific implementation methods described in this specification are only for explaining the present invention, not for limiting the present invention.

[0036] Please refer to Figures 1 to 4 The present invention provides a method for automatically generating a three-dimensional face model with dynamic facial expressions, comprising the following steps:

[0037] S1, making a standard 3D face model with action expressions, recording the 3D vertices P in the standard 3D face model m , and mark the first key point O in the standard 3D face model m ;

[0038] Specifically, in step S1, the nose tip point of the standard three-dimensional face model is the first key point O m .

[0039] The standard 3D face model includes a color 3D model file and an action file; the color 3D model file includes the 3D vertex P in the standard face model. m The action file includes the three-dimensional vertex P in the standard face model.m The binding relationship with the action and the direction of movement of the action.

[0040] S2, using the structured light 3D scanning system to obtain a static 3D face model, and record the 3D vertices P in the static 3D face model n , and mark the second key point O in the static 3D face model n ;

[0041] Specifically, in step S2, the nose tip point of the static three-dimensional face model is the second key point O n .

[0042] The method of obtaining a static three-dimensional face model by using a structured light three-dimensional scanning system in step S2 includes the following steps:

[0043] S21, locating two-dimensional vertices in the static three-dimensional face model image;

[0044] S22, locating the three-dimensional vertex P corresponding to the two-dimensional vertex in step S21 in the static three-dimensional face model image n ;

[0045] S23, using the three-dimensional vertex P in S2 n Weighted calculation of shape reconstruction coefficients of a static three-dimensional face model and imaging model parameters;

[0046] S24, synthesizing the three-dimensional face shape by components using the reconstruction coefficient and the three-dimensional deformation model and performing fusion;

[0047] S25, extracting 3D facial texture using imaging model and facial symmetry.

[0048] S3, by passing the second key point O in the static 3D face model n Compared with the first key point O in the standard 3D face model m Corresponding to the overlap, the three-dimensional vertices P in the standard face model are transformed using the principal component analysis method. m Match the three-dimensional vertex P of the static three-dimensional face model n In the process, the static 3D face model and the standard 3D face model are fused.

[0049] Specifically, in step S3, the second key point O in the static three-dimensional face model is n Compared with the first key point O in the standard 3D face model m The corresponding coincidence includes the following steps:

[0050] S31, respectively taking the first key point O of the standard 3D face model m is the center point and the second key point O of the static 3D face model nAs the center point, take the three-dimensional vertex P of the standard three-dimensional face model and the static three-dimensional face model respectively. m , P n ;

[0051] S32, through the principal component analysis system, the three-dimensional vertices P of the standard three-dimensional face model and the static three-dimensional face model are respectively m , P n Reduced to two-dimensional vertex data; in two-dimensional distribution, the standard system of standard three-dimensional face model is O m -X m Y m Z m , the standard system of static 3D face model is O n -X n Y n Z n

[0052] S33, then the standard system of the static three-dimensional face model is set to O n -X n Y n Z n Through the rotation matrix R and the transfer matrix T transformation, O n -X n Y n Z n The standard system with the standard 3D face model is O m -X m Y m Z m coincide.

[0053] Furthermore, in the two-dimensional distribution described in step S32, the standard system of the standard three-dimensional face model is 0 m -X m Y m Z m , the standard system of static 3D face model is O n -X n Y n Z n The specific process is:

[0054] The standard 3D face model is a standard 3D face model with the first key point O in the front view. m The symmetry line is Y m Axis, through the corresponding first key point O m And vertical Y m The straight line of the axis is X m Axis, respectively vertical Y m Axis, X m The straight line of the axis is Z m Axis, the standard system for establishing a standard three-dimensional face model is O m -X mY m Z m ;

[0055] The static three-dimensional face model is a static three-dimensional face model with a second key point O in the front view. n The symmetry line is Y n Axis, through the second key point O n And vertical Y n The straight line of the axis is X n Axis, respectively vertical Y n Axis, X n The straight line of the axis is Z n Axis, the standard system for establishing a standard three-dimensional face model is O n -X n Y n Z n .

[0056] Please refer to Figure 5 In step S3, the three-dimensional vertices P in the standard face model are analyzed by principal component analysis. m Match the three-dimensional vertex P of the static three-dimensional face model n The following steps are included:

[0057] S34, static 3D face model with the first key point O m is the center of the circle, r m Cutting within the sphere with a radius of the sphere, cutting the static 3D face model into multiple effective areas that can cover the main features of the face;

[0058] S35, each 3D vertex P of the standard 3D face model m Along the corresponding normal direction, search for the nearest 3D vertex P of the static 3D face model in the effective area corresponding to the static 3D face model that covers the main features of the face. n ;

[0059] S36, the three-dimensional vertices P of the static three-dimensional face model are n At the three-dimensional vertex P m The projection along the corresponding normal direction is taken as the corresponding point in the fused 3D face model.

[0060] Specifically, the three-dimensional vertex P of the standard three-dimensional face model m Including P m1 , P m2 , P m3 etc., the three-dimensional vertices P of the static three-dimensional face model n Including P n1 , P n2 , P n3 etc., the three-dimensional vertices P of the standard three-dimensional face model m1 , P m2 , Pm3 etc. respectively with the three-dimensional vertices P of the static three-dimensional face model n1 , P n2 , P n3 Etc., one-to-one correspondence, thereby forming the corresponding point P in the fused 3D face model x1 , P x2 , P x3 The corresponding point P in the fused 3D face model x1 Inherit P m1 Color 3D model files and action files, while inheriting P n1 The color texture and three-dimensional morphology of the face can be automatically generated with the dynamic expression of the three-dimensional face model.

[0061] The method for automatically generating a three-dimensional face model with dynamic facial expressions provided by the present invention is to automatically generate a three-dimensional face model with dynamic facial expressions by converting the three-dimensional vertices P in the standard three-dimensional face model into a m and the three-dimensional vertex P in the static three-dimensional face model n One-to-one correspondence fusion makes the three-dimensional vertex P in the static three-dimensional face model n Inherit the 3D vertex P in the standard 3D face model m The color 3D model file and action file are then automatically generated to form a 3D face model with dynamic expressions. The method for automatically generating a 3D face model with dynamic expressions and actions provided by the invention is easy to use and has high production efficiency.

[0062] The present invention is not limited to what is described in the specification and implementation modes, and therefore additional advantages and modifications can be easily realized by those skilled in the art. Therefore, without departing from the spirit and scope of the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details, representative devices, and illustrative examples shown and described herein.

Claims

1. A method for automatically generating a three-dimensional face model with dynamic facial expressions, characterized in that: The following steps are involved: S1, making a standard 3D face model with action expressions, recording the 3D vertices P in the standard 3D face model m , and mark the first key point O in the standard 3D face model m ; S2, using the structured light 3D scanning system to obtain a static 3D face model, and record the 3D vertices P in the static 3D face model n , and mark the second key point in the static three-dimensional face model; S3, by comparing the second key point in the static three-dimensional face model with the first key point O in the standard three-dimensional face model. m Corresponding to the overlap, the three-dimensional vertices P in the standard face model are transformed using the principal component analysis method. m Match the three-dimensional vertex P of the static three-dimensional face model n In the process of merging the static 3D face model with the standard 3D face model, The nose tip point of the standard three-dimensional face model is the first key point O m The nose tip point of the static three-dimensional face model is the second key point O n ; In step S3, the second key point in the static three-dimensional face model is compared with the first key point in the standard three-dimensional face model. m The corresponding coincidence includes the following steps: S31, respectively taking the first key point O of the standard 3D face model m is the center point and the second key point O of the static 3D face model n As the center point, take the three-dimensional vertex P of the standard three-dimensional face model and the static three-dimensional face model respectively. m , P n ; S32, through the principal component analysis system, the three-dimensional vertices P of the standard three-dimensional face model and the static three-dimensional face model are respectively m , P n Reduced to two-dimensional vertex data; in two-dimensional distribution, the standard system of standard three-dimensional face model is O m -X m Y m Z m , the standard system of static 3D face model is O n -X n Y n Z n ; S33, then the standard system of the static three-dimensional face model is set to O n -X n Y n Z n Through the rotation matrix R and the transfer matrix T transformation, O n -X n Y n Z n The standard system with the standard 3D face model is O m -X m Y m Z m coincide; In the two-dimensional distribution described in step S32, the standard system of the standard three-dimensional face model is 0 m -X m Y m Z m , the standard system of static 3D face model is O n -X n Y n Z n Specifically: The standard 3D face model is a standard 3D face model with the first key point O in the front view. m The symmetry line is Y m Axis, through the corresponding first key point O m And vertical Y m The straight line of the axis is X m Axis, respectively vertical Y m Axis, X m The straight line of the axis is Z m Axis, the standard system for establishing a standard three-dimensional face model is O m -X m Y m Z m ; The static three-dimensional face model is a static three-dimensional face model with a second key point O in the front view. n The symmetry line is Y n Axis, through the second key point O n And vertical Y n The straight line of the axis is X n Axis, respectively vertical Y n Axis, X n The straight line of the axis is Z n Axis, the standard system for establishing a standard three-dimensional face model is O n -X n Y n Z n ; In step S3, the principal component analysis method is used to transform the three-dimensional vertices P in the standard face model into m Match the three-dimensional vertex P of the static three-dimensional face model n The following steps are included: S34, static 3D face model with the first key point O m is the center of the circle, r m Cutting within the sphere with a radius of the sphere, cutting the static 3D face model into a valid area that can cover the main features of the face; S35, each 3D vertex P of the standard 3D face model m Along the corresponding normal direction, search for the nearest 3D vertex P of the static 3D face model in the effective area corresponding to the static 3D face model that covers the main features of the face. n ; S36, the three-dimensional vertices P of the static three-dimensional face model are n At the three-dimensional vertex P m The projection along the corresponding normal direction is taken as the corresponding point in the fused 3D face model; The method of obtaining a static three-dimensional face model by using a structured light three-dimensional scanning system in step S2 includes the following steps: S21, locating two-dimensional vertices in the static three-dimensional face model image; S22, locating the three-dimensional vertex P corresponding to the two-dimensional vertex in step S21 in the static three-dimensional face model image n ; S23, using the three-dimensional vertex P in S2 n Weighted calculation of shape reconstruction coefficients of a static three-dimensional face model and imaging model parameters; S24, synthesizing the three-dimensional face shape by components using the reconstruction coefficient and the three-dimensional deformation model and performing fusion; S25, extracting 3D facial texture using imaging model and facial symmetry.

2. The method for automatically generating a three-dimensional face model with dynamic facial expressions as claimed in claim 1, characterized in that: The standard 3D face model includes a color 3D model file and an action file; the color 3D model file includes the 3D vertex P in the standard face model. m The action file includes the three-dimensional vertex P in the standard face model. m The binding relationship with the action and the direction of movement of the action.

Images