An automatic base expression selection method based on local fusion optimization

Abstract

This invention relates to an automatic base expression selection method based on local hybrid optimization, belonging to the field of computer and automation technology. The method includes: acquiring a sequence of facial expression models with the same topology; segmenting each facial expression model into multiple sub-regions; fitting each sub-region of the base expression in the base expression group to the corresponding sub-region of each target expression model; obtaining a local facial expression model corresponding to each target expression model; fusing the obtained local facial expression models to obtain a complete facial expression model corresponding to each target expression model; comparing each complete facial expression model with its corresponding target expression model, identifying the target expression model with the largest error compared to the corresponding complete facial expression model, and adding this target expression model as the base expression to the base expression group; obtaining the base expression group when the rate of change of the largest error is less than 0.1.

Description

Technical Field

[0001] This invention relates to the fields of computer and automation technology, and in particular to a method for automatic selection of base expressions based on local hybrid optimization. Background Technology

[0002] In the creation of virtual characters for film and television, realistic facial expressions play a crucial role in conveying the character's emotions and language. The commonly used methods for capturing facial expressions involve facial capture devices and hybrid deformation models. To achieve good hybrid deformation effects, it is necessary to create 3D models of as many of the actor's expressions as possible. Currently, this often involves scanning a continuous sequence of facial expressions from the actor using facial capture devices, and then art engineers manually selecting representative base expressions from thousands of 3D facial models based on their experience. This step is too manual and time-consuming; therefore, it is necessary to invent a method for automatically selecting base expressions. Summary of the Invention

[0003] To address the aforementioned technical issues, the base expression selection algorithm based on local fusion optimization proposed in this invention can automatically select base expressions from thousands of facial expression model sequences to replace tedious manual operations. Furthermore, based on the local fusion deformation model method, the entire facial expression model is divided into multiple local regions, each of which can undergo individual fusion deformation, providing more degrees of freedom. This makes the expression base selected by the algorithm more orthogonal and enhances its ability to fit other different facial expressions.

[0004] The purpose of this invention is to provide a method for automatically selecting base expressions based on local blending optimization, which includes the following steps:

[0005] Step 1: Obtain a sequence of facial expression models with the same topology; perform region segmentation on each facial expression model in the sequence to obtain multiple sub-regions; select a neutral expression model from the sequence as the initial base expression in the base expression group; and use the remaining unselected facial expression models in the sequence as the target expression model sequence.

[0006] Step 2: Use each sub-region obtained by region segmentation of the base expressions in the base expression group to fit the corresponding sub-region of each target expression model in the target expression model sequence; obtain the local facial expression model corresponding to each target expression model;

[0007] Step 3: Fuse the local facial expression models corresponding to each target expression model obtained in Step 2 to obtain the complete facial expression model corresponding to each target expression model;

[0008] Step 4: Compare each complete facial expression model obtained in Step 3 with its corresponding target facial expression model, find the target facial expression model with the largest error compared with the corresponding complete facial expression model, add the target facial expression model as the base expression to the base expression group, and delete the target facial expression model from the target facial expression model sequence;

[0009] Step 5: Repeat steps 2 to 4 until the rate of change of the maximum error mentioned in step 4 is less than 0.1, and obtain the base expression set.

[0010] Furthermore, the automatic base expression selection method based on local fusion optimization of the present invention also includes:

[0011] Obtaining a sequence of facial expression models with the same topology involves the following steps:

[0012] Step 1.1: Perform continuous frame facial expression scanning and reconstruct a three-dimensional facial expression model sequence based on the scanning results;

[0013] Step 1.2: Using 3D model topology software, retopologically reconstruct each facial expression model in the reconstructed facial expression model sequence using a facial template with an ideal topological structure to obtain a facial expression model sequence with the same topology.

[0014] Furthermore, the automatic base expression selection method based on local fusion optimization of the present invention also includes:

[0015] Region segmentation for each facial expression model in the facial expression model sequence includes the following steps:

[0016] Step 1.3: Create a facial segmentation mask in texture space according to the distribution area of ​​facial muscles. Obtain the coordinates of each vertex of the three-dimensional facial expression model in texture space through texture unrolling, and divide the coordinates into multiple sub-regions of the mask.

[0017] Step 1.4: Extend the range of each sub-region outward by a portion, so that adjacent sub-regions have 20% overlapping vertices.

[0018] Furthermore, the automatic base expression selection method based on local fusion optimization of the present invention also includes:

[0019] Step two above includes the following steps:

[0020] Step 2.1: Calculate the shape of the i-th sub-region using the following formula.

[0021]

[0022] Among them, P i =[R i |Ti Let be the pose matrix of the i-th sub-region, including the rotation matrix R. i Translation matrix T i u i This represents the shape of the i-th sub-region in the neutral expression model. This represents the weight value of the k-th hybrid deformation in the i-th sub-region. This refers to the change of the k-th hybrid deformation of the i-th sub-region relative to the neutral expression model, where K is the number of base expressions in the base expression group;

[0023] Step 2.2: Establish the objective function, using the hybrid deformation parameters and spatial pose of each sub-region as unknowns, and the 3D vertices of the target facial expression model as input. The formula is as follows:

[0024] E = E M +E O (2)

[0025] Wherein, the objective function E is derived from E M and E O Composition, E M This represents the constraints on the shape and spatial pose of the sub-regions, used to determine the blending deformation parameters of each sub-region in the facial expression model and the spatial position of each sub-region, expressed by the following formula.

[0026]

[0027] Where V represents all sub-regions participating in the local hybrid deformation, and Q i λ represents all vertices contained in the i-th extended sub-region. M These are the weight values ​​for the shape constraints;

[0028] E O It is a constraint on the overlapping parts between adjacent sub-regions, expressed by the following formula.

[0029] E O =λ O ∑ q∈R ∑ (i，j)∈V ||x q，i -x q，j || 2 (4)

[0030] Where q∈R represents a vertex within an adjacent overlapping region, x q，i Let x be the coordinates of vertex q in the overlapping part of the i-th sub-region after deformation. q，j Let λ be the coordinates of vertex q in the overlapping portion of the j-th sub-region after deformation. O These are the weight values ​​for overlapping constraints;

[0031] Step 2.3: Solve for the optimal value so that the objective function E = E M +E O The minimum value is used to obtain the hybrid deformation coefficient for each sub-region. And spatial pose Ri and Ti.

[0032] Step 2.4: Calculate the three-dimensional coordinates x of the vertices in each sub-region after fitting using the above formula (1). i This allows the construction of a fitted local facial expression model;

[0033] Step 2.5: Repeat steps 2.2 to 2.4 until each target expression model in the target expression model sequence has been fitted, and a local facial expression model corresponding to each target expression model is obtained.

[0034] Furthermore, the automatic base expression selection method based on local fusion optimization of the present invention also includes:

[0035] Step three includes:

[0036] Step 3.1: Based on the distance of the vertices of the overlapping portion of each sub-region from the center of the sub-region, calculate the fusion weight of each vertex according to the following formula.

[0037]

[0038] Among them, w q，i μ represents the fusion weight of vertex q in the overlapping part of the i-th sub-region. q，i Let σ be the Euclidean distance between the vertex q of the overlapping region after deformation and the center point of the i-th sub-region. s The standard deviation of a Gaussian kernel is empirically set to 1.6 times the average width of a sub-region;

[0039] Step 3.2: Normalize the fusion weights of the vertices based on the sub-regions to which the overlapping parts belong, using the following formula:

[0040]

[0041] in, This represents the fusion weight of vertex q in the overlapping part of the i-th sub-region after normalization, where i∈I represents the range of sub-regions involved in the overlapping region;

[0042] Step 3.3: Update the positions of vertices belonging to the overlapping region, using the following formula:

[0043]

[0044] in, Let x represent the updated 3D coordinates of vertex q. q,iThis represents the coordinates of vertex q in the overlapping part of the i-th sub-region before the update, and the positions of all vertices of the complete facial expression model after the update.

[0045] Step 3.4: Repeat steps 3.1 to 3.3 until the local facial expression model corresponding to each target expression model is fused to obtain the complete facial expression model corresponding to each target expression model.

[0046] Furthermore, the automatic base expression selection method based on local fusion optimization of the present invention also includes:

[0047] Step four above includes:

[0048] Step 4.1: Use the nearest point iteration algorithm to register the point cloud of the complete facial expression model obtained in Step 3 with its corresponding target facial expression model to make the point cloud distribution position consistent;

[0049] Step 4.2: Calculate the error. The average value of the Euclidean distance between each target expression model and the corresponding complete facial expression model is taken as the error of the target expression model.

[0050] Step 4.3: Calculate the error for each target expression model in the target expression model sequence, add the target expression model corresponding to the maximum error to the base expression group, and delete the target expression model from the target expression model sequence.

[0051] Compared with existing technologies, the local hybrid deformation method used in this invention provides more degrees of freedom than global hybrid deformation, making it more flexible, expressive, and advantageous in uncovering hidden data.

[0052] Furthermore, this invention improves upon the method for facial region segmentation. Existing methods merely divide the facial region into multiple sub-regions by averaging the horizontal and vertical dimensions in texture space, without considering the movement patterns of muscles in different facial areas. This invention segments the facial region based on the medical distribution patterns of facial muscles. This segmentation method allows for more realistic and natural facial expressions fitted by localized blending deformations.

[0053] This invention uses an iterative approach to automatically select a set of base expressions from a sequence of facial expression models to fit the unselected facial expression models, thereby achieving the effect of covering the entire facial expression space with sub-facial expressions. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of the automatic selection process of basic facial expressions based on local fusion optimization provided by the present invention;

[0055] Figure 2This is a schematic diagram of facial region segmentation based on the distribution pattern of facial muscles provided by the present invention. Detailed Implementation

[0056] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a further detailed explanation of the automatic base expression selection method based on local hybridization optimization proposed in this invention. The advantages and features of this invention will become clearer from the following description and claims. It should be noted that the drawings are all in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of this invention.

[0057] This invention provides a method for automatic selection of base expressions based on local fusion optimization, the process of which is as follows: Figure 1 As shown, it includes the following steps:

[0058] Step 1: Obtain a sequence of facial expression models with the same topology; perform region segmentation on each facial expression model in the sequence to obtain multiple sub-regions; select a neutral expression model from the sequence as the initial base expression in the base expression group; and use the remaining unselected facial expression models in the sequence as the target expression model sequence.

[0059] In this step, a three-dimensional facial expression model sequence can be reconstructed by scanning facial expressions frame by frame, such as an actor's face. A multi-view 3D reconstruction method is used, acquiring images of the face from different angles using a multi-view camera array, then extracting and aligning facial feature points to obtain the final facial expression model. Then, using 3D model topology software, the reconstructed facial expression models are retopologically restructured using facial templates with ideal topological structures, ensuring that all facial expression models have a unified ideal topological structure, facilitating subsequent model segmentation and comparison. From the obtained series of facial expression model sequences with the same topology, a neutral expression model is selected as the initial base expression and added to the base expression group. The remaining unselected facial expression models are used as the target expression model sequence for target fitting.

[0060] Step 2: Use each sub-region obtained by region segmentation of the base expressions in the base expression group to fit the corresponding sub-region of each target expression model in the target expression model sequence; obtain the local facial expression model corresponding to each target expression model;

[0061] When choosing base expressions, people generally want the selected set of base expressions to encompass all facial deformation states. In other words, using a smaller set of base expressions (e.g., K) better represents all facial expressions (N) except for neutral expressions. Global blending deformation often requires selecting a large number of base expressions to achieve this. Local blending deformation, on the other hand, allows for independent blending deformation of each sub-region, providing more degrees of freedom and thus greater flexibility.

[0062] Specifically, the facial area can be divided first. For example... Figure 2 As shown, a facial segmentation mask in texture space is created according to the distribution area of ​​facial muscles. The coordinates of the vertices (set to 50,000) of the 3D facial model in texture space can be obtained through texture unwrapping (UV). These coordinates are then divided into V=13 regions of the mask. When performing facial region segmentation, the range of each facial muscle sub-region is defined to be extended outward, so that there are about 20% overlap vertices between adjacent facial muscle sub-regions. This overlap plays an important role in merging adjacent facial muscle sub-regions and maintaining a smooth transition of the sub-region shape.

[0063] In the case of local blending deformation, the facial expression model is divided into multiple sub-regions, and the original blending deformation formula will no longer be applicable. Assuming there are K base expressions in the base expression group, each sub-region has K shapes, which can be individually blended and deformed. The shape of the i-th sub-region can be obtained by formula (1):

[0064]

[0065] Among them, P i =[R i |T i Let be the pose matrix of the i-th sub-region, including the rotation matrix R. i Translation matrix T i u i This represents the shape of the i-th sub-region in the neutral expression. This represents the weight value of the k-th hybrid deformation in the i-th sub-region. This refers to the change in the k-th hybrid deformation of the i-th sub-region relative to the neutral expression. If Let k represent the shape variable of the i-th sub-region.

[0066] The model segmentation and fitting methods used in this invention are both based on facial expression models with the same topological structure. Therefore, the same segmentation method can be used to uniformly divide all facial expression models, and the corresponding facial regions also have similar motion patterns. Furthermore, this invention employs a texture space-based facial region segmentation method; different facial sub-region segmentation methods can be achieved simply by modifying the mask.

[0067] In this step, the remaining target expression models in the target expression model sequence after excluding the base expression group are fitted sequentially. Let the number of target expression models be n, then the initial value of n is 1000. Fitting is performed on the target expression models in the target expression model sequence to obtain the local facial expression model corresponding to each target expression model. The above fitting process requires knowledge of the blending deformation parameters of each sub-region and the pose of the sub-region in 3D space, which can be obtained through the following steps:

[0068] (1) Establish the objective function, taking the hybrid deformation parameters and spatial pose of each sub-region as unknowns, and the three-dimensional vertices of the target expression model as input. The formula is as follows:

[0069] E = E M +E O (2)

[0070] Wherein, the objective function E is derived from E M and E O Composition, E M This represents the constraints on the shape and spatial pose of the sub-regions, used to determine the blending deformation parameters of each sub-region in the face model and the position of the sub-region in space; E O It is a constraint on the overlapping parts between adjacent sub-regions.

[0071] In local blending deformation, each sub-region can undergo independent blending deformation, and the shape of each sub-region is x. i It can be obtained from formula (1), assuming x q，i Let p be the 3D coordinates of the q-th vertex in the i-th sub-region after deformation, and p be the 3D coordinates of the q-th vertex. q，i Let be the coordinates of the three-dimensional vertices of the corresponding input target facial expression model.

[0072]

[0073] Where V represents all sub-regions participating in the local hybrid deformation, and Q i λ represents all vertices contained in the i-th extended sub-region. M Let λ be the weight value of the shape constraint, based on empirical values. M =1.

[0074] Sub-region overlap constraints are used to limit the position and shape of adjacent sub-regions, thereby ensuring the coherence and integrity of facial expressions.

[0075] E O =λ O ∑ q∈R ∑ (i，j)∈V ||x q，i -x q，j || 2 (4)

[0076] Where q∈R represents a vertex within an adjacent overlapping region, x q，i Let x be the coordinates of vertex q in the overlapping part of the i-th sub-region after deformation. q，j Let E be the coordinates of vertex q in the overlapping part of the j-th sub-region after deformation. O The constraint aims to make the vertices of the overlapping portion between the i-th sub-region and the adjacent j-th sub-region as close as possible to each other. λ O Let λ be the weight value of the overlapping constraint, based on empirical values. O =0.6.

[0077] (2) Solve for the optimal value so that the objective function E = E M +E O The minimum value is used to obtain the hybrid deformation coefficient of each sub-region. and pose R i and T i .

[0078] (3) The three-dimensional coordinates x of the vertices in each sub-region after fitting can be calculated from equation (1). i This allows us to construct a fitted local facial expression model.

[0079] (4) Steps (1) to (3) above are the fitting process for one target expression model in the target expression model sequence. They need to be repeated n times until all target expression models are fitted.

[0080] Step 3: Fuse the local facial expression models corresponding to each target expression model obtained in Step 2 to obtain the complete facial expression model corresponding to each target expression model;

[0081] Each model fitted in step two is a local facial expression model after dividing into sub-regions. There are gaps between the sub-regions, and it is necessary to merge all the sub-regions into a complete facial expression model. Based on the distance of the vertex in each overlapping part from the center of the sub-region, the fusion weight of each vertex is calculated according to equation (5).

[0082]

[0083] Among them, wq，i μ represents the fusion weight of vertex q in the overlapping part of the i-th sub-region. q，i Let σ be the Euclidean distance between vertex q of the overlapping portion after deformation and the center point of the i-th sub-region. s The standard deviation of a Gaussian kernel is empirically set to 1.6 times the average width of a sub-region. Then, the fusion weight of vertices farther from the center of the sub-region is smaller.

[0084] (2) The vertex fusion weights are normalized according to the sub-regions to which the overlapping parts belong, as shown in the following formula:

[0085]

[0086] in, This represents the fusion weight of vertex q in the overlapping part of the i-th sub-region after normalization, where i∈I represents the range of sub-regions involved in the overlapping region.

[0087] (3) Update the positions of vertices belonging to the overlapping region, using the following formula:

[0088]

[0089] in, Let x represent the updated 3D coordinates of vertex q. q，i This represents the coordinates of vertex q in the overlapping portion of the i-th sub-region before the update. After the update, the positions of all vertices in the complete facial expression model will be obtained. It should be noted that in this step, the positions of vertices that do not belong to the overlapping region are not adjusted.

[0090] (4) The steps (1) to (3) above are the fusion process for one target expression model in the target expression model sequence. It needs to be repeated n times to fuse all the target expression models individually in turn to obtain the complete facial expression model corresponding to all the target expression models in the target expression model sequence.

[0091] Step 4: Compare each complete facial expression model obtained in Step 3 with its corresponding target facial expression model, find the target facial expression model with the largest error compared with the corresponding complete facial expression model, add the target facial expression model as the base expression to the base expression group, and delete the target facial expression model from the target facial expression model sequence;

[0092] In this step, the n complete facial expression models fitted using the base expression set are compared one by one with the n target expression models in the target expression model sequence. The target expression model with the largest error is identified and added to the base expression set. This is achieved through the following steps:

[0093] (1) First, the complete facial expression model obtained in step three is registered with its corresponding target facial expression model using the nearest point iteration algorithm to make the point cloud distribution position consistent.

[0094] (2) Calculate the error. The error here is the average of the Euclidean distances between each target expression model and the corresponding complete facial expression model.

[0095] (3) Calculate the error of all target expression models in the target expression model sequence, and select the maximum error value δ from these n error values. max The corresponding target expression model is added to the base expression group, and the number of base expressions in the base expression group is updated to K = K + 1, while the number of remaining target expression models in the target expression model sequence is updated to n = n - 1. Then, the new base expression group is used for the next round of iteration.

[0096] Step 5: Repeat steps 2 to 4 until the rate of change of the maximum error mentioned in step 4 is less than 0.1, and obtain the base expression set.

[0097] In this step, steps two through four are repeated until the maximum error value δ is obtained in each iteration. max The iteration stops when the rate of change of δ tends to level off (for example, usually at δ). max When the rate of change is less than 0.1, we consider δ max (When the rate of change slows down), a more suitable set of base expressions can eventually be obtained.

[0098] Therefore, the automatic base expression selection method based on local blending optimization proposed in this invention can select a small number of expression models from a series of expression models as base expressions to cover almost all facial expressions. Using this method, only continuous frame-by-frame facial expression scanning of the face is required, and then the program automatically selects the necessary base expressions. This eliminates the need for extensive manual labor to select base expressions from thousands of 3D facial expression models, effectively simplifying the process of constructing a local blending deformation model and having practical significance for the production of virtual character facial expression animations.

[0099] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.

Claims

1. A method for automatically selecting base expressions based on local hybridization optimization, comprising: Step 1: Obtain a sequence of facial expression models with the same topology, and perform region segmentation on each facial expression model in the sequence to obtain multiple sub-regions; Neutral expression models are selected from the facial expression model sequence as the initial base expressions in the base expression group, and the remaining unselected facial expression models in the facial expression model sequence are used as the target expression model sequence. Step 2: Use each sub-region obtained by region segmentation of the base expressions in the base expression group to fit the corresponding sub-region of each target expression model in the target expression model sequence; obtain the local facial expression model corresponding to each target expression model; Step 3: Fuse the local facial expression models corresponding to each target expression model obtained in Step 2 to obtain the complete facial expression model corresponding to each target expression model; Step 3.1: Based on the distance from the vertex of the overlapping part of each sub-region to the center of the sub-region, calculate the fusion weight of each vertex according to the following formula. Among them, w q,i μ represents the fusion weight of vertex q in the overlapping part of the i-th sub-region. q,i Let σ be the Euclidean distance between the vertex q of the overlapping portion after deformation and the center point of the i-th sub-region. s The standard deviation of a Gaussian kernel is empirically set to 1.6 times the average width of a sub-region; Step 3.2: Normalize the fusion weights of the vertices based on the sub-regions to which the overlapping parts belong, using the following formula: in, This represents the fusion weight of vertex q in the overlapping part of the i-th sub-region after normalization, where i∈I represents the range of sub-regions involved in the overlapping region; Step 3.3: Update the positions of vertices belonging to the overlapping region, using the following formula: in, Let x represent the updated 3D coordinates of vertex q. q,i This represents the coordinates of vertex q in the overlapping part of the i-th sub-region before the update, and the positions of all vertices of the complete facial expression model after the update. Step 3.4: Repeat steps 3.1 to 3.3 until the local facial expression models corresponding to each target expression model are fused to obtain the complete facial expression model corresponding to each target expression model; Step 4: Compare each complete facial expression model obtained in Step 3 with its corresponding target facial expression model, find the target facial expression model with the largest error compared with the corresponding complete facial expression model, add the target facial expression model as the base expression to the base expression group, and delete the target facial expression model from the target facial expression model sequence; Step 5: Repeat steps 2 to 4 until the rate of change of the maximum error mentioned in step 4 is less than 0.1, and obtain the base expression set.

2. The method as described in claim 1, characterized in that, The step one of obtaining the facial expression model sequence with the same topology includes the following steps: Step 1.1: Perform continuous frame facial expression scanning and reconstruct a three-dimensional facial expression model sequence based on the scanning results; Step 1.2: Using 3D model topology software, retopologically reconstruct each facial expression model in the reconstructed facial expression model sequence using a facial template with an ideal topological structure to obtain a facial expression model sequence with the same topology.

3. The method as described in claim 1 or 2, characterized in that, Step one, performing region segmentation on each facial expression model in the facial expression model sequence, includes the following steps: Step 1.3: Create a facial segmentation mask in texture space according to the distribution area of ​​facial muscles. Obtain the coordinates of each vertex of the three-dimensional facial expression model in texture space through texture unrolling, and divide the coordinates into multiple sub-regions of the mask. Step 1.4: Extend the range of each sub-region outward by a portion, so that adjacent sub-regions have 20% overlapping vertices.

4. The method as described in claim 3, characterized in that, Step two includes the following steps: Step 2.1: Calculate the shape of the i-th sub-region using the following formula. Among them, P i =[R i IT i Let be the pose matrix of the i-th sub-region, including the rotation matrix R. i Translation matrix T i u i This represents the shape of the i-th sub-region in the neutral expression model. This represents the weight value of the k-th hybrid deformation in the i-th sub-region. This refers to the change of the k-th hybrid deformation of the i-th sub-region relative to the neutral expression model, where K is the number of base expressions in the base expression group; Step 2.2: Establish the objective function, using the hybrid deformation parameters and spatial pose of each sub-region as unknowns, and the 3D vertices of the target facial expression model as input. The formula is as follows: E=E M +E o (2) Wherein, the objective function E is derived from E M and E o Composition, E M This represents the constraints on the shape and spatial pose of the sub-regions, used to determine the blending deformation parameters of each sub-region in the facial expression model and the spatial position of each sub-region, expressed by the following formula. Where V represents all sub-regions participating in the local hybrid deformation, and Q i λ represents all vertices contained in the i-th extended sub-region. M These are the weight values ​​for the shape constraints; E o It is a constraint on the overlapping parts between adjacent sub-regions, expressed by the following formula. E O =λ O ∑ q∈R ∑ (i，j)∈V ||x q，i -x q，j || 2 (4) Where q∈R represents a vertex within an adjacent overlapping region, x q,i Let x be the coordinates of vertex q in the overlapping part of the i-th sub-region after deformation. q, j represents the coordinates of vertex q in the overlapping portion of the j-th sub-region after deformation, and λ O These are the weight values ​​for overlapping constraints; Step 2.3: Solve for the optimal value so that the objective function E = E M +E O The value of is minimized, therefore... arrive Hybrid deformation coefficient for each sub-region and spatial pose R i and T i ; Step 2.4: Calculate the three-dimensional coordinates of the vertices in each sub-region after fitting using the above formula (1). Standard x i This allows the construction of a fitted local facial expression model; Step 2.5: Repeat steps 2.2 to 2.4 until each target expression model in the target expression model sequence has been fitted, and a local facial expression model corresponding to each target expression model is obtained.

5. The method as described in claim 1, characterized in that, Step four includes: Step 4.1: Use the nearest point iterative algorithm to register the point cloud of the complete facial expression model obtained in Step 3 with its corresponding target facial expression model to make the point cloud distribution position consistent; Step 4.2: Calculate the error. The average value of the Euclidean distance between each target expression model and the corresponding complete facial expression model is taken as the error of the target expression model. Step 4.3: Calculate the error for each target expression model in the target expression model sequence, add the target expression model corresponding to the maximum error to the base expression group, and delete the target expression model from the target expression model sequence.

Images