A face skinning decomposition automatic binding method based on rotation space regularization
Patent Information
- Application Number
- CN202611021801.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-09
- Publication Date
- 2026-10-13
AI Technical Summary
若目标表情对应的关节旋转解中包含不必要的大幅度旋转，即使该关节旋转解在目标状态下能够获得较低的顶点位置重构误差，也可能在中立表情到目标表情的变形过程中引起局部中间状态形变不稳定，使面部局部区域出现不自然的形变变化
1、本发明提供一种基于旋转空间正则的面部蒙皮分解自动绑定方法，在面部蒙皮分解的关节变换矩阵求解中引入用于约束关节旋转矩阵相对于恒等旋转矩阵或参考旋转矩阵的偏离程度的旋转空间正则项，然后利用旋转空间正则项稳定关节旋转解的选择，当多组关节旋转解具有相近目标状态重构误差时，通过旋转空间正则项抑制不必要的大幅度关节旋转，减少中立表情到目标表情变形过程中的局部中间状态形变不稳定，使求解结果倾向于较小幅度的充分旋转解。
Smart Images
Figure CN122906208A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of automatic face binding technology in computer graphics, and particularly relates to an automatic face skin decomposition binding method based on rotational space regularization. Background Technology
[0002] Currently, automatic face binding methods are commonly used in high-fidelity digital human facial animation production. These methods automatically generate facial binding parameters that can be used for animation-driven animation based on the input facial expression data of the character. Existing automatic face binding methods based on template binding and linear hybrid skinning decomposition generally use the reconstruction error of the target expression vertex position as the main constraint during the joint transformation matrix update process, and combine it with soft constraints on joint position to solve the joint rotation and translation corresponding to each target expression. During the skinning weight update process, constraints such as non-negativity of weights, normalization, and fixed sparse distribution are combined to maintain the locality and rationality of the weight distribution.
[0003] The above-mentioned scheme can reproduce the input target expression relatively well, but its optimization objective mainly acts on the target state, that is, it mainly constrains whether the neutral expression can approach the target expression after deformation by the joint transformation matrix and skinning weights. For a target expression, there may be multiple sets of joint rotation solutions that can obtain similar target expression vertex position reconstruction errors. Since the target expression vertex position reconstruction error mainly measures the difference in vertex position in the target state, and the soft constraints on joint position mainly limit the position deviation after joint transformation, the existing basic joint solution objective lacks direct constraints on the joint rotation amplitude itself, and it is also difficult to make a stable choice among multiple sets of joint rotation solutions with similar target state reconstruction errors.
[0004] In practical facial rigging applications, the solved joint transformation matrix is used not only to reproduce the discrete input target expression but also for the deformation process from a neutral expression to the target expression. For example, in animation editing or expression control, intermediate states are often generated based on the joint transformations corresponding to the neutral and target expressions. If the joint rotation solution corresponding to the target expression contains unnecessary large rotations, even if the joint rotation solution can achieve a low vertex position reconstruction error in the target state, it may cause local intermediate state deformation instability during the deformation process from the neutral expression to the target expression, resulting in unnatural deformation changes in local facial areas. Summary of the Invention
[0005] To address the aforementioned issues, this invention provides an automatic binding method for facial skin decomposition based on rotation space regularization. By introducing rotation space regularization terms into the solution of the joint rotation matrix, the joint rotation matrix can reduce unnecessary large-amplitude joint rotations while maintaining the ability to reproduce the target expression, thereby reducing the risk of unstable deformation in local intermediate states.
[0006] An automatic binding method for facial skin decomposition based on rotational space regularization includes the following steps:
[0007] Based on the linear hybrid skin decomposition model, the target facial expression vertex position reconstruction error is constructed. Soft constraints on joint position And rotation space regularization terms used to directly constrain the deviation of the joint rotation matrix from the identity rotation matrix or the reference rotation matrix. ; Based on the reconstruction error of the target facial expression vertex position Soft constraints on joint position Rotational space regularization term Construct an objective function containing rotational space regularization terms. as follows:
[0008] in, The weighting coefficients represent the soft constraints on joint position; Represents the weighting coefficients of the rotation space regularization term; The objective function is solved using an alternating solution method. The output is the automatic face binding result, which includes the character skinning weights, the initial positions of the facial joints, and the joint transformation matrix used to reproduce the target expression of the character.
[0009] Furthermore, the rotation space regularization term Represented as:
[0010] in, Indicates the first solution to be optimized. k Under the target facial expression grid of each character j Joint rotation matrix of 1 joint, Represents the identity rotation matrix. Let f(x) represent the Frobenius norm of the matrix, s represent the number of target facial expressions for the character, and n represent the number of joints used for facial deformation of the character.
[0011] Furthermore, the objective function of the joint rotation matrix is solved by an alternating solution method. The method for outputting the automatic face binding results is as follows: S1: Initialize the character joint positions and character skin weights in the linear hybrid skinning decomposition model; S2: Under the initialized linear hybrid skin decomposition model, calculate the objective function containing the rotational space regularization term. And determine whether the preset number of iterations or the objective function of two adjacent iterations has been reached. The change between Whether it is less than a set threshold, if any judgment result is yes, then the objective function with rotational space regularization term under the current linear hybrid skin decomposition model is... The corresponding automatic face binding result is the final result; if all the judgment results are negative, then proceed to step S3; S3: Under the condition of fixed skin weights for each character, update and optimize the joint transformation matrix corresponding to the target expression of each character using the objective function E containing the rotation space regularization term; then fix the newly calculated joint transformation matrix corresponding to the target expression of each character, update and optimize the skin weights of the characters, and realize the iterative update of the linear hybrid skin decomposition model; re-execute step S2 using the updated linear hybrid skin decomposition model.
[0012] Furthermore, the target facial expression vertex position reconstruction error It is expressed as follows:
[0013] Where s represents the number of target facial expression grids for the character. m The number of vertices contained in each face mesh. Indicates the first k The target expression for the character. i The position of each vertex Neutral emote mesh for characters The first in i vertex positions, n This indicates the number of joints used for facial deformation of the character, and it must be consistent with the total number of joints bound to the template. j For joint indexing, It is the first j Each joint corresponds to the character's neutral facial expression mesh. The Middle i The influence weight of each vertex It is the first one to be optimized. k The first character target facial expression grid j The joint translation vectors of each joint; It is the first one to be optimized. k The first character target facial expression grid j The joint rotation matrix of each joint.
[0014] Furthermore, soft constraints on joint position It is expressed as follows:
[0015] Where s represents the number of target facial expression meshes for the character, and n represents the number of joints used for facial deformation of the character. This indicates the initialization of the neutral character emote mesh. The position of the j-th joint; This represents the target facial expression grid for the k-th character. This indicates the reference position of the j-th joint under the target facial expression grid of the k-th character; It is the joint translation vector of the j-th joint under the k-th character's target facial expression mesh to be optimized; It is the joint rotation matrix of the j-th joint under the k-th character target expression mesh to be optimized.
[0016] Furthermore, the template head model, the character neutral expression mesh, and the character target expression mesh have a consistent mesh topology and a one-to-one correspondence of vertices.
[0017] Furthermore, the joint transformation matrix includes a joint rotation matrix and a joint translation vector, and the joint rotation matrix satisfies the hard constraint of a valid rotation matrix: ,in, Let be the joint rotation matrix of the j-th joint in the k-th character's target facial expression mesh, which needs to be optimized. This represents the transpose of the joint rotation matrix. The determinant of the joint rotation matrix.
[0018] Furthermore, after outputting the automatic facial binding result, the neutral expression mesh of the character is transformed into the target expression state through linear blending skinning based on the output character skinning weights and the joint transformation matrix of the corresponding target expression. At the same time, when an intermediate state is needed to generate from the neutral expression to the target expression, an intermediate joint rotation matrix is generated based on the joint rotation matrices corresponding to the neutral state and the target state. The character skinning weights drive the neutral expression mesh of the character to generate an intermediate expression state that is more natural than the automatic facial skinning decomposition binding method without rotation space regularization.
[0019] Furthermore, the weighting coefficients of the rotation space regularization term It can be set separately for different joints, different facial areas, or different character target expressions.
[0020] Beneficial effects: 1. This invention provides an automatic binding method for facial skin decomposition based on rotational space regularization. In solving the joint transformation matrix of facial skin decomposition, a rotational space regularization term is introduced to constrain the deviation of the joint rotation matrix from the identity rotation matrix or the reference rotation matrix. Then, the rotational space regularization term is used to stabilize the selection of joint rotation solutions. When multiple sets of joint rotation solutions have similar target state reconstruction errors, the rotational space regularization term suppresses unnecessary large-amplitude joint rotations, reduces the instability of local intermediate state deformation during the deformation process from neutral expression to target expression, and makes the solution result tend to be a solution with a smaller amplitude of sufficient rotation.
[0021] 2. This invention provides an automatic binding method for facial skin decomposition based on rotation space regularization. The specific measurement form of the rotation space regularization term can be either the Frobenius norm between the joint rotation matrix and the identity rotation matrix, or other rotation distance forms that can measure the deviation of the joint rotation matrix from the identity rotation matrix or the reference rotation matrix. At the same time, the weight coefficient of the rotation space regularization term can be set uniformly, or it can be set separately according to different joints, different facial regions, or different character target expressions. The rotation space regularization term is compatible with different joint rotation solution methods. The target of solving the joint transformation matrix containing the rotation space regularization term can be solved by gradient optimization, SVD (Singular Value Decomposition) closed-loop update, etc. This invention can make the selection of joint rotation solutions more explicit while maintaining the target expression reproduction ability, and reduce unnecessary large-scale rotations caused by the instability of the selection of approximate equivalent solutions. Attached Figure Description
[0022] Figure 1 A flowchart of an automatic binding method for facial skin decomposition based on rotational space regularization provided by the present invention; Figure 2(a) shows the basic linear hybrid skin decomposition objective function provided by this invention. A schematic diagram illustrating the effect of joint rotation solution selection; Figure 2(b) is a schematic diagram of the effect of the objective function containing the rotation space regularization term provided by the present invention on the selection of joint rotation solutions. Detailed Implementation
[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.
[0024] Automatic facial rigging refers to the process of automatically solving for rigging parameters that can drive facial expression changes based on the character's facial model and expression data. Linear blending skinning refers to a method of weighted deformation of the model's vertex positions through the rotation and translation of multiple joints and their corresponding skinning weights. Facial skinning decomposition refers to inversely solving for the joint transformation matrix and skinning weights from a set of example expressions or 4D expression data, enabling a neutral facial mesh to reproduce all target expression data. Template rigging refers to a pre-defined template facial rigging model, including a template head mesh, facial skeleton, joint positions, and template skinning weights, which can serve as a structural prior in the automatic character rigging process.
[0025] Based on the above definitions, in the automatic facial binding process based on linear hybrid skinning decomposition, this invention, while solving the joint transformation matrix using the target expression vertex position reconstruction error and soft constraints on joint positions, introduces a constraint mechanism directly acting on the joint rotation space. This mechanism selects the rotation amplitude for joint rotation solutions with similar target state reconstruction errors, suppressing unnecessary large-amplitude joint rotations and thus reducing local intermediate state deformation instability during the transformation from neutral to target expression. In other words, this invention uses predefined template binding, a character's neutral expression, and several target expressions as input. Based on the existing linear hybrid skinning decomposition framework, it solves for the joint transformation matrix and character skinning weights used to reproduce each character's target expression. The joint transformation matrix refers to the transformation parameters used to describe the joint transformation from a neutral state to a target expression state. The joint transformation matrix typically includes a joint rotation matrix and a joint translation vector. The core improvement of this invention mainly focuses on the joint rotation matrix.
[0026] The overall process of this invention is as follows: Figure 1 As shown, the process includes: acquiring input data; initializing character joint positions; establishing a basic linear hybrid skin decomposition model; constructing a rotation space regularization term; adding the rotation space regularization term to the joint transformation matrix to solve the objective; updating the joint transformation matrix and character skin weights through an alternating solution method; and outputting the automatic face binding result.
[0027] The template binding, character joint position initialization, basic skin decomposition framework, non-negative skin weight constraint, skin weight normalization constraint, and fixed sparse distribution constraint can be obtained using existing methods and are not considered as the core improvements of this invention.
[0028] The core of this invention lies in the introduction of a rotation space regularization term that directly acts on the joint rotation space during the joint transformation matrix solution process. This term is used to constrain the deviation of the joint rotation matrix from the identity rotation matrix or the reference rotation matrix, thereby introducing a rotation amplitude selection preference among multiple sets of joint rotation solutions with similar target state reconstruction errors.
[0029] In one implementation, suppose there is an existing template-bound header model. The template-bound head model has a predefined facial skeleton and template skinning weights, which can serve as a structural prior in the automatic character face binding process. Let the template facial skeleton contain n joints for facial deformation, and let j denote the joint index. For the character to be bound, the input data includes the character's neutral expression mesh. And a set of character target facial expression grids Where 's' represents the number of target facial expressions for the character. Let represent the mesh for the k-th character's target facial expression. The bound template head model, the character's neutral facial expression mesh, and the character's target facial expression mesh have a consistent mesh topology and a one-to-one correspondence of vertices. Assume each facial mesh contains m vertices, and the position of the i-th vertex in the character's neutral facial expression mesh is denoted as . The position of the i-th vertex in the target facial expression mesh for the k-th character is represented as... .
[0030] The goal of facial skinning decomposition is to solve for the joint stiffness transformation and skinning weights of the character's own model (neutral head mesh) for each target facial expression, given a template structure prior. The joint stiffness transformation consists of joint rotation matrices and joint translation vectors. The joint rotation matrix for the j-th joint under the k-th target facial expression is expressed as: The joint translation vector is represented as ,in , represents a valid three-dimensional rotation matrix; , representing the three-dimensional translation vector. Template skinning weights are represented as... , This indicates that the j-th joint is bound to the template head model. The influence weight of the i-th vertex. The skinning weight of the character to be optimized is represented as... ,in It is the j-th joint for the character's neutral facial expression mesh. The influence weight of the i-th vertex. Hard constraints such as the non-negativity, normalization, and fixed sparse distribution of the character skinning weights can be implemented using existing basic skinning decomposition frameworks and are not considered core improvements in this invention. The above basic representation is used to subsequently construct a basic linear hybrid skinning decomposition model.
[0031] In one implementation, for the k-th target facial expression of a character, the reproduction position of the i-th vertex in the neutral facial expression mesh after linear blending skinning deformation can be represented as: .in, This represents the position of the i-th vertex of the target expression of the k-th character being reproduced.
[0032] In existing automatic binding methods for facial skin decomposition, the accuracy of reproducing the target expression is typically constrained by the target expression vertex position reconstruction error. In one implementation, the target expression vertex position reconstruction error is expressed as: ,in, This indicates the error in reconstructing the vertex position of the target facial expression.
[0033] To limit positional deviation after joint transformation, existing methods can also incorporate soft constraints on joint position. In one implementation, the soft constraint on joint position is expressed as: .in, Indicates soft constraints on joint position; This indicates the initial neutral emote for the character. The position of the j-th joint; This represents the reference position of the j-th joint under the k-th character's target expression. This reference position can be obtained using an existing template-guided method based on the character's target expression.
[0034] In the basic linear hybrid skin decomposition framework, the objective of solving the joint transformation matrix is usually expressed as: .in, This represents the weighting coefficients of the soft constraints on joint positions. The primary objective of this solution is to reproduce the character's facial expressions and limit joint position deviations. Further, this invention introduces a rotation space regularization term into the basic joint transformation matrix solution objective to directly constrain the degree of rotation space deviation of the joint rotation matrix.
[0035] Based on the above input data, basic representation, and basic linear hybrid skin decomposition model, this invention provides an automatic binding method for facial skin decomposition based on rotational space regularization, comprising the following steps: Step 1: Construct the target facial expression vertex position reconstruction error based on the linear hybrid skinning decomposition model Soft constraints on joint position And rotation space regularization terms used to directly constrain the deviation of the joint rotation matrix from the identity rotation matrix or the reference rotation matrix. ; Step 2: Reconstruct error based on the vertex position of the target facial expression Soft constraints on joint position Rotational space regularization term Construct an objective function containing rotational space regularization terms. as follows:
[0036] in, The weighting coefficients represent the soft constraints on joint position; The weight coefficients of the rotation space regularization term; It can be set separately according to different joints, different facial areas, or different character target expressions; Rotation space regularization is a constraint term that directly measures the deviation of the joint rotation matrix from the identity rotation matrix or reference rotation matrix. It is used to suppress unnecessary large rotations in joint rotation solutions. Represented as:
[0037] in, Indicates the first solution to be optimized. kUnder the target facial expression grid of each character j The joint rotation matrix of each joint refers to the rotation part in the joint transformation matrix. The rotation space regularization term of this invention mainly acts on this part. Represents the identity rotation matrix. Let represent the Frobenius norm of the matrix, s represent the number of target facial expressions for the character, and n represent the number of joints used for facial deformation. It should be noted that the reference object in the rotation space regularization term can be an identity rotation matrix that does not produce rotational effects, or a preset reference rotation matrix. The preset reference rotation matrix can be obtained based on template binding priors, neutral state joint rotations, manually set joint reference states, or other prior information. Furthermore, the rotation space regularization term is not limited to using the Frobenius norm form between the joint rotation matrix and the identity rotation matrix; it can also use other rotational distance forms that can measure the deviation of the joint rotation matrix from the identity rotation matrix or the reference rotation matrix.
[0038] The target facial expression vertex position reconstruction error refers to the error between the reproduced target facial expression vertex position and the input target facial expression vertex position, and is a commonly used objective item in basic skinning decomposition methods. It is expressed as follows:
[0039] Where s represents the number of target facial expression grids for the character. m The number of vertices contained in each face mesh. Indicates the first k The target expression for the character. i The position of each vertex Neutral emote mesh for characters The first in i vertex positions, n This indicates the number of joints used for facial deformation of the character, and it must be consistent with the total number of joints bound to the template. j For joint indexing, It is the first j Each joint corresponds to the character's neutral facial expression mesh. The Middle i The influence weight of each vertex It is the first one to be optimized. k The first character target facial expression grid j The joint translation vectors of each joint; It is the first one to be optimized. k The first character target facial expression grid j The joint rotation matrix of each joint.
[0040] Soft constraint of joint position It is expressed as follows:
[0041] Where s represents the number of target facial expression meshes for the character, and n represents the number of joints used for facial deformation of the character. This indicates the initialization of the neutral character emote mesh. The position of the j-th joint; This represents the target facial expression grid for the k-th character. This indicates the reference position of the j-th joint under the target facial expression grid of the k-th character; It is the joint translation vector of the j-th joint under the k-th character's target facial expression mesh to be optimized; It is the joint rotation matrix of the j-th joint under the k-th character target expression mesh to be optimized.
[0042] It should be noted that the template head model, the character's neutral facial expression mesh, and the character's target facial expression mesh have a consistent mesh topology and a one-to-one correspondence of vertices.
[0043] Therefore, the core objective of this invention is to construct a rotation space regularization term that directly acts on the joint rotation space, used to measure the degree of deviation of the joint rotation matrix from the identity rotation matrix or the reference rotation matrix.
[0044] For the same character's target expression, multiple joint rotation solutions may yield similar reconstruction errors of the target expression vertex positions. However, the rotation amplitudes of different joint rotation solutions may differ. As shown in Figure 2(a), when solving the target using only the basic joint transformation matrix, the target state energy... The rotation range may vary slightly, allowing multiple candidate joint rotation solutions to have approximately equivalent target state reconstruction costs. However, if unnecessarily large joint rotations are included, even if the joint rotation solution achieves low reproduction error in the target state, it may cause local intermediate state deformation instability during the deformation process from a neutral expression to the target expression. Here, intermediate state deformation refers to the intermediate expression states generated during the deformation process from a neutral expression to the target expression. This application focuses on the naturalness and stability of local deformations during this process.
[0045] Therefore, this invention introduces a rotation space regularization term during the joint rotation solution process to suppress unnecessary large-amplitude joint rotations. In one embodiment, an identity rotation matrix is used. As a reference rotation matrix, the rotation space regularization term is expressed as: .
[0046] The aforementioned rotation space regularization term measures the deviation between the joint rotation matrix and the identity rotation matrix. The closer the joint rotation matrix is to the identity rotation matrix, the smaller the value of this regularization term; conversely, the larger the rotation amplitude corresponding to the joint rotation matrix, the larger the value of this regularization term. Therefore, adding this rotation space regularization term to the joint transformation matrix solution objective can reduce the objective cost of solutions with unnecessary large joint rotation amplitudes, making the solution process tend to select joint rotation solutions with smaller amplitudes that can reproduce the target expression, as shown in Figure 2(b). It should be noted that Figure 2(b) is a two-dimensional mechanism diagram used to illustrate the effect of the rotation space regularization term on the selection of joint rotation solutions, and does not limit the actual dimension of the joint rotation space, the specific shape of the energy curve, or the numerical solution method.
[0047] In another implementation, the rotation space regularization term can also measure the deviation of the joint rotation matrix from a preset reference rotation matrix. This reference rotation matrix can be obtained based on a neutral state, template binding prior, or a user-defined joint reference state.
[0048] Step 3: Solve the objective function using an alternating solution method. The output is the automatic face binding result, which includes the character skinning weights, the initial positions of the facial joints, and the joint transformation matrix used to reproduce the target expression of the character.
[0049] The joint transformation matrix includes a joint rotation matrix and a joint translation vector, and the joint rotation matrix satisfies the hard constraint of a valid rotation matrix: ,in, Let be the joint rotation matrix of the j-th joint in the k-th character's target facial expression mesh, which needs to be optimized. This represents the transpose of the joint rotation matrix. This represents the determinant of the joint rotation matrix. Hard constraints on this type of legal rotation matrix can be implemented using existing methods such as rotation parameterization.
[0050] Furthermore, after outputting the automatic facial binding result, the neutral expression mesh of the character is transformed into the target expression state through linear blending skinning based on the output character skinning weights and the joint transformation matrix of the corresponding target expression. At the same time, when an intermediate state is needed to generate from the neutral expression to the target expression, an intermediate joint rotation matrix is generated based on the joint rotation matrices corresponding to the neutral state and the target state. The character skinning weights drive the neutral expression mesh of the character to generate an intermediate expression state that is more natural than the automatic facial skinning decomposition binding method without rotation space regularization.
[0051] Specifically, the objective function of the joint rotation matrix is solved by an alternating solution method. The method for outputting the automatic face binding results is as follows: S1: Obtain template bindings, neutral character expressions, and several target character expressions, and initialize the character joint positions and character skinning weights in the linear blending skinning decomposition model; S2: Under the initialized linear hybrid skin decomposition model, calculate the objective function containing the rotational space regularization term. And determine whether the preset number of iterations or the objective function of two adjacent iterations has been reached. The change between Whether it is less than a set threshold, if any judgment result is yes, then the objective function with rotational space regularization term under the current linear hybrid skin decomposition model is... The corresponding automatic face binding result is the final result; if all the judgment results are negative, then proceed to step S3; S3: Under the condition of fixed skin weights for each character, update and optimize the joint transformation matrix corresponding to the target expression of each character using the objective function E containing the rotation space regularization term; then fix the newly calculated joint transformation matrix corresponding to the target expression of each character, update and optimize the skin weights of the characters, and realize the iterative update of the linear hybrid skin decomposition model; re-execute step S2 using the updated linear hybrid skin decomposition model.
[0052] It can be seen that each time the joint transformation matrix is updated, the rotation space regularization term participates in the solution of the joint rotation matrix, thereby continuously introducing rotation amplitude selection preference in the alternating solution process.
[0053] Through the steps S1~S3 described above, the character skinning weights and joint transformation matrices are updated iteratively and alternately. The character skinning weights describe the influence of each joint on the vertices, the joint transformation matrices reproduce the target expressions of each character, and the rotation space regularization term suppresses unnecessary large-amplitude joint rotations during the solution of the joint rotation matrix.
[0054] It should be noted that the purpose of step 3 of the present invention is to add the rotation space regularization term to the target of solving the basic joint transformation matrix, so that the joint transformation matrix solving process simultaneously considers the target expression reproduction error, the soft constraint of joint position, and the degree of rotation space deviation of the joint rotation matrix.
[0055] In one implementation, the objective is solved using the basic joint transformation matrix. Add rotation space regularization terms to the base ,get: It should be noted that the joint transformation matrix includes the joint rotation matrix and the joint translation vector. The rotation space regularization term of this invention mainly acts on the joint rotation matrix and does not directly act on the joint translation vector. The joint translation vector can still be solved by the target facial expression vertex position reconstruction error and the joint position soft constraint.
[0056] Using the objective function described above, when multiple sets of joint rotation solutions have similar costs under the target facial expression vertex position reconstruction error and joint position soft constraints, the rotation space regularization term will impose different costs on candidate solutions with different rotation amplitudes. Unnecessarily large joint rotation solutions will be... Larger, suppressed joint rotation solutions that are smaller in amplitude and can fully reproduce the target expression are more likely to be selected.
[0057] It should be noted that the objective of solving the joint transformation matrix containing rotation space regularization terms can be achieved using different numerical methods, such as gradient optimization methods or closed joint rotation solution methods, such as SVD (Singular Value Decomposition). This invention does not limit the specific optimizer or rotation parameterization method. The above alternatives do not change the core of this application, namely, introducing regularization terms that directly act on the joint rotation space in the solution of the joint rotation matrix of facial skin decomposition.
[0058] In summary, this invention provides an automatic facial skin decomposition and binding method based on rotational space regularization, which has the following advantages compared to existing technologies: 1. By adding a rotation space regularization term to the solution of the joint transformation matrix, the degree of deviation of the rotation space of the joint rotation matrix is directly constrained.
[0059] Existing methods for solving basic joint transformation matrices typically focus on the target expression vertex position reconstruction error and soft constraints on joint positions, primarily constraining vertex position differences and joint position deviations within the target state. This invention incorporates a rotation space regularization term into this basic objective to measure the degree of deviation of the joint rotation matrix from the identity rotation matrix or a reference rotation matrix. Therefore, the solution process for the joint rotation matrix is not only constrained by the target state reconstruction error but also directly constrained by the degree of rotation space deviation, thus imposing additional costs on unnecessary large-amplitude joint rotations.
[0060] 2. By introducing a rotation amplitude selection preference among candidate solutions with similar reconstruction errors of the target state through a rotation space regularization term.
[0061] For the same target expression, multiple joint rotation solutions may exist that all yield similar reconstruction errors of the target expression's vertex positions. Existing basic solution methods struggle to further distinguish differences in rotation amplitude among these candidate solutions. This invention increases the objective cost of solutions with large joint rotation amplitudes through a rotation space regularization term, making it easier to select joint rotation solutions with smaller amplitudes that can adequately reproduce the target expression. This mechanism, while maintaining the ability to reproduce the target expression, makes the selection of joint rotation solutions more explicit and reduces unnecessary large rotations caused by the instability of selecting approximately equivalent solutions.
[0062] 3. By suppressing unnecessary large-amplitude joint rotation, the instability of local intermediate state deformation during the transformation from neutral expression to target expression is reduced.
[0063] In practical facial rigging applications, the solved joint transformation matrix is used not only to reproduce the discrete input target expression but also for the deformation process between the neutral expression and the target expression. If the joint rotation matrix corresponding to the target expression contains unnecessary large rotations, it may cause abrupt changes or unnatural variations in local deformation when generating intermediate states. This invention introduces a rotation space regularization term during the joint rotation matrix solution stage to reduce the possibility of unnecessary large rotations entering the final rigging result, thereby helping to reduce local deformation instability in intermediate states. The effect of this technique stems from the direct constraint of the joint rotation space and does not rely on changing the input data, template rigging structure, or skinning weight update framework.
[0064] Of course, the present invention may have other various embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and modifications according to the present invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.
Claims
1. An automatic binding method for facial skin decomposition based on rotational space regularization, characterized in that, Includes the following steps: Based on the linear hybrid skin decomposition model, the target facial expression vertex position reconstruction error is constructed. Soft constraints on joint position And rotation space regularization terms used to directly constrain the deviation of the joint rotation matrix from the identity rotation matrix or the reference rotation matrix. ; Based on the reconstruction error of the target facial expression vertex position Soft constraints on joint position Rotational space regularization term Construct an objective function containing rotational space regularization terms. as follows: in, The weighting coefficients represent the soft constraints on joint position; Represents the weighting coefficients of the rotation space regularization term; The objective function is solved using an alternating solution method. The output is the automatic face binding result, which includes the character skinning weights, the initial positions of the facial joints, and the joint transformation matrix used to reproduce the target expression of the character.
2. The automatic binding method for facial skin decomposition based on rotational space regularization as described in claim 1, characterized in that, Rotational space regularization term Represented as: in, Indicates the first solution to be optimized. k Under the target facial expression grid of each character j Joint rotation matrix of 1 joint, Represents the identity rotation matrix. Let f(x) represent the Frobenius norm of the matrix, s represent the number of target facial expressions for the character, and n represent the number of joints used for facial deformation of the character.
3. The automatic binding method for facial skin decomposition based on rotational space regularization as described in claim 1, characterized in that, The objective function of the joint rotation matrix is solved by an alternating solution method. The method for outputting the automatic face binding results is as follows: S1: Initialize the character joint positions and character skin weights in the linear hybrid skinning decomposition model; S2: Under the initialized linear hybrid skin decomposition model, calculate the objective function containing the rotational space regularization term. And determine whether the preset number of iterations or the objective function of two adjacent iterations has been reached. The change between Whether it is less than a set threshold, if any judgment result is yes, then the objective function with rotational space regularization term under the current linear hybrid skin decomposition model is... The corresponding automatic face binding result is the final result; if all the judgment results are negative, then proceed to step S3; S3: Under the condition of fixed skin weights for each character, update and optimize the joint transformation matrix corresponding to the target expression of each character using the objective function E containing the rotation space regularization term; then fix the newly calculated joint transformation matrix corresponding to the target expression of each character, update and optimize the skin weights of the characters, and realize the iterative update of the linear hybrid skin decomposition model; re-execute step S2 using the updated linear hybrid skin decomposition model.
4. The automatic binding method for facial skin decomposition based on rotational space regularization as described in claim 1, characterized in that, Target facial expression vertex position reconstruction error It is expressed as follows: Where s represents the number of target facial expression grids for the character. m The number of vertices contained in each face mesh. Indicates the first k The target expression for the character. i The position of each vertex Neutral emote mesh for characters The first in i vertex positions, n This indicates the number of joints used for facial deformation of the character, and it must be consistent with the total number of joints bound to the template. j For joint indexing, It is the first j Each joint corresponds to the character's neutral facial expression mesh. The Middle i The influence weight of each vertex It is the first one to be optimized. k The first character target facial expression grid j The joint translation vectors of each joint; It is the first one to be optimized. k The first character target facial expression grid j The joint rotation matrix of each joint.
5. The automatic binding method for facial skin decomposition based on rotational space regularization as described in claim 1, characterized in that, Soft constraint of joint position It is expressed as follows: Where s represents the number of target facial expression meshes for the character, and n represents the number of joints used for facial deformation of the character. This indicates the initialization of the neutral character emote mesh. The position of the j-th joint; This represents the target facial expression grid for the k-th character. This indicates the reference position of the j-th joint under the target facial expression mesh of the k-th character; It is the joint translation vector of the j-th joint under the k-th character's target facial expression mesh to be optimized; It is the joint rotation matrix of the j-th joint under the k-th character target facial expression mesh to be optimized.
6. The automatic binding method for facial skin decomposition based on rotational space regularization as described in claim 1, characterized in that, The template head model, the neutral character expression mesh, and the target character expression mesh have a consistent mesh topology and a one-to-one correspondence of vertices.
7. The automatic binding method for facial skin decomposition based on rotational space regularization as described in claim 1, characterized in that, The joint transformation matrix includes a joint rotation matrix and a joint translation vector, and the joint rotation matrix satisfies the hard constraint of a valid rotation matrix: ,in, Let be the joint rotation matrix of the j-th joint in the k-th character's target facial expression mesh, which needs to be optimized. This represents the transpose of the joint rotation matrix. The determinant of the joint rotation matrix.
8. The automatic binding method for facial skin decomposition based on rotational space regularization as described in claim 1, characterized in that, After the automatic face binding result is output, the neutral facial expression mesh of the character is deformed into the target expression state through linear blending skinning according to the output character skinning weights and the joint transformation matrix of the corresponding target expression. Meanwhile, when an intermediate state is needed to transition from a neutral expression to a target expression, an intermediate joint rotation matrix is generated based on the joint rotation matrices corresponding to the neutral and target states. The character skinning weights drive the neutral expression mesh to generate a more natural intermediate expression state compared to the automatic binding method of facial skinning decomposition without rotation space regularization.
9. The automatic binding method for facial skin decomposition based on rotational space regularization as described in claim 1, characterized in that, Weight coefficients of the rotation space regularization term It can be set separately for different joints, different facial areas, or different character target expressions.