A method, apparatus, electronic device, and storage medium for facial bone binding

Abstract

This application provides a facial skeleton rigging method, apparatus, electronic device, and storage medium. The facial skeleton rigging method includes: binding facial bones to facial mesh data in the initial state of the character to be constructed, obtaining a first binding result; loading a historical skeleton controller and first controller parameters of the historical skeleton controller; calculating the facial bone position using the first controller parameters, obtaining the vertex position of the first facial mesh; and optimizing the first binding result based on the vertex position of the first facial mesh, the facial mesh data, and the first controller parameters to obtain a second binding result. Calculating the facial bone position using controller parameters and optimizing the first binding result using the vertex position of the facial mesh and the facial mesh data as ground truth significantly improves the rigging effect of facial skeleton rigging.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a facial bone binding method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, facial movements of virtual characters in character animation can be achieved using skeletal skinning animation. Skeletal skinning animation consists of two parts: a skeleton and a skin. When applying skeletal skinning animation to the movement of a virtual character's face, it is necessary to bind the facial bones to the skin.

[0003] In related technologies, the binding information of facial bones and skins for different characters is reused. However, after reusing the binding information, it is necessary to manually fine-tune the binding information based on experience. Even after fine-tuning, the binding information still cannot achieve the best binding for the current character, resulting in poor binding effect. Summary of the Invention

[0004] The purpose of this application is to provide a facial bone binding method, device, electronic device, and storage medium to improve the binding effect of reusing binding information.

[0005] In a first aspect, embodiments of this application provide a facial skeleton binding method, comprising: binding facial bones to facial mesh data in the initial state of a character to be constructed, and obtaining a first binding result; loading a historical skeleton controller and first controller parameters of the historical skeleton controller; calculating the position of the facial bones using the first controller parameters, and obtaining the vertex position of a first facial mesh; optimizing the first binding result based on the vertex position of the first facial mesh, the facial mesh data, and the first controller parameters, and obtaining a second binding result.

[0006] In the implementation of the above scheme, the position of the facial bones is calculated using controller parameters, and the position of the facial mesh vertex and the facial mesh data as ground truth are used to optimize the first binding result, which greatly improves the binding effect of facial bone binding.

[0007] In one implementation of the first aspect, optimizing the first binding result based on the vertex positions of the first facial mesh, the facial mesh data, and the first controller parameters to obtain a second binding result includes: optimizing the first controller parameters based on the vertex positions of the first facial mesh and the facial mesh data to obtain second controller parameters; calculating the facial bone position using the second controller parameters to obtain the vertex positions of the second facial mesh; and optimizing the first binding result based on the vertex positions of the second facial mesh and the facial mesh data to obtain the second binding result.

[0008] In the implementation of the above scheme, by fixing the first binding result, the parameters of the first controller are optimized to obtain the optimized parameters of the second controller, so that the skeleton controller obtains the optimal parameters; by fixing the parameters of the second controller, the first binding result is optimized to obtain the optimized second binding result, so that the character to be constructed obtains the optimal binding result, effectively improving the binding effect of the facial skeleton.

[0009] In one implementation of the first aspect, optimizing the first controller parameters and obtaining the second controller parameters based on the vertex positions of the first facial mesh and the facial mesh data includes: optimizing the first controller parameters with the minimum deviation between the vertex positions of the first facial mesh and the actual positions of facial bones in the facial mesh data as the optimization objective, and then obtaining the second controller parameters.

[0010] In the implementation of the above scheme, the optimization objective is to minimize the deviation between the vertex position of the first facial mesh calculated using the first controller parameters and the actual position of the facial bones in the facial mesh data. The optimized second controller parameters are obtained so that the optimal second controller parameters are obtained while fixing the first binding result. Furthermore, the facial bone position calculated using the second controller parameters can effectively restore the facial mesh data, thus effectively improving the binding effect of the facial bones.

[0011] In one implementation of the first aspect, optimizing the first binding result and obtaining the second binding result includes: optimizing the first binding result and obtaining the second binding result with the objective of minimizing the deviation between the vertex position of the second facial mesh and the actual position of the facial bones in the facial mesh data.

[0012] In the implementation of the above scheme, the first binding result is optimized by minimizing the deviation between the vertex position of the second facial mesh and the actual position of the facial bones in the facial mesh data, so as to obtain the second binding result. This allows the second binding result to effectively fit the character's expression within a limited range of controller parameters, solving the problem of poor expression fitting effect after binding information reuse and effectively improving the binding effect of facial bones.

[0013] In one implementation of the first aspect, optimizing the first binding result based on the vertex positions of the first facial mesh, the facial mesh data, and the first controller parameters to obtain a second binding result includes: obtaining the depth level of the facial bone in the bone tree, and performing layered optimization on the first binding result based on the depth level, the vertex positions of the first facial mesh, the facial mesh data, and the first controller parameters to obtain a second binding result.

[0014] The step of optimizing the first binding result based on the vertex positions of the first facial mesh, the facial mesh data, and the first controller parameters to obtain a second binding result includes:

[0015] Obtain the depth level of the facial bone in the bone tree, and perform layered optimization on the first binding result based on the depth level, the vertex position of the first facial mesh, the facial mesh data, and the first controller parameters to obtain the second binding result.

[0016] In one implementation of the first aspect, the step of calculating the facial bone position using the first controller parameters and obtaining the vertex position of the first facial mesh includes: calculating the facial bone position using linear blending skin (LBS) and the first controller parameters, and obtaining the vertex position of the first facial mesh.

[0017] The step of calculating the facial bone position using the second controller parameters and obtaining the vertex position of the second facial mesh includes: calculating the facial bone position using linear blending skin (LBS) and the second controller parameters, and obtaining the vertex position of the second facial mesh.

[0018] In the implementation of the above scheme, linear hybrid skin LBS is used to accurately calculate the position of facial bones, so that the above facial bone binding method can optimize the binding result based on the accurate position of facial mesh vertex, thereby effectively improving the binding effect of facial bone binding when binding information is reused.

[0019] In one implementation of the first aspect, optimizing the first binding result and obtaining the second binding result includes: optimizing at least one of the following in the first binding result: the weights of the facial bones and facial mesh data vertices, the position of the facial bones in the initial state of the character to be constructed, and the rotation amount of the facial bones in the initial state of the character to be constructed, to obtain the second binding result.

[0020] In the implementation of the above scheme, at least one of the following can be optimized according to the actual application: the weight of the facial bones and the vertices of the facial mesh data, the position of the facial bones in the initial state of the character to be constructed, and the rotation amount of the facial bones in the initial state of the character to be constructed. This makes the above facial bone binding method applicable to more application scenarios and improves the adaptability of the above facial bone binding method.

[0021] Secondly, embodiments of this application provide a facial bone binding device, comprising:

[0022] The initial binding module is used to bind facial bones and facial mesh data in the initial state of the character to be built, and obtain the first binding result;

[0023] A skeleton controller loading module is used to load a historical skeleton controller and the first controller parameters of the historical skeleton controller.

[0024] The first calculation module is used to calculate the position of the facial bones using the parameters of the first controller, and to obtain the vertex position of the first facial mesh.

[0025] The binding result optimization module is used to optimize the first binding result based on the vertex positions of the first facial mesh, the facial mesh data, and the parameters of the first controller to obtain a second binding result.

[0026] Thirdly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when read and executed by a processor, perform the method provided in the first aspect or any possible implementation thereof.

[0027] Fourthly, embodiments of this application provide an electronic device, including: a memory and a processor, wherein the memory stores computer program instructions, and the computer program instructions are read and executed by the processor to perform the method provided in the first aspect or any possible implementation of the first aspect. Attached Figure Description

[0028] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A schematic flowchart illustrating the facial bone binding method provided in this application embodiment;

[0030] Figure 2 This is a schematic diagram of the facial bone binding device provided in the embodiments of this application;

[0031] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of this application, and are therefore merely examples and should not be used to limit the scope of protection of this application.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0034] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary or secondary relationship of the indicated technical features. In the description of the embodiments of this application,

[0035] "Multiple" means two or more, unless otherwise explicitly specified.

[0036] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that...

[0037] The embodiments described herein can be combined with other embodiments.

[0038] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0039] Currently, the skeletal skinning method in character animation generally adopts the following steps:

[0040] Construct 4D data of real human faces, that is, 3D face mesh data over a time series;

[0041] Based on the binding information, the controller parameters within the aforementioned time series are calculated to generate a controller animation.

[0042] The controller parameters are manually fine-tuned based on experience.

[0043] Since the binding information for different characters is currently reused, when making fine adjustments manually, it is generally only possible to fine-tune the controller parameters. The most common situation is that no matter how the controller parameters are adjusted, it is impossible to obtain a mesh that meets the preset error requirements, resulting in the animation effect of the character being far from the effect of real people.

[0044] Based on the above findings, this application provides a facial skeleton binding method. The method uses the first controller parameters of the historical skeleton controller and calculates the facial skeleton position through the first controller parameters to obtain the vertex position of the first facial mesh. The initial binding of the character to be constructed is optimized according to the vertex position of the first facial mesh, the facial mesh data and the first controller parameters to obtain the optimal binding between the facial skeleton and the facial mesh data of the character to be constructed, so that the constructed character can fit a realistic expression.

[0045] The facial bone rigging method described above is explained in detail below. Please refer to [link / reference]. Figure 1 This application provides a facial bone binding method, including:

[0046] Step S110: In the initial state of the character to be constructed, bind the facial bones and facial mesh data to obtain the first binding result;

[0047] Step S120: Load the historical skeleton controller and the first controller parameters of the historical skeleton controller;

[0048] Step S130: Calculate the facial bone position using the first controller parameters and obtain the vertex position of the first facial mesh;

[0049] Step S140: Optimize the first binding result based on the vertex positions of the first facial mesh, the facial mesh data, and the parameters of the first controller to obtain the second binding result.

[0050] In the implementation of the above scheme, the first binding result is automatically optimized by using the vertex position of the first facial mesh, the facial mesh data, and the first controller parameters, which improves the efficiency of facial bone binding. At the same time, compared with the manual fine-tuning method, the facial bone position is calculated by using the controller parameters, and the first binding result is optimized by using the vertex position of the facial mesh and the facial mesh data as the ground truth, which greatly improves the binding effect of facial bone binding.

[0051] Step S110 is described in detail below:

[0052] The initial state of the character to be built in step S110 refers to the character's silent facial pose, also known as the Neutral pose. The character to be built binds the facial bones and facial mesh data in this silent pose.

[0053] The facial bones in step S110 can be pre-set virtual character facial bones, or the desired character can be sculpted in Metahuman, and then the character model and bones can be exported.

[0054] In step S110, the facial mesh data can be obtained by reconstructing the face from the captured face image using multi-view geometry and dense reconstruction techniques, thus obtaining the mesh data of the real face. It can be understood that the mesh data can be 4D data, i.e., 3D facial mesh data over a time series.

[0055] Understandably, after reconstructing a face to obtain the mesh data of a real face, a non-rigid body topology algorithm, such as non-rigid body registration, can be used to retopologically reconstruct the messy mesh data obtained from dense reconstruction into topologically consistent mesh data.

[0056] The data to be acquired in step S110 when binding facial bones to facial mesh data includes: the binding weights of vertices in the bones and facial mesh data, and the rotation and translation of the bones in the initial state. It can be understood that the binding weights of vertices in the bones and facial mesh data, and the rotation and translation of the bones in the initial state, can be reused binding information from historical binding information. Step S110 can apply the historical binding information to the character to be constructed to build the initial binding. The binding information can be exported as an .fbx file for storage.

[0057] Step S120 is described in detail below:

[0058] The bone controller in step S120 can control one or more facial bones. By adjusting the parameters of the bone controller, the one or more facial bones controlled by the bone controller can be adjusted. The first controller parameters of the bone controller obtained in step S120 can be reused from historical bone controllers. The parameters of the bone controller can be converted into the rotation and translation of the corresponding bone in the initial state.

[0059] In this embodiment, the number of skeletal controllers can be 258. The first controller parameter of the skeletal controller is denoted as CRTL, and its value range is (0,1). According to MetaHuman's Rig Logic rules, 476 PSD coefficients can also be obtained from the 258 controller parameters CRTL, denoted as PSD = Q(CRTL), where Q is a constant mapping.

[0060] Step S130 is described in detail below:

[0061] As an optional implementation of the above-mentioned facial bone binding method, step S130 calculates the facial bone position using the first controller parameters and obtains the vertex position of the first facial mesh, including: calculating the facial bone position using linear blending skin (LBS) and the first controller parameters, and obtaining the vertex position of the first facial mesh. For example, this implementation includes:

[0062] The first face mesh vertex position refers to the position information obtained after calculating the mesh vertex positions based on the first controller parameters and the first binding result. The linear hybrid skin LBS uses the following formula to calculate the first face mesh vertex position:

[0063]

[0064] Among them, M i W represents the calculated vertex positions of the facial mesh. ij P represents the binding weight between the j-th bone and the i-th mesh vertex. i R represents the mesh point positions in the initial state. j T represents the initial rotation of the skeleton; j B represents the translation of the bones in the initial state; B represents the number of facial bones.

[0065] It should be noted that M i and P i All are global representations in the world coordinate system, T j and R jThe transformation matrix Bone_mesh is obtained by decomposing the transformation matrix Bone_mesh of the j-th bone. The method for obtaining the transformation matrix Bone_mesh is as follows:

[0066] Taking 3D modeling using Metahuman software as an example, after obtaining the PSD coefficients and CRTL coefficients, they are combined and denoted as Input. Based on the mathematical relationship mapping, a mathematical expression between the facial skeleton and the skeleton controller can be established:

[0067] Bone_delta = D * Input

[0068] Where D is the DNA coefficient matrix obtained when performing character modeling using Metahuman software, D∈R B*9×C Input is a combination matrix of PSD coefficients and CRTL coefficients, where Input ∈ R. C×1 B represents the number of facial bones, and C represents the sum of the number of PSD coefficients and CRTL coefficients. The constant 9 in B*9×C indicates that each bone has 9 editable quantities, including three translation quantities (translationX, translationY, and translationZ); three rotation quantities (RotationX, RotationY, and RotationZ); and three scale quantities (ScaleX, ScaleY, and scaleZ).

[0069] The Input parameter in the above expression obtains the local change of Bone_delta relative to the initial binding. Based on the parent-child relationship of bones, the global change of each bone is updated.

[0070] The expression used to characterize the relationship of change between bones with a parent-child relationship is:

[0071]

[0072] Where Bone_global represents the global transformation matrix of the bone; Bone_parent_global represents the global transformation matrix of the parent bone of the current bone; Bone_local represents the local transformation matrix of the bone; Bone_delta represents the local transformation matrix of the bone calculated from Input; and operators... This indicates that the matrix Bone_local and the matrix Bone_delta will be superimposed.

[0073] Then calculate the transformation matrix of the skeleton in mesh space:

[0074] Bone_mesh=Bone_global*BindInverseMatrix

[0075] BindInverseMatrix is ​​the transformation matrix from skeletal space to mesh space, which can be read from the stored .fbx file.

[0076] Step S140 is described in detail below:

[0077] As an optional implementation of the above-mentioned facial bone binding method, step S140 optimizes the first binding result based on the vertex positions of the first facial mesh, facial mesh data, and first controller parameters to obtain a second binding result. This includes: optimizing the first controller parameters based on the vertex positions of the first facial mesh and facial mesh data to obtain second controller parameters; calculating the facial bone position using the second controller parameters to obtain the vertex positions of the second facial mesh; and optimizing the first binding result based on the vertex positions of the second facial mesh and facial mesh data to obtain the second binding result. For example, this implementation may involve fixing the binding weights of the bones and vertices in the facial mesh data, the rotation and translation of the bones in the initial state, etc., as the first binding result; optimizing the first controller parameters based on the vertex positions of the first facial mesh and facial mesh data; calculating the facial bone position using the second controller parameters to obtain the vertex positions of the second facial mesh; fixing the second controller parameters; and optimizing the first binding result based on the vertex positions of the second facial mesh and facial mesh data to obtain the second binding result.

[0078] As an optional implementation of the above-mentioned facial skeleton binding method, step S140 calculates the facial bone position using the second controller parameters and obtains the vertex position of the second facial mesh, including: calculating the facial bone position using linear blending skin (LBS) and the second controller parameters, and obtaining the vertex position of the second facial mesh. For specific implementation details, please refer to the description of the vertex position of the first facial mesh in step S130 above.

[0079] As an optional implementation of the above-mentioned facial skeleton binding method, step S140 optimizes the first controller parameters and obtains the second controller parameters based on the vertex positions of the first facial mesh and the facial mesh data. This includes: optimizing the first controller parameters with the minimum deviation between the vertex positions of the first facial mesh and the actual positions of the facial bones in the facial mesh data as the optimization objective, and then obtaining the second controller parameters. An example of this implementation is:

[0080] Using the facial mesh data as the objective variable V, and assuming the number of mesh vertices is N, solve for each frame of the animation image:

[0081]

[0082] Among them, M i V represents the vertex position of the first facial mesh. i，t This refers to the facial mesh data in frame t.

[0083] Understandably, the above equation can be solved using an optimization method to obtain the optimized controller parameters Input for each frame, i.e., the second controller parameters.

[0084] As an optional implementation of the above-described facial bone binding method, step S140 optimizes the first binding result and obtains the second binding result, including: optimizing the first binding result with the objective of minimizing the deviation between the vertex positions of the second facial mesh and the actual positions of the facial bones in the facial mesh data, and then obtaining the second binding result. For example, this implementation includes:

[0085] Taking the optimization of facial bone positions in the initial state of a character being built as an example, the optimization method is as follows:

[0086]

[0087] T j，t =G(t) j，0 +c tj，t )

[0088] Where B is the number of bones; T is the total number of frames; t j，0 ct represents the local position of the j-th bone in the initial state; j，t d() represents the amount of local position change generated by the skeleton controller at frame t, i.e., time t; d() represents the calculation of similarity; G() represents the conversion of local facial bone positions into global positions based on parent-child relationships.

[0089] The optimal solution to the above equation can be obtained by using methods such as gradient descent, thereby optimizing the position of the facial bones of the character to be constructed in the initial state in the first binding result.

[0090] Because there is a hierarchical relationship between facial bones, meaning that changes in the parent bone will affect changes in the child bone, it is necessary to optimize the bone binding results in layers according to the depth hierarchy of the bones.

[0091] As an optional implementation of the above-mentioned facial bone binding method, step S140 optimizes the first binding result based on the vertex position of the first facial mesh, the facial mesh data, and the first controller parameters to obtain the second binding result, including: obtaining the depth level of the facial bone in the bone tree, performing layered optimization of the first binding result based on the depth level, the vertex position of the first facial mesh, the facial mesh data, and the first controller parameters to obtain the second binding result.

[0092] As an optional implementation of the above-mentioned facial bone binding method, step S140 optimizes the first binding result and obtains the second binding result, including: optimizing at least one of the following in the first binding result: the weights of the facial bones and facial mesh data vertices, the position of the facial bones in the initial state of the character to be constructed, and the rotation of the facial bones in the initial state of the character to be constructed, to obtain the second binding result. An example of this implementation is:

[0093] For methods to optimize the facial bone position in the initial state of a character being constructed, please refer to the above content;

[0094] The method for optimizing the rotation of the facial bones in the initial state of a character is as follows:

[0095]

[0096] R j，t =G(Rtransform(euler) j，0 +ceuler j，t ))

[0097] Rtransform(x, y, z) = R z (z)*R y (y)*R x (x)

[0098] Among them, euler j，0 This represents the Euler angle of local rotation of the facial bone in the initial state for the j-th bone; ceuler j，t R represents the change in Euler angles of the local facial bone rotation generated by the bone controller at frame t, i.e., time t; Rtransform() represents converting the Euler angles into a rotation matrix, R z R y and R x These are the rotation matrices for the z-axis, y-axis, and x-axis elements, respectively.

[0099] The optimal solution to the above equation can be obtained by using methods such as gradient descent, thereby optimizing the facial bone rotation of the character to be constructed in the initial state in the first binding result.

[0100] Similarly, if it is necessary to optimize the weights of the facial bones and facial mesh data vertices in the first binding result, the weights can be directly substituted into the above optimization formula as independent variables, and solved by methods such as gradient descent, to complete the optimization of the weights of the facial bones and facial mesh data vertices in the first binding result.

[0101] Additionally, it should be noted that if Metahuman software is used for 3D modeling, the DNA coefficients can be fine-tuned after obtaining the second binding result. The specific method is as follows:

[0102] Fine-tuning the DNA coefficients is equivalent to fine-tuning matrix D, which is a sparse matrix containing 865*9 rows and 734 columns, where each row can be represented as:

[0103] Y1 = a1x1 + a2x2 + a3x3...

[0104] Among them, the factors that Y1 is related to x are constant, and fine-tuning the DNA coefficient refers to adjusting the value of the coefficient a. Therefore, the fine-tuning problem is transformed into a linear regression problem, which is to solve 865*9 equations.

[0105] Based on sparsity, equations can be divided into multiple sets of equations to improve computational efficiency. For example, if Y1 is related to x1, x2, and x3, find all Y's related to x1, x2, and x3, and then list the x's of these Y's until the set of Y's and the set of x's can encompass the equations formed by the elements in all sets. This constitutes a set. Y represents a certain attribute of the delta quantity of a certain skeleton, namely one of translationX, translationY, translationZ, RotationX, RotationY, RotationZ, ScaleX, ScaleY, and scaleZ. x represents the CRTL coefficient and PSD coefficient.

[0106] 258 Blendshapes (BSs, a set of references used to compose the overall facial expression, which can be used to calculate the overall expression through linear combination) are created, corresponding to meshes with a controller value of 1. BSs can be obtained from 4D photography or created manually. Based on the methods described above for optimizing bone position and rotation, all Y values ​​corresponding to each BS are obtained. After sparse grouping, the coefficients a = {a1, a2, a3…} are calculated using linear regression or least squares, yielding the final DNA coefficients D.

[0107] Please see Figure 2 Based on the same inventive concept, this application also provides a facial bone binding device 200, comprising:

[0108] The initial binding module 210 is used to bind the facial bones and facial mesh data in the initial state of the character to be built, and obtain the first binding result;

[0109] The skeleton controller loading module 220 is used to load the historical skeleton controller and the first controller parameters of the historical skeleton controller.

[0110] The first calculation module 230 is used to calculate the position of the facial bones using the first controller parameters and obtain the vertex position of the first facial mesh.

[0111] The binding result optimization module 240 is used to optimize the first binding result based on the vertex positions of the first facial mesh, the facial mesh data, and the first controller parameters to obtain a second binding result.

[0112] As an optional implementation of the aforementioned facial bone binding device, the binding result optimization module 240 includes:

[0113] The controller parameter optimization unit is used to optimize the first controller parameters and obtain the second controller parameters based on the vertex positions of the first facial mesh and the facial mesh data.

[0114] The second calculation unit is used to calculate the facial bone position using the second controller parameters and obtain the vertex position of the second facial mesh.

[0115] The binding result optimization unit is used to optimize the first binding result and obtain the second binding result based on the vertex positions of the second facial mesh and the facial mesh data.

[0116] As an optional implementation of the above-mentioned facial skeleton binding device, the controller parameter optimization unit is specifically used to: optimize the first controller parameters and obtain the second controller parameters with the goal of minimizing the deviation between the vertex position of the first facial mesh and the actual position of the facial bones in the facial mesh data.

[0117] As an optional implementation of the above-mentioned facial bone binding device, the binding result optimization unit is specifically used to: optimize the first binding result and obtain the second binding result with the goal of minimizing the deviation between the vertex position of the second facial mesh and the actual position of the facial bones in the facial mesh data.

[0118] As an optional embodiment of the above-mentioned facial bone binding device, the facial bone binding device 200 further includes:

[0119] The bone depth level acquisition module is used to acquire the depth level of the facial bones in the bone tree;

[0120] The binding result optimization module 240 is specifically used to: perform layered optimization on the first binding result based on the depth level, the vertex position of the first facial mesh, the facial mesh data, and the first controller parameters, to obtain a second binding result.

[0121] As an optional implementation of the above-mentioned facial skeleton binding device, the first calculation module 230 is specifically used to: calculate the position of the facial skeleton by using linear hybrid skin LBS and calculating the parameters of the first controller, and obtain the vertex position of the first facial mesh.

[0122] The second computing unit is specifically used to: calculate the facial bone position using linear hybrid skin (LBS) and the second controller parameters, and obtain the vertex position of the second facial mesh.

[0123] As an optional embodiment of the above-mentioned facial bone binding device, the binding result optimization module 240 includes at least one of the following units:

[0124] The weight optimization unit is used to optimize the weights of facial bones and facial mesh data vertices in the first binding result.

[0125] The skeleton position optimization unit is used to optimize the facial bone position of the character to be constructed in the initial state in the first binding result;

[0126] The bone rotation optimization unit is used to optimize the facial bone rotation of the character to be built in the initial state in the first binding result.

[0127] Figure 3 This is a schematic diagram of an electronic device provided in an embodiment of this application. (Refer to...) Figure 3 The electronic device 300 includes a processor 310, a memory 320, and a communication interface 330. These components are interconnected and communicate with each other via a communication bus 340 and / or other forms of connection mechanism (not shown).

[0128] The memory 320 includes one or more (only one is shown in the figure), which may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The processor 310 and other possible components may access the memory 320 to read and / or write data therein.

[0129] Processor 310 includes one or more (only one is shown in the figure), which can be an integrated circuit chip with signal processing capabilities. The processor 310 described above can be a general-purpose processor, including a central processing unit (CPU), a microcontroller unit (MCU), a network processor (NP), or other conventional processors; it can also be a special-purpose processor, including a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0130] Communication interface 330 includes one or more (only one is shown in the figure) and can be used to communicate directly or indirectly with other devices to exchange data. For example, communication interface 330 can be an Ethernet interface; it can be a mobile communication network interface, such as an interface for 3G, 4G, or 5G networks; or it can be other types of interfaces with data transmission and reception functions.

[0131] One or more computer program instructions may be stored in the memory 320, and the processor 310 may read and run these computer program instructions to implement the facial bone binding method provided in the embodiments of this application and other desired functions.

[0132] Understandable. Figure 3The structure shown is for illustrative purposes only; the electronic device 300 may also include components that are more advanced than those shown. Figure 3 The more or fewer components shown, or having the same Figure 3 The different configurations shown. Figure 3 The components shown can be implemented using hardware, software, or a combination thereof. For example, electronic device 300 can be a single server (or other device with computing power), a combination of multiple servers, a cluster of a large number of servers, etc., and can be either a physical device or a virtual device.

[0133] This application also provides a computer-readable storage medium storing computer program instructions. These computer program instructions are read and executed by a computer's processor to perform the facial bone rigging method provided in this application. For example, the computer-readable storage medium can be implemented as follows: Figure 3 The memory 320 in the electronic device 300.

[0134] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0135] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0136] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0137] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for binding facial bones, characterized in that, include: In the initial state of the character to be built, the facial bones and facial mesh data are bound together to obtain the first binding result; Load the historical skeleton controller and the first controller parameters of the historical skeleton controller; The facial bone position is calculated using the first controller parameters to obtain the vertex position of the first facial mesh. The first binding result is optimized based on the vertex positions of the first facial mesh, the facial mesh data, and the first controller parameters to obtain a second binding result; The step of optimizing the first binding result based on the vertex positions of the first facial mesh, the facial mesh data, and the first controller parameters to obtain a second binding result includes: fixing the first binding result; optimizing the first controller parameters based on the vertex positions of the first facial mesh and the facial mesh data, with the goal of minimizing the deviation between the vertex positions of the first facial mesh and the actual positions of facial bones in the facial mesh data, to obtain second controller parameters; calculating the facial bone positions using the second controller parameters to obtain second facial mesh vertex positions; fixing the second controller parameters; and optimizing the first binding result based on the vertex positions of the second facial mesh and the facial mesh data, with the goal of minimizing the deviation between the vertex positions of the second facial mesh and the actual positions of facial bones in the facial mesh data, to obtain the second binding result.

2. The facial bone binding method according to claim 1, characterized in that, The first controller parameters are optimized based on the vertex positions of the first facial mesh and the facial mesh data, with the goal of minimizing the deviation between the vertex positions of the first facial mesh and the actual positions of facial bones in the facial mesh data. The second controller parameters are then obtained, including: The optimization objective is to minimize the deviation between the vertex positions of the first facial mesh and the actual positions of facial bones in the facial mesh data. The first controller parameters are then optimized, and the second controller parameters are obtained, including using the facial mesh data as the target quantity. Assuming the number of mesh vertices is For each frame of the animation, solve: ,in, The vertex positions of the first facial mesh; In order to be in The facial mesh data under the frame; These are the controller parameters.

3. The facial bone binding method according to claim 1, characterized in that, The optimization of the first binding result and the acquisition of the second binding result include: The first binding result is optimized to obtain the second binding result, with the goal of minimizing the deviation between the vertex position of the second facial mesh and the actual position of the facial bones in the facial mesh data. The optimization methods for the first binding result include: in, For the number of bones; Total number of frames; Indicates the first The local position of the root skeleton in its initial state; Indicates in Frame, i.e. The amount of local positional change generated by the skeleton controller at any given time; This indicates the calculation of similarity; This indicates that the local position of the facial bones is converted into the global position based on the parent-child relationship.

4. The facial bone binding method according to claim 1, characterized in that, The step of optimizing the first binding result based on the vertex positions of the first facial mesh, the facial mesh data, and the first controller parameters to obtain a second binding result includes: Obtain the depth level of the facial bone in the bone tree, and perform layered optimization on the first binding result based on the depth level, the vertex position of the first facial mesh, the facial mesh data, and the first controller parameters to obtain the second binding result.

5. The facial bone binding method according to claim 1, characterized in that, The step of calculating the facial bone position using the first controller parameters and obtaining the vertex positions of the first facial mesh includes: The facial bone position is calculated using linear hybrid skin (LBS) and the first controller parameters to obtain the vertex position of the first facial mesh. The step of calculating the facial bone position using the second controller parameters and obtaining the vertex positions of the second facial mesh includes: The facial bone position is calculated using linear hybrid skin (LBS) and the second controller parameters to obtain the vertex position of the second facial mesh.

6. The facial bone binding method according to claim 1, characterized in that, The optimization of the first binding result and the acquisition of the second binding result include: Optimize at least one of the following in the first binding result: the weight of the facial bones and facial mesh data vertices, the position of the facial bones in the initial state of the character to be constructed, and the rotation amount of the facial bones in the initial state of the character to be constructed, to obtain the second binding result.

7. A facial bone binding device, characterized in that, include: The initial binding module is used to bind facial bones and facial mesh data in the initial state of the character to be built, and obtain the first binding result; A skeleton controller loading module is used to load a historical skeleton controller and the first controller parameters of the historical skeleton controller. The first calculation module is used to calculate the position of the facial bones using the parameters of the first controller, and to obtain the vertex position of the first facial mesh. The binding result optimization module is used to optimize the first binding result based on the vertex positions of the first facial mesh, the facial mesh data, and the first controller parameters to obtain a second binding result; The binding result optimization module is specifically used for: fixing the first binding result; optimizing the first controller parameters based on the vertex positions of the first facial mesh and the facial mesh data, with the goal of minimizing the deviation between the vertex positions of the first facial mesh and the actual positions of the facial bones in the facial mesh data, and obtaining second controller parameters; calculating the facial bone positions using the second controller parameters, and obtaining the vertex positions of the second facial mesh; fixing the second controller parameters; optimizing the first binding result based on the vertex positions of the second facial mesh and the facial mesh data, with the goal of minimizing the deviation between the vertex positions of the second facial mesh and the actual positions of the facial bones in the facial mesh data, and obtaining the second binding result.

8. An electronic device, characterized in that, include: A processor, a memory, and a communication bus, wherein the processor and the memory communicate with each other via the communication bus; The memory stores program instructions that can be executed by the processor, and the processor can execute the method as described in any one of claims 1 to 6 by calling the program instructions.

9. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions, which, when executed by a computer, cause the computer to perform the method as described in any one of claims 1 to 6.

Images