Facial skeleton binding automatic migration method and system based on fbx file

Abstract

The present invention discloses an automatic migration method and system for facial skeleton binding based on fbx files, relates to the field of animation production technology based on artificial intelligence, and aims to solve the problem of high cost and low efficiency of facial skeleton binding for new models in the prior art. The key points of the technical solution are to fit the first model with existing skin binding to the second model to be skin bound, obtain a third model, determine the skeleton data and skin information of the second model according to the mapping relationship between the second model and the third model, the skeleton data of the third model and the skin information of the third model, and finally generate the fbx file of the second model to complete the skin binding of the second model. The present invention achieves the automatic migration of existing facial skeleton binding to other models, improves the efficiency of new model binding and reduces labor costs.

Description

Technical Field

[0001] The present invention relates to the field of animation production technology based on artificial intelligence, and in particular to a facial skeleton binding automatic migration method and system based on fbx files. Background Art

[0002] The background technologies involved in the present invention mainly include facial binding, fbx file parsing, 3D landmark detection, and graph theory. Most of the existing facial skeleton binding is done manually, and the main process includes three steps: establishing facial skeleton points, determining the facial skin range, and determining the facial skin weight. Facial skeleton binding of different models cannot be migrated due to different model shapes and model point surfaces, and binding a model requires a lot of time, resulting in high binding costs. Summary of the invention

[0003] The purpose of the present invention is to provide a facial skeleton binding automatic migration method and system based on fbx files, which has the effect of improving the efficiency of new model skinning binding and reducing labor costs.

[0004] The above-mentioned object of the present invention is achieved through the following technical solutions:

[0005] A facial skeleton binding automatic migration method based on fbx files, comprising the following steps:

[0006] Acquire vertex mesh data, skeleton data and skinning information of a first model, and acquire vertex mesh data of a second model, wherein the first model is a three-dimensional model generated after deformation and skinning based on a flame fixed template, and the second model is a three-dimensional model that is not bound by skinning;

[0007] Based on the vertex mesh data of the first model and the vertex mesh data of the second model, respectively obtain key points of the first model and key points of the second model, determine a mapping relationship between the key points of the first model and the key points of the second model based on relative positions of the key points of the first model and the key points of the second model on the face, fit the first model to the second model according to the key points and the mapping relationship, obtain a third model and a mapping relationship between the second model and the third model, wherein the third model includes corresponding vertex mesh data, skeleton data and skinning information;

[0008] Determine the skeleton data of the second model according to the mapping relationship between the second model and the third model and the skeleton data of the third model;

[0009] Determining skin information of the second model according to a mapping relationship between the second model and the third model and skin information of the third model;

[0010] Generate an fbx file corresponding to the second model according to the vertex mesh data, skeleton data and skin information of the second model to complete skin binding;

[0011] The vertex mesh data includes a number of vertex coordinates, the skeleton data includes a number of skeleton point coordinates, and the skinning information includes a number of vertex indexes and corresponding skinning weights.

[0012] The present invention is further configured as follows: the key points of obtaining the second model include:

[0013] Rendering the second model into a two-dimensional space, and obtaining a two-dimensional image corresponding to the second model;

[0014] Performing key point detection on the two-dimensional image to obtain a plurality of two-dimensional key points;

[0015] A mapping relationship between the second model and the two-dimensional image is obtained, and key points of the second model are determined according to the mapping relationship between the second model and the two-dimensional image and the two-dimensional key points.

[0016] The present invention is further configured as follows: the acquiring of the mapping relationship between the second model and the two-dimensional image, and determining the key points of the second model according to the mapping relationship between the second model and the two-dimensional image and the two-dimensional key points, comprises:

[0017] Obtaining the key point pixel position of the two-dimensional key point;

[0018] After the second model is rendered into the two-dimensional space, the corresponding vertex pixel positions of each vertex of the second model in the two-dimensional image are tracked as the mapping relationship between the second model and the two-dimensional image;

[0019] Detect the vertex pixel position closest to the key point pixel position, combine the mapping relationship between the second model and the two-dimensional image, and use the vertex coordinates corresponding to the detection result as the key point coordinates of the second model.

[0020] The present invention is further configured as follows: fitting the first model to the second model according to the key points and the mapping relationship to obtain a third model and a mapping relationship between the second model and the third model includes:

[0021] Tracking the relative position coordinates of key points during the fitting process of the first model, and inputting the relative position coordinates and the key point coordinates of the second model into a loss function;

[0022] When the loss function reaches a minimum, the relative position coordinates at this time are used as the key point coordinates of the third model to determine the third model.

[0023] The present invention is further configured as follows: determining the skeleton data of the second model according to the mapping relationship between the second model and the third model and the skeleton data of the third model comprises:

[0024] Detect a set of vertex coordinates in the second model that are closest to each skeleton point of the third model as the skeleton data of the second model.

[0025] The present invention is further configured as follows: determining the skin information of the second model according to the mapping relationship between the second model and the third model and the skin information of the third model comprises:

[0026] Detecting a vertex coordinate set in the second model that is closest to vertices corresponding to each vertex index of the third model as the vertex index of the skin information of the second model, confirming the skin weight corresponding to the vertex index of the skin information of the second model, and obtaining the skin information of the second model;

[0027] According to the one-to-one correspondence between the vertex index of the second model skin information and the vertex index of the third model skin information, the skin weight of the second model skin information is the same as the skin weight of the third model skin information.

[0028] The present invention is further configured to include:

[0029] The vertex mesh data includes information of a face composed of a number of vertex indices, and the fbx file corresponding to the second model includes face constraint values, and the face constraint values ​​are used to limit the skin diffusion range of the bone points.

[0030] The second object of the present invention is achieved through the following technical solutions:

[0031] An automatic migration system for facial bone binding based on fbx files, including:

[0032] A preparation module, used to obtain vertex mesh data, skeleton data and skinning information of the first model from the fbx file corresponding to the first model, and to obtain vertex mesh data of the second model, wherein the first model is a three-dimensional model generated after deformation and skinning based on a flame fixed template, and the second model is a three-dimensional model without skinning and binding;

[0033] A fitting module, used for acquiring key points of the first model and key points of the second model, determining a mapping relationship between the key points of the first model and the key points of the second model, fitting the first model to the second model according to the key points and the mapping relationship, obtaining a third model and a mapping relationship between the second model and the third model, wherein the third model includes corresponding vertex mesh data, bone data and skinning information;

[0034] A determination module, configured to determine the skeleton data of the second model according to the mapping relationship between the second model and the third model and the skeleton data of the third model; and to determine the skin information of the second model according to the mapping relationship between the second model and the third model and the skin information of the third model;

[0035] A generation module is used to generate an fbx file corresponding to the second model according to the vertex mesh data, skeleton data and skin information of the second model.

[0036] The third object of the present invention is achieved through the following technical solutions:

[0037] An electronic device comprises a memory and a processor, wherein the memory stores a computer program executable by the processor, and the processor implements the above-mentioned automatic migration method of facial bone binding based on fbx files when executing the computer program.

[0038] The fourth object of the present invention is achieved by the following technical solutions:

[0039] A storage medium includes a stored program, and when the program is running, the device where the storage medium is located is controlled to execute the above-mentioned facial skeleton binding automatic migration method based on fbx files.

[0040] In summary, the beneficial technical effects of the present invention are:

[0041] The present invention proposes an automatic migration method and system for facial bone binding based on fbx files, which fits a first model with existing skin binding to a second model to be skin bound to obtain a third model, determines the bone data and skin information of the second model according to the mapping relationship between the second model and the third model, the bone data of the third model and the skin information of the third model, and finally generates an fbx file of the second model to complete the skin binding of the second model. The present invention achieves the automatic migration of existing facial bone binding to other models, improves the efficiency of new model binding and reduces labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is an overall flow chart of Embodiment 1 of the present invention;

[0043] Figure 2 It is a structural block diagram of the second embodiment of the present invention. DETAILED DESCRIPTION

[0044] Embodiment 1

[0045] The present invention discloses an automatic migration method for facial bone binding based on fbx files. The execution subject of the method can be a terminal, a computer, a server, etc., and can be applied to any stage of model development, testing or revision. Figure 1 As shown, the method comprises the following steps:

[0046] S1. Obtain vertex mesh data, skeleton data and skinning information of the first model from the fbx file corresponding to the first model, and obtain vertex mesh data of the second model, wherein the first model is a three-dimensional model generated after deformation and skinning based on a flame fixed template, and the second model is a three-dimensional model that is not skinned and bound.

[0047] Among them, the vertex mesh data includes a number of vertex coordinates, including two aspects: vertices and faces; the bone data includes a number of bone point coordinates; and the skin information includes a number of vertex indices and corresponding skin weights.

[0048] The first model can be constructed by using a model with the same topological structure as in Flame. The format can be obj, ply, etc. The model contains basic information such as vertices, faces, maps, texture coordinates, etc. On this basis, skinning and binding are performed to build an fbx file. This process can be completed through software such as Maya and Blender.

[0049] In actual operation, the flame model needs to be deformed, that is, based on a fixed underlying template, a series of linear combinations of deformations (blendshape) are used to generate a new 3D model. The specific operation is as follows:

[0050] Let flame template be T, which represents the function of shape deformation in represents the shape factor, Represents different shape deformations.

[0051] Functions that represent case variations in represents the shape factor, Represents different shape deformations.

[0052] The model can be expressed as:

[0053] Based on the above deformation, the vertex mesh data of the first model is obtained from the fbx file of the first model. The vertex mesh data includes two aspects: vertices and faces. The vertex data is v represents a point in 3D space; the surface data of the first model is in Indicates the index of the nth vertex that constitutes the face. Get the skeleton data of the first model Represents a 3D coordinate. Get the skin information of the first model where w = {ind v :weight v}, ind v is the vertex index, weight v is the weight of the point. This step can be done through fbx parsing technology. FBX SDK (Software Developer Kit) is a set of tools developed by Autodesk for C++ and Python to facilitate the processing of Fbx files. Without opening the Fbx file, the necessary modifications and processing of the nodes in the Fbx file can be performed.

[0054] The second model is a character model that needs to be skinned and bound. The format can be obj, ply, etc. The second model can be obtained by downloading or by building it with software such as maya and blender. Use OpenGL to load the texture, vertex, and texture coordinates in the second model and obtain the vertex data in the second model. noodle information.

[0055] S2. Obtain key points of the first model and key points of the second model, determine the mapping relationship between the key points of the first model and the key points of the second model, fit the first model to the second model according to the key points and the mapping relationship, obtain a third model and a mapping relationship between the second model and the third model, wherein the third model includes corresponding vertex mesh data, bone data and skinning information.

[0056] S3. Determine the skeleton data of the second model according to the mapping relationship between the second model and the third model and the skeleton data of the third model.

[0057] S4. Determine the skin information of the second model according to the mapping relationship between the second model and the third model and the skin information of the third model.

[0058] S5. Generate an fbx file corresponding to the second model according to the vertex mesh data, skeleton data and skin information of the second model, and complete the skin binding of the second model.

[0059] In the above step S2, the key points of the first model are obtained respectively And the key points of the second model Represents a 3D coordinate. Among them, the key point L of the first model a It can be obtained directly through flame. Get the key point L of the second model b The following steps are involved:

[0060] S21. Render the second model into a two-dimensional space, and obtain a two-dimensional image p corresponding to the second model.

[0061] S22: Detect key points of the two-dimensional image to obtain a number of two-dimensional key points.

[0062] S23: Obtain a mapping relationship between the second model and the two-dimensional image, and determine the key points of the second model according to the mapping relationship between the second model and the two-dimensional image and the two-dimensional key points.

[0063] Step S23 includes the following steps:

[0064] S231, obtaining the key point pixel position of the two-dimensional key point.

[0065] S232 , after the second model is rendered into the two-dimensional space, the corresponding vertex pixel position of each vertex of the second model in the two-dimensional image is tracked.

[0066] S233, detecting the vertex pixel position closest to the key point pixel position, and using the vertex coordinates corresponding to the detection result as the key point coordinates.

[0067] In some embodiments, the key point L of the second model b The specific operation of obtaining is: rendering the second model into a two-dimensional space to form a two-dimensional image p, using dlib or other key point detection tools to detect face key points on the two-dimensional image p, and obtaining Among them l p Represents the pixel position in the two-dimensional image p, and records the pixel position after rendering to the two-dimensional image V b The vertex pixel position of the jth vertex in the image For each By solving Find the corresponding Then determine by index j

[0068] In the above step S2, according to the key points and the mapping relationship, fitting the first model to the second model to obtain the third model and the mapping relationship between the second model and the third model includes the following steps:

[0069] S24, tracking the relative position coordinates of the key points in the first model fitting process, and inputting the relative position coordinates and the key point coordinates of the second model into the loss function. When the loss function reaches a minimum, the relative position coordinates at this time are used as the key point coordinates of the third model to determine the third model.

[0070] In some embodiments, the specific operation of the above steps is: fitting the first model to the second model to obtain a third model, the vertex data of the third model is The skeleton data of the third model is Surface data of the third model This step minimizes L by using flame fitting technology. a , L b 3D landmark loss between, calculate the loss function where w n is the landmark weight. In the specific implementation, the mouth area is set to 10 and other areas are set to 1. When the loss function is minimized, the third model is obtained.

[0071] The above step S3 includes the following steps: detecting a set of vertex coordinates in the second model that are closest to each skeleton point of the third model as the skeleton data of the second model.

[0072] In some implementations, the specific operation of step S3 is: obtaining the index of the corresponding bone point of the third model from the second model. Since the mapping relationship between the second model and the third model has been determined in step 2, this step can be determined by detecting the index of the vertex in the second model that is closest to the bone point in the third model. a′ The i-th element in Solution Obtain the corresponding second model's skeleton data O b .

[0073] The above step S4 includes the following steps: detecting a vertex coordinate set in the second model that is closest to the vertices corresponding to each vertex index of the third model as the vertex index of the skin information of the second model, confirming the skin weight corresponding to the vertex index of the skin information of the second model, and obtaining the skin information of the second model;

[0074] According to the one-to-one correspondence between the vertex index of the second model skin information and the vertex index of the third model skin information, the skin weight of the second model skin information is the same as the skin weight of the third model skin information.

[0075] In some embodiments, the specific operation of step S4 is: by detecting the nearest point of the vertex in the second model to the vertex index of the third model skin information, the weight of the found point is used as the new vertex weight in the second model. It should be noted that the third model has more vertices than the second model.

[0076] In the above step S5, the skinning information and skeleton data of the second model are written into the fbx file through the FBX SDK technology to generate the final fbx file with skin binding.

[0077] As an optimization solution, based on the above implementation, the fbx file corresponding to the second model is added with face constraint values. The vertex mesh data includes face information composed of several vertex indices, that is, The face constraint value is used to limit the skin diffusion range of the bone point. Because the points of some lips are very close, the skin point found by the nearest point is likely to have the upper and lower lips cross, which is not allowed. Therefore, a face constraint is added on this basis. The skin of a certain bone point can only be diffused within n faces. n is the face constraint value, which is generally set to 2 according to experience.

[0078] The above embodiment provides a method for automatic migration of facial bone binding based on fbx files. By obtaining the key points of the first model, the key points of the second model and the mapping relationship between the two, the first model can be fitted to the second model, and the third model and the mapping relationship between the third model and the second model can be obtained. At this point, the third model obtains all the features of the first model, and the relative position of the third model and the second model is determined. The third model is used as an intermediate model to connect the first model and the second model. Through calculation, the second model learns the bone data and skinning information from the third model, thereby completing the process of migrating the facial bone binding of the first model to the second model. The above process establishes an association between the first model and the second model through the third model, and can quickly migrate the facial binding of the existing fbx file to the new model, and automatically generate an fbx file with skin binding, which solves the problem that the existing technology requires a lot of manpower and time for binding a new model.

[0079] Embodiment 2

[0080] The present invention discloses a facial skeleton binding automatic migration system based on fbx files, referring to Figure 2 ,include:

[0081] A preparation module is used to obtain vertex mesh data, skeleton data and skinning information of the first model from the fbx file corresponding to the first model, and to obtain vertex mesh data of the second model, wherein the first model is a three-dimensional model generated after deformation and skinning based on a flame fixed template, and the second model is a three-dimensional model that is not bound by skinning;

[0082] A fitting module is used to obtain key points of the first model and key points of the second model, determine the mapping relationship between the key points of the first model and the key points of the second model, fit the first model to the second model according to the key points and the mapping relationship, obtain a third model and a mapping relationship between the second model and the third model, and the third model includes corresponding vertex mesh data, bone data and skinning information;

[0083] A determination module, used to determine the skeleton data of the second model according to the mapping relationship between the second model and the third model and the skeleton data of the third model; and to determine the skin information of the second model according to the mapping relationship between the second model and the third model and the skin information of the third model;

[0084] The generation module is used to generate an fbx file corresponding to the second model according to the vertex mesh data, skeleton data and skin information of the second model.

[0085] Embodiment 3

[0086] An electronic device includes a memory and a processor, wherein the memory stores a computer program executable by the processor, and the processor implements the above method embodiment when executing the computer program. The specific implementation method and technical effect are similar and will not be repeated here.

[0087] Embodiment 4

[0088] A storage medium includes a stored program, and when the program is running, the device where the storage medium is located is controlled to execute the above method embodiment.

[0089] The embodiments of this specific implementation method are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A facial skeleton binding automatic migration method based on fbx files, characterized in that: The following steps are involved: Acquire vertex mesh data, skeleton data and skinning information of a first model, and acquire vertex mesh data of a second model, wherein the first model is a three-dimensional model generated after deformation and skinning based on a flame fixed template, and the second model is a three-dimensional model that is not bound by skinning; Based on the vertex mesh data of the first model and the vertex mesh data of the second model, respectively obtain key points of the first model and key points of the second model, determine a mapping relationship between the key points of the first model and the key points of the second model based on relative positions of the key points of the first model and the key points of the second model on the face, fit the first model to the second model according to the key points and the mapping relationship, obtain a third model and a mapping relationship between the second model and the third model, wherein the third model includes corresponding vertex mesh data, skeleton data and skinning information; Determine the skeleton data of the second model according to the mapping relationship between the second model and the third model and the skeleton data of the third model; Determining skin information of the second model according to a mapping relationship between the second model and the third model and skin information of the third model; Generate an fbx file corresponding to the second model according to the vertex mesh data, skeleton data and skin information of the second model to complete skin binding; The vertex mesh data includes a number of vertex coordinates, the skeleton data includes a number of skeleton point coordinates, and the skinning information includes a number of vertex indexes and corresponding skinning weights.

2. The facial skeleton binding automatic migration method based on fbx file according to claim 1 is characterized in that: The obtaining of the key points of the second model includes: Rendering the second model into a two-dimensional space, and obtaining a two-dimensional image corresponding to the second model; Performing key point detection on the two-dimensional image to obtain a plurality of two-dimensional key points; A mapping relationship between the second model and the two-dimensional image is obtained, and key points of the second model are determined according to the mapping relationship between the second model and the two-dimensional image and the two-dimensional key points.

3. The facial skeleton binding automatic migration method based on fbx file according to claim 2 is characterized in that: The acquiring a mapping relationship between the second model and the two-dimensional image, and determining the key points of the second model according to the mapping relationship between the second model and the two-dimensional image and the two-dimensional key points, includes: Obtaining the key point pixel position of the two-dimensional key point; After the second model is rendered into the two-dimensional space, the corresponding vertex pixel positions of each vertex of the second model in the two-dimensional image are tracked as the mapping relationship between the second model and the two-dimensional image; Detect the vertex pixel position closest to the key point pixel position, combine the mapping relationship between the second model and the two-dimensional image, and use the vertex coordinates corresponding to the detection result as the key point coordinates of the second model.

4. The facial skeleton binding automatic migration method based on fbx file according to claim 1 is characterized in that: The step of fitting the first model to the second model according to the key points and the mapping relationship to obtain a third model and a mapping relationship between the second model and the third model includes: Tracking the relative position coordinates of key points during the fitting process of the first model, and inputting the relative position coordinates and the key point coordinates of the second model into a loss function; When the loss function reaches a minimum, the relative position coordinates at this time are used as the key point coordinates of the third model to determine the third model.

5. The facial skeleton binding automatic migration method based on fbx file according to claim 1 is characterized in that: Determining the skeleton data of the second model according to the mapping relationship between the second model and the third model and the skeleton data of the third model includes: Detect a set of vertex coordinates in the second model that are closest to each skeleton point of the third model as the skeleton data of the second model.

6. The facial skeleton binding automatic migration method based on fbx file according to claim 1 is characterized in that: The determining the skin information of the second model according to the mapping relationship between the second model and the third model and the skin information of the third model includes: Detecting a vertex coordinate set in the second model that is closest to vertices corresponding to each vertex index of the third model as the vertex index of the skin information of the second model, confirming the skin weight corresponding to the vertex index of the skin information of the second model, and obtaining the skin information of the second model; According to the one-to-one correspondence between the vertex index of the second model skin information and the vertex index of the third model skin information, the skin weight of the second model skin information is the same as the skin weight of the third model skin information.

7. The facial skeleton binding automatic migration method based on fbx file according to any one of claims 1 to 6, characterized in that: Also includes: The vertex mesh data includes information of a face composed of a number of vertex indices, and the fbx file corresponding to the second model includes face constraint values, and the face constraint values ​​are used to limit the skin diffusion range of the bone points.

8. A facial skeleton binding automatic migration system based on fbx files, characterized in that: include: A preparation module, used to obtain vertex mesh data, skeleton data and skinning information of the first model from the fbx file corresponding to the first model, and to obtain vertex mesh data of the second model, wherein the first model is a three-dimensional model generated after deformation and skinning based on a flame fixed template, and the second model is a three-dimensional model without skinning and binding; A fitting module, used for acquiring key points of the first model and key points of the second model, determining a mapping relationship between the key points of the first model and the key points of the second model, fitting the first model to the second model according to the key points and the mapping relationship, obtaining a third model and a mapping relationship between the second model and the third model, wherein the third model includes corresponding vertex mesh data, bone data and skinning information; A determination module, configured to determine the skeleton data of the second model according to the mapping relationship between the second model and the third model and the skeleton data of the third model; and to determine the skin information of the second model according to the mapping relationship between the second model and the third model and the skin information of the third model; A generation module is used to generate an fbx file corresponding to the second model according to the vertex mesh data, skeleton data and skin information of the second model.

9. An electronic device, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program executable by the processor, and when the processor executes the computer program, the method for automatic migration of facial bone binding based on fbx files according to any one of claims 1 to 7 is implemented.

10. A storage medium, characterized in that: The storage medium includes a stored program, and when the program is running, the device where the storage medium is located is controlled to execute the facial skeleton binding automatic migration method based on fbx files according to any one of claims 1 to 7.

Images