Rapid geometric modeling method and device

Abstract

The invention discloses a rapid geometric modeling method and device, and the method comprises the steps: extracting three-dimensional symbol distance field information of all frames of a human body template motion sequence, and carrying out the processing of the three-dimensional symbol distance field information to obtain a data matrix; extracting a human body template trunk skeleton of the first frame of human body action; performing parameter initialization on the data matrix based on human body parts according to a human body template trunk skeleton; inputting the parameters into a reconstruction model based on a quadratic kernel gate function to obtain symbol distance fields for reconstruction of all frames, optimizing and updating the parameters of the model by using an expected value maximum algorithm, and calculating mean square errors of the symbol distance fields for reconstruction of all frames and an original symbol distance field; reconstructing a symbol distance field of each frame by using a parameter corresponding to a minimum mean square error value of the mean square errors; and reconstructing a three-dimensional grid of the human body template for the reconstructed symbol distance fields of all the frames. According to the method, three-dimensional human body dynamic geometry can be effectively modeled, so that the understanding of human body actions has time-space induction generalization.

Description

Technical Field

[0001] The invention relates to the fields of computer graphics, three-dimensional model reconstruction, motion capture and digital human technology, and in particular to a fast geometric modeling method and device. Background Art

[0002] Rapid geometric modeling is one of the important foundations for the development of digital humans and three-dimensional reconstruction technology. It aims to build rapid modeling for dynamic three-dimensional human body data, laying the foundation for characteristic expression for efficient realization of digital human driving and multimodal control.

[0003] Dynamic human body modeling is the key to three-dimensional human body drive research. The existing local shape function expression method of implicit neural template based on Gaussian kernel is applied to the reconstruction of human body template; there is also modeling of dynamic data based on neural radiation field, but these algorithms cannot express the characteristics of dynamic human body template, and therefore are not convenient for subsequent multimodal drive research. In addition, the existing dynamic geometric human body representation based on quadratic kernel model relies on neural network learning and has poor robustness, but it is difficult to withstand the huge amount of dynamic information based on traditional machine learning algorithms. It is necessary to be robust and not rely on deep learning, but also to break through its speed barrier. There is currently a lack of practical solutions. Summary of the invention

[0004] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0005] To this end, the present invention proposes a fast geometric modeling method to overcome the current defects.

[0006] Another object of the present invention is to provide a fast geometric modeling device.

[0007] To achieve the above object, the present invention provides a fast geometric modeling method, comprising:

[0008] Extract the three-dimensional signed distance field information of all frames of the human body template action sequence and process it to obtain a data matrix;

[0009] Extract the human body template torso skeleton of the first frame of human action;

[0010] Determine the number of models to be modeled;

[0011] Initializing the parameters of the data matrix based on the human body parts according to the human body template to initialize the modeling signed distance field to obtain the original signed distance field;

[0012] Use the expected value maximization algorithm to optimize and update the parameters of the model;

[0013] Input the parameters into the reconstruction model based on the quadratic kernel gate function to obtain the reconstructed signed distance fields of all frames, and calculate the mean square error between the reconstructed signed distance fields of all frames and the original signed distance fields;

[0014] Repeatedly iteratively update the model parameters, and reconstruct the symbol distance fields of all frames using the parameters corresponding to the minimum mean square error value of the mean square error;

[0015] The 3D mesh of the human body template is reconstructed using the signed distance fields reconstructed for all frames using the marching cubes algorithm.

[0016] The fast geometric modeling method of the embodiment of the present invention may also have the following additional technical features:

[0017] In one embodiment of the present invention, the data matrix extracted frame by frame includes a signed distance field with a resolution of N0 in T frames and corresponding positions, arranged in rows with a number of N=N0. 3 ×T, a data matrix with 5 columns.

[0018] In one embodiment of the present invention, the human body template torso skeleton is directly obtained from the input human body template parameters, and the human body template vertices are connected in pairs to form a human body skeleton.

[0019] In one embodiment of the present invention, if the number of simulated modeling models is K, the following conditions must be met: the spatial signed distance field is evenly divided into k1×k1×k1 cubes, and k1 3 =K, where k1 needs to be an integer divider of the resolution value N0.

[0020] In one embodiment of the present invention, the initialization modeling signed distance field includes:

[0021] Evenly extract 50 coordinate points on each segment of the extracted skeleton;

[0022] Traverse each cube block. If the points on all the line segments of the human skeleton are not within its range, it will be divided into a group. The remaining blocks will look for sampling points on the skeleton line segment closest to it, and those that match the same segment will be grouped together.

[0023] After the position grouping of the first frame is completed, all the remaining frames are grouped in the same way, and the same grouping in all frames is regarded as a dynamic grouping;

[0024] After matching, the initial dynamic grouping is obtained into K groups, and the mean vector, covariance matrix and weight μ of the K group data are obtained. j ,Σ j ,α j , j = 1, 2, ..., K;

[0025] The parameter μ j ,Σj ,α j , j = 1, 2, ..., K into the five-dimensional quadratic kernel gate function reconstruction formula Initialize the entire signed distance field where δ represents the position, g(δ; α j ,μ j ,Σ j ) represents the quadratic kernel gate function, m(δ; μ j ,Σ j ) represents the quadratic kernel conditional mean.

[0026] In one embodiment of the present invention, the method of optimizing and updating the parameters of the model using the expected value maximization algorithm includes:

[0027] Update the secondary kernel posterior probability Qt ij ,i=1,2,...,N,j=1,2,...,K;

[0028] Update Parameters

[0029] In one embodiment of the present invention, the mean square error is the mean square difference between the values ​​of the original and reconstructed signed distance fields at the same position in each frame.

[0030] In one embodiment of the present invention, the model parameters are iteratively updated 50 times.

[0031] In one embodiment of the present invention, the three-dimensional mesh of the human body template is reconstructed by using the signed distance field reconstructed for each frame using the marching cubes algorithm, comprising:

[0032] For each frame of the reconstructed signed distance field, for each edge, if the signs of the two connected vertices are different, interpolate on this edge, calculate the position where the interpolation value is 0, and place the vertex;

[0033] For each frame of the reconstructed signed distance field, triangle patches are constructed between the vertices.

[0034] To achieve the above object, the present invention provides a fast geometric modeling device, comprising:

[0035] A data processing module is used to extract the three-dimensional symbol distance field information of all frames of the human body template action sequence and process it to obtain a data matrix;

[0036] A skeleton extraction module is used to extract the torso skeleton of the human body template of the first frame of human action;

[0037] A model quantity determination module is used to determine the number of modeling models;

[0038] A parameter initialization module is used to initialize the parameters of the data matrix based on the human body parts according to the human body template torso skeleton, so as to initialize the modeling signed distance field to obtain the original signed distance field;

[0039] A model parameter updating module is used to optimize and update the parameters of the model using the expected value maximization algorithm;

[0040] A mean square error calculation module is used to input parameters into a reconstruction model based on a quadratic kernel gate function to obtain a reconstructed signed distance field for each frame, and calculate the mean square error between the reconstructed signed distance fields of all frames and the original signed distance fields;

[0041] A distance field reconstruction module, used for repeatedly iterating and updating the model parameters, and reconstructing the distance fields of all frame symbols using the parameters corresponding to the minimum mean square error value of the mean square error;

[0042] The human body mesh modeling module is used to reconstruct the three-dimensional mesh of the human body template using the signed distance field reconstructed from all frames using the marching cubes algorithm.

[0043] The fast geometric modeling method and device of the embodiment of the present invention adopts the maximum expected value algorithm to learn the mean vector, covariance matrix, weight and other parameters of the five-dimensional quadratic kernel model, which can effectively model the three-dimensional human body dynamic geometry and make the understanding of human body movements have spatiotemporal inductive generalization.

[0044] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0046] Figure 1 is a flow chart of a fast geometric modeling method according to an embodiment of the present invention;

[0047] Figure 2 is a human body parameterized template motion capture vertex and skeleton graph according to an embodiment of the present invention;

[0048] Figure 3 4 is a structural diagram of a rapid geometric modeling device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0050] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0051] The following describes the rapid geometric modeling method and device proposed according to the embodiments of the present invention with reference to the accompanying drawings.

[0052] Figure 1 is a flow chart of a fast geometric modeling method according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0053] S1, extract the three-dimensional signed distance field information of all frames of the human body template action sequence, and process it to obtain a data matrix.

[0054] Specifically, the signed distance field of each frame of human body template is extracted by TSDF method;

[0055] Specifically, the action template has a total of T frames, each frame is a three-dimensional signed distance field with a spatial voxel grid with a resolution of N0, wherein each voxel stores a signed distance value, indicating the distance from the center of the voxel to the surface of the object.

[0056] Extract the three-dimensional signed distance field position information and signed distance value frame by frame. Extract the three-dimensional position coordinates, time frame number, and signed distance value of each voxel. Arrange the extracted data vertically into rows with the number of N=N0 3 ×T, a data matrix with 5 columns.

[0057] S2, extract the human body template torso skeleton of the first frame of human action.

[0058] Specifically, the human skeleton is directly obtained from the input human template parameters, including the three-dimensional coordinates of 23 vertices such as Figure 2 As shown, the vertices are connected to form a human skeleton, and the skeleton line segments are as follows Figure 2 There are 23 segments shown.

[0059] S3, determine the number of modeling models.

[0060] If the number of models to be modeled is K, the following conditions must be met: the spatial signed distance field is evenly divided into k1×k1×k1 cubes, and k1 3 = K. Where k1 needs to be an integer divider of the resolution value N0.

[0061] S4, initializing the parameters of the data matrix based on the human body parts according to the human body template to initialize the modeling signed distance field to obtain the original signed distance field.

[0062] Specifically, the signed distance field modeled by rapid initialization in step S4 specifically includes the following steps:

[0063] Evenly extract 50 coordinate points on each segment of the extracted skeleton;

[0064] Traverse each cubic block, if all the points on the line segments of the human skeleton are not within its range, it will be divided into a group; the remaining blocks will look for sampling points on the skeleton line segment closest to it, and those matched to the same segment will be grouped; specifically, it includes: for each cubic block, traverse each skeleton segment in the 23 segments, and determine whether the 50 points sampled in each segment are within the block, that is, whether the coordinates of the sampling points are distributed in the three-dimensional coordinate range of the cube in terms of spatial position; if all the points on the line segments of the human skeleton are not within its range, it will be divided into a group; if the cubic block matches multiple sampling points, then match the segment corresponding to the sampling point closest to it through the mean square error calculation of the distance; the cubic blocks matched to the same segment are grouped.

[0065] After the position grouping of the first frame is completed, all the remaining frames are grouped in the same way, and the same grouping in all frames is regarded as a dynamic grouping;

[0066] After matching, the initial dynamic grouping is obtained into K groups, and the mean vector, covariance matrix and weight μ of the K group data are obtained. j ,Σ j ,α j , j = 1, 2, ..., K;

[0067] The parameter μ j ,Σ j ,α j , j = 1, 2, ..., K into the five-dimensional quadratic kernel gate function reconstruction formula Initialize the entire signed distance field where δ represents the position, g(δ; α j ,μ j ,Σ j ) represents the quadratic kernel gate function, m(δ; μ j ,Σ j ) represents the quadratic kernel conditional mean.

[0068] S5, optimize and update the parameters of the model using the expected value maximization algorithm.

[0069] Step E: Update the secondary nuclear posterior probability Qt ij ,i=1,2,...,N,j=1,2,...,K;

[0070] Step M: Update parameters

[0071] S6, inputting the parameters into the reconstruction model based on the quadratic kernel gate function to obtain the reconstructed signed distance fields of all frames, and calculating the mean square error between the reconstructed signed distance fields of all frames and the original signed distance fields.

[0072] Specifically, the parameter μ of the rth iteration is rj ,Σ rj ,α rj , j = 1, 2, ..., K is input into the gate function reconstruction formula In , the model reconstructs the signed distance field for each frame based on these parameters and the kernel function. A signed distance field (SDF) is a function that assigns a value to each point in space that represents the distance from the point to the nearest surface, and the value has a sign to distinguish whether the point is inside or outside the surface.

[0073] Among them, the mean square error is the square mean of the difference between the original symbol distance field and the reconstructed symbol distance field at the same position in each frame, that is, the mean square error of the rth iteration is defined as:

[0074]

[0075] S7, repeatedly iteratively updating the model parameters, and reconstructing the symbol distance fields of all frames using the parameters corresponding to the minimum mean square error value of the mean square error.

[0076] In this embodiment, the number of parameter update iterations is 50.

[0077] The steps of reconstructing each frame of the symbol distance field using the parameters corresponding to the minimum mean square error value of the mean square error include:

[0078] The iteration with the smallest mean square error is r m ,Right now The corresponding parameters are So the reconstructed signed distance field is

[0079]

[0080] S8, reconstructing the 3D mesh of the human body template using the signed distance fields reconstructed from all frames using the marching cubes algorithm.

[0081] For each frame of the reconstructed signed distance field, for each edge, if the signs of the two connected vertices are different, interpolate on this edge, calculate the position where the interpolation value is 0, and place the vertex;

[0082] For each frame of the reconstructed signed distance field, triangle patches are constructed between the vertices.

[0083] According to the fast geometric modeling method of the embodiment of the present invention, the expected value maximum algorithm is used to learn the mean vector, covariance matrix and weight parameters of the five-dimensional quadratic kernel model, which can effectively model the dynamic geometry of the three-dimensional human body and make the understanding of human body movements have temporal and spatial inductive generalization. Data classification initialization based on the human motion capture skeleton can solve the iterative calculation difficulties of high-dimensional data, and does not need to rely on neural networks, and has a certain robustness.

[0084] In order to implement the above embodiment, Figure 3 As shown, this embodiment also provides a fast geometric modeling device 10, including:

[0085] The data processing module 100 is used to extract the three-dimensional signed distance field information of all frames of the human body template action sequence and process it to obtain a data matrix;

[0086] A skeleton extraction module 200 is used to extract the torso skeleton of the human body template of the first frame of human action;

[0087] A model quantity determination module 300 is used to determine the number of modeling models;

[0088] A parameter initialization module 400 is used to initialize the parameters of the data matrix based on the human body parts according to the human body template torso skeleton, so as to initialize the modeling signed distance field to obtain the original signed distance field;

[0089] A model parameter updating module 500 is used to optimize and update the parameters of the model using an expected value maximum algorithm;

[0090] A mean square error calculation module 600 is used to input parameters into a reconstruction model based on a quadratic kernel gate function to obtain signed distance fields reconstructed from all frames, and calculate the mean square error between the signed distance fields reconstructed from all frames and the original signed distance fields;

[0091] The distance field reconstruction module 700 is used to repeatedly iterate and update the model parameters, and reconstruct the distance fields of all frame symbols using the parameters corresponding to the minimum mean square error value of the mean square error;

[0092] The human body mesh modeling module 800 is used to reconstruct the three-dimensional mesh of the human body template using the signed distance fields reconstructed from all frames using the marching cubes algorithm.

[0093] According to the fast geometric modeling device of the embodiment of the present invention, the expected value maximum algorithm is used to learn the mean vector, covariance matrix and weight parameters of the five-dimensional quadratic kernel model, which can effectively model the dynamic geometry of the three-dimensional human body and make the understanding of human body movements have temporal and spatial inductive generalization. Data classification initialization based on the human motion capture skeleton can solve the difficulty of iterative calculation of high-dimensional data, and does not need to rely on neural networks, and has a certain robustness.

[0094] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0095] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

Claims

1. A fast geometric modeling method, characterized in that: include: Extract the three-dimensional signed distance field information of all frames of the human body template action sequence and process it to obtain a data matrix; Extract the human body template torso skeleton of the first frame of human action; Determine the number of models to be modeled; Initializing the parameters of the data matrix based on the human body parts according to the human body template to initialize the modeling signed distance field to obtain the original signed distance field; Use the expected value maximization algorithm to optimize and update the parameters of the model; Input the parameters into the reconstruction model based on the quadratic kernel gate function to obtain the reconstructed signed distance fields of all frames, and calculate the mean square error between the reconstructed signed distance fields of all frames and the original signed distance fields; Repeatedly iteratively update the model parameters, and reconstruct the symbol distance fields of all frames using the parameters corresponding to the minimum mean square error value of the mean square error; The 3D mesh of the human body template is reconstructed using the signed distance fields reconstructed for all frames using the marching cubes algorithm.

2. The method according to claim 1, characterized in that The data matrix extracted frame by frame includes the signed distance field and corresponding positions with a resolution of N0 for T frames, arranged in rows with the number of N=N0. 3 ×T, a data matrix with 5 columns.

3. The method according to claim 1, characterized in that The human body template torso skeleton is directly obtained from the input human body template parameters, and the human body template vertices are connected in pairs to form a human body skeleton.

4. The method according to claim 2, characterized in that: If the number of models to be modeled is K, the following conditions must be met: the spatial signed distance field is evenly divided into k1×k1×k1 cubes, and k1 3 =K, where k1 needs to be an integer divider of the resolution value N0.

5. The method according to claim 4, characterized in that The initialization modeling signed distance field comprises: Evenly extract 50 coordinate points on each segment of the extracted skeleton; Traverse each cube block. If the points on all the line segments of the human skeleton are not within its range, it will be divided into a group. The remaining blocks will look for sampling points on the skeleton line segment closest to it, and those that match the same segment will be grouped together. After the position grouping of the first frame is completed, all the remaining frames are grouped in the same way, and the same grouping in all frames is regarded as a dynamic grouping; After matching, the initial dynamic grouping is obtained into K groups, and the mean vector, covariance matrix and weight μ of the K group data are obtained. j ,Σ j ,α j , j = 1, 2, ..., K; The parameter μ j ,Σ j ,α j , j = 1, 2, ..., K into the five-dimensional quadratic kernel gate function reconstruction formula Initialize the entire signed distance field where δ represents the position, g(δ; α j ,μ j ,Σ j ) represents the quadratic kernel gate function, m(δ; μ j ,Σ j ) represents the quadratic kernel conditional mean.

6. The method according to claim 5, characterized in that The method of optimizing and updating the parameters of the model by using the maximum expected value algorithm includes: Update the secondary kernel posterior probability Qt ij ,i=1,2,...,N,j=1,2,...,K; Update Parameters 7. The method according to claim 1, characterized in that The mean square error is the mean square difference between the values ​​of the original and reconstructed signed distance fields at the same position in each frame.

8. The method according to claim 1, characterized in that The model parameters are updated 50 times.

9. The method according to claim 1, characterized in that: The method of reconstructing the three-dimensional grid of the human body template by using the signed distance field reconstructed from all frames using the marching cubes algorithm comprises: For each frame of the reconstructed signed distance field, for each edge, if the signs of the two connected vertices are different, interpolate on this edge, calculate the position where the interpolation value is 0, and place the vertex; For each frame of the reconstructed signed distance field, triangle patches are constructed between the vertices.

10. A rapid geometric modeling device, characterized in that: include: A data processing module is used to extract the three-dimensional symbol distance field information of all frames of the human body template action sequence and process it to obtain a data matrix; A skeleton extraction module is used to extract the torso skeleton of the human body template of the first frame of human action; A model quantity determination module is used to determine the number of modeling models; A parameter initialization module is used to initialize the parameters of the data matrix based on the human body parts according to the human body template torso skeleton, so as to initialize the modeling signed distance field to obtain the original signed distance field; A model parameter updating module is used to optimize and update the parameters of the model using the expected value maximization algorithm; A mean square error calculation module is used to input parameters into a reconstruction model based on a quadratic kernel gate function to obtain a signed distance field reconstructed for all frames, and calculate a mean square error between the signed distance field reconstructed for all frames and the original signed distance field; A distance field reconstruction module, used for repeatedly iterating and updating the model parameters, and reconstructing the distance fields of all frame symbols using the parameters corresponding to the minimum mean square error value of the mean square error; The human body mesh modeling module is used to reconstruct the three-dimensional mesh of the human body template using the signed distance field reconstructed from all frames using the marching cubes algorithm.

Images