Parametric non-rigid deformable modeling method based on local skeleton pose

Abstract

The present invention relates to a parametric non-rigid deformation modeling method based on local skeleton posture. The non-rigid deformation of an object is modeled by explicitly simulating the interaction between connected components. The deformation of any point on a component is the result of the interaction between the component and its adjacent components. The tendency of a point to remain stationary on a component is defined as the stiffness of the point. A stiffness field of a component relative to each adjacent component is constructed. After the posture transformation data is known, the deformation of this point is solved using this point stiffness field to obtain a non-rigid deformation. A new parametric modeling method for non-rigid deformation based on spatial coordinate object shape expression is provided. The deformation mixing coefficient defined by the local skeleton posture and the explicit modeling of non-rigid deformation achieved by mixing local relative motion can be directly migrated to unknown skeleton movements. The method is suitable for drivable object reconstruction in the field of computer vision and object shape representation and non-rigid deformation parametric modeling in the field of computer graphics.

Description

Technical Field

[0001] The present invention relates to a modeling technology, and in particular to a parameterized non-rigid deformation modeling method based on local skeleton posture. Background Art

[0002] Object shape representations based on spatial coordinates (such as neural implicit functions) have flexible topological structures and expressive capabilities, offering inherent advantages in object shape reconstruction. However, effective solutions for modeling non-rigid deformations of spatial coordinate-based shape representations remain lacking. Existing solutions can be broadly categorized into two types: holistic modeling and component-by-component modeling.

[0003] Holistic modeling involves first defining the shape of an object under a standard skeletal motion and then using a skinning algorithm to transform it into any arbitrary pose. To migrate the commonly used linear blend skinning algorithm on 3D meshes to a shape representation based on spatial coordinates, holistic modeling methods use neural networks to record the skinning coefficient field under specific skeletal motions. However, these neural skinning coefficient fields are only effective under known skeletal motions and have poor generalization capabilities for new motions.

[0004] Component-based modeling approaches model each sub-component separately and then piece them together to form the complete object. These methods effectively utilize the rigid body motion information of the skeleton joints and can well capture the local rigid body transformations during the object's motion. However, they lack non-rigid deformation modeling methods that can generalize to new motions. Summary of the Invention

[0005] To address the problem of poor generalization ability of modeling new actions based on spatial coordinate-based object shape expression, a parametric non-rigid deformation modeling method based on local skeleton pose is proposed, which can directly migrate to unknown skeleton actions.

[0006] The technical solution of the present invention is: a parametric non-rigid deformation modeling method based on local skeleton posture, which models the non-rigid deformation of an object by explicitly simulating the interaction between connected components. The deformation of any point on a component is the result of the interaction between the component and its adjacent components. The tendency of a point to remain stationary on the component is defined as the stiffness of the point. Then, a stiffness field of a component relative to each of its adjacent components is constructed. After the posture transformation data is known, the deformation of this point is solved using this point stiffness field to obtain non-rigid deformation.

[0007] Furthermore, the stiffness of the point is calculated as follows: for bones AO and OB connected by joint O, the unconnected endpoints A and B of the two bones are connected, and then the connecting line is divided into two segments QA and QB according to the ratio of the lengths of the two bones. The stiffness of any point X on the bones b1 and b2 is defined as:

[0008]

[0009]

[0010] where r1 and r2 are the stiffness of point X with respect to bones b1 and b2, respectively; P is the projection of X on the connecting line AB; α1 and β1 are learnable parameters that adjust the stiffness field of bone b1; α2 and β2 are learnable parameters that adjust the stiffness field of bone b2; when point X is close to bone b1, its stiffness with respect to bone b1 increases, while its stiffness with respect to bone b2 decreases.

[0011] Furthermore, the deformation blending weight of the point X with respect to bones b1 and b2 is defined as:

[0012]

[0013] ω1+ω2=1. When the two bones b1 and b2 rotate relative to each other at an angle θ, in order to maintain the connection, the connection section of the two skeletal components should not produce a change in the relative gap, and the sum of the relative rotations ω1θ+ω2θ must be equal to θ.

[0014] Furthermore, the non-rigid deformation modeling method: the skeleton b0 is connected to several bones, and the initial position of any point x on the skeleton b0 component in the standard posture is:

[0015]

[0016] in x is the position of the same point before and after deformation; B is the index set of the joints connected to the bone b0; θ b is the rotation angle of the bth bone connected to bone b0 relative to bone b0; ω b The bth bone connected to bone b0 now has the deformation blending weight for bone b0. Yes b θ b The corresponding rotation matrix; t b It is the rotation center point of the bth bone connected to bone b0.

[0017] The beneficial effects of the present invention are as follows: the parametric non-rigid deformation modeling method based on local skeleton pose provides a new parametric modeling method for non-rigid deformation based on spatial coordinate object shape expression. The deformation blending coefficients defined by the local skeleton pose and the explicit non-rigid deformation modeling achieved by blending local relative motion can be directly transferred to unknown skeleton movements. The method of the present invention is suitable for drivable object reconstruction in the field of computer vision and object shape representation and non-rigid deformation parametric modeling in the field of computer graphics. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is an interaction diagram between the connected components of the present invention;

[0019] Figure 2 A diagram showing the stitching of two connected components of the present invention;

[0020] Figure 3 A diagram showing the stitching of three connected components of the present invention;

[0021] Figure 4 This is a two-dimensional example diagram used to illustrate the definition of stiffness in the present invention;

[0022] Figure 5 Schematic diagram of non-rigid deformation caused by the stitching algorithm of adjacent components of the present invention;

[0023] Figure 6a This is a graph showing the results of comparing the method of the present invention with a new action;

[0024] Figure 6b This is a graph comparing the results of the method of the present invention under the new action 2;

[0025] Figure 6c This is a graph comparing the results of the method of the present invention under the new action three;

[0026] Figure 7a FIG4 is a scaling factor matrix diagram of the optimized stiffness coefficient of the present invention;

[0027] Figure 7b Matrix diagram of the bias factor of the optimized stiffness coefficient of the present invention. DETAILED DESCRIPTION

[0028] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0029] The present invention proposes a method for modeling sub-components, which models the non-rigid deformation of an object by explicitly simulating the interaction between connected components. This method can also be called an "adjacent component stitching" algorithm. It solves the problem of modeling the non-rigid deformation of the connection parts of rigidly connected components and avoids defects in the connection parts when the pose is transformed after modeling. Figure 1 The interaction diagram between the connected parts shown in the figure shows that when the joint rotates, the inner sides of the two parts are squeezed and the outer sides are stretched. In order to model different non-rigid deformation characteristics, a parameterized method based on local skeleton posture is proposed - the "stiffness competition" algorithm.

[0030] Non-rigid deformation modeling based on "adjacent parts stitching":

[0031] Consider the part bound to bone b1 ( Figure 2 ), the adjacent bone b2 has been rotated by an angle θ from the standard pose, and we hope to obtain the initial position of x before the rotation occurs. If the stiffness of bone b1 is infinite, then no matter what the rotation angle of b2 is, point x on b1 will not move. Otherwise, x should be rotated by an angle ωθ, where 0﹤ω﹤1 (currently assuming that the deformation blending weight ω is known, which will be discussed further later). Then, we can get the initial position of x in the standard pose

[0032] where R ωθ is the rotation matrix corresponding to ωθ, and T is the matrix transpose. Although the diagram shown only includes two-dimensional rotations, the same process can be directly extended to three-dimensional rotations based on axis angles.

[0033] The following discusses the situation where three components are connected. Figure 3 As shown in the figure, when considering the initial position of point x on bone b0, it is found that x has two different rotation tendencies. Since the rotation axes of the two rotations are not the same, the rotation angles cannot be simply mixed. Therefore, the mixed offset vector is selected:

[0034]

[0035] Where t1 and t2 are the two rotation center points. So, we get the initial position of x

[0036]

[0037] For the case where more parts are connected, the initial position of any point x on the skeleton b0 part expressed in the above formula can be generalized to

[0038]

[0039] in x is the position of the same point before and after deformation; B is the index set of the joints connected to the bone b0; θ b is the rotation angle of the bth bone connected to bone b0 relative to bone b0; ω b The bth bone connected to bone b0 now has the deformation blending weight for bone b0. Yes b θ b The corresponding rotation matrix; t b It is the rotation center point of the bth bone connected to bone b0.

[0040] Calculation of deformation mixing parameters based on "stiffness competition":

[0041] The basic idea here is that the deformation of any point on a component is the result of the interaction between the component and its neighbors. The tendency of a point to remain stationary on the component is defined as the stiffness of the point. Then, a stiffness field of a component relative to each of its neighbors can be constructed. Figure 4 For example, for bones AO and OB connected by joint 0, connect the unconnected endpoints A and B of the two bones, and then divide the connecting line into two segments QA and QB according to the ratio of the lengths of the two bones. At this point, the stiffness at any point X on bones b1 and b2 is defined as:

[0042]

[0043]

[0044] Where r1 and r2 are the stiffnesses of point X with respect to bones b1 and b2, respectively. P is the projection of X onto line AB. α1 and β1 are learnable parameters that adjust the stiffness field of bone b1. α2 and β2 adjust the stiffness field of bone b2. When point X is close to bone b1, its stiffness with respect to bone b1 increases, while its stiffness with respect to bone b2 decreases.

[0045] When reconstructing a drivable object, the loss between the predicted shape and the true shape is calculated, and the optimization objective is to minimize the shape reconstruction loss function. Because the stiffness coefficients (α1, β1, α2, β2) are involved in the calculation of the predicted shape, they can be updated as parameters when optimizing the shape reconstruction loss function using deep learning frameworks that support automatic differentiation (such as PyTorch).

[0046] Based on the stiffness defined above, the deformation blend weight of point X with respect to bones b1 and b2 is defined as

[0047] According to the above equation, ω1 + ω2 = 1, which is crucial for perfect component stitching. Specifically, when the two bones b1 and b2 rotate relative to each other by an angle θ, the gap between the connecting cross-sections of the two skeletal components should not change to maintain the connection. Therefore, the sum of the relative rotations ω1θ + ω2θ must equal θ. This requires ω1 + ω2 = 1.

[0048] In order to verify the effectiveness of this method, a test was conducted on the human body reconstruction task based on point cloud. Figure 1 When bones b1 and b2 are rotated relative to each other, the algorithm deforms the components of bone b1 so that the cross section through the joint remains aligned with that of bone b2. Figure 1Non-rigid deformation caused by the adjacent component stitching algorithm. When bones b1 and b2 rotate relative to each other, the adjacent component stitching algorithm ensures that the cross-sections of their joints are aligned. To better visualize the cross-sections of the connected parts and confirm that there is no overlap, the spacing between the parts is artificially increased.

[0049] exist Figures 6a to 6c In

[15] , the results of this method under new motions are compared. Without this method, the model will basically lose the ability to model non-rigid deformations, resulting in defects such as cracks in the elbows and knees.

[0050] Since the scaling factor α and the bias factor β of the stiffness coefficient are defined for each pair of adjacent components, they are stored in matrices. As a rule of thumb, the α matrix is ​​initialized to 2 and the β matrix is ​​initialized to 0. Figure 7a 、 7b The optimized stiffness coefficient matrix is ​​shown in . Observe that the matrix for the scaling factor α is symmetric, while the matrix for the bias factor β is skew-symmetric. Only some elements in the matrix deviate from the initial values ​​because the connections between the components are sparse.

[0051] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be determined by the appended claims.

Claims

1. A parametric non-rigid deformable modeling method based on local skeleton pose, characterized in that: The non-rigid deformation of an object is modeled by explicitly simulating the interaction between connected parts. The deformation of any point on a part is the result of the interaction between the part and its adjacent parts. The tendency of a point to remain stationary on the part is defined as the stiffness of the point. Then, a stiffness field of a part relative to each of its adjacent parts is constructed. After the pose transformation data is known, the deformation of this point is solved using this point stiffness field to obtain the non-rigid deformation. The stiffness of the point is calculated as follows: for bones AO and OB connected by joint O, the unconnected endpoints A and B of the two bones are connected, and then the connecting line is divided into two segments QA and QB according to the ratio of the lengths of the two bones. The stiffness of any point X in space with respect to bones b1 and b2 is defined as: Where r1 and r2 are the stiffness of point X with respect to bones b1 and b2 respectively, P is the projection of X on the connecting line AB, α1 and β1 are learnable parameters that adjust the stiffness field of bone b1; α2 and β2 are learnable parameters that adjust the stiffness field of bone b2; when point X is close to bone b1, its stiffness with respect to bone b1 increases, while its stiffness with respect to bone b2 decreases; The deformation blending weight of the point X with respect to bones b1 and b2 is defined as: ω1+ω2=1. When two bones b1 and b2 rotate relative to each other at an angle θ, in order to maintain the connection, the relative gap between the connecting sections of the two skeletal components should not change. The sum of the relative rotations ω1θ+ω2θ must be equal to θ. The non-rigid deformation modeling method: bone b0 is connected to several bones. The initial position of any point x on the bone b0 component in the standard posture is: in x is the position of the same point before and after deformation; B is the index set of the joints connected to the bone b0; θ b is the rotation angle of the bth bone connected to bone b0 relative to bone b0; ω b The bth bone connected to bone b0 now has the deformation blending weight for bone b0. Yes b θ b The corresponding rotation matrix; t b It is the rotation center point of the bth bone connected to bone b0.

Images