Non-rigid matching method and device of geometric model using local and global information

Abstract

The method and device for non-rigid matching of geometric models using local and global information can distinguish the left-right symmetrical parts of a shape and significantly improve the accuracy of point matching. The method includes: (1) inputting a pair of geometric models representing different postures of the same object, and the data type of the model is a mesh type or a point cloud type; (2) calculating the density of each point in the model and cutting the geometric model according to the point density; (3) matching the segments obtained by segmentation using the inter-segment error and the intra-segment error to obtain the segment matching result; (4) using the local-global commutativity preserving model, which includes the segment-related descriptor constraints and the segment-related operator commutativity constraints of the geometric object, and outputting the point matching result using the segment matching information.

Description

Technical Field

[0001] The present invention relates to the technical field of computer graphics, and in particular to a method for non-rigid matching of geometric models using local and global information, and a device for non-rigid matching of geometric models using local and global information. Background Art

[0002] When the movement of an object is not determined by a single bone, but by the movement of multiple bones together, the object will undergo non-rigid deformation. For example, the deformation experienced by the hand when holding an object or snapping fingers is a non-rigid deformation.

[0003] The non-rigid matching method of geometric models seeks the point-to-point or segment-to-segment correspondence between a pair (or a group) of non-rigidly deformed geometric models. The earlier geometric model matching work can be traced back to 2006, when Brostein et al. introduced generalized multidimensional scaling into the surface mapping algorithm to find the minimum distortion embedding from one surface to another. To date, non-rigid matching methods can be roughly divided into the following two categories.

[0004] (1) Sparse-based methods

[0005] Due to the large number of points in the model, it is not feasible to directly constrain the correspondence of all points. The main idea of ​​this type of method is to obtain a set of sparse point correspondences and expand them to a dense mapping. Specifically, this type of method either establishes the correspondence of sub-shapes of the geometric model by pre-segmenting the geometric model; or phrases the segmentation and correspondence (and possible labeling) by jointly optimizing the geometric model. Since sparse point correspondences are discrete in nature, various indicators between point pairs or point sets are usually used to achieve global matching of geometric model pairs, such as geodesic distance, various spectral quantities, or embedding the geometric model into a canonical domain based on landmark correspondence, or a combination of multidimensional geometric and topological tests.

[0006] There are two main drawbacks of this type of work. First, it is difficult to obtain accurate point-to-point mapping. To solve this problem, they usually restore sparse points into several sub-shapes to establish partial or segmented correspondences of geometric models instead of reliable point-to-point matching, which avoids the need to establish reliable point-by-point correspondences. Second, current sparse methods lose a lot of detailed features, making it difficult to distinguish between two symmetrical sub-models, such as distinguishing between left and right hands.

[0007] (2) Method based on function mapping

[0008] The function mapping-based method was proposed by Ovsjanikoc in 2012. This method generalizes the mapping between corresponding points of two geometric models to the mapping of real-valued functions. By selecting a multi-scale basis for the characteristic function space of the Laplace-Beltrami operator of each geometric model, a compact mapping representation is obtained, which can usually be expressed as a set of linear equations. This set of equations can be used to calculate the mapping matrix to represent the mapping relationship between the characteristic function spaces of the Laplace-Beltrami operators of the two geometric models, and then restore the point-to-point mapping from the mapping matrix. This method can achieve better matching effects of isometric geometric models.

[0009] Since it is difficult for sparsification-based methods to distinguish symmetric sub-models, the segmentation results of current methods are usually disconnected, for example, the left and right hands are divided into the same sub-model. This limits the addition of segment constraints in function mapping-based methods. Summary of the invention

[0010] In order to overcome the defects of the prior art, the technical problem to be solved by the present invention is to provide a non-rigid matching method of geometric models using local and global information, which can distinguish the left-right symmetrical parts of the shape and significantly improve the accuracy of point matching.

[0011] The technical solution of the present invention is: the geometric model non-rigid matching method using local and global information comprises the following steps:

[0012] (1) The input is a pair of geometric models representing different postures of the same object, and the data type of the model is a mesh type or a point cloud type;

[0013] (2) Calculate the density of each point in the model and cut the geometric model according to the point density;

[0014] (3) For the segments obtained by segmentation, use the inter-segment error and intra-segment error to match and obtain the segment matching result;

[0015] (4) Through the local-global commutativity preserving model, which includes segment-related descriptor constraints and segment-related operator commutativity constraints of geometric objects, the segment matching information is used to output the point matching results.

[0016] The present invention firstly realizes the sub-shape segmentation of the geometric model based on the density-based shape segmentation algorithm. Compared with other segmentation methods, the sub-shapes segmented by this method are more different, which helps to improve the accuracy of sub-shape matching. Secondly, the method proposes a sub-shape matching algorithm based on inter-segment errors and intra-segment errors. Compared with other segment matching methods, the method is faster and more accurate. Finally, the method proposes a local-global commutativity preserving model, which includes segment-related descriptor constraints and segment-related operator commutativity constraints of geometric objects, thereby improving the accuracy of shape matching.

[0017] A non-rigid matching device for a geometric model using local and global information is also provided, the device comprising:

[0018] A data input module, whose input is a pair of geometric models representing different postures of the same object, and the data type of the model is a mesh type or a point cloud type;

[0019] A cutting module that calculates the density of each point in the model and cuts the geometric model according to the point density;

[0020] A segment matching module matches the segments obtained by segmentation using inter-segment errors and intra-segment errors to obtain segment matching results;

[0021] The point matching module uses the segment matching information to output the point matching results. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A flowchart of non-rigid shape matching based on local and global information is shown.

[0023] Figure 2 The segment cutting results of the RSB method are shown.

[0024] Figure 3 The geodesic error accumulation curve is shown.

[0025] Figure 4 A flow chart of a geometric model non-rigid matching method using local and global information according to the present invention is shown. DETAILED DESCRIPTION

[0026] like Figure 4 As shown, this non-rigid matching method of geometric models using local and global information includes the following steps:

[0027] (1) The input is a pair of geometric models representing different postures of the same object, and the data type of the model is a mesh type or a point cloud type;

[0028] (2) Calculate the density of each point in the model and cut the geometric model according to the point density;

[0029] (3) For the segments obtained by segmentation, use the inter-segment error and intra-segment error to match and obtain the segment matching result;

[0030] (4) Through the local-global commutativity preserving model, which includes segment-related descriptor constraints and segment-related operator commutativity constraints of geometric objects, the segment matching information is used to output the point matching results.

[0031] The present invention firstly realizes the sub-shape segmentation of the geometric model based on the density-based shape segmentation algorithm. Compared with other segmentation methods, the sub-shapes segmented by this method are more different, which helps to improve the accuracy of sub-shape matching. Secondly, the method proposes a sub-shape matching algorithm based on inter-segment errors and intra-segment errors. Compared with other segment matching methods, the method is faster and more accurate. Finally, the method proposes a local-global commutativity preserving model, which includes segment-related descriptor constraints and segment-related operator commutativity constraints of geometric objects, thereby improving the accuracy of shape matching.

[0032] Preferably, in step (1), the input is a pair of geometric models M and N, which respectively contain I and J points, reflecting different postures of the same object.

[0033] Preferably, the step (2) comprises the following sub-steps:

[0034] (2.1) For each geometric model, the density of each point is calculated; for point i in the surface mesh model M, after setting the threshold ε, there is a set G i = {G(i,j1)...G(i,j J )|G(i,j)<ε,j=1,2,…,J} is the set of neighboring points of point i, where G(i,j) is the geodesic distance between two points, and N i G i The number of elements in N is closely related to the structure near point i. The more points and grids there are near point i, the larger the number of elements in N. i The larger the value, the better the N i is the geodesic density of point i. For point i in the point cloud model M, the distance between two points is calculated using the Euclidean distance to obtain the density;

[0035] (2.2) Select several segment centers from all the points;

[0036] (2.3) Cut the geometric model according to the segment center.

[0037] Preferably, the step (2.2) comprises the following sub-steps:

[0038] (2.2.1) Set the initial density critical value η = 0.1;

[0039] (2.2.2) Sort the points in the model from low to high density, the first η×I points are low-density points, and the rest are high-density points;

[0040] (2.2.3) Take the current highest density point as the center of a segment, and start from the segment center to find out whether there are other high-density points in its neighboring points; if there are, continue to find out whether there are other high-density points in the neighboring points of these high-density points; repeat the search process until the neighboring points of the found high-density points do not contain high-density points that have not been searched;

[0041] (2.2.4) Move the high-density points to be found out of the high-density point set;

[0042] (2.2.5) If there are still high-density points, return to step (2.2.3), otherwise go to step (2.2.6);

[0043] (2.2.6) If only one segment center is found in the model, set η = η + 0.1 and return to step (2.2.2); otherwise, terminate the algorithm and output the segment center.

[0044] Preferably, in the step (2.3), the distance from each point to the center of each segment is calculated, and the segment closest to the point is taken as the segment to which the point belongs; for the grid model, the geodesic distance is used to more reasonably evaluate the distance from the point to the segment center; for the point cloud model, the distance between each point and its neighboring points is set to 1, and the number of neighboring points that each point needs to pass through to reach the segment center is calculated, and this is used as the distance from the point to the segment center to achieve an effect similar to the geodesic distance.

[0045] Preferably, in step (3),

[0046] Assume that the geometric models M and N are cut into m and n segments respectively, denoted as M 1 、M 2 ...M m 、N 1 、N 2 ...N n , each segment stores the relevant geometric model fragments, and the segment center is Assume m≤n, and use the following formula to solve the segment matching result:

[0047]

[0048] where Σ n is the set of all permutations of {1, 2, ..., n}, σ = [σ(1), σ(2), ..., σ(n)] is Σ n A permutation in; Formula (1) has multiple solutions, and what is needed are the first m terms of σ: [σ(1),σ(2),...,σ(m)],

[0049] To solve the permutation relationship σ, formula (1) contains three constraints. The first one uses the ICP algorithm to evaluate the similarity between two segments to constrain the internal structure of the segments so that the corresponding pair of segments in the geometric models M and N are as similar as possible. The ICP algorithm accepts a pair of geometric models as input, and after centering, rotation, and registration, returns the sum of squared distances of the pair of geometric models. The more similar the two geometric models are, the smaller the sum of squared distances is.

[0050] The second item constrains the relative angle of the segment centers and allows negative cosine values. This feature helps to distinguish the symmetrical parts of the geometric model. The third item constrains the relative Euclidean distance of the segment centers. These two items ensure that the relative positions of a pair of segments in the geometric models M and N remain consistent in their respective geometric models.

[0051] Preferably, in step (4),

[0052] Assume that the geometric models M and N are segmented and matched into l pairs of corresponding segments. After extending the descriptor constraints and operator commutativity constraints from global to local, we have the objective function:

[0053]

[0054] in is the required mapping matrix, α, β>0 are the weight coefficients of the global exchange constraint and the local exchange constraint respectively, the first item is the descriptor constraint, is the descriptor matrix after the descriptor function of the geometric models M and N is projected into the Laplace space. F and G include both the global descriptor and the descriptor of each segment. The second item is the global exchange constraint. is a global operator; the third term is a local exchange constraint, is the operator on each segment;

[0055] C, F, G, R, S, R i , S i Transform to obtain a closed-form solution for C.

[0056] Preferably, in step (4), in step (4), use Represents the i-th row vector of matrix A, using Denotes the j-th column vector of matrix A, using represents the vectorization operator of the matrix, use Defined as the expanded matrix of A,

[0057]

[0058] Where Diag(·,·,...,·) is a diagonal block matrix construction operator.

[0059]

[0060] Here, each 0 is a zero matrix of size m×n, so formula (2) is equivalent to the following formula:

[0061]

[0062] It is equivalent to the following formula:

[0063]

[0064] Its solution is Here each 0 is m 2 dimensional zero vector, and vec(C) is used to restore the closed solution C of formula (2).

[0065] The present invention experimentally verifies the above method and achieves significant results. All experimental images are from the TOSCA dataset, which contains 9 categories of data. Each category contains triangular surface mesh models of the same geometric model in different postures, with a total of 80 geometric models. Each category of geometric model provides a point-to-point benchmark mapping, which is used to evaluate the accuracy of the method of the present invention.

[0066] This method is applicable to both segment-to-segment shape matching problems and point-to-point shape matching problems. In the segment matching experiment, it is mainly compared with various graph-based methods. This method can also be combined with existing segment matching methods to further improve accuracy, and related experiments are also conducted. In the point matching experiment, the present invention is mainly compared with the function mapping method.

[0067] 1. Segment matching experiment

[0068] In order to achieve comparability, the first geometric model of each category in the TOSCA data is used as the source model, and the other shapes are used as the target models. The source model and the target model are matched one by one to obtain the segment matching accuracy. This approach is consistent with other comparison methods.

[0069] As mentioned above, the present invention can process two data forms: mesh model and point cloud model. In Table 1, Ours-M is the segment matching result of the mesh model, Ours-P is the segment matching result of the point cloud model, and RSB, BIM, BIM-FPS, STB, Reeb, etc. are other comparison methods. The accuracy of segment matching is related to both the segment cutting result and the segment matching result. As can be seen from Table 1, the segment matching method proposed in the present invention can maintain consistent segment cutting results in the source model and the target model, and at the same time, the segment matching result is also good.

[0070] category Ours-M Ours-P RSB BIM BIM-FPS STB Reeb cat 0.9625 0.9475 0.8146 0.8337 0.7858 0.6646 0.1101 centaur 0.9840 0.8336 0.8719 0.9040 0.8400 0.7261 0.2372 david 0.9554 0.9845 0.9483 0.9112 0.8600 0.0177 dog 0.8703 0.6530 0.9155 0.8524 0.8118 0.5470 0.0920 gorilla 0.9989 0.8831 0.9178 0.8966 0.8634 0.4527 0.0166 horse 0.9968 0.7520 0.8959 0.9030 0.8789 0.7463 0.1106 michael 0.9558 0.9284 0.9372 0.9045 0.8614 0.0552 victoria 0.9958 0.6115 0.9192 0.9037 0.8668 0.1025 wolf 0.9976 0.9914 0.9570 0.9816 0.9723 0.9180 0.2446 Average accuracy 0.9686 0.8428 0.9086 0.8990 0.8600 0.6758 0.1096 Average number of segments 5.96 4.39 19.22 - - 9.10 20.68

[0071] Table 1

[0072] 2. Improve other segment matching methods

[0073] This experiment evaluated the improvement effect of formula (3) in the present invention on the RSB method. Figure 2 As shown in FIG. 2 , the segment matching results of the RSB method cannot distinguish the symmetrical parts of the geometric model. As shown in Table 2, after being improved by using formula (3) in the present invention, the symmetrical parts of the geometric model can be distinguished successfully, which greatly increases the average number of segments and maintains the accuracy. Compared with another improved method for breaking symmetry: the RSB-BS method, the speed of the present invention is significantly faster.

[0074]

[0075]

[0076] Table 2

[0077] 3. Point matching experiment

[0078] This experiment adopts the same grouping method as the segment matching experiment and tests the point matching effect of formula (4) in this patent on the TOSCA dataset. Figure 3 As shown, the effect of the method of the present invention is obviously better than that of the function mapping (FMAP) method.

[0079] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the protection scope of the technical solution of the present invention.

Claims

1. A non-rigid matching method of geometric models using local and global information, characterized by: The method comprises the following steps: (1) The input is a pair of geometric models representing different postures of the same object, and the data type of the model is a mesh type or a point cloud type; (2) Calculate the density of each point in the model and cut the geometric model according to the point density; (3) For the segments obtained by segmentation, use the inter-segment error and intra-segment error to match and obtain the segment matching result; (4) Local-global commutativity preserving model, which includes segment-dependent descriptor constraints and segment-dependent operator commutativity constraints of geometric objects; In step (1), the input is a pair of geometric models M and N, which respectively contain I and J points, reflecting different postures of the same object; The step (2) comprises the following sub-steps: (2.1) For each geometric model, the density of each point is calculated; for point i in the surface mesh model M, after setting the threshold ε, there is a set G i = {G(i,j1)...G(i,j J )|G(i,j)<ε,j=1,2,…,J} is the set of neighboring points of point i, where G(i,j) is the geodesic distance between two points, and N i G i The number of elements in N is closely related to the structure near point i. The more points and grids there are near point i, the larger the number of elements in N. i The larger the value, the better the N i is the geodesic density of point i. For point i in the point cloud model M, the distance between two points is calculated using the Euclidean distance to obtain the density; (2.2) Select several segment centers from all the points; (2.3) Cutting the geometric model according to the segment center; The step (2.2) comprises the following sub-steps: (2.2.1) Set the initial density critical value η = 0.1; (2.2.2) Sort the points in the model from low to high density, the first η×I points are low-density points, and the rest are high-density points; (2.2.3) Take the current highest density point as the center of a segment, and start from the segment center to find out whether there are other high-density points in its neighboring points; if there are, continue to find out whether there are other high-density points in the neighboring points of these high-density points; repeat the search process until the neighboring points of the found high-density points do not contain high-density points that have not been searched; (2.2.4) Move the high-density points to be found out of the high-density point set; (2.2.5) If there are still high-density points, return to step (2.2.3), otherwise go to step (2.2.6); (2.2.6) If only one segment center is found in the model, set η = η + 0.1 and return to step (2.2.2). Otherwise, terminate the algorithm and output the segment center. In the step (2.3), the distance from each point to the center of each segment is calculated, and the segment closest to the point is taken as the segment to which the point belongs; for the grid model, the geodesic distance is used to more reasonably evaluate the distance from the point to the segment center; for the point cloud model, the distance between each point and its neighboring points is set to 1, and the number of neighboring points that each point needs to pass through to reach the segment center is calculated, and this is taken as the distance from the point to the segment center, so as to achieve an effect similar to the geodesic distance; In the step (3), Assume that the geometric models M and N are cut into m and n segments respectively, denoted as M 1 、M 2 ...M m 、N 1 、N 2 ...N n , each segment stores the relevant geometric model fragments, and the segment center is Assume m≤n, and use the following formula to solve the segment matching result: where Σ n is the set of all permutations of {1, 2, ..., n}, σ = [σ(1), σ(2), ..., σ(n)] is Σ n A permutation in; Formula (1) has multiple solutions, and what is needed are the first m terms of σ: [σ(1),σ(2),...,σ(m)], To solve the permutation relationship σ, formula (1) contains three constraints. The first one uses the ICP algorithm to evaluate the similarity between two segments to constrain the internal structure of the segments so that the corresponding pair of segments in the geometric models M and N are as similar as possible. The ICP algorithm accepts a pair of geometric models as input, and after centering, rotation, and registration, returns the sum of squared distances of the pair of geometric models. The more similar the two geometric models are, the smaller the sum of squared distances is. The second term constrains the relative angles of the segment centers, allowing negative cosine values. This feature helps to distinguish the symmetrical parts of the geometric model. The third term constrains the relative Euclidean distances of the segment centers. These two terms make the relative positions of a pair of segments in the geometric models M and N consistent in their respective geometric models. In the step (4), Assume that the geometric models M and N are segmented and matched into l pairs of corresponding segments. After extending the descriptor constraints and operator commutativity constraints from global to local, we have the objective function: in is the required mapping matrix, α>0, β>0 are the weight coefficients of the global exchange constraint and the local exchange constraint respectively, the first item is the descriptor constraint, is the descriptor matrix after the descriptor function of the geometric models M and N is projected into the Laplace space. F and G include both the global descriptor and the descriptor of each segment. The second item is the global exchange constraint. is a global operator; the third term is a local exchange constraint, is the operator on each segment; C, F, G, R, S, R i , S i Deform to obtain a closed-form solution to C; In the step (4), it is assumed that use Represents the i-th row vector of matrix A, using Denotes the j-th column vector of matrix A, using represents the vectorization operator of the matrix, use Defined as the expanded matrix of A, Where Diag(·,·,...,·) is a diagonal block matrix construction operator. Here, each 0 is a zero matrix of size m×n, so formula (2) is equivalent to the following formula: It is equivalent to the following formula: Its solution is Here each 0 is m 2 dimensional zero vector, and vec(C) is used to restore the closed solution C of formula (2).

2. A non-rigid matching device for a geometric model using local and global information, characterized in that: The device is used to execute the method according to claim 1, and comprises: A data input module, whose input is a pair of geometric models representing different postures of the same object, and the data type of the model is a mesh type or a point cloud type; The cutting module calculates the density of each point in the model and cuts the geometric model according to the point density; the segment matching module matches the segmented segments using the inter-segment error and intra-segment error. Match, get the segment matching result; The point matching module uses the segment matching information to output the point matching results.

Images