A method for graph matching of three-dimensional models
Patent Information
- Application Number
- CN202410906812.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-08
- Publication Date
- 2026-09-04
- Estimated Expiration
- 2044-07-08
AI Technical Summary
例如，利用高斯曲率和测地距离调整混淆混淆时，对于前后曲率相似以及特征点测地距离相近模型仍会匹配错误；为了解决多对称图形的匹配问题，在对称点检测上引入点对支持程度，但算法针对性较强；利用Gromov-Wasserstein距离提出的图形匹配方法，在求解过程当中采取双向约束，通过区域加点的几何信息判断图形前后左右，可以很好的解决简单模型的匹配，但在对称检测方面上对于多对称图形的对称点检测正确率较低，出现交叉匹配错误和前后错误的现象；利用PointNet框架处理点云信息，由于最大池化考虑全局信息，对发生非刚性形变图形的局部信息表示不足；为了增加对于局部信息的描述，后来引入谱特征描述符，对模型进行局部和全局深度信息提取，算法对部分模型存在局限且计算复杂度较高
[0039]本发明解决了对3D对称图形的初始层采样点进行匹配的过程中，容易发生的对称混淆问题。相较于解决相同问题的基于图形嵌入的方法，本发明极大改善了初始层匹配的准确率和计算时间。基于更准确的初始层匹配结果，后续利用分层匹配算法进行稠密匹配时的准确率也有很大程度地提高。
Smart Images
Figure CN118887430B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of three-dimensional model processing technology, and particularly relates to a method for graphic matching of three-dimensional models. Background Technology
[0002] In existing research on image matching, scholars have proposed many algorithms for both rigid and non-rigid model matching. Since the internal points of a rigidly transformed object remain unchanged, the transformation matrix of the model can be calculated for point cloud registration. Relatively mature methods exist for solving rigid matching problems. However, due to the arbitrary deformation of the model in non-rigid matching, its transformation cannot be represented by simple matrices, making the study of non-rigid matching more challenging. For symmetric models undergoing non-rigid transformations, local features deform. During the matching process, due to the high similarity of geometric features in the symmetrical parts of the object, symmetry confusion often occurs. Solving the symmetry confusion problem and finding the correct correspondence between images remains a research hotspot in image matching. The key to solving the symmetry confusion problem lies in classifying the feature point set; accurate point set classification can effectively solve the symmetry confusion problem.
[0003] To address this challenge, 3D model matching practices typically encompass three aspects: feature point extraction, symmetry detection, and point cloud matching. Since models retain a large number of invariant geometric descriptors after non-rigid transformations, solutions can be proposed for feature point extraction, feature point set classification, symmetry point grouping, and distinguishing the model's top, bottom, left, and right sides. For example, when adjusting the obfuscation using Gaussian curvature and geodesic distance, matching errors still occur for models with similar curvature and similar geodesic distances between feature points. To address the matching problem of multi-symmetric figures, point pair support is introduced in symmetry point detection, but the algorithm is highly specific. The figure matching method proposed using Gromov-Wasserstein distance employs bidirectional constraints during the solution process, determining the front, back, left, and right sides of the figure through the geometric information of the region and points, which can effectively solve the matching of simple models. However, in terms of symmetry detection, the accuracy of symmetry point detection for multi-symmetric figures is low, resulting in cross-matching errors and front-back errors. Using the PointNet framework to process point cloud information, due to the fact that max pooling considers global information, it is insufficient in representing the local information of figures undergoing non-rigid deformation. To improve the description of local information, spectral feature descriptors are introduced to extract local and global depth information of the model, but the algorithm has limitations for some models and has high computational complexity. Therefore, a figure matching method for three-dimensional models is proposed. Summary of the Invention
[0004] The purpose of this invention is to provide a graphical matching method for three-dimensional models, aiming to solve the problems mentioned in the background art.
[0005] To achieve the above objectives, the present invention provides the following technical solution:
[0006] A method for graphical matching of a 3D model includes the following steps:
[0007] Step 1, Feature Point Extraction: Input the simplified 3D model point cloud information to extract feature points. For different models, select 5 to 10 feature points according to the characteristics of the model.
[0008] Step 2, Symmetry Detection: The Gromov-Wasserstein distance is used to compare the symmetry between two sets of graphical points, and the matching probability between model feature points is calculated to classify the feature point sets and complete the matching of asymmetric point sets. The heat kernel difference matrix of the symmetric point sets is calculated, and the symmetry decomposition distance is constructed. Symmetric point pairs are selected from the symmetric point sets, and the geodesic distance between point pairs is calculated. The geodesic distance is used to determine the support of point pairs, and the final symmetric point grouping information is determined, completing the symmetry point detection.
[0009] Step 3, Graphical Matching: Arrange the grouped symmetrical point pairs in descending order of geodesic distance. Calculate two new auxiliary points using the information from the sampling points. The new auxiliary points, existing symmetrical points, and the model center point form a vector. Determine the vertical direction of the model through the positional relationship between the vectors. Then, according to the right-hand rule, determine the horizontal direction using the vector product of the vertical and reference points to complete the final matching.
[0010] Furthermore, the specific process of step 2 is as follows:
[0011] The Gromov-Wasserstein distance is used to compare the symmetry between two point sets of graphs, matching feature points of two models. X and Y are two given metric spaces, and each model in X and Y takes n points. U = μ(X,Y) is defined as the joint measure of spaces X and Y. The matching probability between model feature points is obtained from the Gromov-Wasserstein distance. Points with a matching probability greater than 0.6 are considered as the correct correspondence of the model's asymmetric point set, thus obtaining the correct matching result between the model's asymmetric points. The symmetry difference between two point pairs (x,y) and (x′,y′) of the model is:
[0012] R(x,x′)=max(|d(x,x′)-d(y,y′)|,|d(x,y′)-d(y,x′)|);
[0013] Where d is the geodesic distance;
[0014] The HKS difference of feature points is used to measure the degree of symmetry between feature points, forming a symmetric difference matrix G. The matrix reflecting the symmetry relationship is then decomposed into eigenvalues and embedded into the spectral space. The distance difference matrix obtained by measuring the degree of symmetry using Euclidean distance is denoted as the symmetric decomposition distance.
[0015] G(X i )=(λ1φ1(X i ),λ2φ2(X i ),...,λ n φ n (X i ));
[0016] Where n is the number of feature points, λ i φ is the i-th eigenvalue of the matrix. i (X i X is the point X of the i-th eigenvector of the matrix. i The corresponding item;
[0017] After embedding, a symmetric decomposition matrix is constructed using Euclidean distance to determine the degree of symmetry between points:
[0018] D(X i ,X j )=||G(X i )-G(X j )|| 2 ;
[0019] The value of the symmetric decomposition distance matrix is inversely proportional to the similarity between two points. A threshold is set, and all point pairs whose symmetric decomposition distance is less than the threshold are selected as the set M of possible symmetric point pairs. All point pairs in M satisfy the following condition:
[0020] D(X i ,X j )≤μ×max(D(X i ,X j ));
[0021] Based on the symmetry difference between possible symmetrical points in the geodesic distance calculation model, the symmetry difference is inversely proportional to the model similarity. When R(x,x′)≤μ×max(R), the point pairs (x,y) and (x′,y′) together reflect the symmetry relationship of the model and are a set of support point pairs. The group with the most support point pairs and the largest inner product is selected as the final symmetry point grouping result.
[0022] Furthermore, the specific process of step 3 is as follows:
[0023] Choose an appropriate distance parameter d, calculate the point set Q with a geodesic distance of d from the reference point, calculate the distance between all points in the point set Q and the symmetrical point pair, find the point with the smallest distance difference from the symmetrical point pair as the first added point, and then select the point with the largest Euclidean distance from the first added point from the point set Q as the second added point.
[0024] Determine the top and bottom: Use vector calculation to determine the angle between the vector between the two added points and the center point and symmetric point of the model, determine the positions of the top and bottom points and determine the positive direction of the model; the steps to calculate the positive direction of the model are as follows:
[0025] Input: The feature point set S after adding auxiliary points;
[0026] Output: Positive direction of the model
[0027] Step 1: Extract the coordinates of the feature point set from the 3D model point cloud data, where the first added point is A, the second point is B, and calculate the coordinates M of the model center point;
[0028] Step 2: Calculate the vector between the two added points.
[0029] Step 3: Select the symmetrical point C, and take... calculate like Then, the first added point A is the upper point and B is the lower point; otherwise, the coordinates of A and B are swapped, with A being the upper point and B the lower point. The opposite vector is the positive direction of the model;
[0030] Determining Left and Right: After calibrating the top and bottom points of the model, the left and right sides of the figure are determined using the right-hand rule and the vector product of the top and bottom points and the reference point. Specifically, the process is as follows: Let the vector product of the midpoint O of the two added points and the selected reference point S form a vector... Will Record Compare the vectors formed by the symmetrical points after splitting and point O. The sign of the product is determined by the fact that when the product of vectors formed by points symmetrical to one side is positive, it is defined as the left side.
[0031] Furthermore, the specific operation of step 1 is as follows:
[0032] First, H-WKS sampling points are calculated, which involves calculating the thermal kernel descriptor and wave kernel descriptor for each point in the point cloud. The thermal kernel descriptor and wave kernel descriptor are then fused to form H-WKS. The maximum value of H-WKS in the neighborhood of the model is selected as the H-WKS sampling point. Next, the farthest point is sampled for each model. The two sampling results are put together to remove duplicate points. Then, redundant points are removed by a fusion sampling algorithm. Finally, 5 to 10 points are selected as model feature points.
[0033] Furthermore, the formula for fusing the thermonuclear descriptor and the wavenuclear descriptor to form the H-WKS step is as follows:
[0034] H-WKS(x)=αHKS(x,t)+(1-α)WKS(x,E);
[0035] in:
[0036]
[0037] Where HKS is the thermal kernel descriptor, WKS is the wave kernel descriptor, α is the weighting coefficient, x is a point, E is the energy, t is the time variable, n is the number of feature points, and φ is the weighting coefficient. k It is the k-th eigenfunction of the Laplace matrix, φ k (x) is the corresponding term of the point x of the k-th eigenfunction of the matrix, λ k is the k-th eigenvalue of the matrix, and f represents the energy distribution.
[0038] Compared with the prior art, the beneficial effects of the present invention are:
[0039] This invention solves the symmetry confusion problem that easily occurs during the matching of initial layer sampling points in 3D symmetrical graphics. Compared with graph embedding-based methods that solve the same problem, this invention significantly improves the accuracy and computation time of initial layer matching. Based on the more accurate initial layer matching results, the accuracy of subsequent dense matching using hierarchical matching algorithms is also greatly improved. Attached Figure Description
[0040] Figure 1 This is a flowchart of the algorithm of the present invention.
[0041] Figure 2 This is a schematic diagram of the sampling results of the present invention.
[0042] Figure 3 This is a schematic diagram of the asymmetric point matching results of the present invention.
[0043] Figure 4 This is the result of the symmetry point detection in this invention.
[0044] Figure 5 The results of this invention are shown; the left image is before adjustment, and the right image is after adjustment. Detailed Implementation
[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0046] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.
[0047] like Figure 1 As shown, a graphical matching method for a three-dimensional model provided in one embodiment of the present invention includes the following steps:
[0048] Step 1, Feature Point Extraction: To reduce computational load, feature points are extracted from the simplified 3D model point cloud information. For different models, 5 to 10 feature points are selected according to the model characteristics.
[0049] The specific operation of step 1 is as follows:
[0050] Input point cloud models 1 and 2, and perform initial layer sampling on the point cloud data of the input models. For different models, 5 to 10 feature points are ultimately selected, which satisfies the feature extraction requirements while avoiding the computational cost of too many feature points. The thermal kernel descriptor and the wave kernel descriptor are fused to form H-WKS:
[0051] H-WKS(x)=αHKS(x,t)+(1-α)WKS(x,E);
[0052] in:
[0053]
[0054] Where HKS is the thermal kernel descriptor, WKS is the wave kernel descriptor, α is the weighting coefficient, x is a point, E is the energy, t is the time variable, n is the number of feature points, and φ is the weighting coefficient. k It is the k-th eigenfunction of the Laplace matrix, φ k (x) is the corresponding term of the point x of the k-th eigenfunction of the matrix, λ k is the k-th eigenvalue of the matrix, and f represents the energy distribution.
[0055] The maximum value of H-WKS within the model's neighborhood is selected as the H-WKS sampling point. Then, for each model, the farthest point is sampled. The two sampling results are combined to remove duplicate points. Finally, a fusion sampling algorithm is used to remove redundant points, avoiding sampling point accumulation and ensuring consistency in sampling points for each model class. The sampling results are as follows: Figure 2 As shown.
[0056] Step 2, Symmetry Detection: The Gromov-Wasserstein distance is used to compare the symmetry between two sets of graphical points, and the matching probability between model feature points is calculated to classify the feature point sets and complete the matching of asymmetric point sets. The heat kernel difference matrix of the symmetric point sets is calculated, and the symmetry decomposition distance is constructed. Symmetric point pairs are selected from the symmetric point sets, and the geodesic distance between point pairs is calculated. The geodesic distance is used to determine the support of point pairs, and the final symmetric point grouping information is determined, completing the symmetry point detection.
[0057] The specific process of step 2 is as follows:
[0058] Based on the isometric invariance of the Gromov-Wasserstein distance, this method compares the symmetry between two graph point sets using the Gromov-Wasserstein distance, matching feature points of two models. X and Y are two given metric spaces, and each model in X and Y takes n points. U = μ(X,Y) is defined as the joint measure of spaces X and Y. The matching probability between model feature points is calculated using the Gromov-Wasserstein distance. Points with a matching probability greater than 0.6 are considered as correct correspondences between asymmetric point sets of the model, thus obtaining the correct matching results between asymmetric points of the model. Figure 3 As shown; the symmetry dissimilarity of two point pairs (x,y) and (x′,y′) in the model is:
[0059] R(x,x′)=max(|d(x,x′)-d(y,y′)|,|d(x,y′)-d(y,x′)|);
[0060] Where d is the geodesic distance;
[0061] The HKS difference measure of feature points is used to measure the degree of symmetry between feature points, forming a symmetry difference matrix G. Borrowing from spectral analysis methods, while preserving geometric and topological information, the matrix reflecting symmetry is eigenvalued and embedded into the spectral space. The distance difference matrix obtained by measuring the degree of symmetry using Euclidean distance is denoted as the symmetry decomposition distance.
[0062] G(X i )=(λ1φ1(X i ),λ2φ2(X i ),...,λ n φ n (X i ));
[0063] Where n is the number of feature points, λ i φ is the i-th eigenvalue of the matrix. i (X i X is the point X of the i-th eigenvector of the matrix.i The corresponding item;
[0064] After embedding, a symmetric decomposition matrix is constructed using Euclidean distance to determine the degree of symmetry between points:
[0065] D(X i ,X j )=||G(X i )-G(X j )|| 2 ;
[0066] The value of the symmetric decomposition distance matrix is inversely proportional to the similarity between two points. A threshold is set, and all point pairs whose symmetric decomposition distance is less than the threshold are selected as the set M of possible symmetric point pairs. All point pairs in M satisfy the following condition:
[0067] D(X i ,X j )≤μ×max(D(X i ,X j ));
[0068] Based on the symmetry difference between possible symmetrical points in the geodesic distance calculation model, the symmetry difference is inversely proportional to the model similarity. When R(x,x′)≤μ×max(R), the point pairs (x,y) and (x′,y′) together reflect the symmetry relationship of the model and are a set of support point pairs. The group with the most support point pairs and the largest inner product is selected as the final symmetry point grouping result. Figure 4 This is a partial schematic diagram of the results of symmetry point detection on the model. Points of the same color on the model are a group of symmetry points.
[0069] Step 3, Graphical Matching: Arrange the grouped symmetrical point pairs in descending order of geodesic distance. Calculate two new auxiliary points using the information from the sampling points. The new auxiliary points, existing symmetrical points, and the model center point form a vector. Determine the vertical direction of the model through the positional relationship between the vectors. Then, according to the right-hand rule, determine the horizontal direction using the vector product of the vertical and reference points to complete the final matching.
[0070] The specific process of step 3 is as follows:
[0071] Choose an appropriate distance parameter d, calculate the point set Q with a geodesic distance of d from the reference point, calculate the distance between all points in the point set Q and the symmetrical point pair, find the point with the smallest distance difference from the symmetrical point pair as the first added point, and then select the point with the largest Euclidean distance from the first added point from the point set Q as the second added point.
[0072] Determining the top and bottom: Using vector knowledge, we can calculate the angle between the vector between the two added points and the center point and symmetric point of the model to determine the positions of the top and bottom points, and thus determine the positive direction of the model; the steps for calculating the positive direction of the model are as follows:
[0073] Input: The feature point set S after adding auxiliary points;
[0074] Output: VR in the positive direction of the model;
[0075] Step 1: Extract the coordinates of the feature point set from the 3D model point cloud data, where the first added point is A, the second point is B, and calculate the coordinates M of the model center point;
[0076] Step 2: Calculate the vector between the two added points.
[0077] Step 3: Select the symmetrical point C, and take... calculate like Then, the first added point A is the upper point and B is the lower point; otherwise, the coordinates of A and B are swapped, with A being the upper point and B the lower point. The opposite vector is the approximate positive direction of the model;
[0078] Determining Left and Right: After calibrating the top and bottom points of the model, for matching cases with the same symmetrical direction in model matching, only the left and right sides of the model need to be calibrated. Based on the right-hand rule, the left and right sides of the figure are determined using the vector product of the top and bottom points and the reference point. Specifically, let the vector product of the midpoint O of the two added points and the selected reference point S form a vector... Will Record Compare the vectors formed by the symmetrical points after splitting and point O. The sign of the product is determined by the fact that when the product of vectors formed by points symmetrical to one side is positive, it is defined as the left side.
[0079] The model confirms the up and down information by adding points, determines the left and right by using vector properties, and adjusts the matching objects. Compared with the previous matching results, the matching in the same symmetrical direction has been improved, and matching results in the same direction are obtained between symmetrical points. Figure 5 For display of results.
[0080] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these should also be considered within the scope of protection of the present invention. These modifications and improvements will not affect the effectiveness of the implementation of the present invention or the practicality of the patent.
Claims
1. A method for graphical matching of a three-dimensional model, characterized in that, Includes the following steps: Step 1, Feature Point Extraction: Input the simplified 3D model point cloud information to extract feature points. For different models, select 5 to 10 feature points according to the characteristics of the model. Step 2, Symmetry Detection: The Gromov-Wasserstein distance is used to compare the symmetry between two sets of graphical points, and the matching probability between model feature points is calculated to classify the feature point sets and complete the matching of asymmetric point sets. The heat kernel difference matrix of the symmetric point sets is calculated, and the symmetry decomposition distance is constructed. Symmetric point pairs are selected from the symmetric point sets, and the geodesic distance between point pairs is calculated. The geodesic distance is used to determine the support of point pairs, and the final symmetric point grouping information is determined, completing the symmetry point detection. Step 3, Graphical Matching: Arrange the grouped symmetrical point pairs in descending order of geodesic distance. Calculate two new auxiliary points using the information from the sampling points. The new auxiliary points, existing symmetrical points, and the model center point form a vector. Determine the vertical direction of the model through the positional relationship between the vectors. Then, according to the right-hand rule, determine the horizontal direction using the vector product of the vertical and horizontal points and the reference point to complete the final matching. The specific process of step 2 is as follows: The Gromov-Wasserstein distance is used to compare the symmetry between two sets of graphical points and to match feature points between the two models. X , Y For two given metric spaces, in X , Y The models above all adopt n A point, defined For space X , Y The joint measure is used to calculate the matching probability between model feature points using the Gromov-Wasserstein distance; points with a matching probability greater than 0.6 are considered as the correct correspondence of the model's asymmetric point set, thus obtaining the correct matching result between the model's asymmetric points; for two point pairs of the model... and The degree of symmetry difference is: ； in, d For geodetic distance; The HKS difference measure is used to measure the symmetry between feature points, forming a symmetric difference matrix. G The matrix representing the symmetry relationship is decomposed into eigenvalues and embedded into the spectral space. The distance difference matrix obtained by measuring the degree of symmetry using Euclidean distance is denoted as the symmetry decomposition distance. ； in, It is the number of feature points. It is the first of the matrix 1 eigenvalue, It is the matrix number i Points of eigenvectors The corresponding item; After embedding, a symmetric decomposition matrix is constructed using Euclidean distance to determine the degree of symmetry between points: ； The symmetric decomposition distance matrix value is inversely proportional to the similarity between two points. A threshold is set, and all point pairs whose symmetric decomposition distance is less than the threshold are selected as the set of possible symmetric point pairs. M , M Midpoint pairs all satisfy: ; Based on the calculation of the symmetry difference between possible symmetrical points in the model using geodesic distance, the symmetry difference is inversely proportional to the model similarity. At that time, point to point and The symmetry relationship of the model is reflected by a set of support point pairs. The group with the most support point pairs and the largest inner product is selected as the final symmetry point grouping result.
2. The graphical matching method for three-dimensional models according to claim 1, characterized in that, The specific process of step 3 is as follows: Select appropriate distance parameters d The geodetic distance from the reference point is calculated as follows: d point set Q Calculate the point set Q For all points, find the point with the smallest distance difference to its symmetrical pair and add it as the first point. Then, from the point set... Q Select the point with the largest Euclidean distance from the first point as the second point to add; Determine the top and bottom: Use vector calculation to determine the angle between the vector between the two added points and the center point and symmetric point of the model, determine the positions of the top and bottom points and determine the positive direction of the model; the steps to calculate the positive direction of the model are as follows: Input: Feature point set after adding auxiliary points ; Output: Positive direction of the model ; Step 1: Extract the coordinates of the feature point set from the 3D model point cloud data, where the first added point is... The second point is Calculate the coordinates of the center point of the model. ; Step 2: Calculate the vector between the two added points. ; Step 3: Select symmetrical points ,Pick ,calculate ,like The first point added The point above, The lower point is the one that is directly below; otherwise, they are interchanged. and coordinate, For the best, If it is below, then The opposite vector is the positive direction of the model; Determining Left and Right: After calibrating the top and bottom points of the model, the left and right sides of the figure are determined using the right-hand rule and the vector product of the top and bottom points and the reference point. The specific process is as follows: Set the midpoint of the two added points... With the selected reference point The vector formed ,Will Record Compare the symmetrical points after splitting the sides with The vector formed by the point and The sign of the product is determined by the fact that when the product of vectors formed by points symmetrical to one side is positive, it is defined as the left side.
3. The graphical matching method for a three-dimensional model according to claim 1, characterized in that, The specific operation of step 1 is as follows: First, H-WKS sampling points are calculated, which involves calculating the thermal kernel descriptor and wave kernel descriptor for each point in the point cloud. The thermal kernel descriptor and wave kernel descriptor are then fused to form H-WKS. The maximum value of H-WKS in the neighborhood of the model is selected as the H-WKS sampling point. Next, the farthest point is sampled for each model. The two sampling results are put together to remove duplicate points. Then, redundant points are removed by a fusion sampling algorithm. Finally, 5 to 10 points are selected as model feature points.
4. The graphical matching method for a three-dimensional model according to claim 3, characterized in that, The formula for fusing the thermonuclear descriptor and the wavenuclear descriptor to form the H-WKS is as follows: H-WKS(x)=αHKS(x,t)+(1-α)WKS(x,E); in: Where HKS is the thermal kernel descriptor, WKS is the wave kernel descriptor, α is the weighting coefficient, x is a point, E is the energy, t is the time variable, and n is the number of feature points. φ k It is the k-th eigenfunction of the Laplace matrix. φ k (x) is the corresponding term of point x of the k-th eigenfunction of the matrix. λ k is the k-th eigenvalue of the matrix, and f represents the energy distribution.