A 3D symmetric graphics matching method based on initial layer sampling points
By using a mathematical optimization model based on Gromov-Wasserstein distance and auxiliary point information, the symmetry confusion problem in 3D graphic matching was solved, improving accuracy and reducing computational cost, thus achieving efficient symmetric graphic matching.
Patent Information
- Application Number
- CN202310511209.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-09
- Publication Date
- 2025-10-28
- Estimated Expiration
- 2043-05-09
AI Technical Summary
Existing technologies struggle to accurately distinguish symmetry confusion in 3D graphics matching, resulting in low matching accuracy, high computational cost, and long processing time.
A mathematical optimization model based on Gromov-Wasserstein distance is adopted. By combining the fusion sampling of initial layer sampling points and auxiliary point information with regional heat core signals, the graphic distortion is adjusted to distinguish between symmetrical and asymmetrical points. The outer product of vectors is used to distinguish between left and right, thereby improving the matching accuracy.
It effectively avoids symmetry confusion, improves matching accuracy, reduces computational load, and shortens processing time.
Smart Images
Figure CN116433942B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer graphics research and application, and in particular relates to a 3D symmetric graphics matching method based on initial layer sampling points. Background Technology
[0002] 3D non-rigid graphics matching is a key problem in computer graphics research and applications, with symmetry confusion being a persistent challenge. The accuracy of dense matching of 3D graphics heavily depends on the matching results of the initial layer, and the key challenge in the initial layer matching is the susceptibility to symmetry confusion. Therefore, solving the symmetry confusion problem is crucial for improving the accuracy of 3D graphics matching. Currently, graphics matching problems are often transformed into distance problems in a measure space, such as measuring distance. Existing distance descriptors cannot distinguish symmetric points. Therefore, in practical applications, it is necessary to first distinguish the category of the sampling points and then make judgments on symmetric points to overcome symmetry confusion.
[0003] To address the symmetry confusion problem, some researchers have conducted studies. For example, using thermonuclear signals, an effective and stable solution can be provided for the equidistant shape matching problem when symmetry exists; or symmetrical structures can be detected by analyzing the probability distribution of all possible intrinsic matches between a partial shape and itself; or deep learning methods can be used to generate two high-quality shape correspondences and detect the inherent symmetry of left-right reflections of each shape. These methods have all solved the symmetry confusion problem to some extent, but the procedures are complex and the accuracy needs further improvement.
[0004] Currently, there are methods based on image embedding, which match the initial layer sampling points by embedding the 3D image into 3D Euclidean space. After distortion adjustment, it is projected onto a 2D plane, and then symmetry is adjusted and flipped to distinguish left from right, thus solving the symmetry confusion problem. This method is simple in principle, easy to implement, and has achieved good results in solving this problem. However, it still has drawbacks such as the need to improve the accuracy of distinguishing symmetrical points using the average geodesic distance, and the large amount of computation and long time consumption due to global embedding. Summary of the Invention
[0005] The purpose of this invention is to provide a 3D symmetrical graphic matching method based on initial layer sampling points, which aims to solve the symmetry confusion problem that easily occurs during the matching of initial layer sampling points of 3D symmetrical graphics.
[0006] This invention is implemented as follows: a 3D symmetric graphic matching method based on initial layer sampling points, comprising the following steps:
[0007] Step 1: Initial Layer Sampling
[0008] a. 3D point cloud data, which is the initial layer sampling of the 3D point cloud data of the two input graphics. The sampling points of the initial layer are selected from 5 to 11 key points according to the different categories of graphics.
[0009] b. Calculate the heat core sampling point, that is, calculate the heat core feature descriptor of each point in the point cloud, and select the point with the largest heat core value in the second ring neighborhood as the heat core sampling point;
[0010] c. Add the farthest sampling point. Using the above-mentioned hot core sampling points as the initial point set, add 10 to 30 points using the farthest sampling method to obtain a new feature point set;
[0011] d. Fusion sampling: The new feature point set is processed using a fusion sampling algorithm, and finally 5 to 11 points are selected as the initial layer sampling points;
[0012] Step 2: Initial Layer Matching
[0013] e. Distinguish between symmetrical and asymmetrical points, establish a mathematical optimization model using the GW distance, and distinguish between symmetrical and asymmetrical points by solving the mathematical model. The matching scheme for asymmetrical points adopts this solution result.
[0014] f. To distinguish between the front and back, the same algorithm is used to add four auxiliary points to both images. Using the GW distance optimization model mentioned above, the front and back points of the auxiliary points are matched, and auxiliary points are added and regional heat core information is used.
[0015] g. Adjusting graphic distortion: Utilizing the property that geodesic distances remain approximately equidistant after graphic deformation, an optimization problem is established, and the point positions are changed to adjust the graphic distortion;
[0016] h. Distinguish between left and right: For the two distorted figures after adjustment, use the vector outer product to distinguish between left and right and obtain the matching result of the symmetrical points.
[0017] A further technical solution involves the following specific steps for distinguishing between symmetrical and asymmetrical points:
[0018] 1) Perform initial layer sampling on the point cloud data of the two input 3D graphics to obtain their respective sampling point sets, with the number of sampling points being m and n, respectively;
[0019] 2) Apply the mathematical optimization model based on GW distance to the above sampling point set to obtain the matching result, X=(x i ) mn×1 x i ∈[0,1], the optimization model is established as shown in equation (1):
[0020] minf(x), X∈R mn×1 (1)
[0021] in:
[0022]
[0023] Solving the above optimization problem model using gradient descent yields the matching probability X, which is then transformed into a joint probability matrix U, where U(i,j)=x (i-1)n+j ;
[0024] Set a threshold T, T = 0.05, and make the following judgments to distinguish between symmetrical and asymmetrical points: if |U(i,k)-U(i,l)| < T, then the k-th point and the l-th point in Figure 2 are symmetrical points; if |U(i,k)-U(j,k)| < T, then the i-th point and the j-th point in Figure 1 are symmetrical points; and when U(i,j) ≥ 0.8, the i-th point in Figure 1 and the j-th point in Figure 2 are a pair of correctly matched asymmetrical points.
[0025] A further technical solution, the specific operation of distinguishing before and after is as follows:
[0026] 1) Select an asymmetric point from the initial layer sampling points that has the largest distance from the remaining point set;
[0027] 2) Initialize the distance d0, and find a point around the circle whose distance to the selected asymmetric point is approximately equal to d0;
[0028] 3) Select a point on the axis of symmetry of the graphic from the above points. The point on the axis of symmetry is the first auxiliary point information added.
[0029] 4) Select points on the axis of symmetry of the figure, but the points with larger Euclidean distances from the first point are the second auxiliary points to be added;
[0030] 5) Select the remaining two auxiliary points from the points around the circle, such that each point is equidistant from the first and second auxiliary points and is relatively far apart from the others; that is, among the points around the circle that are equidistant from a certain fixed asymmetric point, select four uniform points that divide the circle into four equal parts.
[0031] 6) The initial layer sampling points and the added auxiliary point information are used together as the set of points to be matched. The matching is performed using the above formula (1), and the corresponding results of the front and back points of the two graphics are output.
[0032] 7) Continue to add auxiliary points: Based on the above sampling points, continue to add auxiliary points in the forward and backward directions;
[0033] 8) Regional thermonuclear information: The thermonuclear signal value of each point involved in the calculation is improved from the single-point thermonuclear signal value to the regional thermonuclear signal value, that is, the original single-point thermonuclear signal value:
[0034] Change to:
[0035] A further technical solution involves adjusting the graphic distortion as follows:
[0036] The input consists of the three-dimensional coordinates X = (x1, x2, ..., xn) of the point set composed of the initial layer sampling points and the aforementioned auxiliary points. m )'∈R m×3 Using this as the initial value, the geodesic distance matrix D of the point set is used. geo And Euclidean distance matrix D Euclid A new optimization problem model is established and solved to obtain the coordinates of a new set of points; the optimization problem is as follows:
[0037] minloss, X∈R m×3 Where, loss = ||D geo -D Euclid ||2.
[0038] A further technical solution, the specific operation of distinguishing left and right is as follows:
[0039] Using the position coordinates of the point sets of the two graphics after the above optimization, calculate the symbol label for each pair of symmetrical points.
[0040] Calculate the symbol labels for the points symmetrical to both sides of the figure:
[0041] For point p i (i = 1, 3), we have Then point p i The label is positive;
[0042] For point p j (j=2,4), we have Then point p j The label is negative;
[0043] Perform the same calculations on graph two. If the labels are the same, the points are judged to be the correct corresponding points, and the final matching result is output.
[0044] Given the initial layer matching result, the two graphics to be matched are then subjected to dense matching using a layered sampling algorithm.
[0045] This invention provides a 3D symmetric graphic matching method based on initial layer sampling points. This application uses the Gromov-Wasserstein distance, which is more effective against topological noise, to establish a corresponding mathematical optimization model and proposes the theoretical framework of the algorithm. It proposes effective solutions for steps such as symmetric point classification, front-back distinction, distortion adjustment, and left-right distinction based on the initial layer sampling points. First, a fusion sampling method is used to perform initial layer sampling on the two input graphics to obtain key points. The results of the optimization model are used to classify symmetric points and match asymmetric points. By adding auxiliary point information, the Gromov-Wasserstein distance of the two feature point sets is used to distinguish the front and back of the graphics. The feature points are embedded into Euclidean space to adjust the distortion of the graphics and accurately determine the left and right sides, more effectively avoiding symmetry confusion. The algorithm proposed in this invention adds only a few points to the initial layer sampling points. By solving the mathematical model established using the Gromov-Wasserstein distance, it distinguishes the symmetric point categories and front-back distinctions. Furthermore, it only embeds the current initial layer sampling points and the added points into 3D Euclidean space. Compared with graphic-based embedding methods, the embedding process requires less computation and saves computation time. At the same time, this invention also significantly improves the final matching accuracy. Attached Figure Description
[0046] Figure 1 This is a flowchart of the initial layer sampling process provided in an embodiment of the present invention;
[0047] Figure 2 The initial layer matching flowchart provided in the embodiments of the present invention;
[0048] Figure 3 This is a schematic diagram illustrating the addition of four auxiliary point information as provided in an embodiment of the present invention;
[0049] Figure 4 A schematic diagram illustrating the addition of auxiliary points as provided in an embodiment of the present invention;
[0050] Figure 5 The positional diagram before optimization provided in the embodiments of the present invention;
[0051] Figure 6 The optimized positional diagram provided for embodiments of the present invention;
[0052] Figure 7 The vector outer product graph provided in the embodiments of the present invention;
[0053] Figure 8 The diagram shows the effect of gradually increasing the matching points according to an embodiment of the present invention. Detailed Implementation
[0054] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0055] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.
[0056] like Figure 1-Figure 2 As shown, this is an embodiment of the present invention providing a 3D symmetric graphic matching method based on initial layer sampling points. The algorithm implementation flow is as follows:
[0057] Step 1: Initial Layer Sampling
[0058] a. 3D point cloud data, which is the initial layer sampling of the 3D point cloud data of the two input graphics. The sampling points of the initial layer are selected from 5 to 11 key points according to the different categories of graphics.
[0059] b. Calculate the heat core sampling point, that is, calculate the heat core feature descriptor of each point in the point cloud, and select the point with the largest heat core value in the second ring neighborhood as the heat core sampling point;
[0060] c. Add the farthest sampling point. Using the above-mentioned hot core sampling points as the initial point set, add 10 to 30 points using the farthest sampling method to obtain a new feature point set;
[0061] d. Fusion sampling: The new feature point set is processed using a fusion sampling algorithm, and finally 5 to 11 points are selected as the initial layer sampling points;
[0062] Step 2: Initial Layer Matching
[0063] This section presents a solution to the symmetric confusion that occurs during initial layer matching;
[0064] e. Distinguish between symmetrical and asymmetrical points, establish a mathematical optimization model using the GW distance, and distinguish between symmetrical and asymmetrical points by solving the mathematical model. The matching scheme for asymmetrical points adopts this solution result.
[0065] f. To distinguish between front and back, the same algorithm is used to add four auxiliary points to both images. Using the GW distance optimization model mentioned above, the front and back points of the auxiliary points are matched. To accurately match the front and back points, two additional improvements are proposed: to continue adding auxiliary points and to use regional heat core information, so as to improve the accuracy of distinguishing between front and back.
[0066] g. Adjusting graphic distortion: Utilizing the property that geodesic distances remain approximately equidistant after graphic deformation, an optimization problem is established, and the point positions are changed to adjust the graphic distortion;
[0067] h. Distinguish between left and right: For the two distorted figures after adjustment, use the vector outer product to distinguish between left and right, and obtain the matching result of symmetrical points, effectively avoiding symmetry confusion.
[0068] In this embodiment of the invention, two input graphics to be matched are sampled according to the initial layer sampling algorithm described above. This process is relatively simple, and the sampling algorithm is stable. The sampling positions and numbers are basically the same for graphics of the same category. The number range in the algorithm is set to 5 to 11, and the specific sampling number is adaptively selected by the algorithm based on the characteristics of the input 3D model itself. The points sampled in the initial layer are matched, and symmetry confusion is adjusted during this matching process. The specific implementation process is as follows;
[0069] The two input point cloud graphics are described as Graphic 1 and Graphic 2, respectively;
[0070] 1) Perform initial layer sampling on the point cloud data of the two input 3D graphics to obtain their respective sampling point sets, with the number of sampling points being m and n, respectively;
[0071] 2) Apply the mathematical optimization model based on GW distance to the above sampling point set to obtain the matching result, X=(x i ) mn×1 x i ∈[0,1], the optimization model is established as shown in equation (1):
[0072] minf(x), X∈R mn×1 (1)
[0073] in:
[0074]
[0075] Solving the above optimization problem model using gradient descent yields the matching probability X, which is then transformed into a joint probability matrix U, where U(i,j)=x (i-1)n+j ;
[0076] Set a threshold T, T = 0.05, and make the following judgments to distinguish between symmetrical and asymmetrical points: if |U(i,k)-U(i,l)| < T, then the k-th point and the l-th point in Figure 2 are considered symmetrical points; if |U(i,k)-U(j,k)| < T, then the i-th point and the j-th point in Figure 1 are considered symmetrical points; and when U(i,j) ≥ 0.8, the i-th point in Figure 1 and the j-th point in Figure 2 are considered a pair of correctly matched asymmetrical points.
[0077] 3) Add auxiliary points based on the initial layer sampling points. The process of adding auxiliary points is as follows:
[0078] In the initial layer sampling points, select an asymmetric point with the largest distance to the remaining point set; initialize the distance d0, and find a circle of points whose distance to the selected asymmetric point is approximately equal to d0; among these circle of points, select points on the graph's axis of symmetry (equal to the distance to the symmetric point), and these points on the graph's axis of symmetry become the first auxiliary point information; select points on the graph's axis of symmetry, but with larger Euclidean distances to the first point, and these points become the second auxiliary point information; among the circle of points, select the remaining two auxiliary point information points, satisfying that each point is equidistant from the first and second auxiliary point information points, and that the distances between them are relatively large; that is, among the circle of points equidistant from a fixed asymmetric point, select four uniform points that divide the circle into four equal parts. Figure 3 Examples of results with added auxiliary points: Points 6 to 9 are the added auxiliary points;
[0079] The initial layer sampling points and the added auxiliary point information are used together as the point set to be matched. The matching is performed using the above equation (1), and the front and back points of the two graphics are output (i.e., Figure 3 The corresponding results for points 6 and 7 in the data;
[0080] For graphic categories that are prone to matching errors, consider improving the algorithm in the following two ways:
[0081] Continue adding auxiliary points: Based on the above sampling points, continue adding auxiliary points in the forward and backward directions, such as... Figure 4 Points 10 and 11 shown are the newly added points;
[0082] Regional thermonuclear information: The thermonuclear signal value of each point involved in the calculation is improved from the single-point thermonuclear signal value to the regional thermonuclear signal value, that is, the original single-point thermonuclear signal value:
[0083] Change to:
[0084] 4) Input the three-dimensional coordinates X = (x1, x2, ..., xn) of the point set consisting of the initial layer sampling points and the aforementioned auxiliary points. m )'∈R m×3 Using this as the initial value, the geodesic distance matrix D of the point set is used. geo And Euclidean distance matrix D Euclid A new optimization problem model is established and solved to obtain the coordinates of a new set of points; the optimization problem is as follows:
[0085] minloss, X∈R m×3 Where, loss = ||D geo -D Euclid ||2
[0086] The changes in the position coordinates of the point set before and after optimization are as follows: Figure 5 and Figure 6 ;
[0087] 5) Using the position coordinates of the point sets of the two graphics after the above optimization, calculate the symbol label for each pair of symmetrical points; that is, perform the following calculations on the two graphics to be matched, where point p5 is the selected asymmetrical point, and points p6 and p7 are the front and back points of the graphics respectively, and have Remaining points Figure 7 The figures shown are symmetrical points on both sides of the graphic.
[0088] Calculate the symbol labels for the points symmetrical to both sides of the figure:
[0089] For point p i (i = 1, 3), we have Then point p i The label is positive;
[0090] For point p j (j=2,4), we have Then point p j The label is negative;
[0091] The same calculation is performed on graph two. If the labels are the same, it is judged as a correct corresponding point. The final matching result is output, which completes the resolution of the symmetry confusion problem in the initial layer matching process.
[0092] 6) Input the initial layer matching results, and perform dense matching on the two graphics to be matched using a layered sampling algorithm. Figure 8 This is a diagram showing the effect of adding more points for matching based on the initial layer matching results.
[0093] The above embodiments of the present invention provide a 3D symmetric graphic matching method based on initial layer sampling points, mainly used to solve the symmetry confusion problem that easily occurs during the matching of initial layer sampling points of 3D symmetric graphics. Compared with graphic embedding-based methods that also solve this problem, the present invention greatly improves the accuracy and computation time of initial layer matching. Based on the more accurate initial layer matching results, the accuracy of subsequent dense matching using hierarchical matching algorithms is also greatly improved.
[0094] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A 3D symmetric graphic matching method based on initial layer sampling points, characterized in that, Includes the following steps: Step 1: Initial Layer Sampling a. 3D point cloud data, which is the initial layer sampling of the 3D point cloud data of the two input graphics. The sampling points of the initial layer are selected from 5 to 11 key points according to the different categories of graphics. b. Calculate the heat core sampling point, that is, calculate the heat core feature descriptor of each point in the point cloud, and select the point with the largest heat core value in the second ring neighborhood as the heat core sampling point; c. Add the farthest sampling point. Using the above-mentioned hot core sampling points as the initial point set, add 10 to 30 points using the farthest sampling method to obtain a new feature point set; d. Fusion sampling: The new feature point set is processed using a fusion sampling algorithm, and finally 5 to 11 points are selected as the initial layer sampling points; Step 2: Initial Layer Matching e. Distinguish between symmetrical and asymmetrical points, establish a mathematical optimization model using the GW distance, and distinguish between symmetrical and asymmetrical points by solving the mathematical model. The matching scheme for asymmetrical points adopts this solution result. f. To distinguish between the front and back, the same algorithm is used to add four auxiliary points to both images. Using the GW distance optimization model mentioned above, the front and back points of the auxiliary points are matched, and auxiliary points are added and regional heat core information is used. g. Adjusting graphic distortion: Utilizing the property that geodesic distances remain approximately equidistant after graphic deformation, an optimization problem is established, and the point positions are changed to adjust the graphic distortion; h. Distinguish between left and right: For the two distorted figures after adjustment, use the vector outer product to distinguish between left and right and obtain the matching result of the symmetrical points.
2. The 3D symmetric graphic matching method based on initial layer sampling points according to claim 1, characterized in that, The specific steps for distinguishing between symmetrical and asymmetrical points are as follows: 1) Perform initial layer sampling on the point cloud data of the two input 3D graphics to obtain their respective sampling point sets, with the number of sampling points being m and n, respectively; 2) Apply the mathematical optimization model based on GW distance to the above sampling point set to obtain the matching result, X=(x i ) mn×1 x i ∈[0,1], the optimization model is established as shown in equation (1): min f(x),X∈R mn×1 (1) in: Solving the above optimization problem model using gradient descent yields the matching probability X, which is then transformed into a joint probability matrix U, where U(i,j)=x (i-1)n+j ; Set a threshold T, T = 0.05, and make the following judgments to distinguish between symmetrical and asymmetrical points: if |U(i,k)-U(i,l)| < T, then the k-th point and the l-th point in Figure 2 are symmetrical points; if |U(i,k)-U(j,k)| < T, then the i-th point and the j-th point in Figure 1 are symmetrical points; and when U(i,j) ≥ 0.8, the i-th point in Figure 1 and the j-th point in Figure 2 are a pair of correctly matched asymmetrical points.
3. The 3D symmetric graphic matching method based on initial layer sampling points according to claim 2, characterized in that, The specific steps for distinguishing between before and after are as follows: 1) Select an asymmetric point from the initial layer sampling points that has the largest distance from the remaining point set; 2) Initialize the distance d0, and find a point around the circle whose distance to the selected asymmetric point is approximately equal to d0; 3) Select a point on the axis of symmetry of the graphic from the above points. The point on the axis of symmetry is the first auxiliary point information added. 4) Select points on the axis of symmetry of the figure, but the points with larger Euclidean distances from the first point are the second auxiliary points to be added; 5) Select the remaining two auxiliary points from the points around the circle, such that each point is equidistant from the first and second auxiliary points and is relatively far apart from the others; that is, among the points around the circle that are equidistant from a certain fixed asymmetric point, select four uniform points that divide the circle into four equal parts. 6) The initial layer sampling points and the added auxiliary point information are used together as the set of points to be matched. The matching is performed using the above formula (1), and the corresponding results of the front and back points of the two graphics are output. 7) Continue to add auxiliary points: Based on the above sampling points, continue to add auxiliary points in the forward and backward directions; 8) Regional thermonuclear information: The thermonuclear signal value of each point involved in the calculation is improved from the single-point thermonuclear signal value to the regional thermonuclear signal value, that is, the original single-point thermonuclear signal value: Change to:
4. The 3D symmetric graphic matching method based on initial layer sampling points according to claim 3, characterized in that, The specific steps for adjusting graphic distortion are as follows: The input consists of the three-dimensional coordinates X = (x1, x2, ..., xn) of the point set composed of the initial layer sampling points and the aforementioned auxiliary points. m )'∈R m×3 Using this as the initial value, the geodesic distance matrix D of the point set is used. geo And Euclidean distance matrix D Euclid A new optimization problem model is established and solved to obtain the coordinates of a new set of points; the optimization problem is as follows: minloss, X∈R m×3 in, loss=D geo -D Euclid2 。 5. The 3D symmetric graphic matching method based on initial layer sampling points according to claim 4, characterized in that, The specific steps for distinguishing left from right are as follows: Using the position coordinates of the point sets of the two graphics after the above optimization, calculate the symbol label for each pair of symmetrical points. Calculate the symbol labels for the points symmetrical to both sides of the figure: For point p i (i = 1, 3), we have Then point p i The label is positive; For point p j (j=2,4), we have Then point p j The label is negative; Perform the same calculations on graph two. If the labels are the same, the points are judged to be the correct corresponding points, and the final matching result is output. Given the initial layer matching result, the two graphics to be matched are then subjected to dense matching using a layered sampling algorithm.