A method for accurate tracking of three-dimensional objects based on region and point cloud fusion

Abstract

The present invention provides a method for accurate tracking of three-dimensional objects based on the fusion of regions and point clouds, belonging to the field of computer vision technology, and comprising: (1) establishing a sparse viewpoint model; (2) establishing a color model on the object projection contour; (3) using a search line to sample candidate contour points on the projection contour image; (4) proposing a new step function to improve the discrimination between the foreground and background areas near the contour; (5) fusing the color and distance information of the object contour to determine the matching object contour points; (6) weighting the energy function with a weight function, solving the pose according to the energy equation of all matching contour points; (7) after obtaining the transformation matrix of inter-frame motion, using it to initialize the model point cloud, and then solving the optimal pose by performing ICP precise registration on the model point cloud and the object point cloud, thereby achieving accurate tracking of the three-dimensional object. The method of the present invention can solve the problems of tracking offset and tracking failure of low-texture three-dimensional objects under complex backgrounds and occlusion.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a three-dimensional object precise tracking method based on region and point cloud fusion. Background Art

[0002] 3D object tracking can continuously obtain the spatial position relationship between a 3D object and a camera, which is an important task in computer vision. 3D object tracking can be roughly divided into three categories according to the type of image features used: feature point-based methods, edge-based methods, and region-based methods.

[0003] The tracking method based on feature points first needs to extract the three-dimensional feature points with significant distinction on the object, then establish the correspondence between the three-dimensional points of the object and the matching two-dimensional points of the image through feature matching, and finally solve the object's posture. Early tracking methods based on feature points use markers to achieve object posture tracking, but it requires placing markers in the scene and the markers cannot be blocked during the tracking process, so this method has poor usability. With the development of feature point detection and description algorithms, it is widely used in the field of image feature tracking. Due to its reliance on feature points on the surface of the object, the tracking method based on feature points is not suitable for tracking weak texture or textureless objects.

[0004] Edge-based methods use edge features to find the correspondence between the three-dimensional model contour points and the two-dimensional image contour points of the object, and iteratively calculate the optimal pose parameters of the object by minimizing the reprojection error of the three-dimensional contour points. This type of method involves fewer sampling points and therefore has a significant speed advantage; however, this type of method relies on the matching effect of the image contour points. When there are a large number of complex edges in the background or the object is occluded, the existing methods are prone to produce a large number of mismatched object contour points, resulting in tracking offset and failure.

[0005] Region-based methods solve the optimal pose parameters by optimizing the color difference between the foreground and background. In this process, the step of dividing the foreground and background regions implicitly searches for the object contour, which gives it a greater advantage in complex backgrounds. This type of method builds a color probability model based on the foreground and background colors. However, in some complex situations, such as scenes with similar foreground and background colors or drastic changes in lighting, the image color may change drastically, resulting in the color model not being updated in time, which leads to tracking failure. Although some current methods use the pixel-by-pixel color posterior probability of the area near the projected contour to achieve more accurate foreground and background color probability modeling, their tracking accuracy depends on the color statistics of the area near the object contour. When the background color is similar to the object's appearance color or the object is occluded, this method is difficult to accurately estimate the object's pose; and because the algorithm needs to repeatedly calculate the color segmentation information of all pixels near the object contour during the color probability modeling and pose estimation process, it leads to a large amount of calculation and is difficult to achieve strong real-time tracking.

[0006] Therefore, a method for accurately tracking three-dimensional objects that can overcome the above problems is needed. Summary of the invention

[0007] The purpose of the present invention is to provide a method for accurate tracking of three-dimensional objects based on region and point cloud fusion. The method adopts a new step function to improve the distinction between foreground and background areas near the contour. By fusing the color and distance information of the object contour, the influence of similar color background and local occlusion on tracking is effectively reduced. Finally, the point cloud registration technology is improved to achieve high-precision estimation of the object's position and posture. The method is characterized in that it includes the following steps:

[0008] S1: Virtual cameras are placed at multiple vertices of the geometric mesh around the model and the 3D geometry is rendered from multiple viewpoints to create a sparse viewpoint model.

[0009] S2: According to the pose of the previous frame, search for the closest calculated view, then project the 3D model into the image, build a search line at the projection contour point, and establish a probability model for the projection contour line to segment the foreground and background;

[0010] S3: Search for projection contour points, foreground points, and background points on the line, and weight each point based on the color and distance of the pixel point to calculate the weight of the object contour point;

[0011] S4: Estimate the optimal pose of the current frame by minimizing the energy function according to the pixel weight;

[0012] S5: The optimal pose obtained in S4 is used to initialize the model point cloud, and then the model point cloud and the object point cloud are precisely aligned by ICP to obtain the optimal pose of the object;

[0013] S6: Evaluate the accuracy of the optimal pose obtained in S5 and the optimal pose obtained in S4. When the accuracy of the optimal pose obtained in S5 is greater than the accuracy of the optimal pose obtained in S4, repeat S2-S5 using the optimal pose in S5 to achieve tracking of the three-dimensional object.

[0014] Furthermore, in S1, creating a sparse viewpoint model specifically includes the following steps:

[0015] S11: Each time you render, randomly sample n from the model outline c points, of which

[0016] S12: For each contour point, estimate its normal vector This normal vector is perpendicular to the contour;

[0017] S13: For each contour point, calculate its depth value This depth value represents the depth information obtained from rendering;

[0018] S14: Use the reconstruction formula to convert the 2D contour points and depth information into the 3D coordinates of the model. The reconstruction formula is expressed by Formula 1 and Formula 2:

[0019] Formula 1 is expressed as:

[0020]

[0021] Formula 2 is expressed as:

[0022]

[0023] 3D Model Point Usage and its homogeneous form Describe it using image coordinates Get the color value y=I from the corresponding color and depth images c (x) and the depth value d z =I d (x); The 3D model is projected onto the image through Formula 3, which is expressed as:

[0024]

[0025] Among them, f x and f y is the focal length, p x and p y are the principal point coordinates.

[0026] In formula 1, π -1 For the inverse operation, according to the image coordinate x and the corresponding depth value d along the optical axis z Reconstruct the 3D model points and calculate the direction vector from the virtual camera to the center of the model The calculated 3D model points, normal vectors and direction vectors are stored separately.

[0027] Furthermore, S2 specifically includes the following steps:

[0028] S21: Get the 3D model points and normal vectors from the nearest view of the sparse viewpoint model according to the previous frame pose, and project them into the image;

[0029] S22: Project contour point m at each model i At , the search line is along the normal direction of the projected contour n i Sampling is performed, where the search line l i The j-th sampling pixel point is x ij Indicates that iRepresents the contour points of an object;

[0030] S23: Assuming pixel-by-pixel probability independence, the probability of the shape of the projection contour is expressed as:

[0031]

[0032] Among them, d(x ij )=n i T (x ij -m i ) is the search line sampling point x ij With the projected contour point m i The normal Euclidean distance between f (x ij ) is the posterior probability of the foreground area for each pixel color; P b (x ij ) is the posterior probability of the background area for each pixel color, calculated using the temporally consistent local color histogram; Among them, η f and η b are the total number of pixels in the foreground area and the background area respectively; the prior probability P(y|M f ) and P(y|M b ) are used to describe the color y is the foreground model M f and background model M b Possibility of f (x) and H b (x) are all step functions, which are used to make the model focus on the pixels near the projection contour;

[0033] In order to prevent the misclassification of pixels due to incomplete segmentation and then assigning them to the wrong color histogram, the step function is defined by Formula 5 and Formula 6. Formula 5 is expressed as:

[0034]

[0035] Formula 6 is expressed as:

[0036]

[0037] In Formula 5 and Formula 6, parameter s is the slope, and the smoothness of the transition is adjusted by controlling the slope of the hyperbolic tangent function; parameter α h ∈[0,0.5] is the amplitude parameter used to adjust the amplitude of the step function.

[0038] Furthermore, S3 specifically includes the following steps:

[0039] S31: For each sampling point x on the search lineij , assign a weight to it, expressed by the formula:

[0040]

[0041] Among them, ω c (x ij ) is the weight calculated based on the color information of the target contour point; ω d (x ij ) is the weight calculated based on the position information of the target contour point;

[0042] S32: Using object contour points s i The color information of the sampling point x ij The weighting function based on the color information of the object contour points is defined by the formula:

[0043]

[0044] Among them, P(s i |S) is s i The probability of belonging to the object contour; c 1 is the minimum color probability threshold.

[0045] S33: Using target contour point s i The distance information of the sample point x ij The weighting function based on the distance information of the object contour points is defined as:

[0046]

[0047] in, is the sampling point x ij To the target contour point s i The normalized distance of Search line l i The length of c 2 is the maximum effective distance.

[0048] Furthermore, in S4, the energy function is expressed as:

[0049]

[0050] Furthermore, S5 specifically includes the following steps:

[0051] S51: Use the pose obtained in S4 to initialize the model point cloud;

[0052] S52: Searching for corresponding points with the closest Euclidean distance to the object point cloud on the model point cloud based on the similarity of curvature features, thereby reducing the matching of erroneous corresponding points;

[0053] S53: Based on the matched corresponding points, a dual-weight function adaptive optimization method of curvature feature similarity is proposed to solve the optimal transformation;

[0054] S54: Using the solved transformation matrix to transform the pose of the model point cloud;

[0055] S55: Algorithm convergence judgment.

[0056] Further, in S52, the curvature feature similarity between two corresponding points is calculated by Formula 11, which is expressed as:

[0057]

[0058] Further, in S53, in a dual weight function adaptive optimization method for curvature feature similarity, the optimization objective function is expressed as:

[0059]

[0060] in, is a dual-weight function. The curvature feature similarity is solved by optimizing the objective function. When the similarity is less than the threshold, When the similarity is greater than the threshold, And assign less weights to local deformation points and outliers.

[0061] Furthermore, S55 is specifically for determining whether the algorithm meets the convergence condition, where the convergence condition is: reaching the maximum number of iterations or the objective function converges to the minimum value; when the algorithm does not meet the convergence condition, S51-S55 are repeated until the algorithm meets the convergence condition.

[0062] Furthermore, in S6, the evaluation is specifically as follows: when the accuracy of the optimal posture solved in S5 is less than the accuracy of the optimal posture solved in S4, the model is initialized to the optimal posture solved in S4; otherwise, the model is initialized to the optimal posture solved in S5.

[0063] Compared with the prior art, the beneficial effects of the present invention are mainly reflected in:

[0064] 1. This invention provides a 3D object tracking method based on region and point cloud fusion, a new region-based cost function, and proposes a smooth step function to reduce the misclassification of sampling points. It not only reduces the computational cost, but also handles complex scenes such as local occlusion and similar backgrounds more effectively than existing region-based methods.

[0065] 2. The present invention solves the negative impact of local occlusion and similar color background on tracking accuracy and robustness by introducing a new weighted energy function based on the color and distance information of the object contour, thereby improving the robustness of tracking.

[0066] 3. The present invention uses point cloud registration to accurately register the target position, which can improve the target tracking accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 Schematic diagram of the structure for calculating 2D rendering from 3D mesh model.

[0068] Figure 2 It is a schematic diagram of the search line sampling in the projected contour.

[0069] Figure 3 It is a schematic diagram of the pose estimation results using point cloud registration.

[0070] Figure 4 It is a schematic diagram of tracking results in different scenarios. DETAILED DESCRIPTION

[0071] The following will be described in more detail with reference to a schematic diagram of a method for accurate tracking of three-dimensional objects based on the fusion of regions and point clouds of the present invention, wherein a preferred embodiment of the present invention is shown. It should be understood that those skilled in the art may modify the present invention described herein while still achieving the beneficial effects of the present invention. Therefore, the following description should be understood as being widely known to those skilled in the art and not as a limitation to the present invention.

[0072] A three-dimensional object tracking method based on region and point cloud fusion specifically includes the following steps:

[0073] Step 1: Arrange virtual cameras at the vertices of a geometric mesh surrounding the model and render the 3D geometry from multiple viewpoints to create a sparse viewpoint model.

[0074] The specific steps include:

[0075] 1) Each time you render, randomly sample n from the model outline c Points

[0076] 2) For each contour point, estimate its normal vector The normal vector is perpendicular to the contour, such as Figure 1 As shown;

[0077] 3) For each sampling point, calculate the depth value This value represents the depth information obtained from rendering;

[0078] 4) Use the reconstruction formulas (1) and (2) to convert the 2D contour points and depth information into the 3D coordinates of the model:

[0079]

[0080]

[0081] 3D Model Point Usage and its homogeneous form To describe, using image coordinates Get the color value y=I from the corresponding color and depth images c (x) and the depth value d z =I d (x). The 3D model is projected onto the image using the following formula:

[0082]

[0083] Among them, f x and f y is the focal length, p x and p y are the principal point coordinates.

[0084] In formula (1), π -1 Represents the inverse operation, based on the image coordinate x and the corresponding depth value d along the optical axis z Reconstruct the 3D model points and calculate the direction vector from the camera to the center of the model The 3D model points, normal vectors and direction vectors calculated above are stored separately

[0085] Step 2: Based on the pose of the previous frame, search for the closest calculated view. Then project the 3D model into the image, construct a search line at the projection contour point, and establish a probability model for the projection contour line to segment the foreground and background.

[0086] The specific steps include:

[0087] 1) Get the 3D model points and normal vectors from the nearest view of the sparse viewpoint model according to the previous frame pose and project them into the image;

[0088] 2) Project contour point m at each model i At , the search line is along the normal direction of the projected contour n i Sampling is performed. Search line l i The j-th sampling pixel point is x ij Indicates that i Represents the contour points of an object, such as Figure 2 As shown;

[0089] 3) Assuming pixel-by-pixel probability independence, the probability of the shape of the projected contour is:

[0090]

[0091] Among them, d(x ij )=n i T (x ij -m i ) is the search line sampling point x ij With the projected contour point m i The normal Euclidean distance between them. f (x ij ) and P b (x ij ) represent the posterior probabilities of the foreground and background regions for each pixel color, respectively, which are calculated from the temporally consistent local color histogram.

[0092] η f , η b is the total number of pixels in the foreground and background areas. f ) and P(y|M b ) describes the color y as the foreground model M f Or background model M b Possibility. f (x), H b (x) are all step functions, which make the model pay more attention to the pixels near the projection contour. In order to prevent the misclassification of pixels due to incomplete segmentation and then assigning them to the wrong color histogram, the step function is redefined as:

[0093]

[0094]

[0095] Among them, parameter s is the slope, which adjusts the smoothness of the transition by controlling the slope of the hyperbolic tangent function; parameter α h ∈[0,0.5] is the amplitude parameter used to adjust the amplitude of the step function.

[0096] Step 3: The search line includes projection contour points, foreground points, and background points. The pixel points are weighted based on their color and distance to calculate their weight as object contour points.

[0097] The specific steps include:

[0098] 1) For each sampling point x on the search line ij , assigning it a weight:

[0099]

[0100] Among them, ω c (x ij ) is the weight calculated based on the color information of the target contour point, ω d (x ij ) is the weight calculated based on the position information of the target contour point.

[0101] 2) Using the object contour point s i The color information of the sampling point x ij For weighting, the weight function based on the color information of the object contour point is defined as:

[0102]

[0103] Among them, P(s i |S) is s i The probability of belonging to the object contour, c 1 is the minimum color probability threshold.

[0104] 3) Using the target contour point s i The distance information of the sample point x ij The weighting function based on the distance information of the object contour points is defined as:

[0105]

[0106] in, is the sampling point x ij To the target contour point s i The normalized distance of Search line l i The length of c 2 is the maximum effective distance.

[0107] Step 4: Estimate the optimal pose of the current frame by minimizing the energy function based on the pixel weights;

[0108] The energy function is:

[0109]

[0110] Where p represents the optimal pose transformation.

[0111] Step 5: Use the pose solved in step 4 to initialize the model point cloud, and then perform ICP precise registration between the model point cloud and the object point cloud to solve the optimal pose, such as Figure 3 As shown;

[0112] The specific steps include:

[0113] 1) Use the pose solved in step (4) to initialize the model point cloud;

[0114] 2) Find any point P′ in the object point cloud P′ i In the model point cloud Q, the set of points with smaller Euclidean distance Q j ;

[0115] 3) Calculate the normal vector and curvature of the object point cloud and the model point cloud. For any point P′ in the object point cloud P′ i , based on the curvature characteristics of the point, establish multiple feature vectors (P i1 ,P i2 ,P i3 ,P i4 ), these vectors correspond to the K, H, K 1 and K 2 At the same time, any point Q in the model point cloud Q j The corresponding multiple eigenvectors (Q j1 ,Q j2 ,Q j3 ,Q j4 ). Among them, K 1 , K 2 is the principal curvature of the point cloud, K is the Gaussian curvature, and H is the mean curvature;

[0116] 4) Use the following formula to calculate the curvature feature similarity between two corresponding points:

[0117]

[0118] The calculated similarity value is equal to the set threshold ω 0 Compare; if the similarity is less than the threshold, the corresponding point does not meet the requirements and needs to be searched again until the point with a similarity value greater than or equal to the threshold is calculated as the corresponding point, thereby reducing the matching of incorrect corresponding points;

[0119] 5) An adaptive optimization method for curvature feature similarity is established, and the designed optimization objective function is as follows:

[0120]

[0121] in, is a dual weight function. According to the similarity of curvature features solved by D, if the similarity is less than the threshold, then If the similarity is greater than the threshold, then Assign less weight to local deformation points and outliers;

[0122] 6) Solve the optimization objective function through singular value decomposition (SVD) to obtain the transformation matrix;

[0123] 7) Apply the solved transformation matrix to the model point cloud to transform the pose of the model point cloud;

[0124] Determine whether the algorithm meets the convergence condition. The convergence condition is to reach the maximum number of iterations or the objective function converges to the minimum value. If the convergence condition is not met, repeat steps 1) to 7) until the convergence condition is met.

[0125] Step 6: Evaluate the accuracy of the pose obtained in step 5 and the pose in step 4. If the pose obtained in step 5 is not as accurate as the pose in step 4, initialize it to the pose in step 4. Conversely, if the pose obtained in step 5 is more accurate, repeat steps 2 to 5 using this pose to achieve three-dimensional object tracking.

[0126] The specific steps include:

[0127] 1) Evaluate the accuracy of the pose obtained in step 5 and the pose in step 4;

[0128] 2) If the pose obtained in step 5 is not as accurate as that in step 4, it will be initialized to the pose in step 4;

[0129] 3) If the posture obtained in step 5 is more accurate, then steps 2 to 5 are repeated using the posture to achieve tracking of the three-dimensional object.

[0130] The three-dimensional object tracking method provided by the present invention is accurately tested, and the data set used is the RBOT data set. This data set uses the real scene captured by the camera as the background, and superimposes the rendered virtual object on the real background to obtain a reference pose. It is the first three-dimensional tracking data set in which the camera and the object move simultaneously. The RBOT data set contains 3D models of 18 objects, 4 motion modes (normal mode, dynamic lighting mode, noise and dynamic lighting mode, occlusion mode), a total of 72 video sequences, and each video sequence contains 1001 frames of images. The three-dimensional object tracking method provided by the present invention is compared with the region-based tracking technology such as Stoiber on the RBOT data set, and the experimental results are shown in Table 1.

[0131]

[0132] Table 1

[0133] It can be seen from Table 1 that the algorithm provided by the present invention is superior to the tracking method provided by the comparative method in all aspects, with an average improvement of 5% under normal, dynamic lighting and occlusion, and an average improvement of 10% in accuracy under noise.

[0134] In order to demonstrate good 3D object tracking results, weak texture objects are placed in different scenarios for experiments, including similar background colors, partial occlusion, fast movement, and reflection. The experimental results are shown in Figure 2. Figure 4 As shown in the figure, it can be seen that the method in this paper can achieve good three-dimensional tracking effect in different scenarios.

[0135] The above is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any technician in the relevant technical field, without departing from the scope of the technical solution of the present invention, makes any form of equivalent replacement or modification to the technical solution and technical content disclosed in the present invention, which does not depart from the content of the technical solution of the present invention and still falls within the protection scope of the present invention.

Claims

1. A three-dimensional object accurate tracking method based on region and point cloud fusion, characterized in that: The steps include: S1: Virtual cameras are placed at multiple vertices of the geometric mesh around the model and the 3D geometry is rendered from multiple viewpoints to create a sparse viewpoint model. S2: According to the pose of the previous frame, search for the closest calculated view, then project the 3D model into the image, build a search line at the projection contour point, and establish a probability model for the projection contour line to segment the foreground and background; S3: Search for projection contour points, foreground points, and background points on the line, and weight each point based on the color and distance of the pixel point to calculate the weight of the object contour point; S4: Estimate the optimal pose of the current frame by minimizing the energy function according to the pixel weight; S5: The optimal pose obtained in S4 is used to initialize the model point cloud, and then the model point cloud and the object point cloud are precisely aligned by ICP to obtain the optimal pose of the object; S6: Evaluate the accuracy of the optimal pose obtained in S5 and the optimal pose obtained in S4. When the accuracy of the optimal pose obtained in S5 is greater than the accuracy of the optimal pose obtained in S4, repeat S2-S5 using the optimal pose in S5 to achieve tracking of the three-dimensional object. In S1, creating a sparse viewpoint model specifically includes the following steps: S11: Each time you render, randomly sample from the model outline points, of which ; S12: For each contour point, estimate its normal vector , the normal vector is perpendicular to the contour; S13: For each contour point, calculate its depth value , the depth value represents the depth information obtained from rendering; S14: Use a reconstruction formula to convert the 2D contour points and depth information into 3D coordinates of the model, and the reconstruction formula is represented by Formula 1 and Formula 2: The formula 1 is expressed as: The formula 2 is expressed as: ， 3D Model Point Usage and its homogeneous form Describe it using image coordinates Get color values ​​from the corresponding color and depth images and depth value ; The 3D model is projected onto the image through Formula 3, which is expressed as: ， in, and is the focal length, and are the principal point coordinates; In the formula 1, For the inverse operation, according to the image coordinates and the corresponding depth values ​​along the optical axis Reconstruct the 3D model points and calculate the direction vector from the virtual camera to the center of the model ; and store the calculated 3D model points, normal vectors and direction vectors respectively; The S2 specifically includes the following steps: S21: Get the 3D model points and normal vectors from the nearest view of the sparse viewpoint model according to the previous frame pose, and project them into the image; S22: Project contour points on each model At , the search line is along the normal direction of the projected contour Sampling is performed, where the search line No. The sampled pixels are express, Represents the contour points of an object; S23: Assuming pixel-by-pixel probability independence, the probability of the shape of the projection contour is expressed as: ， in, Sample points for the search line With projected contour points The normal Euclidean distance between them; is the posterior probability of the foreground area for each pixel color; is the posterior probability of the background area for each pixel color, computed from the temporally consistent local color histogram; ,in, are the total number of pixels in the foreground area and the background area respectively; the prior probability and Used to describe colors It is the foreground model and background model possibility; They are all step functions, which are used to make the model focus on the pixels near the projected contour; In order to prevent the pixels from being misclassified due to incomplete segmentation and then assigned to the wrong color histogram, the step function is defined by Formula 5 and Formula 6, which is expressed as: ， The formula six is ​​expressed as: ， In Formula 5 and Formula 6, the parameters is the slope, which adjusts the smoothness of the transition by controlling the slope of the hyperbolic tangent function; the parameter [0,0.5] Amplitude parameter used to adjust the amplitude of the step function.

2. The method for accurate tracking of three-dimensional objects based on region and point cloud fusion according to claim 1, characterized in that: The S3 specifically includes the following steps: S31: For each sampling point on the search line , assign a weight to it, expressed by the formula: ， in, is the weight calculated based on the color information of the target contour point; is the weight calculated based on the position information of the target contour point; S32: Using object contour points The color information of the sampling point The weighting function based on the color information of the object contour points is defined by the formula: ， in, for The probability of belonging to the object contour; is the minimum color probability threshold; S33: Using target contour points The distance information of the sample point For weighting, the weight function based on the distance information of the object contour points is defined as: ， in, For sampling point To target contour point The normalized distance of For search line Length; is the maximum effective distance.

3. The method for accurate tracking of three-dimensional objects based on region and point cloud fusion according to claim 2, characterized in that: In S4, the energy function is expressed as: 。 4. The method for accurate tracking of three-dimensional objects based on region and point cloud fusion according to claim 1, characterized in that: The S5 specifically includes the following steps: S51: Use the pose obtained in S4 to initialize the model point cloud; S52: Searching for corresponding points with the closest Euclidean distance to the object point cloud on the model point cloud based on the similarity of curvature features, thereby reducing the matching of erroneous corresponding points; S53: Based on the matched corresponding points, a dual-weight function adaptive optimization method of curvature feature similarity is proposed to solve the optimal transformation; S54: Using the solved transformation matrix to transform the model point cloud; S55: Algorithm convergence judgment.

5. The method for accurate tracking of three-dimensional objects based on region and point cloud fusion according to claim 4, characterized in that: In S52, the curvature feature similarity between two corresponding points is calculated by Formula 11, and the Formula 11 is expressed as: 。 6. The method for accurate tracking of three-dimensional objects based on region and point cloud fusion according to claim 4, characterized in that: In the S53, in a dual-weight function adaptive optimization method for curvature feature similarity, the optimization objective function is expressed as: ， in, is a dual weight function. The curvature feature similarity is solved by optimizing the objective function. When the similarity is less than the threshold, =0; when the similarity is greater than the threshold, , and assign less weight to local deformation points and outliers.

7. The method for accurate tracking of three-dimensional objects based on region and point cloud fusion according to claim 4, characterized in that: The S55 is specifically to determine whether the algorithm meets the convergence condition, and the convergence condition is: reaching the maximum number of iterations or the objective function converges to the minimum value; when the algorithm does not meet the convergence condition, S51-S55 are repeated until the algorithm meets the convergence condition.

8. The method for accurate tracking of three-dimensional objects based on region and point cloud fusion according to claim 1, characterized in that: In S6, the evaluation is specifically as follows: when the accuracy of the optimal posture solved in S5 is less than the accuracy of the optimal posture solved in S4, the model is initialized to the optimal posture solved in S4; otherwise, the model is initialized to the optimal posture solved in S5.

Images