A Single-Object Tracking Method for 3D Point Clouds Based on Cascaded Huffman Voting
By employing a cascaded Huffman voting strategy and feature enhancement, the problem of inaccurate target candidate box generation in existing 3D point cloud single-target tracking methods is solved, achieving higher tracking accuracy.
Patent Information
- Application Number
- CN202310356618.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-04
- Publication Date
- 2025-11-14
- Estimated Expiration
- 2043-04-04
AI Technical Summary
Existing 3D point cloud single-target tracking methods struggle to generate high-quality target candidate boxes during regression prediction, resulting in insufficient tracking accuracy.
A cascaded Huffman voting strategy is adopted, which gradually returns to the target center through multiple feature fusions and distance elimination mechanisms. Combined with PointNet for feature enhancement, high-quality 3D candidate boxes are finally generated.
It improves the accuracy of single-target tracking in 3D point clouds, generates higher-quality target candidate boxes, and enhances tracking precision.
Smart Images
Figure CN116385713B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision technology, specifically to a 3D point cloud single-target tracking method based on cascaded Huffman voting. Background Technology
[0002] Single-object tracking is an important branch of computer vision and is widely used in various scenarios such as autonomous driving, video surveillance, and military guidance. In recent years, with the development of LiDAR sensors and the robustness of point clouds obtained by radar scanning to complex environments, single-object tracking methods based on 3D point clouds have shown great potential.
[0003] Current mainstream 3D point cloud single-object tracking methods consist of two parts: feature fusion and regression prediction. Feature fusion aims to enhance the point cloud features of the search region using information from the template target, while regression prediction performs offset regression on the feature-enhanced search region to obtain the final prediction result. Most existing regression prediction methods are based on Huffman voting, first regressing seed points in the search region to the potential target center, and then obtaining the final target candidate box through clustering. To further improve the quality of the target candidate box and thus increase tracking accuracy, some methods assign different weights to the features of points at different locations in the search region and perform Huffman voting based on the weighted features to generate better target candidate boxes. Simultaneously, some other methods use attention mechanisms to enhance the features of the voted points, aggregating global information from the offset points. However, all of the above methods rely on a single Huffman vote based on fused features, making it difficult to generate high-quality target candidate boxes, resulting in an inaccurate tracking prediction structure. Summary of the Invention
[0004] To address the shortcomings of existing technologies, this invention proposes a 3D point cloud single-target tracking method based on cascaded Huffman voting. The proposed cascaded Huffman voting strategy can be embedded into current mainstream 3D point cloud single-target tracking methods, effectively improving tracking accuracy.
[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:
[0006] A single-target tracking method for 3D point clouds based on cascaded Huffman voting specifically includes the following steps:
[0007] Step 1: Construct a dataset containing a target template (a given tracking target in the point cloud sequence) and a search region. The target template specifies the target to be tracked during the tracking process. Using PointNet++, composed of an L-layer Set Abstraction (SA) structure, the points in the target template and search region are downsampled, and features of the sampled points are extracted. The target template is then processed through L... th (1≤lth The features obtained from the SA layer ≤L) are The characteristics of the corresponding search region are Where L is 3. This is achieved by fusing the outputs of each SA layer. and Obtain the features of the feature-enhanced search region. The formula is as follows:
[0008]
[0009] Here, xcorr represents the fusion operator.
[0010] Step 2: Based on fusion features of different depths Perform cascaded Hoffman voting. Indicates based on l th Layer fusion features The Huffman voting process, through cascading voting, shifts the points in the search area L times, gradually returning them to the potential target center. Simultaneously, during the voting process, by calculating the distance between the regressed coordinates and the true target center, some interfering points are successively eliminated.
[0011] Specifically as follows:
[0012] 1) First, for the output of the last layer SA Feature fusion is performed to obtain φ L (S), via Hoffman voting V L Get the offset of each point The formula is as follows:
[0013]
[0014] Here, vote represents a multilayer perceptron consisting of three connection layers.
[0015] 2) After V L The state information of the offset point is It is calculated from equation (3). Let To represent the true target center location, a distance elimination mechanism (DEM) is introduced to remove some interfering points. The location is calculated using equation (4). and Euclidean distance between Select the M1 closest points and keep them, and remove points that are too far away.
[0016]
[0017]
[0018] 3) Calculate the index I1 of the M1 retained points, in Retrieve the status information of M1 points corresponding to the index. (1≤i≤M1), and simultaneously obtain the features of M1 points corresponding to the index from the search region features output by the previous layer SA. And on and Feature fusion is performed to obtain φ L-1 (S). Based on this feature, a second Huffman vote is performed to obtain the offset of each point. The formula is as follows:
[0019]
[0020] 4) Based on The points in the retained M1 search regions are cascaded with a second offset, and the state information of the offset points is as follows: It is calculated using equation (6). Using the DEM, select the M2 closest points to keep and remove points that are too far away.
[0021]
[0022]
[0023] 5) Similarly, obtain the index I2 of M2 points, in Retrieve the status information of M2 points corresponding to the index. (1≤i≤M2), and obtain the features of M2 points corresponding to the index from the search region features output by the previous layer SA. right and Feature fusion is performed to obtain φ L-2 (S), based on this feature, a third Huffman vote is performed to obtain the offset of each point. The formula is as follows:
[0024]
[0025] 6) Based on The points in the retained M2 search regions are cascaded for a third offset, and the state information of the offset points is as follows: The formula is as follows:
[0026]
[0027] Step 3: After the cascaded Huffman voting, the state information of M2 search region points is obtained. right Clustering is performed to obtain the final prediction result. The clustering process first considers location information. M3 points are selected as center points by using the farthest sampling point method, and all points within the radius of the center points are searched, thus dividing the M2 points into several point sets. Then, PointNet is used to enhance the features of each point set. j This refers to the state information of the clustered points.
[0028] Step 4: Select confidence score s j The highest point is used as the center of the predicted target location. j s represents the offset of the 3D position and the rotation angle in the XY plane. j This represents the confidence score. q is the prediction head, composed of MLP-Maxpool-MLP, as shown in the following formula:
[0029] {p j ,s j}=q(T j (10)
[0030] This invention has the following characteristics and beneficial effects:
[0031] 1. This invention proposes a cascaded Huffman voting strategy, in which the state information (including position and features) of the points is adjusted in each voting stage, and the points are gradually returned to the target center in a cascaded manner, thereby achieving more accurate 3D point cloud target localization.
[0032] 2. The template and the point cloud of the search area used in this invention are fused at different depths, and cascaded Huffman voting is performed based on the fused features at different depths to finally generate high-quality 3D candidate boxes, thereby effectively improving the tracking accuracy. Attached Figure Description
[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0034] Figure 1 Flowchart of a 3D point cloud single-target tracking method based on cascaded Huffman voting;
[0035] Figure 2 A framework diagram for single-target tracking of 3D point clouds based on cascaded Huffman voting. Detailed Implementation
[0036] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.
[0037] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.
[0038] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.
[0039] This invention provides a 3D point cloud single-target tracking method based on cascaded Huffman voting, such as... Figure 1 and Figure 2 As shown, taking the P2B tracking algorithm as an example, combined with the proposed cascaded Huffman voting, the specific steps include:
[0040] Step 1: Using PointNet++, which consists of an L-layer Set Abstraction (SA) structure, the points of the target template and the search region are downsampled, and the features of the sampled points are extracted. The target template is processed by l th (1≤l th The features obtained from the SA layer ≤L) are The characteristics of the corresponding search region are Where L is 3. This is achieved by fusing the outputs of each SA layer. and Obtain the features of the feature-enhanced search region. The formula is as follows:
[0041]
[0042] Here, xcorr represents the fusion operator.
[0043] Step 2: Based on fusion features of different depths Perform cascaded Hoffman voting. Indicates based on l th Layer fusion features The Huffman voting process, through cascading voting, shifts the points in the search area L times, gradually returning them to the potential target center. Simultaneously, during the voting process, by calculating the distance between the regressed coordinates and the true target center, some interfering points are successively eliminated.
[0044] Specifically as follows:
[0045] 1) First, for the output of the last layer SA Feature fusion is performed to obtain φ L (S), via Hoffman voting V L Get the offset of each point The formula is as follows:
[0046]
[0047] Here, vote represents a multilayer perceptron consisting of three connection layers.
[0048] 2) After V L The state information of the offset point is It is calculated from equation (3). Let To represent the true target center location, a distance elimination mechanism (DEM) is introduced to remove some interfering points. The location is calculated using equation (4). and Euclidean distance between Select the M1 closest points and keep them, and remove points that are too far away.
[0049]
[0050]
[0051] 3) Calculate the index I1 of the M1 retained points, in Retrieve the status information of M1 points corresponding to the index. (1≤i≤M1), and simultaneously obtain the features of M1 points corresponding to the index from the search region features output by the previous layer SA. And on and Feature fusion is performed to obtain φ L-1 (S). Based on this feature, a second Huffman vote is performed to obtain the offset of each point. The formula is as follows:
[0052]
[0053] 4) Based on The points in the retained M1 search regions are cascaded with a second offset, and the state information of the offset points is as follows: It is calculated using equation (6). Using the DEM, select the M2 closest points to keep and remove points that are too far away.
[0054]
[0055]
[0056] 5) Similarly, obtain the index I2 of M2 points, in Retrieve the status information of M2 points corresponding to the index. (1≤i≤M2), and obtain the features of M2 points corresponding to the index from the search region features output by the previous layer SA. right and Feature fusion is performed to obtain φ L-2 (S), based on this feature, a third Huffman vote is performed to obtain the offset of each point. The formula is as follows:
[0057]
[0058] 6) Based on The points in the retained M2 search regions are cascaded for a third offset, and the state information of the offset points is as follows: The formula is as follows:
[0059]
[0060] Step 3: After the cascaded Huffman voting, the state information of M2 search region points is obtained. right Clustering is performed to obtain the final prediction result. The clustering process first considers location information. M3 points are selected as center points by using the farthest sampling point method, and all points within the radius of the center points are searched, thus dividing the M2 points into several point sets. Then, PointNet is used to enhance the features of each point set. j This refers to the state information of the clustered points.
[0061] Step 4: Select the confidence score s j The highest point is used as the center of the predicted target location. j s represents the offset of the 3D position and the rotation angle in the XY plane. jThis represents the confidence score. q is the prediction head, composed of MLP-Maxpool-MLP, as shown in the following formula:
[0062] {p j ,s j}=q(T j (10)
[0063] It should be noted that the results of the tracking test for the Van class on the publicly available dataset KITTI are shown in the table below:
[0064] Table 1: Performance Comparison of P2B Before and After Embedding the Cascaded Huffman Voting Strategy
[0065] Tracker Success Precision P2B 40.8 48.4 P2B+ Cascaded Hoffman Voting (Ours) 46.2 54.2
[0066] As shown in Table 1, the cascaded Huffman voting strategy proposed in this invention can effectively improve the accuracy of the tracker.
[0067] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments, including components, without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.
Claims
1. A 3D point cloud single-target tracking method based on cascaded Huffman voting, characterized in that, Includes the following steps: S1. Construct datasets for the target template and search region. The target template specifies the target to be tracked during the tracking process. Using PointNet++, downsample the points of the target template and search region and extract the features of the sampled points to obtain the target template features. and search region features By fusing the output of PointNet++ and Obtain the features of the feature-enhanced search region. The PointNet++ consists of L layers of Set Abstraction, where L is 3; S2. Features of the search region based on feature enhancement Cascaded Huffman voting shifts the points in the search region, resulting in the state information of M2 search region points. This gradually returns the data to the potential target center; simultaneously, during the voting process, interference points are eliminated one by one by calculating the distance between the regression coordinates and the true target center. Step S2 includes the following sub-steps: S2-1, For the output of the last layer Set Abstraction Perform feature fusion to obtain V by Hoffman L Get the offset of each point The formula is as follows: Here, vote represents a multilayer perceptron consisting of three connection layers; S2-2, after V L The state information of the offset point is The calculation formula is as follows: Calculated make To represent the true center location of the target, a distance elimination mechanism (DEM) is introduced to remove interference points. By the following formula: calculate and Euclidean distance between Select the M1 closest points and keep them, and remove points that are too far away; S2-3, Calculate the index I1 of the M1 retained points, in Retrieve the status information of M1 points corresponding to the index. Simultaneously, the features of M1 points corresponding to the index are obtained from the search region features output by the previous Set Abstraction layer. And on and Feature fusion is performed to obtain the fused feature. Based on fusion Perform a second Huffman vote to obtain the offset of each point. The formula is as follows: S2-4, based on The points in the retained M1 search regions are cascaded for a second offset, and the state information of the offset points is as follows: The following formula is used to calculate... Using a DEM, select the M2 closest points to keep and remove points that are too far away. The expression is as follows: S2-5, similarly, obtain the index I2 of M2 points, in Retrieve the status information of M2 points corresponding to the index. The features of M2 points corresponding to the index are obtained from the search region features output by the previous Set Abstraction layer. right and Feature fusion is performed to obtain the fused feature. Based on fusion A third Huffman vote is conducted to obtain the offset of each point. The formula is as follows: S2-6, based on The points in the retained M2 search regions are cascaded for a third offset, and the state information of the offset points is as follows: The formula is as follows: S3, to Clustering is performed to obtain the state information T of the clustered points. j ; S4. Prediction is performed based on the state information of the clustered points. q is the prediction head, composed of MLP-Maxpool-MLP, and p... j s represents the offset of the 3D position and the rotation angle in the XY plane. j The confidence score is expressed by the following formula: {p j ,s j }=q(T j ) Select confidence score s j The highest point is used as the center of the predicted target location.
2. The 3D point cloud single-target tracking method based on cascaded Huffman voting according to claim 1, characterized in that, In step S1, the target template undergoes l th The features obtained by layer Set Abstraction are Where, 1≤l th ≤L, the characteristic of the corresponding search region is By fusing the outputs of each Set Abstraction layer and Obtain the features of the feature-enhanced search region. The formula is as follows: Here, xcorr represents the fusion operator.
3. The 3D point cloud single-target tracking method based on cascaded Huffman voting according to claim 1, characterized in that, In step S3, the clustering process first determines the clustering based on location information. M3 points are selected as center points by using the farthest sampling point method, and all points within the radius of the center points are searched to divide the M2 points into several point sets. Then, PointNet is used to enhance the features of each point set.