Three-dimensional single target tracking method based on coarse-to-fine feature matching and point densification

By adopting coarse to thin feature matching and point density methods in three-dimensional single-objective tracking technology, combining cosine similarity and attention mechanism, the accuracy problems caused by tracking failure and point cloud sparseness in complex scenarios are solved. Achieve more robust and accurate tracking effects.

CN119445150AInactive Publication Date: 2025-02-14NANJING UNIV OF AERONAUTICS & ASTRONAUTICS +1
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202411545183.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-01
Publication Date
2025-02-14
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing three-dimensional single-objective tracking technology is prone to failure in complex and dynamic scenarios, and the sparsity of point clouds leads to inaccurate tracking results.

Method used

A method based on feature matching and point density from coarse to thin is adopted, feature matching is performed through the combination of cosine similarity and attention mechanism, and a relatively dense sampling module is introduced to improve the density of target points by segmenting foreground points and background points.

Benefits of technology

It enhances the robustness of feature matching, mitigates tracking failures caused by appearance mutations, and alleviates point cloud sparseness by increasing target point density, improving the accuracy of tracking results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119445150A_ABST
    Figure CN119445150A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of three-dimensional visual single-target tracking, in particular to a three-dimensional single-target tracking method based on coarse-to-fine feature matching and point densification. According to the method provided by the invention, the technical problems of insufficient robustness of feature matching and inaccurate tracking when appearance feature mutation and sparse point cloud are processed by a traditional method are solved; through the method, a new three-dimensional single-target tracking network is constructed, and the new three-dimensional single-target tracking network is called as CDTracker; according to the method, the cosine similarity and the attention mechanism are applied to feature matching from coarse to fine, so that the robustness of feature matching is enhanced, and the tracking failure caused by appearance mutation is reduced. Besides, in order to solve the problem of inaccurate tracking results caused by sparse point clouds, a relatively dense sampling module based on point cloud segmentation is introduced, and target points are densified by segmenting foreground points and background points. Therefore, the method is suitable for three-dimensional single-target tracking tasks of sparse point clouds with huge appearance changes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent visual perception, and in particular relates to a three-dimensional single target tracking method based on coarse-to-fine feature matching and point densification. Background Art

[0002] In intelligent visual perception technology, 3D single target tracking technology has been widely used in fields such as autonomous driving, indoor robots and drones, while the complexity of feature matching and the sparsity of point clouds have always been major issues affecting tracking accuracy. Traditional 3D single target tracking methods based on matching twin paradigms usually involve using twin networks to extract point cloud appearance features from templates and search areas, matching the features of the two areas through cosine similarity and attention mechanisms, and then performing target recognition and positioning. This is suitable for targets with small appearance changes, but is prone to failure in complex and dynamic scenes.

[0003] While the motion-centric paradigm of 3D single-target tracking methods enhances the adaptability of the network to complex dynamic scenes, it requires extensive datasets with multiple motion patterns for training to achieve robust tracking performance and generalization. The incomplete shapes caused by the sparsity of point clouds can cause the network to shift its focus to nearby distractions, which seriously affects the tracking results. The sparsity problem is not addressed in both paradigms. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention proposes a three-dimensional single target tracking method based on coarse-to-fine feature matching and point densification, and constructs a new three-dimensional single target tracking network based on this method, called CDTracker. The present invention combines similarity embedding and attention allocation to achieve coarse-to-fine template and search area feature matching, and combines relatively dense sampling based on the concept of point cloud segmentation to retain more target points and improve positioning accuracy.

[0005] In order to achieve the above technical objectives, the present invention provides the following technical solutions:

[0006] A three-dimensional single target tracking method based on coarse-to-fine feature matching and point densification, comprising:

[0007] S1. Get template point cloud T C And the search area point cloud S C ;

[0008] S2. Construct feature extraction module to obtain template point cloud T C And the search area point cloud S C As input, feature extraction is performed based on the twin paradigm of appearance matching to generate discriminant point features; the template feature set T and the search area feature set S are output;

[0009] S3, build a coarse-to-fine feature matching module; use a hybrid similarity learning mechanism combining cosine embedding and attention allocation to perform coarse-to-fine feature matching on the extracted high-dimensional features to obtain the fused search area feature set F S' ;

[0010] S4, build a relatively dense sampling module, and perform fusion search on the feature set F S' Sampling is performed to obtain the sampled fusion feature F; and the idea of ​​segmentation is used to retain more target points and increase the relative density of target points to alleviate sparsity;

[0011] S5. Build a target positioning module; based on the sampled fusion feature F obtained in step S4, use target positioning to regress the target center and yaw angle to achieve accurate prediction of the tracking result.

[0012] Further, step S2 includes:

[0013] For the template point cloud T in the form of input 3D coordinates C And the search area point cloud S C , feature extraction is performed separately using a shared-weighted appearance matching-based Siamese paradigm backbone network;

[0014] Different weights are assigned to the discriminative features for each backbone network according to the position of the point. The Siamese Point Transformer method is used to capture the long-distance contextual information used for discriminative learning, generate discriminative point features, build a comprehensive point cloud feature extraction module, and output the template feature set T and the search area feature set S; the template feature set T and the search area feature set S contain the C-dimensional feature information extracted from the original three-dimensional coordinates by the backbone network.

[0015] Further, step S3 includes:

[0016] S31, input the search area feature set S into two independent multi-layer perceptrons MLP to obtain two feature subsets, which are respectively denoted as S1 and S2;

[0017] S32, constructing a similarity feature extraction module and a difference feature extraction module based on feature subsets S1 and S2 respectively, performing coarse feature matching, and obtaining a similarity feature set S'1 and a difference feature set S'2 respectively; and concatenating these two feature sets, performing feature fusion, and obtaining a coarse feature map S';

[0018] S33, construct a cross attention module, according to the template feature set T and the coarse feature map S' output by the point cloud feature extraction module, use cross attention to further learn the similarity between them and perform feature fusion to obtain the feature map F S ;

[0019] S34, use the self-attention mechanism to further update the search area point features to obtain the final fusion search area feature set F S' .

[0020] More specifically, step S32 includes:

[0021] S321, construct a similar feature extraction module; calculate the cosine similarity between the template feature set T and the search area feature subset S1, and map it to obtain a similarity graph to obtain the similarity between the template and the search area; the formula is expressed as:

[0022]

[0023] Among them, Cosine(·,·) represents the cosine similarity between two features, N t and Represents the number of point features in T and S1 respectively, Sim represents a Similarity graph, where each entity Sim ij represents the i-th feature t in T i and the jth feature t in S1 j The cosine similarity between ;

[0024] S322, through maximum pooling, obtain the cosine similarity of each point in S1 and its corresponding most similar template point in T, that is, the maximum cosine similarity; collect the features of each point in S1 and the corresponding maximum cosine similarity and the features and original three-dimensional coordinates of the most similar template points of these points in T, perform feature splicing and fusion, and input them into the multi-layer perceptron MLP, embed the template features into the search area features, and obtain the similarity feature set S'1; the formula is expressed as:

[0025]

[0026] Where k is the subscript of the feature of the point in T that is most similar to the template point corresponding to the point in S1, Sim k is the maximum similarity value under this subscript, T k That is, the feature set of these points that are most similar to the template, X k Represents the three-dimensional coordinate set of the most similar template point; C is the dimension of the features contained in the similarity feature set S'1.

[0027] S323, construct a difference feature extraction module; use MLP to calculate the weights between the template feature set T and the corresponding features in the search area feature subset S2, multiply the weights by the corresponding features in T, complete the weighting of the template features, and reduce the feature dimension through the maximum pooling operation to obtain the difference feature set S'2; the formula is expressed as:

[0028]

[0029] Among them, MaxPool(·) is the maximum pooling operation, Represents the weights between corresponding features in T and S2; Represents the number of point features in S2; C is the dimension of the features contained in the difference feature set S'2.

[0030] S324, connecting the similar feature set S'1 and the difference feature set S'2, performing feature fusion, and obtaining a coarse feature map S'.

[0031] More specifically, in step S33, the formula of the cross attention is expressed as:

[0032] F S =CrossAttention(S',T,T+X t );

[0033] in, Represents the position embedding of the template, S', T, T+X t They serve as query, key, and value in the cross-attention mechanism respectively.

[0034] More specifically, step S34 is as follows:

[0035] For each point in the search area, the K-NN algorithm is used to find the K nearest neighbor points containing C-dimensional feature information in the feature space constructed on the cross-attention output. Use the self-attention mechanism to further update the search area point features to obtain the final fused search area feature set Its formula is:

[0036]

[0037] in, Denotes the feature set F S The i-th feature in and Respectively The corresponding feature point and the K-NN neighborhood of the feature point are embedded; They serve as the query, key, and value of the self-attention mechanism respectively.

[0038] Further, step S4 includes:

[0039] Construct a relatively dense sampling module to fuse the search area feature set F S'Input to the multi-layer perceptron MLP to predict the score of each point, and use the ground truth bounding box to generate a smooth one-hot encoded one-hot label Y' for supervision; retain the M points with the highest scores and group them as the center of the ball query, and then compare them with the k adjacent features F n ∈R M×k×C After aggregation, the fusion feature F∈R is obtained M×(3+C) , where 3 is the dimension of the retained point coordinates.

[0040] Further, step S5 includes:

[0041] Build a target positioning module, voxelize the fused feature F and perform the downsampling maximum pooling MaxPool operation, and input it into the prediction head in sequence to obtain the predicted two-dimensional bounding box, deflection angle and center position coordinate information of the target, and accurately predict the tracking result.

[0042] Based on the above technical solution, the present invention has the following beneficial effects:

[0043] The present invention combines cosine similarity and attention mechanism and applies them to coarse-to-fine feature matching, which enhances the robustness of feature matching and reduces tracking failures caused by sudden appearance changes. At the same time, a relatively dense sampling module based on point cloud segmentation is introduced to achieve the densification of target points by segmenting foreground points and background points, thereby solving the problem of inaccurate tracking results caused by sparse point clouds. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic flow chart of the three-dimensional single target tracking method proposed by the present invention;

[0045] Figure 2 It is the overall architecture diagram of the CDTracker network constructed by the method proposed in the present invention;

[0046] Figure 3 This is the specific architecture diagram of the coarse feature matching module of the CDTracker network;

[0047] Figure 4 This is the detailed architecture diagram of the CDTracker network’s fine feature matching module;

[0048] Figure 5 This is the specific architecture diagram of the target positioning prediction module of the CDTracker network. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0050] Although the steps in the present invention are arranged with numbers, they are not used to limit the order of the steps. Unless the order of the steps is clearly stated or the execution of a certain step requires other steps as a basis, the relative order of the steps can be adjusted. It is understood that the term "and / or" used in this article involves and covers any and all possible combinations of one or more of the associated listed items.

[0051] like Figure 1 As shown, the present invention proposes a three-dimensional single target tracking method based on coarse-to-fine feature matching and point densification, and uses this method to construct a Figure 2 The network for three-dimensional single target tracking shown is CDTracker; the method proposed by the present invention specifically includes the following steps:

[0052] S1. Get template point cloud T C And the search area point cloud S C ;

[0053] S2. Construct feature extraction module to obtain template point cloud T C And the search area point cloud S C As input, feature extraction is performed based on the twin paradigm of appearance matching to generate discriminant point features; the template feature set T and the search area feature set S are output;

[0054] As a preferred implementation of step S2, in this embodiment, Figure 2 As shown, S2 is specifically:

[0055] For the template point cloud T in the form of input 3D coordinates C And the search area point cloud S C , feature extraction is performed separately using a shared-weighted appearance matching-based Siamese paradigm backbone network;

[0056] Different weights are assigned to the discriminative features for each backbone network according to the position of the point. The Siamese Point Transformer method is used to capture the long-distance contextual information used for discriminative learning, generate discriminative point features, build a comprehensive point cloud feature extraction module, and output the template feature set T and the search area feature set S; the template feature set T and the search area feature set S contain the C-dimensional feature information extracted from the original three-dimensional coordinates by the backbone network.

[0057] S3, build a coarse-to-fine feature matching module; use a hybrid similarity learning mechanism combining cosine embedding and attention allocation to perform coarse-to-fine feature matching on the extracted high-dimensional features to obtain the fused search area feature set F S' ;

[0058] As a preferred implementation of step S3, in this embodiment, Figure 2 As shown, S3 is specifically:

[0059] S31, input the search area feature set S into two independent multi-layer perceptrons MLP to obtain two feature subsets, which are respectively denoted as S1 and S2;

[0060] S32, constructing a similarity feature extraction module and a difference feature extraction module based on feature subsets S1 and S2 respectively, performing coarse feature matching, and obtaining a similarity feature set S'1 and a difference feature set S'2 respectively; and concatenating these two feature sets, performing feature fusion, and obtaining a coarse feature map S';

[0061] More specifically, the process of coarse feature matching in this embodiment includes:

[0062] S321, construct a similar feature extraction module; calculate the cosine similarity between the template feature set T and the search area feature subset S1, and map it to obtain a similarity graph to obtain the similarity between the template and the search area; the formula is expressed as:

[0063]

[0064] Among them, Cosine(·,·) represents the cosine similarity between two features, N t and Represents the number of point features in T and S1 respectively, Sim represents a Similarity graph, where each entity Sim ij represents the i-th feature t in T i and the jth feature t in S1 j The cosine similarity between ;

[0065] S322, through maximum pooling, obtain the cosine similarity of each point in S1 and its corresponding most similar template point in T, that is, the maximum cosine similarity; collect the features of each point in S1 and the corresponding maximum cosine similarity and the features and original three-dimensional coordinates of the most similar template points of these points in T, perform feature splicing and fusion, and input them into the multi-layer perceptron MLP, embed the template features into the search area features, and obtain the similarity feature set S'1; the formula is expressed as:

[0066]

[0067] Where k is the subscript of the feature of the point in T that is most similar to the template point corresponding to the point in S1, Sim k is the maximum similarity value under this subscript, T k That is, the feature set of these points that are most similar to the template, X k Represents the three-dimensional coordinate set of the most similar template point; C is the dimension of the features contained in the similarity feature set S'1.

[0068] S323, construct a difference feature extraction module; use MLP to calculate the weights between the template feature set T and the corresponding features in the search area feature subset S2, multiply the weights by the corresponding features in T, complete the weighting of the template features, and reduce the feature dimension through the maximum pooling operation to obtain the difference feature set S'2; the formula is expressed as:

[0069]

[0070] Among them, MaxPool(·) is the maximum pooling operation, Represents the weights between corresponding features in T and S2; Represents the number of point features in S2; C is the dimension of the features contained in the difference feature set S'2.

[0071] S324, connecting the similar feature set S'1 and the difference feature set S'2, performing feature fusion, and obtaining a coarse feature map S'.

[0072] In this embodiment, Figure 2 As shown, steps S33-S34 are a fine feature matching process; the fine matching process combines the attention mechanism and the K nearest neighbor search method;

[0073] In step S33, the formula of the cross attention is expressed as:

[0074] F S =CrossAttention(S',T,T+X t );

[0075] in, Represents the position embedding of the template, S', T, T+X t They serve as query, key, and value in the cross-attention mechanism respectively.

[0076] The specific process of step S34 is as follows:

[0077] For each point in the search area, the K-NN (K nearest neighbor) algorithm is used to find its closest K adjacent points containing C-dimensional feature information in the feature space constructed on the cross-attention output. Use the self-attention mechanism to further update the search area point features to obtain the final fused search area feature set The formula is:

[0078]

[0079] in, Denotes the feature set F S The i-th feature in and Respectively The corresponding feature point and the K-NN neighborhood of the feature point are embedded; They serve as the query, key, and value of the self-attention mechanism respectively.

[0080] S4, build a relatively dense sampling module, and perform fusion search on the feature set F S' Sampling is performed to obtain the sampled fusion feature F; and the idea of ​​segmentation is used to retain more target points and increase the relative density of target points to alleviate sparsity;

[0081] As a preferred implementation of step S4, in this embodiment, step S4 is specifically as follows:

[0082] Construct a relatively dense sampling module to fuse the search area feature set F S' Input to the multi-layer perceptron MLP to predict the score of each point, and use the ground truth bounding box to generate a smooth one-hot encoded one-hot label To supervise; the formula of label Y' is as follows:

[0083]

[0084] Among them, when point i is in the ground truth bounding box, y i =1; otherwise y i =0; ε is a hyperparameter, which is set to 0.2 in this embodiment. Considering the imbalance in the number of foreground and background points, a focal loss function is used for supervision;

[0085] The formula is as follows

[0086]

[0087] in, is the prediction score, and α is a hyperparameter, which is set to 2 in the experiment.

[0088] The M points with the highest scores are retained and grouped as the center of the ball query, and then combined with k adjacent features F n ∈R M×k×C After aggregation, the fusion feature F∈R is obtained M×(3+C) , where 3 is the dimension of the retained point coordinates.

[0089] S5. Build a target positioning module; based on the sampled fusion feature F obtained in step S4, use target positioning to regress the target center and yaw angle to achieve accurate prediction of the tracking result.

[0090] As a preferred implementation of step S5, step S5 is specifically:

[0091] Build a target positioning module, voxelize the fused feature F and perform the downsampling maximum pooling MaxPool operation, and input it into the prediction head in sequence to obtain the predicted two-dimensional bounding box, deflection angle and center position coordinate information of the target, and accurately predict the tracking result.

[0092] In this embodiment, Figure 5 As shown in the figure, the fusion feature F obtained by relatively dense sampling is voxelized into a three-dimensional tensor, and then the maximum pooling MaxPool is aggregated into a two-dimensional tensor form, and then the predicted tracking results are obtained through three prediction heads in the form of two-dimensional convolution. Specifically, the two-dimensional center head outputs the predicted two-dimensional bounding box information of the target, including the height H and width W; the offset and rotation head outputs the predicted deflection angle information of the target, including yaw, pitch and roll; the z-axis center head outputs the predicted center position coordinate information of the target. In this way, accurate prediction of the tracked target can be achieved.

[0093] So far, the construction of the three-dimensional single target tracking network CDTracker has been realized. Based on the twin paradigm of appearance matching, a hybrid similarity learning mechanism combining cosine embedding and attention allocation is adopted for coarse-to-fine feature matching. Then, the foreground and background points are segmented to achieve the densification of target points to alleviate sparsity and achieve accurate prediction of tracking results.

[0094] In summary, the method proposed in the present invention can generate discriminant point features through Siamese Point Transformer, combine cosine similarity and attention mechanism to achieve coarse-to-fine feature matching, use relatively dense sampling to improve the relative density of target points, and alleviate the sparsity of point clouds; the CDTracker network constructed based on this method is suitable for three-dimensional single target tracking tasks with sparse point clouds with huge appearance changes.

[0095] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0096] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A three-dimensional single target tracking method based on coarse-to-fine feature matching and point densification, characterized in that: The following steps are involved: S1. Get template point cloud T C And the search area point cloud S C ; S2, construct feature extraction module; To obtain the template point cloud T C And the search area point cloud S C As input, feature extraction is performed based on the twin paradigm of appearance matching to generate discriminant point features; the template feature set T and the search area feature set S are output; S3, build a coarse-to-fine feature matching module; A hybrid similarity learning mechanism combining cosine embedding and attention allocation is used to perform coarse-to-fine feature matching on the extracted high-dimensional features to obtain the fused search area feature set F S' ; S4, build a relatively dense sampling module to fusion search area feature set F S' Sampling is performed to obtain the sampled fusion feature F; and the idea of ​​segmentation is used to retain more target points and increase the relative density of target points to alleviate sparsity; S5. Build a target positioning module; based on the sampled fusion feature F obtained in step S4, use target positioning to regress the target center and yaw angle to achieve accurate prediction of the tracking result.

2. A three-dimensional single target tracking method based on coarse-to-fine feature matching and point densification according to claim 1, characterized in that: Step S2 is specifically as follows: For the template point cloud T in the form of input 3D coordinates C And the search area point cloud S C , feature extraction is performed separately using a shared-weighted appearance matching-based Siamese paradigm backbone network; Different weights are assigned to the discriminative features for each backbone network according to the position of the point. The Siamese Point Transformer method is used to capture the long-distance contextual information used for discriminative learning, generate discriminative point features, build a comprehensive point cloud feature extraction module, and output the template feature set T and the search area feature set S; the template feature set T and the search area feature set S contain the C-dimensional feature information extracted from the original three-dimensional coordinates by the backbone network.

3. The three-dimensional single target tracking method based on coarse-to-fine feature matching and point densification according to claim 1, characterized in that: Step S3 specifically includes: S31, input the search area feature set S into two independent multi-layer perceptrons MLP to obtain two feature subsets, which are denoted as S1 and S2 respectively; S32, constructing a similarity feature extraction module and a difference feature extraction module based on feature subsets S1 and S2 respectively, performing coarse feature matching, and obtaining a similarity feature set S'1 and a difference feature set S'2 respectively; and concatenating these two feature sets, performing feature fusion, and obtaining a coarse feature map S'; S33, construct a cross attention module, according to the template feature set T and the coarse feature map S' output by the point cloud feature extraction module, use cross attention to further learn the similarity between them and perform feature fusion to obtain the feature map F S ; S34, use the self-attention mechanism to further update the search area point features to obtain the final fusion search area feature set F S' .

4. A three-dimensional single target tracking method based on coarse-to-fine feature matching and point densification according to claim 3, characterized in that: Step S32 specifically includes: S321, construct a similar feature extraction module; calculate the cosine similarity between the template feature set T and the search area feature subset S1, and map it to obtain a similarity graph to obtain the similarity between the template and the search area; the formula is expressed as: Among them, Cosine(·,·) represents the cosine similarity between two features, N t and Represents the number of point features in T and S1 respectively, Sim represents a Similarity graph, where each entity Sim ij represents the i-th feature t in T i and the jth feature t in S1 j The cosine similarity between ; S322, through maximum pooling, obtain the cosine similarity of each point in S1 and its corresponding most similar template point in T, that is, the maximum cosine similarity; collect the features of each point in S1 and the corresponding maximum cosine similarity and the features and original three-dimensional coordinates of the most similar template points of these points in T, perform feature splicing and fusion, and input them into the multi-layer perceptron MLP, embed the template features into the search area features, and obtain the similarity feature set S'1; the formula is expressed as: Where k is the subscript of the feature of the point in T that is most similar to the template point corresponding to the point in S1, Sim k is the maximum similarity value under this subscript, T k That is, the feature set of these points that are most similar to the template, X k Represents the three-dimensional coordinate set of the most similar template point; C is the dimension of the similarity feature set S'1 containing features; S323, construct a difference feature extraction module; use MLP to calculate the weights between the template feature set T and the corresponding features in the search area feature subset S2, multiply the weights by the corresponding features in T, complete the weighting of the template features, and reduce the feature dimension through the maximum pooling operation to obtain the difference feature set S'2; the formula is expressed as: Among them, MaxPool(·) is the maximum pooling operation, Represents the weights between corresponding features in T and S2; represents the number of point features in S2; C is the dimension of the features contained in the difference feature set S'2; S324, connecting the similar feature set S'1 and the difference feature set S'2, performing feature fusion, and obtaining a coarse feature map S'.

5. The three-dimensional single target tracking method based on coarse-to-fine feature matching and point densification according to claim 3, characterized in that: In step S33, the formula of the cross attention is expressed as: F S =CrossAttention(S',T,T+X t ); in, Represents the position embedding of the template, S', T, T+X t They serve as query, key, and value in the cross-attention mechanism respectively.

6. The three-dimensional single target tracking method based on coarse-to-fine feature matching and point densification according to claim 3, characterized in that: Step S34 is specifically as follows: For each point in the search area, the K-NN algorithm is used to find the K nearest neighbor points containing C-dimensional feature information in the feature space constructed on the cross-attention output. Use the self-attention mechanism to further update the search area point features to obtain the final fused search area feature set Its formula is: in, Denotes the feature set F S The i-th feature in and Respectively The corresponding feature point and the K-NN neighborhood of the feature point are embedded; They serve as the query, key, and value of the self-attention mechanism respectively.

7. The three-dimensional single target tracking method based on coarse-to-fine feature matching and point densification according to claim 1, characterized in that: Step S4 is specifically as follows: Construct a relatively dense sampling module to fuse the search area feature set F S' Input into the multi-layer perceptron MLP to predict the score of each point, and use the ground truth bounding box to generate a smooth one-hot encoded one-hot label Y' for supervision; retain the M points with the highest scores and group them as the center of the ball query, and then compare them with the k adjacent features F n ∈R M×k×C After aggregation, the fusion feature F∈R is obtained M×(3+C) , where 3 is the dimension of the retained point coordinates.

8. The three-dimensional single target tracking method based on coarse-to-fine feature matching and point densification according to claim 1, characterized in that: Step S5 is specifically as follows: Build a target positioning module, voxelize the fused feature F and perform the downsampling maximum pooling MaxPool operation, and input it into the prediction head in sequence to obtain the predicted two-dimensional bounding box, deflection angle and center position coordinate information of the target, and accurately predict the tracking result.

Citation Information

Patent Citations

  • Three-dimensional point cloud single target tracking method based on graph convolution

    CN114862911A