Joint attack countermeasure method for 3D target tracking

Abstract

The invention provides a joint attack countermeasure method for 3D target tracking. According to the method, point filtering and point disturbance are designed to simulate mixed data defects captured by a laser radar. According to the method, on the basis of voxel segmentation, in order to solve the sparsity characteristic of point cloud data, a voxel-based point filtering module is designed, and the module generates binomial distribution according to density information of points and is used for guiding selection points in a tracking template to be discarded; thirdly, a voxel-based geometric perception point disturbance module is provided, the disturbance direction is defined by using the geometric attribute of the module, disturbance is applied to the filtered tracking template, and it is ensured that the disturbance result is not easily perceived visually; finally, the method provides an objective function which combines confidence loss, shift loss and distance loss to guide the generation and disturbance of binomial distribution. The whole method combines point filtering and point disturbance to simulate the defect of point cloud data, so that potential loopholes and defects of an existing algorithm are revealed.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence security, and specifically to a joint adversarial attack method for 3D target tracking. By designing two modules, point filtering and point perturbation, point cloud data defects are simulated, thereby effectively implementing adversarial attacks on 3D target tracking models. Background Art

[0002] In today's technological landscape, 3D object tracking technology has become a key focus in academic research and numerous practical applications. From autonomous vehicles accurately sensing surrounding road conditions, to precise positioning and interaction in virtual reality scenarios, to component identification and assembly in automated industrial production, numerous scenarios rely on the powerful support of 3D object tracking technology. Its core purpose is to leverage various sensors to continuously acquire key information such as the target object's position and posture in three-dimensional space, providing accurate data support for subsequent decision-making and operations.

[0003] With the rapid development of artificial intelligence technology, deep neural networks (DNNs) have been widely introduced into the field of 3D object tracking. DNNs demonstrate remarkable learning capabilities, capable of extracting deep feature patterns from massive geometric training samples to accurately locate and track targets. Compared with traditional tracking algorithms based on hand-crafted features, deep learning methods can automatically learn more discriminative feature representations, effectively addressing object changes and occlusions in complex scenarios, and significantly improving tracking accuracy and stability, enabling 3D object tracking technology to more reliably meet diverse needs in practical applications.

[0004] While bringing convenience, the widespread application of DDNs in 3D object tracking also exposes its inherent vulnerability. Studies have shown that DNNs' sensitivity to input data makes them extremely vulnerable to adversarial attacks. Attackers can cause serious deviations in the tracking system through carefully designed small perturbations. However, existing adversarial attack methods for 3D object tracking have obvious limitations: on the one hand, most studies focus on a single attack mode (such as only applying geometric perturbations to the point cloud or discarding random points), and fail to fully simulate multi-source composite data defects in real scenes (such as the coupling effect of sensor noise, occlusion loss and transmission errors); on the other hand, traditional adversarial attack strategies often ignore the spatial sparsity and geometric correlation characteristics of point cloud data, resulting in insufficient attack concealment. More importantly, existing methods lack the joint optimization of attack effectiveness and concealment, making it difficult to construct adversarial samples that can both deceive the algorithm and conform to actual physical constraints.

[0005] In view of the above shortcomings, the present invention proposes a joint adversarial attack method based on point filtering and point perturbation. The core idea of this method is to simulate the data defects faced by point cloud data in 3D target tracking through point filtering and point perturbation. First, the method divides the 3D space into uniform voxels, reducing the computational cost and enhancing the attack effect; secondly, the method uses the density information of the points to process the sparsity characteristics of the point cloud, and selects to discard the points in the voxels for point filtering; thirdly, the method combines the local surface information of the voxels to impose constraints on the voxel perturbation direction, and injects the results of the point perturbation into the model; finally, an objective function composed of confidence loss, shift loss and distance loss is designed for back propagation. The entire method simulates the data defect situation in real scenes and develops an effective adversarial attack method to evaluate existing tracking algorithms. It not only helps to reveal the potential loopholes and deficiencies of existing algorithms, but also provides important theoretical basis and technical support for the design of more robust tracking systems. It has extremely important practical significance for ensuring the safe and stable operation of many fields that rely on 3D target tracking technology. Summary of the Invention

[0006] To address the above-mentioned defects in the prior art, the present invention provides a joint adversarial attack method for 3D target tracking. This method designs point filtering and point perturbation to simulate the defects of mixed data captured by lidar. First, the method divides the 3D space into uniform voxels, reducing computational costs and enhancing the attack effect. Second, based on voxel segmentation, the method designs a voxel-based point filtering module to address the sparsity characteristics of point cloud data. This module generates a binomial distribution based on the density information of the points to guide the selection of points to be discarded in the tracking template. Third, the method proposes a voxel-based geometric-aware point perturbation module, which uses its geometric properties to define the perturbation direction to apply perturbations to the filtered tracking template to ensure that the perturbation results are not visually noticeable. Finally, the method proposes an objective function that combines confidence loss, shift loss, and distance loss to guide the generation and perturbation of the binomial distribution. The entire method combines point filtering and point perturbation to simulate the defects of point cloud data, thereby revealing the potential loopholes and shortcomings of existing algorithms.

[0007] The specific contents of the inventive method are as follows:

[0008] Step 1: Voxel segmentation: Take the point cloud data of the target object as input, divide the 3D space where the point cloud data is located into a regular m voxel grid, and obtain the tracking template P after voxel segmentation tmp ;

[0009] Step 2: Weight initialization: Randomly initialize the filter weight θ and perturbation weight l;

[0010] Step 3: Density-aware point filtering: Filter by discarding points in the voxels according to the generated m×1 binomial distribution; Specifically, the voxel density d and the filtering weight θ are calculated through the activation function to obtain the filtering probability p; the threshold constant is set to make the filtering probability p satisfy the binomial distribution; After obtaining the filtering probability p, P' tmp ←P tmp ×p obtains the filtered tracking template;

[0011] Step 4: Geometrically aware point perturbation: Generate an m×3 perturbation in the XYZ directions from the filtered tracking template; based on the local surface direction s of the i-th voxel i The perturbation value Δ of the i-th voxel is calculated with the perturbation weight l i , then by P" tmp ←P' tmp +Δ to get the template after adding perturbation;

[0012] Step 5: Apply the perturbed template as an adversarial sample to the 3D target tracking model and calculate the loss function

[0013] Step 6: Repeat steps 3 to 5 to achieve the attack goal by minimizing the loss function.

[0014] As an implementation method, the filtering probability p is calculated in the following way:

[0015]

[0016] Among them, Sigmoid is the activation function; the density d of the i-th voxel i Calculated by the following formula:

[0017]

[0018] Where R(o) represents the number of neighboring points with point o as the center and r as the radius, v i Represents the points contained in the i-th voxel; the following formula is used to make the filtering probability p satisfy the binomial distribution:

[0019]

[0020] Among them, ∈ is the threshold constant;

[0021] Finally, the filtering probability p is updated using the following formula:

[0022]

[0023] Where η represents the learning rate, is the loss function.

[0024] As an embodiment, the local surface direction s of the i-th voxel i Obtained by the following formula:

[0025]

[0026] Where N(o) represents the normal vector of point o, v i represents the points contained in the i-th voxel, n i Represents the number of points contained in the i-th voxel;

[0027] The perturbation value Δ of the i-th voxel i According to the formula Δ i =s i ×l calculated.

[0028] As an implementation method, the loss function According to the confidence loss Shift loss and distance loss Calculation yields:

[0029]

[0030] Among them, α, β, and γ are trade-off hyperparameters;

[0031] As an embodiment, the confidence loss is calculated as follows:

[0032]

[0033] where R i is the target score obtained by the i-th ranked 3D target tracking model, and q, r, and s represent the index of the target score respectively.

[0034] As an embodiment, the displacement loss is calculated as follows:

[0035]

[0036] Where n is the number of possible targets, C, C i They represent the position prediction center of the target with the highest prediction score of the 3D target tracking model and the XYZ coordinate values of the i-th ranked target respectively.

[0037] As an embodiment, the distance loss is calculated as follows:

[0038]

[0039] Among them, P xis a voxel in the tracking template, P y represents a voxel in the adversarial example.

[0040] As an implementation method, the filtering weight θ and the perturbation weight l are updated according to the calculated loss function.

[0041] Compared with the existing technology, the above technical solution has at least the following beneficial effects:

[0042] (1) A joint adversarial attack method is proposed, which simulates data defects by designing point filtering and point perturbation modules to explore the performance of 3D object tracking under adversarial attacks.

[0043] (2) A density-aware point filtering module is proposed, which uses the density information of the point cloud to encode the binomial distribution of voxels and select points for filtering.

[0044] (3) A geometry-aware point perturbation module is proposed to generate voxel perturbations under the guidance of local geometric information to ensure surface similarity. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a framework diagram of the method of the present invention.

[0046] Figure 2 This is an example diagram of surface direction calculation in the method of the present invention. DETAILED DESCRIPTION

[0047] In order to better understand the purpose, structure and function of the present invention, the following is a further detailed description of a joint counterattack method for 3D target tracking of the present invention in conjunction with the accompanying drawings.

[0048] The present invention is described in detail below with reference to specific examples. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that for those skilled in the art, these embodiments are merely used to explain the technical principles of the present invention and are not intended to limit the scope of evidence collection of the present invention.

[0049] like Figure 1As shown, the present invention provides a joint adversarial attack method for 3D target tracking. First, the 3D space is subdivided into m uniform voxels to voxelize the tracking template. In the density-aware point filtering module, the template filters the points in the voxels according to the m×1 binomial distribution, where the sparsity characteristics of the point cloud are processed by utilizing the density information of the points to enhance the geometric destruction of the template. In the geometry-aware point perturbation module, an m×3 perturbation is generated from the filtered tracking template, which takes into account the constraints imposed by the local surface information of the voxel on the voxel perturbation direction, which ensures that the local surface between the generated adversarial sample and the tracking template remains consistent. In addition, the objective function consists of confidence loss, shift loss and distance loss, which is intended to guide the generation and perturbation of the binomial distribution. The confidence loss tampering with the target score of the attacked tracker, the offset loss further blurs the predicted center position, and the distance loss limits the visual difference caused by the joint attack. The system model of this method contains three modules:

[0050] Density-aware point filtering module: uses the density information of the point cloud to encode the binomial distribution of voxels and select points for filtering.

[0051] Geometry-aware point perturbation module: Generates voxel perturbations guided by local geometric information to ensure surface similarity.

[0052] Objective function module: through The computation modeling of the joint attack is an optimization problem and involves updating the filtering weights and the length of the perturbation.

[0053] Next, combine Figure 1 Detailed description of the workflow of the “Joint Adversarial Attacks for 3D Object Tracking” method:

[0054] Step 1: Voxel segmentation: Voxel segmentation takes the point cloud data of the tracking template (target object such as a car) as input and divides the 3D space where the point cloud data is located into a regular m-voxel grid.

[0055] Step 2: Weight initialization: Randomly initialize the filter weights θ and perturbation weights l.

[0056] Step 3: Density-aware point filtering module uses the tracking template P after voxel segmentation tmp As input, after processing, the filtered template is output to simulate the occlusion situation. The density-aware point filtering module discards the points in the voxels according to the generated m×1 binomial distribution to filter; first calculate the density d of the i-th voxel i :

[0057]

[0058] Where R(o) represents the number of neighboring points with point o as the center and r as the radius, v i Represents the points contained in the i-th voxel; secondly, the voxel density d and the filtering weight θ are calculated through the activation function Sigmoid to obtain the filtering probability p:

[0059]

[0060] The following formula is used to make the filtering probability p satisfy the binomial distribution:

[0061]

[0062] Among them, ∈ is the threshold constant; after obtaining the filtering probability p, P' tmp ←P tmp ×p to obtain the filtered tracking template, where P tmp Represents the tracking template after voxel segmentation.

[0063] Finally, the filtering probability p is updated using the following formula:

[0064]

[0065] Where η represents the learning rate, is the loss function.

[0066] Step 4: Geometric Perturbation Module: This algorithm uses the filtered tracking template P' tmp As input, a perturbation is generated to simulate sensor noise. At the same time, the geometric consistency of the template is maintained to achieve visual inconsistency. The algorithm performs perturbation generation on a voxel basis, generating an m×3 perturbation in the X, Y, and Z directions from the filtered tracking template. The specific steps include:

[0067] Step 4.1: Local surface direction calculation: The algorithm performs perturbation generation in voxel units, where points in the same voxel must be injected with the same perturbation, and the local surface direction s of the voxel is i Obtained by the following formula:

[0068]

[0069] Where N(o) represents the normal vector of point o, v i represents the points contained in the i-th voxel, n i Indicates the number of points contained in the i-th voxel. Figure 2As shown, the cube boxes represent the voxels marked in the tracking template, and the points within the voxels represent the local surface. Each green solid arrow represents the approximate normal vector of a point. These normal vectors are aggregated to calculate the orientation of the local surface, represented by the red dashed arrow.

[0070] Step 4.2: Geometrically Perceptual Perturbation Generation: To achieve the goal of being visually imperceptible, the present invention constrains the perturbation to the local surface direction s i The perturbation value of the i-th voxel is Δ i According to the formula Δ i =s i ×l is calculated. Then, adversarial samples are generated by adding geometric perturbations to the distilled templates, which are given by P” tmp ←P' tmp +Δ to obtain the perturbed template (adversarial sample), where P' tmp Represents the filtered template, P" tmp Represents the template after adding perturbations.

[0071] Step 5: Apply the adversarial example to the 3D object tracking model and calculate the loss function Loss Function According to the confidence loss Shift loss and distance loss Calculation yields:

[0072]

[0073] Among them, α, β, and γ are trade-off hyperparameters.

[0074] Confidence loss The confidence score is blurred by narrowing the gap between high-confidence targets and low-confidence targets, which is calculated as follows:

[0075]

[0076] where R i is the target score obtained by the i-th ranked 3D target tracking model, and q, r, and s represent the index of the target score respectively.

[0077] Shift loss It is used to further reduce the distance between the prediction result center and other target suggestion centers. The calculation method is as follows:

[0078]

[0079] Where n is the number of possible targets, C, C iThey represent the position prediction center of the target with the highest prediction score of the 3D target tracking model and the XYZ coordinate values of the i-th ranked target respectively.

[0080] Distance loss Used to make the attack visually imperceptible, the distance penalty utilizes the bevel distance to limit the visual difference caused by the modification, which is calculated as follows:

[0081]

[0082] Among them, P x is a voxel in the tracking template, P y represents a voxel in the adversarial example.

[0083] Step 6: Repeat steps 3 through 5, i.e., density-aware point filtering and geometry-aware point perturbation, to achieve the attack objective by minimizing the loss function. In each iteration, the density-aware point filtering module generates a binomial distribution based on density information and selects points to discard. Subsequently, the geometry-aware point perturbation module injects perturbations into the filtered tracking template, thereby shifting the points in the filtered tracking template along the surface. Finally, the loss function is calculated to update the filtering weights and perturbation length.

[0084] The effects of the present invention are further described in detail below in conjunction with simulation experiments.

[0085] Specifically, a series of experiments were conducted on the KITTI and NuScenes datasets to verify the robustness of the 3D tracker under the proposed joint attack, where P2B and BAT were used as the attacked 3D tracking models.

[0086] Of the 50 video sequences included in the KITTI dataset, we use 21 for training and the rest for testing. The NusScenes dataset provides 1,000 driving scenes with precise 3D bounding box annotations for 23 object classes. In our experiments, we use 700 scenes for training, 150 for validation, and the remaining for testing.

[0087] Based on the ablation experiments, we set the voxel size to 20 cm and the threshold ∈ to 0.6. All experiments were implemented using Pytorch on a desktop computer with an NVIDIA GTX 1080 GPU. θ and l were initially set to voxel-level random arrays following a standard normal distribution. To update the parameters θ and l, we used the ADAM optimizer to optimize the loss function The initial learning rate is set to 0.01 and decays at a rate of 0.5 after every 20 iterations. The parameters α, β, and γ of the balance loss function are set to 1, 6, and 0.05, respectively. In addition, the number of iterations n itrSet to 100.

[0088] We experimented with different types of adversarial attacks, and the evaluation results are shown in Tables 1-3, where Un-AdvAtt is the name of our proposed attack method. The performance of the attack is measured by the success rate and the decrease in precision of the 3D tracker under adversarial attacks.

[0089] Table 1 Performance of different trackers under different attacks on different datasets

[0090]

[0091]

[0092] To illustrate the effectiveness of the proposed method, the proposed method Un-AdvAtt is compared with the latest attack methods NRTA, TAA and AD-Net. Among them, NRTA generates adversarial samples by rotating the local area of the target template, and TAA and AD-Net generate perturbations and discard points through the designed networks, respectively. As shown in Table 1, in the KITTI dataset, for the attacked BAT model, NRTA reduces the success rate and accuracy to 42.3% and 51.1%, respectively, while TAA reduces the success rate and accuracy to 36.0% and 47.8%, respectively. In contrast, the proposed method Un-AdvAtt outperforms the state-of-the-art on both P2B and BAT. This proves that the mixed data defects simulated by the joint attack are more threatening than those of a single attack method, because Un-AdvAtt destroys the geometric structure of the tracking template in a more effective way.

[0093] Table 2 Attack performance under different loss functions

[0094]

[0095] To verify the performance of confidence loss and offset loss, as shown in Table 2, the success rate and accuracy of the P2B model drop to 21.4% and 33.7% without offset loss, and to 29.1% and 38.5% without confidence loss, respectively. This demonstrates that both confidence loss and offset loss are effective in deceiving 3D trackers. Furthermore, the data in the third row demonstrates that combining these loss functions can further enhance the attack effectiveness. This is because confidence loss and offset loss can guide the victim tracker to generate incorrect object scores and center locations, respectively. Combining these loss functions can effectively improve the attack capability.

[0096] Table 3 Attack performance under different attack methods

[0097]

[0098]

[0099] To verify the effectiveness of the point filtering module and the point perturbation module, we conducted comparative experiments, the results of which are shown in Table 3. The experimental results show that both the proposed point filtering module and the point perturbation module can effectively reduce P2B tracking performance, and the combination of the two modules significantly improves attack performance. This is because, compared with these single attack methods, the joint attack method proposed in this invention tampers with the model input in a more flexible manner, thereby more effectively discovering model vulnerabilities and causing them to fail. In addition, to further evaluate the contribution of density information in point filtering and geometric information in point perturbation, these two modules were compared with random point filtering and regular point perturbation attacks, respectively. Specifically, random point filtering is used to randomly discard points, while regular point perturbation is a method of moving points along the X, Y, and Z directions. It is worth noting that these methods are all designed on a voxel basis. As can be seen from Table 3, the introduction of density information and geometric information enhances the attack performance of the point filtering module and the point perturbation module.

[0100] It will be understood that the present invention is described by way of some embodiments, and it will be appreciated by those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are intended to be protected by the present invention.

Claims

1. A joint adversarial attack method for 3D object tracking, characterized in that: The following steps are involved: Step 1: Voxel segmentation: Take the point cloud data of the target object as input, divide the 3D space where the point cloud data is located into a regular m voxel grid, and obtain the tracking template P after voxel segmentation tmp ; Step 2: Weight initialization: Randomly initialize the filter weight θ and perturbation weight l; Step 3: Density-aware point filtering: Filter by discarding points in the voxels according to the generated m×1 binomial distribution; Specifically, the voxel density d and the filtering weight θ are calculated through the activation function to obtain the filtering probability p; the threshold constant is set to make the filtering probability p satisfy the binomial distribution; After obtaining the filtering probability p, P′ tmp ←P tmp ×p obtains the filtered tracking template; Step 4: Geometrically aware point perturbation: Generate an m×3 perturbation in the XYZ directions from the filtered tracking template; Based on the local surface direction s of the i-th voxel i The perturbation value Δ of the i-th voxel is calculated with the perturbation weight l i , then P″ tmp ←P′ tmp +Δ to get the template after adding perturbation; Step 5: Apply the perturbed template as an adversarial sample to the 3D target tracking model and calculate the loss function Step 6: Repeat steps 3 to 5 to achieve the attack goal by minimizing the loss function.

2. A joint counterattack method for 3D target tracking according to claim 1, characterized in that: The filtering probability p is calculated as follows: Among them, Sigmoid is the activation function; the density d of the i-th voxel i Calculated by the following formula: Where R(o) represents the number of neighboring points with point o as the center and r as the radius, v i represents the points contained in the i-th voxel; The following formula is used to make the filtering probability p satisfy the binomial distribution: Among them, ∈ is the threshold constant; Finally, the filtering probability p is updated using the following formula: Where η represents the learning rate, is the loss function.

3. A joint counterattack method for 3D target tracking according to claim 2, characterized in that: The local surface direction s of the i-th voxel i Obtained by the following formula: Where N(o) represents the normal vector of point o, v i represents the points contained in the i-th voxel, n i Represents the number of points contained in the i-th voxel; The perturbation value Δ of the i-th voxel i According to the formula Δ i =s i ×l calculated.

4. A joint counterattack method for 3D target tracking according to claim 3, characterized in that: The loss function According to the confidence loss Shift loss and distance loss Calculation yields: Among them, α, β, and γ are trade-off hyperparameters.

5. A joint counterattack method for 3D target tracking according to claim 4, characterized in that: The confidence loss is calculated as follows: where R i is the target score obtained by the i-th ranked 3D target tracking model, and q, r, and s represent the index of the target score respectively.

6. A joint counterattack method for 3D target tracking according to claim 5, characterized in that: The displacement loss is calculated as follows: Where n is the number of possible targets, C, C i They represent the position prediction center of the target with the highest prediction score of the 3D target tracking model and the XYZ coordinate values of the i-th ranked target respectively.

7. A joint counterattack method for 3D target tracking according to claim 6, characterized in that: The distance loss is calculated as follows: Among them, P x is a voxel in the tracking template, P y represents a voxel in the adversarial example.

8. A joint counterattack method for 3D target tracking according to claim 7, characterized in that: According to the calculated loss function, the filter weight θ and perturbation weight l are updated.

Images