Three-dimensional point cloud target tracking antagonism attack method and device based on frequency domain importance
By adding perturbations in the frequency domain and utilizing frequency band importance saliency maps and graph Fourier transform techniques, the problems of insufficient transferability and neglect of frequency domain information in existing 3D point cloud target tracking adversarial attack methods are solved, achieving stronger attack generalization and transferability.
Patent Information
- Application Number
- CN202411025498.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-29
- Publication Date
- 2024-11-22
AI Technical Summary
Existing adversarial attack methods for 3D point cloud target tracking lack transferability research and neglect the frequency domain information of point clouds, resulting in poor attack effectiveness.
By adding perturbations in the frequency domain and utilizing frequency band importance saliency maps and graphical Fourier transform techniques, sub-band weights are constructed to enhance the perturbation of important sub-bands for tracking. This guides the tracking model to generate proposed directions with lower confidence, thereby reducing tracking accuracy.
It improves the generalization and adversarial nature of attacks, significantly reduces the performance of tracking models, and enhances the transferability of attacks and the ability to identify generalized features.
Smart Images

Figure CN119006518A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a three-dimensional point cloud target tracking adversarial attack method and device based on frequency domain importance, belonging to the adversarial attack technology of three-dimensional point cloud target tracking. Background Art
[0002] 3D point cloud target tracking is to generate a 3D prediction box based on the given first frame point cloud object as a template to determine the position of the target in subsequent frames. The research on generating adversarial examples plays an important role in evaluating the robustness of the tracker and improving the anti-disturbance ability of the tracker.
[0003] In recent years, adversarial attack methods for 3D target tracking models have been proposed one after another. However, the existing research on adversarial attacks on 3D point cloud target tracking has the following limitations: 1. There are few studies on the transferability of attack methods, and there is a lack of distinction between generalized features and non-generalized features. Currently, only the transferable attack network (TAN) focuses on the transferability of adversarial attack methods for 3D point cloud target tracking. The features learned by the model from the dataset can be subdivided into generalized features and non-generalized features. Generalized features are independent of the model, and different models can learn similar generalized features from the same dataset. The attack dataset data has generalized and fragile features, which can produce transferable adversarial perturbations. 2. The frequency domain information of the point cloud is ignored. The frequency domain of the point cloud contains information about generalization and geometric structure. The frequency domain principle shows that the generalization of deep neural networks is related to the frequency domain. The noise of the high-frequency components of the data is prone to overfitting and thus reduces the generalization, while the noise of the low-frequency components is less. In addition, different frequency bands of point clouds reflect the geometric structure of point clouds. The low frequency band reflects the basic shape of the point cloud, and the high frequency band reflects the details of the point cloud. Attacks on the geometric structure of point clouds will affect the characteristics of point clouds. Summary of the invention
[0004] Purpose of the invention: There is currently little research on the transferability of attack methods for three-dimensional point cloud target tracking adversarial attack methods, and the frequency domain information of the point cloud is ignored, and the attack is only performed in the spatial domain. In order to overcome these shortcomings, the present invention provides a three-dimensional point cloud target tracking adversarial attack method and device based on frequency band importance. Through methods in the fields of spectrogram and signal processing, the search area point cloud is transformed from the spatial domain to the frequency domain, and disturbances are added in the frequency domain. The low-frequency components of the search area are attacked to improve generalization. By designing a frequency band importance saliency map, each unit sub-band in the low-frequency area is scored according to its importance to tracking, and sub-band weights are constructed based on the frequency band importance saliency map scores. The degree of disturbance of sub-bands important for tracking is strengthened, and through a bounding box offset loss function, the candidate target proposals generated by the tracking model are guided to bias towards the proposal direction with lower confidence, so that the tracking prediction bounding box deviates from the target and tracking fails.
[0005] Technical solution: To achieve the above purpose, the technical solution adopted by the present invention is:
[0006] A method for 3D point cloud target tracking against adversarial attacks based on frequency domain importance includes the following steps:
[0007] Step 1. Input the search area point cloud P sea and frequency domain perturbation △, construct the search area point cloud P sea The importance saliency map of the low-frequency band is used to calculate the importance weight σ of the low-frequency band required for the attack;
[0008] Step 2. Use graph Fourier transform to transform the search area point cloud P sea From the spatial domain coordinate sequence Transformed into frequency domain coefficient sequence Search area point cloud P sea The i-th point of is represented by the i-th point in the spatial domain coordinate sequence x i ; According to the low frequency threshold l low The frequency domain coefficient sequence Truncated to low frequency coefficient sequence and high frequency coefficient sequence Where: N represents the search area point cloud P sea Total points of
[0009] Step 3. Add the importance weight σ of the low-frequency band to the frequency domain disturbance △ to obtain the weighted low-frequency frequency domain disturbance, and add the low-frequency frequency domain disturbance to the low-frequency coefficient sequence To enhance the attack of important low-frequency sub-bands, we can obtain the anti-low-frequency coefficient sequence Concatenate adversarial low frequency coefficient sequences and high frequency coefficient sequence Get the adversarial frequency domain coefficient sequence Countermeasure frequency domain coefficient sequence Perform inverse Fourier transform to obtain the adversarial space domain coordinate sequence x * , and then get the adversarial search area point cloud
[0010] Step 4. P the template point cloud tmp And adversarial search area point cloud Input into the tracking model T, get M candidate target proposals, select the candidate target proposal with the highest score as the tracking prediction bounding box B * , calculate the confidence loss L based on the M candidate target proposals conf , bounding box offset loss L box and distance loss D c ;
[0011] Step 5. According to the confidence loss L conf , bounding box offset loss L box and distance loss D c Calculate the total loss L, and use the total loss L to iteratively optimize the frequency domain perturbation △ to obtain an adversarial video frame example;
[0012] Step 6: Use the tracking model T to track the generated adversarial video frame examples to obtain the tracking results of the adversarial attack.
[0013] Specifically, in the point cloud frame video sequence In the example, the real bounding box of the tracked target is used to crop the tracking prediction bounding box in the first frame; from the second frame onwards, the tracking prediction bounding box of the previous frame is called the template point cloud P of the current frame. tmp , according to the template point cloud P of the current frame tmp Crop the search area point cloud P of the current frame sea Then the tracking prediction bounding box B of the current frame is obtained.
[0014] Specifically, the search area point cloud P is transformed using graph Fourier transform sea From the spatial domain coordinate sequence Transformed into frequency domain coefficient sequence The following steps are included: First, the K-NN algorithm is used to calculate the search area point cloud P sea The K nearest neighbors of each point in the search area point cloud P sea From the point set to the graph structure G = {U, ε, W}, the vertex set U is the search area point cloud P sea The edge set ε is the set of adjacent edges connecting each point to all its K nearest neighbors. W represents the adjacency matrix. Next, construct the Laplacian matrix L of the graph structure G: DW. The diagonal elements d in the diagonal matrix D are i represents the sum of the weights of all adjacent edges of point i; then, the Laplace matrix L is subjected to eigendecomposition to obtain L = VΛVT, where V is a standard orthogonal matrix containing the eigenvectors of the Laplace matrix L, and Λ is a diagonal matrix composed of the eigenvalues of the Laplace matrix L; finally, the search area point cloud P sea The spatial domain coordinate sequence x is converted into a frequency domain coefficient sequence through the graph Fourier transform Use the inverse Fourier transform to transform the frequency domain coefficient sequence Convert back to the spatial domain coordinate sequence and get
[0015] Specifically, in Step 1, construct the search area point cloud P sea The low frequency band importance saliency map includes the following steps:
[0016] (111) The template point cloud Ptmp And the search area point cloud P sea Input into the tracking model T to obtain the tracking prediction bounding box B;
[0017] (112) According to the low frequency threshold l low And the length of the low-frequency sub-band m, calculate the number of low-frequency sub-bands Where: low frequency threshold l low It is recommended to take the position of 1 / 3 of the entire frequency band, because experiments show that the influence of discarding 2 / 3 of the higher frequency band on tracking performance is small, which means that most of the information related to tracking in the search area is concentrated in the low-frequency area; the length of the low-frequency sub-band m should generally not be too long, and it is recommended to take a value within 5, and the recommended value is 4;
[0018] (113) The jth low-frequency sub-band The frequency domain coefficient sequence of is set to zero, and the frequency domain coefficient sequence is obtained The remaining frequency domain coefficient sequence of j=1,2,…,N low ;
[0019] (114) Use the inverse Fourier transform to transform the remaining frequency domain coefficient sequence Convert back to the spatial domain coordinate sequence and get Then get the remaining search area point cloud
[0020] (115) The template point cloud P tmp And the remaining search area point cloud Input into the tracking model T to get the tracking prediction bounding box
[0021] (116) Calculate the low frequency sub-band Importance score for tracking B gt is the true bounding box; Recorded in the low frequency band importance saliency map;
[0022] (117) For the search area point cloud P sea The importance scores of all low-frequency sub-bands in the tracking are calculated and recorded to obtain the importance significance map of the low-frequency bands.
[0023] Specifically, in Step 1, the search area point cloud P is calculated sea The importance weight σ of the low-frequency band includes the following steps:
[0024] (121) For the search area point cloud P seaFor the ith frequency domain coefficient in the low-frequency band, first determine the low-frequency sub-band to which it belongs, and then retrieve the importance score of the low-frequency sub-band to which it belongs for tracking from the low-frequency band importance saliency map, denoted as s i ;
[0025] (122) Calculate the search area point cloud P sea Importance weight of the i-th frequency domain coefficient in the low frequency band Among them, α is a hyperparameter that controls the upper limit of the weight, and t is a hyperparameter that controls the importance score-weight mapping;
[0026] (123) Search area point cloud P sea The importance weights of all frequency domain coefficients in the low frequency band constitute the low frequency band importance weight Where: N low Represents the total number of frequency domain coefficients in the low-frequency band.
[0027] Specifically, in Step 3, the low-frequency band importance weight σ is added to the frequency domain perturbation △ to obtain the low-frequency frequency domain perturbation, and the low-frequency frequency domain perturbation is added to the low-frequency coefficient sequence Get the adversarial low-frequency coefficient sequence Obtaining adversarial search area point cloud based on inverse image Fourier transform The steps include:
[0028] (31) The importance weight σ of the low-frequency band is multiplied by the frequency domain perturbation △ to obtain the weighted low-frequency frequency domain perturbation, and the low-frequency frequency domain perturbation is added to the low-frequency coefficient sequence Get the adversarial low-frequency coefficient sequence
[0029]
[0030] (32) Connecting adversarial low-frequency coefficient sequences and high frequency coefficient sequence Get the adversarial frequency domain coefficient sequence
[0031]
[0032] (33) The adversarial frequency domain coefficient sequence Perform inverse Fourier transform to obtain the adversarial space domain coordinate sequence Then we get the adversarial search area point cloud
[0033] Specifically, in Step 4, the template point cloud P tmp And adversarial search area point cloud Input into the tracking model T, get M candidate target proposals, select the candidate target proposal with the highest score as the tracking prediction bounding box B* , calculate the confidence loss L according to M candidate object proposals conf , bounding box offset loss L box and distance loss D c , including the following steps:
[0034] (41) Input the template point cloud P tmp and the adversarial search region point cloud into the tracking model T to obtain M candidate object proposals. After sorting the candidate object proposals in descending order of their scores, the m-th candidate object proposal is denoted as where: represents the score of the m-th candidate object proposal, represents the offset of the center point of the tracking prediction bounding box of the m-th candidate object proposal relative to the center point of the true bounding box B gt of the center point;
[0035] (42) Calculate the confidence loss L conf and the bounding box offset loss L box as:
[0036]
[0037]
[0038] where: represents the confidence of the m-th candidate object proposal, which can take the value of the score of the m-th candidate object proposal; 1 < p < q < r < M, and p, q, r are user-defined hyperparameters, generally taking the values of 15, 36, 50; is a specified non-target position point, and is taken as the one with a confidence ranking around 4 / 5 according to experimental suggestions
[0039] (43) Calculate the distance loss D c as:
[0040]
[0041] where: x i represents the i-th point of the search region point cloud P sea and is also the i-th point spatial domain coordinate in the spatial domain coordinate sequence x; represents the i-th point of the adversarial search region point cloud and is also the i-th point spatial domain coordinate in the adversarial spatial domain coordinate sequence x * of the i-th point.
[0042] Specifically, the Ste pIn 5, the total loss L is calculated, and the frequency domain perturbation △ is iteratively optimized using the total loss L to obtain an adversarial video frame example, including the following steps:
[0043] (51) Calculate the total loss L = L c o nf +aD c +bL box ; Among them, a and b are custom hyperparameters, and their values are generally 0.8 and 0.04;
[0044] (52) Using the Adam optimizer, the frequency domain perturbation △ is optimized according to the total loss L; after a set number of (150) rounds of iterative optimization, the final frequency domain perturbation △ and the final adversarial search area point cloud are obtained.
[0045] (53) The final adversarial search area point cloud Replace the search area point cloud P of the frame to which it belongs sea , and obtain adversarial video frame examples.
[0046] Specifically, in Step 6, the generated adversarial video frame example is tracked using the tracking model T to obtain the tracking result of the adversarial attack; the tracking requires:
[0047] ① Since the adversarial video frame example will guide the tracking box to the wrong position, the template point cloud P is not used in the tracking process. tmp Update, that is, use the offline template point cloud P in the tracking process tmp ;
[0048] ② When the tracking model T is a white box tracking model P2B, the three-dimensional offset and plane rotation angle are used to represent the tracking prediction bounding box;
[0049] ③ When the tracking model T is a black box tracking model BAT or M 2 -Track, the adversarial video frame examples generated by the white-box tracking model P2B are directly used as adversarial video frame examples.
[0050] A three-dimensional point cloud target tracking adversarial attack device based on frequency domain importance is used to implement any of the above three-dimensional point cloud target tracking adversarial attack methods based on frequency domain importance. The device adopts a CW attack framework and mainly includes a GFT-IGFT conversion module, a saliency map generation module, an addition attack module and a tracking model T.
[0051] The GFT-IGFT conversion module converts the input from the spatial domain to the frequency domain by using a graph Fourier transform, and converts the input from the frequency domain to the spatial domain by using an inverse graph Fourier transform;
[0052] The saliency map generation module includes a low-frequency importance saliency map construction unit and a low-frequency band importance weight calculation unit. The search area point cloud P is constructed by the low-frequency importance saliency map construction unit. sea The low-frequency band importance saliency map is used to calculate the search area point cloud P through the low-frequency band importance weight calculation unit. sea The importance weight of the low-frequency band σ;
[0053] The attack adding module first adds the importance weight σ of the low-frequency band to the frequency domain disturbance △ to obtain the low-frequency frequency domain disturbance; then adds the low-frequency frequency domain disturbance to the search area point cloud P sea The frequency domain coefficient sequence Get the adversarial frequency domain coefficient sequence Then, the adversarial frequency domain coefficient sequence is transformed through the GFT-IGFT conversion module Convert to the spatial domain to obtain the adversarial search area point cloud
[0054] The tracking model T is divided into a white-box tracking model and a black-box tracking model. The white-box tracking model adopts a Siamese network architecture, and the black-box tracking model adopts a Siamese network architecture and a motion center network architecture. First, the template point cloud P tmp And adversarial search area point cloud Input into the tracking model T, get M candidate target proposals, select the candidate target proposal with the highest score as the tracking prediction bounding box B * ; Then calculate the confidence loss L based on the M candidate target proposals conf , bounding box offset loss L box , distance loss D c and the total loss L; finally, the total loss L is used to iteratively optimize the frequency domain perturbation △ to obtain an adversarial video frame example.
[0055] Beneficial effects: The adversarial attack method and device for three-dimensional point cloud target tracking based on frequency domain importance provided by the present invention have the following advantages over the prior art: 1. The present invention makes full use of the decoupling relationship between the point cloud frequency domain information and the point cloud geometric structure to attack the geometric features, and the generated adversarial examples can significantly reduce the performance of the tracking model; 2. The present invention utilizes the frequency domain threshold to separate the generalized features and the non-generalized features, attacks the generalized features, and enhances the generalization of the attack. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] Figure 1 1 is a flow chart for implementing the method of the present invention;
[0057] Figure 2 A schematic diagram of the structure of an attack framework based on frequency domain importance for implementing the method of the present invention. DETAILED DESCRIPTION
[0058] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.
[0059] like Figure 1 The figure shows an implementation flow chart of a 3D point cloud target tracking attack method based on frequency domain importance. The attack occurs in the frequency domain and attacks the low-frequency components of the search area point cloud. First, the search area point cloud P is input. sea and template point cloud P tmp , construct a low-frequency importance saliency map and calculate the importance weight σ of the low-frequency band; search the point cloud P in the area sea According to the K-NN algorithm, the set is converted into a graph structure, and the graph Fourier transform (GFT) is used to transform the graph structure from the spatial domain to the frequency domain to obtain the search area point cloud P sea The frequency domain coefficient sequence Then the frequency domain coefficient sequence According to the low frequency threshold l low Divided into low frequency coefficient sequences and high frequency coefficient sequence Multiply the importance weight of the low-frequency band σ by the frequency domain perturbation △ term by term to obtain the weighted low-frequency frequency domain perturbation. The low-frequency frequency domain perturbation and the search area point cloud P sea The low-frequency frequency domain coefficient sequence Add them one by one to get the anti-low-frequency coefficient sequence Will counter the low frequency coefficient sequence With the original high frequency coefficient sequence Connect to get the adversarial frequency domain coefficient sequence The adversarial frequency domain coefficient sequence is transformed using the Inverse Graphic Fourier Transform (IGFT) Transform from frequency domain back to spatial domain to obtain adversarial search area point cloud The adversarial search area point cloud and template point cloud P tmp Input to the tracking model T to obtain candidate target proposals and tracking prediction bounding box B * ; According to the optimization function, the frequency domain perturbation △ is optimized through multiple rounds of iterations, and finally an adversarial video frame example is obtained.
[0060] The method of the present invention fully utilizes the association between the frequency domain information of the point cloud and the decoupling and generalization of the point cloud structure. Each step is specifically described below.
[0061] Step S01: input the search area point cloud and the template point cloud, construct a low-frequency importance saliency map, and calculate the importance weight of the low-frequency band.
[0062] In point cloud frame video sequence In the example, the real bounding box of the tracked target is used to crop the tracking prediction bounding box in the first frame; from the second frame onwards, the tracking prediction bounding box of the previous frame is called the template point cloud P of the current frame.tmp , according to the template point cloud P of the current frame tmp Crop the search area point cloud P of the current frame sea Then we get the tracking prediction bounding box B of the current frame. Randomly initialize the frequency domain perturbation Make it obey the normal distribution of N(0,103).
[0063] (11) Construct the search area point cloud P sea The low frequency band importance saliency map includes the following steps:
[0064] (111) The template point cloud P tmp And the search area point cloud P sea Input into the tracking model T to obtain the tracking prediction bounding box B;
[0065] (112) According to the low frequency threshold l low And the length of the low-frequency sub-band m, calculate the number of low-frequency sub-bands Where: low frequency threshold l low It is recommended to take the position of 1 / 3 of the entire frequency band, because experiments show that the influence of discarding 2 / 3 of the higher frequency band on tracking performance is small, which means that most of the information related to tracking in the search area is concentrated in the low-frequency area; the length of the low-frequency sub-band m should generally not be too long, and it is recommended to take a value within 5, and the recommended value is 4;
[0066] (113) The jth low-frequency sub-band The frequency domain coefficient sequence of is set to zero, and the frequency domain coefficient sequence is obtained The remaining frequency domain coefficient sequence of j=1,2,…,N low ;
[0067] (114) Use the inverse Fourier transform to transform the remaining frequency domain coefficient sequence Convert back to the spatial domain coordinate sequence and get Then get the remaining search area point cloud
[0068] (115) The template point cloud P tmp And the remaining search area point cloud Input into the tracking model T to get the tracking prediction bounding box
[0069] (116) Calculate the low frequency sub-band Importance score for tracking B gt is the true bounding box; Recorded in the low frequency band importance saliency map;
[0070] (117) For the search area point cloud Psea The importance scores of all low-frequency sub-bands in the tracking are calculated and recorded to obtain the importance significance map of the low-frequency bands.
[0071] (12) Calculate the search area point cloud P sea The importance weight σ of the low-frequency band includes the following steps:
[0072] (121) For the search area point cloud P sea For the ith frequency domain coefficient in the low-frequency band, first determine the low-frequency sub-band to which it belongs, and then retrieve the importance score of the low-frequency sub-band to which it belongs for tracking from the low-frequency band importance saliency map, denoted as s i ;
[0073] (122) Calculate the search area point cloud P sea Importance weight of the i-th frequency domain coefficient in the low frequency band Among them, α is a hyperparameter that controls the upper limit of the weight, and t is a hyperparameter that controls the importance score-weight mapping;
[0074] (123) Search area point cloud P sea The importance weights of all frequency domain coefficients in the low frequency band constitute the low frequency band importance weight Where: N low Represents the total number of frequency domain coefficients in the low-frequency band.
[0075] Step S02: transform the search area point cloud from the spatial domain to the frequency domain using graph Fourier transform, and use a low-frequency threshold to truncate a low-frequency coefficient sequence and a high-frequency coefficient sequence.
[0076] Use graph Fourier transform to transform the search area point cloud P sea From the spatial domain coordinate sequence Transformed into frequency domain coefficient sequence The following steps are included: First, the K-NN algorithm is used to calculate the search area point cloud P sea The K nearest neighbors of each point in the search area point cloud P sea From the point set to the graph structure G = {U, ε, W}, the vertex set U is the search area point cloud P sea The edge set ε is the set of adjacent edges connecting each point to all its K nearest neighbors. W represents the adjacency matrix. Next, construct the Laplacian matrix L of the graph structure G: DW. The diagonal elements d in the diagonal matrix D are i represents the sum of the weights of all adjacent edges of point i; then, the Laplace matrix L is decomposed to obtain L = VΛV T , V is a standard orthogonal matrix containing the eigenvectors of the Laplace matrix L, Λ is a diagonal matrix composed of the eigenvalues of the Laplace matrix L; finally, the search area point cloud P seaThe spatial domain coordinate sequence x is converted into a frequency domain coefficient sequence through the graph Fourier transform Use the inverse Fourier transform to transform the frequency domain coefficient sequence Convert back to the spatial domain coordinate sequence and get
[0077] Search area point cloud P sea The i-th point of is represented by the i-th point in the spatial domain coordinate sequence x i ; According to the low frequency threshold l low The frequency domain coefficient sequence Truncated to low frequency coefficient sequence and high frequency coefficient sequence Where: N represents the search area point cloud P sea The total number of points.
[0078] Step S03: Add the weighted low-frequency frequency domain perturbation to the low-frequency coefficient sequence to obtain an adversarial low-frequency coefficient sequence and an adversarial frequency domain coefficient sequence, and obtain the adversarial search area point cloud by inverse image Fourier transform.
[0079] (31) The importance weight σ of the low-frequency band is multiplied by the frequency domain perturbation △ to obtain the weighted low-frequency frequency domain perturbation, and the low-frequency frequency domain perturbation is added to the low-frequency coefficient sequence To enhance the attack of important low-frequency sub-bands, we can obtain the anti-low-frequency coefficient sequence
[0080]
[0081] (32) Connecting adversarial low-frequency coefficient sequences and high frequency coefficient sequence Get the adversarial frequency domain coefficient sequence
[0082]
[0083] (33) The adversarial frequency domain coefficient sequence Perform inverse Fourier transform to obtain the adversarial space domain coordinate sequence Then we get the adversarial search area point cloud
[0084] Step S04: Calculate confidence loss, bounding box offset loss, and distance loss.
[0085] The template point cloud P tmp And adversarial search area point cloud Input into the tracking model T, get M candidate target proposals, select the candidate target proposal with the highest score as the tracking prediction bounding box B *, calculate the confidence loss L according to M candidate object proposals conf , bounding box offset loss L box and distance loss D c , including the following steps:
[0086] (41) Input the template point cloud P tmp and the adversarial search region point cloud into the tracking model T to obtain M candidate object proposals. After sorting the candidate object proposals in descending order of their scores, the m-th candidate object proposal is denoted as where: represents the score of the m-th candidate object proposal, represents the offset of the center point of the tracking prediction bounding box of the m-th candidate object proposal relative to the center point of the true bounding box B gt center point;
[0087] (42) Calculate the confidence loss L conf and the bounding box offset loss L box as:
[0088]
[0089]
[0090] where: represents the confidence of the m-th candidate object proposal, which can take the value of the score of the m-th candidate object proposal; 1 < p < q < r < M, p, q, r are user-defined hyperparameters, and in this case, the values are 15, 36, 50; is a specified non-target position point, and according to the experimental suggestion, it takes the value of the point with a confidence ranking around 4 / 5
[0091] (43) Calculate the distance loss D c as:
[0092]
[0093] where: x i represents the i-th point of the search region point cloud P sea and is also the i-th point spatial domain coordinate in the spatial domain coordinate sequence x; represents the i-th point of the adversarial search region point cloud and is also the i-th point spatial domain coordinate in the adversarial spatial domain coordinate sequence x * ;
[0094] Step S05: Calculate the total loss, and use the total loss to iteratively optimize the frequency domain perturbation to obtain an adversarial video frame example.
[0095] According to the confidence loss Lconf , bounding box offset loss L box and distance loss D c Calculate the total loss L, and use the total loss L to iteratively optimize the frequency domain perturbation △ to obtain an adversarial video frame example, including the following steps:
[0096] (51) Calculate the total loss L = L conf +aD c +bL box ; Among them, a and b are custom hyperparameters, and their values in this case are 0.8 and 0.04;
[0097] (52) Using the Adam optimizer, the frequency domain perturbation △ is optimized according to the total loss L; after 150 iterations of optimization, the final frequency domain perturbation △ and the final adversarial search area point cloud are obtained.
[0098] (53) The final adversarial search area point cloud Replace the search area point cloud P of the frame to which it belongs sea , and obtain adversarial video frame examples.
[0099] Step S06: Use the tracking model to track the adversarial video frame example to obtain the tracking result of the adversarial attack.
[0100] Use the tracking model T to track the generated adversarial video frame examples to obtain the tracking results of the adversarial attack; the tracking requirements are:
[0101] ① Since the adversarial video frame example will guide the tracking box to the wrong position, the template point cloud P is not used in the tracking process. tmp Update, that is, use the offline template point cloud P in the tracking process tmp ;
[0102] ② When the tracking model T is a white box tracking model P2B, the three-dimensional offset and plane rotation angle are used to represent the tracking prediction bounding box;
[0103] ③ When the tracking model T is a black box tracking model BAT or M 2 -Track, the adversarial video frame examples generated by the white-box tracking model P2B are directly used as adversarial video frame examples.
[0104] A device for implementing the above-mentioned frequency domain importance-based three-dimensional point cloud target tracking adversarial attack method, adopting a CW attack framework, mainly including a GFT-IGFT conversion module, a saliency map generation module, an addition attack module and a tracking model T;
[0105] The GFT-IGFT conversion module converts the input from the spatial domain to the frequency domain by using a graph Fourier transform, and converts the input from the frequency domain to the spatial domain by using an inverse graph Fourier transform;
[0106] The saliency map generation module includes a low-frequency importance saliency map construction unit and a low-frequency band importance weight calculation unit. The search area point cloud P is constructed by the low-frequency importance saliency map construction unit. sea The low-frequency band importance saliency map is used to calculate the search area point cloud P through the low-frequency band importance weight calculation unit. sea The importance weight of the low-frequency band σ;
[0107] The attack adding module first adds the importance weight σ of the low-frequency band to the frequency domain disturbance △ to obtain the low-frequency frequency domain disturbance; then adds the low-frequency frequency domain disturbance to the search area point cloud P sea The frequency domain coefficient sequence Get the adversarial frequency domain coefficient sequence Then, the adversarial frequency domain coefficient sequence is transformed through the GFT-IGFT conversion module Convert to the spatial domain to obtain the adversarial search area point cloud
[0108] The tracking model T is divided into a white-box tracking model and a black-box tracking model. The white-box tracking model adopts a Siamese network architecture, and the black-box tracking model adopts a Siamese network architecture and a motion center network architecture. First, the template point cloud P tmp And adversarial search area point cloud Input into the tracking model T, get M candidate target proposals, select the candidate target proposal with the highest score as the tracking prediction bounding box B * ; Then calculate the confidence loss L based on the M candidate target proposals conf , bounding box offset loss L box , distance loss D c and the total loss L; finally, the total loss L is used to iteratively optimize the frequency domain perturbation △ to obtain an adversarial video frame example.
[0109] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any form, and any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.
Claims
1. A method for 3D point cloud target tracking against adversarial attacks based on frequency domain importance, characterized by: The steps include: Step 1. Input the search area point cloud P sea and frequency domain perturbation △, construct the search area point cloud P sea The importance saliency map of the low-frequency band is used to calculate the importance weight σ of the low-frequency band required for the attack; Step 2. Use graph Fourier transform to transform the search area point cloud P sea From the spatial domain coordinate sequence Transformed into frequency domain coefficient sequence Search area point cloud P sea The i-th point of is represented by the i-th point in the spatial domain coordinate sequence x i ; According to the low frequency threshold l low The frequency domain coefficient sequence Truncated to low frequency coefficient sequence and high frequency coefficient sequence Where: N represents the search area point cloud P sea Total points of Step 3. Add the importance weight σ of the low-frequency band to the frequency domain disturbance △ to obtain the weighted low-frequency frequency domain disturbance, and add the low-frequency frequency domain disturbance to the low-frequency coefficient sequence Get the adversarial low-frequency coefficient sequence Concatenate adversarial low frequency coefficient sequences and high frequency coefficient sequence Get the adversarial frequency domain coefficient sequence Countermeasure frequency domain coefficient sequence Perform inverse Fourier transform to obtain the adversarial space domain coordinate sequence x * , and then get the adversarial search area point cloud Step 4. P the template point cloud tmp And adversarial search area point cloud Input into the tracking model T, get M candidate target proposals, select the candidate target proposal with the highest score as the tracking prediction bounding box B * , calculate the confidence loss L based on the M candidate target proposals conf , bounding box offset loss L box and distance loss D c ; Step 5. According to the confidence loss L conf , bounding box offset loss L box and distance loss D c Calculate the total loss L, and use the total loss L to iteratively optimize the frequency domain perturbation △ to obtain an adversarial video frame example; Step 6: Use the tracking model T to track the generated adversarial video frame examples to obtain the tracking results of the adversarial attack.
2. The method for 3D point cloud target tracking against adversarial attacks based on frequency domain importance according to claim 1, characterized in that: In point cloud frame video sequence In the example, the real bounding box of the tracked target is used to crop the tracking prediction bounding box in the first frame; from the second frame onwards, the tracking prediction bounding box of the previous frame is called the template point cloud P of the current frame. tmp , according to the template point cloud P of the current frame tmp Crop the search area point cloud P of the current frame sea Then the tracking prediction bounding box B of the current frame is obtained.
3. The method for 3D point cloud target tracking against adversarial attacks based on frequency domain importance according to claim 1, characterized in that: Use graph Fourier transform to transform the search area point cloud P sea From the spatial domain coordinate sequence Transformed into frequency domain coefficient sequence The following steps are included: First, the K-NN algorithm is used to calculate the search area point cloud P sea The K nearest neighbors of each point in the search area point cloud P sea From the point set to the graph structure G = {U, ε, W}, the vertex set U is the search area point cloud P sea The edge set ε is the set of adjacent edges connecting each point to all its K nearest neighbors. W represents the adjacency matrix. Next, construct the Laplacian matrix L of the graph structure G: DW. The diagonal elements d in the diagonal matrix D are i represents the sum of the weights of all adjacent edges of point i; then, the Laplace matrix L is decomposed to obtain L = VΛV T , V is a standard orthogonal matrix containing the eigenvectors of the Laplace matrix L, Λ is a diagonal matrix composed of the eigenvalues of the Laplace matrix L; finally, the search area point cloud P sea The spatial domain coordinate sequence x is converted into a frequency domain coefficient sequence through the graph Fourier transform Use the inverse Fourier transform to transform the frequency domain coefficient sequence Convert back to the spatial domain coordinate sequence and get 4. The method for 3D point cloud target tracking against adversarial attacks based on frequency domain importance according to claim 1, characterized in that: In Step 1, construct the search area point cloud P sea The low frequency band importance saliency map includes the following steps: (111) The template point cloud P tmp and search area point cloud P sea Input into the tracking model T to obtain the tracking prediction bounding box B; (112) According to the low frequency threshold l low And the length of the low-frequency sub-band m, calculate the number of low-frequency sub-bands (113) The jth low-frequency sub-band The frequency domain coefficient sequence of is set to zero, and the frequency domain coefficient sequence is obtained The remaining frequency domain coefficient sequence of (114) Use the inverse Fourier transform to transform the remaining frequency domain coefficient sequence Convert back to the spatial domain coordinate sequence and get Then get the remaining search area point cloud (115) The template point cloud P tmp And the remaining search area point cloud Input into the tracking model T to get the tracking prediction bounding box (116) Calculate the low frequency sub-band Importance score for tracking B gt is the true bounding box; Recorded in the low frequency band importance saliency map; (117) For the search area point cloud P sea The importance scores of all low-frequency sub-bands in the tracking are calculated and recorded to obtain the importance significance map of the low-frequency bands.
5. The method for 3D point cloud target tracking against adversarial attacks based on frequency domain importance according to claim 1, characterized in that: In Step 1, the search area point cloud P is calculated. sea The low-frequency band importance weight σ includes the following steps: (121) For the search area point cloud P sea For the ith frequency domain coefficient in the low-frequency band, first determine the low-frequency sub-band to which it belongs, and then retrieve the importance score of the low-frequency sub-band to which it belongs for tracking from the low-frequency band importance saliency map, denoted as s i ; (122) Calculate the search area point cloud P sea Importance weight of the i-th frequency domain coefficient in the low frequency band Among them, α is a hyperparameter that controls the upper limit of the weight, and t is a hyperparameter that controls the importance score-weight mapping; (123) Search area point cloud P sea The importance weights of all frequency domain coefficients in the low frequency band constitute the low frequency band importance weight Where: N low Represents the total number of frequency domain coefficients in the low-frequency band.
6. The method for 3D point cloud target tracking against adversarial attacks based on frequency domain importance according to claim 1, characterized in that: In the Step 3, the low frequency band importance weight σ is added to the frequency domain perturbation △ to obtain the low frequency frequency domain perturbation, and the low frequency frequency domain perturbation is added to the low frequency coefficient sequence Get the adversarial low-frequency coefficient sequence Obtaining adversarial search area point cloud based on inverse image Fourier transform The steps include: (31) The importance weight σ of the low-frequency band is multiplied by the frequency domain perturbation △ to obtain the weighted low-frequency frequency domain perturbation, and the low-frequency frequency domain perturbation is added to the low-frequency coefficient sequence Get the adversarial low-frequency coefficient sequence (32) Connecting adversarial low-frequency coefficient sequences and high frequency coefficient sequence Get the adversarial frequency domain coefficient sequence (33) The adversarial frequency domain coefficient sequence Perform inverse Fourier transform to obtain the adversarial space domain coordinate sequence Then we get the adversarial search area point cloud 7. The method for 3D point cloud target tracking against adversarial attacks based on frequency domain importance according to claim 1, characterized in that: In Step 4, the template point cloud P tmp And adversarial search area point cloud Input into the tracking model T, get M candidate target proposals, select the candidate target proposal with the highest score as the tracking prediction bounding box B * , calculate the confidence loss L based on the M candidate target proposals conf , bounding box offset loss L box and distance loss D c , including the following steps: (41) The template point cloud P tmp And adversarial search area point cloud Input into the tracking model T, and obtain M candidate target proposals. After sorting the candidate target proposals from high to low according to their scores, the mth candidate target proposal is expressed as in: represents the score of the mth candidate target proposal, Represents the center point of the tracking prediction bounding box of the mth candidate target proposal relative to the real bounding box B gt Offset of the center point; (42) Calculate the confidence loss L conf and the bounding box offset loss L box for: Wherein: represents the confidence of the m-th candidate target proposal; 1 < p < q < r < M, where p, q, and r are user-defined hyperparameters; is a specified non-target location point; (43) Calculate the distance loss D c for: Where: x i Represents the search area point cloud P sea The i-th point of is also the spatial domain coordinate of the i-th point in the spatial domain coordinate sequence x; Represents the adversarial search area point cloud The i-th point is also the adversarial space domain coordinate sequence x * The spatial domain coordinates of the i-th point in .
8. The method for 3D point cloud target tracking against adversarial attacks based on frequency domain importance according to claim 1, characterized in that: In the Step 5, the total loss L is calculated, and the frequency domain perturbation △ is iteratively optimized using the total loss L to obtain an adversarial video frame example, including the following steps: (51) Calculate the total loss L = L conf +aD c +bL box ; Among them, a and b are custom hyperparameters; (52) Using the Adam optimizer, the frequency domain perturbation △ is optimized according to the total loss L; after a set number of (150) rounds of iterative optimization, the final frequency domain perturbation △ and the final adversarial search area point cloud are obtained. (53) The final adversarial search area point cloud Replace the search area point cloud P of the frame to which it belongs sea , and obtain adversarial video frame examples.
9. The method for 3D point cloud target tracking against adversarial attacks based on frequency domain importance according to claim 1, characterized in that: In Step 6, the generated adversarial video frame example is tracked using the tracking model T to obtain the tracking result of the adversarial attack; the tracking requires: ① The template point cloud P is not correct during the tracking process. tmp Update, that is, use the offline template point cloud P in the tracking process tmp ; ② When the tracking model T is a white box tracking model, the three-dimensional offset and plane rotation angle are used to represent the tracking prediction bounding box; ③ When the tracking model T is a black-box tracking model, the adversarial video frame examples generated by the white-box tracking model P2B are directly used as adversarial video frame examples.
10. A three-dimensional point cloud target tracking adversarial attack device based on frequency domain importance, characterized by: Used to implement any of the frequency domain importance-based three-dimensional point cloud target tracking adversarial attack methods described in claims 1 to 9, the device adopts a CW attack framework, mainly including a GFT-IGFT conversion module, a saliency map generation module, an addition attack module and a tracking model T; The GFT-IGFT conversion module converts the input from the spatial domain to the frequency domain by using a graph Fourier transform, and converts the input from the frequency domain to the spatial domain by using an inverse graph Fourier transform; The saliency map generation module includes a low-frequency importance saliency map construction unit and a low-frequency band importance weight calculation unit. The search area point cloud P is constructed by the low-frequency importance saliency map construction unit. sea The low-frequency band importance saliency map is used to calculate the search area point cloud P through the low-frequency band importance weight calculation unit. sea The importance weight of the low-frequency band σ; The attack adding module first adds the low-frequency band importance weight σ to the frequency domain disturbance △ to obtain the low-frequency frequency domain disturbance; Then add the low-frequency frequency domain perturbation to the search area point cloud P sea The frequency domain coefficient sequence Get the adversarial frequency domain coefficient sequence Then, the adversarial frequency domain coefficient sequence is transformed through the GFT-IGFT conversion module Convert to the spatial domain to obtain the adversarial search area point cloud The tracking model T is divided into a white-box tracking model and a black-box tracking model. The white-box tracking model adopts a Siamese network architecture, and the black-box tracking model adopts a Siamese network architecture and a motion center network architecture. First, the template point cloud P tmp And adversarial search area point cloud Input into the tracking model T, get M candidate target proposals, select the candidate target proposal with the highest score as the tracking prediction bounding box B * ; Then calculate the confidence loss L based on the M candidate target proposals conf , bounding box offset loss L box , distance loss D c and the total loss L; finally, the total loss L is used to iteratively optimize the frequency domain perturbation △ to obtain an adversarial video frame example.
Citation Information
Patent Citations
Visual target tracking transferable black box attack method based on important features
CN114511593A
RF signal classification device incorporating quantum computing with game theoretic optimization and related methods
US20240160978A1
Method and system for improving point cloud classification accuracy based on graph spectral domain
US20250095334A1
Cited By
Method for generating imperceptible three-dimensional point cloud confrontation sample
CN120893479A
