A transferable black-box attack method for visual object tracking based on important features

Abstract

The present invention provides a transferable black-box attack method for visual target tracking based on important features, by attacking important features that are highly correlated with the tracking target and are not affected by the source model, reducing their importance, and enhancing unimportant features to achieve a transferable attack; that is, the importance of the feature is reflected by the corresponding gradient obtained by backpropagation, and then the weighted features obtained by the gradient are used for attack. In addition, the present invention uses the temporal information of the similarity between two adjacent frames of the video to propose a feature similarity descent loss function, and attacks by reducing the feature similarity between adjacent frames. The present invention can achieve an effective black-box attack on the target tracker by generating adversarial samples with strong transferability without knowing the target tracking model and without performing a large number of queries.

Description

Technical Field

[0001] The invention relates to a transferable black box attack method for visual target tracking based on important features, belonging to visual target tracking counterattack technology. Background Art

[0002] Visual object tracking is one of the basic problems in computer vision and has been widely used in fields such as autonomous driving. With the development of convolutional neural networks, it has shown extremely superior performance in the field of computer vision and has been widely used. Similarly, with the application of convolutional neural networks to visual object tracking, the field of visual object tracking has also made great progress. Since Szegedy et al. first proposed adversarial attacks, a large number of studies have shown that CNNs are extremely vulnerable to adversarial attacks, and the field of visual object tracking is also potentially threatened by adversarial attacks.

[0003] Many works have been conducted on adversarial attacks in the field of visual target tracking, but most of the current attack methods are white-box attacks, that is, attacks are performed with the internal information of the tracking model known. Only a small number of studies have targeted more challenging black-box attacks. In the current research on black-box attacks on visual target tracking, most of them are query-based black-box attack methods. This method applies strong noise to the video, inputs it into the tracker to obtain the tracking result, and generates adversarial samples through the query results. However, query-based attacks are often not feasible in reality because it is impossible for models in reality to perform a large number of queries. In addition, the adversarial samples obtained by this method are often more suitable for models with specific attacks, and the attack effect is poor when applied to other models. Summary of the invention

[0004] Purpose of the invention: In order to overcome the deficiencies in the prior art, the present invention provides a transferable black-box attack method for visual target tracking based on important features, which attacks important features that are highly relevant to the tracked target and are not affected by the source model, thereby reducing their importance, and enhancing unimportant features to achieve a transferable attack; that is, the importance of the feature is reflected by the corresponding gradient obtained by backpropagation, and then the weighted feature obtained by the gradient is used for attack. In addition, the present invention uses the temporal information of the similarity between two adjacent frames of the video, proposes a feature similarity descent loss function, and attacks by reducing the feature similarity between adjacent frames.

[0005] A transferable black-box attack method for visual object tracking based on important features, comprising the following steps:

[0006] (1) Obtain the video sample X = {x0, x1, x2, ..., x n}; Take the first frame x0 of the video sample as the template frame z, and use the template frame z to initialize the target; Set the subsequent frames {x1, x2, …, x n} as the search frame, i.e. the frame to be attacked, and n is the number of search frames;

[0007] (2) Calculate the important feature descent loss function L1;

[0008] (3) Calculate the feature similarity descent loss function L2;

[0009] (4) Combine the important feature reduction loss function L1 and the feature similarity reduction loss function L2 to calculate the total loss function L = α1L1 + α2L2 that needs to be optimized; where α1 and α2 are the weights of L1 and L2 respectively;

[0010] (5) Generate adversarial samples by minimizing the total loss function L through momentum iteration method;

[0011] (6) Input the adversarial sample into the visual target tracker f' that needs to be attacked and has other unknown internal information, and obtain the predicted target position of the visual target tracker f' after being attacked.

[0012] Preferably, in step (2), calculating the important feature reduction loss function L1 comprises the following steps:

[0013] (21) inputting the template frame and the search frame into a visual target tracker f with known internal information to obtain a template frame feature map and a search frame feature map; at the same time, obtaining a gradient corresponding to the search frame feature map according to back propagation on the visual target tracker f;

[0014] (22) calculating a weighted feature map of the search frame by searching the feature map of the frame and its corresponding gradient;

[0015]

[0016] Where: x i Search frame for the i-th frame, i = 1, 2, ..., n, feat(x i ) is x i The search frame feature map, Δ i feat(x i ) corresponding to the gradient, ⊙ represents the dot product, ||·||2 represents the L2 norm; for Δ i After regularization, feat(x i ) and multiply them to get x i The weighted feature map feat i ;

[0017] (23) Calculate the important feature descent loss function L1 = ∑feat i .

[0018] Preferably, in step (3), calculating the feature similarity reduction loss function L2 comprises the following steps:

[0019] (31) Calculate the feature similarity graph:

[0020]

[0021] in: For x i The feature similarity graph of

[0022] (32) Calculate the feature similarity descent loss function

[0023] Preferably, in step (5), generating adversarial samples by minimizing the total loss function L through momentum iteration method includes the following steps:

[0024] (51) Apply the visual target tracker f to x i The tracking result is expressed as f(z,x i ), the adversarial attack method of the present invention is to generate adversarial samples under the condition of adding a small disturbance so that the predicted position of the visual target tracker f is wrong, requiring:

[0025]

[0026]

[0027] in: Indicates location and position b i The intersection ratio score between represents the visual target tracker f The tracking results, For x i Adversarial examples of ||·|| p represents the Lp distance, b i For x i The target real annotation position, ε max is the maximum disturbance value added; combining the above two formulas, we have:

[0028]

[0029] (52) The momentum iteration method is used to solve the final adversarial sample:

[0030]

[0031]

[0032] in: represents the x obtained after the tth iteration i Adversarial examples, ||·||1 represents the L1 distance, is the search frame x obtained by back-propagating the total loss function L on the visual object tracker f i The corresponding gradient, g t is the regularized gradient obtained by superposition of iterations t times, μ is the control weight for controlling the iteration, sign(·) is the sign function, ε=ε max / T is the perturbation size added in each iteration, T is the total number of iterations, Add an adversarial perturbation ε·sign(g t+1 )of After pruning, the final adversarial sample is obtained after T iterations. Perform the above operation on each frame of the search frame to obtain a complete adversarial sample.

[0033] Beneficial effect: The transferable black-box attack method for visual target tracking based on important features provided by the present invention can achieve the situation where the target tracking model is unknown and a large number of queries are not performed. By attacking the important features in the features that are highly correlated with the tracking target and are not affected by the source model, the importance of the features is reduced. At the same time, the temporal information of the similarity between two adjacent frames of the video is used to propose a feature similarity reduction loss function. By reducing the feature similarity between adjacent frames, an adversarial attack sample with strong transferability is used to achieve an effective black-box attack on other target trackers. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 1 is a flow chart for implementing the method of the present invention;

[0035] Figure 2 Flowchart for computing visual object tracker and important feature descent function;

[0036] Figure 3 Flowchart for calculating the important feature descent loss function and feature similarity descent loss function;

[0037] Figure 4 is an adversarial sample image of the original image obtained by the method of the present invention. DETAILED DESCRIPTION

[0038] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0039] In the description of the present invention, it is necessary to understand that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0040] The present invention proposes a transferable black-box attack method for visual target tracking based on important features. It is a transfer-based black-box attack method for deep learning target tracking models. It generates adversarial samples by performing white-box attacks on the source model, and uses the strong transferability of adversarial samples to perform black-box attacks on other models with unknown internal information. The present invention starts with the features extracted by the deep network tracker. Since the important part of the features is related to the tracking target, the attack is carried out on important features that are universal and not affected by the source model. In addition, the timing information of the video is used, and according to the similarity between adjacent frames of the video, the attack is carried out by reducing the feature similarity between adjacent frames, thereby making full use of the spatial and temporal information of the video to obtain efficient and strongly transferable adversarial samples. The implementation process of the method of the present invention is as follows: Figure 1 As shown, the following steps are included:

[0041] Step S1: Obtain the video sample X = {x0, x1, x2, ..., x n}; Take the first frame x0 of the video sample as the template frame z, and use the template frame z to initialize the target; Set the subsequent frames {x1, x2, …, x n} as the search frame, that is, the frame that needs to be attacked, and n is the number of search frames.

[0042] Step S2: Calculate the important feature descent loss function, such as Figure 2 The flowchart of computing the visual object tracker f and the important feature descent function is shown; the steps include:

[0043] (21) inputting the template frame and the search frame into a visual target tracker f with known internal information to obtain a template frame feature map and a search frame feature map; at the same time, obtaining a gradient corresponding to the search frame feature map according to back propagation on the visual target tracker f;

[0044] (22) calculating a weighted feature map of the search frame by searching the feature map of the frame and its corresponding gradient;

[0045]

[0046] Where: x i Search frame for the i-th frame, i = 1, 2, ..., n, feat(x i ) is x i The search frame feature map, Δ i feat(x i ) corresponding to the gradient, ⊙ represents the dot product, ||·||2 represents the L2 norm; for Δ i After regularization, feat(x i ) and multiply them to get x i The weighted feature map feat i ;

[0047] (23) Calculate the important feature descent loss function L1 = ∑feat i ;

[0048] Step S3: Calculate the feature similarity descent loss function, such as Figure 3 The flowchart shown is for calculating the important feature descent loss function and the feature similarity descent loss function, including the following steps:

[0049] (31) Calculate the feature similarity graph:

[0050]

[0051] in: For x i The feature similarity graph of

[0052] (32) Calculate the feature similarity descent loss function

[0053] Step S4: Combine the important feature reduction loss function L1 and the feature similarity reduction loss function L2 to calculate the total loss function L=α1L1+α2L2 that needs to be optimized; where α1 and α2 are the weights of L1 and L2 respectively;

[0054] Step S5: Generate adversarial samples by minimizing the total loss function L through momentum iteration method;

[0055] (51) Apply the visual target tracker f to x i The tracking result is expressed as f(z,x i ), the adversarial attack method of the present invention is to generate adversarial samples under the condition of adding a small disturbance so that the predicted position of the visual target tracker f is wrong, requiring:

[0056]

[0057]

[0058] in: Indicates location and position b i The Intersection over Union score between represents the visual target tracker f The tracking results, For x i Adversarial examples of ||·|| p represents the Lp distance, b i For x i The target real annotation position, ε max is the maximum disturbance value added; combining the above two formulas, we have:

[0059]

[0060] (52) The momentum iteration method is used to solve the final adversarial sample:

[0061]

[0062]

[0063] in: represents the x obtained after the tth iteration i Adversarial examples, ||·||1 represents the L1 distance, is the search frame x obtained by back-propagating the total loss function L on the visual object tracker f i The corresponding gradient, g t is the regularized gradient obtained by superposition of iterations t times, μ is the control weight for controlling the iteration, sign(·) is the sign function, ε=ε max / T is the perturbation size added in each iteration, T is the total number of iterations, Add an adversarial perturbation ε·sign(g t+1 )of After pruning, the final adversarial sample is obtained after T iterations. Perform the above operation on each frame of the search frame to obtain a complete adversarial sample;

[0064] Step S6: The adversarial sample Input other unknown internal information to the visual target tracker f' that needs to be attacked, and obtain the predicted target position after the visual target tracker f' is attacked.

[0065] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0066] like Figure 4 Shown is an adversarial sample image of the original image obtained by the method of the present invention.

[0067] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0068] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any form, and any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A transferable black-box attack method for visual object tracking based on important features, characterized by: The steps include: (1) Obtain the video sample X = {x0, x1, x2, ..., x n }; Take the first frame x0 of the video sample as the template frame z, and use the template frame z to initialize the target; Set the subsequent frames {x1, x2, …, x n } as the search frame, i.e. the frame to be attacked, and n is the number of search frames; (2) Calculate the important feature descent loss function L1; (3) Calculate the feature similarity descent loss function L2; (4) Combine the important feature reduction loss function L1 and the feature similarity reduction loss function L2 to calculate the total loss function L = α1L1 + α2L2 that needs to be optimized; where α1 and α2 are the weights of L1 and L2 respectively; (5) Generate adversarial samples by minimizing the total loss function L through momentum iteration method; (6) Input the adversarial sample into the visual target tracker f' that needs to be attacked and has other unknown internal information, and obtain the predicted target position of the visual target tracker f' after being attacked.

2. The transferable black-box attack method for visual target tracking based on important features according to claim 1 is characterized in that: In the step (2), calculating the important feature reduction loss function L1 includes the following steps: (21) inputting the template frame and the search frame into a visual target tracker f with known internal information to obtain a template frame feature map and a search frame feature map; at the same time, obtaining a gradient corresponding to the search frame feature map according to back propagation on the visual target tracker f; (22) calculating a weighted feature map of the search frame by searching the feature map of the frame and its corresponding gradient; Where: x i Search frame for the i-th frame, i = 1, 2, ..., n, feat(x i ) is x i The search frame feature map, Δ i feat(x i ) corresponding to the gradient, ⊙ represents the dot product, ||·||2 represents the L2 norm; for Δ i After regularization, feat(x i ) and multiply them to get x i The weighted feature map feat i ; (23) Calculate the important feature descent loss function L1 = ∑feat i .

3. The method for transferring black-box attack based on important features for visual target tracking according to claim 1 is characterized in that: In the step (3), calculating the feature similarity reduction loss function L2 includes the following steps: (31) Calculate the feature similarity graph: in: For x i The feature similarity graph of (32) Calculate the feature similarity descent loss function 4. The method for transferring black-box attack based on important features for visual target tracking according to claim 1 is characterized in that: In step (5), the adversarial sample is generated by minimizing the total loss function L by the momentum iteration method, which includes the following steps: (51) Apply the visual target tracker f to x i The tracking result is expressed as f(z,x i ),Require: in: Indicates location and position b i The intersection ratio score between represents the visual target tracker f The tracking results, For x i Adversarial examples, ||·|| p represents the Lp distance, b i For x i The target real annotation position, ε max is the maximum disturbance value added; then: (52) The momentum iteration method is used to solve the final adversarial sample: in: represents the x obtained after the tth iteration i Adversarial examples, ||·||1 represents the L1 distance, is the search frame x obtained by back-propagating the total loss function L on the visual object tracker f i The corresponding gradient, g t is the regularized gradient obtained by superposition of iterations t times, μ is the control weight for controlling the iteration, sign(·) is the sign function, ε=ε max / T is the perturbation size added in each iteration, T is the total number of iterations, Add an adversarial perturbation ε·sign(g t+1 )of After pruning, the final adversarial sample is obtained after T iterations.

Images