Target tracking confrontation disturbance generation method based on generative adversarial network

Abstract

The invention discloses a target tracking confrontation disturbance generation method based on a generative adversarial network, which belongs to the technical field of image processing and comprises the following steps: acquiring a video sequence, an initial template and an initial search area; constructing an adversarial generative network comprising a disturbance generator G and a target tracker F, and generating initial disturbance and query disturbance; and constructing a key frame set and performing frame-by-frame processing, generating an adversarial template and an adversarial search area of each key frame, constructing total loss and training and updating G to obtain a shared disturbance generator for subsequent adversarial attacks. According to the method, the concealment problem is solved through an innovative cross-frame disturbance propagation mechanism, a disturbance dynamic size adaptation strategy and a multi-dimensional concealment constraint, so that the calculation complexity can be reduced, the attack speed can be improved, the concealment is maintained while the high attack success rate is ensured, and the method is suitable for large-scale popularization and application. The problem that traditional non-target attacks depend on calculation complexity or are easy to detect due to high-amplitude disturbance is solved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method for generating adversarial disturbances for target tracking based on a generative adversarial network. Background Art

[0002] Object tracking is one of the most challenging tasks in computer vision. It requires continuous positioning of a target in subsequent video frames based on the specified target in the first frame of a video. With the advancement of deep learning research, deep learning-based algorithms have demonstrated advantages in various fields, including object tracking. Deep learning provides powerful feature learning capabilities for object tracking, automatically learning robust target feature representations from large amounts of data and performing effective state estimation. Currently, several commonly used deep learning-based object tracking algorithms include the SiamFC tracker, SiamRPN, SiamRPN++ tracker, and SiamMask tracker.

[0003] Adversarial attacks are a key concept in machine learning. They involve making subtle, imperceptible modifications to input data (such as images, text, and audio) to cause machine learning models to produce incorrect predictions. Adversarial attacks have become an increasingly important research topic in target tracking, as they can reveal potential vulnerabilities in tracking algorithms and challenge the robustness of existing state-of-the-art models. The core of adversarial attacks in target tracking lies in designing perturbation generation strategies that target vulnerable modules within the algorithmic framework, such as the template-search region matching mechanism.

[0004] Although adversarial attack methods in the field of target tracking have made many achievements, they still have many limitations. Based on different attack strategies, current methods have different limitations: Frame-by-frame attack: Attack methods that calculate and inject perturbations for each frame place extremely high demands on hardware performance, which may conflict with the real-time requirements of target tracking. At the same time, the optimized cross-frame perturbation accumulation may make adversarial perturbations easier to detect.

[0005] Single-frame attack: Although the method of generating and injecting perturbations into only a single frame greatly reduces the computational performance requirements, its attack effect is unsatisfactory. Although the impact on only a single frame can cause temporary tracking errors, the tracking model may quickly correct the tracking position in subsequent frames.

[0006] Common limitations: Regardless of the attack method, the success rate is proportional to the perturbation amplitude; that is, the greater the perturbation, the higher the attack rate. However, extremely obvious perturbations are easily filtered out by the tracking model's defense mechanisms and are also easily detected by the human eye. In targeted attacks, the temporal coherence that target tracking relies on to control the target's trajectory after the attack places a semantic consistency requirement on the adversarial perturbation, requiring the adversarial perturbation to maintain consistent semantic features between adjacent frames. However, the trade-off between stealth and temporal coherence remains a challenging problem in the field of target tracking adversarial attacks. Summary of the Invention

[0007] The purpose of the present invention is to provide a target tracking adversarial perturbation generation method based on adversarial generative network, which solves the problem that the above-mentioned attack algorithm cannot take into account the attack effect of frame-by-frame attack and the computational efficiency of single-frame attack.

[0008] In order to achieve the above object, the technical solution adopted by the present invention is as follows: a method for generating adversarial perturbations for target tracking based on a generative adversarial network, comprising the following steps: S1, obtain a video sequence containing N consecutive frames of images , I n For the nth frame image, generate the initial template Z0 and the initial search area S0 according to the first frame image I1; S2, construct a generative adversarial network, including a perturbation generator G and a target tracker F based on a Siamese network, initialize the target tracker and freeze the weights; The perturbation generator G generates an initial perturbation δ0 of the same size as Z0 based on Z0, scales δ0 to S0, and obtains the query perturbation δ search ; S3, from Select K frames as key frames to form a key frame set G key , , KF t is the t-th key frame; S4, process key frames in sequence, KF t The processing method includes steps S41 to S45; S41, from KF via target tracker F t Crop out the search area ; S42, generated The perturbation δ t , δ t =Resize(δ search ,s t ), where s t KF tThe scaling ratio, Resize(∙) is the Resize function; S43, for δ t Smoothing process to obtain smooth disturbance ; S44, add δ0 to Z0 to generate adversarial template Z adv ,Towards Add Generate adversarial search regions ; S45, Z0 and Send it to the target tracker F to get the clean classification map H t and clean regression plot R t , Z adv and Send it to the target tracker F to get the adversarial classification map and adversarial regression plot ; S5, calculate the adversarial deception loss L adv , trajectory deviation loss L traj , disturbance constraint loss L vis , and construct the total loss L total ; , , , L total =L adv +L traj +L vis , L adv In the formula: Ω is the clean classification graph H t The area formed by the anchor points whose upper response value is higher than the response threshold τ, H t The response value of anchor point j in the middle, ReLU(∙) is the ReLU activation function, μ c is the classification map drop threshold, μ w is the prediction box width shrinkage threshold, μ h is the prediction box height shrinkage threshold, 、 are the width and height of the prediction box generated at anchor point j respectively; L traj In the formula, σ(∙) is the Sigmoid function, For adversarial classification graph artificially designated false target areas, To counter the regression graph The artificially designated false target area, b *To preset a false anchor frame, is the L1 norm; L vis In the formula, P is I n The image area is , T(p) is the pixel value of a pixel point p in P, Δp is the offset value of pixel point p, Δp=|T(p)-T'(p)|, T'(p) is the pixel value of pixel point p after adding smooth perturbation, ϵ is the preset maximum pixel change amplitude of perturbation, It means that through the saturate operation, T(p)+Δp is constrained in the interval [T(p)-ϵ,T(p)+ϵ], and λ1~λ5 are the first to fifth weights respectively; S6, repeat S3~S5, and minimize L total Update the network parameters of the disturbance generator G until the preset number of iterations is reached to obtain a shared disturbance generator; S7, obtain the video sequence to be attacked, generate an adversarial template and adversarial search area for each frame based on the shared perturbation generator, send them to the target tracker, and output the adversarial classification map and adversarial regression map.

[0009] As a preference, in S42, calculate s t The method is to obtain the size r1 of the initial search area in I1 and calculate KF t The size of the search area in r2,s t =r1 / r2.

[0010] As a preference, S43 specifically comprises: calculating KF t The disturbance difference ∆δ t , , If ∆δ t ≥0.3, then δ t Perform interpolation smoothing to obtain smooth disturbance , if ∆δ t <0.3 , The key frame KF of the t-1 frame t-1 Smooth perturbation.

[0011] Preferably, S7 is specifically: S71, video sequence to be attacked , obtain the initial template Z0 and initial search area S0 from I1; S72, the shared disturbance generator generates the initial disturbance δ0 and the query disturbance δ search , and generate the adversarial template Z of the first frame adv ; S73, read the next frame image I n , press Z with target tracker advFrom I n Crop out the search area , and calculate I n Scaling ratio s n ; S74, generated The perturbation δ n , smoothing process to obtain smooth disturbance ; S75, to Add smooth perturbations to , generate I n Adversarial search area ; S76, Z adv and Send it to the target tracker and output the adversarial classification map and adversarial regression map; S77, repeat steps S73 to S76 until the video sequence ends.

[0012] The present invention also has the following characteristics: 1. About templates, search areas, and target trackers:

[0013] (1) Template extraction: Take the first frame of the video sequence, I1, as the initial frame and extract the initial template, Z0, from I1. Z0 = Crop(I1, b1), where b1 is the ground-truth bounding box of the target in I1 and Crop is the crop operation, a basic operation in image processing and computer vision.

[0014] (2) Search area generation: The initial search area S0 is specified from I1. The purpose is first to determine the specific location and prediction box size of the target to be tracked in the search area, and secondly to generate δ search δ0 has the same size as Z0, δ search The size is the same as S0. The perturbation δ of subsequent images is t , are based on δ search Press the zoom ratio t Adjusted. Starting from I2, search area According to the formula Get, b t-1 For I t-1 The bounding box of the target in .

[0015] (3) Feature matching: The target tracker calculates the matching response through the dual-branch structure of the twin network, inputting Z0 and When you get a clean classification graph and a clean regression graph, enter Z adv and , we get the adversarial classification graph and adversarial regression graph.

[0016] (4) Result decoding: locate the target position by the maximum response, I t The bounding box of the target in is b i .

[0017] 2. Loss function design: The total loss consists of three parts: adversarial deception loss L adv , trajectory deviation loss L traj , disturbance constraint loss L vis .

[0018] (1) About the anti-deception loss L adv : It aims to systematically destroy the core decision-making mechanism of the target tracking algorithm - that is, through the dual strategy of suppressing classification confidence and amplifying regression error, the tracking model is rendered ineffective in both target positioning and recognition. This is the destroyer mechanism. The right side of the formula is divided into two parts with a plus sign as the boundary. The physical meaning of the first part is: the area that originally belonged to the target (high confidence) is forcibly suppressed to a low confidence interval, confusing the positive and negative sample discrimination ability of the tracking model, resulting in it being unable to distinguish between the target and the background, and thus losing the tracking target. The second part: the regression output of the tracking model gradually deviates from the true target, but is limited within the threshold, which ultimately leads to a complete misalignment between the predicted box and the actual position of the target.

[0019] (2) About trajectory deviation loss L traj : It is a key extension of the adversarial attack strategy. Its goal is not only to suppress the confidence of the real target in the original search area, but also to actively activate the false anchor box far away from the center of the search area. By jointly optimizing the classification and regression branches, the tracking model is forced to mistakenly lock the target at the preset offset position, which eventually causes the target to completely leave the search area and the tracking to fail completely. Therefore, the loss is divided into two parts with a plus sign as the boundary: The physical meaning of the first part is: the introduction of negative log-likelihood loss, which is different from the strategy of only reducing the confidence of the target area. This part maximizes the classification confidence of the artificially specified false target area fa, causing the tracking model to mistakenly believe that the target exists in a false target area fa far away from the true position. The essence of this design is to construct a "bait signal" to interfere with the attention allocation of the tracking model. The second part: by forcing the regression output to approximate the preset false anchor box b * , further solidifying the spatial representation of the false target, making it more inductive in position and shape. Ultimately, the tracking model is attracted by the false anchor frame and completely deviates from the real target position. * The false target area fa is in the original image I n A rectangular box with the corresponding center point in is the center and the shape (length and width) is specified artificially. Its mathematical representation is a four-dimensional vector, and the four elements are the horizontal coordinate of the center point of the preset false anchor box, the vertical coordinate of the center point, the width of the predicted box, and the height of the predicted box.

[0020] (3) About the perturbation constraint loss L vis : The formula uses dual-path constraints. First, local mutations are suppressed based on the L2 energy term; second, global amplitude control is achieved based on pixel-by-pixel truncation to ensure that the perturbation pattern is aligned with the statistical characteristics of natural images.

[0021] 3. Cross-frame perturbation propagation mechanism: To ensure that perturbations remain effective over long videos, the following propagation strategy is designed.

[0022] (1) Single generation and reuse: δ0 is generated only in the initial frame and reused directly in subsequent frames, reducing the computational complexity from O(N) to O(1), where N is the number of images in the video sequence.

[0023] (2) Dynamic size adaptation: According to the size change of the target in subsequent frames, δ search Perform bilinear interpolation scaling to generate the perturbation δ corresponding to each frame image t δ search Also obtained based on δ0 scaling.

[0024] (3) Temporal and spatial coherence guarantee: Through interpolation smoothing, the disturbance difference ∆δ of adjacent frames is t Less than 0.3, avoid time domain filtering detection.

[0025] 4. Template-Search Area Dual-Path Perturbation Injection: This method aims to jointly disrupt the template matching and search positioning processes. The Siamese network-based object tracker is divided into two branches: the template branch and the search branch.

[0026] (1) Template branch attack: The initial perturbation δ0 directly acts on the initial template Z0, polluting its deep feature extraction process and causing the template features to deviate from the true target representation.

[0027] (2) Search branch attack: According to the difference of each image, δ search Scaling and clipping are performed to obtain δ t , and then smoothing to obtain smooth disturbance ,Will Inject the search area of ​​the corresponding image to interfere with the classification and regression branches: interfere with the classification branch to suppress the confidence peak of the target candidate area; interfere with the regression branch to increase the position offset and scale error of the prediction box.

[0028] (3) Collaborative attack effect: The joint perturbation of the template and the search area forces the tracking model to fail simultaneously in feature matching and positioning decisions.

[0029] Compared with the prior art, the advantages of the present invention are: (1) This method uses an innovative cross-frame perturbation propagation mechanism to generate δ0 only in the initial frame, reducing the O(N) computational complexity of traditional frame-by-frame attack to O(1), making the attack speed meet the requirements of real-time systems (>30FPS).

[0030] (2) This paper proposes a strategy for dynamic size adaptation of disturbances. First, a δ is generated based on δ0. search , and then according to the size change of the target in the subsequent frames, search Perform bilinear interpolation scaling to generate the perturbation δ corresponding to each frame image t .

[0031] (3) Solve the concealment problem through multi-dimensional concealment constraints: In the spatial dimension, amplitude truncation and L2 texture smoothing loss are used to ensure that the perturbation is imperceptible in the visual RGB color space; in the temporal dimension, the perturbation difference ∆δ between adjacent frames is constrained. t ,make It is always kept less than 0.3, thus forcing the smooth transition of disturbances between adjacent frames and circumventing time-domain anomaly detection mechanisms such as Kalman filtering.

[0032] In summary, the present invention solves the concealment problem through an innovative cross-frame perturbation propagation mechanism, a dynamic perturbation size adaptation strategy, and multi-dimensional concealment constraints. It not only reduces computational complexity and increases attack speed, but also maintains concealment while ensuring a high attack success rate. This solves the problem of easy detection caused by traditional non-target attacks relying on computational complexity or high-amplitude perturbations. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a training process diagram of the present invention; Figure 2 Comparison images before and after the UEN method adds perturbations to an image. DETAILED DESCRIPTION

[0034] The present invention will be further described below with reference to the embodiments and accompanying drawings.

[0035] Example 1: See Figure 1 ,A target tracking adversarial perturbation generation method based on ,generative adversarial network, comprising the following steps; S1, obtain a video sequence containing N consecutive frames of images , I n For the nth frame image, generate the initial template Z0 and the initial search area S0 according to the first frame image I1; S2, construct a generative adversarial network, including a perturbation generator G and a target tracker F based on a Siamese network, initialize the target tracker and freeze the weights; The perturbation generator G generates an initial perturbation δ0 of the same size as Z0 based on Z0, scales δ0 to S0, and obtains the query perturbation δ search ; S3, from Select K frames as key frames to form a key frame set G key , , KF t is the t-th key frame; S4, process key frames in sequence, KF t The processing method includes steps S41 to S45; S41, from KF via target tracker F t Crop out the search area ; S42, generated The perturbation δ t , δ t =Resize(δ search ,s t ), where s t KF t The scaling ratio, Resize(∙) is the Resize function; S43, for δ t Smoothing process to obtain smooth disturbance ; S44, add δ0 to Z0 to generate adversarial template Z adv ,Towards Add Generate adversarial search regions ; S45, Z0 and Send it to the target tracker F to get the clean classification map H t and clean regression plot R t , Z adv and Send it to the target tracker F to get the adversarial classification map and adversarial regression plot ; S5, calculate the adversarial deception loss L adv , trajectory deviation loss L traj , disturbance constraint loss L vis , and construct the total loss L total ; , , , L total =L adv +L traj +Lvis , L adv In the formula: Ω is the clean classification graph H t The area formed by the anchor points whose upper response value is higher than the response threshold τ, H t The response value of anchor point j in the middle, ReLU(∙) is the ReLU activation function, μ c is the classification map drop threshold, μ w is the prediction box width shrinkage threshold, μ h is the prediction box height shrinkage threshold, 、 are the width and height of the prediction box generated at anchor point j respectively; L traj In the formula, σ(∙) is the Sigmoid function, For adversarial classification graph artificially designated false target areas, To counter the regression graph The artificially designated false target area, b * To preset a false anchor frame, is the L1 norm; L vis In the formula, P is I n The image area is , T(p) is the pixel value of a pixel point p in P, Δp is the offset value of pixel point p, Δp=|T(p)-T'(p)|, T'(p) is the pixel value of pixel point p after adding smooth perturbation, ϵ is the preset maximum pixel change amplitude of perturbation, It means that through the saturate operation, T(p)+Δp is constrained in the interval [T(p)-ϵ,T(p)+ϵ], and λ1~λ5 are the first to fifth weights respectively; S6, repeat S3~S5, and minimize L total Update the network parameters of the disturbance generator G until the preset number of iterations is reached to obtain a shared disturbance generator; S7, obtain the video sequence to be attacked, generate an adversarial template and adversarial search area for each frame based on the shared perturbation generator, send them to the target tracker, and output the adversarial classification map and adversarial regression map.

[0036] In this embodiment S42, s is calculated. t The method is to obtain the size r1 of the initial search area in I1 and calculate KF t The size of the search area in r2,s t =r1 / r2.

[0037] In this embodiment, S43 is specifically as follows: calculating KF t The disturbance difference ∆δ t, , If ∆δ t ≥0.3, then δ t Perform interpolation smoothing to obtain smooth disturbance , if ∆δ t <0.3 , The key frame KF of the t-1 frame t-1 Smooth disturbance.

[0038] The specific embodiment S7 is: S71, video sequence to be attacked , obtain the initial template Z0 and initial search area S0 from I1; S72, the shared disturbance generator generates the initial disturbance δ0 and the query disturbance δ search , and generate the adversarial template Z of the first frame adv ; S73, read the next frame image I n , press Z with target tracker adv From I n Crop out the search area , and calculate I n Scaling ratio s n ; S74, generated The perturbation δ n , smoothing process to obtain smooth disturbance ; S75, to Add smooth perturbations to , generate I n Adversarial search area ; S76, Z adv and Send it to the target tracker and output the adversarial classification map and adversarial regression map; S77, repeat steps S73 to S76 until the video sequence ends.

[0039] The practical application of the present invention is divided into a training phase and an attack phase:

[0040] During the training phase: Get a video sequence from the training dataset , obtain the initial template Z0 and the initial search area S0 from the first frame, then form a key frame set according to step S3, process the key frames in sequence according to S4, and calculate the total loss L according to S5 total , to minimize L total Update the network parameters of the disturbance generator G; repeat S3~S5 to obtain the shared disturbance generator.

[0041] During the attack phase: process each frame of the video sequence to be attacked in sequence. For specific methods, see S71 to S77 in step S7.

[0042] Example 2: See Figure 1 To better illustrate the technical effects of the present invention, this example conducts a comparative experiment. In this comparative experiment, the present invention's method is compared with currently popular and excellent target tracking adversarial attack algorithms, such as EUSA, UEN, FAN, CSA, One-shot, and ABA. The results of each algorithm on the VOT2018, OTB100, and LaSOT datasets are shown in Tables 1, 2, and 3. The target tracker used is SiamRPN++.

[0043] Table 1. Experimental results of various attack algorithms on VOT2018 , Table 2. Experimental results of various attack algorithms on OTB100 , Table 3. Experimental results of various attack algorithms on LaSOT (SiamRPN++) , In Tables 1 to 3, EAO (Expected Average Overlap) represents the expected average overlap. A larger value indicates a higher accuracy. Accuracy represents accuracy, Success represents success rate, and Precision represents precision.

[0044] The EUSA method is an adversarial attack algorithm based on iteratively optimized computational perturbations. It designs a triple composite loss of feature deception, confidence deception, and regression drift. This method randomly extracts videos from the dataset and uses initial perturbations for adversarial attacks. The initial perturbations are then further optimized by minimizing the triple composite loss function based on the attack results. Ultimately, the optimal universal perturbation is selected to implement an adversarial attack on the entire dataset. Although this method appears to achieve an attack computational complexity of O(1) across the entire dataset, the universal perturbation it uses is optimized over the entire dataset at a significant cost in time and computational resources, essentially shifting the computational cost to the pre-processing stage. The EUSA method achieved an 82.6% reduction in the evaluation metric on VOT2018 compared to the original model (0.414 → 0.072), but this method further reduces this metric to 0.013. This advantage is also significant on the OTB100 dataset—the success rate of this method drops by an additional 10.6 percentage points compared to the EUSA method, confirming the superiority of the dynamic generation strategy over static universal perturbations. This phenomenon may be due to two reasons: First, EUSA's universal perturbation needs to be compatible with the entire feature distribution of the dataset. While the perturbation generator used in this method cannot generate a universal perturbation for the entire dataset, it can quickly generate perturbations for each new dataset and video, generating specific perturbations for specific video content. Second, the perturbation of the EUSA method only acts on the search area, while this method causes more systematic damage to the feature matching mechanism of the tracking model through a two-way coordinated attack on the template and the search area. These comparative results show that compared with the static universal perturbation strategy, the dynamically generated specific perturbation exhibits superior adaptability in complex tracking scenarios.

[0045] The CSA method is a generative algorithm based on GANs. Similar to this method, it generates adversarial perturbations by training a perturbation generator. This method implants adversarial perturbations into the search area, causing the target to deviate from its true position. This method requires perturbation calculation and generation for each frame, making it computationally much more complex than this method. Comparative results show that this method reduces the tracking success rate on OTB100 by 56.5 percentage points, a further 21.6 percentage points lower than that of the CSA method, demonstrating that this method is significantly more effective than the CSA method on OTB100. A thorough analysis of the results suggests that the reasons for this discrepancy may be: first, the dual-path injection strategy of this method causes a more systematic disruption to the underlying mechanisms of the target tracking model; second, the loss function of the CSA method only includes classification misleading and regression misleading, while the cross-frame perturbation propagation mechanism introduced by this method ensures that the adversarial perturbation maintains motion coherence between consecutive frames, significantly enhancing the deceptiveness and stealth of the adversarial perturbation.

[0046] The UEN, FAN, and CSA methods are all generative algorithms. They all perform perturbation calculations on video frames by training a perturbation generator. They are all frame-by-frame attack methods targeting the search area, and have the same computational complexity as CSA. However, due to the different designs of the loss functions, the perturbation generators of the three methods have slightly different time consumptions for generating perturbations on a single frame. The UEN method achieved the best attack effect among all algorithms on the OTB100 dataset, but the perturbations it generated exhibited obvious grid-like artifacts, such as Figure 2 As shown in Figure 3, 14. After in-depth analysis, these visual artifacts are caused by the UEN method's overemphasis on attack strength while ignoring visual concealment constraints and failing to enforce spatiotemporal consistency constraints on the perturbations. The FAN algorithm, targeting the SiamFC tracking model, achieved a 72% performance reduction when attacking the model. However, its performance was poor when transferred to the SiamRPN++ tracking model, with a 59.1% performance reduction on the OTB100 dataset, far less than the 82% performance reduction achieved by our method. This discrepancy is attributed to the fact that the FAN method's attack strategy primarily targets the SiamFC tracking model. The perturbations generated based on this strategy primarily act on shallow features of the tracking network, while the deep architecture of SiamRPN++ makes deep features dominate the final prediction results, resulting in poor transferability for the FAN method. Furthermore, regarding training cost, both the UEN and FAN methods do not employ sparse sampling strategies, instead using full-frame training from the entire dataset. Consequently, the training time (1.2 hours) and computing resources consumed are significantly higher than our method.

[0047] One-shot, like EUSA, ABA, DIMBA, IoU-A, and UASE, is an iterative optimization algorithm. The difference is that one-shot methods attack only the template. They optimize the perturbations of the tracking model's initial frame response map using a designed dual attention loss. This allows the adversarial template to suppress the top n high-confidence candidate boxes in the response map, thereby causing the tracking model to lose its semantics of the original ground-truth target. Although CSA also has a template-only attack method, CSA-T, its performance is not considered excellent and is therefore not included in the comparative experiments in this section. In terms of attack results, this method surpasses some frame-by-frame attack algorithms, reducing the success rate to 25.2% on OTB100, surpassing the 32.0% of the FAN method. However, its single-channel attack mode is inherently limited. While adding perturbations to the template contaminates the target baseline representation and affects the similarity matching mechanism for target tracking, it cannot compare to the systematic, coordinated destruction achieved by the dual-channel injection attack strategy of our present invention. This difference stems from the complementary effect of the dual-path injection strategy: when the template attack fails due to changes in the target appearance, the continuous interference in the search area can still maintain the attack effect.

[0048] The ABA method uses motion blur as a method to counter the perturbation. Based on the motion blur and optical flow accumulation process, the adversarial perturbation is disguised as natural motion blur (OP-ABA). However, it requires multiple iterations of optimization during training, which brings greater performance overhead, so it does not have real-time attack capabilities. In further work, the original author introduced JAMANET

[59] To optimize optical flow calculation, a faster attack method, OS-ABA, was proposed. However, its attack speed on the SiamRPN++ tracking model was still only 17.62 fps, failing to meet real-time requirements. Furthermore, OS-ABA suffers a certain performance penalty compared to OP-ABA. For example, on VOT2018, OS-ABA's EAO metric was 0.025 higher than OP-ABA. Compared to this method, the motion blur promoted by the ABA method is more natural and consistent with common sense than traditional adversarial perturbations, making it superior in stealth. However, its attack effectiveness is significantly inferior to this method. On OTB100, the two ABA methods, OP-ABA and OS-ABA, achieved an average success rate metric of 23.85% higher than this method, yet neither achieved the real-time performance of this method. This result reveals a more fundamental limitation: motion blur perturbations are less adaptable to continuous tracking scenarios than conventional adversarial perturbations. Although motion blur can produce a stronger concealment effect, the motion blur calculated by the optical flow method is destined to only cause a temporary dislocation of the target tracking model and cannot cause a lasting impact. The interference information that can be encoded in motion blur is still far inferior to the generative adversarial perturbation.

[0049] Overall, this method excels in low computational overhead during training, a short attack time, and good stealth. It demonstrates excellent attack performance on mainstream target tracking datasets and high-performance tracking models. It also meets the real-time requirements of target tracking countermeasures and has minimal impact on the performance of the tracking model, making this method highly effective while further reducing the likelihood of detection.

[0050] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for generating adversarial perturbations for target tracking based on a generative adversarial network, characterized in that: The following steps are included: S1, obtain a video sequence containing N consecutive frames of images , I n For the nth frame image, generate the initial template Z0 and the initial search area S0 according to the first frame image I1; S2, construct a generative adversarial network, including a perturbation generator G and a target tracker F based on a Siamese network, initialize the target tracker and freeze the weights; The perturbation generator G generates an initial perturbation δ0 of the same size as Z0 based on Z0, scales δ0 to S0, and obtains the query perturbation δ search ; S3, from Select K frames as key frames to form a key frame set G key , , KF t is the t-th key frame; S4, process key frames in sequence, KF t The processing method includes steps S41 to S45; S41, from KF via target tracker F t Crop out the search area ; S42, generated The perturbation δ t , δ t =Resize(δ search ,s t ), where s t KF t The scaling ratio, Resize(∙) is the Resize function; S43, for δ t Smoothing process to obtain smooth disturbance ; S44, add δ0 to Z0 to generate adversarial template Z adv ,Towards Add Generate adversarial search regions ; S45, Z0 and Send it to the target tracker F to get the clean classification map H t and clean regression plot R t , Z adv and Send it to the target tracker F to get the adversarial classification map and adversarial regression plot ; S5, calculate the adversarial deception loss L adv , trajectory deviation loss L traj , disturbance constraint loss L vis , and construct the total loss L total ; ， ， ， L total =L adv +L traj +L vis , L adv In the formula: Ω is the clean classification graph H t The area formed by the anchor points whose upper response value is higher than the response threshold τ, H t The response value of anchor point j in the middle, ReLU(∙) is the ReLU activation function, μ c is the classification map drop threshold, μ w is the prediction box width shrinkage threshold, μ h is the prediction box height shrinkage threshold, 、 are the width and height of the prediction box generated at anchor point j respectively; L traj In the formula, σ(∙) is the Sigmoid function, For adversarial classification graph artificially designated false target areas, To counter the regression graph The artificially designated false target area, b * To preset a false anchor frame, is the L1 norm; L vis In the formula, P is I n The image area is , T(p) is the pixel value of a pixel point p in P, Δp is the offset value of pixel point p, Δp=|T(p)-T'(p)|, T'(p) is the pixel value of pixel point p after adding smooth perturbation, ϵ is the preset maximum pixel change amplitude of perturbation, It means that through the saturate operation, T(p)+Δp is constrained in the interval [T(p)-ϵ,T(p)+ϵ], and λ1~λ5 are the first to fifth weights respectively; S6, repeat S3~S5, and minimize L total Update the network parameters of the disturbance generator G until the preset number of iterations is reached to obtain a shared disturbance generator; S7, obtain the video sequence to be attacked, generate an adversarial template and adversarial search area for each frame based on the shared perturbation generator, send them to the target tracker, and output the adversarial classification map and adversarial regression map.

2. The method for generating adversarial disturbances for target tracking based on a generative adversarial network according to claim 1, wherein: In S42, calculate s t The method is to obtain the size r1 of the initial search area in I1 and calculate KF t The size of the search area in r2,s t =r1 / r2.

3. The method for generating adversarial disturbances for target tracking based on a generative adversarial network according to claim 1, wherein: S43 is specifically: calculating KF t The disturbance difference ∆δ t , , If ∆δ t ≥0.3, then δ t Perform interpolation smoothing to obtain smooth disturbance , if ∆δ t <0.3 , The key frame KF of the t-1 frame t-1 Smooth disturbance.

4. The method for generating adversarial disturbances for target tracking based on a generative adversarial network according to claim 1, wherein: S7 specifically: S71, video sequence to be attacked , obtain the initial template Z0 and initial search area S0 from I1; S72, the shared disturbance generator generates the initial disturbance δ0 and the query disturbance δ search , and generate the adversarial template Z of the first frame adv ; S73, read the next frame image I n , press Z with target tracker adv From I n Crop out the search area , and calculate I n Scaling ratio s n ; S74, generated The perturbation δ n , smoothing process to obtain smooth disturbance ; S75, to Add smooth perturbations to , generate I n Adversarial search area ; S76, Z adv and Send it to the target tracker and output the adversarial classification map and adversarial regression map; S77, repeat steps S73 to S76 until the video sequence ends.

Images