Target tracking attack-resisting method and device based on similar target guidance
By using a similar target-guided approach and leveraging visual language models and dynamic feature weights, this method addresses the problem that existing adversarial attack methods cannot effectively test tracker anti-interference capabilities in complex environments, thereby improving the accuracy and stability of adversarial attacks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- PENG CHENG LAB
- Filing Date
- 2026-04-16
- Publication Date
- 2026-05-15
AI Technical Summary
Existing adversarial attack methods cannot effectively test the anti-interference capabilities of trackers in target tracking, especially when the boundary between the target and the interfering object is blurred or the background is complex. The attack effect is not ideal and the attack target cannot be precisely controlled.
By using a similar target-guided approach, image features are extracted using a visual language model, cosine similarity is calculated, similar target regions are selected and dynamic feature weights are generated for adversarial attacks, including gradient attacks and interference generation models.
Effectively assessing the anti-interference capability and stability of target trackers improves the targeting and accuracy of attacks, enabling stable misdirection of trackers in complex environments.
Smart Images
Figure CN122049786A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of target tracking technology, and more specifically to a target tracking counter-attack method and apparatus based on similar target guidance. Background Technology
[0002] Object tracking is a crucial and fundamental task in the field of computer vision. Its core objective is to automatically identify, locate, and continuously track a specified target object in consecutive video frames. Due to its importance in practical applications, object tracking technology is widely used in video surveillance, autonomous driving, intelligent transportation systems, augmented reality, and robot navigation, becoming an indispensable component in building intelligent perception systems.
[0003] Existing adversarial attack methods typically attempt to mislead the tracker's prediction by randomly selecting a pseudo-target region within the target search area and applying perturbations, causing it to shift from the real target to the pseudo-target. However, this strategy based on random region selection has significant limitations: (1) the selected pseudo-target region often lacks visual similarity to the real target, and the perturbation effect is difficult to correlate sufficiently with the target region, resulting in a significant reduction in attack effectiveness; (2) due to its over-reliance on randomness, such methods cannot precisely control the attack target, making it difficult to continuously and stably mislead the tracker, thereby reducing the attack's specificity and accuracy. Existing technologies have failed to fully exploit the inherent weaknesses of trackers under visually similar object interference, especially when the boundary between the target and the interfering object is blurred or the background is complex, the tracker's recognition ability often shows significant deviations. Therefore, existing adversarial attack methods still cannot fully assess the robustness of trackers under complex adversarial conditions, and the attack effect is difficult to achieve an ideal level.
[0004] Furthermore, in actual target tracking, the appearance features of targets are often variable and unstable, especially during long-term tracking, when the target's appearance may change due to factors such as angle changes, lighting variations, or occlusion. For example, when certain parts of the target are highly similar to the background region, the tracker may mistakenly confuse the target with the background, causing the prediction result to drift to the background region, ultimately leading to tracking failure. Therefore, how to effectively simulate the interaction between target appearance instability and background interference has become a key problem that urgently needs to be solved in adversarial attack methods. Summary of the Invention
[0005] This invention provides a target tracking adversarial attack method and apparatus based on similar target guidance, in order to solve the problem that existing adversarial attack methods cannot effectively test the anti-interference capability of trackers.
[0006] In a first aspect, the present invention provides a target tracking adversarial attack method based on similar target guidance, the method comprising: Based on the text features of the target category, the tracking target is determined in the first frame of the tracking video, and the similarity between the text features of the target category and the features of each frame is calculated to determine the similar target regions of the tracking target in each frame. By calculating similarity, the least relevant text attribute features are selected from the candidate text attribute features, and dynamic weights are generated based on the irrelevant text attribute features. Adversarial attacks are launched against the tracker based on dynamic weights of similar target regions and irrelevant text attribute features of the tracked target.
[0007] The target tracking adversarial attack method based on similar target guidance provided by this invention effectively guides the tracker to predict similar target regions by identifying similar target regions and generating dynamic feature weights, thereby revealing potential vulnerabilities of existing trackers. It has strong attack capabilities in practical applications and can comprehensively evaluate the anti-interference capability and stability of target trackers.
[0008] In one optional implementation, similarity calculation is performed based on the target category text features and the features of each frame image to determine similar target regions in each frame image, including: Image features of each frame are extracted using a visual language basic model. The cosine similarity between the image features and the target category text features is calculated. The corresponding attention map is determined based on the cosine similarity. The attention map is used to characterize the correlation between features at different locations in each frame and the target category text features. Based on the attention map, threshold segmentation is used to determine highly relevant regions, and these highly relevant regions are then regionalized to obtain candidate similar regions. The correlation degree of each candidate similar region is determined based on the spatial relationship between the predicted center point of the tracking target and each candidate similar region; The candidate similar regions with the highest degree of relevance are selected as the similar target regions of the tracking target.
[0009] The target tracking adversarial attack method based on similar target guidance provided by this invention extracts features and calculates cosine similarity through a visual language model, accurately locking candidate regions that highly match the target category and appearance, avoiding the blindness of traditional random region selection. By capturing attention maps with cosine similarity, regions in the image related to target features can be intuitively highlighted, making the selection of similar regions more targeted. Combining the correlation between the target prediction center point and the candidate region selection, it further ensures that similar regions are not only similar in shape but also close in position, making the correlation between interference regions and real targets stronger and greatly improving the misleading nature of subsequent attacks.
[0010] In one optional implementation, based on the attention map, a threshold segmentation is used to determine highly relevant regions, and these highly relevant regions are then regionalized to obtain candidate similar regions, including: Based on the attention map, determine the correlation values between different regions of each frame image and the target category text features; Compare the correlation value with the correlation threshold. If the correlation value is greater than the correlation threshold, the corresponding region is a candidate similar region.
[0011] The target tracking adversarial attack method based on similar target guidance provided by this invention filters candidate similar regions by setting relevant thresholds, accurately filters low-correlation interference regions, ensures that the correlation between the selected regions and the target features meets the standard, and improves the efficiency and accuracy of similar region selection.
[0012] In one optional implementation, the candidate similar regions with the highest relevance are selected as similar target regions for the tracking target, including: Obtain the center points of each candidate similar region and calculate the Euclidean distance between each region center point and the predicted center point; The candidate similar region corresponding to the center point of the region with the smallest Euclidean distance is selected as the similar target region of the tracking target.
[0013] The target tracking adversarial attack method based on similar target guidance provided by this invention calculates the Euclidean distance between the center point of the candidate similar region and the predicted center point of the target, and uses the minimum distance as the screening criterion to ensure that the selected similar region is not only similar in shape to the target, but also highly close to the real target in position. This improves the correlation between the interference region and the target, avoids the bias of subjective judgment, and makes the selection of similar regions more accurate and quantifiable.
[0014] In one optional implementation, the least relevant text attribute features are selected from candidate text attribute features through similarity calculation, including: The similarity between the candidate text attribute features and the target template features in the tracker or the search regions of each frame image is calculated to obtain the text similarity. The text similarity is used to characterize the degree of influence of each candidate text attribute feature on the similarity. The text similarity scores are sorted from smallest to largest, and the candidate text attribute features corresponding to the smallest text similarity score are selected as irrelevant text attribute features.
[0015] The target tracking adversarial attack method based on similar target guidance provided by this invention calculates the similarity between candidate text attribute features and target template features or search region features, and uses a quantitative method to accurately filter the text features with the lowest relevance, avoiding the bias of subjective selection; the feature corresponding to the minimum value is selected according to the similarity ranking, ensuring that the selected irrelevant text features can deviate from the core attributes of the target to the greatest extent, providing a high-quality foundation for subsequent dynamic weight generation.
[0016] In one alternative implementation, dynamic weights are generated based on irrelevant text attribute features, including: Based on the image feature dimensions of the tracker, adjust the dimensions of the irrelevant text attribute features to be the same as the image feature dimensions of the tracker; The unrelated text attribute features after dimension adjustment are normalized to generate dynamic weights.
[0017] The target tracking adversarial attack method based on similar target guidance provided by this invention solves the problem of feature format incompatibility by adjusting the dimension of unrelated text attribute features to make them completely match the dimension of tracker image features, thus laying the foundation for subsequent weighting of image features; dynamic weights are generated through normalization processing to ensure that the weight values are within a uniform and reasonable range, making the intensity of feature modulation controllable and the effect stable.
[0018] In one alternative implementation, an adversarial attack is performed on the tracker based on dynamic weights of similar target regions and irrelevant text attribute features of the tracked target, including: The tracker predicts the location of the tracked target and generates a prediction score map. The gradient attack method is used to generate interference. The loss function is used to make the tracker's prediction score map gradually approach the score map of the similar target region. Alternatively, a perturbation generation model can be trained using dynamic weights of similar target regions and unrelated text attribute features, and then the trained perturbation generation model can be used to generate perturbations for attack trackers.
[0019] The target tracking adversarial attack method based on similar target guidance provided by this invention offers two schemes: gradient attack and interference generation model attack, which are suitable for different application scenarios. The gradient attack guides the predicted score map to move closer to similar regions through the loss function, and the attack is accurate and the response is fast. After the interference generation model is trained with similar regions and dynamic weights, the generated interference is highly targeted and stable.
[0020] Secondly, the present invention provides a target tracking adversarial attack device based on similar target guidance, the device comprising: The similar target region determination module is used to determine the tracking target and its corresponding target features based on the first frame image of the tracking video, and to determine the similar target regions of the tracking target in each frame image by feature comparison based on the target features; The feature dynamic weight determination module is used to filter out the least relevant text attribute features from the candidate text attribute features through similarity calculation, and generate dynamic weights based on the irrelevant text attribute features. The adversarial attack module is used to perform adversarial attacks on the tracker based on dynamic weights of similar target regions and irrelevant text attribute features of the tracked target.
[0021] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the method described in the first aspect or any corresponding embodiment thereof.
[0022] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the method described in the first aspect or any corresponding embodiment thereof. Attached Figure Description
[0023] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0024] Figure 1 This is a schematic diagram of an application scenario according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the first type of target tracking adversarial attack method based on similar target guidance according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the second process of the target tracking adversarial attack method based on similar target guidance according to an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the use of saliency to select similar targets in a target tracking adversarial attack method based on similar targets according to an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the interference generation and counter-attack process in a specific embodiment of the target tracking adversarial attack method based on similar target guidance according to an embodiment of the present invention. Figure 6 This is a structural block diagram of a target tracking counter-attack device based on similar target guidance according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation
[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0026] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.
[0027] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.
[0028] As an optional application scenario of this invention, such as Figure 1 As shown, this target tracking adversarial attack system based on similar target guidance may include at least one terminal device and at least one server. Figure 1 The system is illustrated in the example, which includes a computer 101, a mobile terminal 102, and a server 103, and the terminal devices such as the computer 101 and the mobile terminal 102 are connected to the server 103 through a network 110.
[0029] Specifically, the terminal device can be a smartphone, tablet, laptop, PDA, desktop computer, game console, smart TV, smart wearable device, in-vehicle terminal, VR (Virtual Reality) device, AR (Augmented Reality) device, etc. Server 103 can be a standalone physical server, a server cluster, a distributed system, or a cloud server providing cloud services. Network 110 can be a wired or wireless network, examples of which include, but are not limited to, the Internet, corporate intranet, local area network, wide area network, mobile communication network, and combinations thereof.
[0030] In recent years, the rapid development of deep learning has significantly promoted the advancement of target tracking technology, giving rise to a series of high-performance trackers based on deep neural networks. Among them, the two architectures represented by Convolutional Neural Networks (CNNs) and Transformers have become the mainstream research directions. CNN-based trackers typically extract local features of images and combine similarity metrics or classification mechanisms to achieve target localization and tracking. These methods achieve a good balance between computational efficiency and accuracy, and are therefore widely used in real-world scenarios. On the other hand, Transformer-based trackers, with their global modeling capabilities and powerful contextual representation capabilities, can more fully capture the long-term dependencies between the target and the background, demonstrating superior tracking performance and higher robustness on multiple authoritative benchmark datasets.
[0031] Despite significant breakthroughs in accuracy and robustness achieved by deep learning-based tracking methods, numerous challenges remain in practical applications. One core issue is that trackers are susceptible to interference in complex environments, especially when there are many similar-looking interfering objects. In such cases, models often struggle to effectively distinguish between target and non-target regions, leading to mistracking or even target loss.
[0032] To address the aforementioned issues, this invention provides a target tracking adversarial attack method based on similar target guidance. By identifying similar target regions and generating dynamic feature weights, the method effectively guides the tracker to predict similar target regions, thereby achieving the effect of effectively testing the anti-interference capability of the target tracker.
[0033] According to an embodiment of the present invention, a target tracking adversarial attack method based on similar target guidance is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.
[0034] This embodiment provides a target tracking adversarial attack method based on similar target guidance, which can be used in the aforementioned computer system. Figure 2 This is a flowchart of a target tracking adversarial attack method based on similar target guidance according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: Based on the target category text features, determine the tracking target in the first frame of the tracking video, and calculate the similarity between the target category text features and the features of each frame to determine the similar target regions of the tracking target in each frame.
[0035] Specifically, the tracking video can be a video sequence used for testing. Based on the first frame of the video sequence, the target features of the tracked object are determined. The target category text features are text descriptions (e.g., a dog or a cat), which can be manually input target features. These features are then cross-correlated with the search region to determine which regions in each frame are highly correlated with the text category feature description of the tracked object. Regions highly correlated with the target features of the tracked object are identified as similar target regions. The search region refers to the local area in each frame that the tracker delineates may contain the target.
[0036] Step S202: Through similarity calculation, the least relevant text attribute features are selected from the candidate text attribute features, and dynamic weights are generated based on the irrelevant text attribute features.
[0037] Specifically, the image features or template image features of the search area are modified, and some irrelevant text attribute features (incorrect text description features, for example, if the target feature is "a yellow dog", the irrelevant text attribute feature could be "a black dog") are used to generate weights, which are then applied to the search area features or template image features, affecting the tracker's correct identification of the tracked target.
[0038] Irrelevant text attribute features are processed into dynamic weights that can be matched with image features. Then, they are multiplied with the image features of the tracker in a weighted manner to suppress the stable features of the tracker and amplify the unstable features, so that the tracker can identify similar target areas as tracking targets.
[0039] Step S203: Based on the dynamic weights of similar target regions and irrelevant text attribute features of the tracked target, an adversarial attack is launched against the tracker.
[0040] Specifically, by optimizing the algorithm or overlaying the perturbation generated by the perturbation generator with the search area image, the feature matching process of the tracker can be misled, ultimately achieving the attack objective of making target tracking fail, in order to test the tracker's anti-interference capability.
[0041] The target tracking adversarial attack method based on similar target guidance provided in this embodiment effectively guides the tracker to predict similar target regions by identifying similar target regions and generating dynamic feature weights, thereby revealing potential vulnerabilities of existing trackers. It has strong attack capabilities in practical applications and can comprehensively evaluate the anti-interference capability and stability of target trackers.
[0042] This embodiment provides a target tracking adversarial attack method based on similar target guidance, which can be used in the aforementioned computer system. Figure 3 This is a flowchart of a target tracking adversarial attack method based on similar target guidance according to an embodiment of the present invention, such as... Figure 3As shown, the process includes the following steps: Step S301: Based on the target category text features, determine the tracking target in the first frame of the tracking video, and calculate the similarity between the target category text features and the features of each frame to determine the similar target regions of the tracking target in each frame.
[0043] Specifically, step S301 includes: Step S3011: Extract image features of each frame image using the visual language basic model, calculate the cosine similarity between the image features and the target category text features, and determine the corresponding attention map based on the cosine similarity. The attention map is used to characterize the correlation between features at different positions in each frame image and the target category text features.
[0044] Specifically, image features of each frame are extracted using a visual language foundational model (such as a multimodal neural network model (Contrastive Language-Image Pre-Training, CLIP)). Then, the cosine similarity between the image features and the target features is calculated, as shown in the following formula: (1) in, This represents the category feature vector of the target. For example, when tracking a yellow dog, the feature vector corresponding to "dog + yellow" is extracted using the CLIP model. This represents the feature vector of the k-th region in the image, such as the vector corresponding to the features of a certain region in the image. This indicates the dot product calculation, used to calculate how similar the directions of two feature vectors are (the more similar the directions, the larger the dot product). Let represent the Euclidean norm; This represents cosine similarity, with a value ranging from -1 to 1. The closer it is to 1, the more similar the features of the corresponding region are to the target features (category).
[0045] Step S3012: Based on the attention map, threshold segmentation is used to determine highly relevant regions, and the highly relevant regions are processed by regionalization to obtain candidate similar regions.
[0046] Specifically, attention maps are generated based on cosine similarity. These attention maps can represent which regions in the search area are highly relevant to the target. By using attention maps, regions that are highly similar to the target category features can be selected.
[0047] The cosine similarity between different region features and the tracked target features is compared with a cosine similarity threshold. When the cosine similarity is greater than the cosine similarity threshold, the corresponding region is considered a candidate similar region. First, the target center point predicted by the current tracker and the candidate center points of each candidate similar region are determined. The correlation can be determined by comparing the distance between each candidate center point and the predicted center point.
[0048] Step S3013: Determine the correlation degree of each candidate similar region based on the spatial relationship between the predicted center point of the tracking target and each candidate similar region.
[0049] Step S3014: Select the candidate similar regions with the highest degree of relevance as the similar target regions of the tracking target.
[0050] The target tracking adversarial attack method based on similar target guidance provided in this embodiment extracts features and calculates cosine similarity through a visual language model, accurately locking candidate regions that highly match the target category and appearance. This avoids the blindness of traditional random region selection. By capturing an attention map using cosine similarity, regions in the image related to target features can be intuitively highlighted, making the selection of similar regions more targeted. Combining the correlation between the target prediction center point and the candidate region selection further ensures that similar regions are not only similar in shape but also close in location, making the correlation between interference regions and the real target stronger and significantly improving the misleading nature of subsequent attacks.
[0051] In some optional implementations, the step S3012 above, which determines the candidate similarity region of the tracking target based on cosine similarity, includes: Step a1: Capture attention maps based on cosine similarity. Attention maps are used to characterize the correlation between features at different locations and target features in each frame of the image.
[0052] Specifically, the correlation values between different regions of each frame image and the target category text features are determined based on the attention map.
[0053] Step a2: Compare the correlation value with the correlation threshold. If the correlation value is greater than the correlation threshold, then the corresponding region is a candidate similar region.
[0054] Specifically, attention maps can highlight which areas in an image are most relevant to the category and appearance of the target. For example, if the target is a puppy, the attention map will highlight the areas of the kitten and the plush toy dog in the image. This is just an example, but not a limitation.
[0055] The target tracking adversarial attack method based on similar target guidance provided in this embodiment filters candidate similar regions by setting a relevance threshold, accurately filters low-relevance interference regions, ensures that the correlation between the selected regions and the target features meets the standard, and improves the efficiency and accuracy of similar region selection.
[0056] In some optional implementations, step S3013 above includes: Step b1: Obtain the center point of each candidate similar region and calculate the Euclidean distance between each region center point and the predicted center point.
[0057] Specifically, Euclidean distance is the straight-line distance between two points. Calculating the Euclidean distance between the center point of each region and the predicted center point is a mature existing technology, which will not be elaborated here. The correlation between the center point of each region and the predicted center point can also be calculated using the following formula: (2) in, Represents the set of candidate regions. Indicates one of the candidate similar regions. This indicates the coordinates of the tracker's current predicted center point. Indicates the coordinates of the center point of the candidate similar regions. This represents the square of the straight-line distance between two center points. This indicates that the candidate similar region with the smallest distance between two center points is identified in the candidate region set.
[0058] Step b2: Select the candidate similar region corresponding to the center point of the region with the smallest Euclidean distance as the similar target region of the tracking target.
[0059] Specifically, by calculating the distance between two center points, candidate similar regions that best match the predicted target location are identified and used as similar target regions for tracking. A vector similar to a score map is generated based on these similar target regions. The vector has values in similar regions and zero values in other regions (which can be viewed as an attention map).
[0060] like Figure 4 The diagram illustrates the selection of similar targets using saliency. The left side of the dashed line represents the random selection of pseudo-targets using conventional techniques, while the right side of the dashed line represents the pseudo-targets determined by comparison based on saliency in this embodiment (red solid box). Based on the target attention map, the closest and most similar similar target region to the tracked target (yellow box) is determined from multiple candidate similar regions (red dashed boxes). The identified pseudo-targets are more similar in features to the tracked target.
[0061] The target tracking adversarial attack method based on similar target guidance provided in this embodiment calculates the Euclidean distance between the center point of the candidate similar region and the predicted center point of the target, and uses the minimum distance as the screening criterion to ensure that the selected similar regions not only resemble the target in shape, but are also highly close to the real target in position. This improves the correlation between the interference region and the target, avoids the bias of subjective judgment, and makes the selection of similar regions more accurate and quantifiable.
[0062] Step S302: Through similarity calculation, the least relevant text attribute features are selected from the candidate text attribute features, and dynamic weights are generated based on the irrelevant text attribute features.
[0063] Specifically, step S302 includes: Step S3021: The similarity between the candidate text attribute features and the target template features in the tracker or the search region of each frame image is calculated to obtain the text similarity. The text similarity is used to characterize the degree of influence of each candidate text attribute feature on the similarity.
[0064] Specifically, during the tracking process, the stable and unstable features of the target can affect the tracker's performance. Stable features help the tracker accurately locate the target's position in each frame, while unstable features change over time or with the environment, negatively impacting the tracker. Therefore, dynamically generating image feature weights using specific text attribute features, and then applying these image feature weights to search region features or template features, reduces the tracker's dependence on stable features.
[0065] From the candidate text attribute features, select text description features with low relevance. First, use the above formula (1) to calculate the similarity between the candidate text attribute features and the target template features in the tracker or the search area of each frame image to obtain the text similarity corresponding to each candidate text attribute feature, so as to characterize the degree of influence of each candidate text attribute feature on the similarity.
[0066] Step S3022: Sort the text similarity from smallest to largest, and select the candidate text attribute feature corresponding to the smallest text similarity as the irrelevant text attribute feature.
[0067] Specifically, the following formula is used to compare the similarity of each text (scores of candidate text attribute features): (3) Wherein, represents the text similarity between the k-th candidate attribute text feature and the target template feature or the text similarity of each frame image. Candidate text attribute features that are least relevant to the appearance of the tracked target (i.e., irrelevant text attribute features) can be used to effectively guide the tracker's attention to more unstable features.
[0068] The target tracking adversarial attack method based on similar target guidance provided in this embodiment calculates the similarity between candidate text attribute features and target template features or search region features, and uses a quantitative method to accurately filter the text features with the lowest relevance, avoiding the bias of subjective selection; the feature corresponding to the minimum value is selected according to the similarity ranking, ensuring that the selected irrelevant text features can deviate from the core attributes of the target to the greatest extent, providing a high-quality foundation for subsequent dynamic weight generation.
[0069] In some optional implementations, step S302 above further includes: Step S3023: Adjust the dimensions of the irrelevant text attribute features to be the same as the dimensions of the tracker's image features, based on the tracker's image feature dimensions.
[0070] Specifically, the text attribute features and the tracker's image features (search region features or template features) are of different dimensions and formats, and cannot be directly calculated. In order to recognize the image later, the dimensions of the irrelevant text attribute features are adjusted to be the same as the dimensions of the tracker's image features.
[0071] Suppose the tracker's search region features are a tensor of dimensions [C, H, W] (C = number of channels, H = height, W = width, e.g., the common [256, 32, 32]), while the text attribute features might only be a one-dimensional vector (e.g.,
[768] ). Through dimensionality transformation operations (e.g., using fully connected layers or broadcasting mechanisms), the dimensions of the text attribute features are expanded to be completely consistent with the search region features of [C, H, W]. Irrelevant text attribute features are 1-row, 768-column vectors; these are first transformed into 1-row, 256-column vectors through a fully connected layer, and then copied into a 256-row, 32-column, 32-row tensor through broadcasting, aligning with the search feature dimensions.
[0072] Step S3024: Normalize the unrelated text attribute features after dimension adjustment to generate dynamic weights.
[0073] Specifically, after expanding the dimensions, the numerical range of text attribute features may be arbitrary (e.g., -100 to 200), while the numerical values of image features are usually between 0 and 1. To ensure fair multiplication between the two, the unrelated text attribute features after dimension adjustment are normalized to values in the range of 0 to 1 and used as dynamic weights.
[0074] Dynamic weights can be generated using transformation functions: ,in, Represents the transformation function. This indicates dynamic weights.
[0075] The dynamic weights are applied directly to the tracker's search features, and then multiplied directly: ,in, This represents the search features before weighting. This represents the weighted search features.
[0076] The target tracking adversarial attack method based on similar target guidance provided in this embodiment solves the problem of feature format incompatibility by adjusting the dimension of unrelated text attribute features to make them completely match the dimension of tracker image features, thus laying the foundation for subsequent weights to be applied to image features. Dynamic weights are generated through normalization processing to ensure that the weight values are within a uniform and reasonable range, making the intensity of feature modulation controllable and the effect stable.
[0077] Step S303: Based on the dynamic weights of similar target regions and irrelevant text attribute features of the tracked target, an adversarial attack is launched against the tracker.
[0078] Step S3031: The tracker predicts the position of the tracked target and generates a prediction score map. The gradient attack method is used to generate interference. The loss function is used to make the tracker's prediction score map gradually approach the score map of the similar target region.
[0079] Specifically, the tracker first makes predictions for each frame of the image and outputs a prediction score map. The value of each position in the prediction score map represents the probability that the tracker believes that position is the target. The position where the target is actually tracked has the highest score. The core logic of gradient perturbation is to adjust the pixels in a direction that misleads the tracker. Using the mathematical method of gradient descent, it calculates the interference signal that causes the tracker's prediction to deviate.
[0080] A loss function (such as MSE loss, perturbation constraint loss, etc.) is introduced to calculate the difference between the tracker's predicted score map and the score maps of similar target regions (previously generated maps where only similar regions have high scores). By continuously optimizing the perturbation, the difference is reduced until the tracker's predicted score map converges towards the similar target regions, ultimately treating the similar regions as real targets. This real-time computation and frame-by-frame attack approach is suitable for scenarios requiring high response speed.
[0081] Step S3032, or, using the dynamic weights of similar target regions and unrelated text attribute features to train an interference generation model, and using the trained interference generation model to generate interference, attack tracker.
[0082] Specifically, a tracking dataset (including video frames, target features, similar target regions, dynamic weights, etc.) is collected as training samples for the model. Similar target regions and dynamic weights are used as training constraints and input into the perturbation generation model (i.e., a perturbation generator, such as Unet, GAN, or other image generation models). The model learns how to generate perturbations so that the tracker treats similar regions as targets. The model parameters are iteratively optimized until it can stably generate effective perturbations. After training, when the tracker is running, the current frame image is input into the model, and the model directly generates targeted perturbations. When the perturbation is superimposed on the search region image of the current frame and then input into the tracker, the tracker is misled by the perturbation and cannot identify the real target. This "train first, attack later" approach is suitable for batch processing or scenarios requiring stable perturbation effects.
[0083] Both gradient-based and perturbation generator-based perturbations can effectively interfere with the tracker's prediction results, further revealing that the adversarial attack method provided in this embodiment can effectively evaluate the robustness of existing trackers.
[0084] The target tracking adversarial attack method based on similar target guidance provided in this embodiment offers two schemes: gradient attack and interference generation model attack, which are adapted to different application scenarios. The gradient attack guides the predicted score map to move closer to similar regions through the loss function, and the attack is accurate and the response is fast. After the interference generation model is trained with similar regions and dynamic weights, the generated interference is highly targeted and stable.
[0085] In one specific embodiment, such as Figure 5 As shown, there is a video to be processed. In a target tracking task, the first step is to determine the target. A common approach is to specify the target in the first frame (usually user-specified). Then, the target is tracked in every subsequent frame of the video (the tracker extracts the features of the target in the first frame and matches them in each subsequent frame to determine the target's location). The target tracking adversarial attack method provided in this embodiment operates on each frame, generating perturbations that are added to each frame to interfere with the tracker.
[0086] Specifically, the first step is to identify similar targets. Similarity targets are determined by inputting the target category text description and the search region image for each frame into a visual language model for similarity calculation, thus obtaining the degree of attention given to this category of text in this image (e.g., Figure 4(See the red dashed box in the middle right). This yields several candidate similar targets, which are then compared to determine the final similar target. The second step involves modulating the search region features. This step uses fine-grained attributes of the target's appearance (such as color and material) and the search region image as input into the visual language model. The least relevant attribute is selected based on similarity (e.g., if the target is yellow, the least relevant attributes might be black or red). This least relevant attribute feature is then processed, including normalization, and multiplied with the search region features. Finally, the perturbation can be generated using gradient-based perturbation or a generator.
[0087] like Figure 5 As shown, coarse-grained text category attributes reflect the semantic information of the target's category. Category attributes typically exhibit high stability during video tracking, thus they are used as the semantic prior in the first stage. Specifically, referring to the category definitions of the large-scale computer vision dataset (Microsoft Common Objects in Context, MS COCO) released by Microsoft, 80 common object categories (such as "person", "dog", "cat", "car", etc.) are selected and used as a candidate set for coarse-grained category descriptions. The CLIP model encodes the features of the target image and these category texts respectively, and selects the category terms most relevant to the target semantics through cosine similarity calculation, thereby providing clear category guidance for similar region search.
[0088] Fine-grained appearance attributes are mainly used in the search area feature constraint module to further characterize the appearance characteristics of the target. Fine-grained attributes are divided into three categories: color, material, and texture: (1) Color attributes: 11 high-frequency colors (such as red, yellow, blue, green, white, black, etc.) are selected to describe the main color distribution characteristics of the target; (2) Material attributes: referring to the visual attribute annotation standard, 9 material types (such as metal, wood, textile, glass, plastic, paper, stone, etc.) are selected to reflect the physical texture of the target surface; (3) Texture attributes: 3 common textures (such as smooth, rough, soft) are selected to supplement the description of the micro-features of the target surface.
[0089] This embodiment also provides a target tracking adversarial attack device based on similar target guidance. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.
[0090] This embodiment provides a target tracking adversarial attack device based on similar target guidance, such as... Figure 6 As shown, it includes: The similar target region determination module 601 is used to determine the tracking target and its corresponding target features based on the first frame image of the tracking video, and to determine the similar target regions of the tracking target in each frame image by feature comparison based on the target features.
[0091] The feature dynamic weight determination module 602 is used to filter out the irrelevant text attribute features with the lowest relevance from the candidate text attribute features through similarity calculation, and generate dynamic weights based on the irrelevant text attribute features.
[0092] The adversarial attack module 603 is used to perform adversarial attacks on the tracker based on dynamic weights of similar target regions and irrelevant text attribute features of the tracked target.
[0093] In some optional implementations, the similar target region determination module 601 includes: The cosine similarity calculation unit is used to extract image features from each frame of the image using the visual language basic model, and to calculate the cosine similarity between the image features and the target features.
[0094] The correlation determination unit is used to determine the candidate similar regions of the tracked target based on cosine similarity and to calculate the correlation between the predicted center point of the tracked target and each candidate similar region.
[0095] The region filtering unit is used to filter the candidate similar regions with the highest degree of relevance as similar target regions for tracking.
[0096] In some optional implementations, the feature dynamic weight determination module 602 includes: The text similarity calculation unit is used to calculate the similarity between the candidate text attribute features and the target template features in the tracker or the search region of each frame image to obtain the text similarity. The text similarity is used to characterize the degree of influence of each candidate text attribute feature on the similarity.
[0097] The attribute feature filtering unit is used to sort the text similarity from smallest to largest and select the candidate text attribute feature corresponding to the smallest text similarity as the irrelevant text attribute feature.
[0098] The dimension adjustment unit is used to adjust the dimensions of irrelevant text attribute features to be the same as the dimensions of the tracker's image features, based on the tracker's image feature dimensions.
[0099] The weight generation unit is used to normalize the unrelated text attribute features after dimension adjustment and generate dynamic weights.
[0100] In some alternative implementations, the anti-attack module 603 includes: The gradient attack unit is used to predict the location of the tracked target using the tracker, generate a predicted score map, generate interference using the gradient attack method, and use a loss function to make the tracker's predicted score map gradually approach the score map of the similar target region.
[0101] The generator attack unit is used to train a perturbation generation model using dynamic weights of similar target regions and unrelated text attribute features, and to generate perturbation using the trained perturbation generation model, as well as an attack tracker.
[0102] The target tracking adversarial attack device based on similar target guidance provided in this embodiment of the invention can execute the target tracking adversarial attack method based on similar target guidance provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method. Further functional descriptions of the above modules and units are the same as in the corresponding embodiments described above, and will not be repeated here.
[0103] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.
[0104] The following is a detailed reference. Figure 7 This diagram illustrates a suitable structural schematic for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 701, which can perform various appropriate actions and processes based on a program stored in a read-only memory (ROM) 702 or a program loaded from memory 708 into random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device. The processor 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.
[0105] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 7 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.
[0106] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 709, or installed from a memory 708, or installed from a ROM 702. When the computer program is executed by the processor 701, it performs the functions defined in the target tracking adversarial attack method based on similar target guidance according to embodiments of the present invention.
[0107] Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.
[0108] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the target tracking adversarial attack method based on similar target guidance shown in the above embodiments is implemented.
[0109] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.
Claims
1. A target tracking adversarial attack method based on similar target guidance, characterized in that, The method includes: Based on the text features of the target category, the tracking target is determined in the first frame of the tracking video, and the similarity between the text features of the target category and the features of each frame is calculated to determine the similar target regions of the tracking target in each frame. By calculating similarity, the least relevant text attribute features are selected from the candidate text attribute features, and dynamic weights are generated based on the unrelated text attribute features. Adversarial attacks are launched against the tracker based on the dynamic weights of similar target regions and irrelevant text attribute features of the tracked target.
2. The method according to claim 1, characterized in that, Similarity is calculated based on the text features of the target category and the image features of each frame to determine similar target regions in each frame, including: Image features of each frame are extracted using a visual language basic model. Cosine similarity between the image features and the target category text features is calculated. The corresponding attention map is determined based on the cosine similarity. The attention map is used to characterize the correlation between features at different locations in each frame and the target category text features. Based on the attention map, a threshold segmentation method is used to determine highly relevant regions, and these highly relevant regions are then processed to obtain candidate similar regions. The correlation degree of each candidate similar region is determined based on the spatial relationship between the predicted center point of the tracking target and each candidate similar region; The candidate similar regions with the highest degree of relevance are selected as the similar target regions of the tracking target.
3. The method according to claim 2, characterized in that, Based on the attention map, threshold segmentation is used to determine highly relevant regions, and these highly relevant regions are then regionalized to obtain candidate similar regions, including: Based on the attention map, determine the correlation values between different regions of each frame image and the target category text features; The correlation value is compared with the correlation threshold. If the correlation value is greater than the correlation threshold, the corresponding region is a candidate similar region.
4. The method according to claim 2, characterized in that, The candidate similar regions with the highest relevance are selected as the similar target regions for tracking, including: Obtain the center points of each candidate similar region, and calculate the Euclidean distance between each region center point and the predicted center point; The candidate similar region corresponding to the center point of the region with the smallest Euclidean distance is selected as the similar target region of the tracking target.
5. The method according to claim 1, characterized in that, By calculating similarity, the least relevant text attribute features are selected from the candidate text attribute features, including: The similarity between the candidate text attribute features and the target template features in the tracker or the search regions of each frame image is calculated to obtain the text similarity. The text similarity is used to characterize the degree of influence of each candidate text attribute feature on the similarity. The text similarities are sorted from smallest to largest, and the candidate text attribute features corresponding to the smallest text similarity are selected as irrelevant text attribute features.
6. The method according to claim 1, characterized in that, Generate dynamic weights based on the irrelevant text attribute features, including: Based on the image feature dimensions of the tracker, the dimensions of the irrelevant text attribute features are adjusted to be the same as the image feature dimensions of the tracker; The unrelated text attribute features after dimension adjustment are normalized to generate dynamic weights.
7. The method according to claim 1, characterized in that, Based on the dynamic weights of similar target regions and irrelevant text attribute features of the tracked target, an adversarial attack is launched against the tracker, including: The tracker predicts the position of the tracked target and generates a prediction score map. The gradient attack method is used to generate interference. The loss function is used to make the tracker's prediction score map gradually approach the score map of the similar target region. Alternatively, a perturbation generation model can be trained using dynamic weights of similar target regions and unrelated text attribute features, and then the trained perturbation generation model can be used to generate perturbation to attack the tracker.
8. A target tracking adversarial attack device based on similar target guidance, characterized in that, The device includes: The similar target region determination module is used to determine the tracking target and its corresponding target features based on the first frame image of the tracking video, and to determine the similar target regions of the tracking target in each frame image based on the target features through feature comparison; The feature dynamic weight determination module is used to filter out the least relevant text attribute features from candidate text attribute features through similarity calculation, and generate dynamic weights based on the unrelated text attribute features. The adversarial attack module is used to perform adversarial attacks on the tracker based on the dynamic weights of similar target regions and irrelevant text attribute features of the tracked target.
9. An electronic device, characterized in that, include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the method of any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the method of any one of claims 1 to 7.