A traffic scene-oriented differential denoising interactive target tracking method and system

Abstract

This invention relates to a differential denoising interactive target tracking method and system for traffic scenarios, comprising: acquiring traffic video frames; preprocessing the current frame (search frame) and template frame to obtain a search map and a template map; inputting a trained visual tracking model, extracting multi-level template feature maps, search region feature maps, and template global category tokens through a DINOv3 backbone network with frozen parameters; modulating the search region features using the template category tokens through a heatmap modulation module to enhance target response; fusing template features and enhanced search features through a multi-level feature fusion module to construct a hierarchical feature fusion sequence; performing differential attention processing on the feature sequence through a differential denoising interactive module to obtain interactive search features; and finally, decoding the target bounding box of the current frame using a localizer. This method can effectively improve the robustness and accuracy of target tracking in complex traffic scenarios.

Description

Technical Field

[0001] This invention belongs to the field of visual target tracking technology, specifically relating to a differential denoising interactive target tracking method and system for traffic scenarios. Background Technology

[0002] Visual target tracking is one of the core technologies in fields such as intelligent traffic monitoring and autonomous driving. Its task is to continuously locate the target in subsequent frames after it has been positioned in the initial frame of a video. In recent years, with the development of deep learning, tracking methods based on Siamese networks and Transformer architectures have made significant progress and have demonstrated good performance on a variety of general datasets.

[0003] However, target tracking in complex traffic scenarios still faces numerous challenges. Traffic environments are characterized by cluttered backgrounds, numerous interfering objects (such as similar vehicles, pedestrians, and traffic signs), drastic changes in lighting, and frequent target occlusion, all of which can easily cause tracker drift or even target loss. Existing methods often struggle to effectively distinguish the target from similar background interference when dealing with such complex scenarios, and their adaptability to changes in target appearance is limited. Furthermore, existing tracking methods based on large-scale pre-trained models typically employ full-parameter fine-tuning strategies, which are not only costly to train but may also reduce the model's generalization ability due to overfitting.

[0004] Therefore, improving the reliability of target tracking in complex traffic scenarios is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] To address the aforementioned problems in the existing technology, this invention provides a differential denoising interactive target tracking method and system for traffic scenarios.

[0006] This invention provides a differential denoising interactive target tracking method for traffic scenarios, comprising:

[0007] Collect video frame sequences in traffic scenarios;

[0008] The current video frame obtained from the video frame sequence is used as the search frame, and data preprocessing is performed on the predetermined template frame and the search frame respectively to obtain the preprocessed template image and the preprocessed search area image.

[0009] The preprocessed template image and the preprocessed search region image are input into the trained visual tracking model, and the following operations are performed:

[0010] Using the DINOv3 backbone network with frozen parameters, multi-level feature extraction is performed on the preprocessed search region image and the preprocessed template image to obtain multi-level template feature maps, multi-level search region feature maps, and multi-level template global category tokens.

[0011] The multi-level feature fusion module is used to perform channel alignment processing on the multi-level template feature map, the multi-level search region feature map, and the multi-level template global category token;

[0012] In the heatmap modulation module, the multi-level template global category token after channel alignment is used to perform position-by-position modulation on the multi-level search region feature map after channel alignment to generate a spatial response map; the spatial response map is used to enhance the search region features of the corresponding level in the multi-level search region feature map after channel alignment to obtain an enhanced multi-level search region feature map.

[0013] The multi-level feature fusion module fuses the channel-aligned multi-level template feature map and the enhanced multi-level search region feature map to construct a hierarchical feature fusion sequence.

[0014] The hierarchical feature fusion sequence is subjected to differential attention processing through the differential denoising interaction module to obtain the search region features after interaction.

[0015] The positioning head prediction module decodes the features of the search region after the interaction, calculates the target bounding box coordinates in the current video frame, and completes the target detection of the current video frame.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0017] To address the low reliability of existing visual target tracking methods in complex traffic scenarios, this invention provides a differential denoising interactive target tracking method and system for traffic scenarios. This method employs a DINOv3 backbone network with frozen parameters for multi-level feature extraction, preserving the strong generalization ability of the pre-trained model while avoiding the high training cost and overfitting risk associated with full parameter fine-tuning. A heatmap modulation module utilizes global category tokens from the template to modulate the search region features positionally, explicitly enhancing the target region response and suppressing background interference before deep interaction between the template and the search region, effectively solving the tracking drift problem caused by cluttered backgrounds and similar interfering objects. A differential denoising interaction module performs differential attention processing on the hierarchical feature fusion sequence, eliminating common-mode attention noise and achieving high signal-to-noise ratio interaction between the template and the search region, significantly improving the model's anti-interference ability in complex scenarios such as occlusion and lighting changes. Simultaneously, a multi-level feature fusion module integrates semantic information at different scales, enhancing adaptability to changes in target scale. Through the synergistic effect of the above modules, the present invention can effectively cope with various challenges in complex traffic scenarios, significantly improve the robustness and accuracy of target tracking, and thus effectively solve the technical problem of low reliability of target tracking in the prior art. Attached Figure Description

[0018] Figure 1 This is a flowchart of a differential denoising interactive target tracking method for traffic scenarios provided in an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram illustrating the working principle of the multi-level feature fusion module provided in this embodiment of the invention.

[0020] Figure 3 This is a schematic diagram of the data processing process of the heatmap modulation module provided in an embodiment of the present invention;

[0021] Figure 4 This is a schematic diagram of the data processing process of the differential denoising interactive module provided in an embodiment of the present invention;

[0022] Figure 5 This is an example diagram illustrating the training process of the visual tracking model provided in an embodiment of the present invention. Detailed Implementation

[0023] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0024] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0025] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0026] The present invention will now be described in detail with reference to the accompanying drawings, presenting a differential denoising interactive target tracking method and system for traffic scenarios.

[0027] Figure 1 This is a flowchart of a differential denoising interactive target tracking method for traffic scenarios provided in an embodiment of the present invention. Figure 1 As shown, the method includes:

[0028] S1: Collect video frame sequences in traffic scenarios.

[0029] For example, in complex road conditions such as frequent vehicle obstruction and road surface shadow interference, camera equipment installed at intersections, road sections, or mobile vehicle platforms can be used to collect raw video streams containing traffic participants such as target vehicles and pedestrians in real time or offline. The raw video stream is then decoded into a continuous sequence of video frames and stored in chronological order.

[0030] S2: The current video frame obtained from the video frame sequence is used as the search frame, and data preprocessing is performed on the predetermined template frame and the search frame respectively to obtain the preprocessed template image and the preprocessed search area image.

[0031] It should be noted that the pre-determined template frame refers to a frame that is pre-determined by annotation before tracking the current video frame is performed, or a historical frame that precedes the current video frame selected from the video frame sequence according to the update strategy.

[0032] For example, the template frame includes an initial template frame and a dynamically updated template frame. The initial template frame can be obtained by manual annotation or by an initial detector in the first frame of the video sequence. During continuous tracking, the target's appearance may change significantly due to pose variations, scale changes, illumination changes, or partial occlusion. If the initial template frame is always used, the appearance of the template may differ too much from the current target, leading to tracking drift. Therefore, a historical frame can be selected according to an update strategy to replace the original template frame, achieving dynamic updating of the template frame.

[0033] Based on this, S2 specifically includes S2.1-S2.5:

[0034] S2.1: Obtain the coordinates of the target center point of the pre-determined template frame.

[0035] For example, if the template frame is an initial template frame, its target center point coordinates are directly derived from the initial annotation information of the first frame (including center point coordinates and detection box size). The initial annotation information can be manually annotated real bounding boxes or detection boxes output by the object detection algorithm. If the template frame is a frame selected from historical frames according to an update strategy, its target center point coordinates are derived from the predicted bounding box coordinates saved after target localization of that historical frame in the past.

[0036] S2.2: Using the coordinates of the target center point of the predetermined template frame as a reference, crop out the image containing the target from the predetermined template frame and use it as the template image.

[0037] It should be noted that most Transformer-based vision models (such as DINOv3) require a square image as input (e.g., 224×224, 256×256) to divide the image into a fixed number of patches. Therefore, the image obtained by dynamic cropping is a square image.

[0038] S2.3: Determine the coordinates of the predicted target center point of the search frame based on the prediction result of the previous frame in the video frame sequence.

[0039] For example, suppose the search frame is the first frame in the video frame sequence. Frame, first obtain the first The target bounding box coordinates obtained from frame prediction are usually expressed as ;in It is the first The coordinates of the target's center point in the frame. It is the first The width and height of the target bounding box in the frame. If the tracking system outputs the coordinates of the top-left and bottom-right corners of the bounding box, these must first be converted to center point coordinates. The target center point from the previous frame... Directly as the first The predicted target center point of the frame. This point will be used to crop the search region image in the current frame. This approach is based on the assumption that the target's motion range between adjacent frames is limited, and that the center point of the previous frame can well cover the possible position of the target in the current frame.

[0040] It should be noted that if there is no prediction result from the previous frame, the target location can be determined by manual annotation or by the target detector, or by expanding the search range, re-detection, and motion prediction.

[0041] S2.4: Using the coordinates of the predicted target center point of the search frame as a reference, crop out the search region image containing the candidate region from the search frame.

[0042] It should be understood that the size of the search area image and the size of the template image can be the same or different.

[0043] S2.5: Perform size normalization, image enhancement, and pixel value normalization on the template image and the search region image respectively to obtain the preprocessed template image and the preprocessed search region image.

[0044] Specifically, the cropped template image and the search region image are scaled to a fixed input size preset by the model to ensure that the subsequent feature extraction module can receive input of a uniform size and maintain the consistency of image block division. Then, the image pixel values ​​are mapped from the original range [0,255] to a preset numerical distribution (e.g., divided by 255 to map to [0,1], and then standardized using the mean and standard deviation of the ImageNet dataset) to accelerate model convergence, improve numerical stability, and ensure that the input data distribution is consistent with the expectations of the pre-trained model.

[0045] It should be noted that data preprocessing operations differ slightly between the model training and inference phases. During model training, data preprocessing may include methods such as random flipping, brightness adjustment, and contrast variation to increase data diversity and improve the model's generalization ability to changes in lighting and viewing angle. During inference, only necessary standardization is typically performed without random image augmentation to ensure the determinism of the output results.

[0046] The preprocessed template image and the preprocessed search region image will undergo data processing in S3. S3, as the core part of this invention, involves the actual inference of the visual tracking model. The visual tracking model is based on the DINOv3 backbone network with frozen parameters, and combines a multi-level feature fusion module, a category token-guided heatmap modulation module, a differential denoising interactive module, and a localization head prediction module to form an end-to-end differential denoising interactive visual tracking network.

[0047] The following text will first introduce the working process of the trained visual tracking model (i.e., the actual inference stage), and then introduce the training process of the visual tracking model.

[0048] S3: Input the preprocessed template image and the preprocessed search region image into the trained visual tracking model, and perform the following operations (including S3.1-S3.6):

[0049] S3.1: Using the DINOv3 backbone network with frozen parameters, multi-level feature extraction is performed on the preprocessed search region image and the preprocessed template image to obtain multi-level template feature maps, multi-level search region feature maps, and multi-level template global category tokens.

[0050] It should be noted that the parameters of the DINOv3 backbone network are frozen during both the model training and actual inference phases. This is because: as a large-scale pre-trained visual Transformer model, the DINOv3 backbone network typically has millions to hundreds of millions of parameters. Fine-tuning all parameters during training would significantly increase training time, consume substantial hardware resources, and reduce training efficiency. Secondly, freezing parameters fully preserves the general visual representations learned by the DINOv3 backbone network during large-scale pre-training, avoiding overfitting and decreased generalization ability caused by fine-tuning on limited tracking data. Furthermore, freezing parameters during inference ensures the determinism and real-time performance of the model output, facilitating practical deployment. Finally, the heatmap modulation module and differential denoising interaction module in this invention are designed based on the premise of "fixed DINOv3 features." Through explicit semantic guidance and dual attention differential, accurate target matching and noise suppression are achieved while preserving pre-training knowledge, resulting in highly robust tracking performance in complex traffic scenarios.

[0051] Here, the DINOv3 backbone network for this freeze parameter is configured to perform the following operations:

[0052] (1) Input the preprocessed search region image and the preprocessed template image into the DINOv3 backbone network with frozen parameters, respectively. Perform forward propagation through the DINOv3 backbone network with frozen parameters, and extract the final... Layer output, The preset positive integer;

[0053] (2) From the end In the layer output, obtain the template block token sequence and template global category token corresponding to the preprocessed template image for each layer, as well as the search region block token sequence corresponding to the preprocessed search region image;

[0054] (3) The template block token sequence of each layer is used as the template feature map of the corresponding layer, and the multi-level template feature map is obtained by summarizing them.

[0055] (4) Summarize the template global category tokens of each layer to form a multi-level template global category token;

[0056] (5) The sequence of search region block tokens in each layer is used as the search region feature map of the corresponding layer, and the multi-level search region feature map is obtained by summarizing them.

[0057] It's important to note that in the DINOv3 backbone network, the template global category token, a unique [CLS] token in the Transformer architecture, interacts with all image block tokens through a self-attention mechanism throughout the forward propagation process, aggregating the global contextual information of the entire template image. After iterative encoding through multiple layers of Transformer blocks, this template global category token not only encodes the appearance features of the target but, more importantly, extracts the target's high-level semantic information, including target category attributes, overall structural features, and scene contextual relationships. This semantic information possesses stronger abstraction and invariance compared to lower-level texture, edge, and other detailed features, maintaining relative stability even when the target undergoes deformation, partial occlusion, or changes in illumination.

[0058] It should be noted that the reason for extracting the last... The reason for using layer output is that deep features possess richer semantic information and stronger robustness to target deformation and occlusion, enabling them to more accurately represent the target's category attributes and high-level semantics, while preserving a certain degree of spatial detail to support precise localization. This design leverages the global modeling advantages of Transformer deep features while avoiding redundant computations caused by full features.

[0059] S3.2: Using the multi-level feature fusion module, channel alignment processing is performed on the multi-level template feature map, the multi-level search region feature map, and the multi-level template global category token.

[0060] Here, because the features output from different layers of the DINOv3 backbone network have different channel dimensions, directly concatenating or interacting features of different dimensions would lead to dimension mismatch, making subsequent matrix operations and feature fusion impossible. After processing the multi-level template feature map, multi-level search region feature map, and multi-level template global category token, these are input into the multi-level feature fusion module for channel alignment.

[0061] Here, channel alignment is achieved through a shared projection layer. The shared projection layer uses a 1×1 convolution operation to reduce the channel dimension of each level feature in the multi-level template feature map, the multi-level search region feature map, and the multi-level template global category token from the original dimension to the preset dimension, thereby obtaining the channel-aligned multi-level search region feature map, the channel-aligned multi-level template feature map, and the channel-aligned multi-level template global category token, respectively.

[0062] Figure 2 This is a schematic diagram illustrating the working principle of the multi-level feature fusion module provided in an embodiment of the present invention. Figure 2 As shown, The feature size is set to 12. By using a shared projection layer (1×1 convolution), features from all layers are compressed to the same preset dimension. This approach eliminates dimensional differences between layers, reduces computational complexity, and preserves core semantic information.

[0063] It should be understood that although the channel dimension of each feature layer is compressed, the number of feature layers remains unchanged, i.e., the final output of the original DINOv3 backbone network remains the same. Layer features are preserved after channel alignment The structure is layered, and the spatial dimensions (number of tokens) and the order between layers remain unchanged.

[0064] S3.3: In the heatmap modulation module, the multi-level template global category token after channel alignment is used to perform position-by-position modulation on the multi-level search region feature map after channel alignment to generate a spatial response map; the spatial response map is used to enhance the search region features of the corresponding level in the multi-level search region feature map after channel alignment to obtain the enhanced multi-level search region feature map.

[0065] Figure 3 This is a schematic diagram of the data processing process of the heatmap modulation module provided in an embodiment of the present invention. Figure 3 As shown, the heatmap modulation module is configured to perform the following operations:

[0066] For each level , ,in Total number of levels extracted:

[0067] The first The hierarchical channel-aligned template global category token serves as a semantic prior prototype, and is related to the first... After channel alignment at each level, a dot product operation is performed on each local feature in the feature map of the search region to obtain the first... The original similarity score corresponding to the level; introducing a preset temperature hyperparameter, for the first level The original similarity scores corresponding to each level are scaled to obtain the scaled version of the first level. Original similarity scores for the groups; the scaled scores are then calculated using the SoftMax function. The original similarity scores of the groups are normalized to generate the first group. The spatial response map corresponding to the level; using a residual enhancement mechanism, the first level... The spatial response diagram corresponding to the level and the first After aligning the channels at each level, the feature map of the search region is subjected to element-wise multiplication and addition operations to obtain the first... Enhanced search region feature maps corresponding to each level; Summary The enhanced search region feature maps at each level are used as the enhanced multi-level search region feature maps.

[0068] It should be noted that the heatmap modulation module in this invention is based on category token guidance. Specifically, it utilizes the global category token of the template output from the DINOv3 backbone network as a priori prototype rich in high-level semantic information of the target. This token interacts point-by-point (dot product operation) with each local location in the feature map of the search region, thereby generating a spatial response map reflecting the probability of the target appearing within the search region. This spatial response map is applied to the original search region features through a residual mechanism, achieving explicit semantic enhancement of the target candidate region and effective suppression of irrelevant background. This "semantic guidance first, feature enhancement later" design allows the model to obtain spatial priors about the target's location before performing deep template-search region interaction, significantly improving the input signal-to-noise ratio and overall tracking robustness of the subsequent differential denoising interaction module.

[0069] It should be understood that the first Each local feature in the channel-aligned search region feature map of the hierarchy can be understood as the feature vector corresponding to each image patch in the original search region image in the feature map of that hierarchy. For example, if the... The spatial size of the search region feature map after hierarchical channel alignment is 16×16 (i.e., 256 image blocks), and each local feature refers to any one of the 256 D-dimensional feature vectors.

[0070] In one possible implementation, the specific calculation process in the heatmap modulation module is as follows:

[0071] ;

[0072] ;

[0073] ;

[0074] ;

[0075] in, It is the first Hierarchical spatial response diagram It is the first The feature set in the feature map of the search region after channel alignment at different levels. It is the first A collection of template global category tokens after channel alignment at different levels. It is a preset temperature over-parameter. It is the first The enhanced search region feature map corresponding to the level, It is a shared projection layer. It is the first Hierarchical search region feature map, It is the first A collection of global category tokens for hierarchical templates. This refers to the transpose operation. It is an element-wise product.

[0076] After being processed by the heatmap modulation module, the data is then processed again by the multi-level feature fusion module.

[0077] S3.4: Through the multi-level feature fusion module, the channel-aligned multi-level template feature map and the enhanced multi-level search region feature map are fused to construct a hierarchical feature fusion sequence.

[0078] Specifically, S3 includes: flattening the channel-aligned multi-level template feature map along the token length dimension and serializing and splicing it according to the hierarchical index order to obtain the template feature sequence; flattening the enhanced multi-level search region feature map along the token length dimension and serializing and splicing it according to the hierarchical index order to obtain the search region feature sequence; and splicing the template feature sequence and the search region feature sequence along the token dimension to obtain the hierarchical feature fusion sequence.

[0079] In one possible implementation, the computational expression for the hierarchical feature fusion sequence is:

[0080] ;

[0081] ;

[0082] ;

[0083] in, It is a hierarchical feature fusion sequence. It is a template feature sequence. It is a feature sequence of the search region. It's a concatenation function. It is a flattening operation. It is the first Template feature map after channel alignment at the hierarchical level.

[0084] After constructing the hierarchical feature fusion sequence, the sequence fully contains the multi-scale semantic information of the template and the spatially enhanced search region features. To achieve a higher signal-to-noise ratio deep interaction between the template and the search region, and to effectively suppress common-mode attention noise caused by complex backgrounds in traffic scenes, the above fusion sequence needs to be input into the differential denoising interaction module for further processing.

[0085] S3.5: Through the differential denoising interaction module, differential attention processing is performed on the hierarchical feature fusion sequence to obtain the search region features after interaction.

[0086] Figure 4 This is a schematic diagram of the data processing process of the differential denoising interactive module provided in an embodiment of the present invention. Figure 4 As shown, this differential denoising interaction module is configured to perform the following operations:

[0087] The hierarchical feature fusion sequence is projected onto the first set of query matrices, the first set of key matrices, the second set of query matrices, the second set of key matrices, and the shared value matrix, respectively. A first attention weight distribution is calculated using the first set of query matrices and the first set of key matrices. This first attention weight distribution contains mixed information of the target matching signal and background interference. A second attention weight distribution is calculated using the second set of query matrices and the second set of key matrices. This second attention weight distribution is used to capture common-mode attention noise between the template feature part and the search region feature part in the hierarchical feature fusion sequence. A learnable noise cancellation coefficient is introduced to weight the second attention weight distribution, suppressing common-mode attention noise, resulting in a noise-suppressed second attention weight distribution. The first attention weight distribution and the noise-suppressed second attention weight distribution are subtracted positionally to obtain the net attention weight distribution. The net attention weight distribution is used as weight coefficients to weight and aggregate the value vectors in the shared value matrix, resulting in a weighted feature sequence. Based on the weighted feature sequence and the hierarchical feature fusion sequence, an interactive feature sequence is obtained. The feature sequence corresponding to the search region is extracted from the interactive feature sequence and used as the interactive search region feature.

[0088] Here, the hierarchical feature fusion sequence is normalized in the input layer to obtain a normalized feature sequence; the normalized feature sequence is then input into three independent linear projection layers. Figure 4 The linear layers Q, K, and V in the dataset are used to obtain query projection, key projection, and value projection. The query projection and key projection are then split into two branches using an attention channel splitting operation:

[0089] (1) First branch: The first set of query matrices after splitting and the first set of key matrices The input is fed into the attention calculation unit, where the first attention weight distribution is calculated sequentially through multiplication, scaling dot product, and the SoftMax function.

[0090] (2) The second set of query matrices after splitting Second set of bond matrices The input is fed into the attention calculation unit, where the second attention weight distribution is calculated sequentially through multiplication, scaling dot product, and the Softmax function.

[0091] Here, the net attention weight distribution ( Figure 4 The differential attention map in the matrix is ​​used as the weight coefficient to multiply the value vectors in the shared value matrix (obtained by projecting the normalized feature sequence through the linear layer V) to obtain the weighted feature sequence. The weighted feature sequence is then passed through root mean square normalization, the output linear layer, and the first random dropout layer in sequence, and then added to the hierarchical feature fusion sequence through the residual connection. After passing through the layer normalization, the feedforward network, and the second random dropout layer in sequence, the interactive feature sequence is obtained.

[0092] In one possible implementation, the mathematical calculation expressions in the differential denoising interaction module include:

[0093] ;

[0094] ;

[0095] ;

[0096] in, It is a hierarchical feature fusion sequence The corresponding net attention weight distribution, The full English name is Differential Attention map. It is a scaling factor. It is the noise cancellation coefficient. , , and All are learnable parameters. These are the initialization parameters for the noise cancellation coefficient. It is the deep index of the differential denoising interaction module.

[0097] After processing by the differential denoising interaction module, the resulting interactive search region features have effectively suppressed common-mode attention noise and enhanced the target response, exhibiting a high signal-to-noise ratio (SNR) representation capability. To transform these features into the specific spatial location of the target in the current frame, further decoding processing is required through the localization head prediction module.

[0098] S3.6: The positioning head prediction module decodes the features of the search area after interaction, calculates the target bounding box coordinates in the current video frame, and completes the target detection of the current video frame.

[0099] Here, the positioning head prediction module is configured to perform the following operations:

[0100] Feature reshaping restores the interactive search region features into a two-dimensional feature map with spatial topology. A convolutional neural network, consisting of multiple stacked convolutional layers, batch normalization layers, and activation functions, is used to process the two-dimensional feature map to enhance the discriminative power between the target and the background, resulting in an enhanced feature map. Three independent parallel decoding branches are used to predict the enhanced feature map, generating a target center point probability distribution map, a target bounding box size map, and a local position offset map. The spatial position with the highest response value in the target center point probability distribution map is obtained as the target center point coordinates. Based on the target center point coordinates, the width and height values ​​corresponding to the target center point coordinates in the target bounding box size map, and the offset value corresponding to the target center point coordinates in the local position offset map, geometric transformations are used to calculate the target bounding box coordinates in the current video frame.

[0101] It should be noted that the feature reshaping operation can restore the interactive search region features from a one-dimensional sequence to a two-dimensional feature map with spatial topological structure, recovering their spatial arrangement in the original search region image, so as to facilitate subsequent pixel-by-pixel dense prediction. Furthermore, the three independent parallel decoding branches can include: a center point prediction branch, a size regression branch, and an offset regression branch.

[0102] The training process of the visual tracking model will now be described. Figure 5 This is an example diagram illustrating the training process of the visual tracking model provided in an embodiment of the present invention. Figure 5 This section describes a specific model training process. For example... Figure 5 As shown, this visual tracking model was trained in the following way:

[0103] In the current training process, perform the following steps S110-S190:

[0104] S110: Obtain the tracking model updated after the last training session and use it as the current model. The current model includes the DINOv3 backbone network with frozen parameters, as well as a trainable heatmap modulation module, a multi-level feature fusion module, a differential denoising interaction module, and a localization head prediction module.

[0105] S120: Randomly select template sample frames from the training tracking video sequence according to a preset time interval relationship. Figure 5 B in the middle) and search sample frames ( Figure 5 (A) in the middle.

[0106] S130: Perform data preprocessing on the template sample frame and the search sample frame to construct training sample pairs. The training sample pairs include the template sample image, the search sample image, and the target bounding box annotation information in the search sample image.

[0107] Here, it should be noted that searching for target bounding box annotation information in the sample image is for the following reasons: 1) The output of the localization head prediction module includes the target center point probability distribution map, bounding box size map, and local position offset map. These outputs need to be compared with the real annotation information in order to calculate the loss and guide the model parameter update; 2) Calculate the total loss function; only with real annotation information can these loss functions calculate meaningful error values; 3) The annotation information acts as a bridge, enabling the model to understand which feature patterns correspond to the target center and which correspond to the bounding box boundaries, thereby establishing the mapping ability from the feature space to the image space.

[0108] S140: Input the template sample image and the search sample image from the sample pair into the current model for forward propagation to obtain the coordinates of the current predicted target bounding box. Figure 5 The yellow box in the middle represents the final tracking result.

[0109] It should be understood that the processing procedures in each module of the model training phase are the same as those in the actual inference phase. For the sake of brevity, they will not be elaborated here.

[0110] S150: Based on the target bounding box coordinates of the current predicted target bounding box and the target bounding box annotation information in the search sample image of the sample pair, calculate the focus loss, L1 loss and generalized intersection-union ratio loss respectively, and sum the weighted values ​​of each loss to obtain the total loss value of the current prediction.

[0111] In one possible implementation, the expression for calculating the total loss value for each training iteration is:

[0112] ;

[0113] in, It is the total loss value. It is the focus loss function used for classification prediction. It is the L1 loss function used for size regression. It is the generalized intersection-union loss function. , and All are adjustable weighting coefficients.

[0114] S160: Based on the current total loss value, backpropagate to update the model parameters of the current model's heatmap modulation module, multi-level feature fusion module, differential denoising interaction module, and localization head prediction module to obtain the tracking model updated after the current training.

[0115] S170: Determine whether the tracking model after the current training update has converged;

[0116] S180: If so, end the training and obtain the trained visual tracking model;

[0117] S190: If not, continue with the next model training iteration.

[0118] Through the above steps, a complete description of the differential denoising interactive target tracking method for traffic scenarios proposed in this invention is completed, including the core process of data preprocessing, multi-level feature extraction based on the frozen parameter DINOv3 backbone network, category token-guided heatmap modulation, multi-level feature fusion, differential denoising interaction, and localization head decoding, and the training method of the model is further explained.

[0119] Corresponding to the aforementioned differential denoising interactive target tracking method for traffic scenarios, this invention also provides a differential denoising interactive target tracking system for traffic scenarios. This system is used to implement the aforementioned differential denoising interactive target tracking method for traffic scenarios; the system includes:

[0120] The video acquisition module is used to acquire video frame sequences in traffic scenarios;

[0121] The data preprocessing module is used to take the current video frame obtained from the video frame sequence as the search frame, and to perform data preprocessing on the pre-determined template frame and the search frame respectively to obtain the preprocessed template image and the preprocessed search area image.

[0122] The visual tracking module, which embeds a pre-trained visual tracking model, is used to input the pre-processed template image and the pre-processed search region image into the pre-trained visual tracking model and perform the following operations:

[0123] By using the DINOv3 backbone network with frozen parameters, multi-level feature extraction is performed on the preprocessed search region image and the preprocessed template image to obtain multi-level template feature maps, multi-level search region feature maps, and multi-level template global category tokens.

[0124] The multi-level feature fusion module is used to perform channel alignment processing on the multi-level template feature map, the multi-level search region feature map, and the multi-level template global category token;

[0125] In the heatmap modulation module, the global category token of the multi-level template after channel alignment is used to perform position-by-position modulation on the feature map of the multi-level search region after channel alignment to generate a spatial response map. The spatial response map is used to enhance the search region features of the corresponding level in the feature map of the multi-level search region after channel alignment to obtain the enhanced multi-level search region feature map.

[0126] The multi-level feature fusion module fuses the channel-aligned multi-level template feature map and the enhanced multi-level search region feature map to construct a hierarchical feature fusion sequence.

[0127] The differential denoising interaction module performs differential attention processing on the hierarchical feature fusion sequence to obtain the search region features after interaction.

[0128] The localization head prediction module decodes the features of the search area after interaction, calculates the target bounding box coordinates in the current video frame, and completes the target detection of the current video frame.

[0129] Based on the above technical solution, the present invention can achieve beneficial technical effects in practical applications, which are specifically reflected in the following aspects.

[0130] To address the low reliability of existing visual target tracking methods in complex traffic scenarios, this invention provides a differential denoising interactive target tracking method and system for traffic scenarios. The method first trains a constructed visual tracking model, iteratively optimizing it on a large-scale tracking dataset to ensure that the heatmap modulation module, multi-level feature fusion module, differential denoising interactive module, and localization head prediction module learn optimal parameters, while keeping the DINOv3 backbone network frozen to retain its general visual representation capabilities. During the inference phase, multi-level robust features are extracted using the frozen-parameter DINOv3 backbone network. The heatmap modulation module modulates the search area features position-by-position using a template global category token as a guide. This is done before interaction. This invention explicitly enhances the target response and suppresses background interference. A multi-level feature fusion module integrates the enhanced multi-level search region features with the template features to construct a unified hierarchical feature fusion sequence. A differential denoising interaction module eliminates common-mode attention noise through dual attention differential operations, achieving high signal-to-noise ratio deep interaction between the template and the search region. Finally, a localization head prediction module accurately reconstructs the target bounding box coordinates through parallel decoding branches. Through the synergistic effect of these modules, this invention effectively addresses challenges such as cluttered backgrounds, similar interference, target occlusion, and lighting variations in complex traffic scenarios, significantly improving the robustness and accuracy of target tracking. This effectively solves the technical problem of low reliability in existing visual target tracking methods in complex traffic scenarios.

[0131] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A differential denoising interactive target tracking method for traffic scenarios, characterized in that, include: Collect video frame sequences in traffic scenarios; The current video frame obtained from the video frame sequence is used as the search frame, and data preprocessing is performed on the predetermined template frame and the search frame respectively to obtain the preprocessed template image and the preprocessed search area image. The preprocessed template image and the preprocessed search region image are input into the trained visual tracking model, and the following operations are performed: Using the DINOv3 backbone network with frozen parameters, multi-level feature extraction is performed on the preprocessed search region image and the preprocessed template image to obtain multi-level template feature maps, multi-level search region feature maps, and multi-level template global category tokens. The multi-level feature fusion module is used to perform channel alignment processing on the multi-level template feature map, the multi-level search region feature map, and the multi-level template global category token; In the heatmap modulation module, the multi-level template global category token after channel alignment is used to perform position-by-position modulation on the multi-level search region feature map after channel alignment to generate a spatial response map; the spatial response map is used to enhance the search region features of the corresponding level in the multi-level search region feature map after channel alignment to obtain an enhanced multi-level search region feature map. The multi-level feature fusion module fuses the channel-aligned multi-level template feature map and the enhanced multi-level search region feature map to construct a hierarchical feature fusion sequence. The hierarchical feature fusion sequence is subjected to differential attention processing through the differential denoising interaction module to obtain the search region features after interaction. The positioning head prediction module decodes the features of the search region after the interaction, calculates the target bounding box coordinates in the current video frame, and completes the target detection of the current video frame.

2. The differential denoising interactive target tracking method for traffic scenarios according to claim 1, characterized in that, The predetermined template frame refers to a frame that is predetermined by annotation before tracking the current video frame is performed, or a historical frame that precedes the current video frame selected from the video frame sequence according to an update strategy. The step involves using the current video frame obtained from the video frame sequence as the search frame, and performing data preprocessing on the predetermined template frame and the search frame respectively to obtain a preprocessed template image and a preprocessed search region image, including: Obtain the coordinates of the target center point of the pre-determined template frame; Using the coordinates of the target center point of the predetermined template frame as a reference, an image containing the target is cropped from the predetermined template frame and used as a template image; Based on the prediction result of the previous frame in the video frame sequence, determine the coordinates of the predicted target center point of the search frame; Based on the coordinates of the predicted target center point of the search frame, a search region image containing the candidate region is cropped from the search frame; The template image and the search region image are subjected to size normalization and pixel value normalization respectively to obtain the preprocessed template image and the preprocessed search region image.

3. The differential denoising interactive target tracking method for traffic scenarios according to claim 1, characterized in that, The DINOv3 backbone network for the frozen parameters is configured to perform the following operations: The preprocessed search region image and the preprocessed template image are respectively input into the DINOv3 backbone network with the frozen parameters, and forward propagation is performed through the DINOv3 backbone network with the frozen parameters to extract the final result. Layer output, It is a preset positive integer; From the last In the layer output, the template block token sequence and template global category token corresponding to the preprocessed template image are obtained for each layer, as well as the search region block token sequence corresponding to the preprocessed search region image; The template block token sequence of each layer is used as the template feature map of the corresponding layer, and the multi-level template feature map is obtained by summarizing them. The template global category tokens of each layer are aggregated to form the multi-level template global category token; The sequence of block tokens in the search region of each layer is used as the search region feature map of the corresponding layer, and the multi-level search region feature map is obtained by summarizing them.

4. The differential denoising interactive target tracking method for traffic scenarios according to claim 1, characterized in that, The channel alignment process is implemented through a shared projection layer. The shared projection layer uses a 1×1 convolution operation to reduce the channel dimension of each level feature in the multi-level template feature map, the multi-level search region feature map, and the multi-level template global category token from the original dimension to a preset dimension, thereby obtaining the channel-aligned multi-level search region feature map, the channel-aligned multi-level template feature map, and the channel-aligned multi-level template global category token, respectively.

5. The differential denoising interactive target tracking method for traffic scenarios according to claim 3, characterized in that, The heatmap modulation module is configured to perform the following operations: For each level , ,in Total number of levels extracted: The first The hierarchical channel-aligned template global category token serves as a semantic prior prototype, and is related to the first... After channel alignment at each level, a dot product operation is performed on each local feature in the feature map of the search region to obtain the first... The original similarity score corresponding to the level; Introducing a preset temperature hyperparameter, for the first The original similarity scores corresponding to each level are scaled to obtain the scaled version of the first level. Original similarity score of the group; Use the SoftMax function to scale the first... The original similarity scores of the groups are normalized to generate the first group. Spatial response diagrams corresponding to the levels; Using a residual enhancement mechanism, the first... The spatial response diagram corresponding to the level and the first The feature map of the search region after channel alignment at each level is subjected to element-wise multiplication and addition operations to obtain the first... Enhanced search region feature maps corresponding to each level; Summary The enhanced search region feature maps of each level are used as the enhanced multi-level search region feature maps.

6. The differential denoising interactive target tracking method for traffic scenarios according to claim 1, characterized in that, The step involves fusing the channel-aligned multi-level template feature map and the enhanced multi-level search region feature map through the multi-level feature fusion module to construct a hierarchical feature fusion sequence, including: The multi-level template feature map after channel alignment is flattened along the token length dimension and serialized and spliced ​​according to the hierarchical index order to obtain the template feature sequence. The enhanced multi-level search region feature map is flattened along the token length dimension and serialized and spliced ​​according to the hierarchical index order to obtain the search region feature sequence. The template feature sequence and the search region feature sequence are concatenated along the token dimension to obtain the hierarchical feature fusion sequence.

7. The differential denoising interactive target tracking method for traffic scenarios according to claim 1, characterized in that, The differential denoising interaction module is configured to perform the following operations: The hierarchical feature fusion sequence is projected onto the first set of query matrices and the first set of key matrices, the second set of query matrices and the second set of key matrices, and the shared value matrix, respectively. A first attention weight distribution is calculated using the first set of query matrices and the first set of key matrices. The first attention weight distribution contains mixed information of target matching signal and background interference. The second attention weight distribution is calculated using the second set of query matrices and the second set of key matrices. The second attention weight distribution is used to capture the common-mode attention noise between the template feature part and the search region feature part in the hierarchical feature fusion sequence. A learnable noise cancellation coefficient is introduced to weight the second attention weight distribution to suppress the common-mode attention noise, resulting in a noise-suppressed second attention weight distribution. The net attention weight distribution is obtained by subtracting the first attention weight distribution from the second attention weight distribution after noise suppression at each position. Using the net attention weight distribution as weight coefficients, the value vectors in the shared value matrix are weighted and aggregated to obtain a weighted feature sequence. Based on the weighted feature sequence and the hierarchical feature fusion sequence, the interactive feature sequence is obtained; Extract the feature sequence corresponding to the search region from the feature sequence after the interaction, and use it as the feature of the search region after the interaction.

8. The differential denoising interactive target tracking method for traffic scenarios according to claim 1, characterized in that, The positioning head prediction module is configured to perform the following operations: By reshaping the features, the features of the search region after the interaction are restored into a two-dimensional feature map with a spatial topological structure; The two-dimensional feature map is processed by a convolutional neural network consisting of multiple convolutional layers, batch normalization layers and activation functions stacked alternately to enhance the discriminative power between the target and the background, resulting in an enhanced feature map. Using three independent parallel decoding branches, the enhanced feature map is predicted to generate a target center point probability distribution map, a target bounding box size map, and a local position offset map, respectively. Obtain the spatial location with the highest response value in the probability distribution map of the target center point, and use it as the coordinates of the target center point; Based on the target center point coordinates, the width and height values ​​corresponding to the target center point coordinates in the target bounding box size map, and the offset value corresponding to the target center point coordinates in the local position offset map, the target bounding box coordinates in the current video frame are calculated through geometric transformation.

9. The differential denoising interactive target tracking method for traffic scenarios according to claim 1, characterized in that, The visual tracking model was trained in the following way: In the current training process, perform the following steps: The tracking model updated after the last training is obtained and used as the current model. The current model includes a DINOv3 backbone network with frozen parameters, as well as a trainable heatmap modulation module, a multi-level feature fusion module, a differential denoising interaction module, and a localization head prediction module. Based on a preset time interval, template sample frames and search sample frames are randomly selected from the training tracking video sequence; The template sample frame and the search sample frame are preprocessed to construct a training sample pair. The training sample pair includes the template sample image, the search sample image, and the target bounding box annotation information in the search sample image. The template sample image and the search sample image in the sample pair are input into the current model for forward propagation to obtain the coordinates of the current predicted target bounding box. Based on the current predicted target bounding box coordinates and the target bounding box annotation information in the search sample image of the sample pair, the focus loss, L1 loss and generalized intersection-union ratio loss are calculated respectively, and the weighted sum of each loss is obtained to obtain the current total loss value; Based on the current total loss value, backpropagation is used to update the model parameters of the current model's heatmap modulation module, multi-level feature fusion module, differential denoising interaction module, and localization head prediction module to obtain the tracking model after the current training update. Determine whether the tracking model updated in the current training iteration has converged; If so, end the training and obtain the trained visual tracking model; If not, proceed to the next model training iteration.

10. A differential denoising interactive target tracking system for traffic scenarios, characterized in that, The system is used to implement the differential denoising interactive target tracking method for traffic scenarios according to any one of claims 1 to 9; the system includes: The video acquisition module is used to acquire video frame sequences in traffic scenarios; The data preprocessing module is used to take the current video frame obtained from the video frame sequence as the search frame, and perform data preprocessing on the predetermined template frame and the search frame respectively to obtain the preprocessed template image and the preprocessed search area image. A visual tracking module, on which a pre-trained visual tracking model is embedded, is used to input the pre-processed template image and the pre-processed search region image into the pre-trained visual tracking model and perform the following operations: Using the DINOv3 backbone network with frozen parameters, multi-level feature extraction is performed on the preprocessed search region image and the preprocessed template image to obtain multi-level template feature maps, multi-level search region feature maps, and multi-level template global category tokens. The multi-level feature fusion module is used to perform channel alignment processing on the multi-level template feature map, the multi-level search region feature map, and the multi-level template global category token; In the heatmap modulation module, the multi-level template global category token after channel alignment is used to perform position-by-position modulation on the multi-level search region feature map after channel alignment to generate a spatial response map; the spatial response map is used to enhance the search region features of the corresponding level in the multi-level search region feature map after channel alignment to obtain an enhanced multi-level search region feature map. The multi-level feature fusion module fuses the channel-aligned multi-level template feature map and the enhanced multi-level search region feature map to construct a hierarchical feature fusion sequence. The hierarchical feature fusion sequence is subjected to differential attention processing through the differential denoising interaction module to obtain the search region features after interaction. The positioning head prediction module decodes the features of the search region after the interaction, calculates the target bounding box coordinates in the current video frame, and completes the target detection of the current video frame.

Images