A vehicle multi-target tracking method fusing attention and decoupled memory learning

Abstract

This invention discloses a vehicle multi-target tracking method integrating attention and decoupled memory learning, belonging to the fields of computer vision and intelligent transportation technology. The method includes the following steps: performing vehicle detection and trajectory prediction on traffic video frames to obtain target detection boxes and motion information; utilizing a vehicle appearance feature extraction network, and strengthening key vehicle appearance features with a spatially gated multi-scale backbone network; separating identity features and environment change features through an identity-environment change decoupling module; storing and updating historical identity features by a dynamic identity memory module, and fusing them with current identity features to generate cross-frame consistent appearance features. The obtained appearance features and motion information are fused to construct a joint cost matrix, and target matching and trajectory lifecycle management are completed through the Hungarian algorithm and secondary association. This invention can effectively alleviate matching errors and identity switching caused by interference in complex traffic scenes, and significantly improve the vehicle's cross-frame identity maintenance capability and trajectory continuity.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent transportation technology, and in particular to a method for multi-target vehicle tracking that integrates attention and decoupled memory learning. Background Technology

[0002] Multi-target vehicle tracking, a key technology in intelligent transportation systems, is primarily used to continuously detect, locate, and associate multiple vehicle targets within a video sequence to obtain stable target trajectory information. Existing multi-target tracking methods typically rely on a "detection-association" processing framework. This involves detecting targets in each frame and combining this with data association strategies to achieve cross-frame tracking. One type of method primarily depends on motion models such as Kalman filtering for state prediction, achieving basic tracking functionality in scenarios with relatively stable target motion. However, in situations with occlusion or target interaction, trajectory drift or association errors are prone to occur. To address these issues, some methods introduce appearance features based on re-identification, fusing motion information with appearance similarity for matching, thereby improving target identity preservation. However, in complex real-world traffic scenarios, due to the high appearance similarity between vehicle targets and the frequent occurrence of occlusion, lighting changes, and viewing angle variations, existing methods still suffer from insufficient appearance feature discrimination capabilities, leading to frequent mismatches during data association. Meanwhile, existing appearance features typically combine the identity information used to distinguish different vehicle targets with variations in pose, lighting, and other factors, resulting in unstable feature distributions for the same target across different frames and affecting the accuracy of cross-frame association. Furthermore, most methods rely primarily on current frame or short-term historical features during the association process, lacking an effective modeling mechanism for the target's long-term identity information. When a target reappears after occlusion, it struggles to maintain its original identity, leading to identity switching or trajectory interruption. Therefore, current technologies still struggle to simultaneously achieve stable appearance feature representations and reliable cross-frame data association in complex traffic scenarios, necessitating a vehicle multi-target tracking method that improves feature discriminative power and enhances identity association stability. Summary of the Invention

[0003] To address the shortcomings of existing multi-target vehicle tracking methods in complex traffic scenarios, such as insufficient appearance feature discrimination, unstable feature distribution due to the mixed representation of identity information with changing factors like pose and illumination, and inaccurate cross-frame association and identity switching caused by a lack of long-term identity modeling capabilities, this invention proposes a multi-target vehicle tracking method that integrates attention and decoupled memory learning, leveraging the advantages of existing "detection-association" tracking frameworks. This method makes targeted improvements in appearance feature extraction and data association: First, it enhances the key discrimination regions of the vehicle by introducing a spatial attention mechanism to improve the discrimination capability of appearance features; second, it separates and models features by constructing a decoupled learning mechanism for identity information and environmental change factors, thereby alleviating the problem of unstable feature distribution; further, it continuously models the historical identity features of the target by introducing a dynamic identity memory mechanism to enhance the target's ability to maintain its identity under occlusion or re-emergence conditions; based on this, it fuses the optimized appearance features with motion information to achieve more stable and reliable cross-frame data association, thereby improving the accuracy and continuity of multi-target vehicle tracking in complex traffic scenarios.

[0004] Specifically, this invention provides a vehicle multi-target tracking method that integrates attention and decoupled memory learning, mainly including the following steps: S1. Input a video frame sequence, perform target detection on each frame, obtain the detection box of the vehicle target, and initialize a new trajectory or update an existing trajectory based on the detection results; S2. Maintain the existing trajectory set, and use Kalman filtering on each trajectory to predict its motion state, obtaining the trajectory prediction results to provide prior information for data association. The predicted motion state includes the target's position, velocity, and bounding box scale information. S3. Input the detected vehicle target area into the appearance feature extraction network, perform feature extraction and representation learning through the spatial gating multi-scale backbone network, separate identity features and appearance change features through the identity-environment change decoupling module, and fuse historical identity features through the dynamic identity memory module to finally generate a vehicle appearance feature vector for target association. S4. In the trajectory association stage, the matching association cost is calculated based on the trajectory prediction result obtained in step S2 and the vehicle appearance feature vector obtained in step S3. The matching association cost consists of motion association cost and appearance association cost. The motion association cost is calculated by the Mahalanobis distance between the trajectory prediction position and the detection box position, thus obtaining the motion distance; the appearance association cost is calculated by the cosine distance between the identity features, thus obtaining the appearance distance. Subsequently, a cascaded matching strategy is used for data association. This strategy combines the appearance association cost and the motion association cost to construct a joint cost matrix. The initial matching is completed using the Hungarian algorithm, outputting three types of results: matched trajectories, unmatched detection boxes, and unmatched trajectories. For the unmatched detection boxes and unmatched trajectories, a secondary matching method using intersection-union ratio (IUU) is used for supplementary association to obtain the final matching result.

[0005] S5. Based on the matching results of step S4, perform trajectory status management, marking the trajectory as temporary, confirmed, or deleted, thus completing the trajectory update and maintenance. The final output includes the tracking result containing the vehicle bounding box and the tracking ID, where the tracking ID is a unique identification number assigned to each independent vehicle target, used to distinguish different vehicles and maintain identity consistency across frames.

[0006] The spatially gated multi-scale backbone network described in step S3 extracts features from the target image step by step through a multi-scale feature extraction unit, and introduces a spatial gating mechanism during the feature extraction process to adaptively adjust the spatial response of the feature map in order to enhance the features of key target regions and suppress background interference. The identity-environment change decoupling module described in step S3 is implemented by constructing a feature decomposition and constraint learning mechanism. Appearance features are input into this module, and the features are decomposed through a parallel branch structure to obtain identity-related features and environment change-related features respectively. During the feature learning process, orthogonal constraints are applied to different branches to weaken the response of identity-related features to changes in factors such as pose and illumination, while making environment change-related features mainly represent information about changes in the external environment, thereby achieving decoupled representation of the two types of features in the feature space. The dynamic identity memory module described in step S3 is equipped with an identity dictionary, which is used to store the historical identity features of tracked vehicles and save a fixed identity feature template for each vehicle. During the tracking process, the features are updated using a moving average method. When a vehicle reappears after being obscured, its original identity is retrieved by matching the features in the dictionary, thus avoiding identity switching and improving tracking stability. The trajectory update and maintenance described in step S5 mainly targets trajectories marked as temporary, confirmed, or deleted. Temporary trajectories are newly created initial trajectories that have not completed multi-frame verification and are only temporarily stored without outputting formal results; confirmed trajectories have been verified as valid vehicle trajectories through multi-frame verification, are included in the tracking queue, and are continuously updated; trajectories with target loss, matching failure, or false interference are marked as deleted, removed from the tracking queue, and updates cease.

[0007] Furthermore, in a specific embodiment of the vehicle multi-target tracking method that integrates attention and decoupled memory learning described in this invention, the expression of the overall loss function is as follows: ; in, L id This is the identity classification loss function, used to enhance the ability to distinguish between different vehicle targets and ensure that the extracted features can effectively distinguish different identities; L tri The triplet metric learning loss function is used to constrain the distance between features of the same target to be less than the distance between different target features, thereby improving the discriminativeness of features in the embedding space. L orth It is an orthogonal constraint loss function used to reduce the correlation between identity-related features and environmental change-related features, thereby achieving feature decoupling and reducing the interference of appearance change factors; L gate This is a gated constraint loss function used to adjust the gated response during feature extraction, enhance the feature representation of key discrimination regions, and suppress redundant or irrelevant information. , , The weighting coefficients are used to balance the contributions of each loss term. The overall loss function is optimized through multiple constraints, ensuring that the extracted appearance features simultaneously possess discriminativeness, decoupling, and stability.

[0008] Beneficial Effects: This invention significantly improves the stability and robustness of multi-target vehicle tracking in complex traffic scenarios through the synergistic optimization of spatial attention enhancement, identity-environment change decoupling learning, and dynamic identity memory. Specifically, the spatial attention mechanism strengthens key discrimination region features and suppresses background and occlusion interference; the identity-environment change decoupling learning separates stable identity information from changes in illumination, pose, and viewpoint, effectively reducing cross-frame drift and mismatches; and the dynamic identity memory mechanism continuously models historical features, improving identity recovery capabilities after occlusion and reducing trajectory breaks. Through these combined effects, this invention achieves a comprehensive improvement in tracking accuracy, identity consistency, and trajectory continuity in environments with dense traffic, frequent occlusion, and variable illumination and viewpoints, demonstrating significant practical value. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of the overall process of the vehicle multi-target tracking method that integrates attention and decoupled memory learning according to the present invention. Figure 2 This is a schematic diagram of the overall structure of the appearance feature extraction network of the present invention. Detailed Implementation

[0010] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and their descriptions are used to explain the invention, but are not intended to limit the invention.

[0011] like Figure 1 , Figure 2 As shown, the present invention provides a vehicle multi-target tracking method that integrates attention and decoupled memory learning, including the following steps.

[0012] S1. Input a video frame sequence, perform target detection on each frame, obtain the detection box of the vehicle target, and initialize a new trajectory or update an existing trajectory based on the detection results.

[0013] S2. Maintain the existing trajectory set, and use Kalman filtering on each trajectory to predict its motion state, obtaining the trajectory prediction results to provide prior information for data association. The predicted motion state includes the target's position, velocity, and bounding box scale information.

[0014] S3. Input the detected vehicle target area into the appearance feature extraction network to extract vehicle appearance features. The specific extraction steps are as follows: Figure 2 As shown: Vehicle Image Input: In this invention, the input image is derived from vehicle detection results in traffic monitoring videos, and the input vehicle image size is uniformly adjusted to 256×128×3. Here, 256 represents the image height, 128 represents the image width, and 3 represents the number of channels in the RGB color image, corresponding to the red (R), green (G), and blue (B) color channels respectively. Input preprocessing: The input image is first convolved using a 7×7 kernel with a stride of 2 and 64 output channels, resulting in a shallow feature map. Subsequently, a 3×3 max-pooling layer is used for spatial downsampling with a stride of 2, thereby reducing the feature map resolution and computational cost. After preprocessing, the network obtains basic texture and edge structure features. Spatial Gated Multi-Scale Backbone Network: This network is used for deep appearance feature extraction from vehicle images. The preprocessed feature map undergoes three multi-scale feature learning stages: a second stage, a third stage, and a fourth stage. The second stage consists of two stacked scale-optimized spatial attention units, outputting a feature dimension of 256×64×32, where 256 represents the number of feature channels, and 64 and 32 represent the height and width of the feature map, respectively. The third stage consists of two scale-optimized spatial attention units, outputting a feature dimension of 384×32×16. The fourth stage consists of two scale-optimized spatial attention units, outputting a feature dimension of 512×16×8. Each scale-optimized spatial attention unit employs a multi-branch lightweight convolutional structure, with each branch using 3×3 lightweight convolutions of varying depths to achieve parallel extraction of features from multiple receptive fields, thereby simultaneously learning local vehicle texture information, mesoscale structural information, and global semantic information. Subsequently, the features of each branch are dynamically fused through a feature aggregation gating module. This gating module first generates channel statistics using global average pooling, and then generates channel weight vectors in the range g∈[0,1] using two layers of 1×1 convolutions and a Sigmoid activation function, adaptively weighting features at different scales. Through this spatial gating mechanism, the network can significantly enhance the response of key identity regions such as headlights, windows, body texture, and vehicle outline, while suppressing background noise, road texture, and occlusion interference, thereby improving the discrimination ability and robustness of vehicle appearance features in complex traffic scenarios. Identity-Environment Change Decoupling Module: The vehicle feature map extracted by the spatially gated multi-scale backbone network is input into the identity-environment change decoupling module, which is used to separate and learn the vehicle's stable identity information from external environmental change information. Specifically, the input features are first processed by global average pooling to generate a 512-dimensional global feature vector, and then processed by a dynamic gating network to generate a 512-dimensional gating weight vector g∈[0,1], which is used to control the importance of different channels. The dynamic gating network includes a global average pooling layer, two fully connected layers, ReLU activation, and Sigmoid activation, which can adaptively adjust the ratio of identity information and environmental change information in the features. In the identity feature branch, the input features are element-wise weighted using a gated weight vector g, and then processed through a local average pooling layer, a fully connected layer, and normalization to obtain 256-dimensional identity features, which are used for vehicle identity discrimination and re-identification. In the environmental change feature branch, the complementary gated weight vector 1-g is used to extract environmental change features, which are then processed in the same way to obtain 256-dimensional environmental change features, used to model environmental interference information such as illumination, occlusion, viewpoint, and background changes. To further enhance the independence of the two types of features, an orthogonal constraint loss is introduced, the specific expression of which is as follows: ; In the formula, The decoupled vehicle identity feature vector is obtained. This represents the corresponding environmental change feature vector. Indicates the vector transpose operation, subscript F denoted by Frobenius norm, used to calculate the square root of the sum of squares of all elements of a matrix; This represents the summation of squares over all elements in the matrix, used to measure the correlation strength between two features. This orthogonal constraint loss forces the identity features and environmental change features to be orthogonal in the feature space by minimizing the correlation response between them. This allows the identity features to focus on representing the vehicle's inherent attributes, while the environmental change features focus on representing appearance perturbations such as lighting and viewing angle, thereby improving the robustness and discriminative power of the features. Dynamic Identity Memory Module: This module maintains and updates historical vehicle identity feature information, enhancing the stability of cross-frame vehicle recognition. By establishing an identity memory dictionary, this module stores and manages historical identity identifier memory features, providing a stable long-term temporal identity representation for subsequent feature fusion. When a vehicle's trajectory is interrupted in the surveillance video due to occlusion, brief absence from view, or dense traffic flow, the dynamic identity memory module can utilize the stored historical identity identifier memory features to match the reappearing vehicle, thereby restoring the original identity number, maintaining trajectory continuity, effectively reducing identity switching and recognition errors, and improving the accuracy and robustness of multi-target vehicle tracking and re-identification in complex traffic scenarios. Feature Fusion: This module fuses the identity features of the current frame with historical identity memory features to generate more stable and robust vehicle appearance features. Specifically, this module takes the 256-dimensional identity features of the current frame output from the identity-environment change decoupling module and the 512-dimensional historical identity memory features maintained in the dynamic identity memory module, performs channel alignment, and then performs feature fusion through a gated weighted fusion mechanism. The gated weighted mechanism adaptively adjusts the proportion of the two types of features by learning weight vectors, ensuring that both short-term appearance information and long-term identity information continuity are preserved. The fused feature vector is then mapped to a unified dimension through a fully connected layer and normalized to obtain the final 512-dimensional vehicle appearance feature vector. This feature vector combines the identity representation capability of the current frame with the long-term memory information of historical trajectories, and can be directly used for cross-frame vehicle association, multi-target tracking, and trajectory management, significantly improving the stability of identity discrimination and the robustness of the system in complex traffic scenarios.

[0015] S4. In the trajectory association stage, the matching association cost is calculated based on the trajectory prediction result obtained in step S2 and the vehicle appearance feature vector obtained in step S3. The matching association cost consists of motion association cost and appearance association cost. The motion association cost is calculated by the Mahalanobis distance between the trajectory prediction position and the detection box position, thus obtaining the motion distance; the appearance association cost is calculated by the cosine distance between the identity features, thus obtaining the appearance distance. Subsequently, a cascaded matching strategy is used for data association. This strategy combines the appearance association cost and the motion association cost to construct a joint cost matrix. The initial matching is completed using the Hungarian algorithm, outputting three types of results: matched trajectories, unmatched detection boxes, and unmatched trajectories. For the unmatched detection boxes and unmatched trajectories, a secondary matching method using intersection-union ratio (IUU) is used for supplementary association to obtain the final matching result.

[0016] S5. Based on the matching results of step S4, perform trajectory status management, marking the trajectory as temporary, confirmed, or deleted, thus completing the trajectory update and maintenance. The final output includes the tracking result containing the vehicle bounding box and the tracking ID, where the tracking ID is a unique identification number assigned to each independent vehicle target, used to distinguish different vehicles and maintain identity consistency across frames.

[0017] Furthermore, in a specific embodiment of the present invention, the VehicleID dataset is used to supervise the training of the vehicle appearance feature extraction network during the vehicle appearance feature learning stage. During the training process, the input image resolution is uniformly adjusted to 256×128, the batch size is set to 64, and the optimizer is AdamW.

[0018] Furthermore, in a specific embodiment of this invention, the multi-target tracking performance is evaluated using the UA-DETRAC dataset. During the experiment, 10 representative traffic video sequences from the UA-DETRAC dataset are selected as a test subset to evaluate the tracking stability and robustness of the method in complex traffic scenarios. In the tracking phase, the trained vehicle appearance feature extraction network is used to extract appearance features from the detected targets, and cross-frame data association is completed by combining trajectory motion information. During feature extraction, the detection region is first cropped and uniformly adjusted to a 128×64 resolution for input to the network, with a batch size of 32. During trajectory association, the initial frame count is set to 3 frames, the cosine distance threshold is set to 0.18, the maximum survival frame count is set to 30 frames, the feature cache capacity is set to 100, the target detection confidence threshold is set to 0.55, the non-maximum suppression threshold is set to 0.8, and the intersection-union ratio (IUU) matching threshold is set to 0.7.

[0019] Furthermore, in a specific embodiment of the present invention, the expression for the total loss function is as follows: ; in, L id This is the identity classification loss function, used to enhance the ability to distinguish between different vehicle targets and ensure that the extracted features can effectively distinguish different identities; L tri The triplet metric learning loss function is used to constrain the distance between features of the same target to be less than the distance between different target features, thereby improving the discriminativeness of features in the embedding space. L orth It is an orthogonal constraint loss function used to reduce the correlation between identity-related features and environmental change-related features, thereby achieving feature decoupling and reducing the interference of appearance change factors; L gate This is a gated constraint loss function used to adjust the gated response during feature extraction, enhance the feature representation of key discrimination regions, and suppress redundant or irrelevant information. , , The weighting coefficients are used to balance the contributions of each loss term. The overall loss function is optimized through multiple constraints, ensuring that the extracted appearance features simultaneously possess discriminativeness, decoupling, and stability.

[0020] The technical means disclosed in this invention are not limited to those disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications are also considered within the scope of protection of this invention.

Claims

1. A method for multi-target vehicle tracking that integrates attention and decoupled memory learning, characterized in that: The method includes the following steps: S1. Input a video frame sequence, perform vehicle target detection on each frame image, obtain the detection box, and initialize a new trajectory or update an existing trajectory based on the detection results; S2. Maintain the existing trajectory set, predict the motion state of each trajectory, and obtain the trajectory prediction results to provide prior information for data association; S3. Input the detected vehicle target area into the appearance feature extraction network. This network includes a spatial gating multi-scale backbone network, an identity-environment change decoupling module, and a dynamic identity memory module, which are used to extract discriminative appearance features, separate identity information from environmental change information, and store and update vehicle historical identity information, respectively. Finally, the output is a vehicle appearance feature vector for target association. A spatially gated multi-scale backbone network is used for deep appearance feature extraction from vehicle images. The preprocessed feature map undergoes three multi-scale feature learning stages for feature extraction: a second stage, a third stage, and a fourth stage. The second stage consists of two stacked scale-optimized spatial attention units, outputting a feature dimension of 256×64×32, where 256 represents the number of feature channels, and 64 and 32 represent the height and width of the feature map, respectively. The third stage consists of two scale-optimized spatial attention units, outputting a feature dimension of 384×32×16. The fourth stage… It consists of two scale-optimized spatial attention units, with an output feature dimension of 512×16×8. Each scale-optimized spatial attention unit adopts a multi-branch lightweight convolution structure. Each branch adopts a 3×3 lightweight convolution with different convolution depths. Then, the features of each branch are dynamically fused through a feature aggregation gating module. The gating module first uses global average pooling to generate channel statistics, and then uses two layers of 1×1 convolution and a Sigmoid activation function to generate a channel weight vector in the range of g∈[0,1], and adaptively weights the features at different scales. The identity-environment change decoupling module first generates a 512-dimensional global feature vector through global average pooling, and then generates a 512-dimensional gating weight vector g∈[0,1] through a dynamic gating network to control the importance of different channels. The dynamic gating network includes a global average pooling layer, two fully connected layers, ReLU activation, and Sigmoid activation. In the identity feature branch, the gating weight vector g is used to weight the input features element-wise, and after passing through the global average pooling layer, fully connected layer, and normalization processing, a 256-dimensional identity feature is obtained. In the environment change feature branch, the complementary gating weight vector 1-g is used to extract the environment change feature, and after the same processing, a 256-dimensional environment change feature is obtained. The dynamic identity memory module is equipped with an identity dictionary for storing historical identity information of tracked vehicles. During the feature extraction process, the historical identity information is iteratively updated using a moving average method based on the identity features extracted in the current frame and the historical identity information stored in the identity dictionary. S4. In the trajectory association stage, the matching association cost is calculated based on the trajectory prediction result obtained in step S2 and the vehicle appearance feature vector obtained in step S3. The matching association cost consists of motion association cost and appearance association cost. The motion association cost is calculated by the Mahalanobis distance between the trajectory prediction position and the detection box position to obtain the motion distance. The appearance association cost is calculated by the cosine distance between the identity features to obtain the appearance distance. Subsequently, a cascaded matching strategy is adopted for data association. This strategy combines the appearance association cost and the motion association cost to construct a joint cost matrix. The initial matching is completed by the Hungarian algorithm, and three types of results are output: matched trajectory, unmatched detection box, and unmatched trajectory. For the unmatched detection box and unmatched trajectory, a second-order cross-union matching is used to supplement the association to obtain the final matching result. S5. Based on the matching results of step S4, perform trajectory status management, mark the trajectory as temporary, confirmed or deleted, complete the trajectory update and maintenance, and finally output the tracking result containing the vehicle bounding box and tracking ID, where the tracking ID is a unique identification number assigned to each independent vehicle target to distinguish different vehicles.

2. The vehicle multi-target tracking method integrating attention and decoupled memory learning according to claim 1, characterized in that: The motion state described in step S2 includes the target's position, the target's velocity, and the target bounding box scale information.

3. The vehicle multi-target tracking method integrating attention and decoupled memory learning according to claim 1, characterized in that: The spatially gated multi-scale backbone network described in step S3 is used to receive the input vehicle image, extract local texture information, mesoscale structural information and global semantic information of the vehicle step by step, and assign weights to different spatial regions of the feature map through a spatial gating mechanism.

4. The vehicle multi-target tracking method integrating attention and decoupled memory learning according to claim 1, characterized in that: The identity-environment change decoupling module described in step S3 is used to decompose the multi-scale features extracted by the spatial gated multi-scale backbone network into identity-related features and environment change-related features, and to perform constraint learning on the two types of features through orthogonal constraints.

Images