A vehicle multi-target tracking method based on improved BoT-SORT

Abstract

This invention discloses a vehicle multi-target tracking method based on an improved BoT-SORT, belonging to the field of multi-target tracking technology. The method includes the following steps: Step 1, target detection and confidence grouping; Step 2, trajectory prediction; Step 3, motion compensation correction; Step 4, feature extraction and orientation enhancement; Step 5, residual fusion and identity feature generation; Step 6, two-stage association and trajectory management. By improving the vehicle re-identification network and motion state prediction, this method can address the challenges of vehicle tracking in traffic monitoring scenarios. It not only uses an OSNet multi-scale network to extract global features but also designs a HeightWidthFeatureDepthwiseConv orientation feature enhancement method to specifically strengthen horizontal and vertical structural features. Furthermore, it achieves deep fusion of multi-scale global features and orientation enhancement features through residual connections, making vehicle identity features more distinctive and stable.

Description

Technical Field

[0001] This invention relates to the field of multi-target tracking technology, and more specifically, to a vehicle multi-target tracking method based on an improved BoT-SORT. Background Technology

[0002] With the rapid development of intelligent transportation systems and autonomous driving technology, multi-target vehicle tracking, as a core underlying technology in traffic monitoring, vehicle-road cooperation, and trajectory analysis, is tasked with continuously locating multiple vehicle targets in a video sequence and maintaining their identity consistency to provide reliable support for subsequent behavior understanding and decision-making. However, the frequent occurrence of vehicle occlusion, dense distribution, similar appearance, and sudden changes in motion state in real traffic scenarios makes tracking algorithms prone to identity switching and trajectory breakage. Therefore, a new tracking method is needed.

[0003] In the prior art, document CN116500603B discloses a "multi-target tracking method, device and vehicle". The method includes: acquiring lidar data and millimeter-wave radar data collected by lidar and millimeter-wave radar at the current moment, and determining the detection target information at the current moment based on the target data; matching and tracking multiple obstacles within a preset range based on the detection target information and historical tracking target information; the historical tracking target information includes: the position information, speed information and heading angle of each obstacle to be tracked in the tracking list.

[0004] The document CN113963168A discloses "an improved high-precision end-to-end vehicle multi-target tracking method", which includes: detecting vehicles based on an improved Faster RCNN model to obtain a target detection dataset; inputting the target detection dataset into a region generation network module for similarity learning to obtain a candidate target dataset to be matched; and finally matching the candidate target dataset to be matched by a nearest neighbor matching method.

[0005] In summary, existing technologies, by improving network structure and fusing multi-sensor information, can effectively enhance the model's ability to detect and track targets with large scale variations, significantly reduce the number of identity switching during the tracking process, and enable unmanned machinery to have stable tracking capabilities for multiple targets, thereby improving overall work efficiency. However, most existing technologies are geared towards multi-sensor fusion or end-to-end integrated detection and tracking designs in unmanned driving scenarios, and do not pay enough attention to specific problems such as vehicle occlusion, drastic scale changes, and high similarity in appearance in pure visual image sequences in traffic monitoring. Furthermore, the re-identification network has limited ability to extract fine-grained structural features of vehicles, and motion prediction is difficult to adapt to fluctuations in detection quality, resulting in frequent vehicle identity switching and difficulty in trajectory recovery after occlusion. Overall tracking capabilities still need to be improved. Summary of the Invention

[0006] This invention provides a vehicle multi-target tracking method based on an improved BoT-SORT, which can solve the problems mentioned in the background.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a vehicle multi-target tracking method based on improved BoT-SORT, comprising: S1. Acquire traffic monitoring video stream, use YOLOX target detector to perform vehicle target detection on each frame of the video stream, output detection results including the center coordinates, width, height and detection confidence of the detection box, and divide the detection results into high confidence detection set and low confidence detection set according to the detection confidence. S2. The motion state of the existing vehicle trajectory is described by an eight-dimensional state vector. The position and scale of the vehicle in the current frame are predicted by the state transition matrix to generate a prediction box. The observation noise covariance matrix is ​​dynamically adjusted according to the detection confidence output by the detector. S3. Extract ORB features from two consecutive frames and perform optical flow tracking. Calculate the camera motion affine matrix between the two consecutive frames using the RANSAC algorithm. Use the affine matrix to correct the generated prediction box to obtain the compensated prediction trajectory. S4. Crop the vehicle region image according to the detection box position and uniformly scale it to 256×256 resolution. Input the scaled image into the OSNet network to extract multi-scale vehicle features, and then input the multi-scale vehicle features into HeightWidthFeatureDepthwiseConv for feature enhancement. S5. The multi-directional features obtained by feature enhancement are fused with the original multi-scale features output by OSNet through residual connection. Global average pooling and L2 normalization are performed on the fused vehicle identity features to generate vehicle identity feature vector. S6. Construct an association cost matrix based on motion and appearance information. Use the Hungarian algorithm to perform a first-stage matching between the trajectory and the high-confidence detection set. Then, perform a second-stage matching between the unmatched trajectory and the low-confidence detection set. Based on the matching results, update the trajectory status, generate new trajectories, and delete inactive trajectories. Finally, output the vehicle tracking results.

[0008] Furthermore, in step S1, only unmatched high-confidence detection boxes are used to generate new vehicle trajectories, while low-confidence detection boxes do not participate in the generation of new trajectories and are only used for trajectory matching in the subsequent second stage.

[0009] Furthermore, in step S2, the eight-dimensional state vector consists of the target center horizontal coordinate, center vertical coordinate, target width, target height, horizontal velocity, vertical velocity, width change rate, and height change rate. The motion state prediction of the vehicle in the current frame is completed through the state transition matrix.

[0010] Furthermore, in step S3, sparse optical flow tracking is performed based on the extracted ORB feature points to obtain matching features. After removing the erroneous features using the RANSAC algorithm, the camera motion affine matrix is ​​calculated, and the coordinate transformation of all predicted boxes is performed using the affine matrix to complete the compensation.

[0011] Furthermore, in step S4, the HeightWidthFeatureDepthwiseConv feature enhancement is set with three parallel feature extraction branches. The three branches use convolution kernels of different sizes to perform orientation-specific feature extraction on the input multi-scale vehicle features.

[0012] Furthermore, in step S5, the three directional features of the HeightWidthFeatureDepthwiseConv feature enhancement output are first concatenated, and then the channel dimension features are fused through 1×1 convolution. The fused features are then residually connected with the original multi-scale features output by OSNet.

[0013] Furthermore, in step S6, the first-stage matching constructs an association cost matrix containing the intersection-union distance and appearance distance, and uses the Hungarian algorithm to complete the matching of the trajectory with the high-confidence detection box; during the trajectory state update process, the appearance feature library of the trajectory is smoothly updated using an exponential moving average strategy.

[0014] Furthermore, the observation noise covariance matrix of the Kalman filter is negatively correlated with the detection confidence level, and the value of the observation noise covariance matrix is ​​adjusted in real time according to the detection confidence level corresponding to each detection box.

[0015] Furthermore, the first branch uses a 3×3 depthwise separable convolution to extract local texture features of the vehicle, the second branch uses a 1×5 convolution to extract horizontal structural features of the vehicle, and the third branch uses a 5×1 convolution to extract vertical structural features of the vehicle.

[0016] The beneficial effects of this invention's improved BoT-SORT-based vehicle multi-target tracking method are as follows: By improving the vehicle re-identification network and motion state prediction, it can address the challenges of vehicle tracking in traffic monitoring scenarios. It not only employs an OSNet multi-scale network to extract global features but also designs a HeightWidthFeatureDepthwiseConv directional feature enhancement method to specifically strengthen horizontal and vertical structural features. Furthermore, it achieves deep fusion of multi-scale global features and directional enhancement features through residual connections, making vehicle identity features more distinctive and stable. In addition, it introduces noise-adaptive Kalman filtering based on detection confidence, dynamically adjusting the observation noise covariance matrix according to detection quality, enabling motion prediction to adapt to fluctuations in detection results and effectively mitigating trajectory drift caused by occlusion and false detections. Simultaneously, the improved re-identification and adaptive motion prediction are synergistically integrated into the BoTSORT two-stage association framework, allowing motion information and appearance information to complement each other during the matching process. This significantly improves the trajectory recovery and identity preservation capabilities after vehicle occlusion, effectively reducing the number of identity switching operations and providing more reliable technical support for vehicle tracking in complex traffic environments. Attached Figure Description

[0017] The present invention will now be described in further detail with reference to the accompanying drawings and specific implementation methods.

[0018] Figure 1 This is a schematic diagram of the process flow of a vehicle multi-target tracking method based on an improved BoT-SORT according to the present invention. Detailed Implementation

[0019] To make the technical solution of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] Example like Figure 1 As shown, a technical solution is provided: a vehicle multi-target tracking method based on improved BoT-SORT, comprising: Step 1: Target Detection and Confidence Grouping The system acquires traffic monitoring video streams and uses the YOLOX object detector to detect vehicle targets in each frame of the video stream. It outputs detection results including the center coordinates, width, height, and detection confidence of the detection box. The detection results are then divided into a high-confidence detection set and a low-confidence detection set based on the detection confidence. Specifically, only unmatched high-confidence detection boxes are used to generate new vehicle trajectories, while low-confidence detection boxes do not participate in the generation of new trajectories and are only used for trajectory matching in the subsequent second stage.

[0021] The process begins by uniformly encoding the acquired continuous video frames, representing the traffic monitoring video stream as follows: In the formula Let t represent the image at frame t. Each frame is input into the pre-trained YOLOX object detector. The detector extracts multi-scale vehicle features through the backbone network, then performs feature fusion and enhancement through the neck network. Finally, the detection head outputs the set of detection results for all vehicle targets in the current frame, as shown below:

[0022] In the formula, Let be the set of vehicle detection results for frame t, where each detection result corresponds to a five-dimensional vector, as follows:

[0023] In the formula, Let i be the detection result for the i-th vehicle target. and These are the x-coordinate and y-coordinate of the center of the detection frame, respectively. The width of the detection frame. The height of the detection frame, The confidence level for the vehicle target detection corresponding to the detection box, with a value ranging from 0 to 1; Then, based on the detection confidence level corresponding to each detection box... A single confidence threshold is set to divide the detection results into a high-confidence detection set and a low-confidence detection set. Detection boxes with a detection confidence greater than or equal to the threshold are assigned to the high-confidence detection set and used for high-confidence association and new trajectory generation in the first stage. Detection boxes with a detection confidence less than the threshold but greater than 0 are assigned to the low-confidence detection set and used only for low-confidence association in the second stage. Next, invalid detection boxes are filtered out from the two divided detection sets. Detection boxes with a width or height smaller than the preset minimum pixel value are removed to avoid false detection results that are too small from interfering with the subsequent association process. At the same time, the high-confidence detection set and the low-confidence detection set are stored in independent memory buffers, waiting for the subsequent two-stage association module to read and call them in sequence. After grouping and preprocessing all detection results in the current frame, the high-confidence detection set is matched with the existing trajectory set output from the previous frame in the first stage. Only the high-confidence detection boxes that fail to match after the first stage of association are initialized with new trajectories. They are assigned a globally unique tracking ID and the corresponding motion state vector and appearance feature library are initialized. All low-confidence detection boxes do not participate in the initialization process of new trajectories.

[0024] Step 2, Trajectory Prediction The motion state of the existing vehicle trajectory is described by an eight-dimensional state vector. The position and scale of the vehicle in the current frame are predicted by the state transition matrix to generate a prediction box. The observation noise covariance matrix is ​​dynamically adjusted according to the detection confidence output by the detector. Specifically, the eight-dimensional state vector consists of the target center x-coordinate, center y-coordinate, target width, target height, horizontal velocity, vertical velocity, width change rate, and height change rate. The motion state prediction of the vehicle in the current frame is completed through the state transition matrix.

[0025] First, all valid vehicle trajectories retained after the tracking of the previous frame are read. For each active trajectory, an eight-dimensional state vector is used to fully represent its motion state and scale change characteristics. This state vector is represented as follows: In the formula, The x-coordinate of the target center The vertical coordinate of the target center is... For target width, For the target height, For horizontal velocity, Vertical velocity, For the speed of width change, As the height change rate, this vector simultaneously covers the vehicle's planar displacement information and its own dynamic size change information, which can accurately adapt to the scaling phenomenon of vehicles due to changes in distance in traffic monitoring scenarios. Then, based on the linear state recursion rule of Kalman filtering, the state vector of the previous frame for each trajectory is calculated using a preset state transition matrix to derive the predicted state of the vehicle in the current frame. The specific calculation formula is as follows:

[0026] In the formula, Here is the state transition matrix. The vehicle motion state vector of frame t-1 is used to describe the linear evolution of the vehicle motion state over time. The position and scale components in the predicted state vector are extracted to generate the corresponding prediction box of the vehicle in the current frame. The prediction boxes of all trajectories are stored in the prediction result queue in order of trajectory ID. Next, the confidence score of the detection box corresponding to the trajectory to be matched is obtained. Based on this confidence score, the observation noise covariance matrix of the Kalman filter is adjusted in real time. First, the noise scaling factor is calculated as follows:

[0027] In the formula, To determine the detection confidence of the corresponding detection box, and based on this scaling factor and the preset baseline observation noise covariance matrix, the adaptively adjusted observation noise covariance matrix for the current frame is obtained, expressed as:

[0028] In the formula, This is the noise scaling factor. The preset benchmark observation noise covariance matrix is ​​used. When the detection confidence is high, the noise scaling factor approaches 1, the observation noise is small, and the state update process will rely more on the current detection result. When the detection confidence is low, the noise scaling factor grows exponentially, the observation noise increases accordingly, and the state update process will rely more on the prediction results of historical trajectories. Finally, the adjusted observation noise covariance matrix of each trajectory is bound to the corresponding prediction box one by one, and the status information of the trajectory is updated synchronously. After the motion state prediction and noise parameter adjustment of all active trajectories are completed, all prediction boxes are output to the subsequent camera motion compensation step for coordinate correction, providing accurate prior position information for subsequent two-stage association matching.

[0029] Step 3: Motion Compensation Correction ORB features are extracted from two consecutive frames of images and optical flow is performed. The camera motion affine matrix between the two consecutive frames is calculated using the RANSAC algorithm. The generated prediction box is corrected using the affine matrix to obtain the compensated prediction trajectory. Specifically, the matching features are obtained by sparse optical flow tracking based on the extracted ORB feature points. After removing the erroneous features using the RANSAC algorithm, the camera motion affine matrix is ​​calculated. The affine matrix is ​​then used to perform coordinate transformation on all predicted boxes to complete the compensation.

[0030] The process begins by acquiring grayscale images of the current frame and the previous frame. Rotation-invariant and scale-invariant ORB feature points are extracted from both grayscale images. Binary descriptors are calculated for each extracted feature point. A brute-force matching method is then used to perform preliminary matching of the feature descriptors between the two frames, resulting in an initial set of feature matching pairs. Subsequently, a sparse optical flow method is used to track and verify the feature points in the initial matching pairs. The optical flow displacement of each feature point between the two frames is calculated, and erroneous matching pairs with optical flow displacements exceeding a preset threshold are removed, resulting in the retained valid feature matching pairs. Next, the effective feature matching pairs are robustly estimated using the input RANSAC algorithm. Inlier feature pairs that conform to the global motion model are iteratively selected. Based on the selected inlier feature pairs, the affine matrix of camera motion between two consecutive frames is obtained. This matrix can completely represent the translation, rotation, and scaling motion of the camera between two frames. Subsequently, the obtained affine matrix is ​​used to perform coordinate transformation on all predicted boxes output in step 2 to correct the prediction position deviation caused by camera motion. The transformation process is expressed as follows:

[0031] In the formula, The original prediction box generated in step 2, The predicted bounding box after camera motion compensation. The camera motion matrix between two consecutive frames; Finally, all compensated prediction boxes are validated for validity. Invalid prediction boxes whose center coordinates exceed the image boundary or whose aspect ratio exceeds a reasonable range are removed. The validated compensated prediction boxes are bound to their corresponding trajectory IDs and stored sequentially in the compensation prediction result queue, thus completing the camera motion compensation process and providing accurate position information that has eliminated camera motion interference for the subsequent first-stage association matching.

[0032] Step 4: Feature Extraction and Orientation Enhancement The vehicle region image is cropped according to the detection box position and uniformly scaled to 256×256 resolution. The scaled image is then input into the OSNet network to extract multi-scale vehicle features. The multi-scale vehicle features are then input into HeightWidthFeatureDepthwiseConv for feature enhancement. Specifically, the HeightWidthFeatureDepthwiseConv feature enhancement setup has three parallel feature extraction branches. Each of the three branches uses a convolutional kernel of a different size to extract orientation-specific features from the input multi-scale vehicle features.

[0033] First, all high-confidence detection boxes from step 1 are traversed. Based on the center coordinates, width, and height information of each detection box, the corresponding vehicle region image is cropped from the original image of the current frame. Bilinear interpolation is used to uniformly scale all cropped vehicle region images to a resolution of 256×256 to ensure the consistency of the input image size. The scaled vehicle images are then input into the pre-trained and fine-tuned OSNet network. The network extracts local features, mesoscale features, large-scale features, and global features simultaneously through a multi-branch parallel convolutional structure, and outputs fused multi-scale vehicle features. These multi-scale features can effectively adapt to the appearance changes of vehicles at different distances and from different perspectives. Then, the multi-scale vehicle features output by OSNet are input into three parallel feature extraction branches for orientation-specific enhancement. The first branch uses a 3×3 depthwise separable convolution to process the input features and extract local texture features of the vehicle surface. The second branch uses a 1×5 horizontal convolution kernel to perform convolution operations, focusing on extracting structural features with horizontal orientation such as headlights, windows, bumpers, and horizontal body contours. The third branch uses a 5×1 vertical convolution kernel to perform convolution operations, focusing on extracting structural features with vertical orientation such as A-pillars, B-pillars, and vertical body contours. The parallel processing of the three branches can simultaneously capture fine-grained features of the vehicle in different directions, overcoming the limitation of traditional square convolutions that cannot specifically extract directional structural information. In addition, it includes concatenating the local texture features, horizontal structure features, and vertical structure features output from the three branches along the channel dimension to obtain the concatenated multi-directional fusion feature, represented as:

[0034] In the formula, Local texture features extracted for the square region branch. The horizontal structural features extracted for horizontal region branches. The vertical structural features extracted from the vertical region branches are then subjected to channel dimension interaction and compression through 1×1 convolution to eliminate redundant information between features of different branches, resulting in enhanced features after channel fusion. The 1×1 convolution achieves effective cross-channel fusion of features in different directions without changing the feature space size. Finally, the enhanced features after channel fusion are mapped one-to-one with the multi-scale vehicle features of the original OSNet output. After binding the trajectory ID with the detection box ID, they are stored in the feature buffer, waiting to be called in the subsequent residual fusion and identity feature generation steps to complete the entire vehicle feature extraction and orientation enhancement process, providing a more discriminative and stable feature foundation for subsequent appearance similarity calculation.

[0035] Step 5: Residual Fusion and Identity Feature Generation The multi-directional features obtained by feature enhancement are fused with the original multi-scale features output by OSNet through residual connections. The final vehicle identity features after fusion are then subjected to global average pooling and L2 normalization to generate a vehicle identity feature vector. Specifically, the three directional features of the HeightWidthFeatureDepthwiseConv feature enhancement output are first concatenated, and then the channel dimension features are fused through 1×1 convolution. The fused features are then residually connected to the original multi-scale features output by OSNet.

[0036] First, the enhanced features after channel fusion stored in step 4 are read along with the corresponding original OSNet multi-scale vehicle features. A residual connection structure is then constructed for feature fusion. The fusion process is represented as follows:

[0037] In the formula, For the final vehicle identification characteristics, This refers to the directional enhancement features after channel fusion. The residual connection is the original multi-scale feature output by OSNet. It not only fully preserves the global multi-scale semantic information extracted by OSNet, but also incorporates direction-specific fine-grained structural features. At the same time, it effectively avoids the gradient vanishing and feature degradation problems that may occur during the training of deep networks, so that the final feature has both global discriminativeness and local detail discrimination. Then, a global average pooling operation is performed on the final vehicle identity features obtained by fusion. The average value of each channel is calculated in the spatial dimension of the feature map, and the two-dimensional feature map is converted into a one-dimensional feature vector. The global average pooling operation can aggregate the global spatial information of the feature map, while significantly reducing the feature dimension, reducing the computational cost of subsequent similarity calculation, and avoiding the risk of overfitting caused by fully connected layers. In addition, the one-dimensional feature vector obtained by global average pooling is subjected to L2 normalization to make the Euclidean norm of the feature vector equal to 1. The normalized feature vector eliminates the influence of scale difference on similarity calculation, ensuring the accuracy and stability of subsequent appearance distance calculation based on cosine similarity, and making the identity features of different vehicles comparable in the same metric space. Finally, the normalized vehicle identity feature vector is bound one by one with the corresponding detection box ID and detection confidence, and stored in the identity feature buffer in sequence to complete the entire vehicle identity feature generation process. This provides accurate and highly discriminative identity feature basis for the appearance information cost calculation in the subsequent first-stage association matching.

[0038] Step 6: Two-stage association and trajectory management Based on motion and appearance information, an association cost matrix is ​​constructed. The Hungarian algorithm is used to perform the first stage matching of the trajectory with the high-confidence detection set. Then, the unmatched trajectory is matched with the low-confidence detection set. Based on the matching results, the trajectory status is updated, a new trajectory is generated, and an inactive trajectory is deleted. Finally, the vehicle tracking result is output. Specifically, in the first stage of matching, an association cost matrix including intersection-union distance and appearance distance is constructed, and the Hungarian algorithm is used to complete the matching of trajectory with high-confidence detection boxes. During the trajectory state update process, an exponential moving average strategy is used to smoothly update the appearance feature library of the trajectory.

[0039] The process involves reading the compensated predicted bounding box set obtained in step 3, the high-confidence detection set obtained in step 1, and the corresponding vehicle identity feature vector generated in step 5. Motion information cost and appearance information cost are calculated respectively. The motion information cost is calculated based on the intersection-union ratio (IoU) of the predicted bounding box and the high-confidence detection box to obtain the IoU distance. The appearance information cost is calculated based on the cosine similarity of the vehicle identity feature vector to obtain the appearance distance. A BoT-SORT association cost matrix is ​​constructed based on these two distances, calculated as follows:

[0040] In the formula, The IoU distance between the predicted bounding box and the high-confidence detection box. The cost matrix is ​​the cosine distance between vehicle identity feature vectors. The minimum value fusion strategy is adopted for this cost matrix. When the vehicle motion state changes suddenly and the motion information is unreliable, the appearance information can dominate the matching. When the vehicle is partially occluded and the appearance information is unreliable, the motion information can dominate the matching. The Hungarian algorithm is used to solve the cost matrix to complete the optimal matching between the trajectory and the high confidence detection box, and obtain the set of matched trajectories, the set of unmatched trajectories, and the set of unmatched high confidence detection boxes. Then, the unmatched trajectory set obtained in the first stage is matched with the low-confidence detection set obtained in step 1 in the second stage using IoU matching. Association is completed using only the intersection-union distance (IoU) to recover the trajectories of vehicles with short-term occlusion and those that were missed. For all successfully matched trajectories, their eight-dimensional motion state vector is first updated based on the corresponding detection results. Simultaneously, an exponential moving average strategy is used to update the trajectory's appearance feature library. The update formula is as follows:

[0041] In the formula, To smooth out fluctuations in single-frame appearance features, EMA updates can mitigate abrupt changes in appearance features caused by illumination variations or local occlusion. For high-confidence detection boxes that did not match in the first stage, a new vehicle trajectory is created and a globally unique tracking ID is assigned. Simultaneously, its motion state vector and appearance feature library are initialized. Low-confidence detection boxes do not participate in the generation of new trajectories throughout the process. For continuous... Trajectories that do not match any detection boxes are identified as inactive trajectories and removed from the active trajectory set to reduce computational complexity. Finally, the current information of all active trajectories is collected, and the vehicle tracking result of each frame is output, including the vehicle's unique ID, detection box coordinates, real-time vehicle motion trajectory, historical vehicle trajectory, and vehicle motion state information. At the same time, the updated active trajectory set, appearance feature library, and state parameters of the current frame are saved as input for trajectory prediction in the next frame, completing the closed loop of the entire tracking process and realizing continuous and stable tracking of vehicle targets in the video stream.

[0042] To illustrate the effectiveness of this method, we conducted the following experiment: Experimental Environment and Dataset Preparation Experimental environment setup: A PyTorch deep learning environment was set up to reproduce and run the BoT-SORT basic algorithm framework. Based on this, the improved OSNet vehicle re-identification network and NSA-KF motion state prediction model proposed in this paper were integrated to build a complete experimental testing environment.

[0043] Experimental Dataset Selection: The UA-DETRAC vehicle dataset was selected as the main experimental platform. This dataset is a typical vehicle detection and tracking dataset in traffic scenarios. It contains a large number of vehicle video sequences under different weather conditions, lighting conditions, and traffic densities, which can well reflect the vehicle tracking problem in real traffic environments.

[0044] Experimental Procedure Design Complete the preprocessing operations for the UA-DETRAC dataset, including frame extraction, annotation information parsing, and format conversion of the original video sequences; Following the official dataset partitioning standards, complete the partitioning of the training and test sets to ensure the consistency and comparability of the experimental data; The original BoT-SORT algorithm and the improved algorithm in this paper were trained respectively. During the training process, the dual loss function training strategy designed in this paper was used to optimize the re-identification network. Under the same testing conditions, the performance of each trained model was evaluated, and the experimental results were recorded and compared.

[0045] Selection of evaluation indicators The experiment mainly uses core evaluation metrics commonly used in the field of multi-target tracking to quantitatively evaluate the algorithm performance, specifically including: Multi-target tracking accuracy (MOTA): A comprehensive measure of the algorithm's target detection accuracy and trajectory tracking continuity; Identity Preservation Capability Index (IDF1): Evaluates the algorithm's ability to maintain the target identity; ID Switch Count: The total number of times the target identity was incorrectly switched during the tracking process.

[0046] Experimental Results and Analysis Through experimental comparative analysis, it was found that: The improved vehicle re-identification network proposed in this paper can effectively enhance the ability to distinguish vehicle identity features, significantly reduce identity switching problems caused by vehicle occlusion and similar appearance, significantly improve the IDF1 index, and greatly reduce the number of IDSwitches.

[0047] After introducing the NSA-KF motion state prediction method based on detection confidence, the stability of vehicle trajectory prediction is enhanced. In complex scenarios such as vehicle acceleration, deceleration, turning, and short-term occlusion, it can maintain a more continuous and accurate target trajectory, and the MOTA index is improved.

[0048] By integrating the improved vehicle re-identification module with the NSA-KF motion prediction module, the algorithm as a whole achieves a synergistic improvement in vehicle identity preservation and trajectory continuity, verifying the effectiveness and feasibility of the proposed improvement method.

[0049] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A vehicle multi-target tracking method based on improved BoT-SORT, characterized in that: S1. Acquire traffic monitoring video stream, use YOLOX target detector to perform vehicle target detection on each frame of the video stream, output detection results including the center coordinates, width, height and detection confidence of the detection box, and divide the detection results into high confidence detection set and low confidence detection set according to the detection confidence. S2. The motion state of the existing vehicle trajectory is described by an eight-dimensional state vector. The position and scale of the vehicle in the current frame are predicted by the state transition matrix to generate a prediction box. The observation noise covariance matrix is ​​dynamically adjusted according to the detection confidence output by the detector. S3. Extract ORB features from two consecutive frames and perform optical flow tracking. Calculate the camera motion affine matrix between the two consecutive frames using the RANSAC algorithm. Use the affine matrix to correct the generated prediction box to obtain the compensated prediction trajectory. S4. Crop the vehicle region image according to the detection box position and uniformly scale it to 256×256 resolution. Input the scaled image into the OSNet network to extract multi-scale vehicle features, and then input the multi-scale vehicle features into HeightWidthFeatureDepthwiseConv for feature enhancement. S5. The multi-directional features obtained by feature enhancement are fused with the original multi-scale features output by OSNet through residual connection. Global average pooling and L2 normalization are performed on the fused vehicle identity features to generate vehicle identity feature vector. S6. Construct an association cost matrix based on motion and appearance information. Use the Hungarian algorithm to perform a first-stage matching between the trajectory and the high-confidence detection set. Then, perform a second-stage matching between the unmatched trajectory and the low-confidence detection set. Based on the matching results, update the trajectory status, generate new trajectories, and delete inactive trajectories. Finally, output the vehicle tracking results.

2. The vehicle multi-target tracking method based on improved BoT-SORT according to claim 1, characterized in that: In step S1, only unmatched high-confidence detection boxes are used to generate new vehicle trajectories, while low-confidence detection boxes do not participate in the generation of new trajectories and are only used for trajectory matching in the subsequent second stage.

3. The vehicle multi-target tracking method based on improved BoT-SORT according to claim 1, characterized in that: In step S2, the eight-dimensional state vector consists of the target center x-coordinate, center y-coordinate, target width, target height, horizontal velocity, vertical velocity, width change rate, and height change rate. The motion state prediction of the vehicle in the current frame is completed through the state transition matrix.

4. A vehicle multi-target tracking method based on improved BoT-SORT according to claim 1, characterized in that: In step S3, sparse optical flow tracking is performed based on the extracted ORB feature points to obtain matching features. After removing the erroneous features using the RANSAC algorithm, the camera motion affine matrix is ​​calculated. The affine matrix is ​​then used to perform coordinate transformation on all predicted boxes to complete the compensation.

5. A vehicle multi-target tracking method based on improved BoT-SORT according to claim 1, characterized in that: In step S4, the HeightWidthFeatureDepthwiseConv feature enhancement is set with three parallel feature extraction branches. The three branches use convolution kernels of different sizes to extract orientation-specific features from the input multi-scale vehicle features.

6. A vehicle multi-target tracking method based on improved BoT-SORT according to claim 1, characterized in that: In step S5, the three directional features of the HeightWidthFeatureDepthwiseConv feature enhancement output are first concatenated, and then the channel dimension features are fused through 1×1 convolution. The fused features are then residually connected with the original multi-scale features output by OSNet.

7. A vehicle multi-target tracking method based on improved BoT-SORT according to claim 1, characterized in that: In step S6, the first-stage matching constructs an association cost matrix containing the intersection-union distance and appearance distance, and uses the Hungarian algorithm to complete the matching of the trajectory with the high-confidence detection box; during the trajectory state update process, the appearance feature library of the trajectory is smoothly updated using an exponential moving average strategy.

8. A vehicle multi-target tracking method based on improved BoT-SORT according to claim 1, characterized in that: The observation noise covariance matrix of Kalman filtering is negatively correlated with the detection confidence. The value of the observation noise covariance matrix is ​​adjusted in real time according to the detection confidence corresponding to each detection box.

9. A vehicle multi-target tracking method based on improved BoT-SORT according to claim 6, characterized in that: The first branch uses a 3×3 depthwise separable convolution to extract local texture features of the vehicle, the second branch uses a 1×5 convolution to extract horizontal structural features of the vehicle, and the third branch uses a 5×1 convolution to extract vertical structural features of the vehicle.

Images