A vehicle multi-target tracking system and method based on hybrid tracking
By employing a hybrid approach combining vehicle detection models and feature point tracking, the robustness and efficiency issues of multi-target vehicle tracking technology in complex scenarios are addressed, achieving efficient vehicle trajectory tracking.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- AI SUPER EYE TECH CO LTD
- Filing Date
- 2026-03-18
- Publication Date
- 2026-06-02
AI Technical Summary
Existing vehicle multi-target tracking technologies lack robustness in complex scenarios and struggle to balance tracking accuracy with computational efficiency.
The vehicle detection model obtains bounding boxes and performs correlation matching. The occlusion status determines whether to enable feature point tracking. If not enabled, the cached feature points are extracted and updated. If enabled, the position is predicted by the point tracking algorithm. The vehicle trajectory is generated by fusing the bounding boxes and feature point results.
It improves tracking robustness in complex scenarios while balancing vehicle tracking accuracy and computational efficiency.
Smart Images
Figure CN122135315A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of intelligent transportation, and in particular to a vehicle multi-target tracking system and method based on hybrid tracking. Background Technology
[0002] Multi-target vehicle tracking is a key technology in intelligent transportation and computer vision, providing core target trajectory and identity data support for downstream applications such as traffic flow statistics, violation detection, and autonomous driving decision-making. Currently, the mainstream technologies for solving multi-target vehicle tracking problems fall into two categories: one is the "detection + association" method based on bounding box matching, and the other is the feature point tracking method used throughout the entire process. The former relies on target detection boxes to complete inter-frame matching, while the latter achieves position prediction by tracking target feature points. The bounding box matching method, because it only relies on the spatial and appearance features of the detection boxes for matching, has high requirements for the stability of the detection results and is prone to trajectory breaks and identity confusion in occlusion and missed detection scenarios. On the other hand, the feature point tracking method requires complex calculations on all feature points in each frame, resulting in excessive computational consumption and failing to meet the real-time requirements of practical applications. Neither of these methods can simultaneously guarantee robustness and operational efficiency in complex scenarios.
[0003] At present, vehicle multi-target tracking technology has technical problems such as insufficient tracking robustness in complex scenarios and difficulty in balancing tracking accuracy and computational efficiency. Summary of the Invention
[0004] This application provides a vehicle multi-target tracking system and method based on hybrid tracking. It uses a vehicle detection model to detect vehicles in consecutive video frames, obtaining vehicle detection boxes for each frame. The current frame's detection box is then matched with historical detection boxes corresponding to the vehicle trajectory in the previous frame to obtain a matching result. Combining the matching result and the detection box occlusion status, it determines whether to enable feature point tracking for each vehicle trajectory. If not enabled, feature points are extracted from successfully matched detection boxes, and the cached feature point set is updated. If enabled, the current frame position of feature points is predicted using a point tracking algorithm based on the cached feature point set. The system integrates the detection box matching and feature point tracking results to generate candidate trajectories, outputting a globally unique vehicle ID and the corresponding vehicle trajectory. These techniques address the technical problems of insufficient tracking robustness in complex scenarios and the difficulty in balancing tracking accuracy and computational efficiency in existing vehicle multi-target tracking technologies. This achieves the technical effect of improving tracking robustness in complex scenarios while simultaneously considering vehicle tracking accuracy and computational efficiency.
[0005] This application provides a vehicle multi-target tracking system based on hybrid tracking, comprising: a vehicle detection module, used to acquire continuous video frame images and perform target detection on the continuous video frame images using a vehicle detection model to obtain vehicle detection boxes in each frame image; a detection box matching module, used to associate and match the vehicle detection box of the current frame with the historical detection boxes corresponding to the vehicle trajectory of the previous frame to obtain detection box matching results, the detection box matching results including matching success results and matching failure results; a hybrid tracking decision module, used to perform hybrid tracking decision on each vehicle trajectory based on the detection box matching results and the occlusion state of the vehicle detection boxes to determine whether to enable feature point tracking; a point acquisition module, used to extract feature points inside the successfully matched vehicle detection boxes and update the cached feature point set of the corresponding trajectory when feature point tracking is not enabled; a point tracking module, used to predict the position of feature points in the current frame based on the cached feature point set using a point tracking algorithm when feature point tracking is enabled; and a trajectory clustering and identity matching module, used to fuse the detection box matching results and feature point tracking results to generate candidate trajectories, and use the candidate trajectories to output a globally unique vehicle ID and vehicle trajectory.
[0006] In a possible implementation, the system may also include a trajectory management module, which performs lifecycle management of trajectory creation, updating, and state switching.
[0007] In a possible implementation, the vehicle detection box matching module associates and matches the vehicle detection box of the current frame with the historical detection box corresponding to the vehicle trajectory of the previous frame, including: calculating the cross-union ratio similarity and appearance feature similarity between the vehicle detection box of the current frame and the historical detection box; constructing a matching cost matrix based on the cross-union ratio similarity and the appearance feature similarity; and completing the association matching between the vehicle detection box and the historical detection box using the Hungarian algorithm based on the matching cost matrix.
[0008] In a possible implementation, the hybrid tracking decision module performs hybrid tracking decision for each vehicle trajectory to determine whether to enable feature point tracking, including: enabling feature point tracking when the detection box matching result is a failed match; enabling feature point tracking in advance when the detection box matching result is a successful match and the vehicle detection box is determined to be occluded; and disabling feature point tracking and performing vehicle detection box matching tracking when the detection box matching result is a successful match and the vehicle detection box is determined to be unoccluded.
[0009] In a possible implementation, in the point acquisition module, when feature point tracking is not enabled, extracting feature points within the successfully matched vehicle detection box and updating the cached feature point set of the corresponding trajectory includes: performing image preprocessing operations on the vehicle detection box region, the image preprocessing operations including grayscale processing, Gaussian noise reduction, and contrast enhancement; extracting multiple feature points distributed on the vehicle surface within the vehicle detection box region after image preprocessing; and updating the cached feature point set using the feature points when conditions such as vehicle scale change, vehicle orientation angle change, or preset time interval are met.
[0010] In a possible implementation, in the point tracking module, when feature point tracking is enabled, predicting the position of feature points in the current frame based on the cached feature point set using a point tracking algorithm includes: predicting the position of feature points in the current frame based on the cached feature point set of the previous frame using optical flow integration; filtering the predicted feature points based on feature point matching confidence and removing drifting feature points; and updating the cached feature point set with the spatial distribution of the remaining valid feature points.
[0011] In a possible implementation, the trajectory clustering and identity matching module is further used to: when the detection box matching result is a successful match and the vehicle detection box is determined to be in an unoccluded state, directly use the vehicle detection box coordinates and keep the identity unchanged; when the detection box matching result is a successful match and the vehicle detection box is determined to be in an occluded state, perform weighted correction on the vehicle detection box position based on the feature point tracking result; when the detection box matching result is a failed match, if the number of valid feature points is greater than a preset threshold, generate a virtual tracking box based on the spatial distribution of feature points and update the vehicle trajectory; when the detection box matching result is a failed match, if the number of valid feature points is less than or equal to a preset threshold, mark the vehicle trajectory as lost.
[0012] This application also provides a vehicle multi-target tracking method based on hybrid tracking, comprising: acquiring continuous video frame images, and performing target detection on the continuous video frame images using a vehicle detection model to obtain vehicle detection boxes in each frame image; associating and matching the vehicle detection boxes of the current frame with historical detection boxes corresponding to the vehicle trajectory of the previous frame to obtain detection box matching results, wherein the detection box matching results include matching success results and matching failure results; performing hybrid tracking decision on each vehicle trajectory based on the detection box matching results and the occlusion state of the vehicle detection boxes to determine whether to enable feature point tracking; when feature point tracking is not enabled, extracting feature points inside the successfully matched vehicle detection boxes and updating the cached feature point set of the corresponding trajectory; when feature point tracking is enabled, predicting the position of the feature points in the current frame using a point tracking algorithm based on the cached feature point set; fusing the detection box matching results and feature point tracking results to generate candidate trajectories, and using the candidate trajectories to output a globally unique vehicle ID and vehicle trajectory.
[0013] This application proposes a vehicle multi-target tracking system and method based on hybrid tracking. The system acquires continuous video frame images through a vehicle detection module and performs target detection on these frames using a vehicle detection model, obtaining vehicle detection boxes in each frame. A detection box matching module associates and matches the vehicle detection boxes in the current frame with historical detection boxes corresponding to the vehicle trajectory in the previous frame, obtaining matching results. These matching results include successful and unsuccessful matches. Based on the matching results and the occlusion status of the vehicle detection boxes, a hybrid tracking decision module performs hybrid tracking decisions on each vehicle trajectory to determine whether feature point tracking should be enabled. When feature point tracking is not enabled, a point acquisition module extracts feature points within successfully matched vehicle detection boxes and updates the cached feature point set of the corresponding trajectory. When feature point tracking is enabled, a point tracking module predicts the position of feature points in the current frame using a point tracking algorithm based on the cached feature point set. A trajectory clustering and identity matching module fuses the detection box matching results and feature point tracking results to generate candidate trajectories. Finally, the candidate trajectories are used to output a globally unique vehicle ID and vehicle trajectory. Through the above process, the system and method proposed in this application achieve the technical effect of improving tracking robustness in complex scenarios while taking into account vehicle tracking accuracy and computational processing efficiency. Attached Figure Description
[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments of the present invention will be briefly described below. Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from these processes.
[0015] Figure 1 This is a schematic diagram of a vehicle multi-target tracking system based on hybrid tracking, provided as an embodiment of this application.
[0016] Figure 2 This is a flowchart illustrating a vehicle multi-target tracking method based on hybrid tracking, provided in an embodiment of this application.
[0017] Explanation of reference numerals in the attached diagram: Vehicle detection module 10, detection box matching module 20, hybrid tracking decision module 30, point acquisition module 40, point tracking module 50, trajectory clustering and identity matching module 60. Detailed Implementation
[0018] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.
[0019] This application provides a vehicle multi-target tracking system based on hybrid tracking, such as... Figure 1 As shown, the system includes:
[0020] The vehicle detection module 10 is used to acquire continuous video frame images and perform target detection on the continuous video frame images through a vehicle detection model to obtain vehicle detection boxes in each frame image.
[0021] Specifically, continuous video frames of road traffic scenes are captured at a fixed frame rate using image acquisition devices such as high-definition surveillance cameras and vehicle-mounted cameras. The captured video frames are then input into a pre-trained vehicle detection model in chronological order. This model uses deep neural network models such as the YOLO series and Faster R-CNN. The input to the model is standardized image data. The model extracts features and performs classification regression through network structures such as convolutional layers, pooling layers, and fully connected layers. It outputs the detection box coordinates of each vehicle target in each frame of the image. The coordinates are represented in the form of a quadruple of the top-left corner x-coordinate, top-left corner y-coordinate, bottom-right corner x-coordinate, and bottom-right corner y-coordinate, with the top-left corner as the origin. The model also outputs the confidence score of the detection box and a 512-dimensional appearance feature vector. Furthermore, it calculates the intersection-union ratio between detection boxes and determines the occlusion state of the detection boxes by combining the position of the vehicle in the image. Finally, all detection results are classified and organized according to vehicle targets to form a vehicle detection box dataset for each frame.
[0022] The detection box matching module 20 is used to associate and match the vehicle detection box of the current frame with the historical detection box corresponding to the vehicle trajectory of the previous frame to obtain the detection box matching result, which includes the matching success result and the matching failure result.
[0023] Specifically, the association between the vehicle detection box in the current frame and the corresponding historical detection box of the vehicle trajectory in the previous frame is established to complete the preliminary matching of vehicle targets between frames. Two results are output: successful matching and unsuccessful matching. Successful matching means that the detection boxes in the previous and next frames correspond to the same vehicle target. Unsuccessful matching means that the detection box in the current frame does not have a corresponding historical detection box in the previous frame, or the historical detection box in the previous frame does not have a corresponding detection box in the current frame.
[0024] In one possible implementation, the vehicle detection box matching module 20 performs association matching between the vehicle detection box of the current frame and the historical detection box corresponding to the vehicle trajectory of the previous frame, including: calculating the cross-union ratio similarity and appearance feature similarity between the vehicle detection box of the current frame and the historical detection box; constructing a matching cost matrix based on the cross-union ratio similarity and the appearance feature similarity; and completing the association matching between the vehicle detection box and the historical detection box using the Hungarian algorithm based on the matching cost matrix.
[0025] Specifically, the cross-union ratio (CUP) similarity and appearance feature similarity are calculated between the vehicle detection box in the current frame and the historical detection box in the previous frame. When calculating the CUP similarity, the areas of the intersection and union regions of the two detection boxes are first calculated, and then the area of the intersection region is divided by the area of the union region to obtain the CUP value. The value ranges from 0 to 1, and the closer the value is to 1, the higher the spatial overlap between the two detection boxes. When calculating the appearance feature similarity, the cosine similarity algorithm is used to calculate the cosine similarity of the 512-dimensional appearance feature vectors corresponding to the two detection boxes to obtain the appearance feature similarity value. The value ranges from 0 to 1, and the closer the value is to 1, the more similar the vehicle appearance features corresponding to the two detection boxes are. A matching cost matrix is constructed based on the intersection-union similarity (CI / UN) and appearance feature similarity. CI / UN similarity is assigned a weight of 0.7, and appearance feature similarity is assigned a weight of 0.3. The two similarity values are multiplied by their respective weights and summed to obtain the comprehensive similarity. The comprehensive similarity is then subtracted from 1 to obtain the matching cost. Rows in the matching cost matrix correspond to vehicle detection boxes in the current frame, and columns correspond to historical detection boxes in the previous frame. Each element in the matrix represents the matching cost for the corresponding detection box pair; a smaller cost value indicates a higher matching degree. Based on the constructed matching cost matrix, the Hungarian algorithm is used to perform allocation calculations. The algorithm traverses the elements in the matrix and, according to the minimum cost allocation principle, matches each current frame detection box with a unique historical detection box from the previous frame. After allocation, a matching result is considered successful if there is a corresponding match, and a matching result is considered unsuccessful if there is no corresponding match. The final output contains the detection box matching results, including the matching status of all detection box pairs.
[0026] The hybrid tracking decision module 30 is used to perform hybrid tracking decision for each vehicle trajectory based on the detection box matching result and the occlusion state of the vehicle detection box, so as to determine whether to enable feature point tracking.
[0027] Specifically, based on the matching results output by the detection box matching module 20 and combined with the occlusion state of the vehicle detection box, a decision is made individually for each existing vehicle trajectory to determine whether the feature point tracking method needs to be enabled for that trajectory, thereby realizing the on-demand switching between detection box matching tracking and feature point tracking.
[0028] In one possible implementation, the hybrid tracking decision module 30 performs hybrid tracking decision-making for each vehicle trajectory to determine whether to enable feature point tracking, including: enabling feature point tracking when the detection box matching result is a matching failure; enabling feature point tracking in advance when the detection box matching result is a matching success and the vehicle detection box is determined to be in an occluded state; and disabling feature point tracking and performing vehicle detection box matching tracking when the detection box matching result is a matching success and the vehicle detection box is determined to be in an unoccluded state.
[0029] Specifically, each active vehicle trajectory in the system is traversed, and the detection box matching result and vehicle detection box occlusion status for each trajectory are obtained one by one. The occlusion status is pre-determined by the vehicle detection module 10, for example, based on an intersection-union ratio (IU) greater than 0.6 between detection boxes and the vehicle being relatively far from the camera. If the detection box matching result of a trajectory fails, a feature point tracking enable command is directly triggered to start the feature point tracking process for that trajectory. If the detection box matching result of a trajectory succeeds, but the corresponding vehicle detection box is determined to be occluded, a preventative start operation is performed to enable feature point tracking for that trajectory in advance to avoid subsequent occlusion aggravation leading to tracking failure. If the detection box matching result of a trajectory succeeds, and the corresponding vehicle detection box is determined to be unoccluded, and if feature point tracking has been enabled for that trajectory before, a feature point tracking disable command is immediately triggered to stop the feature point tracking process for that trajectory, retaining only the detection box matching tracking method. If it has not been enabled before, the detection box matching tracking method continues to be maintained. Finally, a clear decision result for enabling or disabling feature point tracking is generated for each trajectory.
[0030] The point acquisition module 40 is used to extract feature points within the successfully matched vehicle detection box and update the cached feature point set of the corresponding trajectory when feature point tracking is not enabled.
[0031] Specifically, when feature point tracking is not enabled, feature points are extracted from the successfully matched vehicle detection box regions. At the same time, the cached feature point set of the corresponding vehicle trajectory is dynamically updated according to the vehicle's motion and shape changes, so as to prepare effective feature point data for feature point tracking that may be enabled later.
[0032] In one possible implementation, when feature point tracking is not enabled, the point acquisition module 40 extracts feature points within the successfully matched vehicle detection box and updates the cached feature point set of the corresponding trajectory by: performing image preprocessing operations on the vehicle detection box region, including grayscale processing, Gaussian noise reduction, and contrast enhancement; extracting multiple feature points distributed on the vehicle surface within the vehicle detection box region after image preprocessing; and updating the cached feature point set using the feature points when conditions such as vehicle scale change, vehicle orientation angle change, or preset time interval are met.
[0033] Specifically, vehicle trajectories for which feature point tracking is not enabled in the current system are identified. For the successfully matched vehicle detection bounding boxes corresponding to these trajectories, image preprocessing operations are performed sequentially. First, the image within the detection box is converted to grayscale by using a weighted average method to convert the red, green, and blue channel pixel values of the color image into single-channel grayscale values. Then, Gaussian noise reduction is applied to the grayscale image, filtering noise through Gaussian convolution operations. Finally, contrast enhancement is performed using a histogram equalization algorithm to stretch the dynamic range of grayscale values and improve image contrast. Feature points are extracted within the preprocessed detection bounding box regions using ORB, SIFT, and other feature point extraction algorithms. The number of extracted feature points is controlled between 25 and 50 to ensure that the feature points are evenly distributed in key areas of the vehicle surface, such as the front, body, and wheels. The system monitors the vehicle's motion and shape in real time to determine whether the update conditions for the cached feature point set are met. These conditions include changes in vehicle scale, changes in vehicle orientation angle, or preset time intervals, such as changes in vehicle scale greater than 10%, changes in vehicle orientation angle greater than 15°, a distance greater than 5 frames from the last feature point sampling, or the vehicle detection box being determined to be occluded. If any one of these conditions is met, the extracted feature points are replaced or added to the cached feature point set for the corresponding trajectory. During replacement, points with drifted positions or invalid features are removed from the set. During addition, the total number of feature points in the set is maintained between 25 and 50, thus completing the dynamic update of the cached feature point set.
[0034] The point tracking module 50 is used to predict the position of feature points in the current frame based on the cached feature point set using a point tracking algorithm when feature point tracking is enabled.
[0035] Specifically, when feature point tracking is enabled, the position of the feature point in the current frame is predicted by the point tracking algorithm based on the cached feature point set of the corresponding trajectory. At the same time, the prediction results are filtered and optimized, and the cached feature point set is updated to ensure the accuracy and continuity of feature point tracking.
[0036] In one possible implementation, in the point tracking module 50, when feature point tracking is enabled, predicting the position of a feature point in the current frame based on the cached feature point set using a point tracking algorithm includes: predicting the position of the feature point in the current frame based on the cached feature point set of the previous frame using optical flow integration; filtering the predicted feature points according to the feature point matching confidence and removing drifting feature points; and updating the cached feature point set with the spatial distribution of the remaining valid feature points.
[0037] Specifically, the system obtains the cached feature point set from the previous frame corresponding to the vehicle trajectory with feature point tracking enabled. This set contains the coordinates and feature descriptor of each feature point. Based on this set, an optical flow integral algorithm is used to predict the position of the feature point in the current frame. The optical flow integral algorithm first calculates the optical flow field between adjacent frames to obtain the instantaneous velocity of each feature point, then integrates the velocity over time to obtain the displacement of the feature point from the previous frame to the current frame. Combined with the coordinates of the feature points from the previous frame, the predicted coordinates for the current frame are calculated. The matching confidence score for each predicted feature point is calculated. The confidence score is determined comprehensively based on indicators such as the stability of the feature point's optical flow vector and the matching degree of its feature descriptor. A confidence score threshold is set, and predicted feature points with confidence scores below the threshold are identified as drifting feature points and removed. Valid feature points with confidence scores above the threshold are retained. The cached feature point set is updated based on the spatial distribution of valid feature points. First, points in the set that deviate too much from the current valid feature point position are removed. Then, a small number of new feature points are added in the visible area of the vehicle to ensure that the feature points are evenly distributed. At the same time, the total number of feature points in the set is maintained between 25 and 50. The update of the cached feature point set is completed, providing basic data for feature point tracking in the next frame.
[0038] The trajectory clustering and identity matching module 60 is used to fuse the detection box matching results and feature point tracking results to generate candidate trajectories, and to output a globally unique vehicle ID and vehicle trajectory using the candidate trajectories.
[0039] Specifically, the matching results of the detection box matching module 20 and the tracking results of the point tracking module 50 are fused to generate vehicle candidate trajectories. Then, based on the candidate trajectories, a globally unique vehicle identity identifier is assigned and output for each vehicle target, while the corresponding vehicle trajectory is updated to ensure the consistency of the vehicle target's identity and the continuity of its trajectory in consecutive frames.
[0040] In one possible implementation, the trajectory clustering and identity matching module 60 is further configured to: when the detection box matching result is a successful match and the vehicle detection box is determined to be in an unoccluded state, directly use the vehicle detection box coordinates and keep the identity unchanged; when the detection box matching result is a successful match and the vehicle detection box is determined to be in an occluded state, perform weighted correction on the vehicle detection box position based on the feature point tracking result; when the detection box matching result is a failed match, if the number of valid feature points is greater than a preset threshold, generate a virtual tracking box based on the spatial distribution of feature points and update the vehicle trajectory; when the detection box matching result is a failed match, if the number of valid feature points is less than or equal to a preset threshold, mark the vehicle trajectory as lost.
[0041] Specifically, the system iterates and counts the detection box matching results, occlusion status, and number of valid feature points for all vehicle trajectories. The number of valid feature points is determined by the point tracking module 50 after filtering, with a preset threshold of 10. Different cases are then handled as follows: First, if the trajectory detection box matching result is successful and the vehicle detection box is unoccluded, the coordinates of the current frame detection box obtained from the detection box matching are directly used as the current position of the trajectory, keeping the globally unique vehicle identity identifier assigned to the trajectory unchanged, and the vehicle trajectory is updated based on these coordinates. Second, if the trajectory detection box matching result is successful but the vehicle detection box is occluded, the coordinates of the valid feature points output by the point tracking module 50 are first obtained, the center position of the valid feature points is calculated, and weights are assigned to the detection box coordinates and the feature point center positions respectively. The detection points are then weighted and summed to update the detection points. In the first case, if the bounding box position is corrected and the corrected coordinates are used as the current position of the trajectory, the vehicle identification remains unchanged, and the vehicle trajectory is updated. In the second case, if the trajectory detection box matching result fails and the number of valid feature points is greater than a threshold, the valid feature points are spatially clustered. The minimum bounding rectangle algorithm is used to find the minimum bounding rectangle of all valid feature points, and this rectangle is used as the virtual tracking box. Simultaneously, the prior position of the vehicle motion model obtained from Kalman filtering is fused, and the coordinates of the virtual tracking box are fine-tuned. The fine-tuned virtual tracking box coordinates are used as the current position of the trajectory, the vehicle identification remains unchanged, a new candidate trajectory is generated, and the vehicle trajectory is updated. In the third case, if the trajectory detection box matching result fails and the number of valid feature points is less than or equal to a threshold, no tracking box is generated, the vehicle trajectory is directly marked as lost, and the trajectory coordinates are not updated. Finally, all processed trajectories are integrated, and a globally unique vehicle identification is assigned to the newly generated vehicle trajectory. The identification uses a numerical sequence, for example, starting from 1 and incrementing sequentially, ensuring that the identification of each vehicle target is unique throughout the tracking process. Finally, the globally unique identification of all vehicles and the corresponding complete vehicle trajectory are output.
[0042] In one possible implementation, the system further includes a trajectory management module for performing lifecycle management of trajectory creation, updating, and state switching.
[0043] Specifically, full lifecycle management is performed on all vehicle trajectories in the system, including: Trajectory creation: When the detection box matching module 20 detects a new detection box in the current frame that has no corresponding historical detection box from the previous frame, and the confidence level of this detection box is higher than a preset threshold, a new vehicle trajectory is created for the vehicle target corresponding to this detection box, a globally unique vehicle identity is assigned, the trajectory's cached feature point set, motion model parameters, and other data are initialized, and the trajectory status is marked as active. Trajectory update: For active trajectories, based on the current frame position coordinates output by the trajectory clustering and identity matching module 60, the coordinates are added to the trajectory data in chronological order, and the trajectory's motion model parameters, cached feature point set, and other related data are updated. The system performs trajectory status switching operations: It monitors the tracking status of each trajectory in real time. If an active trajectory fails to achieve a valid match for 10 consecutive frames and point tracking fails, the trajectory status is switched from active to deleted, and all data for that trajectory is removed from the system. If an active trajectory experiences a brief matching failure but point tracking is effective, or the number of effective feature points is less than or equal to a threshold, the trajectory status is switched from active to lost, retaining the trajectory's historical data and identifier. If a lost trajectory successfully achieves a detection box match or point tracking recovers, the trajectory status is switched from lost to active, and trajectory data continues to be updated. Throughout this process, the system traverses and checks all trajectories at fixed frame intervals to ensure the real-time performance and accuracy of trajectory lifecycle management.
[0044] This application embodiment uses a vehicle detection model to detect vehicles in continuous video frames and obtain vehicle detection boxes for each frame. It then performs association matching between the current frame detection box and the historical detection boxes corresponding to the vehicle trajectory in the previous frame to obtain matching results. Combining the matching results and the occlusion status of the detection boxes, it decides whether to enable feature point tracking for each vehicle trajectory. If not enabled, feature points are extracted from the successfully matched detection boxes and the cached feature point set is updated. If enabled, the current frame position of the feature points is predicted based on the cached feature point set using a point tracking algorithm. The detection box matching and feature point tracking results are fused to generate candidate trajectories, and a globally unique vehicle ID and corresponding vehicle trajectory are output. These technical means solve the technical problems of insufficient tracking robustness in complex scenarios and difficulty in balancing tracking accuracy and computational efficiency in existing multi-target vehicle tracking technologies. This achieves the technical effect of improving tracking robustness in complex scenarios while taking into account vehicle tracking accuracy and computational efficiency.
[0045] In the above text, refer to Figure 1 A vehicle multi-target tracking system based on hybrid tracking according to an embodiment of the present invention is described in detail. Next, reference will be made to... Figure 2 This invention describes a vehicle multi-target tracking method based on hybrid tracking according to an embodiment of the present invention.
[0046] According to an embodiment of the present invention, a vehicle multi-target tracking method based on hybrid tracking is used to solve the technical problems of insufficient tracking robustness in complex scenarios and difficulty in balancing tracking accuracy and computational efficiency in existing vehicle multi-target tracking technologies. The method achieves the technical effect of improving tracking robustness in complex scenarios while taking into account vehicle tracking accuracy and computational efficiency. A vehicle multi-target tracking method based on hybrid tracking includes: acquiring continuous video frame images, and performing target detection on the continuous video frame images using a vehicle detection model to obtain vehicle detection boxes in each frame image; associating and matching the vehicle detection boxes of the current frame with historical detection boxes corresponding to the vehicle trajectory of the previous frame to obtain detection box matching results, the detection box matching results including matching success results and matching failure results; performing hybrid tracking decision on each vehicle trajectory based on the detection box matching results and the occlusion state of the vehicle detection boxes to determine whether to enable feature point tracking; when feature point tracking is not enabled, extracting feature points inside the successfully matched vehicle detection boxes and updating the cached feature point set of the corresponding trajectory; when feature point tracking is enabled, predicting the position of the feature points in the current frame using a point tracking algorithm based on the cached feature point set; fusing the detection box matching results and feature point tracking results to generate candidate trajectories, and using the candidate trajectories to output a globally unique vehicle ID and vehicle trajectory.
[0047] The method may further include: lifecycle management of execution trajectory creation, updating, and state switching.
[0048] The process of associating and matching the vehicle detection box in the current frame with the historical detection box corresponding to the vehicle trajectory in the previous frame can further include: calculating the intersection-union ratio (IU) similarity and appearance feature similarity between the vehicle detection box in the current frame and the historical detection boxes; constructing a matching cost matrix based on the IU and appearance feature similarities; and completing the association and matching of the vehicle detection box and the historical detection boxes using the Hungarian algorithm based on the matching cost matrix.
[0049] Specifically, performing a hybrid tracking decision for each vehicle trajectory to determine whether to enable feature point tracking may further include: enabling feature point tracking when the detection box matching result is a failed match; enabling feature point tracking in advance when the detection box matching result is a successful match and the vehicle detection box is determined to be occluded; and disabling feature point tracking and performing vehicle detection box matching tracking when the detection box matching result is a successful match and the vehicle detection box is determined to be unoccluded.
[0050] When feature point tracking is not enabled, extracting feature points within the successfully matched vehicle detection box and updating the cached feature point set of the corresponding trajectory may further include: performing image preprocessing operations on the vehicle detection box region, including grayscale processing, Gaussian noise reduction, and contrast enhancement; extracting multiple feature points distributed on the vehicle surface within the vehicle detection box region after image preprocessing; and updating the cached feature point set using the feature points when conditions such as vehicle scale change, vehicle orientation angle change, or preset time interval are met.
[0051] When feature point tracking is enabled, predicting the position of feature points in the current frame using a point tracking algorithm based on the cached feature point set can further include: predicting the position of feature points in the current frame using optical flow integration based on the cached feature point set of the previous frame; filtering the predicted feature points based on the feature point matching confidence and removing drifting feature points; and updating the cached feature point set with the spatial distribution of the remaining valid feature points.
[0052] The process of generating candidate trajectories by fusing detection box matching results and feature point tracking results, and outputting a globally unique vehicle ID and vehicle trajectory using the candidate trajectories, can further include: when the detection box matching result is a successful match and the vehicle detection box is determined to be in an unoccluded state, the vehicle detection box coordinates are directly used and the identity remains unchanged; when the detection box matching result is a successful match and the vehicle detection box is determined to be in an occluded state, the vehicle detection box position is weighted and corrected according to the feature point tracking results; when the detection box matching result is a failed match, if the number of valid feature points is greater than a preset threshold, a virtual tracking box is generated according to the spatial distribution of feature points and the vehicle trajectory is updated; when the detection box matching result is a failed match, if the number of valid feature points is less than or equal to a preset threshold, the vehicle trajectory is marked as lost.
[0053] The vehicle multi-target tracking system based on hybrid tracking provided in this embodiment of the invention can execute the vehicle multi-target tracking method based on hybrid tracking provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0054] Although this application makes various references to certain modules in the system according to the embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy distinction between each other and are not used to limit the scope of protection of this invention.
[0055] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.
Claims
1. A vehicle multi-target tracking system based on hybrid tracking, characterized in that, The system includes: The vehicle detection module is used to acquire continuous video frame images and perform target detection on the continuous video frame images through a vehicle detection model to obtain the vehicle detection box in each frame image. The detection box matching module is used to associate and match the vehicle detection box in the current frame with the historical detection box corresponding to the vehicle trajectory in the previous frame, and obtain the detection box matching result, which includes the matching success result and the matching failure result. The hybrid tracking decision module is used to perform hybrid tracking decision for each vehicle trajectory based on the detection box matching result and the occlusion state of the vehicle detection box, so as to determine whether to enable feature point tracking. The point acquisition module is used to extract feature points within the successfully matched vehicle detection box and update the cached feature point set of the corresponding trajectory when feature point tracking is not enabled. The point tracking module is used to predict the position of feature points in the current frame based on the cached feature point set and the point tracking algorithm when feature point tracking is enabled. The trajectory clustering and identity matching module is used to fuse detection box matching results and feature point tracking results to generate candidate trajectories, and outputs a globally unique vehicle ID and vehicle trajectory using the candidate trajectories.
2. The vehicle multi-target tracking system based on hybrid tracking as described in claim 1, characterized in that, The system also includes: The trajectory management module is used to perform lifecycle management of trajectory creation, updates, and status switching.
3. The vehicle multi-target tracking system based on hybrid tracking as described in claim 1, characterized in that, In the detection box matching module, the association and matching of the vehicle detection box in the current frame with the historical detection boxes corresponding to the vehicle trajectory in the previous frame includes: Calculate the intersection-union similarity and appearance feature similarity between the vehicle detection bounding box in the current frame and the historical detection bounding boxes; Construct a matching cost matrix based on the intersection-union similarity and the appearance feature similarity; Based on the matching cost matrix, the Hungarian algorithm is used to complete the association matching between vehicle detection boxes and historical detection boxes.
4. A vehicle multi-target tracking system based on hybrid tracking as described in claim 1, characterized in that, In the hybrid tracking decision module, hybrid tracking decision is performed for each vehicle trajectory to determine whether feature point tracking should be enabled, including: When the detection box matching result is a matching failure result, feature point tracking is enabled; When the detection box matching result is a successful match and the vehicle detection box is determined to be in an occluded state, feature point tracking is enabled in advance. When the detection box matching result is a successful match and the vehicle detection box is determined to be in an unoccluded state, feature point tracking is turned off and vehicle detection box matching tracking is performed.
5. A vehicle multi-target tracking system based on hybrid tracking as described in claim 1, characterized in that, In the feature point acquisition module, when feature point tracking is not enabled, feature points are extracted within the successfully matched vehicle detection bounding boxes, and the cached feature point set of the corresponding trajectory is updated, including: Image preprocessing operations are performed on the vehicle detection box region, including grayscale conversion, Gaussian noise reduction, and contrast enhancement. Multiple feature points distributed on the vehicle surface are extracted within the vehicle detection bounding box area after image preprocessing. When conditions such as changes in vehicle dimensions, changes in vehicle orientation angle, or preset time intervals are met, the cached feature point set is updated using the feature points.
6. A vehicle multi-target tracking system based on hybrid tracking as described in claim 1, characterized in that, In the point tracking module, when feature point tracking is enabled, the position of feature points in the current frame is predicted using a point tracking algorithm based on the cached feature point set, including: Based on the cached feature point set of the previous frame, the position of the feature point in the current frame is predicted by optical flow integration; Predicted feature points are filtered based on feature point matching confidence, and drifting feature points are removed. Update the cached feature point set by updating the spatial distribution of the remaining valid feature points.
7. A vehicle multi-target tracking system based on hybrid tracking as described in claim 1, characterized in that, The trajectory clustering and identity matching module is also used for: When the detection box matching result is a successful match and the vehicle detection box is determined to be in an unobstructed state, the vehicle detection box coordinates are used directly and the identity remains unchanged. When the detection box matching result is a successful match and the vehicle detection box is determined to be in an occluded state, the position of the vehicle detection box is weighted and corrected according to the feature point tracking result. When the detection box matching result is a matching failure result, if the number of valid feature points is greater than the preset number threshold, a virtual tracking box is generated according to the spatial distribution of feature points and the vehicle trajectory is updated. When the detection box matching result is a matching failure result, if the number of valid feature points is less than or equal to a preset number threshold, the vehicle trajectory is marked as lost.
8. A vehicle multi-target tracking method based on hybrid tracking, characterized in that, The method is implemented by a vehicle multi-target tracking system based on hybrid tracking as described in any one of claims 1-7, and the method includes: Acquire continuous video frame images, and perform target detection on the continuous video frame images using a vehicle detection model to obtain vehicle detection boxes in each frame image; The vehicle detection box in the current frame is associated and matched with the historical detection box corresponding to the vehicle trajectory in the previous frame to obtain the detection box matching result, which includes the matching success result and the matching failure result. Based on the detection box matching results and the occlusion state of the vehicle detection box, a hybrid tracking decision is performed on each vehicle trajectory to determine whether feature point tracking should be enabled. When feature point tracking is not enabled, feature points are extracted from the successfully matched vehicle detection boxes and the cached feature point set of the corresponding trajectory is updated. When feature point tracking is enabled, the position of the feature points in the current frame is predicted using a point tracking algorithm based on the cached feature point set. The candidate trajectory is generated by fusing the detection box matching result and the feature point tracking result, and the globally unique vehicle ID and vehicle trajectory are output using the candidate trajectory.