Large frame spacing traffic object tracking method fusing consistency measure and motion prediction

Abstract

The large frame spacing traffic target tracking method fuses the consistency measure and motion prediction, and comprises the following steps: step 1, setting initial parameters of a traffic target tracking system; step 2, acquiring visible light images continuously shot by a vehicle-mounted monocular camera and performing size normalization, obtaining a target cluster to be matched in the images after pretreatment, inputting the two into a target matching module, and obtaining a successful matching set and a mismatching set of front and rear images; step 3, inputting the successful matching set and the mismatching set into a target tracking module, performing matching recovery of the mismatching set on the preceding occluded target, and performing tracking target id assignment on the generated successful matching set; step 4, inputting the target cluster to be matched into a target merging module; step 5, adding a second matching mismatching set of the current image frame into an occlusion queue, updating the latest storage lost target in the occlusion queue, and displaying the mismatching id of the current frame in a visualization result. The application greatly reduces the deployment cost and meets the needs of most mobile terminals.

Description

Technical Field

[0001] This invention belongs to the field of vehicle navigation and positioning technology, specifically relating to a large-frame-space traffic target tracking method that integrates consistency measurement and motion prediction. Background Technology

[0002] Target tracking technology is currently a hot research topic both domestically and internationally. It refers to the technology of continuously tracking and locating targets in video or image sequences by extracting and matching their features. It is widely used in video surveillance, autonomous driving, drones, robotics, and other fields. Large frame spacing traffic target tracking based on vehicle-mounted monocular cameras is a particularly challenging task with complex scenarios and high real-time requirements.

[0003] Currently available traffic target tracking technologies mainly achieve this through the following methods:

[0004] 1. Color Model-Based Target Tracking: For example, patent number CN103985139A describes a method that first acquires an image sequence of the current environment using a monocular camera. It then selects a suitable color model for the target, commonly RGB or HSV. By acquiring sample images of the target, its color features are extracted, and a color model is established. This color model is then used to detect the target's location within the video or image sequence. Based on the target's color model, the method tracks the target's position changes across consecutive frames to achieve target tracking.

[0005] 2. Feature-point-based target tracking: For example, patent number CN106846377A. This type of method first acquires an image sequence of the current environment using a monocular camera. In the target image, feature extractors are used to extract identifiable feature points, such as corner points and edge points. The extracted feature points are then described using feature descriptors; common methods include SIFT, SURF, and ORB. Next, the most similar feature points to the target feature points in the current image frame are searched in consecutive frames; common matching algorithms include nearest neighbor matching and RANSAC. Based on the matched feature points, target tracking is achieved by calculating changes in their positions.

[0006] All of the above applications have certain shortcomings.

[0007] Among them, the target tracking method based on the color model is more sensitive to lighting conditions and background interference. Especially in outdoor tracking tasks such as traffic targets, changes in brightness caused by day and night and sunshine and rain will have a significant impact on the tracking results and cannot meet the accuracy requirements of traffic target tracking.

[0008] Feature-point-based target tracking methods are not only affected by changes in outdoor lighting, but also suffer from problems such as target occlusion leading to tracking failure. Complex road conditions in outdoor traffic often cause the tracked target to be occluded by obstacles, making it difficult for feature-point-based methods to guarantee continuous target tracking in real-world scenarios with sudden changes in occlusion.

[0009] Therefore, it is urgent to solve the problem of how to design a traffic target tracking method based on a vehicle-mounted monocular camera that has low deployment cost, low requirements for image quality, strong real-time performance, and takes into account the characteristics of the target itself. Summary of the Invention

[0010] To overcome the shortcomings of existing technologies, this invention aims to provide a large-frame-space traffic target tracking method that integrates consistency metrics and motion prediction. Based on a vehicle-mounted monocular camera, it calculates consistency metrics for target images, such as structural similarity, to define the geometric constraints of the tracking image. Based on statistical analysis of different tracking target metrics, the Hungarian algorithm is used to obtain the maximum matching set, and the motion direction is estimated. When tracking at a constant speed, Kalman filtering is used to assist in stable target tracking. Simultaneously, target boxes are merged after the first matching, and a second matching is performed on the merged boxes to improve matching accuracy. Based on the final result set, the system will provide a notification when the tracked target is lost. This significantly reduces deployment costs and meets the needs of most mobile devices.

[0011] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0012] A large-frame-space traffic target tracking method that integrates consistency measurement and motion prediction includes the following steps;

[0013] Step 1: Set the initial parameters for the traffic target tracking system to define and initialize system operation;

[0014] Step 2: Based on the frame interval (fps) set in the initial system parameters in Step 1, acquire the visible light images continuously captured by the vehicle-mounted monocular camera and normalize their size. After YOLO preprocessing, obtain the target clusters to be matched in the images. Input the previous and next frames into the target matching module to obtain the successful matching set and mismatch set of the previous and next images.

[0015] Step 3: Input the successful matching set and mismatch set described in Step 2 into the target tracking module, perform matching and recovery of the preceding occluded target in the image in the mismatch set, if the recovery is successful, it is included in the successful matching set, if it fails, it becomes the preset occluded target of this frame, and assign the tracking target ID to the updated successful matching set.

[0016] Step 4: Input target merging module for the target cluster to be matched described in Step 2;

[0017] The target merging module first performs an internal distance judgment on the target cluster. It merges any two boxes whose center coordinates <(cx1,cy1),(cx2,cy2)> are less than the target box merging threshold parameter close or which overlap, generating a larger target box to indicate that the synthesized target boxes should theoretically be one. The successfully merged front and back frame boxes are then passed to the target matching module to obtain the successful matching set and mismatch set of the front and back image merged boxes. For the successfully matched merged boxes, the tracking target IDs of the sub-boxes of the front and back boxes are changed to be consistent. If the merged box of the next frame does not have a corresponding registered object, the box ID is updated, while ensuring the consistency of the tracking target IDs of each sub-box.

[0018] Step 5: Add the set of mismatches in the secondary matching of this image frame to the occlusion queue, update the latest stored lost target in the occlusion queue, and display the mismatch ID of this frame in the visualization results.

[0019] Furthermore, in step 1, the initial parameters include the interval frame number fps, the initial tracking statistic flag, the tracking target aspect ratio change threshold parameter t1, the tracking target displacement threshold parameter t2, the matching algorithm threshold parameter conf, the initial occlusion queue sq, the target box merging threshold parameter close, and the maximum occlusion frame interval parameter gap.

[0020] The frame interval (fps) determines the time interval between frames when processing image sequences, affecting the real-time performance and processing speed of tracking. The initial tracking statistic (flag) is used to record and control statistical information during the tracking process. The target aspect ratio change threshold parameter (t1) sets the maximum allowable value for the aspect ratio change of the target in consecutive frames, used to filter target pairs that do not meet the aspect ratio change condition. The target displacement threshold parameter (t2) sets the maximum allowable value for the displacement change of the target in consecutive frames, used to determine whether the targets are on the same trajectory. The matching algorithm threshold parameter (conf) is used to determine the confidence threshold for successful matching, affecting the strictness of the matching process. The initial occlusion queue (sq) is used to store target information that may be occluded during tracking, for subsequent occlusion recovery processing. The target box merging threshold parameter (close) is used to determine whether the target boxes in consecutive frames are close enough to require merging. The maximum occlusion frame interval parameter (gap) sets the maximum number of frame intervals at which the system attempts to resume tracking after the target is occluded.

[0021] In step 1, the default values ​​for each parameter are: interval frame count fps = 0, initial tracking statistic flag = 0, target aspect ratio change threshold parameter t1 = 2, target displacement threshold parameter t2 = 80, matching algorithm threshold parameter conf = 0.15, initial occlusion queue sq = list(), target box merging threshold parameter close = 40, and maximum occlusion frame interval parameter gap = 1.

[0022] Furthermore, the workflow of the target matching module used in step 2 is as follows:

[0023] Step 2.1: Input the images of the previous and next frames, the target cluster to be matched, and the matching parameters set in Step 1. Compare the target pairs in the two images one by one. Set geometric constraints according to the target aspect ratio change threshold parameter t1 and the target displacement threshold parameter t2 to filter and prune the target pairs. That is, it is necessary to ensure that the aspect ratio change of the two target pairs to be calculated is within t1 and the pixel displacement of the box is within t2. Otherwise, the target pair will definitely not match and the value will be set to 0.

[0024] Step 2.2: Calculate the consistency measure of target images such as Structural Similarity (SSIM) for target pairs with filtering conditions. At the same time, use the Kalman filter algorithm to predict the position of the current frame based on the previous tracking trajectory, i.e. the movement path of the same bounding box ID in the previous frame. Calculate the IOU distance between the target position and the preset target pair in the current frame. The two calculated values ​​are weighted according to the current aspect ratio change to obtain the cost threshold of the target pair. The initial weight is set to 0.5.

[0025] Step 2.3: Combine the cost thresholds from Step 2.2 into a cost matrix, use the Hungarian algorithm for matching, retain the matching pairs with confidence scores greater than the matching algorithm threshold, and feed them back into the Kalman filter algorithm for updating, to obtain the optimal estimate of the current frame, and return the successful matching set and the mismatch set.

[0026] The workflow of the Kalman filter algorithm described in step 2.2 is as follows:

[0027] 1) Initialization: Set the initial state and covariance matrix of the system;

[0028] 2) Prediction (time update): Based on the dynamic model of the system, the next state of the system is predicted through the state transition equation, and the covariance matrix of the state is updated.

[0029] 3) Update (measurement update): Based on the observation data, calculate the residual between the predicted state and the actual observation, i.e., innovation. Then, by calculating the Kalman gain, combine the residual with the uncertainty of the predicted state to obtain the corrected state estimate and covariance matrix.

[0030] 4) Repeat the prediction and update steps: Repeat the prediction and update steps to estimate the state of the system in a recursive manner.

[0031] The workflow of the Hungarian algorithm described in step 2.3 is as follows:

[0032] 1) Create a cost matrix: Construct an n×n cost matrix according to the task, where n represents the maximum number of target boxes in the previous and next frames. Each element in the matrix represents the cost required to match a target box in the previous frame to a target box in the next frame. The cost matrix in this task is the combination of the consistency metric passed in step 2.2 and the cost thresholds calculated by the Kalman algorithm.

[0033] 2) Row subtraction: For each row of the weighted cost matrix, find the minimum value of the row and subtract the minimum value from each element of the row; this ensures that each row has at least one zero element.

[0034] 3) Column subtraction: For the weighted cost matrix that has been subtracted from the minimum value of a row, find the minimum value of each column and subtract the minimum value from each element of that column; this ensures that each column has at least one zero element.

[0035] 4) Marking zero elements: In the weighted cost matrix after subtracting the minimum row and column values, find all zero elements and mark them;

[0036] 5) Attempt to assign tasks: Starting from any unmarked zero element, attempt to assign tasks to executors. If there are already other assigned tasks in the row or column where the zero element is located, recursively try. If an unmarked zero element is found and a task is successfully assigned, mark the zero element and the other zero elements in its row and column.

[0037] 6) Check the matching results: Check whether the matching results meet the requirements. If the number of matched tasks is equal to the total number of tasks, the algorithm ends; otherwise, further adjustments are needed.

[0038] 7) Adjust the cost matrix: For unassigned rows and assigned columns, find their minimum values ​​and subtract them from the other elements of the cost matrix. For assigned rows and unassigned columns, find their minimum values ​​and add them to the other elements of the weighted cost matrix.

[0039] 8) Return to step 4): Return to step 4) and continue the process of marking and attempting to assign tasks until all tasks are assigned, and obtain an optimal target box matching scheme that minimizes the total cost.

[0040] Furthermore, the target tracking module used in step 3 has the following workflow:

[0041] Step 3.1: Input the successful matching set and mismatch set of the previous and next frame images, and assign the same tracking ID to the target pairs that match successfully;

[0042] Step 3.2: For the unregistered tracking target in the next frame, first match it with the target in the occlusion queue to recover the lost target, which is the same as step 2, but does not require geometric constraint filtering and pruning.

[0043] Step 3.3: If the rematch calculation value is greater than the threshold conf, it is considered an occluded target and assigned the same tracking ID; otherwise, it is a new tracking target and assigned a new ID.

[0044] Furthermore, the workflow of the target merging module used in step 4 is as follows:

[0045] Step 4.1: Input the target clusters to be matched in the image from Step 2, traverse the target boxes that have not yet been merged, compare them one by one with the remaining target boxes in the image, and determine whether they should be merged.

[0046] Step 4.2: The merging judgment condition is whether the center coordinates of the two current boxes <(cx1,cy1),(cx2,cy2)> are less than the target box merging threshold parameter close, and the nearest distance on the target box is compared with the target box merging threshold parameter close for filtering and pruning.

[0047] Step 4.3: Add the merged result to the set of remaining target boxes in the image. After traversing the set, return the set of target boxes to be matched after merging the image.

[0048] Furthermore, in step 5, the system will display all successfully matched and unmatched target boxes in the visualization results and label the corresponding tracking IDs; when the system loses a target during the tracking process, it will provide a prompt so that the user or the system can take appropriate measures.

[0049] The traffic target tracking method can be applied to tasks such as autonomous driving, smart mobility, or digital mapping.

[0050] The beneficial effects of this invention are:

[0051] This invention solves the problems of heavy reliance on image quality pixels, poor real-time performance, and high deployment costs in traditional traffic target tracking. This method only requires RGB images from an onboard monocular camera, greatly reducing deployment costs and meeting the needs of most mobile devices. The computational complexity of this method mainly lies in the target matching module, target tracking module, and target merging module of the preceding and following frames. In step 2, the target matching module uses the YOLO model for target detection to quickly obtain the target clusters to be matched. Then, it quickly calculates the similarity between target pairs using consistency metrics such as Structural Similarity Scale (SSIM), and then uses the Hungarian algorithm for matching. This algorithm is an efficient bipartite graph matching algorithm that can quickly find the optimal matching scheme. Step 2.1 uses geometric constraint filtering pruning, which pre-filters impossible-to-match target pairs by setting geometric constraints (such as aspect ratio change threshold parameter t1 and displacement threshold parameter t2), reducing the burden of subsequent computation. The Kalman filter mentioned in step 2.2 is used to predict the target's position in the next frame. This is a recursive estimation algorithm that can quickly provide the optimal estimate of the target's position in sequential data. Step 3, the target tracking module, performs matching and recovery of occluded targets in the mismatch set. If the recovery is successful, it quickly updates the set to a successful matching set; otherwise, it marks the target as an occluded object. This step improves the robustness of the algorithm. Step 4, the target merging module, reduces the number of target IDs by merging similar target boxes, improving matching accuracy and reducing the complexity of subsequent processing. All of the above designs demonstrate that the method is fast, accurate, and can meet the real-time traffic target tracking requirements. Furthermore, when the system loses a tracked target, it provides a prompt, indicating that the system can quickly identify and respond to tracking failures, facilitating timely remedial measures.

[0052] This invention employs an image matching method, requiring only real-time image data to perform traffic target detection and real-time target tracking, without the need for a dedicated motion sensor. The camera's current relative motion can be determined by analyzing the trajectory of the tracked target. Attached Figure Description

[0053] Figure 1 This is a flowchart of the method of the present invention.

[0054] Figure 2 This is a flowchart of the target matching module of the present invention.

[0055] Figure 3 This is a flowchart of the target tracking module of the present invention.

[0056] Figure 4 This is a flowchart of the target merging module of the present invention.

[0057] Figure 5 This is a schematic diagram of the actual tracking effect of the two frames before and after the present invention. Figure 1The number next to the target bounding box in the image is the ID of the target being tracked.

[0058] Figure 6 This is a schematic diagram of the actual tracking effect of the two frames before and after the present invention. Figure 2 The number next to the target bounding box in the image is the ID of the target being tracked. Detailed Implementation

[0059] The present invention will now be described in further detail with reference to the accompanying drawings.

[0060] This invention provides a traffic target tracking method based on consistency measurement and motion prediction, the workflow of which is as follows: Figure 1 As shown.

[0061] First, set the initial system parameters, including the interval frame count fps, the initial tracking statistics flag, the target aspect ratio change threshold parameter t1, the target displacement threshold parameter t2, the matching algorithm threshold parameter conf, the initial occlusion queue sq, the target box merging threshold parameter close, and the maximum occlusion frame interval parameter, etc.

[0062] Then the traffic target tracking system starts working, with a monocular camera acquiring visible light images and the mobile device's YOLO model obtaining the target clusters to be matched.

[0063] The bounding boxes are traversed and paired between consecutive frames. First, geometric constraints are applied for pruning and filtering. Then, consistency metrics and Intersection over Union (IOU) distance are calculated, and the Hungarian algorithm is used to match the bounding box pairs. Successfully matched bounding box pairs are added to the successful matching set; otherwise, they are added to the mismatch set.

[0064] Simultaneously, the target bounding box clusters to be matched in the current frame are merged, the merged bounding boxes from the previous and next frames are matched, and the results are updated to the successful matching set and the mismatch set.

[0065] Finally, the tracking IDs of the target boxes in the next frame of the successfully matched set are synchronized, new tracking IDs are assigned to the target boxes in the next frame of the mismatched set, and the target boxes in the previous frame of the mismatched set are added to the occlusion queue and updated.

[0066] The workflow of the target matching module is as follows: Figure 2 As shown.

[0067] The module takes two frames of images and their target clusters as input, compares the target pairs in the two images one by one, first performs pruning filtering by geometric constraints such as tracking changes in the aspect ratio and displacement range of the target. If they do not match, the value is set to 0 to indicate that they do not match. Otherwise, the consistency measure between the two is calculated.

[0068] The previous frame's bounding box is used to predict its position in the current frame using Kalman filtering. Its Interchange of Union (IOU) value is calculated with the next frame's bounding box. A weighted consistency metric based on the current aspect ratio change is used to obtain the cost matrix for the bounding box pairs. The Hungarian algorithm is then used to match the bounding box pairs. Successfully matched bounding box pairs are added to the successful matching set; otherwise, they are added to the mismatch set. The estimated optimal result is fed back into the Kalman filter to update its parameters.

[0069] The target tracking module workflow is as follows: Figure 3 As shown. The module takes into account the successful matching set, mismatch set, and global occlusion queue of the preceding and following frames.

[0070] The algorithm iterates through the target bounding boxes in subsequent frames to check for successful matches. If a match is found, the tracking target ID from the previous frame is synchronized. Otherwise, a secondary match is performed with targets in the occlusion queue, which involves calculating the cost matrix and using the Hungarian algorithm for matching. If the matching recovery accuracy is greater than the matching algorithm threshold `conf`, it indicates that the occluded target has been lost and recovered, and the target bounding box in the subsequent frame is synchronized with the tracking target ID of the lost bounding box in the previous frame. If the secondary match fails, it indicates that this target is a new target, and a new tracking target ID is assigned.

[0071] The workflow of the target merging module is as follows: Figure 4 As shown, the module takes the target clusters to be matched in the current frame image as input, traverses the target boxes, and compares them one by one with the set of remaining target boxes in the image.

[0072] First, determine if the nearest distance between the target boxes is greater than close*0.8. If it is less than 0.8, it indicates that the boxes are small and far traffic targets and are removed and filtered out.

[0073] For boxes that meet the criteria, the module checks if the center coordinates of the two current boxes are less than the target box merging threshold (close). If they are, the boxes need to be merged into a new box and added to the traversal sequence; otherwise, merging is not required. After traversal, the module returns a set of merged target boxes to be matched for subsequent calls.

[0074] Example:

[0075] 1. Initialize system parameters: Set the interval frame count (fps), initial tracking statistics (flag), aspect ratio change threshold parameter (t1), displacement threshold parameter (t2), matching algorithm threshold parameter (conf), initial occlusion queue (sq), target box merging threshold parameter (close), and maximum occlusion frame interval parameter (gap);

[0076] 2. Target detection and matching: First, a vehicle-mounted monocular camera is used to capture continuous visible light images. The images are preprocessed using the YOLO model to obtain target clusters to be matched. Then, the target matching module is used to match the images of consecutive frames to generate a successful matching set and a mismatch set.

[0077] 3. Target tracking: Successfully matched targets are assigned the same tracking ID to achieve continuous tracking. For mismatch sets, check for occlusion and attempt to resume tracking.

[0078] 4. Target merging: For multiple detected similar target boxes, the target merging module merges them to improve tracking accuracy;

[0079] 5. Occlusion handling: If the target is occluded, the system will use the occlusion queue to track and recover the occluded target;

[0080] 6. Tracking result update: Update the occlusion queue, record the currently lost target information, and assign a new tracking ID to newly detected targets or targets that failed to be recovered;

[0081] 7. Visual Display: The visual interface displays all successfully matched and unmatched bounding boxes, labeled with their corresponding tracking IDs. Figure 5 This demonstrates how the system accurately tracks each target in unobstructed conditions. Figure 6 This demonstrates how the system maintains tracking in the presence of occlusion by merging and handling occlusion queues. The upper part of the diagram shows the target tracking effect in the previous frame, and the lower part shows the target tracking effect in the next frame. When the target trajectory changes, due to the change in the target detection results, the traffic targets that were originally tracked independently are merged and tracked synchronously. This not only improves the tracking accuracy but also facilitates the visualization of the results, showing that they are a cluster of synchronously changing traffic targets.

[0082] 8. Real-time feedback: If the system loses the target during tracking, it will provide real-time prompts so that intervention or adjustments can be made;

[0083] 9. Looping Processing: The system will run continuously, constantly processing new image data, updating tracking results, and displaying them in real time.

[0084] The actual tracking visualization of the two frames before and after this invention is as follows: Figure 5 As shown, the number next to the target box in the image is the ID of the tracked target.

[0085] in Figure 5 This is the result of traffic camera tracking recorded by a vehicle-mounted camera in a practical application.

[0086] The actual merging and tracking effect of the two frames in this invention is as follows: Figure 6 As shown, the number next to the target box in the image is the ID of the tracked target.

[0087] in Figure 6The upper part of the diagram shows the target tracking effect in the previous frame, and the lower part shows the target tracking effect in the next frame. When the target trajectory changes, due to the change in the target detection result, the traffic targets that were originally tracked independently are merged and tracked synchronously. This is beneficial to improving the tracking accuracy and also to displaying the visualization results, showing that they are a cluster of synchronously changing traffic targets.

Claims

1. A large-frame-interval traffic target tracking method that integrates consistency measurement and motion prediction, characterized in that, Includes the following steps; Step 1: Set the initial parameters for the traffic target tracking system to define and initialize system operation; Step 2: Based on the frame interval (fps) set in the initial system parameters in Step 1, acquire the visible light images continuously captured by the vehicle-mounted monocular camera and normalize their size. After YOLO preprocessing, obtain the target clusters to be matched in the images. Input the images of the previous and next frames into the target matching module to obtain the successful matching set and mismatch set of the previous and next images. Step 3: Input the successful matching set and mismatch set described in Step 2 into the target tracking module, perform matching and recovery of the preceding occluded target in the image in the mismatch set, if the recovery is successful, it is included in the successful matching set, if it fails, it becomes the preset occluded target of the current frame, and assign the tracking target ID to the updated successful matching set. Step 4: Pass the target clusters to be matched described in Step 2 to the target merging module; The target merging module first performs an internal distance judgment on the target cluster. It merges any two boxes whose center coordinates <(cx1,cy1),(cx2,cy2)> are less than the target box merging threshold parameter close or which overlap, generating a larger target box to indicate that the synthesized target boxes should theoretically be one. The successfully merged front and back frame boxes are then passed to the target matching module to obtain the successful matching set and mismatch set of the front and back image merged boxes. For the successfully matched merged boxes, the tracking target IDs of the sub-boxes of the front and back boxes are changed to be consistent. If the merged box of the next frame does not have a corresponding registered object, the box ID is updated, while ensuring the consistency of the tracking target IDs of each sub-box. Step 5: Add the set of mismatches in the secondary matching of this image frame to the occlusion queue, update the latest stored lost target in the occlusion queue, and display the mismatch ID of this frame in the visualization results; In step 1, the initial parameters include the interval frame number fps, the initial tracking statistic flag, the tracking target aspect ratio change threshold parameter t1, the tracking target displacement threshold parameter t2, the matching algorithm threshold parameter conf, the initial occlusion queue sq, the target box merging threshold parameter close, and the maximum occlusion frame interval parameter gap. The workflow of the target matching module used in step 2 is as follows: Step 2.1: Input the previous and next frame images, the target cluster to be matched, and the matching parameters set in Step 1. Compare the target pairs in the two images one by one. Set geometric constraints according to the target aspect ratio change threshold parameter t1 and the target displacement threshold parameter t2 to filter and prune the target pairs. If the aspect ratio change of the target box in the previous and next frames is within t1 and the pixel displacement of the box is within t2, the target pair will definitely not match and the value will be set to 0. Step 2.2: Calculate the consistency measure of the structural similarity (SSIM) target image for the target pair through the filtering conditions. At the same time, use the Kalman filter algorithm to predict the position of the current frame based on the previous tracking trajectory, that is, the movement path of the same bounding box ID in the previous frame. Calculate the IOU distance between the target position and the preset target pair in the current frame. The two calculated values ​​are weighted according to the current aspect ratio change to obtain the cost threshold of the target pair. The initial weight is set to 0.

5. Step 2.3: Combine the cost thresholds from Step 2.2 into a cost matrix, use the Hungarian algorithm for matching, retain the matching pairs with confidence scores greater than the matching algorithm threshold, and feed them back into the Kalman filter algorithm for updating, to obtain the optimal estimate of the current frame, and return the successful matching set and the mismatch set.

2. The large-frame-interval traffic target tracking method according to claim 1, characterized in that, The frame interval (fps) determines the time interval between frames when processing image sequences, affecting the real-time performance and processing speed of tracking; the initial tracking statistic (flag) is used to record and control statistical information during the tracking process; the target aspect ratio change threshold parameter (t1) sets the maximum allowable value for the target's aspect ratio change in consecutive frames, used to filter target pairs that do not meet the aspect ratio change condition; the target displacement threshold parameter (t2) sets the maximum allowable value for the target's displacement change in consecutive frames, used to determine whether the targets are on the same trajectory; the matching algorithm threshold parameter (conf) is used to determine the confidence threshold for successful matching, affecting the strictness of the matching process. The initial occlusion queue sq is used to store target information that may be occluded during the tracking process, so as to facilitate subsequent occlusion recovery processing; The target bounding box merging threshold parameter `close` is used to determine whether the target bounding boxes in consecutive frames are close enough to be merged. The maximum occlusion frame interval parameter gap indicates the maximum number of frame intervals at which the system attempts to resume tracking after the target is occluded.

3. The large-frame-interval traffic target tracking method according to claim 2, characterized in that, In step 1, the default values ​​for each parameter are: interval frame count fps=0, initial tracking statistic flag=0, target aspect ratio change threshold parameter t1=2, target displacement threshold parameter t2=80, matching algorithm threshold parameter conf=0.15, initial occlusion queue sq=list(), target box merging threshold parameter close=40, and maximum occlusion frame interval parameter gap=1.

4. The large-frame-interval traffic target tracking method according to claim 1, characterized in that, The workflow of the Kalman filter algorithm described in step 2.2 is as follows: 1) Initialization: Set the initial state and covariance matrix of the system; 2) Prediction: Based on the dynamic model of the system, the next state of the system is predicted through the state transition equation, and the covariance matrix of the state is updated. 3) Update: Based on the observation data, calculate the residual between the predicted state and the actual observation. Then, by calculating the Kalman gain, combine the residual with the uncertainty of the predicted state to obtain the corrected state estimate and covariance matrix. 4) Repeat the prediction and update steps: Repeat the prediction and update steps to estimate the state of the system in a recursive manner.

5. The large-frame-interval traffic target tracking method according to claim 1, characterized in that, The workflow of the Hungarian algorithm described in step 2.3 is as follows: 1) Create a cost matrix: Construct an n×n cost matrix according to the task, where n represents the maximum number of target boxes in the previous and next frames. Each element in the matrix represents the cost required to match a target box in the previous frame to a target box in the next frame. The cost matrix in this task is the combination of the consistency metric passed in step 2.2 and the cost thresholds calculated by the Kalman algorithm. 2) Row subtraction: For each row of the weighted cost matrix, find the minimum value of that row and subtract that minimum value from each element of that row; this ensures that each row has at least one zero element. 3) Column subtraction: For the weighted cost matrix after subtracting the minimum value of each row, find the minimum value of each column and subtract the minimum value from each element of that column; this ensures that each column has at least one zero element. 4) Marking zero elements: In the weighted cost matrix after subtracting the minimum row and column values, find all zero elements and mark them; 5) Attempt to assign tasks: Starting from any unmarked zero element, attempt to assign tasks to executors. If there are already other assigned tasks in the row or column where the zero element is located, recursively try. If an unmarked zero element is found and a task is successfully assigned, mark the zero element and the other zero elements in its row and column. 6) Check the matching results: Check whether the matching results meet the requirements. If the number of matched tasks is equal to the total number of tasks, the algorithm ends; otherwise, further adjustments are needed. 7) Adjust the cost matrix: For unassigned rows and assigned columns, find their minimum values ​​and subtract them from the other elements of the cost matrix. For assigned rows and unassigned columns, find their minimum values ​​and add them to the other elements of the weighted cost matrix. 8) Return to step 4): Return to step 4) and continue the process of marking and attempting to assign tasks until all tasks are assigned and an optimal target box matching scheme is obtained, which minimizes the total cost.

6. The large-frame-interval traffic target tracking method according to claim 1, characterized in that, The target tracking module used in step 3 has the following workflow: Step 3.1: Input the successful matching set and mismatch set of the previous and next frame images, and assign the same tracking ID to the target pairs that match successfully; Step 3.2: For the unregistered tracking target in the next frame, first match it with the target in the occlusion queue to recover the lost target, which is the same as step 2, but does not require geometric constraint filtering and pruning. Step 3.3: If the rematch calculation value is greater than the threshold conf, it is considered an occluded target and assigned the same tracking ID; otherwise, it is a new tracking target and assigned a new ID.

7. The large-frame-interval traffic target tracking method according to claim 1, characterized in that, The workflow of the target merging module used in step 4 is as follows: Step 4.1: Input the target clusters to be matched in the image from Step 2, traverse the target boxes that have not yet been merged, compare them one by one with the remaining target boxes in the image, and determine whether they should be merged. Step 4.2: The merging judgment condition is whether the center coordinates of the two current boxes <(cx1,cy1),(cx2,cy2)> are less than the target box merging threshold parameter close, and the nearest distance on the target box is compared with the target box merging threshold parameter close for filtering and pruning. Step 4.3: Add the merged result to the set of remaining target boxes in the image. After traversing the set, return the set of target boxes to be matched after merging the image.

8. The large-frame-interval traffic target tracking method according to claim 1, characterized in that, In step 5, the system will display all successfully matched and unmatched target boxes in the visualization results and label the corresponding tracking IDs; when the system loses a target during the tracking process, it will provide a prompt so that the user or the system can take appropriate measures.

9. The application of the large-frame-interval traffic target tracking method according to any one of claims 1-8, characterized in that, The traffic target tracking method can be applied to tasks such as autonomous driving, smart mobility, or digital mapping.

Images