Video Tracklet Splitting and Merging for Fewer ID Switches
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object tracking technologies, such as ByteTrack, suffer from ID switches where different objects are incorrectly assigned to the same tracklet, necessitating post-processing to improve tracking purity and accuracy.
Innovation Solution
An information processing device and method that calculates similarity sets for tracklet frames, determines splitting points based on predefined conditions, splits tracklets into sub-segments, and merges non-overlapping sub-segments to improve tracking purity, using KL distance, slope ratios, and average differences to enhance accuracy and speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If post-processing is applied to correct ID switches in tracking, then tracking purity is improved, but processing time increases and real-time performance deteriorates
Solution Approach 1:
The patent divides tracklets into multiple sub-segments based on splitting points detected through appearance feature analysis. This segmentation allows the system to process only relevant portions of tracklets, reducing overall processing time while maintaining tracking purity by correctly identifying and separating objects that undergo ID switches.
Solution Approach 2:
The patent performs preliminary splitting of tracklets into sub-segments before merging operations. By pre-identifying splitting points using appearance features and motion information, the system prepares the data structure in advance, enabling faster merging operations and reducing total post-processing time while improving tracking accuracy.
2Measurement precision
If complex post-processing operations are performed to improve tracking accuracy, then ID switch correction is enhanced, but computational complexity increases
Solution Approach 1:
The patent segments tracklets into sub-segments based on detected splitting points, allowing complex merging operations to be performed on smaller, more manageable units. This reduces the overall computational complexity compared to processing entire tracklets at once, while still achieving high tracking accuracy through targeted refinement.
Solution Approach 2:
The patent applies different processing strategies to different portions of tracklets based on local characteristics. By identifying splitting points where appearance features or motion information indicate object changes, the system applies refined merging operations only where needed, rather than uniformly processing all tracklet portions, thus reducing overall computational complexity.
3Reliability
If tracklets are split into multiple sub-segments for refined processing, then tracking purity is improved, but the number of processing operations increases
Solution Approach 1:
The patent performs preliminary splitting of tracklets into sub-segments based on detected splitting points before merging operations. This preliminary organization of data into meaningful units enables more efficient merging processing, as the system only needs to merge adjacent sub-segments with consistent appearance features, rather than processing all possible combinations, thus improving processing efficiency while maintaining high tracking purity.
4Measurement precision
If appearance features are used to merge sub-segments, then merging accuracy is improved, but the computational cost of feature comparison increases
Solution Approach 1:
The patent segments tracklets into sub-segments and performs appearance feature comparison only between adjacent sub-segments that need to be merged. This localized approach to feature comparison significantly reduces the total number of comparisons needed compared to comparing all possible sub-segment pairs, thus lowering computational cost while maintaining high merging accuracy through targeted feature analysis.
Data Source
Figure 1
Figure 2(a)~2(d)
Figure 3(a)~3(d)
AI summary
An information processing device, an information processing method and a computer readable storage medium are disclosed. The information processing device includes: a similarity calculating unit, a splitting point determining unit, a splitting unit, and a merging unit. The similarity calculating unit calculates, for each tracklet, a similarity set of each frame of a predetermined frame set included in the tracklet, where the similarity set includes a similarity between the frame and a frame in a first predetermined time period immediately before the frame and a similarity between the frame and a frame in a second predetermined time period immediately after the frame. The splitting point determining unit determines, for each tracklet, a splitting point of the tracklet from the predetermined frame set based on the similarity set calculated by the similarity calculating unit. The splitting unit splits the tracklet into multiple sub-segments by using the determined splitting point. The merging unit merges sub-segments which involve a same object and do not overlap temporally among sub-segments to be merged, to obtain a merged segment, where the sub-segments to be merged include the multiple sub-segments obtained by the splitting unit.