Video Visual Relation Detection via Segmentation and Tracklet Merging

The video sequence is divided into multiple overlapping segments, where each segment is processed independently for visual relation detection. The segments are then merged by associating detections across segment boundaries to form complete relation trajectories. This segmentation approach reduces the computational burden of processing the entire video as a single unit while maintaining comprehensive detection capability through the association of results across segments.

Object detection is performed at key frames or selected frames rather than every frame. The detected objects are then tracked across subsequent frames using tracking algorithms that maintain object identities and trajectories. This preliminary detection approach reduces the number of expensive detection operations while maintaining accurate localization through the combination of detection results and tracking information.

The system uses a smaller set of training examples with focused annotation effort on critical relation types, rather than attempting to annotate all possible relations exhaustively. The method leverages the temporal continuity and consistency of visual relations in videos to compensate for the limited training data, achieving acceptable accuracy with reduced annotation burden.

The video is divided into segments that are processed independently for tracking. This segmentation allows the use of simpler tracking algorithms within each segment, as the temporal duration of each segment is limited. The tracking results are then associated across segment boundaries to maintain long-term object identities, reducing the computational complexity required for tracking within individual segments while maintaining overall tracking accuracy.