Keypoint Trajectory Coding for Video Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video analysis methods are inefficient in extracting compact descriptors from video sequences, leading to redundant representations and increased bandwidth and complexity, particularly in human action recognition and video retrieval tasks, as they fail to effectively exploit similarities between successive frames and do not provide a full representation of keypoint trajectories.
Innovation Solution
The method introduces scalable trajectory coding (STC) and interframe trajectory coding (ITC) to predictively code keypoint trajectories, using constrained matrix factorization to extract and encode keypoint trajectories efficiently, while ensuring backward compatibility with image-based systems, allowing for low-complexity encoding and advanced motion analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If individual image descriptors are extracted from each video frame separately, then complete feature information is obtained, but bandwidth efficiency decreases and matching complexity increases
Solution Approach 1:
The patent applies preliminary action by extracting and encoding keypoint trajectories from video frames in advance, creating a compact representation that captures temporal motion patterns. This pre-processing enables efficient retrieval and matching operations without needing to process complete individual frame descriptors during query operations, thus reducing matching complexity while preserving essential feature information.
2Speed
If conventional affine transformation is used to code keypoint locations, then motion prediction is achieved, but computational complexity increases and accurate motion details are lost
Solution Approach 1:
The patent segments the motion representation into keypoint trajectories that track individual feature points across frames. Instead of applying global affine transformations to all keypoints, the method independently tracks and encodes the motion of each keypoint, allowing for more accurate local motion details while reducing overall computational complexity through selective tracking of salient features.
3Quantity of substance
If low-rank non-negative matrix factorization is used to exploit near stationarity, then low dimensional clusters are obtained, but stability decreases as cluster number increases and full trajectory representation is lost
Solution Approach 1:
The patent adds the temporal dimension to the descriptor representation by encoding keypoint trajectories that span multiple frames. Instead of reducing descriptors to low-dimensional static clusters, the method represents keypoints as temporal sequences of positions and orientations, capturing full trajectory information while maintaining a compact representation through efficient encoding of motion patterns over time.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method processes keypoint trajectories in a video, wherein the keypoint trajectories describe motion of a plurality of keypoints across pictures of the video over time, by first acquiring the video of a scene using a camera. Kkeypoints and associated feature descriptors are detected in each picture. The keypoints and associated features descriptors are matched between neighboring pictures to generate keypoint trajectories. Then, the keypoint trajectories are coded predictively into a bitstream, which is outputted.