Video Decoder Motion Vector Extraction for Computer Vision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video processing technologies discard motion vector data during decoding, making it inaccessible to computer vision applications, which are computationally expensive and resource-intensive, limiting their use on devices with limited resources like smartphones.
Innovation Solution
A method and device that enable motion vector data to be saved and outputted alongside decoded image data, allowing it to be passed from a lower system level to higher levels, where computer vision applications can utilize it, reducing the computational resources required for video understanding tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If motion vector data is discarded after decoding (conventional approach), then the decoding process is simple and fast, but motion vector data becomes inaccessible to computer vision applications
Solution Approach 1:
The patent extracts motion vector data from the decoding process and outputs it separately through a dedicated output buffer, making it accessible to higher-level applications without disrupting the core decoding function. This extraction approach allows motion vector data to be reused by computer vision applications while maintaining decoder simplicity.
Solution Approach 2:
The output buffer is designed to serve multiple purposes: it outputs both decoded image data and motion vector data to different destinations. This multi-functionality allows the same buffer structure to support both traditional video playback and computer vision applications that require motion information.
2Productivity
If decoded video frames are used as input for computer vision applications, then complete image data is available, but processing time and computational cost increase significantly
Solution Approach 1:
The motion vector data is extracted and prepared during the decoding phase itself, before computer vision processing begins. This preliminary extraction eliminates the need for separate motion estimation computations in computer vision applications, significantly reducing their computational burden and processing time.
Solution Approach 2:
Instead of requiring computer vision applications to process complete decoded frames, the patent provides motion vector data as a compact representation that captures essential motion information. This copying of motion information from the decoding process allows CV applications to work with smaller, more efficient data structures.
3Loss of information
If motion vector data is stored and outputted alongside decoded image data, then motion vector data becomes accessible to higher-level applications, but buffer storage requirements increase
Solution Approach 1:
The output buffer is segmented into distinct regions: one for decoded image data and another for motion vector data. This segmentation allows the buffer to efficiently organize different types of information with different storage requirements, making motion vector data accessible while managing memory usage effectively.
Data Source
AI summary
Devices and methods for extracting motion vector data during decoding compressed of video data are described. At a video decoder, an encoded video data for a frame of video from an input buffer is obtained. The encoded video data is decoded to obtain decoded image data for a decoded frame, where the decoding includes extracting corresponding motion vector data for the decoded frame. The decoded image data is stored in a temporary storage indexed with a given index, and the corresponding motion vector data is stored in a same or different temporary storage indexed with the given index. An output buffer indexed with the given index is filled with the decoded image data and the corresponding motion vector data stored in the respective temporary storage indexed with the given index.