Multi-view Video Encoding via Motion Vector Borrowing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing encoding processes for multi-view color and depth videos are inefficient due to increased bandwidth and calculation complexity, particularly in free viewpoint 3D video systems, where obtaining motion vectors between frames from multiple viewpoints leads to significant computational demands.
Innovation Solution
An image processing apparatus that calculates motion vectors for blocks in a color video of one viewpoint and borrows motion vectors from corresponding blocks in the same viewpoint using disparity vectors derived from depth videos, allowing for efficient encoding by dividing blocks into sub-blocks when necessary to reduce redundant calculations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If motion vectors are calculated individually for each macro block in color videos from multiple viewpoints, then encoding accuracy is improved, but calculation amount significantly increases
Solution Approach 1:
The patent pre-calculates motion vectors for a reference viewpoint and stores them in a lookup table. When encoding videos from other viewpoints, the system retrieves pre-calculated motion vectors from the table rather than calculating them in real-time, significantly reducing computational burden while maintaining encoding accuracy.
Solution Approach 2:
The patent copies and reuses motion vectors from the reference viewpoint across multiple viewpoints. By leveraging the spatial relationship between viewpoints and using disparity information, the system copies motion vectors from the reference view and adapts them to other viewpoints, avoiding redundant calculations.
2Adaptability or versatility
If color videos and depth videos are obtained from multiple viewpoints for free viewpoint 3D video system, then system functionality is improved, but bandwidth of data significantly increases
Solution Approach 1:
The patent extracts and utilizes only the necessary motion vector information from the reference viewpoint to encode videos from other viewpoints. Instead of transmitting all color and depth data from multiple viewpoints, the system extracts motion vectors from the reference view and uses disparity information to reconstruct motion for other viewpoints, significantly reducing data bandwidth requirements.
Solution Approach 2:
The patent makes the motion vector data from the reference viewpoint serve multiple functions across different viewpoints. The same pre-calculated motion vectors are reused for encoding videos from multiple viewpoints by applying disparity-based transformation, making the data universally applicable and reducing overall bandwidth requirements.
3Measurement precision
If disparity vectors are calculated for all blocks in color videos, then encoding precision is improved, but device complexity increases
Solution Approach 1:
The patent applies local quality by calculating disparity vectors only for blocks that require precise encoding, rather than uniformly for all blocks. The system uses depth information to identify regions with significant depth variations and applies disparity-based motion vector borrowing selectively to those regions, reducing overall processing complexity while maintaining encoding precision where needed.
Data Source
AI summary
Provided is an image processing apparatus, method and computer-readable medium. When encoding a multi-view color and depth video, a motion vector calculator of the image processing apparatus may calculate a block motion vector with respect to a first viewpoint color image. A motion vector borrowing unit may borrow, as a block motion vector with respect to a second viewpoint color image, a block motion vector with respect to the first viewpoint color image based on a disparity calculated from a depth video.