Multi-View Video Encoding with Depth Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio/video interface formats like HDMI have limited support for image-based rendering that requires image and depth information, particularly lacking comprehensive support for stereo images and associated depth data, as well as supplemental metadata transmission.
Innovation Solution
The method involves encoding video data signals for multi-view rendering devices by subdividing 2160p video frames into quadrants for left and right views, with depth information encoded in luminance components and metadata in chrominance components, ensuring backwards compatibility and error correction, and transmitting this data over interfaces like HDMI for efficient rendering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If existing HDMI interface formats are used for transmitting video data, then device compatibility is maintained, but support for image-based rendering with depth information and metadata is limited
Solution Approach 1:
The video data signal is segmented into multiple components: image data, depth information, and metadata are separated and transmitted through different channels (video data signal, auxiliary data signal, and metadata packets respectively). This segmentation allows each component to be optimized for its specific transmission requirements while maintaining overall system compatibility.
Solution Approach 2:
The patent introduces a temporal dimension by transmitting metadata in supplemental enhancement information (SEI) packets that accompany the video stream. This allows metadata to be transmitted in synchronization with the video data without interfering with the existing HDMI interface protocol, effectively adding a new dimension of information transmission capability.
2Adaptability or versatility
If video frames are subdivided into quadrants for stereo views, then multi-view rendering capability is improved, but data transmission complexity increases
Solution Approach 1:
The encoding method is designed to be universal and compatible with existing HDMI interface specifications. The same interface can transmit both traditional 2D video and the new subdivided stereo video format with depth and metadata, making the system multi-functional without requiring separate transmission channels or protocols.
Solution Approach 2:
The patent changes the organizational parameters of video data transmission by introducing quadrant-based subdivision and associated metadata structures. However, these parameter changes are implemented in a way that maintains backward compatibility, allowing receivers to interpret the data according to existing HDMI specifications while supporting the enhanced multi-view format.
3Productivity
If depth information is encoded in luminance components and metadata in chrominance components, then transmission efficiency is improved, but color accuracy may be affected
Solution Approach 1:
The patent applies local quality by encoding depth information in the luminance component where human vision is most sensitive to variations, and metadata in the chrominance component where color information resides. This localized encoding strategy optimizes the use of available bandwidth by placing each type of information in the component best suited for it according to human visual characteristics.
Data Source
AI summary
The present invention relates to a method of encoding a video data signal for use with a multi-view rendering device, a method of decoding the video data signal, the video data signal, an encoder of the video data signal, a decoder of the video data signal, a computer program product comprising instructions for encoding the video data signal and a computer program product comprising instructions for decoding a video data signal. The method of encoding provides (401) a first image (10) of a scene associated with a first viewpoint, a depth map (20) associated with the first image, metadata (30) for use in depth map processing or rendering one or more views for further viewpoints by the multi-view rendering device, and generates (404) the video data signal. The video data signal comprises video frames partitioned in sub-images comprising a sub-image based on the first image and a depth sub-image based on the depth map, and metadata encoded in a color component of the depth sub-image.