Multi-Camera Image Coding for Head-Mounted Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 3D video recording technologies require high bandwidth for transmitting high-quality image data from multiple cameras, which is inefficient and not optimized for head-mounted display (HMD) usage, where users typically view central scenes with higher quality and peripheral scenes with lower quality.
Innovation Solution
A method and system that encode image data from a multi-camera capture device using geometry information to prioritize higher quality encoding for central image regions and lower quality encoding for peripheral regions, based on the user's head orientation and camera setup, allowing for reduced bandwidth usage and seamless transitions when the user turns their head.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If all image data from multiple cameras is encoded with high quality, then image quality is improved, but bandwidth requirements increase
Solution Approach 1:
The patent applies local quality by differentiating encoding quality across different spatial regions of the image. Central regions that correspond to the user's foveal vision are encoded with high quality, while peripheral regions are encoded with lower quality. This resolves the contradiction by maintaining high image quality where the user actually looks (central regions) while reducing bandwidth consumption in peripheral regions where the user's attention is naturally lower.
2Reliability
If high-quality encoding is applied to all regions, then viewing experience is improved, but data transmission efficiency deteriorates
Solution Approach 1:
The patent implements local quality encoding where central image regions are transmitted with high fidelity to ensure excellent viewing experience in the user's focal area, while peripheral regions use compressed encoding to maximize data transmission efficiency. This resolves the contradiction by optimizing viewing experience locally where needed rather than uniformly across the entire image.
3Device complexity
If uniform encoding quality is used across all image regions, then encoding simplicity is maintained, but visual perception quality deteriorates
Solution Approach 1:
The patent improves visual perception quality by applying different encoding qualities to different image regions based on human visual system characteristics. Central regions receive high-quality encoding while peripheral regions use lower quality encoding. This is achieved through region classification based on camera geometry and expected viewing behavior, adding moderate complexity to the encoding process while significantly improving overall visual perception quality.
Solution Approach 2:
The patent changes encoding parameters dynamically based on spatial location within the image. Different quantization parameters, compression ratios, or resolution levels are applied to central versus peripheral regions. This parameter variation allows the system to optimize visual perception quality in critical areas while managing overall data throughput, resolving the contradiction between encoding complexity and visual quality.
Data Source
AI summary
The invention relates to a method, where a first and second stereoscopic image are formed comprising a left eye image and a right eye image, a first central image region and a first peripheral image region are determined in the first stereoscopic image, the first central image region comprising a first central scene feature and the first peripheral image region comprising a first peripheral scene feature, and in the second stereoscopic image a second central image region and a second peripheral image region in said second stereoscopic image are determined, and based on said determining that said second central image region comprises said first peripheral scene feature, encoding said first stereoscopic image such that said first peripheral image region is encoded with a reduced quality with respect to said first central image region.