Foveated Image Encoding via Spherical Projection and Ring Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image encoders and decoders for extended reality (XR) environments suffer from pixel deformation and uneven pixel density due to the use of planar surfaces, leading to poor image quality and user immersion, especially when trying to replicate the wide field of view of a human eye.
Innovation Solution
The method involves generating a curved image by projecting the XR scene onto an imaginary 3D geometric shape, dividing it into an input portion and rings, and encoding these into planar images for efficient storage and transmission, allowing for a wide field of view similar to human vision while maintaining high image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If planar projection is used for XR images, then image rendering is simple, but pixel density becomes uneven and image quality deteriorates at edges
Solution Approach 1:
The patent applies curvature by projecting the visual scene onto the inner surface of a sphere instead of a planar surface. This spherical projection creates a curved image that naturally distributes pixels more uniformly across the field of view, eliminating the edge distortion and pixel density unevenness inherent in planar projections while maintaining rendering simplicity.
2Area of moving object
If wide field of view is increased, then coverage of human eye vision improves, but pixel density at edges increases excessively
Solution Approach 1:
The patent applies local quality by dividing the curved image into multiple regions (input portion and input rings) with different resolutions. The central region maintains high pixel density for detailed viewing, while peripheral regions use lower pixel density, thereby providing wide field of view coverage without excessively increasing overall pixel quantity and data transmission requirements.
3Productivity
If conventional downsampling is applied, then data transmission efficiency improves, but image quality is lost and pixels appear deformed
Solution Approach 1:
The patent applies preliminary action by performing resolution adaptation during the image encoding stage rather than during transmission or decoding. The curved image is pre-processed to assign different resolutions to different regions, creating an encoded representation that maintains image quality while reducing data size. This preliminary resolution assignment prevents the quality loss and pixel deformation that occur when conventional decoders perform aggressive downsampling.
Data Source
AI summary
An encoding method and a decoding method. The encoding method includes generating curved image by creating projection of visual scene onto inner surface of imaginary 3D geometric shape that is curved in at least one dimension; dividing curved image into input portion and plurality of input rings; encoding input portion and input rings into first planar image and second planar image, respectively, such that input portion is stored into first planar image, and input rings are packed into corresponding rows of second planar image; and communicating, to display apparatus, first and second planar images and information indicative of sizes of input portion and input rings.