Foveated Image Compression for Bandwidth-Limited Mixed Reality
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing encoder/decoder systems struggle to handle high-resolution images efficiently, particularly in mixed reality systems where head-mounted devices with lower computational power receive images from remote devices with higher computational power, exceeding the processing capabilities of the encoding/decoding units.
Innovation Solution
A method of compressing high-resolution source images to lower-resolution target images using a distortion function that maps source pixels to target pixels with a one-to-one mapping within a foveal region and a more-than-one-to-one mapping outside the foveal region, preserving image quality in the foveal region while reducing resolution elsewhere.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-resolution images are transmitted to maintain image quality, then image quality is improved, but network bandwidth consumption increases
Solution Approach 1:
The patent applies local quality by differentiating between the foveal region (central vision area) and peripheral regions. The foveal region is mapped with one-to-one pixel correspondence to preserve high quality, while peripheral regions use many-to-one mapping to reduce data transmission. This resolves the contradiction by maintaining high image quality only where the human eye is most sensitive, thereby reducing overall bandwidth consumption while preserving perceived quality.
2Measurement precision
If high-resolution images are processed to maintain quality, then image quality is improved, but processing capability requirements increase
Solution Approach 1:
The patent reduces processing capability requirements by applying different mapping strategies to different regions. The foveal region uses straightforward one-to-one mapping which is computationally simple, while peripheral regions use many-to-one mapping that reduces the number of pixels requiring processing. This localized approach maintains image quality in critical areas while significantly reducing the overall processing burden on encoding/decoding units.
3Measurement precision
If full-resolution images are transmitted to ensure quality, then image quality is improved, but transmission efficiency decreases
Solution Approach 1:
The patent improves transmission efficiency by transmitting fewer pixels for peripheral regions while maintaining full resolution for the foveal region. The many-to-one mapping in peripheral areas reduces the total pixel count that needs to be transmitted across the network, thereby increasing transmission efficiency without noticeably degrading perceived image quality, since the human eye is less sensitive to details in peripheral vision.
4Quantity of substance
If image resolution is reduced to decrease bandwidth usage, then bandwidth usage is reduced, but image quality deteriorates
Solution Approach 1:
The patent resolves this contradiction by strategically reducing resolution only in peripheral regions where the human eye is less sensitive, while maintaining full resolution in the foveal region where visual acuity is highest. This selective resolution reduction decreases overall bandwidth usage while preserving perceived image quality, as the most visually critical areas remain high-resolution.
5Productivity
If high frame rate video streams are transmitted to improve display quality, then display quality is improved, but network bandwidth consumption increases
Solution Approach 1:
The patent enables high frame rate video streams to be transmitted more efficiently by applying the foveal-peripheral mapping strategy to each frame. This reduces the data volume per frame by compressing peripheral regions, thereby allowing higher frame rates to be sustained within the same bandwidth constraints, or reducing bandwidth consumption while maintaining the desired frame rate and perceived display quality.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided are an apparatus and a method for compressing a source image, the method comprising receiving the source image, the source image having a source resolution; and mapping a source pixel of the source image to a target pixel of a target image using a distortion function, the target image having a lower resolution than the source resolution, wherein the distortion function defines a mapping, the mapping comprising a one-to-one source-to-target pixel mapping within a foveal region, and the mapping comprising a more-than-one-to-one source-to-target pixel mapping outside of the foveal region, and wherein the foveal region is a defined area of pixels.