Omnidirectional Image Encoding for Immediate Field-of-View Display
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing systems face challenges in immediately displaying the field-of-view range of a user due to the inefficiencies in encoding and decoding omnidirectional images, particularly when the line-of-sight vector changes abruptly, as they rely on general encoding schemes like MPEG 2 and AVC that restrict random access points and hinder efficient decoding of specific image areas.
Innovation Solution
The system encodes omnidirectional images by combining high-resolution and low-resolution images across opposite surfaces of a 3D model, allowing for efficient decoding and display of only the necessary image areas corresponding to the user's line-of-sight vector, reducing the decoding load and enabling immediate image updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If general encoding schemes like MPEG 2 or AVC are used to encode omnidirectional images, then compression efficiency is improved, but the ability to decode only specific image areas corresponding to the user's line-of-sight vector is hindered
Solution Approach 1:
The omnidirectional image is divided into multiple regions based on the user's line-of-sight vector and field-of-view range. Only the relevant regions (front, side, or back regions) are decoded and displayed, while other regions remain encoded. This segmentation allows the system to maintain high compression efficiency for the entire omnidirectional image while enabling efficient decoding of only the necessary portions.
2Speed
If the entire omnidirectional image is decoded to enable immediate display of the field-of-view range, then display responsiveness is improved, but the decoding process amount increases
Solution Approach 1:
Instead of decoding the entire omnidirectional image, the system performs partial decoding only on the regions corresponding to the user's field of view. This partial action approach maintains display responsiveness by avoiding the need to decode unnecessary regions, thereby reducing the overall decoding process amount and computational power required.
3Adaptability or versatility
If the line-of-sight vector is decided by the user, then adaptability is improved, but the recording device's ability to encode only the corresponding area is reduced
Solution Approach 1:
The system pre-encodes the entire omnidirectional image using general encoding schemes, creating a complete encoded stream that contains all possible viewing regions. When the user changes their line-of-sight vector, the reproducing device can immediately switch to decoding only the relevant pre-encoded regions without requiring the recording device to re-encode. This preliminary encoding action maintains user adaptability while simplifying the encoding process.
Data Source
Figure 1~2
Figure 3
Figure 4~5
AI summary
[Object] To enable an image of a field-of-view range of a user to be displayed immediately. [Solution] An encoder encodes, for a pair of opposite surfaces among a plurality of surfaces of a model for an omnidirectional image, a first composite image obtained by combining an omnidirectional image of a first surface of a high resolution and an omnidirectional image of a second surface of a resolution lower than the high resolution and a second composite image obtained by combining an omnidirectional image of the second surface of the high resolution and an omnidirectional image of the first surface of a low resolution. For example, the present disclosure can be applied to an image display system or the like.