360-Degree Image Decoding with Projection-Based Intra Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing systems struggle with the massive data generated from multi-view images for 360-degree images in virtual and augmented reality, leading to insufficient performance in encoding and decoding high-resolution images.
Innovation Solution
A method for decoding 360-degree images involves generating a predicted image using syntax information, combining it with a residual image, and reconstructing the image in specific projection formats like ERP, CMP, OHP, or ISP, with image expansion and intra-prediction techniques to enhance compression performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional image encoding/decoding methods are used for 360-degree images, then the processing can be performed with standard techniques, but the performance is insufficient for handling the massive data generated by multi-camera systems
Solution Approach 1:
The patent applies segmentation by dividing the 360-degree image into multiple projection formats (e.g., equirectangular, cube map, octahedron, icosahedron) and processing each segment independently with appropriate encoding techniques. This allows the system to handle massive data volumes from multi-camera systems by processing smaller, manageable segments rather than treating the entire spherical image as a single unit, thereby improving overall processing performance.
2Adaptability or versatility
If 360-degree images are captured with multiple cameras for virtual and augmented reality, then realistic media service is enabled, but the amount of data generated increases massively
Solution Approach 1:
The patent employs parameter changes by transforming 360-degree images captured from multiple cameras into different projection format parameters (equirectangular, cube map, octahedron, icosahedron). This transformation changes the spatial representation parameters of the image data, enabling versatile virtual and augmented reality applications while reducing the overall data amount through efficient packing and compression techniques specific to each projection format.
3Productivity
If image expansion and intra-prediction techniques are used, then compression performance is enhanced, but the encoding and decoding process becomes more complex
Solution Approach 1:
The patent applies preliminary action by performing image expansion and generating predicted images before the actual encoding process. Syntax information including projection format information is prepared in advance and embedded in the bitstream. This preliminary preparation enables more efficient compression during the main encoding process, as the decoder can use the pre-prepared predicted images and syntax information to reconstruct the 360-degree image with reduced complexity compared to generating all components during decoding.
Data Source
AI summary
A method for decoding a 360-degree image includes: receiving a bitstream obtained by encoding a 360-degree image; generating a prediction image by making reference to syntax information obtained from the received bitstream; combining the generated prediction image with a residual image obtained by dequantizing and inverse-transforming the bitstream, so as to obtain a decoded image; and reconstructing the decoded image into a 360-degree image according to a projection format. Here, generating the prediction image includes: checking, from the syntax information, prediction mode accuracy for a current block to be decoded; determining whether the checked prediction mode accuracy corresponds to most probable mode (MPM) information obtained from the syntax information; and when the checked prediction mode accuracy does not correspond to the MPM information, reconfiguring the MPM information according to the prediction mode accuracy for the current block.