360-Degree Image Encoding And Decoding With Projection Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing systems struggle with the massive data generated from processing multi-view 360-degree images for virtual and augmented reality, leading to insufficient performance in encoding and decoding high-resolution images.
Innovation Solution
A method for encoding and decoding 360-degree images that includes generating a predicted image using syntax information, combining it with a residual image, and reconstructing the image in specific projection formats like Equi-Rectangular, CubeMap, OctaHedron, and IcoSahedral, with image expansion and motion vector prediction to enhance compression performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional image encoding/decoding methods are used for 360-degree images, then the processing can be performed with standard algorithms, but the performance is insufficient for handling massive data volumes from multi-view images
Solution Approach 1:
The patent divides the 360-degree image processing into multiple projection formats (ERP, CMP, OHP, ISP) and processes different regions with different encoding strategies. The image is segmented into face units and tile units that can be independently encoded and decoded, allowing parallel processing and improved performance for large data volumes.
Solution Approach 2:
The patent applies different projection format parameters and encoding parameters based on the specific region and content requirements. By changing projection parameters (ERP for equirectangular, CMP for cubemap, etc.) and encoding parameters adaptively, the system optimizes processing performance for different types of 360-degree image content.
2Manufacturing precision
If high-resolution 360-degree images are processed, then the image quality is improved, but the data amount increases massively making processing difficult
Solution Approach 1:
The patent extracts and processes only the necessary syntax information from the bitstream for prediction, rather than processing the entire high-resolution image data. By extracting key motion vectors, prediction modes, and projection format information, the system maintains high image quality while reducing the computational burden of processing massive data volumes.
Solution Approach 2:
The patent transforms the 360-degree image data into different projection format dimensions (ERP 2D plane, CMP cube, OHP octahedron, ISP polyhedron) to facilitate more efficient encoding and decoding. This dimensional transformation allows high-resolution images to be processed in formats better suited for compression and rendering.
3Reliability
If projection format information is included in syntax information, then the image can be reconstructed in correct format, but the bitstream size increases
Solution Approach 1:
The patent includes projection format information in the syntax elements only when necessary for correct reconstruction, rather than always transmitting complete format specifications. By using partial information (e.g., indicating only the active projection format among a set of possibilities), the system ensures reliable image reconstruction while minimizing bitstream overhead.
4Measurement precision
If image expansion is performed on partitioning units, then the prediction accuracy is improved, but the processing complexity increases
Solution Approach 1:
The patent performs image expansion on individual partitioning units (tiles and faces) rather than on the entire 360-degree image. This segmentation allows prediction to be performed on smaller, manageable units with higher accuracy, while the overall processing complexity is controlled by processing each unit independently and in parallel.
Data Source
AI summary
A method of decoding an image, includes obtaining at least one offset for a picture, deriving a variable for scaling for the picture based on the at least one offset, and performing inter prediction based on the variable for scaling for the picture. The at least one offset is defined with a direction of scaling.