360-Degree Image Reconstruction Using Projection-Aware Inter Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing systems struggle with the massive data generated for 360-degree images in virtual and augmented reality, requiring improved performance in encoding and decoding methods.
Innovation Solution
A method for encoding and decoding 360-degree images that includes generating a predicted image using syntax information, combining it with a residual image, and reconstructing the image based on projection formats like ERP, CMP, and ISP, with image expansion and motion vector prediction to enhance compression performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional image encoding/decoding methods are used for 360-degree images, then the processing can be performed with standard algorithms, but the performance is insufficient for handling the massive data volume generated by multi-view images
Solution Approach 1:
The 360-degree image is divided into multiple projection formats (ERP, CMP, OHP, ISP) and processed separately. The encoding apparatus segments the image data according to different projection types, allowing specialized processing for each format. This segmentation enables the system to handle large data volumes by processing divided regions rather than the entire 360-degree image at once, improving overall processing performance.
Solution Approach 2:
The patent applies different encoding parameters and syntax information based on the projection format type. The system changes processing parameters dynamically according to whether the image is in ERP, CMP, OHP, or ISP format, optimizing the encoding/decoding performance for each specific projection type. This parameter adaptation allows the system to efficiently process massive data volumes by tailoring the processing approach to each format's characteristics.
2Measurement precision
If image expansion is performed on the reference picture to generate predicted images, then the prediction accuracy improves, but the computational complexity increases
Solution Approach 1:
The reference picture is expanded in advance before the actual prediction process. By performing image expansion on the reference picture beforehand, the system prepares the expanded regions that can be directly used for generating predicted images without requiring complex real-time expansion operations during prediction. This preliminary action reduces computational complexity during the main decoding process while maintaining high prediction accuracy.
Solution Approach 2:
The expanded reference picture acts as an intermediary between the original reference data and the final predicted image. Instead of directly computing complex predictions from the original reference picture, the system uses the pre-expanded reference picture as an intermediate step, simplifying the prediction process while improving accuracy through the additional boundary information provided by the expansion.
3Loss of information
If motion vector prediction is performed using adjacent blocks and face continuity information, then the compression performance improves, but the processing time increases
Solution Approach 1:
The system uses feedback from adjacent blocks and face continuity information to improve motion vector prediction. By analyzing the motion vectors of adjacent blocks and the continuity characteristics of the current face, the system refines the motion vector prediction accuracy. This feedback mechanism allows the system to achieve better compression performance by more accurately predicting motion vectors, reducing the amount of data that needs to be encoded and transmitted.
Solution Approach 2:
The motion vector prediction is performed with different levels of detail depending on the local characteristics of each block. The system applies more sophisticated prediction using face continuity information where needed, while using simpler methods in regions where high precision is less critical. This local quality approach optimizes the balance between compression performance and processing time by applying computational resources selectively.
Data Source
AI summary
A method of decoding an image, includes obtaining at least one offset for a picture, deriving a variable for scaling for the picture based on the at least one offset, and performing inter prediction based on the variable for scaling for the picture. The at least one offset is defined with a direction of scaling.