Immersive Video Encoding with Spherical Harmonic Patch Pruning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual reality technologies struggle to provide immersive 6 Degrees of Freedom (DoF) experiences, as most omnidirectional images only support rotary motion, lacking depth and translation motion, which limits the realism and immersion in virtual reality services.
Innovation Solution
The method involves classifying view images into basic and additional images, pruning them, generating an atlas, and encoding spherical harmonic function information, including coefficients and metadata, to enhance image encoding and decoding for immersive video processing, allowing for more realistic texture representation and reduced data redundancy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If spherical harmonic function information is encoded for all view images, then image quality and realism are improved, but data encoding amount and complexity increase
Solution Approach 1:
The patent applies local quality by selectively encoding spherical harmonic function information only for specific regions (patches) in the basic image rather than uniformly across all view images. The encoder determines which patches require spherical harmonic encoding based on local image characteristics, and the decoder reconstructs only those specific regions with enhanced quality, thereby improving image quality where needed while minimizing overall data encoding amount.
2Manufacturing precision
If spherical harmonic function information is encoded for all points in three-dimensional space, then texture representation realism is improved, but device complexity and processing overhead increase
Solution Approach 1:
The patent segments the three-dimensional space into discrete patches based on the basic image, and selectively applies spherical harmonic function encoding only to specific patches rather than the entire space. The encoder divides the scene into manageable patches, identifies which patches benefit from spherical harmonic encoding, and processes only those regions. This segmentation approach reduces processing overhead while maintaining texture representation realism in critical areas.
3Quantity of substance
If pruning is performed on additional images, then data redundancy is reduced, but information loss may occur without proper compensation
Solution Approach 1:
The patent uses spherical harmonic function information as an intermediary to compensate for information loss when pruning additional images. Instead of directly storing all redundant information from additional images, the encoder extracts essential visual characteristics and represents them using spherical harmonic functions in the basic image. This intermediary representation preserves critical information while significantly reducing data redundancy, as the spherical harmonic coefficients efficiently encode lighting and texture variations that would otherwise require multiple full-resolution images.
Data Source
AI summary
An image encoding method according to the present disclosure may include classifying a plurality of view images into a basic image and an additional image; performing pruning for at least one of the plurality of view images based on a result of the classification; generating an atlas based on a result of performing the pruning; and encoding the atlas and metadata for the atlas. In this case, the metadata may include spherical harmonic function information on a point in a three-dimensional space.