360 Video to 2D Conversion via Region of Interest Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 360-degree video systems face inefficiencies in compression, decoding, and streaming due to the need to process and transmit entire panoramic frames, even when only a small portion is viewed, leading to increased computational and bandwidth requirements.
Innovation Solution
A system that aggregates region of interest information from multiple sources to generate 2D videos based on viewer preferences, using machine learning to fine-tune parameters and dynamically switch between 360-degree and 2D video formats based on viewer device capabilities, bandwidth, and viewing angles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If entire panoramic frames are processed and transmitted in 360-degree video systems, then complete viewing coverage is achieved, but bandwidth consumption and computational requirements increase significantly
Solution Approach 1:
The patent divides the complete 360-degree panoramic video into multiple 2D video segments corresponding to different viewing directions. Instead of transmitting the entire panoramic frame, only the relevant 2D segments are selected and transmitted based on the viewer's selected viewing direction, thereby reducing bandwidth consumption while maintaining complete viewing coverage capability.
Solution Approach 2:
The patent applies different processing quality levels to different regions of the panoramic video. By identifying the region of interest (ROI) corresponding to the viewer's selected direction and transmitting it in higher quality 2D format, while potentially using lower quality or compressed representations for other regions, the system optimizes bandwidth usage while ensuring high quality where needed.
2Adaptability or versatility
If entire panoramic frames are decoded, then all viewing directions are available, but decoder cycles and computational efficiency decrease
Solution Approach 1:
The patent pre-processes the panoramic video into multiple 2D video segments corresponding to different viewing directions. During playback, the decoder only needs to decode the specific 2D segment corresponding to the viewer's selected direction, rather than decoding the entire panoramic frame, thereby significantly improving decoder efficiency while maintaining the ability to provide all viewing directions on demand.
Solution Approach 2:
The patent performs the segmentation and 2D video generation as a preliminary action during the encoding phase. This pre-processing creates ready-to-playback 2D video segments that can be quickly decoded during playback without requiring complex real-time panoramic decoding operations, thus improving decoder efficiency during actual use.
3Reliability
If 360-degree video is streamed in full resolution, then video quality is maintained, but streaming bandwidth requirements increase
Solution Approach 1:
The patent segments the full-resolution panoramic video into multiple 2D video segments. During streaming, only the relevant 2D segment corresponding to the viewer's selected viewing direction is transmitted at full resolution, while other segments are either not transmitted or transmitted at lower resolutions, thereby maintaining video quality for the viewed portion while reducing overall streaming bandwidth requirements.
4Productivity
If region of interest information is aggregated from multiple sources and used to generate 2D videos, then bandwidth and computational efficiency improve, but system complexity increases
Solution Approach 1:
The patent implements a multi-functional system that can aggregate region of interest information from multiple sources (viewer preferences, viewing history, device capabilities, bandwidth conditions) and use this aggregated information to dynamically control video generation, selection, and transmission. This universal approach handles multiple factors and decision criteria within a single integrated system, improving processing efficiency while managing complexity through unified control logic.
Data Source
AI summary
A semiconductor package apparatus may include technology to aggregate region of interest information for omni-directional video content from two or more sources, select video information from the omni-directional video content based on the aggregated region of interest information, and generate one or more two-dimensional videos based on the selected video information. Other embodiments are disclosed and claimed.