360-Degree Video Streaming Tile Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 360-degree video streaming technologies are inefficient due to the need to download entire video frames, leading to high bandwidth consumption and resource inefficiency, especially on cellular networks, where users often fetch both visible and invisible portions of spherical videos, resulting in poor Quality of Experience (QoE) and significant resource wastage.
Innovation Solution
A cellular-friendly streaming scheme that predicts a user's head movement and viewport orientation to deliver only the visible spatial segments or tiles of a spherical video, using spatial segmentation and predictive algorithms to minimize bandwidth usage, incorporating machine learning and content analysis for accurate viewport prediction and efficient tile delivery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If entire video frames are downloaded for 360-degree video streaming, then complete video content is available for display, but bandwidth consumption increases significantly
Solution Approach 1:
The patent divides the 360-degree video content into multiple spatial segments or tiles corresponding to different regions of the spherical video. Instead of downloading the entire video frame, the system segments the content and delivers only the specific tiles that correspond to the user's current field of view, thereby reducing bandwidth consumption while maintaining video quality in the visible region.
Solution Approach 2:
The patent extracts and delivers only the necessary portion of the video content that corresponds to the user's viewport. By identifying which spatial segments are within the field of view and extracting only those tiles for delivery, the system eliminates the need to transmit the entire video frame, thus resolving the contradiction between content completeness and bandwidth usage.
2Loss of energy
If spatial segmentation is used to deliver only visible portions, then bandwidth consumption is reduced, but system complexity increases due to predictive algorithms and content analysis
Solution Approach 1:
The patent performs video content analysis and spatial segmentation in advance, before the user views the video. The system pre-processes the 360-degree video into spatial segments and prepares the content for efficient delivery. This preliminary action reduces the complexity during real-time streaming, as the segmentation and analysis work has already been completed.
Solution Approach 2:
The patent introduces an intermediary processing layer that handles the complex tasks of video analysis, segmentation, and tile generation. This intermediary system acts as a mediator between the source video and the user's display, managing the complexity of predictive algorithms and content analysis in a dedicated component rather than requiring the user's device to handle all processing tasks.
3Productivity
If only visible spatial segments are fetched, then resource usage is optimized, but video quality may deteriorate if prediction accuracy is insufficient
Solution Approach 1:
The patent implements feedback mechanisms where the system monitors the user's actual viewing behavior and compares it with the predicted viewport. Based on this feedback, the system adjusts its prediction algorithms and delivers appropriate video tiles to maintain quality. The feedback loop ensures that even with selective fetching, the delivered content meets quality requirements by adapting to actual user behavior patterns.
Data Source
AI summary
Aspects of the subject disclosure may include, for example, a method performed by a processing system of determining a present orientation of a display region presented at a first time on a display of a video viewer, predicting a future orientation of the display region occurring at a second time based on data collected, to obtain a predicted orientation of the display region to be presented at the second time on the display of the video viewer, identifying, based on the predicted orientation of the display region, a first group of tiles from a video frame of a panoramic video being displayed by the video viewer, wherein the first group of tiles covers the display region in the video frame at the predicted orientation, and a plurality of objects moving in the video frame from the first time to the second time, wherein each object of the plurality of objects is located in a separate spatial region of the video frame at the second time, wherein a second group of tiles collectively covers the separate spatial regions, wherein tiles in the first group of tiles and tiles in the second group of tiles are different, and facilitating wireless transmission of the first group of tiles and a second tile from the second group of tiles, for presentation at the video viewer at the second time. Other embodiments are disclosed.