360 Video Seek Frame Segmentation Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current streaming technologies for 360-degree videos face challenges during seek operations, as they often fail to seamlessly transition frames, leading to missed video elements outside the user's viewpoint, especially during fast-forward or review operations, resulting in a suboptimal viewing experience.
Innovation Solution
The method involves encoding and decoding 360-degree video frames using a projection technique that selects and communicates both a primary and secondary frame portion, with the secondary frame extending beyond the user's viewpoint, allowing for a wider field of view during seek operations, such as fast-forward or review, by encoding and communicating additional video elements outside the initial view point.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If only the primary frame portion corresponding to the user's viewpoint is encoded and communicated during seek operations, then the bandwidth consumption and processing load are reduced, but video elements outside the user's viewpoint are missed resulting in suboptimal viewing experience
Solution Approach 1:
The frame is divided into multiple portions: a first portion corresponding to the user's viewpoint and a second portion extending outward from the center. This segmentation allows the system to encode and communicate both the primary view and additional surrounding content, ensuring no video elements are missed during seek operations while managing bandwidth through selective encoding of different frame regions
Solution Approach 2:
The patent transitions from encoding only the central viewpoint portion to encoding an expanded portion that includes the center plus additional outward-extending regions. This dimensional expansion in the frame structure enables capture of video elements outside the traditional viewpoint without fundamentally changing the seek operation mechanism
2Productivity
If the secondary frame portion extending beyond the viewpoint is encoded and communicated, then a wider field of view is provided during seek operations, but the encoding time and data transmission time increase
Solution Approach 1:
The system pre-encodes and prepares the second frame portion (extending beyond the viewpoint) in advance during the seek operation. By having this additional content ready before the user needs it, the system can provide immediate wider field of view during seek operations without significant delays, as the encoding is performed preliminarily rather than in real-time upon demand
3Reliability
If the entire 360 degree video frame is encoded during seek operations, then all video elements are captured, but the encoding complexity and processing load increase significantly
Solution Approach 1:
Instead of encoding the entire 360 degree frame, the system segments the frame into a first portion (user viewpoint) and a second portion (outward extension). This segmentation enables reliable capture of video elements outside the viewpoint during seek operations while significantly reducing the encoding processing load compared to encoding the complete frame
Solution Approach 2:
The patent applies different encoding strategies to different portions of the frame. The first portion corresponding to the user's viewpoint receives encoding treatment optimized for that region, while the second portion extending outward is encoded with appropriate quality for capturing additional video elements. This local quality approach ensures reliability for out-of-viewpoint content without uniformly processing the entire frame at maximum quality
Data Source
AI summary
A method includes receiving, from a device displaying a portion of a first frame of a streaming 360 degree video, an indication that a seek operation is triggered, the portion of the first frame being based on a view point of a user of the device, selecting a second frame of the streaming 360 degree video based on the seek operation, encoding a first portion of the second frame and a second portion of the second frame, the first portion of the second frame corresponding to the portion of the first frame, and communicating the encoded portion of the second frame to the device.