Spatially tiled omnidirectional video streaming with viewpoint-based resolution selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in efficiently streaming high-quality omnidirectional video due to high bandwidth requirements and lack of standardization in representing spherical content, leading to interoperability issues and inflexibility in existing proprietary solutions.
Innovation Solution
A method and system that utilize a client device to select and process omnidirectional video tiles based on a 3D coordinate system, allowing for flexible selection of tile streams with varying resolutions and projections, enabling efficient streaming and rendering of omnidirectional video content using conventional coding schemes like AVC, SVC, and HEVC.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If omnidirectional video is streamed in high resolution to provide high-quality user experience, then video quality is improved, but bandwidth requirements increase
Solution Approach 1:
The omnidirectional video is divided into multiple tiles that can be independently encoded and transmitted. Each tile represents a specific region of the spherical video, allowing selective streaming based on user viewpoint and quality requirements, thereby reducing overall bandwidth consumption while maintaining high quality for viewed regions
Solution Approach 2:
Different quality levels are applied to different regions of the omnidirectional video. High-resolution tiles are transmitted for regions likely to be viewed (central field of view), while lower-resolution tiles are used for peripheral regions, optimizing the trade-off between video quality and bandwidth usage
2Quantity of substance
If a proprietary encoding technique is used to reduce bandwidth, then bandwidth efficiency is improved, but interoperability deteriorates
Solution Approach 1:
The patent employs standard video coding techniques (such as H.264/AVC, H.265/HEVC) that are widely supported across different platforms and devices. By using conventional coding schemes with tile-based segmentation rather than proprietary encodings, the solution achieves both bandwidth efficiency and broad interoperability
3Productivity
If a hard-coded pyramid-mapping technique is used, then encoding efficiency is improved, but flexibility deteriorates
Solution Approach 1:
Instead of using a fixed hard-coded pyramid mapping, the patent implements a flexible tile-based system where the number, size, and arrangement of tiles can be dynamically adjusted based on content requirements, application needs, and available bandwidth. This allows the same framework to adapt to different omnidirectional video resolutions and streaming scenarios
4Ease of manufacture
If conventional tiled-based streaming schemes are used, then implementation simplicity is improved, but suitability for omnidirectional video deteriorates
Solution Approach 1:
The patent extends conventional 2D tiled streaming to 3D spherical coordinates by mapping tiles to regions on a sphere using spherical coordinate systems (azimuth and elevation angles). This allows the straightforward tile-based approach to be adapted for omnidirectional video while maintaining implementation simplicity and leveraging existing video coding infrastructure
Data Source
AI summary
A method is described for processing an omnidirectional video by a client device, said omnidirectional video being associated with a 3D coordinate system for determining a spatial position of video data of said omnidirectional video on a curved surface, preferably a sphere, said method comprising: receiving a manifest file, the manifest file comprising a plurality of tile stream identifiers for identifying a plurality of tile streams, the tile streams comprising video frames having image views, whereby the image views of video frames of different tile streams cover different regions of a 2D projection of the omnidirectional video, a region defining a tile; selecting on the basis of spatial relation information in the manifest file and on the basis of a viewpoint of a user of the client device a first tile streams associated with a first resolution and a first tile position and a second tile stream associated with a second resolution and a second tile position, the second resolution being lower than the first resolution, wherein the spatial relation information defines for each tile a tile position and wherein the viewpoint defines a viewing direction of the user, tile position and the viewpoint being defined on the basis of coordinates of the 3D coordinate system.