Immersive Media Streaming via Adaptive Tile Density

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional systems for streaming mixed reality videos face challenges in efficiently transmitting high-immersive-quality content, particularly in handling user head position changes and objects with six Degrees of Freedom (DOF), leading to latency issues and unsuitable Region of Interest (ROI) calculations, which result in low-quality content delivery.

Innovation Solution

A method and system for adaptive streaming of immersive media that uses a dominance calculation module to determine the Region of Interest (ROI) based on user motion parameters, dynamically adjusting pixel density and tile selection to optimize content delivery, incorporating a server-client interaction for efficient bandwidth utilization and multidimensional media buffering.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If high bandwidth is used to maintain immersive quality throughout the frame, then image quality is improved, but bandwidth consumption increases

Engineering Contradiction:
Improveimage qualityVSAvoidbandwidth consumption
Core Design Contradiction:
Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent applies local quality by differentiating image quality across different spatial regions. The viewport region (where the user is looking) receives high-quality rendering with full pixel density, while peripheral regions use lower quality with reduced pixel density. This resolves the contradiction by maintaining high image quality where needed while reducing overall bandwidth consumption through selective quality adjustment in different frame regions.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The patent segments the omnidirectional video frame into multiple tiles or regions, allowing independent quality control for each segment. The viewport-containing tiles are transmitted at high quality, while other tiles use lower quality settings. This segmentation approach enables targeted bandwidth allocation that improves perceived image quality without proportionally increasing total bandwidth consumption.

Inventive Principle:
Principle #1Segmentation

2Adaptability or versatility

If the user's head position changes frequently, then immersive experience is improved, but download latency increases

Engineering Contradiction:
Improveimmersive experienceVSAvoiddownload latency
Core Design Contradiction:
Adaptability or versatilityVSLoss of time

Solution Approach 1:

The patent implements preliminary action by pre-calculating and buffering video tiles for anticipated viewport positions before the user actually looks in those directions. The system predicts future head movements and prepares corresponding high-quality tiles in advance, so when the user turns their head, the required content is already available or nearly available, significantly reducing perceived download latency while maintaining adaptability to head position changes.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent applies dynamics by making the video delivery system adaptive to real-time head position changes. The system dynamically adjusts which tiles are prioritized for download and rendering based on current and predicted viewport positions. This dynamic adaptation allows the system to respond to frequent head movements without being constrained by fixed download schedules, reducing latency while preserving immersive experience.

Inventive Principle:
Principle #15Dynamics

3Measurement precision

If Region of Interest (ROI) calculations are performed for objects with six Degrees of Freedom (DOF), then accuracy is improved, but computational complexity increases

Engineering Contradiction:
ImproveROI calculation accuracyVSAvoidcomputational complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent segments the complex 6DOF ROI calculation problem into manageable components by processing tiles independently. Instead of calculating ROI for the entire omnidirectional frame at once, the system divides the frame into multiple tiles and performs ROI calculations for each tile based on its relationship to the viewport. This segmentation reduces per-tile computational complexity while maintaining overall accuracy through coordinated processing of all tiles.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent applies partial action by focusing computational resources only on tiles that contain or are near the viewport region. Rather than performing exhaustive 6DOF ROI calculations for all tiles in the omnidirectional frame, the system identifies and processes only the relevant subset of tiles that contribute to the current view. This partial processing approach maintains ROI calculation accuracy for visible regions while significantly reducing overall computational complexity.

Inventive Principle:
Principle #16Partial or excessive action

Data Source

PatentUS11481026B2Immersive device and method for streaming of immersive media
Publication Date: 2022.10.25 SAMSUNG ELECTRONICS CO LTD
  • US11481026B2 patent drawing
  • US11481026B2 patent drawing
  • US11481026B2 patent drawing

AI summary

An immersive device and method for streaming of immersive media are provided. The method includes receiving body motion coordinates of a user in each of three dimensions and determining deviation in the body motion by a first immersive device. A dominant direction of motion based on the deviation calculated is then determined. A request by the first immersive device for at least one media frame to be displayed is received by a second immersive device. The received request is processed by the second immersive device and the media frame to be displayed is segmented into a plurality of tiles by the second immersive device. One tile with the first pixel density and a remainder of the tiles of the plurality of tiles with a second pixel density are stitched by the second immersive device to render one immersive media frame which is displayed to the user.