360-Degree Video Encoding With Predictive Viewport Feedback

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing technologies face challenges in efficiently delivering immersive 360-degree video content due to the high degree of user freedom and uncertainty in viewport orientation, leading to suboptimal quality of experience in viewport-dependent streaming.

Innovation Solution

Implementing a method and apparatus for early RTCP feedback to signal viewport orientation information from the receiver to the sender, using thresholds based on viewport distance and velocity to optimize viewport-dependent streaming.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If viewport orientation information is sent frequently to enable timely adaptation to user head motion, then the quality of experience is improved, but the network bandwidth consumption increases

Engineering Contradiction:

Improvequality of experienceVSAvoidnetwork bandwidth consumption

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

The receiver device predicts future viewport orientation based on current head motion velocity and sends this predicted information to the sender in advance. This preliminary action allows the sender to prepare the optimized video stream before the user's head actually rotates to the new position, reducing the need for frequent feedback while maintaining quality of experience.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system dynamically adjusts the frequency and timing of viewport orientation feedback based on the detected head motion velocity. When head motion is fast, the system sends feedback more frequently; when motion is slow or stationary, feedback is reduced. This dynamic adaptation resolves the contradiction between maintaining quality and reducing bandwidth consumption.

Inventive Principle:

Principle #15Dynamics

2Speed

If viewport orientation feedback is sent immediately upon head motion detection, then adaptation speed is improved, but network traffic overhead increases

Engineering Contradiction:

Improveadaptation speedVSAvoidnetwork traffic overhead

Core Design Contradiction:

SpeedVSDevice complexity

Solution Approach 1:

Instead of immediately sending viewport orientation feedback, the receiver device performs a prediction based on current head motion velocity and sends this predicted orientation information in advance. This preliminary prediction action maintains fast adaptation while reducing network traffic overhead by avoiding redundant immediate feedback transmissions.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements a feedback mechanism where the receiver sends viewport orientation information to the sender, but optimizes this feedback by using predicted values rather than immediate raw measurements. This feedback principle allows the system to maintain adaptation speed while reducing network traffic through intelligent data selection and prediction.

Inventive Principle:

Principle #23Feedback

3Stability of the object's composition

If the system waits for scheduled RTCP feedback intervals, then network stability is improved, but viewport adaptation delay increases

Engineering Contradiction:

Improvenetwork stabilityVSAvoidviewport adaptation delay

Core Design Contradiction:

Stability of the object's compositionVSLoss of time

Solution Approach 1:

The receiver device performs preliminary prediction of future viewport orientation based on current head motion and sends this predicted information before the next scheduled RTCP feedback interval. This preliminary action allows the system to maintain network stability by sticking to scheduled intervals while reducing adaptation delay through advance preparation of viewport information.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system uses periodic RTCP feedback intervals to maintain network stability, but enhances this periodic action by inserting predicted viewport orientation information at strategic moments. This periodic action with enhanced timing allows the system to balance network stability with reduced adaptation delay by utilizing the prediction capability to send information slightly before or at optimal moments within the periodic cycle.

Inventive Principle:

Principle #19Periodic action

Data Source

PatentUS12464103B2Method, an apparatus and a computer program product for video encoding and video decoding

Publication Date: 2025.11.04 NOKIA TECHNOLOGIES OY

AI summary

The embodiments relate to a method and technical equipment for consumption of omnidirectional visual media content, comprising receiving from a sender a bitstream defining a presentation, the presentation comprising an omnidirectional visual media content; transmitting the presentation on a receiver device; determining a condition when to send viewport orientation information to the sender, wherein the determining is based on at least a distance relating to a moving viewport and a corresponding threshold; when the determined condition is met, generating a packet comprising the viewport orientation information and sending the packet to the sender; and receiving a presentation from the sender to be streamed on the receiver device according to the sent viewport orientation.