VR Video Streaming System with Predictive Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing VR devices face issues with picture delay and jitter due to insufficient support for video streaming, leading to suboptimal user experience in applications like online games and live games.
Innovation Solution
A video streaming system and method that involves a terminal and a VR device, where application platform software and server streaming software are installed on the terminal, and client streaming software on the VR device, enabling the transmission of posture and positioning data for real-time picture rendering, with a positioning and prediction unit to predict data delays and improve rendering stability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If video streaming is implemented on existing VR devices, then video playback capability is improved, but picture delay and picture jitter occur due to insufficient device support
Solution Approach 1:
The system divides video processing into separate modules: video decoding module, video rendering module, and synchronization module. The terminal handles video decoding and rendering, while the VR device handles display, segmenting the processing tasks to match device capabilities and reduce latency.
Solution Approach 2:
The terminal performs video decoding and rendering in advance before transmitting to the VR device. The system pre-processes video data and predicts playback timing to compensate for transmission delays, ensuring synchronized playback without real-time processing pressure on the VR device.
2Speed
If video data is transmitted in real-time, then playback timeliness is improved, but network delays cause picture jitter
Solution Approach 1:
The system pre-calculates playback timing and pre-positions video data for transmission. The terminal predicts when video frames should be displayed and transmits them accordingly, compensating for network transmission time in advance to maintain synchronized playback.
Solution Approach 2:
The system implements timing feedback mechanisms where the terminal monitors actual playback timing and adjusts subsequent video transmission schedules. The synchronization module uses timing information from previous frames to optimize transmission timing for future frames, reducing jitter over time.
3Manufacturing precision
If video processing is performed on the VR device, then playback quality is improved, but device performance limitations cause delay and jitter
Solution Approach 1:
The patent extracts video decoding and rendering functions from the VR device and relocates them to the terminal. The terminal's more powerful processing capabilities handle computationally intensive video processing, while the VR device focuses on display output, eliminating the bottleneck of limited device performance.
Solution Approach 2:
The terminal acts as an intermediary between the video source and VR device. It decodes and renders video data, then transmits processed data to the VR device for display. This intermediary approach allows high-quality processing without overloading the VR device's limited resources.
Data Source
AI summary
Provided are a video streaming system, and a corresponding video streaming method and video streaming apparatus. The video streaming system includes a terminal and a VR device. Application platform software and server streaming software are installed on the terminal. Client streaming software is installed on the VR device, the client streaming software is configured to send posture data to the server streaming software on the terminal, the server streaming software is configured to send the posture data to the application platform software, and the application platform software is configured to render a picture. The hardware configuration of the terminal itself is utilized for picture processing, and satisfactory pictures can be obtained on the screen of the VR device. On the other hand, the picture is rendered according to prediction positioning data/prediction posture data, thereby effectively reducing picture jitter and display delay.