Edge Image Streaming Using Partial Frames for VR Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing edge computing technologies face challenges in efficiently transmitting high-resolution image content to electronic devices, particularly in scenarios involving user interaction and dynamic changes in orientation and network conditions, leading to potential disruptions in immersive experiences like virtual reality.
Innovation Solution
An edge data network adaptively streams image content by encoding partial images based on sensor information, orientation, and state parameters, including user gaze and rotational speed, to ensure seamless reproduction and minimize latency and black edges.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If high-resolution image content is transmitted to electronic devices using existing edge computing technologies, then image quality is improved, but latency increases and black edges may occur due to dynamic orientation changes
Solution Approach 1:
The patent divides the image content into multiple resolutions, with a first resolution for the central region (fovea) and a second resolution for peripheral regions. This segmentation allows the system to transmit high-resolution content only where the user is looking, reducing overall data transmission requirements and latency while maintaining perceived image quality.
Solution Approach 2:
The system performs preliminary actions by predicting future orientation changes based on current sensor data and pre-processing image content accordingly. This allows the system to prepare and transmit appropriate image segments before the user actually moves their gaze, reducing perceived latency and preventing black edges.
2Manufacturing precision
If high-resolution image content is transmitted using existing edge computing technologies, then image quality is improved, but black edges occur due to orientation changes
Solution Approach 1:
The patent implements dynamic adaptation by continuously monitoring sensor information (orientation, angular velocity) and adjusting image processing parameters in real-time. The system dynamically changes which image regions are transmitted at high resolution based on current and predicted gaze direction, ensuring continuous immersive streaming without black edges.
Solution Approach 2:
The system uses sensor feedback from the electronic device (orientation sensors, gyroscopes) to continuously adjust image processing. This feedback loop allows the system to respond to user movements and maintain accurate image alignment, preventing black edges while preserving image quality.
3Loss of time
If partial images are encoded based on sensor information and orientation, then latency is reduced and black edges are prevented, but device complexity increases
Solution Approach 1:
The patent applies local quality by encoding only specific regions of the image (the fovea or central region) at high resolution, while peripheral regions are encoded at lower resolution. This selective encoding reduces overall processing complexity and data transmission requirements while maintaining perceived image quality where it matters most to the user.
Data Source
AI summary
The present disclosure relates to a server, an electronic device and their methods of transmitting or receiving image content. In one embodiment, a method performed by the server includes obtaining sensor information including orientation information detected by the electronic device connected to the server; obtaining state information regarding the server and the electronic device; identifying a size of a first partial image and a position of the first partial image, based on the sensor information and the state information; generating a first frame by encoding the first partial image, based on the identified size of the first partial image or identified position of the first partial image; and transmitting the generated first frame to the electronic device.