Rendering Map for Latency Reduction in VR Vehicle Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
First person view video from remote control vehicles, such as UAVs, can cause motion sickness in users due to latency in camera view changes in response to user control movements.
Innovation Solution
A rendering map is generated during initialization to map texture frame coordinates to virtual sphere coordinates, allowing immediate determination and display of new texture values for the viewing device without waiting for the camera system to move, reducing latency and motion sickness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the camera system waits for vehicle movement before updating the video view, then the camera can capture accurate first person view video, but latency increases causing motion sickness
Solution Approach 1:
The system pre-renders video frames for multiple possible vehicle positions and orientations before they are actually needed. When the vehicle moves, the viewing device can immediately display the pre-computed frame corresponding to the new position, eliminating the latency that would otherwise occur while waiting for video capture and transmission after movement.
Solution Approach 2:
The system dynamically adjusts the video update mechanism based on vehicle movement. Instead of a static wait-for-movement approach, the system continuously pre-computes video frames for anticipated vehicle positions, creating a dynamic buffer of ready-to-display frames that adapts to the vehicle's motion state.
2Loss of time
If the system pre-renders video frames for multiple vehicle positions, then latency is reduced, but computational complexity increases
Solution Approach 1:
The rendering process is segmented into discrete frames corresponding to specific vehicle positions and orientations. Instead of continuously rendering at full complexity, the system divides the problem into manageable segments (individual frames for specific positions) that can be pre-computed and stored, reducing the real-time computational burden.
Solution Approach 2:
The system performs rendering in advance to excess - pre-computing frames for positions that may or may not be immediately needed. This partial advance computation creates a buffer that reduces latency, while the actual computational load is distributed over time rather than concentrated during critical display moments.
Data Source
AI summary
A vehicle camera system captures and transmits video to a user device, which includes a viewing device for playback of the captured video, such as virtual reality or augmented reality glasses. A rendering map is generated that indicates which pixels of the video frame (as identified by particular coordinates of the video frame) correspond to which coordinates of a virtual sphere in which a portion of the video frame is rendered for display. When a video frame is received, the rendering map is used to determine the texture values (e.g., colors) for coordinates in the virtual sphere, which is used to generate the display for the user. This technique reduces the rendering time when a user turns his or her head (e.g., while in virtual reality) and so it reduces motion and/or virtual reality sickness induced by the rendering lag.