VR Headset Neural Network Orientation Prediction Reducing Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Virtual reality headsets suffer from motion-to-photon latency, leading to inaccurate perceptual illusions and motion sickness due to the time gap between user head motion and image updates, which conventional methods like extrapolation and filter-based prediction fail to adequately address.
Innovation Solution
An orientation predicting method using a trained neural network model, based on orientation training data adjusted for application latency, to predict user head movements by inputting real-time orientation data from sensors, thereby reducing latency and improving prediction accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional extrapolation and filter-based prediction methods are used to predict head movement, then the prediction can be made with existing methods, but the prediction accuracy is insufficient to adequately address motion-to-photon latency
Solution Approach 1:
The patent changes the fundamental parameter of the prediction approach by transitioning from conventional extrapolation and filter-based methods to a neural network-based prediction method. The neural network model learns complex temporal patterns and non-linear relationships in head movement data, enabling significantly improved prediction accuracy that can adequately compensate for motion-to-photon latency in VR systems
Solution Approach 2:
The patent replaces the mechanical/mathematical prediction system (extrapolation and filter-based methods) with an intelligent system using neural networks. This substitution allows the system to adaptively learn and predict head movements with higher accuracy, effectively addressing the latency issue that conventional methods cannot resolve
2Productivity
If there is a time interval between user motion and HMD frame update, then the system can process data sequentially, but motion-to-photon latency occurs causing motion sickness
Solution Approach 1:
The patent applies preliminary action by predicting future head movement positions before the actual motion-to-photon latency period elapses. The neural network model forecasts where the user's head will be when the next frame is displayed, allowing the system to pre-render images for the predicted orientation. This preliminary prediction compensates for the time interval, eliminating the harmful effect of motion sickness while maintaining frame processing throughput
Data Source
AI summary
An orientation predicting method, adapted to a virtual reality headset, comprises obtaining an orientation training data and an adjusted orientation data, wherein the adjusted orientation data is obtained by cutting a data segment off from the orientation training data, wherein the data segment corresponds to a time interval determined by an application latency; training an initial neural network model based on the orientation training data and the adjusted orientation data corresponding to the time interval; retrieving a real-time orientation data by an orientation sensor of the virtual reality headset; and inputting the real-time orientation data to the trained neural network model to output a predicted orientation data. The present disclosure further discloses a virtual reality headset and a non-transitory computer-readable medium.