Near-Eye AR/VR Display Architecture for Dynamic Object Positioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional displays, including augmented reality (AR) and virtual reality (VR) displays, fail to account for the movement of user's head position and gaze direction, leading to challenges in object positioning and integration, which is not feasible for small form factor near-eye display devices due to size, weight, and power consumption issues.
Innovation Solution
A hardware architecture employing a system-on-chip (SoC) to compose and submit textures, combined with a software stack that includes an applications level, display service level, and driver level, allowing for dynamic object positioning based on user head movement and gaze, and distributing functionality between a portable controller device and the near-eye display device to reduce size, weight, and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional displays are used for AR/VR, then object positioning and integration can be achieved, but the device size, weight, and power consumption become unacceptable for near-eye display devices
Solution Approach 1:
The display system is segmented into two parts: a portable controller device containing the display screen and processing components, and a lightweight near-eye display device containing only the optical elements and sensors. This segmentation allows the heavy components to be carried in the controller while the near-eye device remains lightweight enough for comfortable wear.
Solution Approach 2:
The patent introduces an intermediary computational system (the portable controller device) that handles the complex processing of head movement tracking and object positioning. This intermediary performs the computationally intensive tasks remotely, allowing the near-eye display device to remain simple and lightweight while still achieving reliable object positioning and integration.
2Adaptability or versatility
If conventional displays account for head movement and gaze direction, then dynamic object positioning is achieved, but the device complexity and power consumption increase
Solution Approach 1:
The system separates the complex processing functions (head movement tracking, gaze detection, object positioning calculations) into the portable controller device, while the near-eye display device contains only the necessary optical elements and basic sensors. This segmentation reduces the complexity of the near-eye device while maintaining dynamic object positioning capability.
Solution Approach 2:
The system uses the device's own sensors (cameras, gyroscopes, accelerometers) to automatically track head movement and determine gaze direction without requiring external tracking equipment. This self-service approach enables dynamic object positioning while minimizing additional system complexity.
3Speed
If processing functionality is integrated into the near-eye display device, then object positioning responsiveness is improved, but the device size and power consumption increase
Solution Approach 1:
The processing functionality is segmented between the portable controller device (containing the display screen and main processor) and the near-eye display device (containing sensors and optical elements). The controller handles intensive processing while the near-eye device remains compact, achieving a balance between responsiveness and size.
Solution Approach 2:
The system performs preliminary processing of sensor data and object positioning calculations in the portable controller device before transmitting the final rendered image to the near-eye display. This preliminary action allows complex computations to be completed in advance, maintaining responsiveness while keeping the near-eye device small.
Data Source
AI summary
A system for implementing an augmented reality (AR)/virtual reality (VR) display includes a hardware (HW) architecture that employs a system-on-chip (SoC) to compose and submit textures. The SoC composes and submits textures by using display serial interface (DSI) display lines to transmit the textures to a physical display. Software (SW) architecture includes a stack display architecture with an applications level using the surface rendered by applications, a display service level to compose superframes, provide an abstraction layer on top of a surface flinger (SL) application programming interface (API), and allow assignment of textures and metadata to different hardware planes, and a driver level. The system uses the surfaces or enhanced effects that are rendered by the applications or the compositor, depending on the type of the application, and computes the metadata related to the location or coordinates.