Hardware-Accelerated Tracking for Head-Mounted Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional head-mounted displays experience noticeable visual delays and high power consumption due to inefficiencies in positional tracking, often sacrificing accuracy by using low-resolution image data to reduce processing time and power requirements.
Innovation Solution
A special-purpose hardware device with a hardware-accelerated positional tracking component that caches incoming image data and performs image processing using a hardware-accelerated image processing unit, reducing DRAM accesses and enabling precise tracking by analyzing relevant image frame portions, allowing for high-resolution data processing and efficient power management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional head-mounted displays use low-resolution image data to reduce processing time and power consumption, then processing speed and power efficiency improve, but positional tracking accuracy deteriorates
Solution Approach 1:
The patent divides the image processing task into two segments: a hardware-accelerated component that performs initial feature detection and tracking on high-resolution data, and a software component that handles less time-critical processing. This segmentation allows the most demanding operations to be performed in hardware, achieving both high speed and high accuracy simultaneously.
Solution Approach 2:
The patent replaces traditional software-based image processing with a hardware-accelerated processing component that uses dedicated circuitry (such as FPGAs or ASICs) to perform feature detection and positional tracking. This substitution of mechanical/software systems with hardware systems enables parallel processing and significantly reduces computation time while maintaining full image resolution.
2Measurement precision
If traditional head-mounted displays use high-resolution image data for accurate positional tracking, then tracking accuracy improves, but processing time and power consumption increase
Solution Approach 1:
The patent replaces software-based image processing with a hardware-accelerated processing component that uses dedicated circuitry to perform feature detection and positional tracking. This hardware substitution enables parallel processing operations and significantly reduces computation time while maintaining full image resolution for accurate tracking.
Solution Approach 2:
The hardware-accelerated processing component performs feature detection and tracking operations in advance and in parallel with other system operations. By preparing positional tracking data beforehand using dedicated hardware, the system reduces the time required for subsequent processing steps while maintaining high accuracy.
3Measurement precision
If traditional head-mounted displays increase computing resources for accurate positional tracking, then tracking accuracy improves, but power consumption increases
Solution Approach 1:
The patent replaces power-intensive software-based image processing with a hardware-accelerated processing component using dedicated circuitry. This hardware substitution performs the same computational tasks with significantly lower power consumption by utilizing specialized processors designed for specific image processing operations rather than general-purpose computing.
Solution Approach 2:
The patent changes the computational parameters by using hardware-based processing with fixed-function circuitry optimized for specific image processing tasks. This parameter change from general-purpose software processing to specialized hardware processing reduces the energy required per computation while maintaining or improving processing accuracy.
4Loss of time
If traditional head-mounted displays reduce processing delays by using low-resolution data, then motion-to-photon latency improves, but visual quality and tracking precision deteriorate
Solution Approach 1:
The patent replaces software-based processing with hardware-accelerated processing that operates directly on high-resolution image data. This substitution enables the system to maintain full visual quality and tracking precision while achieving lower motion-to-photon latency through parallel hardware operations and optimized data flow paths.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The disclosed special-purpose hardware device may include an image signal processor or a camera interface that receives, from a camera device of a head-mounted-display system, image frames of a physical environment. The special-purpose hardware device may also include a positional tracking component that (1) stores at least a portion of the image frames in a cache and/or hardware-accelerated block of the special-purpose hardware device that has a faster access speed than a main memory of the special-purpose hardware device, (2) tracks, based on the portion of the image frames stored in the cache and/or processed by the hardware-accelerated block, a change in the position of the head-mounted display system within the physical environment, and (3) stores the change in the position of the head-mounted-display system, in particular in a main memory, for use in generating one or more augmented-reality frames. The special-purpose hardware device may further include a frame-output interface that feeds the augmented-reality frames to a display device of the head-mounted-display system.