Hybrid Dynamic Vision Sensor Tracking for Smooth VR Controller Motion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing VR and AR implementations face challenges in accurate and fast motion tracking of game controllers due to reliance on expensive hardware and inefficient data processing, especially when using cameras and machine learning algorithms, which generate excessive data and require complex setups.
Innovation Solution
Utilizing a Dynamic Vision Sensor (DVS) with high update rates to detect changes in light intensity from multiple light sources on a game controller, combined with inertial measurement units (IMUs) and machine learning algorithms, to track controller position and orientation efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If infrared camera at 200 frames per second is used for motion tracking, then basic tracking functionality is achieved, but the frame rate is not fast enough to provide smooth feedback for motion
Solution Approach 1:
The patent changes the temporal sampling parameter from fixed frame-based capture (200 fps) to event-based continuous capture. The DVS captures light intensity changes asynchronously at microsecond resolution, effectively increasing the temporal resolution and update rate to provide smooth motion feedback without being constrained by traditional frame rate limitations.
2Measurement precision
If high frame rate camera is used for smooth motion detection, then motion tracking accuracy is improved, but a large amount of data is generated that must be processed quickly requiring expensive hardware
Solution Approach 1:
The patent extracts only the relevant information for motion tracking by using event-based detection that captures only changes in light intensity. Instead of processing all pixels in every frame, the system processes only the subset of pixels that detected changes, significantly reducing data volume while maintaining tracking accuracy, thereby avoiding the need for expensive processing hardware.
Solution Approach 2:
The system discards redundant data by not capturing or processing frames where no motion occurred. By using asynchronous event-based sampling, the system recovers only the essential motion information needed for tracking, eliminating unnecessary data processing and reducing hardware requirements.
3Measurement precision
If traditional infrared camera setup is used, then controller tracking is achieved, but fixed position of screen and controller is required which limits user movement
Solution Approach 1:
The patent transitions from a static, fixed-position tracking system to a dynamic, adaptive system. The DVS continuously captures motion events and the system dynamically updates controller position and orientation in real-time, allowing users to move freely within the field of view while maintaining accurate tracking. The system adapts to changing spatial relationships between the camera and controller without requiring fixed positions.
4Measurement precision
If machine learning algorithms are used for hand and controller detection, then detection accuracy is improved, but extensive frame data must be processed requiring expensive hardware
Solution Approach 1:
The patent extracts only the essential features needed for detection by using event-based data that captures temporal changes in light intensity. Instead of feeding complete high-resolution frames to machine learning algorithms, the system processes a reduced set of motion events containing the critical information for hand and controller detection, thereby reducing computational requirements and hardware costs while maintaining detection accuracy.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables accurate and fast motion tracking with reduced data processing requirements, eliminating the need for expensive hardware and complex setups, while providing smooth motion feedback.
Implementation Method 1
A new type of vision system has recently been developed called, a Dynamic Vision System (DVS) the DVS utilizes only the change in light intensity of an array of light sensitive pixels to resolve changes in a scene
Implementation Method 2
Some of the earliest implementations use infrared lights detected by an infrared camera with a defined detection radius on a game controller
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A DVS has an array of light-sensitive elements in a known configuration and outputs signals corresponding to events at corresponding light-sensitive elements in the array in response to changes in light output from two or more light sources in a known configuration with respect to each other and with respect to a controller body. The signals indicate times of the events and array locations of the corresponding light-sensitive elements. Filters selectively transmit light from the light sources to light-sensitive elements in the array and selectively block other light from reaching those elements and vice versa. A processor determines a position and orientation of the controller from times of the events, array locations of corresponding light-sensitive elements, and the known light source configuration and determines a position and orientation of one or more objects from signals generated by two or more light-sensitive elements resulting from other light.