Multi-Controller Gesture Detection via SLAM and IMU Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual, augmented, and mixed reality applications face challenges in accurately detecting and interpreting gestures performed by multiple controllers, particularly in determining gestures without physical contact and in multi-user environments, where processing limitations of Head-Mounted Devices (HMDs) restrict the complexity and accuracy of gesture recognition.
Innovation Solution
An Information Handling System (IHS) is designed to receive Simultaneous Localization and Mapping (SLAM) landmarks and Inertial Measurement Unit (IMU) data from multiple controllers to identify gestures, such as tapping, and execute associated commands, even in the absence of physical contact, by determining the proximity and orientation of the controllers within a threshold distance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If gesture detection is performed using only HMD sensors, then device complexity is reduced, but measurement precision of gestures deteriorates
Solution Approach 1:
The system combines sensors from multiple sources: HMD sensors (accelerometers, gyroscopes) and controller sensors (IMUs with accelerometers and gyroscopes) to detect gestures. This merging of sensor data from different devices enables accurate gesture detection while keeping individual device complexity low, as each device only needs basic sensing capabilities rather than complex processing.
2Measurement precision
If physical contact between controllers is required for gesture detection, then measurement precision improves, but ease of operation deteriorates
Solution Approach 1:
The system replaces the mechanical requirement of physical contact between controllers with an electromagnetic/sensor-based detection method. By using sensor data (accelerometer and gyroscope readings) to detect proximity and orientation, the system can identify gestures like tapping without requiring actual physical contact, making interactions more convenient while maintaining detection accuracy.
3Measurement precision
If SLAM landmarks are used to determine controller proximity, then measurement precision improves, but loss of information increases
Solution Approach 1:
The system extracts only the essential information needed for gesture detection from the SLAM landmarks data - specifically the position and orientation of controllers relative to each other. Rather than transmitting or processing all SLAM landmark information, the system extracts the relevant spatial relationships to determine proximity and orientation, reducing data loss while maintaining measurement precision.
Data Source
AI summary
Embodiments of systems and methods for providing controller gestures in virtual, augmented, or mixed reality (xR) applications are described. In some embodiments, an Information Handling System (IHS) may include a processor and a memory coupled to the processor, the memory having program instructions stored thereon that, upon execution, cause the IHS to: receive one or more first Simultaneous Localization and Mapping (SLAM) landmarks corresponding to a first controller; receive one or more second SLAM landmarks corresponding to a second controller; determine, using the first and second SLAM landmarks, that the first controller is within a threshold distance of the second controller; in response to the determination, receive first Inertial Measurement Unit (IMU) data from the first controller and second IMU data from the second controller; identify, using the first and second IMU data, a gesture performed with the first and second controllers; and execute a command associated with the gesture.