SLAM-Based Gesture Recognition in HMDs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current Information Handling Systems (IHSs) for virtual, augmented, and mixed reality applications lack effective gesture recognition capabilities, particularly in environments where processing power is limited, such as in Head-Mounted Devices (HMDs), which hinders seamless user interaction and control within immersive experiences.
Innovation Solution
The implementation of Simultaneous Localization and Mapping (SLAM) components in IHSs, utilizing a combination of image capture devices, including gesture cameras and ambient light sensors, to detect and recognize gesture sequences by concurrently processing video frames and determining the most frequent gestures, allowing for accurate execution of commands associated with these gestures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If gesture recognition is implemented in HMDs with limited processing power, then user interaction capability is improved, but system performance and accuracy deteriorate
Solution Approach 1:
The system segments the gesture recognition task by separating feature extraction (performed locally in HMD using SLAM components) from complex gesture sequence analysis (performed on host system). This allows the HMD to handle basic tracking while the host processes sophisticated gesture patterns, resolving the contradiction between local interactivity and accurate recognition.
Solution Approach 2:
The patent introduces an intermediary processing layer that uses SLAM components to extract hand pose features and trajectory data. This intermediary layer bridges the gap between limited HMD processing capabilities and the need for accurate gesture recognition, enabling complex gestures to be recognized through simplified intermediate representations.
2Measurement precision
If multiple image capture devices are used for gesture detection, then gesture recognition accuracy is improved, but device complexity increases
Solution Approach 1:
The system makes existing SLAM image capture devices multi-functional by using them for both environmental mapping and gesture recognition. Instead of adding dedicated gesture cameras, the patent repurposes the existing cameras to serve dual functions, thereby improving gesture detection accuracy without increasing device complexity.
Solution Approach 2:
The system enables the existing image capture devices to serve themselves by extracting additional gesture information from the data they already collect for SLAM operations. The same cameras and processing pipelines are utilized for both navigation/mapping and gesture recognition, eliminating the need for separate dedicated gesture sensing hardware.
3Measurement precision
If concurrent processing of video frames from multiple devices is performed, then gesture sequence recognition precision is improved, but processing power requirements increase
Solution Approach 1:
The concurrent processing task is segmented between HMD and host system. The HMD performs lightweight feature extraction on video frames using SLAM components, while the host system handles the computationally intensive gesture sequence analysis. This segmentation allows precise gesture recognition through multi-device processing without overwhelming the limited processing power of the HMD.
Data Source
AI summary
Systems and methods for gesture sequence recognition using Simultaneous Localization and Mapping (SLAM) components in virtual, augmented, and mixed reality (xR) applications are described. In an illustrative, non-limiting embodiment, an Information Handling System (IHS) may include: a processor; and a memory coupled to the processor, the memory having program instructions stored thereon that, upon execution, cause the IHS to: detect a gesture performed by a user wearing a Head-Mounted Device (HMD) using a first image capture device and a second image capture device concurrently; evaluate the first and second image capture devices; identify the gesture based upon the evaluation; and execute a command associated with the gesture.