XR Device Motion Detection Using IMU Sensors and Machine Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial reality (AR) devices require users to navigate virtual menus and select options, which is time-consuming, limits the field-of-view, and conserves compute resources, but adding physical buttons is technically complex and expensive.
Innovation Solution
Implementing a system that uses sensors, such as inertial measurement units (IMUs), to detect physical interactions and apply machine learning models to identify specific motions, allowing users to trigger actions without displaying virtual menus.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If physical buttons are added to AR devices, then ease of operation is improved, but device complexity and cost increase
Solution Approach 1:
The patent replaces mechanical buttons with sensor-based detection systems. IMUs and other sensors detect user actions (head movements, hand gestures, body motions) and translate these into control signals, eliminating the need for physical buttons while maintaining ease of operation.
Solution Approach 2:
The patent introduces sensors (IMUs, cameras, microphones) as intermediaries between the user and the AR system. These sensors detect user actions and provide input data to the system, serving as a mediator that replaces the direct mechanical button interface.
2Adaptability or versatility
If virtual menus are displayed for user interaction, then adaptability is improved, but field-of-view is limited and time is consumed
Solution Approach 1:
The patent implements preliminary action by pre-configuring the AR system with multiple sensors and detection algorithms ready to immediately recognize user actions. The system is pre-programmed with motion recognition models that can instantly identify gestures, head movements, and body actions without requiring users to navigate through virtual menus.
Solution Approach 2:
The patent substitutes the virtual menu navigation system with direct sensor-based action triggering. Instead of requiring users to interact with graphical interfaces, the system directly translates sensor-detected motions into actions, eliminating the time-consuming menu navigation process.
3Use of energy by moving object
If compute resources are conserved by avoiding virtual menus, then energy efficiency is improved, but ease of operation deteriorates
Solution Approach 1:
The patent implements self-service by enabling the AR device to autonomously detect and interpret user actions through sensors without requiring computational resources for rendering and interacting with virtual menus. The system self-determines user intentions through motion detection and automatically triggers appropriate actions, eliminating the need for energy-intensive graphical interface processing.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables quick and efficient triggering of actions on AR devices without the need for virtual menus, conserving compute resources and enhancing user experience by eliminating the complexity and cost of physical buttons.
Implementation Method 1
The XR device can detect a physical interaction with the XR device using one or more sensors (e.g., sensors of an inertial measurement unit (IMU))
Data Source
AI summary
Aspects of the present disclosure can trigger an action based on a motion detected by an artificial reality (XR) device, such as a head-mounted display (HMD). The XR device can display an XR experience to a user. While displaying the XR experience, the XR device can detect a physical interaction with the XR device using one of more sensors (e.g., sensors of an inertial measurement unit (IMU)). The physical interaction can generate a movement profile captured by the one or more sensors. The XR device can identify the physical interaction as a particular motion (e.g., one or more taps on the XR device) by applying a machine learning model to the movement profile. In response to identifying the particular motion, the XR device can trigger an action on the XR device (e.g., activating pass-through on the XR device).