LiDAR–EMG Fusion for Mediated Reality Object Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mediated reality devices struggle to accurately classify real-world objects based on user interactions, limiting the effectiveness of mediated reality applications.
Innovation Solution
A system that combines LiDAR data with electromyography data to classify objects by mapping user interactions, such as touch, force, or movement, using position data to enhance object recognition and control mediated reality functions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LiDAR data alone is used for object classification, then the system complexity is low, but the classification accuracy is insufficient
Solution Approach 1:
The patent combines LiDAR data with electromyography (EMG) data from the user's hand to improve object classification accuracy. The system merges spatial information from LiDAR with physiological interaction data from EMG sensors, creating a multi-modal data fusion approach that enhances classification precision beyond what either sensor could achieve alone.
Solution Approach 2:
The patent introduces position data as an intermediary element that maps the user's hand position to the LiDAR data points. This intermediary mapping layer enables the system to correlate EMG interaction data with specific spatial locations on the object, improving the accuracy of classification by establishing a spatial context for the physiological signals.
2Measurement precision
If multiple data sources are combined for object classification, then the classification accuracy improves, but the device complexity increases
Solution Approach 1:
The patent segments the classification system into distinct functional modules: LiDAR data acquisition, EMG data acquisition, position data calculation, and data fusion. Each module processes specific types of data independently before combining them, which manages complexity by organizing the integration process into manageable, specialized components rather than a monolithic system.
Solution Approach 2:
The patent creates a unified classification framework that can process multiple data types (LiDAR, EMG, position) through a common processing pipeline. The system uses a single classification algorithm that accepts multi-modal input, providing a universal solution that handles diverse data sources without requiring separate processing systems for each sensor type.
3Measurement precision
If interaction data from electromyography device is integrated, then the object classification becomes more accurate, but the ease of operation decreases
Solution Approach 1:
The patent implements self-calibration mechanisms where the system automatically adapts to individual user characteristics. The electromyography sensors are calibrated through automated procedures that learn the user's specific muscle activation patterns, eliminating the need for manual setup and reducing operational complexity while maintaining high measurement precision.
Solution Approach 2:
The patent performs preliminary calibration of the electromyography system during an initial setup phase, storing user-specific parameters for later use. This preliminary action captures the user's physiological characteristics in advance, so that during normal operation the system can use pre-stored calibration data to maintain accuracy without requiring the user to perform complex setup procedures each time.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enhances the accuracy of object classification in mediated reality environments, allowing for improved rendering and interaction with real-world objects by aligning mediated reality content with user interactions.
Implementation Method 1
obtaining LiDAR data from a mediated reality device wherein the mediated reality device is used by a user and the LiDAR data comprises data points representing at least part of an object
Implementation Method 2
obtaining interaction data indicating how the user's hand is interacting with the at least part of the object wherein at least some of the interaction data is obtained from an electromyography device coupled to the user
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Examples of the disclosure data enable interactions of a user with an object to be used to improve the classification of the object in mediated reality applications. In examples of the disclosure LiDAR data is obtained from a mediated reality device. The mediated reality device is used by a user and the LiDAR data comprises data points representing at least part of an object. Position data relating to a position of the user's hand relative to the at least part of the object is obtained and interaction data indicating how the user's hand is interacting with the at least part of the object is also obtained. At least some of the interaction data is obtained from an electromyography device coupled to the user. The position data is used to map the interaction data to the LiDAR data to enable the interaction data to be used to classify the at least part of the object.