AR Motion Conflict Detection via Per-Pixel Probability Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality (AR) systems face challenges in accurately detecting primary and secondary motion in dynamic environments with multiple coordinate frames, leading to inconsistent sensor data and reduced accuracy in localization and tracking.
Innovation Solution
The implementation of Motion Conflict aware Visual Inertial Odometry (MC-VIO) systems, which utilize a convolutional neural network encoder/decoder and a Hidden Markov Model to generate a probability map for pixel-level frame analysis, identifying landmarks, and estimating secondary motion relative to a local coordinate frame.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If prior-art AR systems assume a static environment and handle only primary motion consistent with the inertial coordinate frame, then localization can be performed in a stationary room, but the system becomes substantially less effective in handling dynamic environments with multiple coordinate frames
Solution Approach 1:
The patent segments the image data into different coordinate frame regions using a probability map generated by a convolutional neural network. Each pixel is classified as belonging to either the inertial coordinate frame or the local coordinate frame, allowing the system to process each segment with appropriate motion estimation algorithms.
Solution Approach 2:
The system transitions from a static environment assumption to a dynamic model by introducing a Hidden Markov Model that can switch between different coordinate frame states. This allows the system to adapt to changing environmental conditions and handle multiple coordinate frames dynamically.
2Adaptability or versatility
If the system processes conflicting sensor data from multiple coordinate frames, then it can operate in dynamic environments, but the measurement precision of motion detection deteriorates due to inconsistent data
Solution Approach 1:
The patent divides the image into regions belonging to different coordinate frames using per-pixel probability classification. This segmentation allows the system to process each region with the appropriate motion model, preventing mixing of conflicting data and maintaining measurement precision within each coordinate frame context.
Solution Approach 2:
The probability map generated by the convolutional neural network acts as an intermediary that mediates between the conflicting inertial and local coordinate frame data. It provides a probabilistic classification that guides the Hidden Markov Model in selecting the appropriate coordinate frame for each pixel, resolving the conflict without losing information.
3Measurement precision
If the system uses both camera system and inertial sensors for localization, then it can perform visual-inertial odometry, but the device complexity increases due to integrating multiple sensor types
Solution Approach 1:
The patent merges the camera system and inertial sensors into a unified visual-inertial odometry framework. The probabilistic segmentation model combines data from both sensor types, using the convolutional neural network to process camera images and the Hidden Markov Model to integrate inertial measurements, achieving synergistic improvement in localization accuracy.
4Measurement precision
If the system performs per-pixel analysis to identify coordinate frame membership, then it can resolve motion conflicts, but the computational complexity and processing time increase
Solution Approach 1:
The patent replaces traditional mechanical or algorithmic image analysis methods with a deep learning-based convolutional neural network. This substitution enables efficient per-pixel classification by learning complex patterns directly from image data, reducing computational complexity compared to exhaustive analysis methods while maintaining high identification accuracy.
Data Source
AI summary
A method for motion estimation in an augmented reality (AR) system includes receiving inertial sensor data and image data during movement of the AR system generating a probability map based on the inertial sensor data and the image data, the probability map corresponding to one frame in the image data and including probability values indicating that each pixel in the one frame is in an inertial coordinate frame or a local coordinate frame with a convolutional neural network encoder/decoder, identifying visual observations of at least one landmark in the local coordinate frame based on the image data and the probability map, and generating an estimate of secondary motion in the local coordinate frame based on a first prior state in a hidden Markov model (HMM) corresponding to the local coordinate frame and the visual observations of the at least one landmark in the local coordinate frame.