Mixed Reality Object Guidance Using 3D Interaction Indicators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing e-commerce applications in mixed reality (MR) lack effective methods for guiding users to interact with objects based on contextual information, relying on 2D user-interface elements for product information display.
Innovation Solution
A computer-implemented method and system that uses an MR device with a camera and processor to identify objects, extract contextual information, and guide users to perform interactions through interaction indicators, utilizing machine learning and computer vision to determine relevant recommendations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If 2D user-interface elements are used to display product information, then information can be presented, but user engagement and interaction guidance are insufficient
Solution Approach 1:
The patent transitions from 2D user-interface elements to 3D spatial indicators in mixed reality. Interaction indicators are positioned in 3D space relative to the physical object, providing directional guidance and contextual information that 2D interfaces cannot convey. This dimensional transformation enables users to understand object interactions more naturally through spatial relationships.
Solution Approach 2:
The patent introduces interaction indicators as intermediary elements between the user and the physical object. These indicators serve as mediators that convey contextual information about possible interactions without requiring direct contact with the object. The indicators act as a bridge, translating object properties and potential interactions into visible guidance cues.
2Adaptability or versatility
If generic product information is displayed, then basic details are provided, but context-specific recommendations are lacking
Solution Approach 1:
The system continuously monitors user interactions and object states, using this feedback to dynamically update interaction indicators. When users interact with objects or when object conditions change, the system adjusts the displayed recommendations in real-time. This feedback loop enables context-specific recommendations without requiring complex pre-programming for every scenario.
Solution Approach 2:
The system automatically detects objects, extracts their properties, and generates relevant interaction indicators without requiring manual configuration. The mixed reality device autonomously processes sensor data, identifies objects, and presents contextualized recommendations, reducing the complexity burden on users while maintaining high adaptability.
3Loss of information
If comprehensive product information is provided through traditional interfaces, then detailed data is available, but user engagement and interaction motivation are reduced
Solution Approach 1:
Instead of presenting comprehensive information uniformly, the system delivers information locally at the specific location of each object. Interaction indicators are positioned adjacent to or on the object itself, providing relevant information exactly where the user needs it. This localized information delivery maintains completeness while enhancing engagement through spatial context.
Solution Approach 2:
The patent moves information display from 2D screens to 3D spatial positioning around objects. Comprehensive product information is distributed in 3D space, with different information elements positioned at different locations and orientations. This spatial distribution makes information more engaging and easier to access during natural interaction with objects.
Data Source
AI summary
A computer-implemented method for guiding a user interacting with an object is provided. The method comprises receiving informational cues of a scene including the object, the informational cues pertaining to sensor-retrieved data; identifying the object based on the informational cues; extracting contextual information based on the informational cues; determining at least one interaction indicator configured to indicate an interaction performable on the identified object, the at least one interaction indicator being based at least in part on the contextual information; and guiding the user to perform the interaction on the object based on the at least one interaction indicator through control of a mixed reality (MR) device.