Contextual Applications in Mixed Reality via Object Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual and augmented reality systems lack the ability to seamlessly integrate contextual applications based on real-world environments and user interactions, limiting their functionality and user experience.
Innovation Solution
A mixed reality (MR) system that uses object detection and recognition algorithms to trigger contextual applications based on user location, proximity to real-world objects, and user actions, allowing for the presentation of virtual objects and interactions within the MR environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If contextual applications are integrated based on real-world environment detection, then user experience and interactivity are improved, but system complexity and computational requirements increase
Solution Approach 1:
The system segments the environment into discrete identifiable objects and features that can be independently detected and processed. Object detection algorithms break down the complex real-world scene into individual detectable elements, allowing contextual applications to be triggered by specific objects rather than requiring full scene understanding, thus managing system complexity while maintaining adaptability.
Solution Approach 2:
The patent introduces intermediate layers including virtual objects, markers, and detection algorithms that mediate between the real-world environment and the virtual reality system. These intermediaries translate physical environmental features into computable data structures that can trigger contextual applications, reducing the direct complexity burden on the core VR system while enabling versatile environmental integration.
2Productivity
If object detection and recognition algorithms are implemented in real-time, then contextual application triggering is improved, but processing time and computational resources increase
Solution Approach 1:
The system performs preliminary actions by pre-processing and pre-identifying key environmental features and objects before they are needed for application triggering. Detection algorithms continuously monitor and catalog environmental objects in advance, so when a contextual application needs to be triggered, the identification work is already done or nearly done, reducing real-time processing delays while maintaining high productivity in application response.
3Adaptability or versatility
If virtual objects are integrated with real-world environment, then immersion and user experience are improved, but alignment precision and spatial accuracy requirements increase
Solution Approach 1:
The system creates simplified digital copies or representations of real-world environmental features rather than attempting to perfectly replicate all spatial details. Virtual objects are positioned and oriented based on detected environmental landmarks and reference points, providing sufficient spatial accuracy for contextual applications without requiring millimeter-level precision, thus balancing immersion with practical measurement capabilities.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method of implementing a plurality of contextual applications within a mixed reality (MR) environment on an MR-capable device of a user is disclosed. At least one real-world object is identified in the MR environment by applying an object recognition algorithm to one or more attributes of the at least one real- world object that are captured by sensors of the MR-capable device. A first contextual application of the plurality of contextual applications is used to determine an association between a first set of contextual triggers and a second contextual application of the plurality of contextual applications. A second contextual application is initiated based on a satisfying of the at least one contextual trigger. A function is invoked within the second contextual application based on an interaction of the user with at least one virtual object satisfying a second set of contextual triggers associated with the second contextual application.