Semantic Construction of Physical Environments for XR Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing devices that generate and present environments, such as smartphones, are ineffective in allowing users to interact with the environment effectively.
Innovation Solution
A device with processors and memory that obtains environmental data, determines a bounding surface, detects physical elements, and generates a semantic construction of the environment, including representations and labels, to create an extended reality environment that allows interaction with XR representations of physical elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional environment presentation devices (smartphones) are used, then environment generation capability is provided, but user interaction effectiveness with the environment deteriorates
Solution Approach 1:
The patent creates virtual copies of physical elements (objects, surfaces, spaces) in the environment and allows users to interact with these copies. The system generates virtual representations that mirror physical elements, enabling interaction with the environment through these copies rather than direct physical interaction, thus improving ease of operation while maintaining environmental fidelity
Solution Approach 2:
The patent introduces an intermediary layer between the user and the physical environment - a virtual environment that mediates interaction. This intermediary virtual space allows users to interact with environmental elements in a more effective manner while still representing the physical environment accurately, resolving the contradiction between interaction ease and environmental fidelity
2Adaptability or versatility
If environmental data is processed to generate detailed semantic construction, then interaction capability improves, but computational complexity increases
Solution Approach 1:
The patent segments the environment into distinct elements (objects, surfaces, spaces) and processes each segment separately. By dividing the environmental data into manageable components and generating virtual copies for each segment individually, the system achieves detailed interaction capability while reducing overall computational complexity through modular processing
Solution Approach 2:
The patent applies different levels of processing and detail to different regions or elements within the environment based on their importance or interaction potential. Rather than uniformly processing all environmental data at high complexity, the system focuses computational resources on key interactive elements, improving adaptability while managing device complexity
Data Source
AI summary
In some implementations, a method includes obtaining environmental data corresponding to a physical environment. In some implementations, the method includes determining, based on the environmental data, a bounding surface of the physical environment. In some implementations, the method includes detecting a physical element located within the physical environment based on the environmental data. In some implementations, the method includes determining a semantic label for the physical element based on at least a portion of the environmental data corresponding to the physical element. In some implementations, the method includes generating a semantic construction of the physical environment based on the environmental data. In some implementations, the semantic construction of the physical environment includes a representation of the bounding surface, a representation of the physical element and the semantic label for the physical element.