3D Environment User Interface for 360 Degree Image Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing user interfaces for 3D environments created from 360° cameras face challenges in providing a robust and simple method for users to interact effectively, particularly due to inaccuracies in mapping 3D meshes to 360° images, especially under strong light conditions or outdoors.
Innovation Solution
A method and system that displays a sequence of 360° images, allows users to interact with detected points of interest associated with 3D coordinates, and processes these interactions to enable content association and activation, simplifying user navigation and content placement within the 3D environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If text tags are added to the 3D mesh during configuration, then the 3D environment can be augmented with additional content, but the process becomes difficult when the 3D mesh does not map to the 360° image accurately
Solution Approach 1:
The patent introduces an intermediary system that detects points of interest directly within the 360° image sequence rather than relying on the inaccurate 3D mesh. This intermediary detection mechanism bridges the gap between the visual content and user interaction, allowing text tags to be associated with detected points of interest in the image space where the mapping is accurate, thus resolving the contradiction between mapping reliability and configuration ease
2Adaptability or versatility
If a proprietary camera with multiple sensors is used to capture 3D environments, then the capture capability is enhanced, but the technology complexity increases
Solution Approach 1:
The patent replaces the complex proprietary camera system with a standard 360° camera that captures image sequences. Instead of relying on multiple sensors and complex hardware, the system creates a virtual 3D environment by processing sequences of 360° images. This copying approach uses software processing of simple image captures to achieve the same functional result, dramatically reducing device complexity while maintaining capture versatility
Solution Approach 2:
The patent substitutes the mechanical sensor-based 3D capture system with an optical image sequence-based system. Rather than using infrared sensors, gyroscopes, and mechanical rotation mechanisms to build 3D meshes, the system uses standard 360° camera images processed through point of interest detection and coordinate mapping algorithms, replacing complex mechanical sensing with simpler optical capture and computational processing
3Ease of operation
If the 3D mesh is used for user interaction, then navigation in the 3D environment is enabled, but the interaction becomes less accurate when lighting conditions are strong or outdoor
Solution Approach 1:
The patent performs preliminary detection of points of interest within the 360° image sequences before user interaction occurs. By pre-identifying and marking points of interest in the image space where lighting conditions are already accounted for in the captured images, the system ensures accurate point detection regardless of lighting conditions. This preliminary action in the image domain avoids the lighting-sensitive 3D mesh generation step while maintaining navigation capability
Solution Approach 2:
The patent shifts the interaction paradigm from 3D mesh coordinate space to 2D/360° image space with associated 3D coordinates. Instead of interacting directly with the lighting-vulnerable 3D mesh, users interact with points of interest detected in the robust 360° image sequences, where point detection is not affected by lighting conditions. This dimensionality change moves the detection process to a more reliable space while preserving navigation functionality through coordinate association
Data Source
AI summary
A method is shown for providing a user interface for a 3D environment of 360° images. The method includes displaying a first 360 degree image in a sequence of 360 degree images; receiving user input to interact with one of a plurality of points of interest detected within the first 360 degree image; and processing the interaction. Each point of interest is associated with a 3D coordinate. A system and software are also disclosed.