AR Floorplan Generation with Voice and Gesture Object Annotation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic floorplan generation methods struggle to accurately capture and identify physical objects within a space, leading to improper object recognition and labeling.
Innovation Solution
A system and method that utilizes augmented reality to capture physical object data by displaying mesh data over an image of the space, allowing users to annotate objects via voice commands, eye gaze, or gestures, and integrating these annotations into the floorplan generation process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automatic floorplan generation uses only triangle mesh data, then the process is fast and automated, but physical objects cannot be properly identified or labeled
Solution Approach 1:
The system segments the floorplan generation process into two distinct phases: (1) automatic mesh data collection for structural elements like walls and floors, and (2) interactive object annotation phase where users specifically identify and label physical objects. This segmentation allows each phase to optimize for its specific purpose while combining to produce a complete floorplan with proper object identification.
Solution Approach 2:
The patent introduces an intermediary annotation layer that bridges the gap between automatic mesh data and object identification. This intermediary system allows users to interact with the mesh data to assign semantic meanings and labels to specific regions, effectively mediating between the automated capture process and the need for detailed object information.
2Measurement precision
If expert skills are used for manual floorplan creation, then accurate object identification is achieved, but the process is time-consuming and expensive
Solution Approach 1:
The system enables users to perform object identification themselves through intuitive interaction with the mesh data, eliminating the need for expert floorplan creators. Users can independently annotate and label objects by interacting with the automatically captured mesh, making the expert-level task of object identification accessible to non-experts while maintaining accuracy.
Solution Approach 2:
The system performs preliminary automatic capture of the spatial environment as mesh data before the user arrives. This preliminary action prepares the foundation work of capturing walls, floors, and spatial relationships, so that when the user arrives, only the object identification and labeling tasks remain, significantly reducing the time and expertise required compared to complete manual floorplan creation.
3Manufacturing precision
If traditional manual measuring and data entry is performed, then accurate floorplans are produced, but the process requires expert skills and is expensive
Solution Approach 1:
The patent replaces the mechanical process of manual measuring and data entry with an automated optical/electronic system that captures the environment as mesh data. Instead of physically measuring walls and angles with tools and manually entering data, the system uses cameras and processors to automatically generate the spatial representation, eliminating the need for expert measuring skills while maintaining accuracy.
Solution Approach 2:
The system creates a digital copy of the physical environment in the form of mesh data that can be manipulated and annotated computationally. This digital copy preserves the geometric accuracy of the physical space while allowing for easier modification, annotation, and analysis compared to traditional manual drawings, reducing both the complexity and expertise required.
Data Source
AI summary
A system and method for captures of physical object data for environment modeling is provided. Mesh data is displayed over an image of a physical space for which a floorplan is to be generated. Instructions to mark an object are received and the instructions include a voice command and eye gaze or gesture. The mesh data is annotated with a marker based on the instructions by placing the marker at a location at which an object is to be identified. A floorplan of the space is generated based on the mesh data. The marker is placed in the floorplan.