3D Robot Interaction Mapping for Accurate Action Targeting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
User interactions with robots, particularly those involving visual control, are often unintuitive and misleading due to the use of 2D visual content, which can distort object distances, complicating control tasks in robots like drones and cleaning robots.
Innovation Solution
A method and system utilizing three-dimensional (3D) mapping to provide a geometrical user interface, where key points in images are detected and descriptors are used to create a 3D map, allowing users to input actions based on accurate 3D environment representations, enabling precise control of robotic devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If 2D visual content is used for robot control interface, then the interface is simple to implement, but the object distances are distorted and control becomes unintuitive
Solution Approach 1:
The patent transitions from 2D visual content to 3D mapped environment representation, adding a spatial dimension to the control interface. This allows users to interact with a three-dimensional model that accurately represents real-world distances and spatial relationships, making control more intuitive while maintaining implementation feasibility through software-based 3D mapping
2Ease of manufacture
If 2D visual content is used for robot control interface, then the implementation is straightforward, but the visual content is misleading regarding object distances
Solution Approach 1:
By implementing a 3D mapping system that reconstructs the environment in three dimensions, the patent preserves accurate distance and spatial information while maintaining ease of implementation through software processing of sensor data. The 3D model allows precise distance representation without complex hardware modifications
Solution Approach 2:
The patent creates a virtual 3D copy of the real environment that accurately replicates spatial relationships and distances. This virtual model serves as an intermediate representation that preserves measurement accuracy while allowing for intuitive user interaction, separating the complexity of precise measurement from the user interface
Data Source
AI summary
Systems and methods for providing geometric interactions via three-dimensional mapping. A method includes determining a plurality of first descriptors for a plurality of key points in a plurality of first images, wherein each first image shows a portion of a 3D environment in which a robotic device and a visual sensor are deployed; generating a 3D map of the 3D environment based on the plurality of key points and the plurality of descriptors; determining a pose of the visual sensor based on at least one second descriptor and the plurality of first descriptors, wherein the second image is captured by the visual sensor; and determining a target action location based on at least one user input made with respect to a display of the second image and the pose of the visual sensor, wherein the target action location is a location within the 3D environment.