AR Robot Assembly Teaching Without Full VR Modeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for programming robots to perform assembly tasks are limited by the need for physical interaction with large, heavy, or dangerous objects, and recreating complex environments in virtual reality requires significant computational resources.
Innovation Solution
A method using augmented reality (AR) devices allows a human operator to demonstrate assembly tasks with both virtual and physical objects, enabling a robotic system to learn the task by tracking the operator's manipulation of virtual objects and generating a program based on their pose and trajectory, without the need for external sensors in the physical environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If programming is done in a complete virtual environment using VR devices, then the robot can be taught tasks without physical interaction with dangerous objects, but extensive computational resources are required to recreate complex physical environments
Solution Approach 1:
The patent introduces an augmented reality environment as an intermediary between the physical world and the virtual programming environment. This AR environment overlays virtual objects and interfaces onto the real physical environment, allowing operators to program robots remotely without being physically present with dangerous objects, while avoiding the need to fully recreate complex physical environments in pure virtual reality, thus reducing computational resource requirements
Solution Approach 2:
The patent segments the programming environment into virtual elements (AR overlays, virtual objects, interfaces) that are superimposed on the actual physical environment. This segmentation allows the system to use minimal virtual elements for programming interactions while leveraging the existing physical environment, rather than requiring complete virtual recreation of all physical assets
2Measurement precision
If programming is done through physical demonstration with actual objects, then the robot learns accurate real-world interactions, but the operator must physically handle large, heavy, or dangerous components
Solution Approach 1:
The patent uses virtual copies of physical objects displayed through AR technology. These virtual objects replicate the appearance, position, and interaction characteristics of real objects, allowing operators to demonstrate tasks with accurate visual feedback without physically handling dangerous components. The virtual objects are sufficiently realistic to teach precise robotic interactions while eliminating physical hazards
3Adaptability or versatility
If complex physical environments are recreated in VR for programming, then complete virtual immersion is achieved, but significant computational resources are consumed
Solution Approach 1:
Instead of immersing the operator completely in a virtual environment (inverting the traditional VR approach), the patent inverts the approach by bringing the virtual interface into the physical world through AR. The operator remains in the real environment with minimal virtual overlays, achieving sufficient adaptability for programming while dramatically reducing the computational infrastructure required compared to full VR immersion
Data Source
AI summary
A method for programming a robotic system by demonstration is described. In one aspect, the method includes displaying a first virtual object in a display of an augmented reality (AR) device, the first virtual object corresponding to a first physical object in a physical environment of the AR device, tracking, using the AR device, a manipulation of the first virtual object by a user of the AR device, identifying an initial state and a final state of the first virtual object based on the tracking, the initial state corresponding to an initial pose of the first virtual object, the final state corresponding to a final pose of the first virtual object, and programming by demonstration a robotic system using the tracking of the manipulation of the first virtual object, the first initial state of the first virtual object, and the final state of the first virtual object.