Mixed Reality Robot Programming for Reusable Spatial Skills
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current robotic programming methods require expert knowledge, are not scalable, and lack reusability and transferability, as they rely on code-based programming, teaching pendants, and vision-based systems that struggle to capture 3D interactions and dynamical relationships between robots and objects.
Innovation Solution
A mixed reality (MR) assisted programming platform that uses MR tools to teach robotic systems different skill sets by learning spatial parameters through interactive tasks, allowing for the installation of a skill engine in an MR environment, reducing dependency on expert knowledge and enabling scalable skill accumulation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If code-based programming is used to program robotic systems, then the robot can perform complex tasks with predefined motions and sensor analysis, but it requires expert knowledge in robotics and programming languages
Solution Approach 1:
The patent introduces motion capture technology as an intermediary between human demonstration and robot execution. The motion capture system records human movements and translates them into robot commands, serving as a mediator that bridges the gap between simple demonstration and complex task execution without requiring programming expertise from the user.
Solution Approach 2:
The patent replaces traditional code-based programming mechanisms with motion capture-based programming. Instead of using programming languages and code editors, users demonstrate tasks physically through motion capture, and the system automatically generates the control program, substituting the mechanical programming process with a more intuitive physical demonstration approach.
2Ease of operation
If teaching pendant approach is used to program industrial robots, then common interfaces are provided for production and assembly tasks, but the teaching process must be repeated for each task and lacks reusability
Solution Approach 1:
The patent uses motion capture to record and store task demonstrations as reusable digital copies. Once a task is demonstrated and captured, it can be copied and reused for multiple similar tasks or modified for variations, eliminating the need to repeat the teaching process for each new task while maintaining standardized interfaces.
Solution Approach 2:
The patent performs preliminary action by capturing and storing task demonstrations in advance. The motion capture system records human expertise and task knowledge beforehand, creating a library of reusable task templates that can be quickly deployed and adapted for future tasks, rather than teaching each task from scratch.
3Loss of information
If vision based systems are used for robot programming, then object recognition and path planning are enabled, but the system cannot capture physical connections and dynamical interactions between robots and objects
Solution Approach 1:
The patent merges vision-based systems with motion capture technology to create a comprehensive programming system. The vision system captures visual information about objects and environments, while the motion capture system simultaneously records physical interactions and dynamical relationships. By combining these two systems, the patent achieves both visual information capture and reliable understanding of physical interactions.
4Measurement precision
If motion tracking is used to learn sequences of observed motions, then complex movements and realistic physical interactions are accurately recreated, but the equipment cost is high and requires highly-skilled technicians to operate
Solution Approach 1:
The patent implements self-service by enabling non-experts to program robots through simple physical demonstration. The motion capture system automatically processes the demonstrated motions and generates executable programs without requiring user intervention or expertise in programming or system operation. The system serves itself by automatically translating human movement into robot commands, eliminating the need for highly-skilled technicians.
Data Source
AI summary
A computer-based system and method is disclosed for spatial programming of a robotic device. A mixed reality tool may select an object related to one or more interactive tasks for the robotic device. A spatial location of the object may be located including Cartesian coordinates and orientation coordinates of the object. An application program may be executed to operate the robotic device using the spatial location. Based on initial parameters, execution of the one or more tasks by the robotic device on the object related to a skill set may be simulated in a mixed reality environment.