Mixed Reality Robot Training System for Intuitive Path Programming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current industrial robot training systems are complex and require familiarity with programming languages and abstract concepts, making it difficult to visualize and troubleshoot robot programs, especially in real-world scenarios.
Innovation Solution
A mixed reality training system that combines a transparent display, video input, and user input devices to superimpose virtual robot paths and objects onto real-world scenes, allowing users to control, program, and simulate industrial robots using hand gestures and head movements, with a computing device processing inputs to visualize and adjust robot paths for collision avoidance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional offline software or teach pendant is used for robot programming, then robot training can be performed, but the system complexity and difficulty of operation increase significantly
Solution Approach 1:
The patent creates a virtual copy of the real-world scene and robot within a mixed reality environment. Users interact with virtual representations of physical objects, robot paths, and program elements, allowing training without direct manipulation of complex programming systems. This virtual copying simplifies the interface while maintaining fidelity to the actual robot operations.
Solution Approach 2:
The mixed reality environment serves as an intermediary between the user and the actual robot programming system. Instead of directly using complex programming languages or teach pendants, users interact with intuitive virtual representations that translate their actions into robot commands, reducing the operational barrier while preserving programming capability.
2Loss of information
If traditional robot programming methods are used, then robot tasks can be programmed, but visualization and troubleshooting of robot programs in real scenes become difficult
Solution Approach 1:
The patent merges the virtual robot program visualization with the real-world scene in a mixed reality environment. Virtual robot paths, program execution traces, and diagnostic information are overlaid directly onto the physical robot and workspace, allowing users to see program behavior in context without switching between separate simulation and real-world views.
Solution Approach 2:
The patent adds a virtual dimension to the real-world scene, creating a layered visualization where digital program information exists alongside physical robot components. This dimensional overlay allows simultaneous viewing of both the physical robot operation and the virtual program representation, enhancing troubleshooting capability through multi-layered information presentation.
3Measurement precision
If abstract programming concepts like coordinate frames are used, then precise robot control is achieved, but user understanding and programming efficiency decrease
Solution Approach 1:
The patent replaces abstract mathematical coordinate frame concepts with visual, spatial representations in mixed reality. Instead of requiring users to understand and manipulate mathematical coordinate transformations, the system provides intuitive visual cues and spatial references that naturally convey position and orientation information, maintaining precision while eliminating mathematical abstraction barriers.
Data Source
AI summary
Unique systems, methods, techniques and apparatuses of a robot training system are disclosed. One exemplary embodiment is an industrial robot training system comprising a mixed reality display device structured to superimpose a virtual scene on a real-world view of a real-world scene including a plurality of physical objects including an industrial robot, a video input device, and a computing device. The computing device is structured to detect physical objects using video output from the video input device, generate virtual objects using the detected physical objects, simulate a virtual robot path, determine one movement of the series of robot movements causes a collision, adjust the virtual robot path so as to avoid the collision between the two virtual objects of the plurality of virtual objects, and program the industrial robot to perform a real robot path using the adjusted virtual robot path.