AR Robot Simulation for Moving Workpiece Interference Checking

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing robot simulation methods fail to accurately predict interference with peripheral devices when a robot operates on a moving workpiece, as the motion path taught in offline simulations may not account for real-world conditions, leading to potential collisions and operational inefficiencies.

Innovation Solution

A simulation device and system that uses an image sensor to capture the real space, an augmented reality display to overlay a virtual robot on the actual robot, a workpiece management section to track the moving workpiece, and a motion control section to control the virtual robot's motion based on the workpiece's position, allowing for real-time visualization and control of potential interference with peripheral devices.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If offline simulation is performed to plan robot motion path, then robot operation efficiency is improved, but accuracy of predicting interference with peripheral devices deteriorates

Engineering Contradiction:
Improverobot operation efficiencyVSAvoidaccuracy of predicting interference
Core Design Contradiction:
ProductivityVSReliability

Solution Approach 1:

The system performs preliminary action by conducting an offline simulation to generate a motion path for the robot before actual operation. This allows the robot to operate efficiently based on pre-planned paths while the augmented reality component provides real-time verification to ensure reliability by showing potential interference with peripheral devices.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The augmented reality display acts as an intermediary between the offline simulation results and the actual robot operation. It visualizes the virtual robot's motion path overlaid on the real workspace, allowing operators to verify potential interference with peripheral devices before executing the actual motion, thus bridging the gap between simulated efficiency and real-world safety.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Adaptability or versatility

If motion path is corrected based on workpiece position, then adaptability to moving workpiece is improved, but understandability of actual robot motion deteriorates

Engineering Contradiction:
Improveadaptability to moving workpieceVSAvoidunderstandability of actual robot motion
Core Design Contradiction:
Adaptability or versatilityVSLoss of information

Solution Approach 1:

The system creates a virtual copy of the robot that mirrors the actual robot's motion path in the augmented reality display. This virtual robot is controlled based on the detected workpiece position, allowing the system to adapt to moving workpieces while maintaining understandability through the visual representation of the robot's intended motion path.

Inventive Principle:
Principle #26Copying

Solution Approach 2:

The system implements feedback by detecting the actual workpiece position and using this information to adjust the virtual robot's motion path in real-time. The augmented reality display continuously updates to show the corrected motion path based on workpiece position, providing visual feedback that maintains understandability even as the system adapts to moving workpieces.

Inventive Principle:
Principle #23Feedback

3Loss of information

If augmented reality display shows virtual robot overlaid on actual robot, then visualization of motion path is improved, but device complexity increases

Engineering Contradiction:
Improvevisualization of motion pathVSAvoidsystem complexity
Core Design Contradiction:
Loss of informationVSDevice complexity

Solution Approach 1:

The image sensor serves multiple functions: it captures the real workspace for augmented reality display, detects the workpiece position for motion correction, and provides the base image for overlaying the virtual robot. This multi-functionality reduces the need for separate dedicated components, thereby managing device complexity while achieving improved visualization.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system merges the offline simulation results with the real-time augmented reality display by overlaying the virtual robot image on the captured real workspace image. This combination provides comprehensive visualization of the motion path while using integrated processing to manage the complexity of handling both simulated and real-world data streams.

Inventive Principle:
Principle #5Merging (Combining)

Data Source

PatentUS11904478B2Simulation device and robot system using augmented reality
Publication Date: 2024.02.20 FANUC LTD
  • US11904478B2 patent drawing
  • US11904478B2 patent drawing
  • US11904478B2 patent drawing

AI summary

A simulation device includes an image sensor which captures an image of a real space including an actual robot and a peripheral device arranged at the periphery of the actual robot, an augmented reality display section which displays a virtual robot overlaid on the actual robot shown in the captured image, a workpiece management section which manages a position of a moving workpiece, and a motion control section which controls a motion of the virtual robot on the basis of the position of the workpiece.