Robot Control Using 2D Vision and Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current robot control methods for pick-and-place operations, such as those using robot arms in factory automation, require extensive trial and error for position and orientation estimation, especially with varied workpiece shapes, and often necessitate expensive 3D cameras for accurate positioning.
Innovation Solution
A neural network-based system that performs robot control using 2D video images, allowing for intuitive and simple configuration, and employs reinforcement learning to quickly converge on a learned model, either on a simulator or an actual robot, with domain randomization to adapt to real-world conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 3D camera is used to acquire position and orientation information with high accuracy, then measurement precision is improved, but device cost increases
Solution Approach 1:
The patent uses a 2D camera to capture images of the workpiece and generates a 3D model through image processing and neural network analysis. This creates a virtual copy of the workpiece from 2D data, eliminating the need for expensive 3D cameras while achieving accurate position and orientation information.
Solution Approach 2:
The patent replaces the mechanical/optical 3D camera system with a computational approach using 2D image capture and neural network-based 3D reconstruction. This substitutes hardware complexity with software intelligence, reducing device cost while maintaining measurement precision.
2Manufacturing precision
If teaching is performed for each workpiece with various shapes, then manufacturing precision is improved, but loss of time increases
Solution Approach 1:
The robot system performs self-learning through reinforcement learning, automatically improving its position and shape estimation capabilities without requiring manual teaching for each workpiece. The neural network learns from trial and error during operation, enabling the system to adapt to various workpiece shapes autonomously.
Solution Approach 2:
The patent uses domain randomization during the learning phase, randomly varying parameters such as lighting conditions, camera angles, and workpiece positions. This enables the neural network to learn robust features that generalize across different workpiece shapes and conditions, eliminating the need for separate teaching for each shape.
3Adaptability or versatility
If reinforcement learning is performed on actual robot, then adaptability to real-world conditions is improved, but loss of time increases
Solution Approach 1:
The patent performs preliminary reinforcement learning in a simulated environment before deploying the model to the actual robot. This pre-training in simulation allows the neural network to learn basic skills and adapt to general conditions without consuming actual robot time, reducing the learning time required on the real system.
Solution Approach 2:
The patent introduces a simulation environment as an intermediary between theory and practice. The simulation acts as a training ground where the robot can learn extensively without physical constraints, and the learned model is then transferred to the actual robot with minimal additional learning time.
Data Source
Figure 1~2
Figure 3~4B
Figure 5
AI summary
This invention provides a robot control device for controlling a robot configured to perform a predetermined operation, where the robot control device comprises an acquisition unit configured to acquire a plurality of images captured by a plurality of image capturing devices including a first image capturing device and a second image capturing device different from the first image capturing device; and a specification unit configured to use the plurality of captured images acquired by the acquisition unit as inputs to a neural network, and configured to specify a control instruction for the robot based on an output as a result from the neural network.