Image Generation Device for Robot Training

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Generating a large number of learning images for robot training is challenging, as capturing real operation space images with various workpieces is difficult, while artificially generated virtual images may not accurately represent real robots, leading to inaccurate positional relationship identification and control command output.

Innovation Solution

An image generation device that combines real and virtual images to create learning images, using a first image of a real operation space and a second image with a virtual operation space, and a learner trained through machine learning to convert the virtual image to approximate the real image, thereby generating a learning image suitable for robot training.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If a large number of captured images from real operation space are used to generate learning images, then the accuracy of robot training is improved, but the difficulty of image acquisition and time consumption increase

Engineering Contradiction:

Improveaccuracy of robot trainingVSAvoidtime consumption for image acquisition

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent uses virtual images that copy the appearance and spatial relationships of real operation spaces. These virtual images are generated through computer rendering rather than physical capture, allowing unlimited数量的 images to be produced without time loss. The virtual images replicate the visual characteristics of real scenes, enabling the robot to learn from diverse scenarios efficiently.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent replaces the mechanical capture process (physical cameras recording real scenes) with a computational generation process (virtual rendering). This substitution eliminates the time-consuming aspect of capturing diverse real-world images while maintaining the educational value for robot training through synthesized visual data.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Ease of manufacture

If virtual images are used to generate learning images, then the ease of image generation is improved, but the accuracy of positional relationship identification deteriorates

Engineering Contradiction:

Improveease of image generationVSAvoidaccuracy of positional relationship identification

Core Design Contradiction:

Ease of manufactureVSMeasurement precision

Solution Approach 1:

The patent modifies parameters of virtual images through post-processing techniques. By adjusting rendering parameters, lighting conditions, and spatial coordinates, the virtual images are optimized to accurately represent positional relationships. This parameter optimization ensures that despite being artificially generated, the images maintain the precision needed for accurate robot training.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent incorporates feedback mechanisms where the generated virtual images are evaluated against real image characteristics. This feedback loop allows continuous refinement of the virtual image generation process to ensure accurate representation of positional relationships, while maintaining the ease of generation through automated rendering pipelines.

Inventive Principle:

Principle #23Feedback

3Quantity of substance

If virtual operation space is rendered to create learning images, then the quantity of available images is improved, but the realism and applicability of the images deteriorate

Engineering Contradiction:

Improvequantity of learning imagesVSAvoidrealism and applicability

Core Design Contradiction:

Quantity of substanceVSReliability

Solution Approach 1:

The patent merges virtual images with real image characteristics through a hybrid generation approach. By combining the quantity advantage of virtual rendering with the realism of captured images, the system creates learning images that are both numerous and reliable. The merging process integrates the benefits of both virtual and real image generation methods.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent creates composite learning images that combine virtual and real image elements. These composite images leverage the unlimited quantity capability of virtual generation while incorporating the realism and accuracy of captured images, resulting in a diverse and reliable training dataset that maintains both high quantity and high reliability.

Inventive Principle:

Principle #40Composite materials

Data Source

PatentUS12246456B2Image generation device, robot training system, image generation method, and non-transitory computer readable storage medium

Publication Date: 2025.03.11 OMRON CORP

AI summary

Provided is an image generation device capable of generating, on the basis of inputted images, a learning image for training an action of a robot which carries out a prescribed operation on a workpiece. The image generation device comprises: a first image acquisition unit which acquires a first image capturing a real operation space including the robot and not including the workpiece; a second image acquisition unit which acquires a second image in which a virtual operation space including a virtual robot corresponding to the robot and a virtual workpiece corresponding to the workpiece is rendered; and a learning apparatus so that, in response to an input of the first image and the second image, the apparatus outputs a third image obtained by transforming the second image such that at least the virtual robot included in the second image is made to approximate the robot included in the first image.