Industrial Robot Gripping Control With Synthetic-to-Real Training

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing systems for gripping objects in industrial processes face challenges in accuracy and flexibility, especially when objects are randomly arranged, requiring precise position and orientation estimation and recognition.

Innovation Solution

A distributed, computer-implemented system that uses a central training computer and local computing units to control robots for gripping objects. This system employs an artificial neural network (ANN) pre-trained with synthetically generated data and post-trained with real data for improved object recognition and position estimation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If manually designed features are used for object recognition, then the system can recognize objects in images, but the system lacks flexibility and requires expert knowledge for optimization

Engineering Contradiction:

Improveobject recognition accuracyVSAvoidsystem flexibility

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent replaces manually designed feature extraction systems with machine learning-based automatic feature learning. Instead of relying on expert-designed features, the system uses neural networks to automatically learn optimal features from data, substituting mechanical feature engineering with intelligent automated learning.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system enables objects to serve themselves by automatically learning their own recognition features without human intervention. The machine learning model autonomously extracts and optimizes features from training data, eliminating the need for expert knowledge in feature design.

Inventive Principle:

Principle #25Self-service

2Reliability

If real training data is used for machine learning, then the system can be trained on actual objects, but data acquisition is time-consuming and labor-intensive

Engineering Contradiction:

Improvetraining data accuracyVSAvoidtraining data preparation time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent uses synthetic data generated from 3D models as copies of real objects for training. Instead of capturing actual physical objects, the system creates virtual representations that replicate the essential characteristics needed for training, significantly reducing data preparation time while maintaining training effectiveness.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The system changes the state of training data from physical real-world objects to synthetic digital representations. By transforming the parameters and format of training data, the system eliminates the time-consuming process of physical data collection while preserving the essential information needed for learning.

Inventive Principle:

Principle #35Parameter changes

3Ease of manufacture

If synthetically generated training data is used, then data acquisition is fast and easy, but the simulation lacks realism and accuracy

Engineering Contradiction:

Improvetraining data generation easeVSAvoidgrasping task accuracy

Core Design Contradiction:

Ease of manufactureVSMeasurement precision

Solution Approach 1:

The patent performs preliminary rendering of synthetic training data at different resolutions before the actual training process. This preliminary action creates a hierarchy of synthetic data quality levels, allowing the system to start training with lower-resolution data and progressively improve with higher-resolution synthetic data, bridging the gap between synthetic ease and real-world accuracy.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system dynamically adjusts the resolution and quality of synthetic training data during the training process. By transitioning from lower-resolution to higher-resolution synthetic data, the system adapts the training process to progressively reduce the reality gap while maintaining computational efficiency.

Inventive Principle:

Principle #15Dynamics

4Measurement precision

If high-resolution rendering is performed for all training data, then realism is improved, but computational resources and time are excessively consumed

Engineering Contradiction:

Improvetraining data realismVSAvoidrendering time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent applies partial high-resolution rendering only to the extent necessary for effective training. Instead of rendering all training data at maximum resolution, the system uses a multi-resolution approach where only essential portions are rendered at high quality, reducing overall computational burden while maintaining sufficient realism for accurate training.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS20250028300A1Control of an industrial robot for a gripping task

Publication Date: 2025.01.23 VATHOS GMBH

AI summary

In one aspect, the present invention relates to a distributed system for controlling at least one robot (R) in a gripping task for gripping objects (O) of different types which are arranged in a working area (CB, B, T) of the robot (R). The system comprises a central training computer (CTR), which is designed for pre- and post-training, and at least one local processing unit (LCU), on which real image data of the object (O) is recorded, which is used to generate post-training data.