Vision-Guided Robot Training for Non-Expert Object Recognition

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional robot programming for object interaction requires expert knowledge and specialized programming languages, making it time-consuming and inaccessible to non-experts, especially in high-mix low-volume applications where frequent reconfiguration is needed.

Innovation Solution

A method and system enabling non-experts to intuitively define workspace objects and gripping locations using vision-based interaction modes, such as a wand resembling a robot gripper, combined with touch screens and gesture recognition, allowing users to teach robots through gestures, language, or touch interactions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If traditional robot programming using specialized programming languages is used, then the robot can perform object interaction tasks, but the programming process becomes time-consuming and requires expert knowledge

Engineering Contradiction:

Improveease of robot programmingVSAvoidprogramming time

Core Design Contradiction:

Ease of operationVSLoss of time

Solution Approach 1:

The patent replaces traditional mechanical programming approaches (specialized programming languages and expert systems) with vision-based interaction modes. The robot uses computer vision to automatically detect objects, determine their positions, and identify gripping locations through image processing and pattern recognition, eliminating the need for manual coordinate programming and expert knowledge.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The robot system performs self-programming by automatically defining workspace objects and gripping information through vision-based interaction. The system independently identifies objects, determines their locations, and programs gripping parameters without human intervention in the traditional programming sense, enabling non-experts to quickly reconfigure the robot for different tasks.

Inventive Principle:

Principle #25Self-service

2Adaptability or versatility

If expert-level programming is used to define object locations and gripping information, then accurate robot control is achieved, but the system becomes inaccessible to non-experts

Engineering Contradiction:

Improveaccessibility to non-expertsVSAvoidprogramming complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent replaces complex programming languages and expert systems with vision-based interaction modes. The robot uses computer vision to automatically detect objects, determine their positions, and identify gripping locations through image processing and pattern recognition, eliminating the need for manual coordinate programming and expert knowledge.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The vision system acts as an intermediary between the user and the robot's control system. Instead of requiring users to directly program the robot using complex languages, the vision-based interaction mode serves as a mediator that automatically translates visual information into actionable robot commands, simplifying the interface for non-experts.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Adaptability or versatility

If frequent reconfiguration is needed for high-mix low-volume applications, then flexibility is improved, but traditional programming methods become increasingly time-consuming

Engineering Contradiction:

Improvereconfiguration flexibilityVSAvoidreconfiguration speed

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system performs preliminary object recognition and parameter identification automatically before execution. By pre-defining objects and their characteristics through vision-based interaction, the robot is prepared for rapid task switching and reconfiguration without requiring time-consuming reprogramming for each new task.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS12427660B2Method and system for training a robot

Publication Date: 2025.09.30 ABB (SCHWEIZ) AG

AI summary

A method and system are provided for training a robot to recognize objects in the workspace of the robot. Objects in the workspace are identified by the user, and the robot determines candidate objects. Feedback may be used in order for the user to confirm whether the candidate object determined by the robot system matches the object intended by the user. Gripping information for the object may also be identified by the user to train the robot how to grip the object.