Robot Picking Learning From 3D State Feedback in Bulk Loads

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional robot systems require human intervention and trial-and-error to optimize the picking up of workpieces in bulk-loaded states, leading to a low success rate due to the need for presetting workpiece extraction settings and operation programming.

Innovation Solution

A machine learning device that observes state variables from a three-dimensional measuring device and learns manipulated variables for robot operations, including command data and measurement parameters, to optimize the picking process without human intervention, using a state variable observation unit, operation result obtaining unit, and learning unit.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If conventional robot systems use presetting and programming for workpiece picking, then the system can operate with structured control, but the success rate of picking up workpieces in bulk-loaded states remains low and requires extensive human trial-and-error adjustment

Engineering Contradiction:
Improvesuccess rate of picking operationVSAvoidcomplexity of presetting and programming
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The robot system performs self-learning through autonomous trial-and-error picking operations. The learning unit automatically acquires knowledge about workpiece positions and picking strategies by observing successful and unsuccessful operations, eliminating the need for human operators to manually program and adjust picking parameters. This self-service mechanism directly improves reliability while reducing the complexity of presetting and programming.

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

The system implements a feedback loop where the learning unit receives information about picking results (success or failure) and uses this feedback to refine future picking operations. The state variable observation unit monitors robot states and workpiece positions, providing continuous feedback that enables the system to adapt and improve its picking strategy automatically, thereby increasing success rate without requiring complex manual programming.

Inventive Principle:
Principle #23Feedback

2Reliability

If human operators perform trial-and-error adjustment to optimize picking operations, then the success rate can be improved, but the time and effort required for setup and optimization increases significantly

Engineering Contradiction:
Improvesuccess rate of picking operationVSAvoidtime for trial-and-error adjustment
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The robot system autonomously performs the optimization process that would otherwise require extensive human trial-and-error adjustment. The learning unit automatically learns from each picking attempt and refines its strategy over time, eliminating the need for human operators to spend significant time adjusting parameters. This self-service capability directly reduces the time loss associated with manual optimization while maintaining or improving success rates.

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

The system performs preliminary learning actions by conducting autonomous trial picking operations to gather data about workpiece positions and optimal picking strategies. This preliminary action phase allows the robot to build its knowledge base automatically, so that when actual production begins, the system is already optimized without requiring time-consuming human adjustment during setup.

Inventive Principle:
Principle #10Preliminary action

3Adaptability or versatility

If the robot system operates with pre-programmed instructions, then the control process remains simple, but the system cannot adapt to variations in workpiece positioning and bulk-loaded states

Engineering Contradiction:
Improveadaptability to random workpiece placementVSAvoidcomplexity of learning and observation units
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent replaces traditional mechanical control systems (pre-programmed instructions) with an intelligent learning system. The learning unit uses machine learning algorithms to process sensor data and determine picking strategies, substituting rigid mechanical programming with adaptive computational intelligence. This enables the system to handle random workpiece placement while the complexity is managed through software-based learning rather than mechanical reconfiguration.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system transitions from static pre-programmed control to dynamic adaptive control. The learning unit continuously updates its knowledge base based on observed states and outcomes, allowing the control strategy to evolve dynamically in response to varying workpiece positions and configurations. This dynamic approach provides adaptability to random placement while the complexity is contained within the learning algorithm rather than requiring complex mechanical or programming changes.

Inventive Principle:
Principle #15Dynamics

Data Source

PatentUS11780095B2Machine learning device, robot system, and machine learning method for learning object picking operation
Publication Date: 2023.10.10 FANUC LTD
  • US11780095B2 patent drawing
  • US11780095B2 patent drawing
  • US11780095B2 patent drawing

AI summary

A machine learning device that learns an operation of a robot for picking up, by a hand unit, any of a plurality of objects placed in a random fashion, including a bulk-loaded state, includes a state variable observation unit that observes a state variable representing a state of the robot, including data output from a three-dimensional measuring device that obtains a three-dimensional map for each object, an operation result obtaining unit that obtains a result of a picking operation of the robot for picking up the object by the hand unit, and a learning unit that learns a manipulated variable including command data for commanding the robot to perform the picking operation of the object, in association with the state variable of the robot and the result of the picking operation, upon receiving output from the state variable observation unit and output from the operation result obtaining unit.