Robotic Item Picking With Recognition Error Recovery Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object recognition systems for robotic depalletization, especially in mixed-SKU environments, face challenges with incorrect recognition leading to failed palletization and depalletization, requiring time-consuming manual remote recovery and not improving recognition capabilities.
Innovation Solution
A method and system that iteratively collect object data through a vision sensor, perform object recognition to determine orientation and location, and for unrecognized objects, initiate a recovery process involving human operator input to correct recognition errors, thereby improving object recognition functions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual remote recovery is used to correct recognition errors, then recognition accuracy can be improved, but the process becomes time-consuming and labor-intensive
Solution Approach 1:
The system performs self-correction of recognition errors through automated error detection and self-learning mechanisms. When recognition errors occur, the system automatically detects them, requests remote assistance only when necessary, and learns from the corrections to improve future recognition accuracy without requiring continuous manual intervention.
Solution Approach 2:
The system implements a feedback loop where recognition results are continuously evaluated, errors are detected and communicated to remote operators when found, and the learned information from corrections is fed back into the recognition model to improve future performance. This creates a closed-loop system that progressively improves accuracy while minimizing manual intervention.
2Reliability
If manual object selection and annotation is performed by human operators, then recognition errors can be corrected, but the process incurs additional cost and time
Solution Approach 1:
Instead of requiring manual annotation for all objects or continuous manual monitoring, the system applies partial manual intervention only when recognition errors are detected. The automated system handles the majority of recognition tasks independently, requesting remote assistance only for specific error cases, thereby reducing overall annotation workload while maintaining reliability.
Solution Approach 2:
The system performs self-correction by automatically detecting recognition errors and using remote operator input to improve its own recognition capabilities. The learned information from error corrections is automatically integrated into the recognition model, eliminating the need for continuous manual annotation and enabling the system to serve itself progressively.
3Ease of operation
If remote recovery is performed without providing object boundary information, then the process is simpler, but the recognition software capability is not improved
Solution Approach 1:
The system performs preliminary error detection and analysis before initiating remote recovery, automatically identifying specific recognition errors and preparing relevant information. This preliminary preparation ensures that when remote assistance is requested, the operator receives targeted information about the error, enabling efficient correction while maintaining system simplicity.
Solution Approach 2:
The system implements feedback by automatically detecting recognition errors, requesting remote assistance with specific error information, and using the corrections received to improve future recognition. This feedback loop enables the system to learn from errors and continuously improve recognition capability while maintaining operational simplicity through automated error management.
Data Source
AI summary
A method for recognizing and unloading a plurality of objects, which may include, until the plurality of objects has been unloaded, iteratively performing: collecting, by a processor, object data through a vision sensor; performing, by the processor, object recognition by determining object orientation and object location based on the object data; for an object of the plurality of objects being recognized and determined as available for picking, picking up and unloading the object using a robotic device; and for no object being determined as available for picking, performing: determining, by the processor, occurrence of object recognition error; for the object recognition error being detected, performing, by the processor, a recovery process to address the object recognition error; and for the object recognition error not being detected, recognizing, by the processor, completion in unloading of the plurality of objects.