Object Pose Recognition With Adaptive Depth Estimation Fallback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing robot systems face issues with incorrect depth estimation when input images significantly differ from teacher images, affecting the accuracy of object position and posture recognition.
Innovation Solution
A method and system that utilize a depth estimator to generate estimated depths, an identifier to assess depth usability, and a depth sensor to measure depths, ensuring accurate recognition by using either estimated or measured depths based on their usability, with training processes to enhance performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If depth estimation is used for quick recognition, then recognition speed is improved, but depth accuracy deteriorates when images differ significantly from training data
Solution Approach 1:
The patent introduces an identifier as an intermediary component that evaluates the quality of estimated depths before they are used for recognition. The identifier receives the object image and estimated depths, determines whether the depths are usable, and guides the selection between estimated and measured depths, thus mediating between the speed advantage of estimation and the accuracy requirement of measurement
Solution Approach 2:
The system dynamically switches between two depth acquisition modes: using estimated depths when they are deemed usable (for speed) and using measured depths when estimation quality is insufficient (for accuracy). This dynamic adaptation allows the system to optimize recognition performance based on real-time conditions rather than relying on a fixed approach
2Measurement precision
If depth sensor is used for accurate depth measurement, then depth accuracy is improved, but recognition speed deteriorates due to additional measurement time
Solution Approach 1:
Instead of always using the more accurate but slower depth sensor, the system applies partial action by using the depth sensor only when necessary (when estimated depths are unusable). For cases where estimated depths are sufficient, the system uses only the faster estimation process, thus avoiding the excessive time cost of always using the depth sensor
3Device complexity
If only depth estimator is used, then system complexity is reduced, but depth reliability deteriorates when images differ from training data
Solution Approach 1:
The system performs a preliminary evaluation of depth quality using the identifier before committing to using estimated depths for recognition. This preliminary action allows the system to detect potential reliability issues early and switch to measured depths if necessary, preventing unreliable estimates from compromising recognition accuracy
Solution Approach 2:
The system dynamically adjusts its depth acquisition strategy based on the identifier's assessment. When estimated depths are reliable, the system uses them; when they become unreliable (as detected by the identifier), the system switches to using measured depths, thus maintaining reliability under varying conditions
Data Source
AI summary
A method according to an aspect of the present disclosure includes generating an object image by capturing an image of the object with a camera, determining estimated depths of the object by using a depth estimator that receives the object image as input and outputs the estimated depths in the object image, determining whether the estimated depths are useable by using an identifier that receives at least one of the object image and an object image feature quantity extracted from the object image as input, when the estimated depths are usable, recognizing the position and posture of the object by using the estimated depths and the object image, and when the estimated depths are unusable, determining measured depths of the object with a depth sensor and recognizing the position and posture of the object by using the measured depths and the object image.