3D Object Pose Estimation for Transparent and Opaque Objects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in accurately acquiring three-dimensional (3D) pose information for objects, particularly transparent objects, and have limitations in measuring distance ranges due to narrow field of view differences between cameras and require fixed camera positions.
Innovation Solution
An electronic device equipped with multiple cameras, a depth sensor, and neural network models to identify object transparency and symmetry, enabling efficient 3D pose information acquisition through stereo matching and depth information, adaptable to changing camera positions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If pose information is acquired based on one image using a neural network model, then the acquisition process is simple and fast, but it is difficult to acquire accurate pose information for transparent objects and objects without established 3D models
Solution Approach 1:
The patent segments the pose estimation task into two distinct pathways: one for transparent objects using stereo matching, and another for opaque objects using depth sensors. This segmentation allows each pathway to use the most appropriate technique for its specific object type, thereby improving overall accuracy while maintaining efficiency.
Solution Approach 2:
The system dynamically selects between different pose estimation methods (stereo matching vs. depth sensor) based on object transparency detection. This dynamic adaptation allows the system to optimize its approach for each specific object type encountered, improving measurement precision without sacrificing productivity.
2Measurement precision
If a stereo camera is used to acquire pose information, then depth measurement capability is improved, but the range of measurable distances is limited due to narrow field of view difference between cameras
Solution Approach 1:
The patent makes the stereo camera system multi-functional by combining it with both depth sensors and transparency detection. This universal system can now handle both close-range stereo matching for transparent objects and far-range depth sensing for opaque objects, greatly expanding the measurable distance range while maintaining depth measurement precision.
Solution Approach 2:
The transparency detection mechanism acts as an intermediary that determines which measurement pathway to use. For transparent objects, it enables stereo matching; for opaque objects, it triggers depth sensor usage. This intermediary function resolves the field of view limitation by selecting the appropriate measurement method based on object properties.
3Adaptability or versatility
If the positional relationship between two cameras is changed to expand field of view, then the measurable distance range increases, but a trained neural network model may not be used as the premise of fixed positional relationship is violated
Solution Approach 1:
The patent replaces reliance on fixed-camera-position neural network models with a more flexible approach using depth sensors and transparency-based method selection. This substitution removes the mechanical constraint of fixed camera positions, allowing camera placement flexibility while maintaining measurement reliability through alternative techniques.
4Measurement precision
If depth information is acquired through a depth sensor for opaque objects, then accurate pose information is obtained, but the system complexity increases compared to single-image processing
Solution Approach 1:
The patent applies local quality by using depth sensors only where needed (for opaque objects) rather than universally. This selective application maintains high measurement precision for opaque objects while minimizing system complexity by avoiding unnecessary depth sensing for transparent objects where stereo matching suffices.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Disclosed are an electronic device and a control method of an electronic device. In particular, the control method of an electronic device according the disclosure comprises the steps of: acquiring a plurality of images through at least one camera; inputting RGB data for each of the plurality of images into a first neural network model to obtain two-dimensional pose information on an object included in the plurality of images; inputting RGB data for at least one image of the plurality of images into a second neural network model to identify whether the object is transparent; if the object is a transparent object, performing stereo matching based on the two-dimensional pose information on each of the plurality of images to obtain three-dimensional pose information on the object; and if the object is an opaque object, acquiring three-dimensional pose information on the object based on one image of the plurality of images and depth information corresponding to the one image.