Multi-Camera 3D Pose Estimation for Transparent Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in accurately acquiring three-dimensional (3D) pose information for objects, particularly transparent objects, and have limitations in measuring distance range and camera positional changes, leading to inefficiencies in pose information acquisition.
Innovation Solution
An electronic device equipped with multiple cameras and neural network models processes RGB data to determine object transparency and symmetry, using stereo matching for opaque objects and depth information for transparent objects, while adjusting camera positions to enhance pose information acquisition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If pose information is acquired based on one image using a neural network model, then the process is simple and fast, but it is difficult to acquire accurate pose information for transparent objects and objects without established 3D models
Solution Approach 1:
The system dynamically selects between single-image processing and stereo-matching processing based on object characteristics. For transparent objects or objects without established 3D models, the system automatically switches to stereo-matching mode using multiple images, while for standard opaque objects with established models, it uses the faster single-image neural network approach.
Solution Approach 2:
The system changes the processing parameters by switching between different algorithms based on object transparency and model availability. When transparency is detected or no 3D model exists, the system changes from single-image neural network processing to multi-image stereo-matching processing, adjusting the measurement parameters accordingly.
2Measurement precision
If a stereo camera is used to acquire pose information, then depth measurement is improved, but the field of view difference between two cameras is narrow and the measurable distance range is limited
Solution Approach 1:
The system dynamically adjusts the baseline distance between cameras based on the measurement requirements. For distant objects, it increases the separation distance to improve depth resolution, while for close objects, it reduces the separation to maintain stereo matching accuracy, thus adapting to different distance ranges.
Solution Approach 2:
The system uses multiple baseline distances by repositioning cameras at different locations. Instead of relying on a fixed stereo pair, it captures images from multiple camera positions and combines them, effectively adding a temporal dimension to the stereo measurement process and expanding the measurable distance range.
3Adaptability or versatility
If the positional relationship between two cameras is changed to expand measurement range, then the measurable distance range increases, but a trained neural network model cannot be used as the premise of fixed camera positions is violated
Solution Approach 1:
The system performs preliminary calibration by capturing images of known calibration objects at multiple camera positions before actual measurement. This pre-calibration process establishes the geometric relationships between different camera positions, allowing the system to use these pre-computed parameters for subsequent measurements without requiring real-time re-calibration.
Solution Approach 2:
The system creates a virtual calibrated model that replicates the calibration results for different camera positions. Instead of re-calibrating for each position, it uses the preliminary calibration data to generate virtual calibration models for various camera configurations, allowing flexible positioning while maintaining measurement accuracy.
4Measurement precision
If multiple images are processed through stereo matching to acquire 3D pose information, then accuracy for transparent objects is improved, but processing time and computational complexity increase
Solution Approach 1:
The system applies stereo-matching processing selectively only to regions containing transparent objects or objects without established 3D models, rather than processing the entire image. By identifying problem areas and applying the computationally intensive stereo-matching algorithm only where needed, it reduces overall processing time while maintaining accuracy for critical objects.
Solution Approach 2:
The system segments the processing task into two stages: first, rapid classification of objects by transparency and model availability using lightweight neural networks, then selective application of full stereo-matching processing only to objects requiring it. This segmentation allows the system to maintain high speed for most objects while ensuring accuracy for challenging cases.
Data Source
AI summary
An electronic device and a control method of an electronic device are provided. The method acquiring a plurality of images through at least one camera, inputting red green blue (RGB) data for each of the plurality of images into a first neural network model to obtain two-dimensional pose information on an object included in the plurality of images, inputting RGB data for at least one image of the plurality of images into a second neural network model to identify whether the object is transparent, if the object is a transparent object, performing stereo matching based on the two-dimensional pose information on each of the plurality of images to obtain three-dimensional pose information on the object, and if the object is an opaque object, acquiring three-dimensional pose information on the object based on one image of the plurality of images and depth information corresponding to the one image.