3D CAD Surface Point Selection for Robust Industrial Object Localization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer vision and machine learning techniques for 3D object localization in industrial environments struggle with highly variable backgrounds, lighting conditions, and reflective or poorly textured objects, failing to achieve millimeter-level precision and robustness.
Innovation Solution
A method for selecting 3D surface points from a CAD model involves generating a geometric model, sampling points from sharp edges, associating images with reference viewpoints, refining camera poses, and using a convolutional neural network to identify and select relevant surface points for accurate 3D localization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If local descriptors are extracted from training images to construct a bag of visual words for object recognition and pose estimation, then object recognition capability is improved, but the system becomes not robust when context changes significantly or objects are poorly textured and highly reflective
Solution Approach 1:
The patent transforms the 6D pose estimation problem into a 2D point detection problem by changing the parameters being optimized. Instead of directly estimating pose parameters, the system detects 2D projection points of 3D surface points, which are then used to calculate pose. This parameter transformation enables the system to handle highly variable contexts and reflective surfaces more robustly.
Solution Approach 2:
The patent introduces an intermediate representation layer between the image input and pose output. The intermediate 2D projection points serve as mediators that bridge the gap between visual features and pose parameters. This intermediate step decouples the complexity of direct pose estimation from image variations, improving robustness to context changes.
2Measurement precision
If 2D-3D matching is performed for accurate 6D pose estimation, then measurement precision is improved, but the system requires selection of relevant surface points which is complex and computationally intensive
Solution Approach 1:
The patent performs preliminary selection and projection of 3D surface points to 2D space before the actual detection process. By pre-defining the projection relationships between 3D surface points and their 2D projections, the system simplifies the subsequent detection task and reduces computational complexity during runtime while maintaining high precision.
Solution Approach 2:
The patent segments the complex pose estimation task into distinct components: 3D surface point selection, 2D projection, point detection, and pose calculation. This segmentation allows each component to be optimized independently, reducing overall system complexity while maintaining measurement precision.
3Ease of operation
If a completely mobile camera is used in industrial environments with highly variable backgrounds and lighting conditions, then operational flexibility is improved, but localization precision deteriorates due to reflective surfaces and poor textures
Solution Approach 1:
The patent replaces traditional mechanical/physical markers or instrumented sensors with a computational vision system. Instead of using physical beacons, markers, or instrumented scenes, the system uses image processing and deep learning to achieve precise localization, maintaining camera mobility while overcoming environmental challenges.
Solution Approach 2:
The patent changes the approach from direct pose estimation to 2D point detection, which is more robust to lighting and texture variations. This parameter change allows the system to maintain high precision with mobile cameras in challenging industrial environments by focusing on detecting stable 2D projection points rather than relying on texture or lighting conditions.
Data Source
Figure 1
Figure 2~3
Figure 4~5
AI summary
A method for automatically selecting 3D points from a CAD model, for 3D locating of an industrial object, comprising the following steps: generating a geometric model from the CAD model; previously designating reference viewpoints from a priori knowledge of the current viewpoints of the camera; acquiring, by means of a camera, a plurality of images taken from viewpoints of interest of said object; associating each acquired image with a reference viewpoint in order to form sub-sets of images (partitions) and identifying a reference image for each partition; and selecting n 3D surface points for each partition. Use, in particular for locating industrial 3D objects, in augmented reality equipment.