Speckle Projection 3D Measurement With Deep Stereo Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Speckle projection profilometry (SPP) suffers from low matching accuracy in traditional stereo matching algorithms, limiting its effectiveness in achieving precise 3D shape measurement.
Innovation Solution
A 3D measurement method based on end-to-end deep learning, involving stereo rectification of speckle images, feature extraction using shared weights, salient object detection, and 4D matching cost volume generation with 3D convolution layers to achieve robust disparity mapping.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional stereo matching algorithm is used for speckle projection profilometry, then the system is simple and easy to implement, but the matching accuracy is low
Solution Approach 1:
The patent replaces traditional stereo matching algorithms with a deep learning-based matching network. The matching network uses convolutional neural networks to automatically learn and extract features from speckle patterns, substituting the manual feature matching process with an intelligent system that achieves higher accuracy without requiring manual feature selection or complex parameter tuning.
Solution Approach 2:
The patent transforms the matching problem from traditional parameter-based algorithms to a learning-based approach where the system automatically adapts to different speckle patterns. The matching network learns optimal matching parameters through training data, allowing the system to handle various lighting conditions, speckle densities, and surface properties without manual parameter adjustment.
2Measurement precision
If multiple patterns are projected for 3D measurement, then measurement accuracy improves, but measurement time increases
Solution Approach 1:
The patent performs preliminary actions by pre-training the matching network with large datasets of speckle patterns. This preliminary training enables the network to quickly and accurately match single-shot speckle patterns without requiring multiple projections. The system prepares the intelligent matching capability in advance, allowing fast single-shot measurement during actual operation.
Solution Approach 2:
The patent achieves continuous useful action by using a single speckle pattern projection that captures all necessary information for 3D measurement. The deep learning matching network continuously processes the captured images in real-time, eliminating the need for sequential multiple pattern projections and enabling continuous measurement of dynamic objects.
Data Source
AI summary
The invention discloses a three-dimensional (3D) measurement method based on end-to-end deep learning for speckle projection. First, the speckle pattern was projected by the projector and collected simultaneously by the stereo camera. The speckle images after stereo rectification are fed into the stereo matching network. A feature extraction sub-network based on shared weights processes the speckle images to obtain a series of low-resolution 3D feature tensors. The feature tensor is fed into the saliency object detection sub-network to detect foreground information in the speckle images, producing a full-resolution valid mask map. A 4D matching cost volume is generated using the feature tensor of both views based on the candidate disparity range, filtered by a series of 3D convolutional layers to achieve cost aggregation, so that the initial disparity map is obtained by disparity regression. The final disparity map is obtained by combining the mask map and the initial disparity map to achieve a single-frame, robust, and absolute 3D shape measurement. The invention achieves a single-frame, robust, and absolute 3D shape measurement by projecting a single speckle pattern.