Speckle Image Block Matching Using Convolution Techniques
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional 3D reconstruction methods using speckle structured light are limited by speed and accuracy, making them unsuitable for real-time applications due to reliance on general-purpose processors and inefficient block matching algorithms.
Innovation Solution
Implementing convolution techniques using a processor with convolutional neural network hardware to preprocess reference images, perform convolution operations, and generate disparity and depth maps, which accelerates speckle image block matching by reducing calculation complexity and improving data parallelization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If conventional block matching algorithms are executed on general purpose processors, then the system is simple to implement, but the reconstruction speed is not fast enough
Solution Approach 1:
The patent replaces general purpose processors with specialized convolutional neural network hardware to execute block matching algorithms. This substitution of computing architecture dramatically increases reconstruction speed while accepting increased device complexity, as the specialized hardware is designed specifically for parallel convolution operations required in speckle image processing
Solution Approach 2:
The patent segments the reference images into multiple patches and processes them through parallel convolution operations. By dividing the image data into manageable segments that can be processed simultaneously, the system achieves faster reconstruction speeds while maintaining manageable computational complexity through organized parallel processing
2Measurement precision
If conventional block matching algorithms are used, then the algorithm is simple to implement, but the reconstruction accuracy is not accurate enough
Solution Approach 1:
The patent performs preliminary preprocessing of reference images by dividing them into patches and pre-calculating convolution kernels before the actual block matching process. This preliminary preparation enables more accurate reconstruction by ensuring that all necessary computational elements are ready and optimized before processing the speckle images, thereby improving measurement precision
Solution Approach 2:
The patent transforms the traditional 2D block matching approach into a 4D tensor operation by incorporating depth information and multiple reference patches. This dimensional expansion allows the convolutional neural network to process spatial and depth information simultaneously, improving reconstruction accuracy while managing algorithm complexity through structured tensor operations
3Speed
If 192 reference offset images are matched using original image, then comprehensive matching is achieved, but the matching speed becomes unsatisfactory
Solution Approach 1:
The patent applies partial action by processing only relevant patches of reference images through convolution operations rather than performing exhaustive matching on all 192 reference offset images. This selective processing maintains comprehensive matching coverage where needed while significantly improving matching speed by avoiding redundant computations in regions where full matching is not necessary
Solution Approach 2:
The patent performs preliminary organization of the 192 reference offset images into structured tensor formats and pre-computes convolution kernels before the actual matching process. This preliminary preparation enables efficient parallel processing during matching, maintaining comprehensive coverage of all reference images while achieving satisfactory matching speed through optimized computational workflows
Data Source
AI summary
An apparatus comprising an interface, a light projector and a processor. The interface may be configured to receive pixel data. The light projector may be configured to generate a structured light pattern. The processor may be configured to process the pixel data arranged as video frames and generate disparity and depth maps. The processor may comprise convolutional neural network hardware that may arrange reference images into a tensor, perform logical operations on one of the video frames in a depth direction of the tensor to generate a tensor of feature maps of the video frames, use a convolution to reduce an amount of calculations performed in the depth direction of the tensor of feature maps, perform convolution filtering on the tensor of the feature maps, determine an index map location, and search lookup data based on the index map location to determine the disparity and depth maps.