Structured Light Super-Resolution via Temporal Frame Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High resolution cameras in structured light imaging devices have low frame rates, limiting the overall frame rate of the system and hindering the generation of high resolution depth images.
Innovation Solution
Implementing multiple-frame super-resolution algorithms using low resolution cameras with high frame rates to combine multiple images, aligning and interpolating them to generate a high resolution depth image, thereby overcoming the limitations of high resolution cameras with low frame rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a high resolution camera is used in structured light imaging, then the precision of depth estimation is improved, but the frame rate decreases
Solution Approach 1:
The patent segments the imaging process into multiple temporal frames, capturing multiple lower-resolution images at high frame rates and then combining them through super-resolution algorithms to achieve the equivalent of a single high-resolution image, thereby resolving the contradiction between resolution and frame rate
Solution Approach 2:
The patent transitions from spatial resolution to temporal resolution by capturing multiple frames over time, then synthesizes the temporal information to achieve high spatial resolution in the final depth image, effectively moving the resolution gain from the spatial domain to the temporal domain
2Measurement precision
If multiple low resolution images are combined using super-resolution, then the resolution of the depth image is improved, but the complexity of image processing increases
Solution Approach 1:
The patent performs preliminary feature extraction and matching on individual low-resolution frames before combining them, pre-identifying key correspondences that guide the subsequent super-resolution synthesis, thereby reducing the overall processing complexity compared to naive full-image processing
Solution Approach 2:
The patent uses feature copying and matching between frames, where detected features in one frame are correlated with corresponding features in other frames, allowing the system to leverage repeated pattern recognition rather than processing entire images multiple times
Data Source
AI summary
A method of image processing in a structured light imaging device is provided that includes capturing a plurality of images of a scene into which a structured light pattern is projected by a projector in the structured light imaging device, extracting features in each of the captured images, finding feature matches between a reference image of the plurality of captured images and each of the other images in the plurality of captured images, rectifying each of the other images to align with the reference image, wherein each image of the other images is rectified based on feature matches between the image and the reference image, combining the rectified other images and the reference image using interpolation to generate a high resolution image, and generating a depth image using the high resolution image.