3D Point Cloud Generation Through Segmented MVS Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D point cloud generation methods face computational inefficiencies and loss of fine details when scaling with increasing image resolutions, particularly in applications requiring real-time spatial understanding.
Innovation Solution
Segment digital images into segments and apply Multi-View Stereo (MVS) processing on a subset of pixels per segment, determining depth and normal vectors for each segment, and combining these to generate a 3D point cloud.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If MVS processing is applied on full-resolution digital images, then the quality and detail of the 3D point cloud is improved, but the computational time and processing load increase significantly
Solution Approach 1:
The patent divides digital images into multiple segments or patches, and applies MVS processing to each segment independently rather than processing the entire high-resolution image at once. This segmentation approach maintains the quality of 3D point cloud generation by preserving fine details in each segment while significantly reducing the computational burden and processing time through parallel processing of smaller segments.
2Productivity
If digital images are downscaled to accelerate MVS processing, then the computational burden is reduced, but fine details and sharp boundary edges are lost
Solution Approach 1:
Instead of downsampling the entire image which would lose details, the patent segments the high-resolution image into smaller patches. Each patch is processed independently at full resolution, preserving fine details and sharp edges. The segmentation enables faster processing by reducing the computational complexity of processing the entire image at once, while maintaining quality through full-resolution processing of individual segments.
3Manufacturing precision
If MVS processing is applied to all pixels in high-resolution images, then the completeness of the 3D point cloud is improved, but the scalability and real-time performance deteriorate
Solution Approach 1:
The patent processes high-resolution images by dividing them into segments and applying MVS processing to each segment independently. This approach maintains point cloud completeness by ensuring all segments are processed, while achieving real-time performance through parallel processing of multiple smaller segments rather than processing the entire image sequentially. The method improves scalability by reducing the memory and computational requirements for processing large high-resolution images.
Data Source
AI summary
A method for generating a 3D point cloud of a scene is performed by an image processing device. The method obtains digital images depicting the scene. Each digital image is composed of pixels. The method includes segmenting each of the digital images into digital image segments. The method includes determining a depth vector and a normal vector per each of the digital image segments by applying MVS processing to a subset of the pixels per each digital image segment. The method includes forming a map of depth vectors and normal vectors per each pixel in the digital images by, based on the determined depth and normal vectors per each of the digital image segments, estimating a 3D plane per digital image segment. The method includes generating the 3D point cloud of the scene as a combination of all the maps of depth vectors and normal vectors per each pixel in the digital images.