Real-Time Urban Scene Reconstruction via Object Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current three-dimensional scene reconstruction methods face challenges in achieving both accuracy and efficiency, particularly in reconstructing urban scenes, as they often result in low-efficiency reconstruction processes and low-accuracy sparse point clouds when using two-dimensional images.
Innovation Solution
A method involving the processing of target image frames and adjacent frames to obtain object point clouds, images, and coordinate transformation matrices, followed by determining global characteristics and surface parameters to reconstruct a three-dimensional scene model using a plane combination matrix and coordinate transformation matrix, enabling real-time urban scene reconstruction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If three-dimensional point cloud is reconstructed based on two-dimensional image with complex post-processing, then three-dimensional model can be obtained, but reconstruction efficiency is low
Solution Approach 1:
The patent segments the reconstruction process by separating object detection (2D image processing) from point cloud generation and scene reconstruction. This allows parallel processing of multiple objects simultaneously, improving efficiency while maintaining accuracy through dedicated processing pipelines for each object.
Solution Approach 2:
The patent performs preliminary object detection and segmentation on 2D images before point cloud reconstruction. By pre-identifying object boundaries and characteristics in the 2D domain, the system prepares structured data that accelerates the subsequent 3D reconstruction process, reducing overall processing time.
2Productivity
If three-dimensional model is reconstructed directly from two-dimensional image, then reconstruction efficiency is improved, but only sparse point cloud can be obtained resulting in low accuracy
Solution Approach 1:
The patent introduces an intermediary process that uses 2D image segmentation masks to guide point cloud extraction from depth data. This intermediary step bridges direct 2D-to-3D reconstruction with traditional point cloud methods, enabling efficient processing while achieving dense, accurate point clouds by constraining the search space with segmentation information.
3Manufacturing precision
If complex post-processing is applied to reconstructed three-dimensional point cloud, then three-dimensional model accuracy is improved, but reconstruction process complexity increases
Solution Approach 1:
The patent performs preliminary organization of point cloud data during the detection and segmentation phase. By structuring point cloud data according to detected object boundaries and characteristics before reconstruction, the system eliminates the need for complex post-processing operations, reducing overall process complexity while maintaining accuracy.
Data Source
AI summary
A method, a device, a computer device and a storage medium for a real-time urban scene reconstruction are provided. The method comprises: obtaining a target image frame and an adjacent image frame corresponding to a target urban scene; locating a position of an object in the target image frame according to the target image frame and the adjacent image frame and obtaining an object point cloud, an object image and a coordinate transformation matrix corresponding to a target object; determining a global characteristic of the target object and parameters of surfaces to be selected of the target object which is configured to determine a characteristic of the surface to be selected; determining a plane combination matrix of the target object; reconstructing a three-dimensional scene model of the target urban scene according to the plane combination matrix, the parameters of the surfaces to be selected and the coordinate transformation matrix.