3D Image Processing Using Segmented Depth and Color Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for processing and rendering three-dimensional (3D) image data obtained from depth cameras and color cameras face challenges in efficiently storing and merging data, particularly in handling different resolutions and complex stitching operations, which affect the performance of 3D representation and rendering.
Innovation Solution
The method involves storing depth images as 3D data in a hierarchical format like octree and color images as 2D data, using a moving least squares (MLS) projection operator to generate a surface, and merging data through ray-surface intersection to achieve efficient resolution adjustment and rendering of 3D images from various viewpoints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of moving object
If depth image data and color image data are stored together in a unified data structure, then data integration is improved, but data processing complexity and rendering efficiency deteriorate
Solution Approach 1:
The patent divides 3D image data into two independent data structures: one for depth image data and another for color image data. This segmentation allows each data type to be processed and stored separately, reducing the complexity of unified data management while maintaining the ability to integrate them during rendering operations.
2Reliability
If mesh-based representation is used to connect adjacent pixels, then noise removal and hole filling are improved, but stitching complexity for multiple images increases
Solution Approach 1:
The patent extracts the stitching operation from the mesh generation process. Instead of performing complex stitching during mesh creation, the system processes each depth image independently to generate local meshes, then uses point cloud integration to combine them. This extraction simplifies individual processing steps while maintaining overall mesh quality.
3Productivity
If point-based representation is used to spatially merge 3D points, then 3D model generation efficiency is improved, but storage requirements for 3D and color information increase
Solution Approach 1:
The patent merges point cloud data from multiple depth images into a single integrated 3D structure. By combining spatial information from multiple sources into one point cloud, the system reduces redundant storage while maintaining efficient 3D model generation capabilities. The merged point cloud serves as a unified foundation for subsequent rendering operations.
Data Source
AI summary
Provided is point-based efficient three-dimensional (3D) information representation from a color image that is obtained from a general Charge-Coupled Device (CCD)/Complementary Metal Oxide Semiconductor (CMOS) camera, and a depth image that is obtained from a depth camera. A 3D image processing method includes storing a depth image associated with an object as first data of a 3D data format, and storing a color image associated with the object as color image data of a 2D image format, independent of the first data.