Foreground 3D Model Generation from Depth and Silhouette Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods lack the capability to generate a 3D model of only the foreground from depth images and foreground images captured by multiple cameras.
Innovation Solution
An image processing apparatus and method that utilize depth images and foreground images from multiple viewpoints to generate a 3D model of the foreground, incorporating techniques such as visual hull generation, correction, and polygon mesh creation to isolate and reconstruct the foreground.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional methods are used to generate 3D models from multiple depth images, then complete 3D models can be generated, but the ability to generate only foreground 3D models is lost
Solution Approach 1:
The patent extracts and separates foreground information from the complete 3D model generation process. By using depth-related images that contain both depth data and foreground silhouette information, the system extracts only the foreground portion for 3D model generation, eliminating the need to process entire scenes and enabling foreground-only reconstruction.
Solution Approach 2:
The patent segments the 3D model generation process into distinct stages: first generating a complete 3D model from multiple depth-related images, then separating the foreground from the background. This segmentation allows the system to handle complex scenes by processing them in manageable parts, ultimately producing only the foreground 3D model.
2Ease of manufacture
If depth images and foreground images from multiple viewpoints are processed separately, then processing simplicity is maintained, but generation accuracy of foreground 3D model decreases
Solution Approach 1:
The patent merges depth information and foreground silhouette information into a single depth-related image structure. This combination allows the system to process multiple viewpoints simultaneously while maintaining both depth accuracy and foreground identification, achieving high reconstruction accuracy without complicating the processing architecture.
Solution Approach 2:
The depth-related image serves multiple functions: it provides depth data for 3D reconstruction, contains foreground silhouette information for segmentation, and maintains compatibility with existing multi-view processing pipelines. This multi-functionality achieves accurate foreground 3D model generation without requiring separate specialized processing stages.
3Loss of information
If complete 3D models including background are generated, then comprehensive information is obtained, but data transmission bandwidth requirements increase
Solution Approach 1:
The patent extracts only the essential foreground information from complete 3D models by using depth-related images that inherently contain foreground silhouette data. This extraction eliminates the need to transmit and process entire scene models, significantly reducing bandwidth requirements while preserving all necessary foreground 3D model information.
Solution Approach 2:
Instead of generating and transmitting complete 3D models of entire scenes (excessive action), the system generates only the necessary foreground portions (partial action). This partial generation approach reduces data transmission bandwidth while maintaining information completeness for the specific foreground objects of interest.
Data Source
AI summary
The present disclosure relates to an image processing apparatus and an image processing method that enable generation of only a 3D model of a foreground. A reconstruction unit generates a 3D model of the foreground, on the basis of depth images of a plurality of viewpoints and foreground images of the plurality of viewpoints. The present disclosure can be applied to a decoding apparatus or the like that decodes an encoded stream of depth-related images and color images of a 3D model of a plurality of viewpoints, and generates a 3D model of the foreground on the basis of the resultant depth-related images and color images, and virtual viewpoint information including internal parameters and external parameters for virtual cameras of the respective viewpoints, for example.