3D Model Visibility Filtering for Virtual Viewpoint Data Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating three-dimensional models for virtual viewpoint images retain unnecessary data points that cannot be seen from outside, leading to inefficient data storage and processing requirements.
Innovation Solution
An image processing apparatus that acquires three-dimensional shape data from multiple cameras, generates visibility data based on camera positions, and corrects point cloud data to retain only observable points, reducing data storage needs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the visual hull method is used to generate a three-dimensional model, then a complete three-dimensional model including all points is obtained, but data points that cannot be seen from outside are retained, increasing data storage requirements
Solution Approach 1:
The patent extracts and removes unnecessary data points from the three-dimensional model. Specifically, it identifies and deletes points that cannot be observed from any camera position, keeping only the visible surface points that are necessary for generating virtual viewpoint images. This extraction process reduces data storage requirements while preserving the essential visual information.
Solution Approach 2:
The patent discards unnecessary internal points that cannot be seen from outside, and recovers only the essential visible surface points. By selectively removing redundant data and retaining only the necessary observable points, the system optimizes the balance between model completeness and data efficiency.
2Manufacturing precision
If erosion and dilation processing is applied to generate solid models, then model refinement is achieved, but processing complexity and time increase
Solution Approach 1:
Instead of applying complex erosion and dilation processing, the patent extracts the essential visible surface points directly from the point cloud data by checking visibility from camera positions. This extraction approach achieves model accuracy by retaining only observable points without requiring iterative morphological operations.
Solution Approach 2:
The patent replaces the mechanical erosion and dilation processing with a visibility-based filtering approach. By substituting complex iterative morphological operations with a direct visibility check from camera positions, the system achieves similar model refinement results with significantly reduced processing complexity.
3Reliability
If all points from the visual hull are retained for virtual viewpoint image generation, then complete coverage is achieved, but storage and processing efficiency decrease
Solution Approach 1:
The patent extracts only the necessary visible surface points from the complete point cloud data by determining which points can be observed from camera positions. This extraction ensures complete coverage of all observable areas while removing redundant internal points, thereby improving storage and processing efficiency without sacrificing coverage completeness.
4Manufacturing precision
If maximum-flow/minimum-cut algorithm is applied to optimize the three-dimensional model, then energy minimization is achieved, but computational complexity increases
Solution Approach 1:
The patent replaces the complex maximum-flow/minimum-cut algorithm with a simpler visibility-based filtering method. By substituting iterative graph-theoretic optimization with direct geometric visibility checks from camera positions, the system achieves model optimization with significantly reduced computational complexity while maintaining effectiveness.
Data Source
AI summary
An image processing apparatus includes an acquisition unit configured to acquire a three-dimensional shape data of an object based on images captured by a plurality of cameras, a generation unit configured to generate information based on a relationship between the three-dimensional shape data acquired by the acquisition unit and positions of the plurality of cameras, and a correction unit configured to correct the three-dimensional shape data based on the information generated by the generation unit.