Deep Learning Multi-View Image Selection for 3D Face Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of selecting appropriate images from multiple perspectives for 3D face reconstruction is difficult, leading to increased resource consumption and poor reconstruction results due to noise and ambiguity from too many input images or closely continuous viewing angles.
Innovation Solution
An input optimization method using deep learning models to process multi-view images, including background removal, image grouping, and facial landmark detection to select optimal images for 3D face reconstruction, utilizing Part Grouping Network (PGN) and Dlib models, along with homography matrix calculations to minimize image noise and resource consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If too many face images from different perspectives are input into the model, then the completeness of 3D face reconstruction is improved, but resource consumption increases and reconstruction quality deteriorates due to noise and ambiguity
Solution Approach 1:
The patent extracts and removes background elements from the input images using deep learning models (Part Grouping Network), retaining only the facial regions. This extraction process reduces the amount of processing data while maintaining the essential facial information needed for accurate 3D reconstruction, thereby reducing resource consumption without sacrificing reconstruction quality
Solution Approach 2:
The patent segments the input images into different categories (frontal, left profile, right profile) using the Part Grouping Network and selects representative images from each segment. This segmentation approach allows the system to process a manageable subset of images that captures the essential 3D face information, reducing overall processing resources while maintaining reconstruction accuracy
2Manufacturing precision
If too many face images from different perspectives are input into the model, then the completeness of 3D face reconstruction is improved, but reconstruction quality deteriorates due to noise and ambiguity
Solution Approach 1:
The patent extracts and removes background elements from the input images using deep learning models (Part Grouping Network), retaining only the facial regions. This extraction process reduces the amount of processing data while maintaining the essential facial information needed for accurate 3D reconstruction, thereby reducing resource consumption without sacrificing reconstruction quality
Solution Approach 2:
The patent creates a simplified representation of the 3D face by selecting and processing a subset of optimized 2D images. The system copies essential facial features from multiple perspectives into a compressed 3D model, avoiding the need to process all input images while maintaining reconstruction accuracy and reducing noise/ambiguity
3Manufacturing precision
If manually selecting reliable images is performed, then image quality for reconstruction is improved, but the complexity of the selection process increases
Solution Approach 1:
The patent implements self-service by using deep learning models (Part Grouping Network) to automatically categorize and select optimal images without manual intervention. The system autonomously processes images, identifies reliable facial views, and prepares them for 3D reconstruction, eliminating the complex manual selection process while maintaining high image quality standards
Solution Approach 2:
The patent incorporates feedback mechanisms where the system evaluates image quality and selection effectiveness during the processing stages. The Part Grouping Network provides feedback on image suitability, and the system iteratively refines its selections based on reconstruction progress, automatically optimizing the image set without manual guidance
Data Source
AI summary
An input optimization method for multi-view images based on deep learning model for 3D face reconstruction is disclosed. The method first removes background from the modeling images containing a face, and then groups these modeling images without background into left face images, front face images and right face images, and marks several facial landmarks on them. From these left face images and right face images, the ones with smaller differences between the new facial landmark positions after homography transformation and facial landmark positions of the front face images are selected and used for 3D face reconstruction.