Face Image Processing With 3D Reconstruction for Natural Edits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing portrait editing methods using 3D models for face orientation and angle adjustment in 2D images face accuracy limitations, leading to unnatural rendering results due to sharp boundaries and angularity issues.
Innovation Solution
Perform three-dimensional reconstruction on a face image to obtain an initial 3D model, apply transformation and expansion processing to create an expanded 3D model, determine a displacement image, and transform the original face image using this image to achieve a more natural result.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If 3D model reconstruction is used for face orientation and angle adjustment, then the transformation can be applied to the face image, but the rendering result becomes unnatural due to sharp boundaries and angularity
Solution Approach 1:
The patent segments the face image into multiple regions (face region and background region) and applies different processing strategies to each. The face region uses 3D model transformation for orientation adjustment, while the background region is processed separately to maintain natural transitions and avoid sharp boundaries at the interface between face and background.
Solution Approach 2:
The patent applies local quality by treating the face region and background region differently. The face region benefits from 3D transformation for accurate orientation, while the background region is processed with different parameters to ensure smooth transitions. This localized approach resolves the contradiction by optimizing each region's processing according to its specific requirements.
2Adaptability or versatility
If 3D model transformation is applied to transform face image, then the face orientation can be changed, but sharp boundaries and angularity appear at the interface between face and background
Solution Approach 1:
The patent divides the image into face region and background region, applying 3D transformation only to the face region while keeping the background region separate. This segmentation allows orientation transformation to be applied effectively to the face while avoiding the propagation of sharp boundaries to the background interface.
Solution Approach 2:
The patent introduces an intermediary processing step that generates a transformed background image based on the transformed face image. This intermediary step acts as a bridge between the 3D-transformed face region and the original background, smoothing the transition and eliminating sharp boundaries at the interface.
3Extent of automation
If 3D model is used for face editing, then the transformation can be applied, but the accuracy is limited by the 3D model precision
Solution Approach 1:
The patent creates a copy of the face image (the transformed face image) that preserves the 3D transformation effects. This copy is then used to generate the transformed background image, allowing the transformation accuracy to be maintained while enabling automated processing of the entire image including background regions.
Solution Approach 2:
The patent uses the transformed face image as feedback to generate the transformed background image. This feedback mechanism allows the system to automatically adjust the background transformation based on the face transformation results, improving the overall transformation accuracy without requiring manual intervention.
Data Source
AI summary
The disclosure provides a method, apparatus, device and storage medium for image processing. The method for image processing includes: performing three-dimensional reconstruction on an original face image to obtain an initial 3D model; transforming the initial 3D model according to predetermined transformation information to obtain a transformed 3D model; performing expansion processing on the transformed 3D model to obtain an expanded 3D model; determining a displacement image based on the expanded 3D model and the initial 3D model; and performing transformation processing on the original face image according to the displacement image to obtain a target face image.