Portrait Sketch Synthesis Using MRF Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing face sketch synthesis methods fail to accurately generate sketches from photos under varying lighting and pose conditions due to the independent synthesis of patches, which ignores spatial relationships and results in blurring effects.
Innovation Solution
A method that divides a photo into patches, determines matching information with training patches, and uses a Markov Random Field (MRF) network model to optimize the selection of matched training sketch patches, incorporating shape priors and compatibility information to synthesize a portrait sketch, ensuring robustness to lighting and pose variations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If patch-based reconstruction is used to synthesize sketches, then local details can be preserved, but spatial relationships between patches are ignored causing blurring effects
Solution Approach 1:
The patent combines local patch-based reconstruction with global spatial relationship constraints by integrating both approaches in a unified synthesis framework. The system merges the advantages of local detail preservation from patch-based methods with the spatial coherence from global structure analysis, eliminating the blurring effect while maintaining local fidelity.
Solution Approach 2:
The patent implements feedback mechanisms where the synthesized sketch is continuously evaluated against both local patch matching criteria and global spatial relationship constraints. The system uses iterative optimization with feedback loops that adjust patch selections to simultaneously satisfy local detail requirements and maintain proper spatial relationships between facial features.
2Device complexity
If global feature transform is applied to synthesize sketch, then processing is simplified, but hair regions with significant style variations cannot be synthesized accurately
Solution Approach 1:
The patent segments the face into multiple regions including hair, face, and background areas. Different synthesis strategies are applied to different segments: patch-based reconstruction is used for hair regions to capture style variations, while other regions may use more efficient methods. This segmentation allows the system to handle complex hair styles without overly complicating the overall processing.
Solution Approach 2:
The patent applies local quality by using different synthesis approaches for different facial regions based on their specific characteristics. Hair regions with high style variability receive more computationally intensive patch-based processing, while stable regions use simpler methods. This localized approach optimizes both accuracy and efficiency across different parts of the face.
3Productivity
If linear combinations of training sketch patches are used, then synthesis speed is improved, but blurring effects occur
Solution Approach 1:
The patent extracts and removes the blurring effect caused by linear combinations by using a different synthesis mechanism. Instead of averaging multiple patches, the system selects and stitches individual patches that best match the target photo, extracting only the sharp, relevant features while discarding the blurring artifacts inherent in linear combination methods.
4Manufacturing precision
If previous methods are used under well controlled conditions, then synthesis accuracy is maintained, but they fail when lighting and pose variations occur
Solution Approach 1:
The patent implements dynamic adaptation by making the synthesis process responsive to varying lighting and pose conditions. The system dynamically adjusts its patch selection and matching criteria based on the input photo's characteristics, allowing it to maintain accuracy across diverse conditions rather than relying on fixed, pre-defined transformations.
Solution Approach 2:
The patent changes key parameters such as lighting normalization factors, pose alignment transformations, and patch matching thresholds based on the input conditions. By dynamically adjusting these parameters according to the specific lighting and pose of each input photo, the system maintains synthesis accuracy across varying conditions rather than failing as previous methods did.
Data Source
AI summary
The present invention discloses a system and method for synthesizing a portrait sketch from a photo. The method includes: dividing the photo into a set of photo patches; determining first matching information between each of the photo patches and training photo patches pre-divided from a set of training photos; determining second matching information between each of the photo patches and training sketch patches pre-divided from a set of training sketches; determining a shape prior for the portrait sketch to be synthesized; determining a set of matched training sketch patches for each of the photo patches based on the first and the second matching information and the shape prior; and synthesizing the portrait sketch from the determined matched training sketch patches.