Multi-View Mesh Texturing With Variational Patch Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for texturing 3D polygon meshes from source images suffer from artifacts due to assumptions about a single 'good' image source, leading to suboptimal visual quality and increased user time and resource expenditure in retouching, hindering the adoption of SfM photogrammetry in infrastructure modeling.
Innovation Solution
A variational technique that jointly searches for a texture map and patch correspondence across multiple source images, using an energy function with matching and regularization terms to optimize similarity and complexity, employing a hybrid ZNCC function to consider exposure and color, and an alternate minimization algorithm for efficient computation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single source image is selected as texture source based on simple selection criteria, then the process is simple and fast, but artifacts such as ghosting, misalignments, and moving objects appear in the textured mesh
Solution Approach 1:
The patent merges multiple source images into a single texture map by combining their pixel information. Instead of selecting one image, the system integrates information from multiple images at each pixel location, using a combination of selection weights and blending functions to produce a unified texture that avoids artifacts from single-image selection.
Solution Approach 2:
The patent changes the parameter of image selection from binary (one image or none) to continuous (multiple images with varying weights). By introducing selection weights and blending parameters, the system can smoothly transition between different source images and optimize the contribution of each image based on its quality and relevance.
2Device complexity
If simple averaging of source images is used to produce textures, then the process is computationally simple, but unacceptable artifacts still appear in the textured mesh
Solution Approach 1:
The patent applies local quality by treating different regions of the texture map differently. Instead of uniform averaging, the system applies selective blending where certain pixels from certain images are weighted more heavily than others, based on local image quality, occlusion status, and correspondence accuracy. This allows high-quality regions to dominate while suppressing artifact-prone regions.
Solution Approach 2:
The patent introduces an intermediary blending function that mediates between multiple source images and the final texture map. This blending function acts as a filter and selector, determining which pixels from which images contribute to the final result, thereby preventing direct artifacts from appearing in the output.
3Manufacturing precision
If combinatorial optimization is used to select texture sources, then texture quality improves, but the process becomes computationally intensive and time-consuming
Solution Approach 1:
The patent segments the texture mapping process into independent pixel-wise operations. Instead of optimizing the entire texture map simultaneously through complex combinatorial optimization, the system processes each pixel independently using pre-computed selection weights and blending functions, dramatically reducing computational complexity while maintaining quality.
Solution Approach 2:
The patent performs preliminary actions by pre-computing image selection weights, correspondence relationships, and blending parameters before the actual texturing process. This allows the main texturing operation to proceed much faster using simple pixel-wise blending rather than iterative optimization during the rendering phase.
4Productivity
If conventional texturing methods are used, then the process is efficient, but visual quality and realism are insufficient, requiring extra user time for retouching
Solution Approach 1:
The patent incorporates feedback mechanisms that continuously evaluate the quality of the generated texture and adjust the blending weights accordingly. By monitoring metrics such as image quality scores, occlusion detection, and correspondence accuracy, the system can adaptively refine the texture map to ensure high visual quality while maintaining processing efficiency.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
In various example embodiments, a variational technique is provided for texturing a polygon mesh based on source images of a scene. The technique combines information from multiple source images to generate a texture map with patches that are the most similar to corresponding patches of the source images. The texture map may be the direct result of an optimization that jointly searches for the texture map and patch correspondence, which is initially unknown. The optimization may use a variational formulation that scores every possible texture map using an energy function that includes a matching term that quantifies how similar patches of the texture map are to the corresponding patches of the source images, and a regularization term that quantifies how complex the patch correspondence is.