3D Mesh Texturing via Markov Random Field Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D reconstruction and texturing methods face challenges in achieving accurate texturing due to inaccuracies in camera pose estimation, sensor synchronization, and distorted images, leading to suboptimal representation of real objects in 3D models.
Innovation Solution
A computer-implemented method that optimizes pixel shifts for 3D mesh texturing using a discrete Markov Random Field optimization scheme, maximizing a probability function to improve texture mapping and correct inaccuracies in 3D model representation, allowing for high-quality texturing even with inaccurate 3D meshes and distorted images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If standard image projection methods are used for texturing 3D models, then the texturing process is simple and fast, but the texturing accuracy deteriorates due to camera pose estimation errors and distorted images
Solution Approach 1:
The patent transforms the texturing problem from direct image projection to an optimization problem by changing the parameters being optimized. Instead of directly mapping images to 3D models using estimated camera poses, the method optimizes pixel shifts for each vertex of the 3D mesh. This parameter transformation allows the system to compensate for camera pose errors and image distortions by adjusting pixel positions rather than relying on accurate initial estimates.
Solution Approach 2:
The patent creates an optimized copy of the texture coordinates by generating pixel shift values for each vertex. Rather than using the original projected texture coordinates directly, the method creates an improved version through optimization, where each vertex's texture coordinate is adjusted by a computed pixel shift. This copying approach preserves the original simple projection method while adding an optimization layer that corrects accuracy issues.
2Manufacturing precision
If pixel shifts are optimized for each vertex to improve texturing accuracy, then texturing quality improves, but computational time and complexity increase
Solution Approach 1:
The patent extracts the essential correction needed for accurate texturing by separating the pixel shift optimization from the full 3D reconstruction optimization. Instead of optimizing all camera pose parameters and 3D point positions, the method extracts only the pixel shift component as the optimization variable. This extraction focuses computational effort on the specific parameter (pixel shift) that directly affects texturing accuracy, rather than optimizing the entire reconstruction pipeline.
Solution Approach 2:
The patent applies local optimization by computing pixel shifts independently for each vertex of the 3D mesh rather than applying a global transformation. Each vertex's texture coordinate is adjusted locally based on its specific position and the corresponding image region. This local approach allows the system to achieve high texturing accuracy at each point without requiring global optimization of all parameters, reducing overall computational complexity.
3Manufacturing precision
If accurate camera pose estimation and sensor synchronization are used to improve texturing accuracy, then the representation quality improves, but the system complexity and measurement requirements increase
Solution Approach 1:
The patent converts the harmful effect of inaccurate camera pose estimation and image distortion into a benefit by using these inaccuracies as the motivation for pixel shift optimization. Rather than requiring perfect input data, the method accepts imperfect camera poses and distorted images, then compensates for these errors through optimization. The pixel shifts are specifically designed to correct the errors introduced by inaccurate sensor synchronization and camera pose estimation, turning the problem of data inaccuracy into an opportunity for improved texturing through optimization.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
A computer-implemented method for designing a 3D modeled object representing a real object comprises the steps of providing (S10) a 3D mesh representing the real object, a texturing image and a mapping between the vertices of the 3D mesh and pixels of the texturing image; then maximizing (S20) a probability P(L(V)) of the form: PLV=1Zexp-∑i=1nφiʹLvi -Σf∈Fψf′Lνii∈f. The step of maximizing is performed with a predetermined discrete Markov Random Field optimization scheme viewing the 3D mesh and the pixel shifts associated to the texture coordinates of the vertices of the 3D mesh as a Markov Random Field of energy -log(P(L(V))) - log(Z). The method then comprises texturing (S30) the 3D mesh according to the texturing image, to the mapping, and to the result of the step of maximizing. This provides an improved solution for designing a 3D modeled object representing a real object.