3D Mesh Rendering with Multi-View Texture Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 3D polygon mesh rendering methods fail to accurately reproduce the appearance of objects in 2D images, particularly in terms of sharpness, lighting effects, and silhouette details, due to the averaging of colors from multiple capture positions, which results in a loss of realism and image quality.
Innovation Solution
The method involves selecting multi-view images based on render and capture camera positions to determine optimal texture images, using weights to minimize positional errors, and rendering 3D polygon meshes as 2D images with alpha channels to incorporate detailed alpha values and lighting effects, thereby enhancing image fidelity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple captured images from different camera positions are averaged to create texture, then comprehensive object coverage is achieved, but image sharpness and realism are lost
Solution Approach 1:
The patent applies local quality by selecting different texture images based on the local viewing direction. For each pixel or small region in the rendered image, the system chooses the most appropriate captured image from multiple views, rather than averaging all images globally. This ensures that each local area uses the highest quality texture available for that specific viewpoint, maintaining sharpness while achieving comprehensive coverage through multi-view selection.
2Adaptability or versatility
If multiple captured images are used to create texture, then complete object surface coverage is achieved, but lighting effect realism deteriorates
Solution Approach 1:
The patent implements dynamics by making the texture selection adaptive to the render camera position. As the camera moves around the object, the system dynamically selects different captured images as textures based on which view best matches the current viewing angle. This dynamic selection preserves realistic lighting effects because each view captures the object under its specific lighting conditions, rather than creating a static averaged texture that loses lighting information.
3Productivity
If traditional 3D polygon mesh rendering is used, then rendering speed is maintained, but image quality and realism are insufficient
Solution Approach 1:
The patent uses copying by taking actual captured images of the object and using them directly as textures on the 3D polygon mesh, rather than generating textures through complex rendering calculations. This approach copies the real-world appearance information from captured images onto the mesh surfaces, preserving photo-realistic quality including natural lighting and details, while maintaining fast rendering speeds since it avoids computationally intensive texture generation during rendering.
4Measurement precision
If high-resolution textures are used for all mesh surfaces, then image fidelity is improved, but data size increases
Solution Approach 1:
The patent applies partial action by using high-resolution captured images as textures only where needed on the mesh surfaces, rather than uniformly applying high-resolution textures to all surfaces. The system selects from multiple captured images and applies appropriate portions to different mesh faces based on viewing angle and surface orientation, achieving high image fidelity for visible surfaces while keeping overall data size manageable by not storing or processing excessive texture data for all possible views simultaneously.
Data Source
AI summary
An apparatus for rendering a three-dimensional (3D) polygon mesh includes a processor, and a memory storing instructions, wherein the processor is configured to execute the instructions to obtain a plurality of multi-view images corresponding to a plurality of captured images obtained by capturing an object at different capture camera positions, obtain a 3D polygon mesh for the object, select one or more of the plurality of multi-view images as one or more texture images based on a render camera position and capture camera positions of the plurality of multi-view images, and render the 3D polygon mesh as a two-dimensional (2D) image based on the selected one or more texture images.