3D Image Viewpoint Transformation Using Residual Texture Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for generating images from three-dimensional data using texture maps and meshes face challenges in achieving high-quality viewpoint transformations with significant shifts, leading to quality degradation and increased data rates, especially in virtual reality applications where efficient data distribution and processing are crucial.
Innovation Solution
The proposed solution involves an apparatus and method that utilize residual data texture maps and meshes to generate images for new viewpoints by combining intermediate texture maps and residual data texture maps, allowing for efficient viewpoint transformations without requiring full texture maps for each viewpoint, thereby reducing data rates and maintaining high image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full texture maps are provided for each viewpoint, then image quality is maintained, but data rate increases significantly
Solution Approach 1:
The patent extracts and transmits only the residual difference data between reference texture maps and target viewpoint texture maps, rather than transmitting complete texture maps for each viewpoint. This residual data contains only the necessary corrections needed to achieve high-quality viewpoint transformation, significantly reducing data transmission requirements while maintaining image quality.
Solution Approach 2:
The patent performs preliminary viewpoint transformation using reference texture maps and mesh data before transmission. The residual data transmitted contains pre-computed corrections that can be efficiently applied at the receiver end, reducing the computational and data burden during real-time playback.
2Measurement precision
If multiple residual data texture maps are used for a single viewpoint, then accuracy of viewpoint transformation is improved, but processing complexity increases
Solution Approach 1:
The patent combines multiple residual data texture maps corresponding to different reference viewpoints into a single composite texture map for the target viewpoint. By merging these multiple corrections, the system achieves higher transformation accuracy while managing processing complexity through efficient combination operations.
Solution Approach 2:
The patent applies multiple residual corrections (excessive action) to ensure high accuracy in viewpoint transformation. By using more correction data than minimally necessary and combining them, the system achieves superior transformation quality while the processing complexity remains manageable through optimized combination algorithms.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An apparatus comprises a receiver (201) for receiving an image signal comprising a number of three dimensional images representing a scene from different viewpoints, each three dimensional image comprising a mesh and a texture map, the signal further comprising a plurality of residual data texture maps for a first viewpoint being different from the different viewpoints of the number of three dimensional images, a first residual data texture map of the plurality of residual texture maps providing residual data for a texture map for the first viewpoint relative to a texture map resulting from a viewpoint shift of a first reference texture map being a texture map of a first three dimensional image of the number of three dimensional images and a second residual data texture map of the plurality of residual texture maps providing residual data for a texture map for the first viewpoint relative to a texture map resulting from a viewpoint shift of a second reference texture map being a texture map of a second three dimensional image of the number of three dimensional images. A mesh processor (203) generates a first mesh for the first viewpoint and a second mesh for the first viewpoint from an additional reference mesh being a mesh of the second three dimensional image of the plurality of three dimensional images. A first texture processor (205) generates a first intermediate texture map in response to a viewpoint transformation of the first reference texture map from a viewpoint of the first three dimensional image to the first viewpoint and an second intermediate texture map in response to a viewpoint transformation of the second reference texture map from a viewpoint of the second three dimensional image to the first viewpoint. A second texture processor (207) generates a first texture map in response to a combination of the first intermediate texture map and the first residual data texture map, and a second texture map in response to a combination of the second intermediate texture map and the second additional residual data texture map. An image generator (209) generates an image for a second viewpoint from the first texture map, the first mesh, the second texture map, and the second mesh.