Texture Synthesis Match Table Indexing for Real-Time Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing texture synthesis methods are inefficient for real-time rendering on low-cost and mobile devices due to high storage requirements and processing resources, often resulting in unsatisfactory visual artefacts and repeating patterns.
Innovation Solution
The method involves creating a match table and index table from sample blocks of an input image to generate a larger synthesised image, using overlapping regions and cuts to reduce blocky artefacts, allowing for real-time texture synthesis without repeating patterns.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If existing texture synthesis algorithms are used to generate large textures, then visual quality is improved, but storage requirements and processing resources become too large for real-time rendering on low-cost devices
Solution Approach 1:
The patent divides the texture synthesis problem into two distinct phases: an analysis phase that pre-computes a compact representation (jump map and cost map) from the input texture, and a synthesis phase that rapidly generates output textures using this pre-computed data. This segmentation allows high-quality texture synthesis while keeping real-time storage requirements minimal, as only the compact representation needs to be stored rather than multiple pre-generated texture variants.
Solution Approach 2:
The analysis phase performs preliminary computation to create the jump map and cost map from the input texture before actual rendering occurs. This pre-processing step captures the essential structure and patterns of the input texture in a compact form, enabling fast real-time synthesis without needing to re-analyze the full input texture during rendering. The preliminary action reduces both storage requirements and real-time processing burden.
2Manufacturing precision
If existing texture synthesis algorithms are used to generate large textures, then visual quality is improved, but processing time becomes too long for real-time rendering
Solution Approach 1:
By separating texture synthesis into analysis and synthesis phases, the patent enables heavy computation to occur offline during the analysis phase, while the real-time synthesis phase only requires simple lookups and copying operations based on the pre-computed jump map. This segmentation allows high visual quality to be achieved through thorough analysis without sacrificing real-time performance during actual rendering.
Solution Approach 2:
The analysis phase performs all complex pattern recognition and jump map computation in advance, before rendering is needed. This preliminary action transfers the computational burden from real-time synthesis to offline preparation, enabling the synthesis phase to operate quickly by simply following the pre-determined jump map instructions to copy pixel values from the input texture.
3Productivity
If Wang Tiles are used for real-time texture synthesis, then processing speed is improved, but visual artefacts and repeating patterns appear in the synthesised image
Solution Approach 1:
The patent uses a copying-based approach where the synthesis phase rapidly reproduces texture patterns by copying pixel values from the input texture according to the jump map instructions. This copying mechanism maintains visual fidelity to the original texture while achieving real-time performance, avoiding the tiling artefacts that occur with Wang Tile methods that stitch together discrete patches.
Solution Approach 2:
The patent replaces the mechanical patch-stitching mechanism of Wang Tiles with a data-driven copying mechanism guided by the jump map. Instead of physically assembling texture patches like Wang Tiles, the system uses pre-computed directional instructions to copy pixel values, eliminating the diamond-shaped artefacts and repeating patterns inherent in tile-based approaches while maintaining real-time performance.
4Quantity of substance
If a small sample image is used for texture synthesis, then storage requirements are reduced, but it becomes difficult to create enough variations to avoid repeating patterns
Solution Approach 1:
The analysis phase performs preliminary examination of the small sample image to extract all possible pattern variations and encode them in the jump map and cost map. This preliminary action identifies and stores directional information and pixel value relationships that can be reused to generate unlimited texture variations without repeating patterns, allowing small input images to produce large, non-repetitive output textures.
Solution Approach 2:
The patent transforms the small sample image into a different parameter representation (the jump map with directional instructions and cost values) that encodes the essential patterns and variations. This parameter transformation allows the system to generate diverse texture variations by following different paths through the jump map, effectively expanding the visual content available from a small input image without increasing storage requirements.
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
Image synthesis methods and units are described which can be implemented in real-time on low-cost and mobile devices. A set of sample blocks from an input image are defined. For each combination of sample blocks, a matching block is determined, which provides a best match for that combination when positioned in conjunction with a predetermined arrangement of the sample blocks of the combination. A match table can then be created and stored which includes an indication of the determined matching block for each combination of sample blocks from the input image. The match table can be used to generate a synthesised image, which may be much larger than the input image. For example, the match table may be used to determine an index table which comprises indications of sample blocks to be positioned in a synthesis arrangement for generating the synthesised image.