Dual Tiling for Real-Time Image Compositing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image compositing systems face challenges in achieving real-time performance due to hardware limitations, particularly with large image files, which require significant time and bandwidth, leading to the use of lower quality proxies for editing and previewing, resulting in suboptimal artistic and business decisions.
Innovation Solution
The system employs dual tiling methods for processing and storage tiles, optimized for memory transfers and disk I/O respectively, with a persistent data structure mapping between them, using two coordinate systems to ensure efficient retrieval and storage of image tiles, allowing for real-time compositing of images of arbitrary sizes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If disk-based tiling with fixed-size tiles is used for storage, then disk I/O efficiency is improved, but real-time compositing performance deteriorates due to excessive disk I/O operations
Solution Approach 1:
The patent segments image data into two distinct tile types: storage tiles optimized for disk I/O and processing tiles optimized for memory operations and compositing. This segmentation allows each tile type to be independently optimized for its specific function, resolving the contradiction between disk I/O efficiency and real-time processing performance.
Solution Approach 2:
The patent introduces an intermediary mapping structure that translates between storage tile coordinates and processing tile coordinates. This intermediary layer enables the system to leverage both disk-optimized storage tiles and memory-optimized processing tiles simultaneously, bridging the gap between storage efficiency and processing speed.
2Quantity of substance
If larger storage tiles are used to minimize disk I/O, then disk access efficiency is improved, but processing time increases due to larger tile sizes
Solution Approach 1:
The patent divides the image into storage tiles for disk I/O and processing tiles for memory operations. Storage tiles can be larger for optimal disk access, while processing tiles remain smaller for faster memory processing, eliminating the time penalty associated with processing large tiles.
Solution Approach 2:
Different tile types have different optimization characteristics tailored to their specific functions. Storage tiles are locally optimized for disk I/O patterns, while processing tiles are locally optimized for memory bandwidth and processing speed, allowing each to operate at its optimal performance point.
3Productivity
If proprietary file systems with controlled allocation are used, then processing efficiency is improved, but system versatility deteriorates due to inability to use open file systems
Solution Approach 1:
The patent introduces a virtual tile mapping layer that acts as an intermediary between the file system and the processing system. This virtual layer manages the mapping between disk blocks and processing tiles, allowing the system to use standard open file systems while maintaining optimized processing characteristics through the virtual mapping abstraction.
4Speed
If lower resolution proxies are used for previewing, then processing speed is improved, but image quality and decision accuracy deteriorate
Solution Approach 1:
The patent segments the image into processing tiles that can be handled at full resolution. By dividing the large image into manageable processing tiles, the system can process and display portions of the image at full quality without the performance penalties associated with handling the entire large image at once, eliminating the need for proxy rendering.
Data Source
AI summary
A computer-implemented method, apparatus, and article of manufacture provide the ability to store image data for use in a real-time compositing computer application. A storage tile size is determined based on disk input/output (I/O) hardware testing. A processing tile size is determined based on graphics I/O testing. An image is obtained and processing tiles (of the processing tile size) are mapped over the image. A center of the image is used as a point of origin for the processing tiles. The processing tiles are mapped to storage tiles. The storage tile point of origin is located at a lower left corner of the processing tiles. Each storage tile is configured to contain complete processing tiles that are stored in the storage tiles based on the storage tile size, processing tile size, and the mappings.