Enclosed Block Convolution for Image Processing Precision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing techniques for convolutions, such as those using running histograms, pyramidal data structures, and summed-linear tables, lack accuracy and precision, especially for operations like max filters, due to quantizing errors in finite precision floating-point systems.
Innovation Solution
The method involves pre-computing acceleration data for enclosed blocks of pixels and using these data structures to perform associative operations like addition, multiplication, max, min, binary AND, and binary OR, allowing for exact convolutions by combining pre-computed intermediate results stored in image pyramids, which can be processed in parallel on GPUs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If summed-linear tables are used to speed up convolutions, then computation speed is improved, but numerical precision deteriorates due to quantizing errors in finite precision floating point systems
Solution Approach 1:
The patent pre-computes integral images (summed-area tables) with higher precision than the final output requires, storing these intermediate results with full precision. This preliminary computation at high precision avoids quantizing errors, and the pre-computed data is reused during convolution operations to maintain both speed and accuracy.
Solution Approach 2:
The patent introduces an intermediary high-precision data structure (integral image with extended precision) that mediates between the input image and the final convolution result. This intermediary stores cumulative sums with sufficient precision to prevent error accumulation, allowing fast table-lookup convolutions without sacrificing numerical accuracy.
2Productivity
If MIP maps with down sampling are used to accelerate convolutions, then processing speed is improved, but accuracy deteriorates because results are not as accurate as convolutions on original image data
Solution Approach 1:
The patent pre-computes integral images at full resolution before any downsampling or convolution operations. These pre-computed integral images serve as the foundation for all subsequent operations, ensuring that the base data maintains original precision while enabling accelerated processing through efficient query operations.
Solution Approach 2:
The patent transforms the 2D convolution problem into a 1D range sum problem by computing integral images. This dimensional transformation allows the use of prefix sum techniques that can be efficiently implemented with simple additions and subtractions, achieving O(1) convolution complexity while maintaining full precision without actual downsampling.
3Measurement precision
If explicit convolution or binary trees are used for max filters, then accuracy is maintained, but computation speed deteriorates making these operations slow
Solution Approach 1:
The patent pre-computes integral images that store cumulative maximum values (or other associative operations) in addition to or instead of simple sums. These pre-computed structures enable O(1) max filter operations by directly querying the pre-computed data, eliminating the need for slow explicit convolution or dynamic binary tree construction during the filtering operation.
Data Source
AI summary
A system and method for performing convolutions on image data using pre-computed acceleration data structures is disclosed. The method may include calculating intermediate convolution values for each of a plurality of blocks of pixels by performing an associative operation on the pixel values in each block. Each intermediate value may be associated with the block and indexed dependent on index values of pixels in the block. An image pyramid may include intermediate convolution values for multiple levels of acceleration by calculating intermediate convolution values for multiple block sizes. A convolution result for a kernel of an image may be produced by performing the associative operation on intermediate convolution values for non-overlapping blocks enclosed within the kernel and on pixel values associated with pixels in the kernel but not in one of the non-overlapping blocks. The methods may be implemented by program instructions executing in parallel on CPU(s) or GPUs.