Convolution Tile Boundary Layout for Lossless Large-Image Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Processing large-sized images in machine learning models, such as Convolutional Neural Networks, is computationally expensive and often results in loss of information due to downsampling or requires complex patch-based approaches with insufficient labels, making efficient computation challenging.
Innovation Solution
Implementing tiling strategies in neural networks using reconfigurable data processors like GPUs and FPGAs, where tensors are divided into non-overlapping or overlapping tiles, with zero-padding and convolution operations applied to each tile, optimizing memory usage and computation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large-sized images are processed directly in machine learning models, then processing accuracy is maintained, but computational cost and memory requirements increase significantly
Solution Approach 1:
The patent divides large input images into multiple smaller tiles or patches that can be processed independently and in parallel. This segmentation reduces the computational burden on each processing unit while maintaining the ability to reconstruct the full image output, thereby lowering overall computational cost and energy consumption without sacrificing processing accuracy.
2Productivity
If downsampling is applied to reduce computational cost, then processing speed increases, but information loss occurs
Solution Approach 1:
Instead of downsampling the entire image, the patent segments the image into tiles and processes each tile at full resolution independently. This approach maintains processing speed by enabling parallel computation on smaller tiles while preserving all original image information, avoiding the information loss inherent in downsampling.
Solution Approach 2:
The patent transforms the problem from processing one large image sequentially to processing multiple smaller tiles in parallel across the spatial dimension. This dimensional transformation increases processing throughput and speed while maintaining full resolution and information integrity in each tile.
3Quantity of substance
If patch-based approaches are used to process large images, then memory requirements are reduced, but label insufficiency and complexity increase
Solution Approach 1:
The patent implements a systematic tiling approach where the input image is divided into non-overlapping or overlapping patches with defined boundary handling strategies. This segmentation reduces memory requirements by processing smaller units while maintaining a relatively simple overall framework compared to complex patch-based methods, using standardized convolution operations on each tile.
Data Source
AI summary
Disclosed is a data processing system that includes compile time logic to section a graph into a sequence of sections, configure a first section to generate a first set of output tiles in a first target tiling configuration in response to processing a first set of input tiles in a first input tiling configuration, and configure a second section to generate a second set of output tiles in a second target tiling configuration in response to processing the first set of output tiles in a second input tiling configuration. Runtime logic is configured to pad a first input into a first padded input, read the first set of input tiles from the first padded input in the first input tiling configuration, and process the first set of input tiles through the first section to generate the first set of output tiles in the first target tiling configuration.