Convolution Tensor Tiling With Zero-Padded Boundary Re-Tiling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Processing large-sized images in machine learning models, such as Convolutional Neural Networks, is computationally inefficient and often requires complex procedures like downsampling, patch-based approaches, or data parallelism, which can lead to loss of information or insufficient label usage.
Innovation Solution
Implementing tiling strategies in neural networks that divide input tensors into non-overlapping or overlapping tiles, with optional zero-padding, to optimize memory usage and processing efficiency, using reconfigurable data processors like GPUs, FPGAs, and CGAs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If tiling is used to process large images in neural networks, then memory usage is optimized and processing efficiency is improved, but information loss occurs at section boundaries
Solution Approach 1:
The patent applies segmentation by dividing the input image into multiple non-overlapping tiles or sections that can be processed independently in parallel. This segmentation enables efficient memory usage and processing while the patent specifically addresses boundary information loss through overlap mechanisms where adjacent tiles share boundary regions, ensuring that information at section boundaries is preserved across multiple processing units.
2Productivity
If downsampling is used to handle large images, then processing becomes more efficient, but information is lost
Solution Approach 1:
The patent applies partial action by processing only the necessary portions of the image at full resolution while using efficient tiling for the remainder. The overlap mechanism ensures that boundary regions are processed with sufficient detail, avoiding the need for complete downsampling while maintaining processing efficiency for large images.
3Productivity
If patch-based approaches are used, then large images can be processed, but label usage becomes insufficient
Solution Approach 1:
The patent applies merging by combining multiple adjacent tiles that share boundary overlaps into a unified processing framework. This allows labels from overlapping boundary regions to be utilized across multiple tiles, increasing the effective use of available labels while maintaining the ability to process large images through the tiling structure.
Data Source
AI summary
Disclosed is a method that includes generating by an output processing node of a first section of a processing graph, a plurality of output tiles of an output tensor. The plurality of output tiles of the output tensor is written in a memory, where the writing includes zero-padding the plurality of output tiles of the output tensor in the memory. The zero-padded plurality of output tiles of the output tensor are tiled, to generate a plurality of input tiles of an input tensor. The plurality of input tiles of the input tensor is processed in a second section of the processing graph.