Lossless Tiling in Convolution Networks for Weight Gradient Calculation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Processing large-sized images in neural networks is computationally expensive and often requires downsampling, which leads to loss of information and inaccurate analysis, while existing methods like data parallelism across spatial dimensions are complex and time-consuming.
Innovation Solution
The implementation of tiling techniques in neural networks, where input tensors are divided into non-overlapping or overlapping tiles for convolution operations, allowing for efficient processing and memory management through zero-padding and re-tiling, facilitating faster execution and reduced memory requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If downsampling is used to process large images, then computational cost is reduced, but information loss occurs and analysis accuracy decreases
Solution Approach 1:
The patent divides large input tensors into smaller non-overlapping tiles, allowing the neural network to process manageable chunks of data without downsampling. Each tile is processed independently through the convolutional layers, preserving all original image information while reducing memory requirements and computational complexity at each layer.
Solution Approach 2:
The patent introduces a spatial tiling dimension to the processing architecture. Instead of reducing image resolution (downsampling), the system processes the full-resolution image by dividing it into multiple spatial tiles that are handled in parallel or sequential passes, effectively adding a processing dimension without losing image detail.
2Productivity
If data parallelism across spatial dimensions is used, then processing capability is improved, but system complexity and execution time increase
Solution Approach 1:
The patent segments the input tensor into non-overlapping tiles that can be processed in parallel across multiple computing units. This segmentation enables data parallelism while maintaining a relatively simple system architecture, as each tile follows the same processing pipeline without requiring complex coordination or communication between processing units.
Solution Approach 2:
The patent processes data in partial chunks (tiles) rather than requiring the entire dataset to be loaded and processed at once. This allows progressive processing where results can be accumulated incrementally, improving productivity without requiring the system to handle the full complexity of the entire large image simultaneously.
3Loss of information
If tiling with overlapping is used, then information loss is prevented, but computation time increases
Solution Approach 1:
The patent uses non-overlapping tile segmentation that strategically divides the input tensor into distinct regions. By carefully selecting tile boundaries and processing order, the method ensures complete coverage of all input data without redundancy, maintaining information completeness while avoiding the computational overhead of processing overlapping regions multiple times.
Data Source
AI summary
Disclosed is a data processing system that includes compile time logic configured to process a processing graph to generate a modified processing graph, which includes a plurality of forward processing nodes of a forward pass and a plurality of backward processing nodes of a backward pass. The data processing system also includes runtime logic configured with the compile time logic to execute the modified processing graph to generate, at a backward processing node of the plurality of backward processing nodes, a plurality of partial weight gradients, based on processing a corresponding plurality of gradient tiles of a gradient tensor, and generate, based on the plurality of partial weight gradients, a final weight gradient corresponding to the gradient tensor.