Tiling Control Circuit for Neural Network Tile Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks face performance issues due to insufficient memory resources, requiring tiling of input feature maps and complex software-controlled pipelines for efficient processing, which can lead to suboptimal performance unless specifically configured to maximize hardware parallel processing and throughput.
Innovation Solution
A circuit arrangement comprising a download dispatch circuit, an operation dispatch circuit, and an upload dispatch circuit, which initiate and manage the download, processing, and upload of input and output feature map tiles, respectively, using multiple source and destination buffers, and a tile index generator to determine tile boundaries, enabling efficient pipelined processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If software-controlled pipelines are used to manage tiling and processing, then flexibility and control over hardware architecture are improved, but system complexity and performance optimization difficulty increase
Solution Approach 1:
The patent introduces hardware control circuits as intermediary components between the software and the tiling/pipelining operations. These control circuits include a tiling control circuit that receives control signals from a processor and automatically manages the division of input feature maps into tiles, and a pipeline control circuit that coordinates the pipelined processing across multiple source and destination buffers. This intermediary hardware layer translates high-level software intent into low-level hardware control signals, maintaining flexibility while reducing software complexity.
Solution Approach 2:
The hardware control circuits are designed to autonomously manage the tiling and pipelining operations without requiring detailed software intervention. The tiling control circuit automatically divides input feature maps into tiles based on received control parameters, and the pipeline control circuit self-coordinates buffer management and data flow across processing stages. This self-service capability allows the hardware to optimize its own operation while responding to software control signals, reducing the burden on software developers.
2Quantity of substance
If tiling is implemented to handle insufficient memory resources, then memory capacity requirements are reduced, but processing overhead and system complexity increase
Solution Approach 1:
The patent implements tiling by segmenting the input feature map into multiple smaller tiles that can be processed individually. The tiling control circuit receives control signals specifying tile dimensions and automatically divides the input feature map accordingly. This segmentation allows processing with limited memory resources by loading only one tile at a time into the processing pipeline, significantly reducing memory capacity requirements while the hardware automation keeps control complexity manageable.
3Productivity
If pipelined hardware is used to improve throughput, then processing speed and throughput increase, but control complexity and coordination overhead increase
Solution Approach 1:
The patent introduces a pipeline control circuit as an intermediary that manages the complex coordination required for pipelined processing. This control circuit receives control signals from the processor and automatically coordinates data flow between multiple source buffers and destination buffers across different processing stages. It handles buffer allocation, data routing, and synchronization without requiring software to manage each individual pipeline stage, thus enabling high throughput while keeping control complexity hardware-managed rather than software-managed.
4Productivity
If multiple source and destination buffers are used to enable parallel processing, then processing parallelism and throughput are improved, but memory resource requirements and system complexity increase
Solution Approach 1:
The patent implements multiple source and destination buffers with specialized roles in the processing pipeline. Each buffer is optimized for its specific function: source buffers hold input tiles ready for processing, while destination buffers store processed output tiles. The pipeline control circuit intelligently allocates and manages these buffers based on the current processing stage and data flow requirements. This localized optimization allows parallel processing to proceed efficiently while minimizing the total buffer memory resources required compared to a uniform buffer allocation approach.
Data Source
AI summary
A download dispatch circuit initiates download of an input tile of an input feature map in response to a source buffer of two or more source buffers being available for the input tile, and indicates that the input tile is available in response to completion of the download. An operation dispatch circuit initiates a neural network operation on the input tile in response to the input tile being available and a first destination buffer of two or more destination buffers being available for an output tile of an output feature map, and indicates that the output tile is available in response to completion of the neural network operation. An upload dispatch circuit initiates upload of the output tile to the output feature map in response to the output tile being available, and indicates that the first destination buffer is available in response to completion of the upload.