CNN Controller Circuit Address Generation for Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face performance bottlenecks due to increased latency and overhead processing, particularly as batch size increases, and the need to handle zero-padding computations which add to computation time and storage requirements.
Innovation Solution
The implementation of a circuit arrangement with multiple registers, address generation circuits, and pipelined computation circuits that eliminate zero-padding computations and reduce context switching overhead by generating addresses for input and output feature maps, allowing concurrent processing of successive layers using ping-pong memories.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If batch processing is used to reduce computational costs, then computational efficiency is improved, but latency increases as batch size increases
Solution Approach 1:
The processing pipeline is segmented into multiple independent stages (address generation, data retrieval, convolution computation, output generation) that can operate concurrently. Multiple streaming processors work in parallel on different batches, allowing overlapping of computation and data transfer operations, thus reducing overall latency while maintaining high computational throughput
Solution Approach 2:
Address generation is performed in advance during the data retrieval phase, and intermediate results are pre-computed and stored in buffer memory before being passed to the next processing stage. This preliminary preparation eliminates waiting time between stages and reduces overall processing latency
2Reliability
If zero-padding is applied to input feature maps, then computational completeness is improved, but computation time and storage requirements increase
Solution Approach 1:
The zero-padding operation is completely removed from the processing pipeline. Instead of padding the input feature map with zeros, the system dynamically adjusts the convolution window boundaries to process only the valid data region, eliminating unnecessary computations and storage requirements while maintaining correct output results
Solution Approach 2:
Rather than expanding the input data with padding to accommodate fixed-size convolution operations, the approach inverts the strategy by adjusting the convolution operation itself to fit the actual data boundaries, processing features only where valid input data exists
3Adaptability or versatility
If context switching is implemented between layers, then processing flexibility is improved, but processing speed decreases
Solution Approach 1:
The system maintains continuous data flow through the neural network layers using ping-pong buffer architecture. While one layer is computing, the next layer is simultaneously preparing its address generation and data retrieval operations. This overlapping eliminates idle context switching delays and maintains continuous useful computation across all layers
Solution Approach 2:
The system uses periodic ping-pong buffer switching between layers, where data is alternately written to and read from paired buffers in a rhythmic fashion. This periodic action pattern synchronizes data movement with computation cycles, eliminating random context switching overhead while maintaining processing flexibility
Data Source
AI summary
Disclosed circuits and methods involve a first register configured to store of a first convolutional neural network (CNN) instruction during processing of the first CNN instruction and a second register configured to store a second CNN instruction during processing of the second CNN instruction. Each of a plurality of address generation circuits is configured to generate one or more addresses in response to an input CNN instruction. Control circuitry is configured to select one of the first CNN instruction or the second CNN instruction as input to the address generation circuits.