Dynamic IFM Partitioning Circuit for FPGA Convolution Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing CNN processor designs face challenges in efficiently partitioning data and computation across different configurations, leading to inefficiencies due to static expansion of IFMs, which increases latency and reduces performance, especially on FPGAs where dynamic expansion is not scalable and resource-intensive.
Innovation Solution
The proposed solution involves a circuit arrangement that dynamically partitions a 3-D input feature map into sub-volumes based on available storage, using a request generator and pixel iterator circuit to manage data access and perform MAC operations efficiently, allowing for runtime programmability and adaptability to various CNN layers and configurations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If static expansion of the input feature map (IFM) is performed to enable matrix multiplication operations, then the convolution operation can be executed on GPUs, but the memory bandwidth is limited and latency increases
Solution Approach 1:
The patent implements dynamic expansion of the input feature map at runtime rather than static expansion. The system uses a request generator to dynamically generate read requests for IFM data elements based on the current computation needs, allowing the expansion to adapt to varying computational requirements and reduce latency by only loading necessary data.
Solution Approach 2:
The patent segments the large IFM into smaller sub-volumes that can be processed in batches. The request generator divides the IFM access into multiple smaller read operations, allowing better utilization of memory bandwidth and reducing the latency impact by processing data in manageable chunks rather than requiring the entire expanded IFM to be available simultaneously.
2Power
If static expansion of the volume is performed in FPGA accelerators, then matrix multiplication can be performed, but the computational capacity to memory ratio is not efficiently utilized due to bandwidth limitations
Solution Approach 1:
The patent employs dynamic expansion where the IFM is expanded at runtime based on actual computation requirements. This allows FPGAs to maintain high computational efficiency by only expanding the necessary portions of the IFM dynamically, rather than statically expanding the entire volume, thus better utilizing the computational capacity to memory ratio.
Solution Approach 2:
The request generator performs preliminary actions by pre-fetching and storing subsets of IFM data elements into line buffers before they are needed for computation. This preliminary data preparation reduces the bandwidth burden during actual MAC operations, allowing the FPGA to maintain high computational efficiency without being bottlenecked by memory bandwidth limitations.
3Speed
If the entire IFM is stored in local memory for dynamic expansion, then processing speed improves, but the resource requirements exceed available FPGA memory
Solution Approach 1:
The patent segments the IFM into smaller sub-volumes and processes them in batches. Only the currently needed sub-volume is loaded into local memory for dynamic expansion, rather than loading the entire IFM. This segmentation allows fast access to the active data subset while keeping total memory usage within FPGA constraints.
Solution Approach 2:
The patent implements partial expansion by only expanding and loading the necessary subset of IFM data elements into local memory at any given time, rather than expanding the entire IFM. The request generator manages this partial expansion dynamically, ensuring that local memory resources are used efficiently for speed-critical operations while external memory handles the bulk storage.
4Adaptability or versatility
If a unified control structure is used for dynamic expansion, then the system can handle various CNN configurations, but the design complexity increases and scalability is reduced
Solution Approach 1:
The request generator is designed as a universal control structure that can handle various CNN configurations through a standardized interface. It uses configurable parameters such as IFM dimensions, sub-volume sizes, and expansion factors that can be programmed at runtime to adapt to different CNN layers and configurations, providing versatility without increasing structural complexity.
Solution Approach 2:
The patent achieves adaptability through parameter changes rather than structural changes. The request generator uses configurable parameters (IFM width, height, depth, sub-volume dimensions, expansion factor) that can be modified at runtime to suit different CNN configurations. This approach maintains a simple control structure while providing high versatility through parameter programmability.
Data Source
AI summary
A and a request generator circuit is configured to read data elements of a three-dimensional (3-D) input feature map (IFM) from a memory and store a subset of the data elements in one of a plurality of N line buffers. Each line buffer is configured for storage of M data elements. A pixel iterator circuit is coupled to the line buffers and is configured to generate a sequence of addresses for reading the stored data elements from the line buffers based on a sequence of IFM height values and a sequence of IFM width values.