Neural Network Accelerator Resource Utilization via Feature Map Slicing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face memory constraints and latency issues due to the use of single accelerators, which leads to underutilization of on-chip memory and processing elements, especially when processing high-resolution or deep data, resulting in idle accelerators and prolonged inference times.
Innovation Solution
Implementing a fine-grained row/column buffering and pipelining scheme to configure accelerators differently for each layer of the CNN, allowing for efficient data processing by distributing data slices across multiple accelerators and utilizing vertical and horizontal stride increments to reduce memory requirements and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single accelerator is used to process CNN data, then device complexity is reduced, but memory requirements increase and utilization efficiency decreases
Solution Approach 1:
The patent segments the feature map processing into multiple data slices that are distributed across multiple accelerators. Each accelerator processes a specific portion of the feature map, dividing the computational workload to reduce individual memory requirements while maintaining overall processing capability.
Solution Approach 2:
The patent introduces a new dimension to accelerator configuration by assigning different accelerators to different spatial dimensions (rows or columns) of the feature map. This dimensional distribution allows parallel processing while reducing the memory footprint required at each individual accelerator.
2Device complexity
If a single accelerator processes all data, then device complexity is reduced, but processing time increases due to idle accelerators
Solution Approach 1:
The patent divides the feature map into multiple data slices and assigns them to different accelerators, enabling parallel processing. This segmentation eliminates idle time by ensuring all accelerators are actively processing different portions of the data simultaneously, reducing overall inference time.
Solution Approach 2:
The patent implements continuous data flow where feature maps are processed in sliding windows across multiple accelerators. This continuous processing approach ensures that accelerators remain busy and productive throughout the inference process, eliminating idle periods and reducing total processing time.
3Reliability
If feature map data is processed without slicing, then data integrity is maintained, but memory requirements increase significantly
Solution Approach 1:
The patent slices the feature map into multiple data slices along spatial dimensions, processing only the necessary portions at each accelerator. This segmentation maintains data integrity through proper indexing and reconstruction while significantly reducing the memory requirements at each individual accelerator device.
Solution Approach 2:
The patent applies local quality by processing different regions of the feature map at different accelerators with appropriate precision and resolution. Each accelerator handles its specific local portion of the data, maintaining overall data integrity while optimizing memory usage at each location.
Data Source
AI summary
A method for configuring a set of hardware accelerators to process a CNN. In an embodiment, the method includes one or more computer processors determining a set of parameters related to a feature map to analyze at a respective layer of the CNN, the set of parameters include quantization value and respective values that describe a shape of the feature map. The method further includes configuring a set of hardware accelerators for the respective layer of the CNN. The method further includes receiving a portion of the feature map to the configured set of hardware accelerators for the respective layer of the CNN, wherein the received portion of the feature map includes a group of sequential data slices. The method further includes analyzing the group of sequential data slices among the configured set of hardware accelerators.