Hybrid CNN Accelerator Traversal for Kernel Reuse and Power Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep convolutional neural networks (CNNs) face challenges in achieving high throughput and power efficiency on portable devices with limited battery capacity, particularly in edge devices that require real-time machine learning tasks, necessitating power-efficient acceleration of compute-intensive operations like direct convolution.
Innovation Solution
A hybrid traversal method for CNN accelerator architectures that involves receiving and reusing kernel microbatches for direct and Winograd convolutions, combined with quantization and pooling, to reduce power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If direct convolution operations are performed with high precision to maintain accuracy, then recognition accuracy is preserved, but power consumption increases significantly
Solution Approach 1:
The patent segments the convolution operation into two distinct paths: a high-precision path for critical computations that maintain recognition accuracy, and a low-precision quantized path for less critical computations that reduce power consumption. This segmentation allows the system to selectively apply different precision levels based on the importance of each computation, resolving the contradiction between accuracy and power usage.
Solution Approach 2:
The patent applies local quality by assigning different precision characteristics to different parts of the convolution operation. Specifically, certain channels or feature maps are processed with high precision while others use lower precision quantization. This localized differentiation allows the system to maintain overall recognition accuracy while reducing power consumption in less critical regions.
2Use of energy by moving object
If quantization to lower precision is applied to reduce power consumption, then power efficiency improves, but recognition accuracy may degrade
Solution Approach 1:
The patent implements dynamic precision adjustment where the system can adaptively switch between high-precision and low-precision processing modes based on runtime conditions such as available power, performance requirements, and input data characteristics. This dynamic approach allows the system to optimize the balance between power efficiency and recognition accuracy in real-time rather than being fixed at a single precision level.
3Productivity
If zero-skip logic is implemented to exploit data sparsity and accelerate computations, then performance improves, but power consumption increases due to expensive zero-skip logic
Solution Approach 1:
The patent extracts and eliminates the expensive zero-skip logic component from the system. Instead of implementing complex logic to detect and skip zero-valued operations, the system simply performs the multiplication operations anyway but uses lower precision arithmetic, which naturally handles zero values efficiently without requiring additional detection logic, thereby removing the source of excessive power consumption.
4Productivity
If kernel pruning is applied to increase sparsity for improved acceleration, then acceleration factor increases, but model size reduction requires careful balancing
Solution Approach 1:
The patent changes the precision parameter of the kernel weights from high precision to low precision quantized values. This parameter change increases sparsity in the computational sense (many quantized values map to the same representable value), which improves acceleration factor without requiring physical removal of kernel elements. The model structure remains intact, avoiding the complexity issues associated with aggressive kernel pruning while still achieving performance benefits.
Data Source
AI summary
Disclosed is a hybrid traversal apparatus and method for a convolution neural network (CNN) accelerator architecture that receives input feature map (IFM) microbatches from a pixel memory and receiving kernel microbatches from a kernel memory, multiplies the IFM microbatches by the kernel microbatches while reusing the kernel microbatches based on a kernel reuse factor for at least one of a direct convolution (DConv) or a Winograd convolution (WgConv), to obtain output feature map (OFM) microbatches, and writes the generated OFM microbatches to the pixel memory, after quantization, non-linear function, and pooling on a result of the multiplying.