CNN Accelerator Processing Blocks with Local Memory Data Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing convolutional neural network (CNN) accelerators face limitations in scalability, flexibility, and energy efficiency due to limited bandwidth, high energy consumption, memory capacity issues, and limited data reuse, which hinder the processing efficiency of deep neural networks.
Innovation Solution
A processing method and architecture that utilize an array of unitary processing blocks with local memories, enabling data transmission and computation in both vertical and horizontal directions, implementing broadcast and multicast communication modes, and prioritizing data requests to optimize data reuse and reduce energy consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional CNN accelerators use large memory capacity to store data for processing, then data storage capability is improved, but energy consumption increases significantly and memory capacity wall problem occurs
Solution Approach 1:
The patent divides the accelerator architecture into multiple processing blocks (PBs) with distributed local memories, each handling specific data locally. This segmentation eliminates the need for a single large centralized memory, reducing the memory capacity wall problem while lowering energy consumption through localized data processing.
Solution Approach 2:
Each processing block is equipped with local memories tailored to its specific computational needs, allowing data to be stored and processed locally rather than being fetched from a centralized memory system. This local quality approach reduces memory access energy consumption and avoids the scalability limitations of large centralized memory.
2Productivity
If CNN accelerators increase bandwidth through effective broadcast medium, then data transmission efficiency is improved, but device complexity increases
Solution Approach 1:
The interconnection network in the patent is designed to handle multiple communication patterns (broadcast, multicast, point-to-point) using a unified architecture. This multi-functional communication infrastructure achieves high data transmission efficiency without requiring separate specialized hardware for each communication mode, thus avoiding excessive complexity.
3Productivity
If CNN accelerators reuse data extensively to improve processing efficiency, then productivity is improved, but communication overhead and energy consumption increase
Solution Approach 1:
The patent loads and caches frequently reused data into local memories of processing blocks before computation begins. This preliminary action ensures that data is readily available locally during processing, maximizing data reuse benefits while minimizing the energy cost of repeated data communication during the actual computation phases.
Data Source
AI summary
A processing method in a convolutional neural network accelerator includes an array of unitary processing blocks associated with a set of respective local memories and performing computing operations on data stored in its local memories, wherein: during respective processing cycles, some unitary blocks receive and/or transmit data from or to neighbouring unitary blocks in at least one direction selected, on the basis of the data, from among the vertical and horizontal directions in the array; during the same cycles, some unitary blocks perform a computing operation in relation to data stored in their local memories during at least one previous processing cycle.