CNN Accelerator Cache-Integrated Processing Modules
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing convolutional neural network (CNN) accelerators face inefficiencies due to bandwidth limitations and excessive data transfer, which degrade processing efficiency, and are often limited by the size and arrangement of input data, leading to suboptimal computational resource utilization.
Innovation Solution
An apparatus comprising a cache module integrated with processing modules, allowing for direct data exchange and reuse within the cache, reducing external data transmission and enabling flexible configuration for various operations, with a control module managing data flow and scheduling to optimize resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If dedicated hardware accelerators are used to accelerate convolution operations, then processing speed is improved, but data transfer bandwidth limitations and excessive data transfer reduce processing efficiency
Solution Approach 1:
The patent combines multiple processing elements into processing modules that share common input and output interfaces. This merging reduces the total number of interfaces needed and minimizes data transfer overhead, as multiple processing elements can access shared input data through a single interface rather than requiring separate interfaces for each element.
Solution Approach 2:
The processing modules are designed with universal input and output interfaces that can handle data for multiple processing elements. These interfaces serve multiple functions by providing data to various processing elements and collecting results from them, reducing the need for specialized dedicated connections for each processing element.
2Productivity
If conventional accelerator architectures are used, then specific operations are accelerated, but adaptability to different input data sizes and arrangements is limited
Solution Approach 1:
The processing modules are designed with dynamic configurability, allowing the number and arrangement of processing elements within each module to be adjusted based on the specific convolution operation requirements. This dynamic structure enables the accelerator to adapt to different input data sizes, weight kernel sizes, and operational configurations while maintaining high computational efficiency.
Solution Approach 2:
The accelerator is divided into multiple processing modules, each containing multiple processing elements that can be independently configured. This segmentation allows flexible arrangement of processing elements within modules to match different computational requirements, enabling adaptation to various input data arrangements and sizes.
3Productivity
If more processing elements are added to increase computational power, then processing capability is improved, but device complexity increases
Solution Approach 1:
Multiple processing elements are merged into processing modules with shared input and output interfaces. This merging approach increases computational power by adding processing elements while reducing device complexity by eliminating the need for separate dedicated interfaces for each processing element. The shared interfaces serve multiple elements, simplifying the overall system architecture.
4Ease of operation
If processing modules use separate input interfaces for each processing element, then data access is simplified, but bandwidth limitations and excessive data transfer occur
Solution Approach 1:
The patent merges multiple individual processing element interfaces into shared input and output interfaces at the processing module level. This merging maintains ease of operation by providing clear defined interfaces for data access while improving bandwidth utilization by reducing redundant data transfers. Input data is fetched once at the module level and then distributed to multiple processing elements within the module, eliminating repeated data transfers that would occur with separate interfaces for each processing element.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An accelerator and a system for accelerating operations are disclosed. A respective apparatus comprises an interface configured to couple the apparatus to an interconnect, a plurality of processing modules, each configured to process data, a control module configured to control processing of each of the plurality of processing modules, and a cache module configured to store at least a portion of data processed by at least one of the plurality of processing modules. Each processing module further includes a processing core configured to process data by performing an operation on the data using a plurality of processing elements, an input control unit configured to retrieve data via the interface and data stored in the cache module and to provide the retrieved data to the processing core, and an output control unit configured to provide data processed by the processing core to the interface and the cache module.