CNN Hardware Accelerator with Vector MAC Units

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing hardware and software systems face challenges in achieving high computational efficiency when processing convolutional neural networks (CNNs) due to varied data access patterns and layer sizes, leading to inefficiencies in processing complex CNNs.

Innovation Solution

A hardware accelerator architecture that includes a compute core with multiple compute units, each equipped with memory caches and vector multiply-accumulate units, optimized for efficient processing of CNNs by loading contiguous data traces and weight vectors, and operating in cooperative or independent modes to maximize parallelism and minimize latency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If standard hardware and software systems are used to process CNNs, then general-purpose computing is achieved, but computational efficiency deteriorates due to varied data access patterns and layer sizes

Engineering Contradiction:

Improveability to handle varied data access patternsVSAvoidcomputational efficiency

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The processing system is segmented into multiple compute units, each capable of independently processing different data access patterns. Each compute unit contains its own memory caches and vector MAC units, allowing parallel processing of different CNN layers and operations without interfering with each other, thus maintaining high computational efficiency across varied workloads

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The compute units can dynamically switch between cooperative and independent operating modes based on the specific CNN layer being processed. This dynamic adaptability allows the system to optimize for either parallelism (cooperative mode) or independent data access patterns (independent mode), resolving the contradiction between versatility and efficiency

Inventive Principle:

Principle #15Dynamics

2Speed

If multiple memory caches are provided in each compute unit, then data access efficiency is improved, but device complexity increases

Engineering Contradiction:

Improvedata access speedVSAvoidcomplexity of compute unit structure

Core Design Contradiction:

SpeedVSDevice complexity

Solution Approach 1:

Multiple memory caches (first memory cache for map traces and second memory cache for kernel traces) are merged into a single compute unit structure, sharing common control logic and interconnect resources. This integration reduces the overall device complexity while maintaining fast data access speeds through localized caching of frequently accessed data

Inventive Principle:

Principle #5Merging (Combining)

3Productivity

If vector multiply-accumulate units are used, then computational throughput is improved, but energy consumption increases

Engineering Contradiction:

Improvecomputational throughputVSAvoidenergy consumption of accelerator

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The vector MAC units are designed to perform continuous multiply-accumulate operations without interruption, processing data in streamlined vectors rather than scalar operations. This continuous operation maximizes throughput while minimizing the energy cost per computation by keeping the computational units actively engaged rather than idle

Inventive Principle:

Principle #20Continuity of useful action

Solution Approach 2:

Traditional scalar multiplication and addition operations are replaced with vectorized MAC units that perform multiple operations in parallel. This substitution increases computational throughput significantly while the energy consumption increases only proportionally, not exponentially, due to the efficient vector processing architecture

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS11775313B2Hardware accelerator for convolutional neural networks and method of operation thereof

Publication Date: 2023.10.03 PURDUE RES FOUND

AI summary

An accelerator for processing of a convolutional neural network (CNN) includes a compute core having a plurality of compute units. Each compute unit includes a first memory cache configured to store at least one vector in a map trace, a second memory cache configured to store at least one vector in a kernel trace, and a plurality of vector multiply-accumulate units (vMACs) connected to the first and second memory caches. Each vMAC includes a plurality of multiply-accumulate units (MACs). Each MAC includes a multiplier unit configured to multiply a first word that of the at least one vector in the map trace by a second word of the at least one vector in the kernel trace to produce an intermediate product, and an adder unit that adds the intermediate product to a third word to generate a sum of the intermediate product and the third word.