PIM MAC Operator Using Fixed-Point Accumulation for Faster AI Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning neural networks require increased computational resources due to the exponential growth in data processing needs, especially when processors and memory are separated, leading to performance degradation due to limited data communication between them.
Innovation Solution
A PIM (Processing-In-Memory) device integrates processors and memory on a semiconductor chip, utilizing a MAC operator with multipliers, floating-point to fixed-point converters, an adder tree, and an accumulator to enhance data processing speed within the neural network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If processors and memory are separated, then device architecture is conventional and flexible, but data communication between memory and processor is limited causing performance degradation
Solution Approach 1:
The patent merges memory and processing functions into a single PIM device by integrating multipliers, adder trees, and accumulators directly within the memory structure. This allows arithmetic operations to be performed on data while it resides in memory, eliminating the need for frequent data transfer between separate memory and processor components, thereby resolving the contradiction between maintaining conventional architecture flexibility and achieving high data processing speed.
2Productivity
If the number of neural network layers is increased, then artificial intelligence performance is improved, but the amount of computation required increases exponentially
Solution Approach 1:
The patent segments the computation process by distributing arithmetic operations across multiple PIM devices that can operate in parallel. Each PIM device handles specific MAC operations on portions of the neural network data, allowing the system to process deeper neural networks with more layers by dividing the computational workload across multiple integrated memory-processing units rather than requiring a single high-power processor.
3Productivity
If data communication between memory and processor is increased, then more computation can be performed, but communication limitations cause performance degradation
Solution Approach 1:
The PIM device enables memory to serve its own processing needs by performing arithmetic operations directly on stored data. The integrated multipliers, adder trees, and accumulators allow the memory structure to execute MAC operations without requiring external processor intervention, thereby eliminating communication delays and allowing computation throughput to scale without being constrained by memory-processor communication bandwidth.
Data Source
AI summary
A MAC operator includes a plurality of multipliers, a plurality of floating-point to fixed-point converters, an adder tree, an accumulator, and a fixed-point to floating-point converter. Each of the plurality of multipliers may perform a multiplication operation on first data and second data of a single-precision floating-point (FP32) format to output multiplication result data of the FP 32 format. Each of the plurality of floating-point to fixed-point converters may convert the FP 32 format into a fixed-point format. The adder tree may perform a first addition operation on the data of the fixed-point format. The accumulator may perform an accumulation operation on the data output from the adder tree. And the fixed-point to floating-point converter may convert the data of the fixed-point format into data of the FP32 format.