PIM MAC Operator With Floating-Point Modulation for Faster AI Compute

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Deep learning neural networks face performance degradation due to limitations in data communication between separate memory and processor units, leading to increased computational demands and inefficiencies in AI systems.

Innovation Solution

A neural network system incorporating a PIM device with a data type converter that converts 32-bit floating-point formats to 16-bit formats and a MAC operator with a data type modulator, enabling efficient MAC operations within the PIM device to improve data processing speed.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If separate memory and processor units are used in AI systems, then data storage and computation can be performed independently, but data communication limitations between memory and processor degrade AI performance

Engineering Contradiction:

ImproveAI computation performanceVSAvoiddata communication time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent merges memory and processor units into a single integrated PIM device, where memory cells store data and dedicated MAC circuits perform computations directly on the stored data. This integration eliminates the need for data to be transferred between separate memory and processor units, thereby reducing communication time and improving AI computation performance.

Inventive Principle:

Principle #5Merging (Combining)

2Productivity

If more layers are added to neural networks to improve AI performance, then learning capability increases, but computational requirements increase exponentially

Engineering Contradiction:

ImproveAI learning performanceVSAvoidcomputational power requirement

Core Design Contradiction:

ProductivityVSPower

Solution Approach 1:

The PIM device enables memory to perform computations autonomously through integrated MAC circuits that execute multiply-accumulate operations directly on data stored in memory cells. This self-service capability eliminates the need to transfer data to external processors for computation, reducing the computational power requirement at the processor level while maintaining high AI learning performance through increased network depth.

Inventive Principle:

Principle #25Self-service

3Speed

If PIM device performs arithmetic operations internally, then data processing speed improves, but data communication overhead between separate units is reduced

Engineering Contradiction:

Improvedata processing speedVSAvoiddata communication energy

Core Design Contradiction:

SpeedVSLoss of energy

Solution Approach 1:

The patent combines memory storage and arithmetic computation functions into a single PIM device, allowing data processing to occur in-place without transfer between separate units. This merging architecture enables internal MAC operations that improve data processing speed while simultaneously eliminating the energy consumption associated with data communication between memory and processor units.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS11817884B2Neural network system with multiplication and accumulation(MAC) operator

Publication Date: 2023.11.14 SK HYNIX INC

AI summary

A neural network system includes a data type converter and a MAC operator. The data type converter may convert 32-bit floating-point format into one of a plurality of 16-bit floating-point formats. The MAC operator may perform MAC operations using 16-bit floating-point format data converted by the data type converter. The MAC operator includes a data type modulator configured to modulate the bit number of the converted 16-bit floating-point format to provide a modulated floating-point format with bit number different from the bit number of the converted 16-bit floating-point format.