Approximate In-DRAM Acceleration for GPU Memory Bottlenecks

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

GPUs face memory bandwidth bottlenecks and high data transfer energy costs due to their throughput processing model, making near-data acceleration challenging, especially with DRAM integration, which is constrained by area, power, and thermal limitations.

Innovation Solution

The AXRAM architecture simplifies in-DRAM accelerators by approximating neural transformations into MAC and LUT operations, integrating them into DRAM banks without altering the memory structure, using shift-add and LUT operations with early termination, and leveraging DRAM's high internal bandwidth.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If many accelerators are integrated close to DRAM data to enable near-data acceleration for GPU workloads, then computation efficiency improves, but area, power, and thermal constraints of DRAM are exceeded

Engineering Contradiction:

Improvecomputation efficiencyVSAvoidDRAM area

Core Design Contradiction:

ProductivityVSArea of stationary object

Solution Approach 1:

The patent uses approximate MAC units that sacrifice precision for area efficiency. These simplified units use limited iterations of shift-add and LUT access operations with early termination, accepting approximate results rather than exact computations. This allows many such units to be integrated within DRAM area constraints, enabling near-data acceleration without exceeding the area budget.

Inventive Principle:

Principle #27Cheap short-living objects (Disposable)

Solution Approach 2:

The patent changes the operational parameters of MAC units from full-precision to approximate computation. By limiting the number of iterations and accepting approximation errors, the area requirement per MAC unit is dramatically reduced. This parameter change enables integration of many accelerators within DRAM while maintaining acceptable computational accuracy for the target applications.

Inventive Principle:

Principle #35Parameter changes

2Productivity

If 3D/2.5D stacking is used to integrate accelerators close to DRAM, then near-data processing capability improves, but the cost of TSVs increases significantly

Engineering Contradiction:

Improvenear-data processing capabilityVSAvoidTSV cost

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent integrates MAC units and LUT units within the same DRAM chip, allowing them to share the same substrate and interconnect resources. This multi-functional integration within a single chip eliminates the need for separate 3D stacking and TSV connections, reducing complexity while maintaining near-data processing capability.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Productivity

If DRAM is modified to integrate complex accelerators, then computation capability improves, but the overhead and complexity of DRAM structure increases

Engineering Contradiction:

Improvecomputation capabilityVSAvoidDRAM structure complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent uses simplified approximate MAC units that require minimal integration resources. These units use limited iterations and early termination, reducing the complexity of integration logic and control structures needed within DRAM. The simplicity of these units allows integration without significantly increasing DRAM structural complexity.

Inventive Principle:

Principle #27Cheap short-living objects (Disposable)

Solution Approach 2:

The patent changes the computational precision parameter from exact to approximate, which dramatically simplifies the integration requirements. The approximate MAC units require fewer transistors, simpler control logic, and less complex interconnect routing within DRAM, enabling integration without excessive structural overhead.

Inventive Principle:

Principle #35Parameter changes

4Productivity

If data transfer rate is increased to provide higher bandwidth to accelerators, then near-data acceleration effectiveness improves, but signal integrity deteriorates and power consumption increases superlinearly

Engineering Contradiction:

ImprovebandwidthVSAvoiddata transfer energy

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The patent merges the MAC units and LUT units into a unified accelerator structure that is directly integrated within the DRAM chip. This consolidation allows the units to share common resources including data paths, control logic, and power distribution, reducing redundant data transfers and lowering overall energy consumption while maintaining high effective bandwidth.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS12461712B2In-memory near-data approximate acceleration

Publication Date: 2025.11.04 GEORGIA TECH RES CORP

AI summary

A random access memory may include memory banks and arithmetic approximation units. Each arithmetic approximation unit may be dedicated to one or more of the memory banks and include a respective multiply-and-accumulate unit and a respective lookup-table unit. The respective multiply-and-accumulate unit is configured to iteratively perform shift and add operations with two inputs and to provide a result of the shift and add operations to the respective lookup-table unit. The result approximates or is a product of the two inputs. The respective lookup-table unit is configured produce an output by applying a pre-defined function to the result. The arithmetic approximation units are configured for parallel operation. The random access memory may also include a memory controller configured to receive instructions, from a processor, regarding locations within the memory banks from which to obtain the two inputs and in which to write the output.