Near-Memory Units for Directional Blocked Data Quantization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional computer systems face bottlenecks in data processing due to the back-and-forth movement of data between the CPU and memory, especially in data-intensive tasks, which slows down system performance and increases energy consumption.

Innovation Solution

Implementing processing-in-memory (PIM) technology, which integrates processing units within memory chips to process data where it is stored, and employing smart data organization and processing strategies to convert scalar data formats to directional blocked data formats, optimizing memory usage and computational efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If data is processed by CPU after retrieval from memory, then processing can be performed, but data movement between CPU and memory creates bottlenecks and slows down system performance

Engineering Contradiction:

Improvedata processing speedVSAvoiddata movement time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent merges the processing unit with the memory structure by integrating simple processors directly into the memory chip. This allows data to be processed within the memory itself rather than being transferred to a separate CPU, eliminating the data movement bottleneck and significantly reducing latency for data-intensive tasks.

Inventive Principle:

Principle #5Merging (Combining)

2Productivity

If processing units are integrated within memory chips (PIM), then data processing speed and efficiency improve, but computational complexity and energy consumption for data movement increase

Engineering Contradiction:

Improvedata processing speedVSAvoidenergy consumption

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

By integrating processing units directly into the memory chip, the patent eliminates the need for frequent data transfers between separate memory and processing components. This merger significantly reduces energy consumption associated with data movement while maintaining processing capability, as data can be processed in-place within the memory structure.

Inventive Principle:

Principle #5Merging (Combining)

3Productivity

If scalar data formats are converted to directional blocked data formats using PIM, then memory usage and computational efficiency are optimized, but the complexity of data organization and processing increases

Engineering Contradiction:

Improvecomputational efficiencyVSAvoiddata organization complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent segments data into directional blocked formats that are optimized for PIM processing. By dividing data into structured blocks with specific directional arrangements, the system optimizes memory usage and computational efficiency while the integrated PIM processing units automatically handle the complexity of data organization, reducing the burden on external control systems.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS20250307133A1Offloading Quantization of Directional Blocked Data Formats to Near-Memory Units

Publication Date: 2025.10.02 ADVANCED MICRO DEVICES INC

AI summary

Offloading quantization of directional blocked data formats to near-memory units is described. In one or more implementations, a system includes a processing-in-memory memory circuit configured to store data. The system can also include a memory controller circuit configured to map the data to the processing-in-memory memory circuit. The system can also include a processing-in-memory arithmetic logic unit configured to quantize the data from a scalar format to a directional blocked data format. The scalar format can be or can include a scalar format such as BFloat 16. The directional blocked data format can be or can include a microscaling format.