Memory Device Parallel Arithmetic Processing via Interposer

Solution Overview

Problem

Current memory devices face limitations in efficiently processing large amounts of computations in parallel, particularly in high-performance electronic systems, as they often require hardware accelerators for data communication and result transmission, leading to latency and increased costs in systems like neural networks.

Innovation Solution

A memory module architecture that includes multiple high-bandwidth memory (HBM) devices with interposer-based signal transmission and through-silicon via (TSV) connections, allowing direct data communication and arithmetic processing between HBMs without relying on hardware accelerators, enabling distributed and parallel arithmetic processing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If memory devices use hardware accelerators for data communication and result transmission, then computation processing capability is improved, but latency increases and system cost increases

Engineering Contradiction:

Improvecomputation processing capabilityVSAvoidlatency

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent merges memory functions with arithmetic processing functions into a single integrated memory device. The memory device includes memory cells for data storage and arithmetic circuits for performing computations, allowing both data communication and computation to occur within the same device without requiring separate hardware accelerators, thereby reducing latency while maintaining high computation processing capability

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The memory device is designed to perform multiple functions: it can store data in its memory cells and simultaneously perform arithmetic operations using its arithmetic circuits. This multi-functional design eliminates the need for dedicated hardware accelerators for specific computation tasks, reducing both latency and system cost while maintaining high productivity

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Productivity

If memory devices use hardware accelerators for data communication and result transmission, then computation processing capability is improved, but system cost increases

Engineering Contradiction:

Improvecomputation processing capabilityVSAvoidsystem cost

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent merges memory functions with arithmetic processing functions into a single integrated memory device. The memory device includes memory cells for data storage and arithmetic circuits for performing computations, allowing both data communication and computation to occur within the same device without requiring separate hardware accelerators, thereby reducing latency while maintaining high computation processing capability

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The memory device is designed to perform multiple functions: it can store data in its memory cells and simultaneously perform arithmetic operations using its arithmetic circuits. This multi-functional design eliminates the need for dedicated hardware accelerators for specific computation tasks, reducing both latency and system cost while maintaining high productivity

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Productivity

If multiple memory devices communicate through hardware accelerators, then arithmetic processing can be performed, but bandwidth characteristics are limited

Engineering Contradiction:

Improvearithmetic processing capabilityVSAvoidbandwidth characteristics

Core Design Contradiction:

ProductivityVSSpeed

Solution Approach 1:

The patent divides the memory module into multiple independent memory devices, each capable of performing arithmetic operations locally. This segmentation allows parallel arithmetic processing across multiple devices, significantly increasing bandwidth characteristics and overall processing speed without requiring centralized hardware accelerator communication

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS10642612B2Memory device performing parallel arithmetic processing and memory module including the same

Publication Date: 2020.05.05 SAMSUNG ELECTRONICS CO LTD

AI summary

A memory module includes a first memory device configured to receive data and first information from a hardware accelerator, to generate an arithmetic result by performing arithmetic processing using the data and the first information, and to output the arithmetic result through an interface with at least one other memory device; and a second memory device configured to receive the arithmetic result from the first memory device through the interface without using the hardware accelerator, and to store the arithmetic result.