Memory Chiplet Formula Execution to Reduce Sparse Access Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing sparse matrix multiplication (SpMM) performance is limited by memory bandwidth and imbalanced compute and memory access due to inefficient storage formats and memory indirection, leading to high latency and bandwidth bottlenecks.
Innovation Solution
Implementing in-memory compute circuitry within memory chiplets that register formulas with physical memory addresses, allowing direct communication between chiplets through an interconnect chiplet, balancing memory access and compute operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If memory indirection is used for sparse matrix storage, then storage flexibility is improved, but memory access latency increases
Solution Approach 1:
The system pre-computes and caches the results of memory access formulas during a registration phase. When a formula like addr = base + index * stride is encountered, the system pre-calculates the physical addresses and stores them in a cache, avoiding repeated computation and memory indirection during actual data access operations.
Solution Approach 2:
The patent introduces an address calculation cache as an intermediary between the logical memory addresses and physical memory locations. This cache stores pre-computed address mappings, acting as a mediator that eliminates the need for real-time formula evaluation and reduces memory access latency while preserving the flexibility of sparse storage formats.
2Quantity of substance
If data is distributed across multiple memory chiplets, then memory capacity is improved, but communication overhead increases
Solution Approach 1:
The patent divides the memory system into multiple independent chiplet segments, each with its own address calculation cache. This segmentation allows each chiplet to autonomously handle address translation for its local data, reducing the need for inter-chiplet communication and minimizing communication overhead while maintaining scalable memory capacity.
Solution Approach 2:
Each memory chiplet is equipped with local address calculation caches that are optimized for its specific data partition. This local quality approach ensures that address translation and data access operations are performed locally within each chiplet, eliminating the need for remote communication and reducing overall system communication overhead.
3Productivity
If in-memory compute circuitry is added to memory chiplets, then computing performance is improved, but device complexity increases
Solution Approach 1:
The memory chiplet is designed with multi-functional circuitry that can perform both traditional memory access operations and in-memory compute operations using the same physical infrastructure. The address calculation caches and compute units are integrated into the existing memory controller, allowing the same hardware to serve multiple functions without proportionally increasing complexity.
Solution Approach 2:
The patent merges the address calculation logic, cache memory, and compute units into a single integrated in-memory compute engine within each chiplet. This consolidation combines multiple functions into unified circuitry, improving computing performance while avoiding the complexity overhead that would result from separate independent components.
Data Source
AI summary
Systems, apparatus, articles of manufacture, and methods are disclosed for executing memory access formulas in memory chiplets. An example system includes a plurality of memory chiplets including a first memory chiplet and a second memory chiplet. In the example system, the first memory chiplet is to register a formula with an identifier in a formula data structure. In the example system, the first memory chiplet is also to determine, based on the formula data structure, that the second memory chiplet stores data corresponding to the formula. The example system also includes interconnect chiplet circuitry connected to the plurality of memory chiplets, the interconnect chiplet circuitry to communicate a request from the first memory chiplet to the second memory chiplet to obtain the data corresponding to the formula.