Hierarchical Multicast Decoders for In-Memory Compute Data Loading
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing in-memory computing architectures face challenges in efficiently providing input data to a large number of memory cells for highly concurrent processing, leading to latency overhead and efficiency loss due to serial data access and limited parallelization.
Innovation Solution
Implementing a memory architecture with multicast capability and co-located compute elements, utilizing hierarchical decoders to distribute data and commands to multiple subarrays, and employing an architecture-aware compiler framework to optimize dataflow for parallelizable workloads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If in-memory computing architectures provide relevant input data to a large number of memory cells for highly concurrent processing, then processing speed and parallelization are improved, but latency overhead and efficiency loss increase due to serial data access
Solution Approach 1:
The memory architecture is segmented into multiple hierarchical levels (memory banks, sub-banks, and arrays) with distributed processing elements. Each segment can independently receive and process data, enabling parallel operations across segments while reducing the serialization bottleneck at any single level.
Solution Approach 2:
The patent introduces a new dimension of parallelization by implementing multicast capability that allows a single data source to simultaneously distribute data to multiple destinations across different memory banks and processing elements. This transforms serial data access into parallel data distribution, reducing latency overhead while maintaining high processing speed.
2Productivity
If compute elements are deployed in or near memory cells to reduce data movement costs, then data efficiency is improved, but providing input data to multiple memory cells for concurrent processing becomes more complex
Solution Approach 1:
The memory architecture implements a universal data distribution network that serves multiple functions: it can perform traditional point-to-point data access, multicast data to multiple memory banks simultaneously, and support in-memory compute operations. This multi-functional approach maintains data efficiency while managing distribution complexity through a unified interface.
Solution Approach 2:
The patent introduces intermediate control logic and address translation mechanisms that mediate between simple memory access requests and the complex multicast distribution requirements. These intermediaries handle the complexity of routing data to multiple processing elements while presenting a simplified interface to applications, thereby maintaining data efficiency without exposing distribution complexity.
3Quantity of substance
If hierarchical memory structure with multiple levels is used, then memory capacity and parallelization potential are improved, but data access paths become longer and more complex
Solution Approach 1:
The architecture pre-establishes multicast routing paths and data distribution patterns during system initialization or before compute operations begin. By preparing data distribution routes in advance, the system reduces the complexity of real-time data access paths while maintaining high memory capacity and parallelization potential across hierarchical levels.
Data Source
AI summary
A memory architecture includes processing circuits co-located with memory subarrays for performing computations within the memory architecture. The memory architecture includes a plurality of decoders in hierarchical levels that include a multicast capability for distributing data or compute operations to individual subarrays. The multicast may be configurable with respect to individual fan-outs at each hierarchical level. A computation workflow may be organized into a compute supertile representing one or more “supertiles” of input data to be processed in the compute supertile. The individual data tiles of the input data supertile may be used by multiple compute tiles executed by the processing circuits of the subarrays, and the data tiles multicast to the respective processing circuits for efficient data loading and parallel computation.