Hybrid Memory Cube Interposed Between Dataflow Units
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current integrated circuits face challenges in efficiently handling large amounts of data due to limitations in communication between dataflow processing units, which hampers performance in applications like artificial intelligence and machine learning.
Innovation Solution
The introduction of a first memory unit, such as a hybrid memory cube (HMC), interposed between dataflow processing units to facilitate high-bandwidth communication, utilizing circular buffers for controlling data transfer and implementing shared memory architecture to allocate operations across processing units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data transfer between dataflow processing units is controlled using traditional memory architectures, then device complexity is reduced, but communication bandwidth and data handling performance deteriorate
Solution Approach 1:
The memory architecture is segmented into multiple banks, each capable of independent operation. This segmentation allows parallel data access and transfer operations, significantly increasing communication bandwidth between dataflow processing units without requiring a monolithic complex memory structure.
Solution Approach 2:
The patent introduces a three-dimensional stacked memory architecture where memory banks are arranged in vertical layers with through-silicon vias for inter-layer communication. This dimensional change enables high-bandwidth data transfer without increasing planar footprint, resolving the contradiction between performance and complexity.
2Speed
If high-bandwidth memory communication is implemented between dataflow processing units, then communication efficiency improves, but device complexity increases
Solution Approach 1:
Buffer memory structures are introduced as intermediary components between dataflow processing units and the high-bandwidth memory system. These buffers absorb data transfer variations and simplify the interface complexity, enabling high communication efficiency without proportionally increasing overall device complexity.
Solution Approach 2:
Data is pre-loaded into buffer memories and organized in advance before being transferred to dataflow processing units. This preliminary action reduces the complexity of real-time data management during high-speed operations, maintaining communication efficiency while controlling device complexity.
3Productivity
If shared memory architecture is used to allocate operations across processing units, then productivity increases, but device complexity increases
Solution Approach 1:
The shared memory architecture is designed with universal interfaces and control logic that can serve multiple dataflow processing units simultaneously. This multi-functionality allows the same memory structure to support diverse operation allocation patterns, increasing processing throughput without proportionally increasing architecture complexity.
Data Source
AI summary
A combination of memory units and dataflow processing units is disclosed for computation. A first memory unit is interposed between a first dataflow processing unit and a second dataflow processing unit. Operations for a dataflow graph are allocated across the first dataflow processing unit and the second dataflow processing unit. The first memory unit passes data between the first dataflow processing unit and the second dataflow processing unit to execute the dataflow graph. The first memory unit is a high bandwidth, shared memory device including a hybrid memory cube. The first dataflow processing unit and second dataflow processing unit include a plurality of circular buffers containing instructions for controlling data transfer between the first dataflow processing unit and second dataflow processing unit. Additional dataflow processing units and additional memory units are included for additional functionality and efficiency.