On-Chip Interconnect for Load-Balanced Memory Channel Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network computations face performance penalties due to load imbalances caused by hard mapping of channel controllers to specific memory channels, leading to inefficient data access and processing delays.
Innovation Solution
Implementing a crossbar/on-chip interconnect (OCI) that allows each channel controller to access any memory channel, dynamically balancing address allocations and improving performance by enabling load-balanced data access across all memory channels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If channel controllers are hard mapped to specific memory channels, then the system structure is simplified and easier to implement, but load imbalances occur leading to performance penalties and processing delays
Solution Approach 1:
The patent implements a universal on-chip interconnect that allows any channel controller to access any memory channel, transforming the system from a dedicated one-to-one mapping to a many-to-many configuration. This enables load balancing across all channel controllers while maintaining simplified individual controller designs, as each controller can dynamically access any memory channel based on workload distribution.
2Device complexity
If channel controllers are hard mapped to specific memory channels, then the control logic is simplified, but data access efficiency deteriorates due to load imbalances
Solution Approach 1:
The patent introduces an on-chip interconnect as an intermediary layer between channel controllers and memory channels. This mediator handles the complexity of dynamic routing and load balancing, allowing individual channel controllers to maintain simple control logic while achieving efficient data access through the interconnect's intelligent resource allocation.
3Device complexity
If hard mapping is used, then the system architecture is simpler, but processing delays increase due to inefficient data access
Solution Approach 1:
The patent transitions from a static hard-mapped architecture to a dynamic on-chip interconnect system where channel controllers can adaptively access any memory channel based on real-time workload conditions. This dynamic allocation optimizes data access patterns and reduces processing delays while maintaining architectural simplicity through standardized interconnect interfaces.
Data Source
AI summary
Methods, systems, and apparatus, including computer-readable media, are described for an integrated circuit that accelerates machine-learning computations. The circuit includes processor cores that each include multiple channel controllers; an interface controller for coupling each channel controller to any memory channel of a system memory; and a fetch unit in each channel controller. Each fetch is configured to: receive channel data that encodes addressing information; obtain, based on the addressing information, data from any memory channel of the system memory using the interface controller; and write the obtained data to a vector memory of the processor core via the corresponding channel controller that includes the respective fetch unit.