Request Broadcasting Masters for High Bandwidth Memory Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hardware platforms face challenges in achieving efficient memory access performance for neural networks due to high memory transfer costs, which are exacerbated by the computational and data-intensive nature of complex artificial intelligence problems.
Innovation Solution
A high bandwidth memory system utilizing request broadcasting masters is implemented, where processing elements are connected to multiple memory units via a network-on-chip subsystem, with designated request broadcasting masters managing memory access requests to minimize network collisions and optimize data distribution across memory units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If each processing element directly manages memory access requests, then memory access performance is improved, but network collisions increase and device complexity increases
Solution Approach 1:
The processing elements are segmented into groups with designated request broadcasting masters, where each master manages memory access requests for its group. This segmentation reduces the number of active request generators on the network, thereby reducing collisions while maintaining efficient memory access through hierarchical request management.
Solution Approach 2:
Request broadcasting masters serve as intermediary components between processing elements and memory units. These masters aggregate and broadcast memory requests to multiple memory units simultaneously, reducing network traffic and collisions while maintaining high memory access performance through coordinated request management.
2Productivity
If data is distributed across multiple memory units, then memory utilization is improved, but the number of memory access requests increases
Solution Approach 1:
Multiple memory access requests from processing elements in the same group are merged into a single broadcast request by the request broadcasting master. This merged request is then distributed to multiple memory units simultaneously, achieving high memory utilization while reducing the total number of individual requests on the network.
Solution Approach 2:
The request broadcasting master performs multiple functions: collecting requests from multiple processing elements, merging them into a single broadcast request, and distributing to multiple memory units. This multi-functionality reduces network traffic while maintaining efficient data distribution across memory units.
3Object-generated harmful factors
If request broadcasting masters merge memory requests, then network collisions are reduced, but request processing complexity increases
Solution Approach 1:
The request broadcasting master performs preliminary actions by collecting and merging memory requests from multiple processing elements before broadcasting to memory units. This preliminary consolidation reduces the number of simultaneous requests on the network, thereby reducing collisions, while the complexity is contained within the master's processing logic.
Data Source
AI summary
A system comprises a processor and a plurality of memory units. The processor is coupled to each of the plurality of memory units by a plurality of network connections. The processor includes a plurality of processing elements arranged in a two-dimensional array and a corresponding two-dimensional communication network communicatively connecting each of the plurality of processing elements to other processing elements on same axes of the two-dimensional array. Each processing element that is located along a diagonal of the two-dimensional array is configured as a request broadcasting master for a respective group of processing elements located along a same axis of the two-dimensional array.