Broadcast Read Response Protocol for Memory Subsystems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In computing systems, the shared memory subsystem's finite throughput leads to inefficiencies in memory transactions, particularly when multiple processing engines need to access the same data, resulting in increased power consumption and latency.
Innovation Solution
Implementing a broadcast read response protocol where a host processing unit designates a primary processing engine to fetch data from memory and broadcast it to other engines, reducing the number of memory transactions by converting traditional read instructions into read and broadcast response commands, thereby optimizing data access and processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple processing engines independently fetch data from memory, then each engine can access required data, but the number of memory transactions increases leading to higher power consumption and latency
Solution Approach 1:
Multiple independent memory transactions from multiple processing engines are merged into a single memory transaction. The hub consolidates read requests from multiple engines into one broadcast read command, reducing the total number of transactions to memory while ensuring all engines receive their required data.
Solution Approach 2:
The hub acts as an intermediary between processing engines and memory subsystem. It receives read requests from multiple engines, consolidates them, and manages the single memory transaction, thereby reducing power consumption while maintaining data access reliability for all engines.
2Reliability
If multiple processing engines independently fetch data from memory, then each engine can access required data, but the number of memory transactions increases leading to increased latency
Solution Approach 1:
Multiple independent memory transactions are merged into a single memory transaction through the hub, which consolidates read requests from multiple processing engines. This reduces the total time required for data access by eliminating redundant transactions and their associated latencies.
Solution Approach 2:
The hub performs preliminary consolidation of read requests from multiple processing engines before issuing a single memory transaction. This preliminary action ensures that all required data is fetched in one operation, reducing overall latency while maintaining data access reliability.
3Reliability
If traditional read instructions are used by multiple processing engines, then data can be accessed, but the finite throughput of the memory subsystem is exceeded reducing efficiency
Solution Approach 1:
The hub merges multiple read requests into a single memory transaction, ensuring that the finite throughput of the memory subsystem is not exceeded. This consolidation maintains data access reliability while improving memory throughput efficiency by reducing transaction overhead.
Solution Approach 2:
The hub provides multi-functional capability by consolidating read requests from multiple processing engines into a single memory transaction. This universal approach serves multiple engines simultaneously, maintaining data access reliability while optimizing memory subsystem throughput efficiency.
Data Source
AI summary
Systems, apparatuses, and methods for implementing a broadcast read response protocol are disclosed. A computing system includes a plurality of processing engines coupled to a memory subsystem. A first processing engine executes a read and broadcast response command, wherein the read and broadcast response command targets first data at a first address in the memory subsystem. One or more other processing engines execute a wait command to wait to receive the first data requested by the first processing engine. After receiving the first data from the memory subsystem, the plurality of processing engines process the first data as part of completing a first operation. In one implementation, the first operation is implementing a given layer of a machine learning model. In one implementation, the given layer is a convolutional layer of a neural network.