SIMD Processor Core Shared Data Broadcast Architecture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing systems with SIMD architectures face performance degradation due to high memory bandwidth limitations when executing tasks with high data reuse, leading to multiple unnecessary memory accesses during General Matrix to Matrix Multiplication (GEMM) operations.
Innovation Solution
Implementing a shared data broadcast protocol within the SIMD processor core, where a single access request is generated when multiple compute units request the same data, reducing memory fetches by broadcasting the retrieved data to all requesting units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If SIMD architecture executes data parallel workloads, then computing performance and cost efficiency are improved, but memory bandwidth limitations cause performance degradation when high data reuse requires multiple memory accesses
Solution Approach 1:
Multiple compute units that request the same data are merged into a single memory access operation. The memory subsystem receives one consolidated request instead of multiple separate requests, reducing memory bandwidth consumption while maintaining the computational performance of all participating compute units.
Solution Approach 2:
The memory access request mechanism is enhanced to serve multiple functions: it simultaneously handles individual compute unit requests and aggregated broadcast requests. A single memory access can serve multiple compute units through data broadcasting, making the memory subsystem more versatile and efficient.
2Reliability
If multiple compute units access the same data independently, then each unit gets its data, but the number of memory accesses increases unnecessarily
Solution Approach 1:
Independent memory access requests from multiple compute units targeting the same data are combined into a single request. This ensures all compute units receive their required data while reducing the total number of memory accesses, thereby improving memory access efficiency without compromising data delivery reliability.
Solution Approach 2:
The retrieved data is copied and broadcast to multiple compute units simultaneously. Instead of fetching the same data multiple times, the system fetches it once and creates copies for each requesting compute unit, maintaining data delivery reliability while improving efficiency.
3Ease of operation
If data is fetched multiple times for the same data item, then each compute unit has its data, but memory subsystem load and latency increase
Solution Approach 1:
Multiple data fetch operations for the same data item are merged into a single fetch operation. This reduces memory subsystem load and eliminates redundant wait times, thereby reducing memory latency while ensuring data is made available to all requiring compute units through broadcasting.
Solution Approach 2:
The system performs preliminary aggregation of memory access requests before executing the actual memory fetch. By collecting and consolidating requests in advance, the system prepares a single optimized memory access operation that serves multiple compute units, reducing overall latency and memory subsystem load.
Data Source
AI summary
Systems, apparatuses, and methods for efficient parallel execution of multiple work units in a processor by reducing a number of memory accesses are disclosed. A computing system includes a processor core with a parallel data architecture. The processor core executes a software application with matrix operations. The processor core supports the broadcast of shared data to multiple compute units of the processor core. A compiler or other code assigns thread groups to compute units based on detecting shared data among the compute units. Rather than send multiple read accesses to a memory subsystem for the shared data, the processor core generates a single access request. The single access request includes information to identify the multiple compute units for receiving the shared data when broadcasted by the processor core.