Decoupled Load-Store Architecture for Memory Latency Mismatch
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional processing systems experience mismatches in processing and data transfer speeds, leading to reduced overall system performance and inefficiencies in handling complex datasets, particularly in processors like the Pixel Processing Engine (PPE) where there is a significant mismatch between SIMD vector compute and Data Memory.
Innovation Solution
Implementing a decoupled load store unit (DLSU) that prefetches data from memory to a stream buffer, decoupling memory accesses from the processor pipeline to minimize stalls and ensure efficient data transfer between a processor and memory devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is transferred directly between processor and memory device, then data transfer speed matches processor speed, but processor stalls occur due to memory latency
Solution Approach 1:
The stream processor prefetches data from the memory device into its internal stream buffer before the processor needs it, performing the data transfer operation in advance. This preliminary action ensures data is ready when the processor requires it, eliminating stalls caused by memory latency while maintaining matched data transfer speeds.
Solution Approach 2:
The stream processor acts as an intermediary component between the memory device and the processor. It receives data from memory, buffers it internally, and provides it to the processor at the required speed, mediating the speed mismatch and preventing processor stalls without requiring direct processor-memory synchronization.
2Productivity
If stream processor prefetches data into stream buffer, then processor stalls are eliminated, but device complexity increases
Solution Approach 1:
The memory architecture is segmented into distinct functional components: the stream processor for data prefetching and buffering, and the processor for computation. This segmentation allows the stream processor to handle data transfer operations independently, improving productivity while containing complexity within dedicated modules rather than the entire system.
Solution Approach 2:
The stream processor serves multiple functions: it acts as a data prefetcher, a buffer, and a data provider to the processor. By consolidating these functions into a single multi-functional component, the architecture achieves high productivity without proportionally increasing overall device complexity, as one component performs multiple critical roles.
Data Source
AI summary
Aspects of this technical solution can provide at least a technical improvement to reading and writing data between a memory device and a processor, including, for example, by providing a technical solution to configure one or more load streams with stream sizes configured based on relative speed of a processor and a memory. For example, this technical solution can provide a technical improvement to processing speed of computations by a processor with data obtained from or stored to a memory device. For example, a system in accordance with this technical solution can provide a decoupled load store unit (DLSU) distinct from a processor and a memory device to prefetch a sufficient amount of data from a memory device into a stream buffer of a DLSU, to provide instructions to a processor at a rate that eliminates waiting by the processor for memory over one or more cycles.