DSP Streaming Engine Architecture for Memory Latency Hiding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern digital signal processors face challenges such as increasing workloads, memory system latency, and limited bandwidth, which are exacerbated by the complexity of systems on a chip and the in-order nature of central processing units, leading to subpar performance and scalability issues.
Innovation Solution
The integration of a streaming engine with vector support, scoreboarded loads, speculative loads, and software-directed prefetch, along with a credit-based bidirectional bus protocol, enables efficient data transfer and coherence maintenance, allowing for improved bus utilization and memory system latency hiding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the bus width and clock rate are increased to increase bandwidth, then bandwidth is improved, but wire count and routing complexity increase
Solution Approach 1:
The system segments the memory interface into multiple independent channels (e.g., 4 channels of 64-bit width instead of one 256-bit channel), allowing bandwidth scaling without proportionally increasing routing complexity. Each channel can be routed independently through the interconnect fabric, distributing the bandwidth demand across multiple manageable paths.
Solution Approach 2:
The patent introduces a time dimension to bandwidth by implementing pipelined memory operations and asynchronous transfer protocols. Multiple transfers can occur in parallel across different channels and memory banks, effectively increasing throughput without requiring a single ultra-wide bus that would create routing nightmares.
2Loss of time
If deeper buffering is implemented to hide memory latency, then memory latency is reduced, but device complexity increases
Solution Approach 1:
The system implements hardware prefetchers that proactively load data into buffers before it is actually needed by the processor. Multiple prefetch queues maintain data ahead of time, hiding memory latency without requiring excessively deep buffers. The prefetcher uses address pattern recognition to predict future memory accesses and pre-fetch relevant data.
Solution Approach 2:
The patent introduces an intelligent memory management unit (MMU) and cache controller that act as intermediaries between the processor and memory system. These components manage buffer contents, perform address translation, and coordinate data movement, reducing the burden on the processor while effectively hiding memory latency through sophisticated buffer management.
3Reliability
If separate busses are used for demand traffic, DMA traffic and snoop traffic, then traffic isolation is improved, but bus utilization decreases
Solution Approach 1:
The system merges multiple traffic types (demand, DMA, snoop) onto shared memory channels with intelligent arbitration. Instead of dedicated busses for each traffic type, the patent implements a credit-based arbitration scheme that allows different traffic types to share the same physical channels while maintaining isolation through virtual channels and priority schemes, thereby improving bus utilization without sacrificing traffic isolation.
Solution Approach 2:
The memory interconnect implements dynamic resource allocation where channel bandwidth is allocated based on current traffic demands and priorities. The arbitration logic adapts in real-time to changing workload conditions, dynamically adjusting which traffic types receive access to which channels, thus optimizing overall bus utilization while maintaining necessary traffic isolation when required.
Data Source
AI summary
Disclosed embodiments include a data processing apparatus having a processing core, a memory, and a streaming engine. The streaming engine is configured to receive a plurality of data elements stored in the memory and to provide the plurality of data elements as a data stream to the processing core, and includes an address generator to generate addresses corresponding to locations in the memory, a buffer to store the data elements received from the locations in the memory corresponding to the generated addresses, and an output to supply the data elements received from the memory to the processing core as the data stream.