Processor Streaming Unit for Efficient Data Stream Handling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current processors lack an efficient mechanism for handling streaming data, which involves processing data streams from memory with regular intervals, as existing load/store units are not optimized for concurrent and transformative operations on non-contiguous data blocks, leading to suboptimal cache usage and increased workload on execution units.
Innovation Solution
A processor with a dedicated streaming unit that includes an internal network of queues and operations circuits, allowing for software-programmable coupling of operations circuits to perform operations on multiple data streams, thereby enabling efficient loading, transformation, and output of data streams independently of the cache subsystem and execution units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a load/store unit is used to handle streaming data, then data can be fetched from memory, but the processing efficiency is reduced due to lack of optimization for concurrent operations on non-contiguous data blocks
Solution Approach 1:
The streaming unit is divided into multiple independent operation circuits (e.g., add circuit, subtract circuit, multiply circuit, divide circuit, logical operation circuit) that can concurrently process different data streams. Each circuit handles specific operations independently, allowing parallel processing of multiple streaming data blocks without interfering with the cache subsystem.
Solution Approach 2:
The streaming unit acts as an intermediary component between the cache subsystem and execution units. It receives streaming data from the cache subsystem, performs preprocessing operations (addition, subtraction, multiplication, division, logical operations) on non-contiguous data blocks, and outputs processed data to execution units or memory, thereby reducing the workload on both the cache subsystem and execution units.
2Productivity
If execution units process all streaming data operations, then data processing can be performed, but the workload on execution units increases and performance decreases
Solution Approach 1:
The processing workload is segmented between the streaming unit and execution units. The streaming unit handles specific streaming data operations (concurrent addition, subtraction, multiplication, division, logical operations on multiple data streams), while execution units focus on other computational tasks. This segmentation reduces execution unit workload and improves overall system throughput.
Solution Approach 2:
The streaming unit performs self-service by autonomously processing streaming data operations without requiring execution unit intervention. It can independently execute operations on streaming data blocks, manage its own operation circuits, and handle data flow between memory and execution units, thereby reducing the burden on execution units.
3Productivity
If conventional load/store units handle streaming data, then data can be loaded from memory, but cache pollution increases due to non-contiguous data block access
Solution Approach 1:
The streaming unit serves as an intermediary that handles non-contiguous data block access patterns. It receives memory addresses with stride values from the load/store unit, calculates the actual memory addresses for non-contiguous blocks, and performs data loading operations. This mediation prevents random non-contiguous accesses from directly polluting the cache subsystem, as the streaming unit manages the access pattern optimization.
Solution Approach 2:
The streaming unit utilizes stride values as parameters to change the data access pattern from random non-contiguous addresses to a systematic access pattern. By calculating addresses based on base addresses and stride values, it transforms the access pattern in a way that reduces cache pollution while maintaining the ability to handle non-contiguous data blocks efficiently.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A processor having a streaming unit is disclosed. In one embodiment, a processor includes a streaming unit configured to load one or more input data streams from a memory coupled to the processor, to the execution units of the processor. The streaming unit includes an internal network having a plurality of queues configured to store streams of data. The streaming unit further includes a plurality of operations circuits configured to perform operations on the streams of data. The streaming unit is configurable in response to an instruction from the processor to operatively couple two or more of the plurality of operations circuits together via one or more of the plurality of queues. The operations circuits may perform operations on multiple streams of data, resulting in corresponding output streams of data.