Data Streaming Unit Bypasses Cache for NUMA Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional wireless baseband systems face inefficiencies due to non-uniform memory access architecture (NUMA) causing cache misses and wasting DSP resources, as they struggle with strided memory access patterns that exceed cache capabilities, leading to inefficient data processing and resource utilization.
Innovation Solution
A data streaming unit (DSU) is introduced to bypass the cache, offloading address calculation and data reorganization from the compute engine to the DSU, which manages address data and reorganizes data into vector form, thereby accelerating data access and reducing NUMA architecture-related stalls.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional NUMA architecture with cache is used for data access, then data storage capacity is maintained, but cache misses occur and DSP resources are wasted due to strided memory access patterns
Solution Approach 1:
The patent extracts the address calculation and data reorganization functions from the DSP and creates a separate data streaming unit (DSU) to handle these tasks. This extraction allows the DSP to focus solely on signal processing while the DSU manages memory access patterns, eliminating the resource waste caused by the DSP attempting to handle both computation and memory management.
Solution Approach 2:
The data streaming unit acts as an intermediary between the DSP and the memory system. It mediates the strided memory access patterns by generating appropriate address sequences and reorganizing data in vector format before presenting it to the DSP, thereby eliminating cache misses without burdening the DSP with memory management overhead.
2Speed
If cache is used for data access in NUMA architecture, then data access speed is improved for sequential patterns, but cache conflicts occur with strided access patterns exceeding cache capabilities
Solution Approach 1:
The data streaming unit performs preliminary actions by pre-calculating address sequences and pre-reorganizing data into vector format before the DSP needs the data. This preliminary processing ensures that when data is accessed from memory, it is already in the correct format and location, eliminating cache conflicts and ensuring high hit rates.
Solution Approach 2:
The patent changes the parameter of data organization from scalar to vector format. By reorganizing data in vector format with proper alignment and contiguity before it reaches the cache, the system transforms problematic strided access patterns into efficient sequential access patterns that the cache can handle effectively.
3Device complexity
If DSP handles both computation and memory management, then system complexity is reduced, but processing latency increases due to resource contention
Solution Approach 1:
The patent segments the system into distinct functional units: the DSP for computation and the data streaming unit for memory management. This segmentation separates the computation task from the memory management task, allowing both to operate in parallel without resource contention, thereby reducing processing latency despite the increased architectural complexity.
4Productivity
If data is reorganized in vector format within the compute engine, then processing efficiency is improved, but power consumption increases due to additional processing overhead
Solution Approach 1:
The patent extracts the vector format reorganization function from the compute engine and places it in the data streaming unit. This extraction allows the compute engine to receive data already in the optimal vector format, improving processing efficiency while reducing the power consumption of the compute engine by eliminating the redundant reorganization operation.
Data Source
AI summary
A data streaming unit (DSU) and a method for operating a DSU are disclosed. In an embodiment the DSU includes a memory interface configured to be connected to a storage unit, a compute engine interface configured to be connected to a compute engine (CE) and an address generator configured to manage address data representing address locations in the storage unit. The data streaming unit further includes a data organization unit configured to access data in the storage unit and to reorganize the data to be forwarded to the compute engine, wherein the memory interface is communicatively connected to the address generator and the data organization unit, wherein the address generator is communicatively connected to the data organization unit, and wherein the data organization unit is communicatively connected to the compute engine interface.