Spare Scheduler Extends Hardware Accelerator Local Buffer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Hardware accelerator (HWA) sub-systems face challenges in managing variable data sizes and memory requirements, leading to potential over-engineering and performance degradation when handling worst-case scenarios, particularly in scenarios requiring larger memory capacities than the local memory can provide.
Innovation Solution
The implementation of a hardware thread scheduler (HTS) with spare schedulers allows for dynamic management of data movement between local and on-chip or external memory, effectively extending the local memory capacity and converting between data line and block formats as needed, thereby supporting variable-sized data transfers without decreasing system performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the local memory capacity is increased to handle worst-case scenarios, then the memory capacity is improved, but the device complexity and cost increase due to over-engineering
Solution Approach 1:
The memory system is segmented into local memory (fast, small capacity) and external memory (slower, large capacity). The spare scheduler dynamically segments data transfer operations, moving only necessary data between memory tiers rather than requiring all memory to be local.
Solution Approach 2:
The spare scheduler acts as an intermediary component that manages data movement between local and external memory. It mediates memory access requests, determining when to use local memory and when to spill data to external memory, eliminating the need for oversized local memory.
2Adaptability or versatility
If the local memory capacity is increased to handle variable data sizes, then the adaptability is improved, but the device complexity increases due to over-engineering for worst-case scenarios
Solution Approach 1:
The system dynamically adjusts memory usage based on actual data size requirements. The spare scheduler monitors data flow and dynamically spills data to external memory when local memory is full, or pre-loads data when available, rather than statically allocating fixed memory capacity for worst-case scenarios.
Solution Approach 2:
The spare scheduler provides universal memory management functionality, handling both local memory operations and external memory spilling through a unified interface. This multi-functional component adapts to varying data sizes without requiring different hardware configurations.
3Ease of operation
If data movement management is simplified, then the ease of operation is improved, but the productivity decreases due to performance degradation
Solution Approach 1:
The spare scheduler implements self-service mechanisms by automatically monitoring local memory status and initiating data spilling to external memory when needed. It autonomously manages the complexity of coordinated data movement between memory tiers without requiring external intervention, maintaining both ease of operation and productivity.
Solution Approach 2:
The system incorporates feedback mechanisms where the spare scheduler continuously monitors local memory utilization and adjusts data movement operations accordingly. When local memory approaches capacity, the scheduler receives feedback and automatically spills data to external memory, maintaining system performance without manual intervention.
Data Source
AI summary
Methods and apparatus to extend local buffer of a hardware accelerator are disclosed herein. In some examples, an apparatus, including a local memory, a first hardware accelerator (HWA), a second HWA, the second HWA and the first HWA connected in a flexible data pipeline, and a spare scheduler to manage, in response to the spare scheduler inserted in the flexible data pipeline, data movement between the first HWA and the second HWA through the local memory and a memory. Local buffer extension may be performed by software to control data movement between local memory and other system memory. The other system memory may be on-chip memory and/or external memory. The HWA sub-system includes a set of spare schedulers to manage the data movement. Data aggregation may be performed in the other system memory. Additionally, the other system memory may be utilized for conversion between data line and data block.