On-Chip Cache for Low Latency Task Metadata Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high latency associated with loading tasks from memory in processor systems leads to inefficiencies in task scheduling and launch, as memory read requests cause processor stalls and delays.
Innovation Solution
A hardware scheduling mechanism with a local cache for task metadata data structures allows for low-latency scheduling and launch by storing recently written tasks in on-chip memory, reducing the need for high-latency memory access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If tasks are stored in off-chip memory and loaded on demand, then memory capacity is sufficient, but processor stalls occur due to high latency memory read requests
Solution Approach 1:
The memory system is segmented into two parts: off-chip memory for bulk storage and on-chip cache memory for frequently accessed task metadata. This segmentation allows the system to maintain large storage capacity while providing fast access to critical task data, resolving the contradiction between memory capacity and access speed.
Solution Approach 2:
Task metadata is pre-loaded into on-chip cache memory before the processor needs to execute tasks. The scheduling unit proactively loads task metadata from off-chip memory into the cache, so that when task launch is required, the data is already available in fast on-chip memory, eliminating processor stalls.
2Speed
If task metadata is kept in on-chip cache memory, then access speed increases, but cache memory capacity is limited
Solution Approach 1:
The on-chip cache memory is designed to handle multiple types of data structures including task metadata, task state, and other scheduling information. This multi-functional cache design maximizes the utilization of limited cache capacity while maintaining fast access speeds for various processor operations.
Solution Approach 2:
The cache memory implements a replacement policy where less frequently accessed task metadata is evicted from the cache to make room for newly required task metadata. The scheduling unit manages cache contents dynamically, discarding old entries and recovering cache space for active tasks, thus maintaining high access speed within limited capacity.
3Reliability
If task metadata is copied from cache to memory, then data consistency is maintained, but additional memory operations increase latency
Solution Approach 1:
Instead of maintaining a single copy of task metadata that must be updated synchronously across memory and cache, the system creates a copy of task metadata in the on-chip cache when needed for scheduling. This copying approach allows the cache to have a local copy for fast access while the original remains in off-chip memory, maintaining consistency without requiring continuous synchronization overhead.
Data Source
AI summary
A system, method, and computer program product for low-latency scheduling and launch of memory defined tasks. The method includes the steps of receiving a task metadata data structure to be stored in a memory associated with a processor, transmitting the task metadata data structure to a scheduling unit of the processor, storing the task metadata data structure in a cache unit included in the scheduling unit, and copying the task metadata data structure from the cache unit to the memory.