Accelerator Preemption Module Context Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI hardware accelerators face challenges in efficiently managing task preemption and resource allocation, leading to suboptimal performance and latency in executing AI tasks, particularly in scenarios requiring high-priority tasks and deep neural network operations.
Innovation Solution
The method involves a preemption module that moves context information of a first task from an internal memory to an external memory and executes a second task associated with the preemption request, while determining whether to execute target instructions based on movement time and expected execution time, ensuring resource conflict-free execution and high utilization rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If context information is moved from internal memory to external memory during task preemption, then task switching flexibility is improved, but memory transfer time increases system latency
Solution Approach 1:
The system determines whether to execute target instructions during the context transfer process by comparing movement time with expected execution time. This preliminary assessment allows the system to proactively decide on instruction execution timing, optimizing the balance between task switching flexibility and time efficiency.
Solution Approach 2:
The system dynamically adjusts task execution based on real-time conditions. By evaluating movement time and expected execution time, the system can flexibly determine whether to execute target instructions during context transfer or after completion, adapting to varying workload conditions and priority levels.
2Speed
If high-priority tasks preempt low-priority tasks, then system responsiveness is improved, but resource conflicts may occur
Solution Approach 1:
The system uses feedback from resource usage information to determine the start point of second task execution. By monitoring resource availability and adjusting execution timing accordingly, the system ensures that high-priority task preemption does not lead to resource conflicts, maintaining both responsiveness and reliability.
Solution Approach 2:
The system determines the start point of second task execution in advance based on resource usage information before actual execution begins. This preliminary planning prevents resource conflicts by ensuring that resource allocation is coordinated between preempting and preempted tasks.
3Quantity of substance
If context information is transferred to external memory, then internal memory resources are freed for other tasks, but data access speed decreases
Solution Approach 1:
The system selectively transfers only the necessary context information to external memory, maintaining optimal internal memory utilization. By determining the precise amount of context information needed and transferring only that portion, the system balances memory resource availability with data access speed requirements.
Data Source
AI summary
A method of operating the accelerator includes receiving a request for preemption during an execution of a first task using one or more processing elements included in the accelerator, in response to the request for preemption, moving context information of the first task stored in an internal memory of the accelerator to an external memory of the accelerator, and executing a second task associated with the request for preemption using the processing elements.