AI Processor Task Scheduling Through Prefetch and Real Task Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional multi-threading technologies in central processing units (CPUs) and graphics processing units (GPUs) for artificial intelligence increase hardware complexity and thread switching overheads, leading to unstable performance benefits.
Innovation Solution
A dual-threaded architecture is implemented by splitting tasks into prefetch and real tasks, where the prefetch task is executed during the execution of the real task, allowing for parallel execution and reducing thread switching overheads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multi-threading technology is used to improve parallel processing performance, then parallelism between threads is enhanced, but hardware complexity and thread switching overheads increase
Solution Approach 1:
The patent divides a traditional multi-threaded architecture into two independent single-threaded processing units: a control thread unit and a user thread unit. Each unit has its own program counter and instruction queue, eliminating the need for complex thread switching mechanisms while maintaining parallel execution capability. This segmentation resolves the contradiction by achieving parallelism through architectural separation rather than through multiple threads sharing resources.
2Productivity
If the number of threads is increased to enhance parallelism, then processing capacity improves, but control logic complexity and thread switching overheads increase
Solution Approach 1:
The patent segments the processing architecture into two independent single-threaded units that execute in parallel. Each unit maintains its own program counter and instruction queue, eliminating the need for complex thread management logic. This approach achieves improved processing capacity through architectural parallelism rather than through increasing thread count, thereby avoiding control logic complexity.
3Productivity
If multi-threading is implemented to achieve higher level parallelism, then task execution efficiency improves, but thread switching overheads increase
Solution Approach 1:
The patent divides the processing system into two independent single-threaded units that operate simultaneously. The control thread unit handles control flow while the user thread unit executes user programs, with each maintaining its own program counter and instruction queue. This segmentation eliminates thread switching overheads by using architectural parallelism instead of thread-based parallelism, while still achieving high task execution efficiency.
Data Source
AI summary
A task scheduler is arranged in an artificial intelligence processor. The artificial intelligence processor includes an executing circuit configured to execute a task. The task scheduler includes a first sending circuit configured to send a prefetch task of a next task to the executing circuit during an execution of a real task of a current task by the executing circuit, wherein a task in the task scheduler is split into a prefetch task and a real task that are interrelated, and a second sending circuit configured to send a real task of the next task to the executing circuit after the executing circuit has completed an execution of the prefetch task of the next task, so that the executing circuit executes the real task of the next task after the executing circuit has completed the execution of the real task of the current task.