Work Scheduler Offloading Linked Work Assignments
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current workflow management and scheduling in computer applications, particularly in CMP architectures, face challenges due to high overhead and latency in C2C communication, which limits performance scaling and accelerator integration, mainly because of the need for complex software management of shared memory and hardware coherency traffic.
Innovation Solution
The introduction of a work scheduler that enables accelerator chaining by constructing arrays of work items, offloading load balancing and flow-specific communication, using a universal API to abstract accelerator locality, and providing quality of service (QoS) across cores and hardware devices, allowing for efficient data processing without CPU involvement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If shared memory space is used for C2C communication in CMP architectures, then efficient communication is achieved, but software management complexity and hardware coherency traffic increase significantly
Solution Approach 1:
The patent introduces a work scheduler as an intermediary component that mediates between CPU cores and hardware accelerators. The work scheduler manages workload distribution, data transfer coordination, and accelerator invocation without requiring complex software management of shared memory or handling hardware coherency traffic directly. This mediator approach resolves the contradiction by maintaining communication efficiency while significantly reducing software management complexity.
2Ease of operation
If hardware coherency traffic is increased to support data sharing, then data access is improved, but effective latency and bandwidth utilization increase
Solution Approach 1:
The patent extracts hardware coherency traffic management from the CPU core responsibilities and relocates it to the work scheduler and hardware accelerators. By taking out coherency traffic handling from the CPU path, the system eliminates the bottleneck that caused increased effective latency while maintaining data sharing capability through direct accelerator-to-accelerator communication.
Solution Approach 2:
Hardware accelerators are designed to perform self-service operations including autonomous workload execution, self-managed data transfer, and self-coherency maintenance. This eliminates the need for CPU-mediated data sharing and coherency management, thereby reducing effective latency while preserving data sharing functionality through direct hardware-to-hardware communication.
3Adaptability or versatility
If CPU cores are used for workflow management, then flexibility is maintained, but performance scaling and accelerator integration are limited
Solution Approach 1:
The patent segments workflow management responsibilities by separating CPU core functions from accelerator functions. CPU cores handle high-level workflow orchestration and data generation, while the work scheduler handles workload distribution and data transfer coordination, and hardware accelerators handle execution and self-coherency. This segmentation enables performance scaling by allowing accelerators to operate independently without CPU bottlenecking.
Solution Approach 2:
The work scheduler serves as an intermediary layer between CPU cores and hardware accelerators, enabling flexible workflow management while supporting performance scaling. It provides a simplified interface that abstracts away the complexity of direct CPU-accelerator communication, allowing multiple accelerators to be integrated and scaled without increasing CPU burden.
4Productivity
If direct accelerator-to-accelerator communication is implemented, then overhead is reduced, but complex coordination and data transfer management are required
Solution Approach 1:
Hardware accelerators implement self-service capabilities including autonomous workload execution, self-managed data transfer via DMA, and self-coherency maintenance without requiring complex CPU-mediated coordination. This self-service approach reduces overhead and improves processing efficiency while the work scheduler provides high-level coordination only when needed, significantly reducing overall system complexity.
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
Examples are described herein that can be used to offload a sequence of work events to one or more accelerators to a work scheduler. An application can issue a universal work descriptor to a work scheduler. The universal work descriptor can specify a policy for scheduling and execution of one or more work events. The universal work descriptor can refer to one or more work events for execution. The work scheduler can, in some cases, perform translation of the universal work descriptor or a work event descriptor for compatibility and execution by an accelerator. The application can receive notice of completion of the sequence of work from the work scheduler or an accelerator.