Task Scheduling Layer for Fair GPU and NPU Resource Sharing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Efficient and convenient scheduling of heterogeneous hardware resources in cloud servers, particularly GPUs and NPUs, is challenging due to the need for seamless sharing across multiple containers, especially when dealing with black-box GPU device drivers that are not open source.

Innovation Solution

A scheduling method that acquires task information, determines actual execution time, and adjusts actual scheduling time based on comparison with preset scheduling time to optimize task allocation, ensuring fair and efficient utilization of hardware resources without relying on device drivers.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If traditional device driver-based scheduling is used, then hardware resource scheduling can be implemented, but it cannot support non-open source device drivers and lacks flexibility

Engineering Contradiction:

Improvecompatibility with different device driversVSAvoidscheduling system complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent introduces an intermediate scheduling layer that sits between the task manager and device drivers. This intermediary scheduling system manages hardware resources without requiring direct access to or modification of device driver code, enabling it to work with both open-source and non-open-source drivers while maintaining scheduling flexibility

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent extracts the scheduling functionality from the device driver layer and places it in a separate, independent scheduling system. This extraction allows the scheduling logic to be decoupled from driver-specific implementations, enabling universal application across different hardware platforms and driver types without modifying the drivers themselves

Inventive Principle:

Principle #2Taking out (Extraction)

2Productivity

If hardware resources are scheduled without time adjustment, then scheduling is simple, but resource allocation becomes unfair and inefficient

Engineering Contradiction:

Improvehardware resource utilization efficiencyVSAvoidscheduling time adjustment overhead

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent implements a feedback mechanism where the actual execution time of tasks is measured and fed back into the scheduling system. Based on this feedback, the system dynamically adjusts scheduling times to compensate for variations in task execution duration, ensuring fairer resource allocation and improved overall productivity

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The scheduling system transitions from static, fixed-time scheduling to dynamic scheduling that adapts to actual task execution characteristics. The system continuously adjusts scheduling parameters based on real-time performance data, allowing it to optimize resource allocation in response to varying workloads and execution times

Inventive Principle:

Principle #15Dynamics

3Ease of operation

If task scheduling does not consider actual execution time, then scheduling is fast, but scheduling time is not balanced among tasks

Engineering Contradiction:

Improvescheduling operation simplicityVSAvoidscheduling time balance

Core Design Contradiction:

Ease of operationVSStability of the object's composition

Solution Approach 1:

The system performs preliminary measurement of actual execution times for tasks before final scheduling decisions are made. This preliminary data collection enables the scheduler to make more informed decisions about time allocation, ensuring balanced scheduling while maintaining operational simplicity through automated time adjustment

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentEP4113298B1Task scheduling method, computing device and storage medium

Publication Date: 2026.04.01 CLOUD INTELLIGENCE ASSETS HOLDING (SINGAPORE) PTE LTD

Figure 1~2
Figure 3
Figure 4~6

AI summary

Provided are a scheduling method for tasks, a computing device, and a storage medium. The method comprises acquiring task information of a current task to be executed, determining an actual execution time for the current task to be executed according to the task information, comparing the actual execution time with a preset scheduling time corresponding to the current task to be executed, and adjusting an actual scheduling time corresponding to a next task to be executed according to the comparison result so as to determine whether to schedule the next task to be executed. By comparing the actual execution time with the preset scheduling time after determining the actual execution time, the actual scheduling time corresponding to the next task to be executed can be adjusted so as to decide whether to perform task scheduling, thus achieving quick and easy task scheduling and full utilization of hardware resources. Moreover, since the actual scheduling time is adjusted, the actual scheduling time for each task can be allocated accurately and efficiently, and the scheduling time can be balanced among the tasks.