Heterogeneous GPU resource scheduling method and system based on combination of meta learning and reinforcement learning

Abstract

The invention relates to a heterogeneous GPU (Graphics Processing Unit) resource scheduling method and system based on the combination of meta learning and reinforcement learning, and aims to improve the utilization efficiency of computing resources in large model training and reasoning and improve the adaptive capacity of tasks and the response speed of the system. According to the method, efficient scheduling of GPU resources is achieved through the five steps of task feature extraction, meta-learning strategy initialization, reinforcement learning scheduling optimization, task scheduling execution and dynamic strategy adjustment. Wherein the meta-learning module generates an initial scheduling strategy by utilizing knowledge accumulated by historical tasks, so that the exploration time is shortened; and the reinforcement learning module continuously optimizes a scheduling strategy through environment feedback to realize efficient distribution of tasks. The system can adaptively adjust strategies according to different task requirements and dynamic changes of GPU hardware resources, and therefore higher resource utilization rate and overall performance optimization are achieved.

Description

Technical Field

[0001] The present invention relates to the field of resource allocation, and in particular to a heterogeneous GPU resource scheduling method and system based on the combination of meta-learning and reinforcement learning. Background Art

[0002] With the widespread application of large-scale model training, inference, and large-scale data processing, GPU, as the core hardware for accelerated computing, has become an important resource for computing-intensive tasks. Since different models of GPUs have different computing capabilities, memory capacity, and power consumption characteristics, how to efficiently schedule heterogeneous GPU resources has become an important issue for improving system performance and optimizing resource utilization. In cloud computing and data center environments, facing multi-task requirements from multiple users, reasonable scheduling of GPU resources is of great significance for reducing costs and improving throughput. In these scenarios, the dynamic nature of tasks and the heterogeneity of GPUs increase the complexity of scheduling. For example, different tasks have different requirements for computing resources. Some tasks require higher computing power, while others may have higher requirements for memory and bandwidth. At the same time, the diversity of GPU hardware also requires scheduling strategies to have a certain degree of flexibility in order to give full play to the advantages of various types of hardware.

[0003] At present, heterogeneous GPU resource scheduling is mainly implemented in the following ways: Scheduling based on heuristic rules: This method relies on preset scheduling rules, such as selecting a specific GPU according to the task type, or allocating resources according to the first-come, first-served principle. These methods are easy to implement, but they cannot guarantee optimal performance for complex and dynamically changing task requirements. Scheduling based on static optimization models: Some methods solve the GPU scheduling problem by establishing static optimization models, such as finding the optimal allocation strategy through linear programming or genetic algorithms. However, static optimization models lack flexibility and real-time performance when facing changing task requirements and resource conditions. Scheduling based on reinforcement learning: Reinforcement learning is applied to resource scheduling, using intelligent agents to learn the optimal scheduling strategy by interacting with the environment. Although reinforcement learning has improved scheduling performance to a certain extent, when faced with new tasks, the learning process still requires a long exploration time and it is difficult to quickly adapt to new task requirements.

[0004] However, there are some shortcomings in the existing technical solutions. Methods based on heuristic rules or static optimization models lack the ability to adapt to dynamic environments and new task types, and have poor adaptability. This makes it impossible to guarantee the performance of the system when facing changing task requirements and hardware conditions. When facing new task types, reinforcement learning methods need to learn from scratch, resulting in low learning efficiency, especially in heterogeneous environments, where the model is difficult to generalize effectively. This inefficient learning process will cause the system to have long delays when adapting to new tasks. Existing optimization methods are usually computationally complex and difficult to expand in large-scale environments, especially when faced with the diversity of heterogeneous GPUs, and cannot flexibly adjust scheduling strategies, complexity and computational overhead. These methods have high computational overhead and are difficult to apply in real time in actual systems. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention proposes a heterogeneous GPU resource scheduling method based on the combination of meta-learning and reinforcement learning. The resource scheduling method generates an initial scheduling strategy for task characteristics through meta-learning, and continuously optimizes the scheduling strategy through reinforcement learning combined with real-time environmental feedback. First, the feature vector of the input task is extracted for the meta-learning module to generate the initial scheduling strategy; then the scheduling strategy parameters are optimized through the reinforcement learning agent to improve the efficiency of task allocation; then the scheduling decision is executed to allocate the task to the most suitable GPU; finally, the strategy is dynamically adjusted according to the system feedback and GPU status to ensure that the system always maintains the best performance in a constantly changing environment, specifically including:

[0006] Step 1: Task feature extraction. The system extracts key features from the input task through the task feature extraction module and uses them as input to the meta-learning module and the reinforcement learning scheduling module.

[0007] Step 2: Meta-learning initialization strategy. The extracted task feature vector is input into the meta-learning module. The model is trained using the meta-learning algorithm to obtain a more appropriate initial scheduling strategy for the new task.

[0008] Step 3: Reinforcement learning optimization. The initial scheduling strategy initialized by meta-learning and the task feature representation are input into the reinforcement learning module. The reinforcement learning agent interacts with the environment through the strategy generated by the policy network. Based on the actual performance during the task execution, it obtains new feedback information. Based on the feedback information, the system dynamically adjusts the scheduling strategy through a reward mechanism.

[0009] Step 4: Task scheduling and execution. Taking the scheduling strategy optimized by reinforcement learning as input, the task scheduling execution module assigns tasks to the most suitable GPU. The task scheduling execution module assigns tasks to computing units in the GPU resource pool as needed according to the scheduling strategy.

[0010] The present invention proposes a heterogeneous GPU resource scheduling system based on the combination of meta-learning and reinforcement learning. The resource scheduling device includes a task representation module, a meta-learning module, a reinforcement learning scheduling module and a GPU resource management module, wherein:

[0011] The task representation module is used to extract features of the input tasks, including the task's computational requirements, priority, and estimated execution time, and represent the task features as vectors to be input into subsequent models.

[0012] The meta-learning module is used to learn historical tasks and extract features from historical task data. The meta-optimization process uses the meta-learning algorithm to train the model so that the reinforcement learning agent can adapt to new tasks more quickly. The strategy generator forms an initial scheduling strategy to help the reinforcement learning agent adapt to new tasks more quickly.

[0013] The reinforcement learning scheduling module is based on the classic agent-environment interaction paradigm. The reinforcement learning agent makes resource allocation decisions based on the initial strategy provided by meta-learning and the current task characteristics. The reinforcement learning scheduling module includes a policy network, an environment interaction module, and a reward calculation module. The policy network is responsible for generating resource allocation strategies based on the current task characteristics and environmental status. The environment interaction module interacts with the system environment to collect feedback information on task execution. The reward calculation module calculates the reward value of each scheduling decision to help the reinforcement learning model continuously optimize the decision strategy.

[0014] The GPU resource management module is used to manage and monitor the usage status of the GPU and apply scheduling decisions to specific hardware resources. Among them, the resource monitoring module is used to monitor the usage of the GPU in real time, including status information such as computing load and memory usage. The scheduling execution module is used to apply the decisions of the scheduling strategy to specific GPU hardware to realize task allocation and execution. The exception management module is used to monitor abnormal situations during task execution, such as GPU overload or failure, and take corresponding measures.

[0015] Compared with the prior art, the present invention has the following beneficial effects:

[0016] (1) Efficient task adaptability: Through meta-learning training, the system can quickly adapt to new task types, reducing its reliance on the exploration phase. This enables the scheduling system to respond to new tasks more quickly, improving the real-time performance of the system.

[0017] (2) Flexibility in dynamic environments: The system can adjust the scheduling strategy in real time according to the changes in different tasks and GPU hardware resources to optimize the overall performance. The system can adaptively adjust the strategy to cope with changes in task requirements and hardware resources, ensuring that the system is always in the best operating state.

[0018] (3) Improve resource utilization: Allocate GPU resources more reasonably to avoid idle or overloaded resources, thereby improving the system's throughput and computing efficiency. By optimizing the matching of tasks and GPUs, the present invention effectively improves the system's resource utilization efficiency and reduces computing costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a processing flow chart of the scheduling method of the present invention;

[0020] Figure 2 It is a processing flow chart of the scheduling system of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention. In addition, in the following description, the description of well-known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention.

[0022] The following is a detailed description with reference to the accompanying drawings.

[0023] The present invention proposes a heterogeneous GPU resource scheduling method and system based on the combination of meta-learning and reinforcement learning. Most of the existing GPU resource scheduling methods rely on static rules or optimization models, which are difficult to cope with dynamically changing task requirements and hardware resource differences, resulting in low resource utilization efficiency and poor task adaptability. In response to these problems, the present invention introduces a meta-learning module, utilizes the knowledge accumulation of historical tasks, generates an initial scheduling strategy, thereby reducing the exploration time on new tasks and improving task adaptability. At the same time, combined with the reinforcement learning module, the system can continuously optimize the scheduling strategy according to task execution feedback to improve the efficiency of resource allocation. In a multi-task, multi-GPU environment, the present invention can adjust the scheduling strategy in real time to ensure the best match between tasks and resources, maximize the utilization of computing resources, and ultimately improve the overall performance of the system.

[0024] The present invention aims to solve the problems of low resource utilization efficiency, poor task adaptability and difficulty in rapid generalization of scheduling strategies in the current heterogeneous GPU resource scheduling. Existing scheduling methods usually rely on static rules or optimization models based on specific scenarios, which cannot cope with dynamically changing task requirements and the diversity of GPU hardware. Therefore, the present invention proposes a method combining meta-learning and reinforcement learning to quickly adapt to different types of task requirements and optimize resource allocation, thereby improving the overall performance of the system and GPU resource utilization.

[0025] Specifically, the technical problems solved by the present invention include: first, how to achieve efficient task scheduling in a heterogeneous GPU environment and improve the system's computing resource utilization rate; second, how to enable the scheduling system to quickly adapt to changing task types and hardware conditions by introducing meta-learning; and third, how to reduce the exploration time in task scheduling and improve the system's response speed.

[0026] By combining meta-learning with reinforcement learning, the present invention can quickly adapt and find efficient scheduling strategies when faced with new task requirements and changing resource environments. The introduction of meta-learning allows the model to accumulate knowledge from historical tasks, shorten the learning time on new tasks, and greatly improve resource utilization and system response speed.

[0027] Figure 1 It is a processing flow chart of the scheduling method of the present invention. Figure 1 As shown, the method of the present invention mainly includes five steps: task feature extraction, meta-learning initialization, reinforcement learning optimization, task scheduling execution, and dynamic strategy adjustment.

[0028] Step 1: Task feature extraction, the system extracts key features from the input task through the task feature extraction module. Task features include information such as computing requirements, memory requirements, and data transmission bandwidth. The extracted features are represented in the form of feature vectors and used as input to the meta-learning module and reinforcement learning scheduling module.

[0029] Step 2: Meta-learning initialization strategy. The extracted task feature vector is input into the meta-learning module. The model is trained using a meta-learning algorithm (such as MAML) to obtain a more appropriate initial scheduling strategy for the new task, so that the reinforcement learning agent can adapt to the new task faster, thereby reducing exploration time and avoiding learning from scratch.

[0030] Step 3: Reinforcement learning optimization. The initial scheduling strategy initialized by meta-learning and the task feature representation are input into the reinforcement learning module. The reinforcement learning agent interacts with the environment through the strategy generated by the policy network, and obtains new feedback information based on the actual performance during task execution (such as execution time, GPU load and other indicators). Based on these feedbacks, the system will dynamically adjust the strategy through the reward mechanism to ensure that the scheduling strategy can cope with dynamic environments such as task load and GPU resource changes, and further improve resource utilization and task execution efficiency.

[0031] Step 4: Task scheduling and execution. Taking the scheduling strategy optimized by reinforcement learning as input, the task scheduling execution module assigns tasks to the most suitable GPU. The task scheduling execution module assigns tasks to computing units in the GPU resource pool as needed according to the scheduling strategy, ensuring that tasks can run efficiently on the most suitable hardware and avoiding resource idleness or overload.

[0032] Figure 2 It is a schematic diagram of the structure of the dispatching system of the present invention. Figure 2 As shown, the scheduling system proposed in the present invention includes a task representation module, a meta-learning module, a reinforcement learning scheduling module and a GPU resource management module.

[0033] The task representation module is used to extract features from the input tasks, including the computational requirements, priority, and estimated execution time of the tasks, and represents the task features as vectors for input into subsequent models. The task representation module converts complex task information into feature vectors that are easy for the model to process by describing the tasks in detail, providing support for subsequent meta-learning and reinforcement learning processes.

[0034] The meta-learning module is used to learn historical tasks and extract features from historical task data; the meta-optimization process uses a meta-learning algorithm (such as MAML) to train the model so that the reinforcement learning agent can adapt to new tasks faster; the strategy generator forms an initial scheduling strategy to help the reinforcement learning agent adapt to new tasks faster.

[0035] The reinforcement learning scheduling module is used to make resource allocation decisions based on the classic agent-environment interaction paradigm. The reinforcement learning agent makes decisions based on the initial strategy provided by meta-learning and the current task characteristics. The reinforcement learning scheduling module includes a policy network, an environment interaction module, and a reward calculation module. Among them, the policy network is responsible for generating resource allocation strategies based on the current task characteristics and environmental status. The environment interaction module interacts with the system environment and collects feedback information on task execution. The reward calculation module calculates the reward value of each scheduling decision to help the reinforcement learning model continuously optimize the decision-making strategy.

[0036] The GPU resource management module is responsible for managing and monitoring the usage status of the GPU and applying scheduling decisions to specific hardware resources. Among them, the resource monitoring module is used to monitor the usage of the GPU in real time, including status information such as computing load and memory usage. Scheduling execution module: responsible for applying the decisions of the scheduling strategy to specific GPU hardware to achieve task allocation and execution. Exception management module: responsible for monitoring abnormal situations during task execution, such as GPU overload or failure, and taking corresponding measures.

[0037] It should be noted that the above specific embodiments are exemplary, and those skilled in the art can come up with various solutions inspired by the disclosure of the present invention, and these solutions also belong to the disclosure scope of the present invention and fall within the protection scope of the present invention. Those skilled in the art should understand that the present invention description and its drawings are illustrative and do not constitute a limitation of the claims. The protection scope of the present invention is defined by the claims and their equivalents.

Claims

1. A heterogeneous GPU resource scheduling method based on the combination of meta-learning and reinforcement learning, characterized in that: The resource scheduling method generates an initial scheduling strategy for task features through meta-learning, and continuously optimizes the scheduling strategy through reinforcement learning combined with real-time environmental feedback. First, the feature vector of the input task is extracted and used in the meta-learning module to generate the initial scheduling strategy; then the scheduling strategy parameters are optimized through the reinforcement learning agent to improve the efficiency of task allocation; then the scheduling decision is executed to allocate the task to the most suitable GPU; finally, the strategy is dynamically adjusted according to the system feedback and GPU status to ensure that the system always maintains the best performance in a changing environment, including: Step 1: Task feature extraction. The system extracts key features from the input task through the task feature extraction module and uses them as input to the meta-learning module and the reinforcement learning scheduling module. Step 2: Meta-learning initialization strategy. The extracted task feature vector is input into the meta-learning module. The model is trained using the meta-learning algorithm to obtain a more appropriate initial scheduling strategy for the new task. Step 3: Reinforcement learning optimization. The initial scheduling strategy initialized by meta-learning and the task feature representation are input into the reinforcement learning module. The reinforcement learning agent interacts with the environment through the strategy generated by the policy network. Based on the actual performance during the task execution, it obtains new feedback information. Based on the feedback information, the system dynamically adjusts the scheduling strategy through a reward mechanism. Step 4: Task scheduling and execution. Taking the scheduling strategy optimized by reinforcement learning as input, the task scheduling execution module assigns tasks to the most suitable GPU. The task scheduling execution module assigns tasks to computing units in the GPU resource pool as needed according to the scheduling strategy.

2. A heterogeneous GPU resource scheduling system based on the combination of meta-learning and reinforcement learning, characterized in that: The resource scheduling device includes a task representation module, a meta-learning module, a reinforcement learning scheduling module and a GPU resource management module, wherein: The task representation module is used to extract features of the input tasks, including the computing requirements, priority, and estimated execution time of the tasks, and represent the task features as vectors to input into subsequent models; The meta-learning module is used to learn historical tasks and extract features from historical task data. The meta-optimization process uses the meta-learning algorithm to train the model so that the reinforcement learning agent can adapt to new tasks more quickly. The strategy generator forms an initial scheduling strategy to help the reinforcement learning agent adapt to new tasks more quickly. The reinforcement learning scheduling module is based on the classic agent-environment interaction paradigm. The reinforcement learning agent makes resource allocation decisions based on the initial strategy provided by meta-learning and the current task characteristics. The reinforcement learning scheduling module includes a policy network, an environment interaction module, and a reward calculation module. The policy network is responsible for generating resource allocation strategies based on the current task characteristics and environmental status. The environment interaction module interacts with the system environment to collect feedback information on task execution. The reward calculation module calculates the reward value of each scheduling decision to help the reinforcement learning model continuously optimize the decision strategy. The GPU resource management module is used to manage and monitor the usage status of the GPU and apply scheduling decisions to specific hardware resources. Among them, the resource monitoring module is used to monitor the usage of the GPU in real time, including status information such as computing load and memory usage. The scheduling execution module is used to apply the decisions of the scheduling strategy to specific GPU hardware to realize task allocation and execution. The exception management module is used to monitor abnormal situations during task execution, such as GPU overload or failure, and take corresponding measures.

Images