Machine learning task scheduling method and system based on GPU perception and DQN

Abstract

The invention relates to the technical field of task scheduling, and provides a machine learning task scheduling method and system based on GPU perception and DQN, and the method comprises the steps: carrying out the extension of a cluster, and achieving the perception and sharing of GPU resources; receiving machine learning training task requests, adding the machine learning training task requests into a to-be-scheduled task queue for queuing, and polling the queue until task requests which can be met by residual resources of a cluster are found; acquiring a real-time state of each node of the cluster, and generating a cluster state; based on the cluster state, the intelligent agent selects a decision action from the action space and schedules a task to a node; and after the agent selects the decision action, a reward corresponding to the action is obtained, the cluster state, action and reward information of the same task are put into an experience playback pool as an experience, experience playback pool sampling is carried out, and a DQN algorithm is used for training and updating of the agent. The average completion time of the training tasks and the total processing time of a batch of training tasks can be effectively shortened.

Description

Technical Field

[0001] This invention belongs to the field of task scheduling technology, and in particular relates to a machine learning task scheduling method and system based on GPU perception and DQN (Deep Q-Network). Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of machine learning technology, the demand for machine learning training is increasing. At the same time, with the rise of large models, the computing power of ordinary computers is increasingly unable to support users' training requirements. Therefore, in order to meet people's expectations for accelerating the execution speed of training tasks, machine learning server clusters independent of personal computers have gradually gained widespread favor from laboratories and enterprises.

[0004] Meanwhile, with the popularization of cloud computing and the rapid development of container technology in recent years, container management frameworks like Kubernetes have emerged. Kubernetes is an open-source container orchestration and management platform that can manage cluster environments composed of multiple nodes. Developed by Google, it has become a widely used framework in production environments. Kubernetes can automatically deploy, scale, and manage containerized applications within a cluster, greatly reducing the workload for developers. Furthermore, because it can instantly detect the load status and node health on the cluster and automatically replace failed containers, it also significantly improves the robustness of container systems. Due to the excellent isolation and portability of containers, this provides convenient conditions for the efficient training of machine learning tasks on clusters. Therefore, encapsulating machine learning training tasks in containers and running them on server clusters, then using Kubernetes to manage these containers, becomes very valuable.

[0005] However, while Kubernetes can schedule containers well, the scheduling methods used by Kubernetes are only general in nature, as the tasks running within containers are diverse. For specific tasks, the scheduling method may not be the optimal solution. Therefore, Kubernetes has certain limitations when dealing with the specific problem of machine learning training tasks. (1) The default Kubernetes scheduler, Kube-Scheduler, only supports scheduling of conventional resources such as CPU and memory, and cannot perceive or process the status of GPU resources, nor can it achieve GPU sharing to deploy multiple tasks on a single GPU. This deficiency means that when performing tasks that require a large amount of GPU computing power, the scheduler cannot understand key information such as the load, idle status, and utilization efficiency of the GPUs in the current cluster. Therefore, when faced with tasks that are highly sensitive to GPU status, such as machine learning model training and other tasks that require frequent use of GPU acceleration, Kube-Scheduler cannot make reasonable scheduling decisions based on the actual GPU resource status.

[0006] (2) In addition, the default scheduler of Kubernetes is more focused on ensuring the health of the cluster and the expected goal is to balance the load of the cluster. However, for machine learning training tasks, the training completion time is the goal that users are more concerned about. Therefore, it is difficult for Kubernetes to achieve the shortest average task completion time when scheduling training tasks.

[0007] (3) In current practice, many users often use custom schedulers officially supported by Kubernetes to replace the default Kubernetes scheduler in order to achieve better task scheduling results. Among these custom schedulers, common heuristic scheduling algorithms (such as BestFit, Shortest Job First (SJF), and load balancing strategies) have shown certain effectiveness in certain application scenarios due to their relatively simple and straightforward design. However, a major limitation of these heuristic algorithms is that they often rely on pre-set fixed rules or rules of thumb, and these rules are prone to causing the scheduler to get stuck in local optima when faced with complex and dynamic resource allocation scenarios, making further optimization impossible. Summary of the Invention

[0008] To address the technical problems mentioned above, this invention provides a machine learning task scheduling method and system based on GPU perception and DQN. The deep reinforcement learning algorithm DQN is used to train an agent to schedule machine learning training tasks. The agent can select a more suitable node to place the task based on the remaining resources of the cluster (including GPU resources) and the resource request of the task, which can effectively shorten the average completion time of the training task and the total processing time of a batch of training tasks.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: The first aspect of the present invention provides a machine learning task scheduling method based on GPU-aware and DQN, comprising: The cluster is expanded to use GPU status awareness as the basis for scheduling, while also enabling GPU sharing; It receives machine learning training task requests and adds them to the queue of tasks to be scheduled, polling the task queue until it finds a task request that can be satisfied by the remaining resources of the cluster. Obtain the real-time status of each node in the cluster and generate the cluster status; Based on the cluster state, the agent selects a decision action from the action space. After the agent selects a decision action, it obtains the corresponding reward. The cluster state, action and reward information of the same task are put into the experience replay pool as an experience. When the number of experiences in the experience replay pool reaches the threshold, the experience replay pool is sampled to train and update the agent. The training of the agent adopts the DQN algorithm. Based on decision-making actions, tasks are scheduled to nodes.

[0010] Furthermore, the reward is: ;in, This represents the relative execution time of the scheduled task after it has been completed. This refers to the interference that a task, once scheduled to a node, causes to other tasks already existing on that node. This indicates the adjustment parameter.

[0011] Furthermore, the interference is: Among them, the time it takes for a certain task to slow down due to interference. The training tasks can be performed individually or in pairs simultaneously. Indicates the time of execution when performed alone. This represents the time for two tasks to execute simultaneously, where n represents the number of nodes in the cluster, m represents the number of training task types, and k represents the number of tasks that were already running on the node.

[0012] Furthermore, the relative running time is: ;in, It is the task completion time. It is the baseline time for task completion.

[0013] Furthermore, before the agent makes a decision on an action, actions in the action space corresponding to nodes with insufficient resources are set as illegal.

[0014] Furthermore, the machine learning training task request includes the requested number of CPUs, the requested amount of memory, the requested number of GPUs, and the type of training task.

[0015] Furthermore, the real-time status of the node includes the remaining number of CPUs, the remaining amount of memory, the remaining amount of GPU memory, the number of tasks running on the node, and the GPU model of the node.

[0016] A second aspect of the present invention provides a machine learning task scheduling system based on GPU-aware and DQN, comprising: The task queue module is configured to receive machine learning training task requests, add them to the task queue to be scheduled, and poll the task queue until a task request can be satisfied by the remaining resources of the cluster. The cluster status monitoring module is configured to: acquire the real-time status of each node in the cluster and generate the cluster status; wherein, the cluster is expanded to use the GPU status as a scheduling basis and to realize GPU sharing. The agent decision-making module is configured as follows: based on the cluster state, the agent selects an action from the action space; after the agent selects an action, it obtains the reward corresponding to the action; the cluster state, action and reward information of the same task are put into the experience replay pool as an experience; after the number of experiences in the experience replay pool reaches the threshold, the experience replay pool is sampled for the training and updating of the agent; and the training of the agent adopts the DQN algorithm. The task scheduling module is configured to schedule tasks to nodes based on decision-making actions.

[0017] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the machine learning task scheduling method based on GPU perception and DQN as described above.

[0018] A fourth aspect of the present invention provides a computer device including a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, wherein the processor executes the program to implement the steps of the machine learning task scheduling method based on GPU perception and DQN as described above.

[0019] Compared with the prior art, the beneficial effects of the present invention are: This invention extends the original cluster with GPU state awareness and uses the deep reinforcement learning algorithm DQN to train agents to schedule machine learning training tasks. The agents can select a more suitable node to place tasks based on the remaining resources of the cluster's heterogeneous resources (including CPU, memory, and GPU) and the resource requests of the tasks. This can effectively shorten the average completion time of training tasks and the total processing time of a batch of training tasks.

[0020] This invention quantifies the interference caused by the current scheduled task to existing tasks on a node and incorporates it into the reward. This allows the agent to take into account the interference between different tasks after training. This method solves the following problem: the completion time of a task is not only related to the node it is deployed on, but also to the interference from other tasks deployed on the same node in the future. However, since it is impossible to predict future interference in advance during scheduling, interference quantification is adopted. This requires the decision-makers to bear the negative impact on previously existing tasks, so as to minimize interference and shorten the average completion time. Attached Figure Description

[0021] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0022] Figure 1 This is a flowchart of a machine learning task scheduling method based on GPU perception and DQN according to Embodiment 1 of the present invention; Figure 2 This is an architecture diagram of a machine learning task scheduling system based on GPU perception and DQN according to Embodiment 2 of the present invention; Figure 3 This is an architecture diagram of the distributed deployment of the machine learning task scheduling system based on GPU perception and DQN in Embodiment 2 of the present invention on two servers. Figure 4 This is a schematic diagram of the structure of a computer device according to Embodiment 4 of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0024] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0025] Example 1 This embodiment provides a machine learning task scheduling method based on GPU awareness and DQN.

[0026] In recent years, Deep Reinforcement Learning (DRL) has rapidly emerged as a powerful tool in numerous fields. Reinforcement Learning (RL) is a type of machine learning method based on behavior and feedback. Its core idea is to continuously obtain feedback from the environment through the interaction between the agent and the environment, and optimize behavioral strategies through reward mechanisms. A significant characteristic of reinforcement learning is its decision-making ability, making it particularly suitable for solving complex problems that require long-term interaction to optimize strategies. It has been successfully applied in many practical scenarios. For example, in the gaming field, deep reinforcement learning algorithms have been successfully applied to the development of game AI, with reinforcement learning models like AlphaGo defeating world-class Go players. Furthermore, reinforcement learning has shown great potential in areas such as autonomous driving, robot control, intelligent trading, and dynamic advertising. Another prominent application area of ​​reinforcement learning is resource scheduling. In cloud computing, edge computing, and distributed systems, resource scheduling has always been a challenging task, especially under conditions of dynamically changing resources and diverse task requirements. Through reinforcement learning, scheduling strategies can be intelligently learned and optimized, solving local optima and inflexibility problems that traditional heuristic scheduling algorithms cannot effectively address.

[0027] The machine learning task scheduling method based on GPU awareness and DQN provided in this embodiment expands the resource awareness scope of Kubernetes by utilizing the gpushare-device-plugin supported by Kubernetes. This allows Kubernetes to be aware of heterogeneous resources (including CPU, memory, GPU memory availability, and GPU type) during scheduling and to achieve GPU sharing. This successfully overcomes the limitations of the default Kubernetes scheduler in perceiving GPU status and achieving GPU sharing, enabling real-time monitoring of GPU status information in the cluster, thus providing more optimized decisions for tasks to be scheduled. Furthermore, it addresses the challenges of heterogeneous Kubernetes environments. This paper addresses the challenges of scheduling machine learning training tasks in Kubernetes clusters. An improved Deep Reinforcement Learning (DQN) algorithm is used to train an agent to schedule these tasks. The agent can select a more suitable node to place tasks based on the remaining resources of the heterogeneous cluster and the resource requests of the tasks, effectively shortening the average completion time of training tasks and the total processing time for a batch of training tasks. The paper also defines the states, actions, and reward mechanisms required for reinforcement learning. Furthermore, innovative reward design mechanisms, including delayed rewards, action masks, and quantization interference, are introduced to specifically improve the traditional DQN algorithm, significantly enhancing its performance in handling machine learning training task scheduling decisions in heterogeneous Kubernetes clusters.

[0028] The machine learning task scheduling method based on GPU awareness and DQN provided in this embodiment first deploys a GPU device plugin on a Kubernetes cluster to achieve GPU awareness and sharing. Then, it continuously receives machine learning training requests from users and puts them into a scheduling queue in order. The queue is polled and the most suitable task is retrieved. Then, the real-time status of each node in the cluster is obtained (including five pieces of information: CPU remaining amount, memory remaining amount, GPU memory remaining amount, GPU type, and number of tasks running on the node), thereby generating the state required for reinforcement learning. After that, the improved DQN algorithm is used to train and update the model. The improvement methods used include reward design mechanisms such as action masking, delayed reward, and quantized interference. Finally, after multiple rounds of training, a mature agent is obtained, which can make appropriate decisions on the scheduling of machine learning training tasks in the cluster nodes. This solution addresses the issues of the Kubernetes default scheduler's inability to perceive GPU status and achieve GPU sharing. It enables the scheduler to make more suitable decisions for machine learning training tasks by being aware of GPU status, and supports the deployment of multiple tasks on the same GPU, significantly improving GPU resource utilization and thus enhancing overall performance and efficiency. Furthermore, it addresses the problem that the Kubernetes default scheduler cannot specifically handle machine learning training task scheduling and therefore cannot make targeted optimal decisions. In terms of performance, compared with several advanced baseline algorithms, including the Kubernetes default scheduler, the method in this embodiment shows significant performance improvements in both reducing the average task completion time and the total processing time of a batch of training tasks.

[0029] The machine learning task scheduling method based on GPU-awareness and DQN provided in this embodiment, such as Figure 1 As shown, it includes the following steps: Step 1: Configure the GPU device plugin in the cluster so that the scheduler can be aware of the GPU status and realize GPU sharing.

[0030] Device plugins refer to plugins supported by Kubernetes that extend the management of devices in the cluster other than CPU and memory.

[0031] Deploy the gpushare-device-plugin in a Kubernetes cluster. This device plugin can obtain the relevant status of the GPU, including the amount of GPU memory remaining, and also enables GPU sharing, allowing multiple tasks to share the GPU.

[0032] Step 2: The user submits their machine learning training request. The request should include the number of CPUs requested, the amount of memory requested, the number of GPUs requested, and the type of training task.

[0033] For example, the user submitted a request for: a 3-core CPU, 5GB of RAM, 4GB of GPU memory, and a training task type of training a CNN structure model using the Fashion-MNIST dataset.

[0034] Step 3: Add the user request from Step 2 to the task queue for queuing. The scheduling system will then continuously poll the user requests in the queue and check whether the remaining cluster resources can meet the requirements of each request. The specific information monitored includes the remaining number of CPUs, remaining memory, remaining GPU memory, number of tasks running on the node, and the model of the node's GPU. If a task is polled and it is found that the cluster can meet its resource requirements, it is selected as the task to be scheduled and the subsequent scheduling operation is performed.

[0035] Suppose that at this point, the request submitted in step 2 is polled—a 3-core CPU, 5GB of memory, 4GB of GPU memory, and the training task is to train a CNN model using the Fashion-MNIST dataset. Furthermore, if it is detected that the remaining resources of a node can satisfy the user's request, for example, if three nodes meet the conditions: node 0 has 4 remaining CPU cores, 5GB of remaining memory, and 5GB of remaining GPU memory; node 1 has 5 remaining CPU cores, 6GB of remaining memory, and 6GB of remaining GPU memory; and node 2 has 8 remaining CPU cores, 8GB of remaining memory, and 10GB of remaining GPU memory, then the request from step 2 is retrieved from the queue, and subsequent scheduling operations are performed.

[0036] Step 4: For each task that needs to be scheduled, train the agent using the improved deep reinforcement learning (DQN) algorithm. By exploring different node choices, the agent makes decisions based on the current cluster state and rewards or punishes tasks based on the feedback of the scheduling results, thereby continuously optimizing the scheduling strategy and ultimately enabling the agent to make the best decision and achieve optimal task allocation.

[0037] (1) Obtain the real-time status of each node in the cluster and generate the cluster status. .

[0038] state space This refers to the set of all possible states that the environment can be in, including the real-time states of each node in the cluster. and the resource requests of the task itself , can be represented as: ; exist middle, This represents the real-time status of all nodes in the cluster, i.e. = ; exist middle, This represents the real-time status of a specific node, including the main factors affecting the completion time of the training task, such as: the number of remaining CPUs. Remaining memory Remaining GPU memory The number of tasks currently running on this node The GPU model of this node , is represented as: = .

[0039] Specifically, in the above example, consider the remaining resource conditions of the three nodes in step 3: Assume that node 0 has 3 running tasks and GPU model number 0, node 1 has 2 running tasks and GPU model number 1, and node 2 has 0 running tasks and GPU model number 2. It can be represented as: ; ; ; Assuming that the remaining CPU count, remaining memory, remaining GPU memory, number of running tasks on this node, and GPU model on this node are all 0, then the remaining... All can be represented as: ; In the above examples, It can be represented as: = .

[0040] exist middle, This indicates the details of the training task to be scheduled, including the number of CPUs requested. The amount of memory requested Request GPU memory. Task Type , is represented as: .

[0041] Specifically, in the example above, such as the request content in step 2, assuming the requested task type number is 1, then... It can be represented as: .

[0042] In the above examples, It can be fully represented as: = .

[0043] (2) The agent receives Then, it will select the action to make the decision from the action space.

[0044] Among them, action space Let represent the actions that the agent can choose. For this problem, the action the agent needs to take is to place the task on the most suitable node. Therefore, the action space is the set of all nodes in the cluster. All nodes in the cluster are labeled using integers from 0 to n, which can be represented as: .

[0045] Assuming the cluster has a total of 8 nodes, the action space can be represented as: .

[0046] (3) Use action masks to block illegal actions in the action space.

[0047] Before making scheduling decisions, the agent identifies nodes with insufficient resources, sets the actions in the action space corresponding to these nodes as invalid, and assigns the Q-value in the DQN algorithm corresponding to the action to a value that is not specified. In this way, the agent will choose the action with the highest Q value as the final decision, so the agent will never choose an illegal action.

[0048] As in step 3, only the remaining resources of nodes 0, 1, and 2 satisfy the selected task requests. Therefore, action masks are set for nodes 3 to 7. The specific steps are as follows: In the DQN algorithm, the agent receives... Then, the neural network outputs a Q-value for each action in the action space, selects the action with the largest Q-value, and uses this action as the final decision action; therefore, the Q-values ​​output by nodes 3 to 7 are assigned a value. This way, the agent will choose the action with the highest Q value when making decisions, and will never choose an illegal action.

[0049] In this embodiment, it is assumed that the agent gives the largest Q value to node 0, so node 0 is selected as the final decision and the task request will be scheduled to node 0.

[0050] (4) The agent takes an action and schedules the task to a certain node.

[0051] (5) After the agent performs an action, a reward corresponding to that action will be given.

[0052] Reinforcement learning trains an agent through a reward mechanism, enabling the agent to perceive whether performing a certain action in a given state is correct. In this embodiment, the reward... It can be represented as: ; in, This represents the relative running time after the scheduled task is completed. This metric can only be obtained when the task is finished. However, the completion time of a machine learning training task is usually long. Therefore, it is not possible to give a reward immediately after the decision is made, as is the case with the traditional DQN algorithm. Since it takes some time for the task to be completed, a delayed reward method is introduced in the traditional DQN algorithm. This indicates that the task being scheduled to a node interferes with other tasks already existing on that node. This represents an adjustment parameter used to adjust... Adjusting the range to [-10, 10] ensures more stable reinforcement learning training.

[0053] Among them, the relative running time after the scheduled task is completed. , can be represented as: ;in, This is the task completion time. The scheduling system will monitor the completion status of each task and record the completion time of the task once it is completed. The baseline time for task completion is calculated as follows: Run a sufficient number of rounds of experiments in the cluster in advance. The tasks in the experiments are first scheduled by the Kubernetes default scheduler to obtain the completion times of a sufficient number of various machine learning training tasks. Then, calculate the average completion time of each task under the scheduling of the Kubernetes default scheduler as the baseline time.

[0054] This includes the interference that a task, once scheduled to a node, causes to other tasks already existing on that node. , can be represented as: ;in, Each training task can be executed individually or in pairs simultaneously. Indicates the time of execution when performed alone. If the time represents the time when two pairs of items are executed simultaneously, then... This represents the time a task slows down due to interference; therefore, with n nodes and a total of m task types, the time it takes for each task to slow down due to interference on each node can be experimentally determined and denoted as . So, the average time that these tasks slow down due to interference when executed in pairs simultaneously is... However, in reality, at any given moment during the execution of a task, a new task may arrive and preempt resources, causing interference. Assuming the probability of interference is the same at any given moment, the average time slowed down due to interference in a real environment can be expressed as... So, after scheduling a task to a node, assuming that node already has k tasks that are already running, the interference it causes to these k tasks is... It can be represented as .

[0055] In the specific example above, the scheduling system first monitors the completion status of the task; if the task is completed, it records the completion time. Assuming a timeframe of 60 seconds, we then find the pre-calculated average completion time for this type of task as the baseline time. Assuming it's 90 seconds, then calculate... : = = .

[0056] In the specific example above, if the average time to slow down due to interference is set to 4 seconds, then after scheduling the task to node 0, since there are 3 previously existing running tasks on that node, the interference it causes to these 3 tasks is... It can be represented as: .

[0057] In this embodiment, the following settings are provided: The value is 3, to ensure Within the range of [-10, 10], to avoid an excessively large reward range that could lead to training instability, then... It can be represented as: .

[0058] (6) The state, reward, action and other information of the same task are put into the experience replay pool as an experience. After the number of experiences in the experience replay pool reaches 600, the experience replay pool is sampled every five decisions, and the agent is trained and updated. Finally, the training ends after the reward in the training process converges to a relatively high value, and the trained agent is used as the agent that makes the final decision in the scheduling system.

[0059] In this embodiment, a delayed reward method is used, that is, the reward is not given immediately after the scheduling is made, but only after the task is completed. , and Only after it is given will the corresponding task be assigned. , , Information such as these are stored as a single experience in the experience replay pool. In this embodiment, a warm-up process of 600 decisions is first performed, that is, 600 experiences are initially stored in the experience replay pool. Subsequently, 64 experiences are sampled from the experience replay pool every 5 decisions made. Then, gradient descent is used to train and update the agent. After about 500 episodes, the reward function converges to a stable high value, the agent training is complete, and the trained agent is integrated into the scheduling system for subsequent decision-making.

[0060] The machine learning task scheduling method based on GPU awareness and DQN provided in this embodiment first expands the resource awareness scope in a heterogeneous Kubernetes cluster, enabling the scheduling method to perceive the status of cluster GPUs and achieve GPU sharing. Next, for machine learning training task requests submitted by users, they are added to the task scheduling queue in sequence. Then, the scheduling queue is polled until a task request that can be satisfied by the current remaining resources of the cluster is found and removed from the queue. Then, the remaining resource status of each node in the cluster is obtained and combined with the task request status to train the agent to make a decision on which node the task will be placed on, so as to shorten the average completion time of the task and the total processing time of a batch of training tasks. Finally, the task scheduling module is responsible for generating the corresponding pod for the task request and binding it to the specific node of the decision.

[0061] The machine learning task scheduling method based on GPU awareness and DQN provided in this embodiment solves the problem that the default Kubernetes scheduler cannot be aware of the GPU status and cannot achieve GPU sharing by using the Kubernetes-supported gpushare-device-plugin device plugin and a custom scheduling system. This allows the scheduling method to obtain the GPU status of the cluster and make better decisions for the tasks to be scheduled. It can also deploy multiple tasks on the same GPU, optimize the scheduling strategy and improve the utilization of the GPU.

[0062] This embodiment presents a machine learning task scheduling method based on GPU awareness and DQN. Addressing the problem of scheduling machine learning training tasks on heterogeneous Kubernetes clusters, it proposes a solution using deep reinforcement learning (DQN) to train agent decision-making. Specifically, it defines the necessary states, actions, and rewards for reinforcement learning and employs innovative reward design mechanisms such as delayed rewards, action masks, and quantization interference to improve the traditional DQN algorithm, thereby specifically enhancing the performance of DQN in solving the machine learning training task scheduling problem on heterogeneous Kubernetes clusters.

[0063] The machine learning task scheduling method based on GPU perception and DQN provided in this embodiment adopts the delayed reward method mainly because the traditional DQN algorithm requires an immediate reward after an action is performed. However, for this problem, one of the rewards is the relative running time, which can only be obtained after the task is completed, so an immediate reward cannot be given.

[0064] The machine learning task scheduling method based on GPU perception and DQN provided in this embodiment employs action masking primarily to filter out illegal actions. In common reinforcement learning problems, illegal actions are often assigned very low rewards, allowing the agent to learn after multiple rounds of training that certain illegal actions cannot be performed. However, in this embodiment, such illegal actions mean that the agent schedules tasks to nodes with insufficient resources, causing tasks to malfunction or even node crashes. The consequences of such scheduling actions are extremely serious and unacceptable at any stage of training. Therefore, action masking is introduced into the traditional DQN algorithm to directly prevent the agent from selecting these known illegal nodes.

[0065] The machine learning task scheduling method based on GPU perception and DQN provided in this embodiment features an innovative reward design mechanism that quantifies interference. This mechanism quantifies the interference caused by the current scheduled task to existing tasks on the node and incorporates it into the reward. This allows the agent to consider the interference between different tasks after training. This method solves the following problem: the completion time of a task is not only related to the node it is deployed on, but also to the interference from other tasks deployed on the same node in the future. However, since it is impossible to predict future interference in advance during scheduling, interference quantification is adopted. This requires the decision-maker to bear the negative impact on previously existing tasks, so as to minimize interference and shorten the average completion time.

[0066] To verify the effectiveness and superiority of this invention, several advanced baseline algorithms were compared, and evaluation experiments were conducted using three workloads and four arrival frequencies for each workload. The experiments show that the machine learning task scheduling method based on GPU awareness and DQN provided in this embodiment has significant performance advantages in solving the machine learning training task scheduling problem on heterogeneous Kubernetes clusters and in reducing the average task completion time and the total processing time of a batch of training tasks, respectively, by 22.22% and 5.91%.

[0067] Example 2 The machine learning task scheduling system based on GPU-awareness and DQN provided in this embodiment, such as Figure 2 As shown, it includes: The task queue module is configured as follows: a queue is set up, in which user requests to be scheduled are placed from the front to the back of the queue in the order of arrival. The agent continuously polls the task queue. If there are enough resources in the cluster to place the polled task, the task is taken out of the queue and enters other modules for subsequent decision-making and scheduling. The cluster status monitoring module is configured to perform three tasks: first, to continuously monitor the remaining resources of the cluster and determine whether the remaining resources in the cluster can meet the tasks polled in the task queue; second, to obtain the resource usage of each node in the cluster at this time when the agent makes a decision and generate the State information required for the agent's decision; and third, to detect in real time whether the task has been completed after it has been scheduled, and if it has been completed, to calculate the time spent on the task completion. The agent decision-making module is configured to: the agent considers the real-time cluster status monitored by the cluster status monitoring module and the task requests selected by the task queue module, make scheduling decisions, and decide to place the task on a specific node in the cluster. The task scheduling module is configured to: be responsible for executing the scheduling decisions made by the agent decision-making module; and, in each scheduling process, deploy the tasks to be processed as Pods to the cluster nodes selected by the agent, based on the decisions made by the agent.

[0068] like Figure 3 As shown, a machine learning task scheduling system based on GPU awareness and DQN can be deployed on multiple servers on a heterogeneous Kubernetes cluster. Each server has three modules in addition to the task queue module. The server nodes are connected and communicate through distributed technology and share a task queue module. The load balancing algorithm selects different user requests and distributes them to the agent decision-making module on each server for decision-making.

[0069] When the machine learning task scheduling system based on GPU perception and DQN provided in this embodiment is deployed in a distributed manner, the system only needs to be deployed on multiple servers. Then, the user requests submitted by users are placed in the same task queue module for unified queuing to ensure the fairness of the queue. Then, the user requests are distributed to different servers for decision-making and scheduling through a load balancing algorithm. The load balancing algorithm can be implemented by setting up an Nginx server. At the same time, the scheduling results and other information that need to be stored also need to be stored in the same database.

[0070] It should be noted that each module in this embodiment corresponds one-to-one with each step in Embodiment 1, and their specific implementation processes are the same, so they will not be repeated here.

[0071] Example 3 This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the machine learning task scheduling method based on GPU perception and DQN as described in Embodiment 1 above.

[0072] Example 4 This embodiment provides a computer device, such as... Figure 4 As shown, the system includes a computer-readable storage medium 1003, a processor 1001, a communication interface 1002, and a computer program stored on the computer-readable storage medium 1003 and executable on the processor 1001. The processor 1001, communication interface 1002, and computer-readable storage medium 1003 can be connected via a bus or other means. The communication interface 1002 is used to receive and send data. When the processor 1001 executes the program, it implements the steps in the machine learning task scheduling method based on GPU perception and DQN as described in Embodiment 1 above.

[0073] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A machine learning task scheduling method based on GPU-aware and DQN, characterized in that, include: The cluster is expanded to use GPU status awareness as the basis for scheduling, while also enabling GPU sharing; It receives machine learning training task requests and adds them to the queue of tasks to be scheduled, polling the task queue until it finds a task request that can be satisfied by the remaining resources of the cluster. Obtain the real-time status of each node in the cluster and generate the cluster status; Based on the cluster state, the agent selects a decision action from the action space. After the agent selects a decision action, it obtains the corresponding reward. The cluster state, action and reward information of the same task are put into the experience replay pool as an experience. When the number of experiences in the experience replay pool reaches the threshold, the experience replay pool is sampled to train and update the agent. The training of the agent adopts the DQN algorithm. Based on decision-making actions, tasks are scheduled to nodes.

2. The machine learning task scheduling method based on GPU-awareness and DQN as described in claim 1, characterized in that, The reward is: ;in, This represents the relative execution time of the scheduled task after it has been completed. This refers to the interference that a task, once scheduled to a node, causes to other tasks already existing on that node. This indicates the adjustment parameter.

3. The machine learning task scheduling method based on GPU-awareness and DQN as described in claim 2, characterized in that, The interference is: Among them, the time it takes for a certain task to slow down due to interference. The training tasks can be performed individually or in pairs simultaneously. Indicates the time of execution when performed alone. This represents the time for two tasks to execute simultaneously, where n represents the number of nodes in the cluster, m represents the number of training task types, and k represents the number of tasks that were already running on the node.

4. The machine learning task scheduling method based on GPU-awareness and DQN as described in claim 2, characterized in that, The relative running time is: ;in, It is the task completion time. It is the baseline time for task completion.

5. The machine learning task scheduling method based on GPU-awareness and DQN as described in claim 1, characterized in that, Before the agent makes a decision, actions in the action space corresponding to nodes with insufficient resources are set as illegal.

6. The machine learning task scheduling method based on GPU-awareness and DQN as described in claim 1, characterized in that, The machine learning training task request includes the requested number of CPUs, the requested amount of memory, the requested number of GPUs, and the type of training task.

7. The machine learning task scheduling method based on GPU-awareness and DQN as described in claim 1, characterized in that, The real-time status of the node includes the remaining number of CPUs, the remaining amount of memory, the remaining GPU memory, the number of tasks running on the node, and the GPU model of the node.

8. A machine learning task scheduling system based on GPU-aware and DQN, characterized in that, include: The task queue module is configured to receive machine learning training task requests, add them to the task queue to be scheduled, and poll the task queue until a task request can be satisfied by the remaining resources of the cluster. The cluster status monitoring module is configured to: acquire the real-time status of each node in the cluster and generate the cluster status; wherein, the cluster is expanded to use the GPU status as a scheduling basis and to realize GPU sharing. The agent decision-making module is configured as follows: based on the cluster state, the agent selects an action from the action space; after the agent selects an action, it obtains the reward corresponding to the action; the cluster state, action and reward information of the same task are put into the experience replay pool as an experience; after the number of experiences in the experience replay pool reaches the threshold, the experience replay pool is sampled for the training and updating of the agent; and the training of the agent adopts the DQN algorithm. The task scheduling module is configured to schedule tasks to nodes based on decision-making actions.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the machine learning task scheduling method based on GPU awareness and DQN as described in any one of claims 1-7.

10. A computer device comprising a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the machine learning task scheduling method based on GPU perception and DQN as described in any one of claims 1-7.

Images