A resource scheduling method and related apparatus

Abstract

The application provides a resource scheduling method and related device, which divides a first task into a plurality of task segments according to a first parameter of the first task, the first parameter comprising a node representing a first task execution stage, the first task comprising a large model training task; selects hardware resources for the first task segment from a resource pool according to the usage of each type of hardware resource, the resource pool comprising a plurality of types of hardware resources, the usage being a ratio of a first quantity to a second quantity, the first quantity being a quantity of task segments using a first hardware resource among the plurality of task segments, the second quantity being a quantity of task segments pre-allocated to use the first hardware resource among the plurality of task segments, the first hardware resource being any one of the plurality of types of hardware resources, and the first task segment being any one of the plurality of task segments; and executes the first task segment through the hardware resources selected for the first task segment. The method can guarantee task experience consistency in a Serverless scenario.

Description

Technical Field

[0001] This application relates to the field of computers, and more particularly to a resource scheduling method and related apparatus. Background Technology

[0002] With the rapid development of artificial intelligence (AI) technology, the complexity of AI models and training requirements are constantly increasing. The limitations of purchasing cloud-based on-demand computing power clusters in terms of ease of use, scalability, and cost-effectiveness are gradually being exposed. For small and medium-sized customers, there are various problems, such as the inability to continuously and fully utilize the purchased computing power clusters, leading to resource idleness and waste; the use of different hardware configurations, resulting in additional model development workload; and the need for professional information technology (IT) personnel and technical knowledge to manage and optimize local computing power clusters.

[0003] Serverless computing, as an emerging cloud computing paradigm, offers entirely new possibilities for AI training. By abstracting underlying hardware resources, serverless technology enables developers to run their businesses correctly on different hardware configurations, improving development speed, reducing elastic usage costs, and enhancing the resource operation efficiency of cloud vendors.

[0004] However, due to the variety of underlying hardware resources, some existing serverless services contain multiple hardware resources with inconsistent underlying hardware resources. The performance of AI training tasks varies on different hardware resources or at different time points during the training process, resulting in inconsistent final performance of AI training tasks and thus inconsistent user performance experience. Summary of the Invention

[0005] This application discloses a resource scheduling method and related apparatus, which are applied to serverless scenarios. It can ensure that different types of hardware resources are used in a balanced manner in serverless scenarios, guarantee the uniform performance of training tasks, and solve the problem of consistent user experience for AI training tasks in serverless scenarios.

[0006] The first aspect of this application discloses a resource scheduling method, including:

[0007] The first task is divided into multiple task segments based on the first parameter of the first task. The first parameter includes nodes that represent the execution phase of the first task. The first task includes a large model training task.

[0008] Hardware resources are selected for the first task segment from the resource pool based on the utilization rate of various hardware resources. The resource pool includes multiple types of hardware resources, and the utilization rate is the ratio of a first quantity to a second quantity. The first quantity is the number of task segments that have used the first hardware resources among the multiple task segments, and the second quantity is the number of task segments that have been pre-allocated to use the first hardware resources among the multiple task segments. The first hardware resource is any one of the multiple types of hardware resources, and the first task segment is any one of the multiple task segments.

[0009] The first task segment is executed using the hardware resources selected for the first task segment.

[0010] This method divides large model training tasks submitted to the Serverless platform into multiple task segments. Then, based on the utilization rate, hardware resources are selected to execute each task segment. This ensures that the training task can achieve a full and balanced utilization of various hardware resources in the Serverless platform's resource pool, regardless of when the training task stops (e.g., midway through training or after training ends). This avoids using only one or a few types of resources in the resource pool, thus achieving performance consistency and user experience consistency for large model training tasks.

[0011] In one possible implementation of the first aspect, the difference between the first resource proportion and the second resource proportion is less than a preset threshold. The first resource proportion is the ratio of the third quantity to the fourth quantity. The third quantity is the number of task segments that select the first hardware resource from multiple task segments. The fourth quantity is the total number of task segments in the multiple task segments. The second resource proportion is the proportion of the first hardware resource in the total hardware resources included in the resource pool.

[0012] In this method, the ratio of the number of task segments using various types of resources in the resource pool for large model training tasks submitted to the serverless platform (i.e., the ratio of the duration of use of various types of resources) is close to the ratio of the amount of various types of resources in the resource pool. Resources that account for a large proportion in the resource pool are used for a long time, which can ensure that the training tasks use different resources in the resource pool in a balanced way and avoid a certain type of resource becoming a bottleneck.

[0013] In another possible implementation of the first aspect, the hardware resource selected for the first task segment is the hardware resource with the lowest usage rate in the resource pool.

[0014] In this method, the Serverless platform selects the type of hardware resource with the lowest current usage rate for each task segment based on the usage rate of various resources in the resource pool, ensuring balanced utilization of various hardware resources and avoiding frequent use of a certain type of resource.

[0015] In another possible implementation of the first aspect, when there are at least two hardware resources with the lowest utilization, the hardware resource selected for the first task segment is one of the at least two hardware resources with the lowest utilization, selected according to a preset strategy. For example, the preset strategy may be to give priority to the resource with the larger proportion in the resource pool.

[0016] In this method, when the Serverless platform selects the hardware resource with the lowest current utilization for each task segment, if there are at least two hardware resources with the lowest utilization, then one of them is selected as the hardware resource to execute the current task segment according to the preset strategy. This still ensures the balanced utilization of various types of hardware resources in the resource pool and avoids the frequent use of a certain type of resource.

[0017] In yet another possible implementation of the first aspect, the method further includes:

[0018] Before selecting hardware resources for each task segment, the utilization rate of various hardware resources in the resource pool is recalculated.

[0019] Using this method, each time hardware resources are selected for the current task segment, the utilization rate of various hardware resources in the resource pool needs to be recalculated and the calculation results refreshed. This ensures that the hardware resources selected for the current task segment based on the latest utilization rate results are the hardware resources with the lowest current utilization rate, so as to achieve the goal of making balanced use of various hardware resources in the resource pool.

[0020] In yet another possible implementation of the first aspect, the method further includes:

[0021] The number of task segments that the first task uses the first hardware resources is pre-allocated based on the second resource proportion and the fourth quantity.

[0022] Specifically, before selecting hardware resources for each task segment, the Serverless platform will pre-allocate the number of task segments for each type of hardware resource to be used by the training task based on the proportion of each type of hardware resource in the resource pool and the total number of task segments into which the training task is divided.

[0023] Using this method, training tasks submitted to the Serverless platform will utilize various resources according to the system's pre-allocation, so as to ensure that the difference between the proportion of the first resource and the proportion of the second resource is less than a preset threshold, thus making balanced use of various resources in the resource pool.

[0024] In another possible implementation of the first aspect, the first parameter includes a checkpoint.

[0025] In this method, the serverless platform segments the task according to the checkpoints of the first task. Since each checkpoint stores the parameters and results of the training task, segmenting by checkpoint can ensure the continuous progress of the training task by scheduling the checkpoints.

[0026] In one possible implementation of the first aspect, the number of checkpoints N and the number of the plurality of task segments M satisfy: M = N + 1, where M and N are both positive integers.

[0027] In yet another possible implementation of the first aspect, the first task segment is executed using hardware resources selected for the first task segment, including:

[0028] When the first task segment is the starting task segment, the first task segment is executed using the hardware resources selected for the first task segment.

[0029] If the first task segment is not the starting task segment, the data generated after the execution of the first task segment and the previous task segment is executed using the hardware resources selected for the first task segment.

[0030] Using this method, training data is saved after each task segment is completed. Before executing the next task segment, the data saved in the previous task segment needs to be scheduled and loaded onto the hardware resources selected for the current task segment. The current task segment continues to be executed based on this data, ensuring that the training task can continuously train and update parameters based on the training in the previous task segments.

[0031] Secondly, embodiments of this application provide a resource scheduling apparatus, the apparatus comprising:

[0032] The segmentation unit is used to divide the first task into multiple task segments according to the first parameter of the first task, wherein the first parameter includes nodes that represent the execution stage of the first task, and the first task includes a large model training task.

[0033] The selection unit is used to select hardware resources from the resource pool for the first task segment based on the utilization rate of various types of hardware resources. The resource pool includes multiple types of hardware resources, and the utilization rate is the ratio of a first quantity to a second quantity. The first quantity is the number of task segments that have used the first hardware resources among the multiple task segments, and the second quantity is the number of task segments that have been pre-allocated to use the first hardware resources among the multiple task segments. The first hardware resource is any one type of hardware resource among the multiple types of hardware resources, and the first task segment is any one of the multiple task segments.

[0034] An execution unit is used to execute the first task segment using the hardware resources selected for the first task segment.

[0035] This method divides large model training tasks submitted to the Serverless platform into multiple task segments. Then, based on the utilization rate, hardware resources are selected to execute each task segment. This ensures that the training task can achieve a full and balanced utilization of various hardware resources in the Serverless platform's resource pool, regardless of when the training task stops (e.g., midway through training or after training ends). This avoids using only one or a few types of resources in the resource pool, thus achieving performance consistency and user experience consistency for large model training tasks.

[0036] In one possible implementation of the second aspect, the difference between the first resource proportion and the second resource proportion is less than a preset threshold. The first resource proportion is the ratio of the third quantity to the fourth quantity. The third quantity is the number of task segments that select the first hardware resource from multiple task segments. The fourth quantity is the total number of task segments in the multiple task segments. The second resource proportion is the proportion of the first hardware resource in the total hardware resources included in the resource pool.

[0037] In this method, the ratio of the number of task segments using various types of resources in the resource pool for large model training tasks submitted to the serverless platform (i.e., the ratio of the duration of use of various types of resources) is close to the ratio of the amount of various types of resources in the resource pool. Resources that account for a large proportion in the resource pool are used for a long time, which can ensure that the training tasks use different resources in the resource pool in a balanced way and avoid a certain type of resource becoming a bottleneck.

[0038] In one possible implementation of the second aspect, the selection unit selects the hardware resource with the lowest utilization rate in the resource pool for the first task segment.

[0039] In this method, the Serverless platform selects the type of hardware resource with the lowest current usage rate for each task segment based on the usage rate of various resources in the resource pool, ensuring balanced utilization of various hardware resources and avoiding frequent use of a certain type of resource.

[0040] In a second possible implementation of the second aspect, when there are at least two hardware resources with the lowest utilization, the selection unit selects one of the hardware resources with the lowest utilization from the at least two hardware resources according to a preset strategy for the first task segment.

[0041] In this method, when the Serverless platform selects the hardware resource with the lowest current utilization for each task segment, if there are at least two hardware resources with the lowest utilization, then one of them is selected as the hardware resource to execute the current task segment according to the preset strategy. This still ensures the balanced utilization of various types of hardware resources in the resource pool and avoids the frequent use of a certain type of resource.

[0042] In another possible implementation of the second aspect, the device further includes:

[0043] The computing unit is used to recalculate the utilization rate of various hardware resources in the resource pool before selecting hardware resources for each task segment.

[0044] Using this method, each time hardware resources are selected for the current task segment, the utilization rate of various hardware resources in the resource pool needs to be recalculated and the calculation results refreshed. This ensures that the hardware resources selected for the current task segment based on the latest utilization rate results are the hardware resources with the lowest current utilization rate, so as to achieve the goal of making balanced use of various hardware resources in the resource pool.

[0045] In yet another possible implementation of the second aspect, the device further includes:

[0046] The pre-allocation unit is used to pre-allocate the number of task segments that the first task uses the first hardware resources based on the second resource ratio and the fourth quantity.

[0047] Using this method, training tasks submitted to the Serverless platform will utilize various resources according to the system's pre-allocation, so as to ensure that the difference between the proportion of the first resource and the proportion of the second resource is less than a preset threshold, thus making balanced use of various resources in the resource pool.

[0048] In one possible implementation of the second aspect, the first parameter includes a checkpoint.

[0049] In this method, the serverless platform segments the task according to the checkpoints of the first task. Since each checkpoint stores the parameters and results of the training task, segmenting by checkpoint can ensure the continuous progress of the training task by scheduling the checkpoints.

[0050] In one possible implementation of the second aspect, the number of checkpoints N and the number of multiple task segments M satisfy: M = N + 1, where M and N are both positive integers.

[0051] In another possible implementation of the second aspect, the execution unit is used to execute the first task segment using hardware resources selected for the first task segment, including:

[0052] When the first task segment is the initial task segment, the execution unit is used to execute the first task segment using the hardware resources selected for the first task segment;

[0053] If the first task segment is not the starting task segment, the execution unit is used to execute the first task segment and the data generated after the execution of the previous task segment using the hardware resources selected for the first task segment.

[0054] Using this method, training data is saved after each task segment is completed. Before executing the next task segment, the data saved in the previous task segment needs to be scheduled and loaded onto the hardware resources selected for the current task segment. The current task segment continues to be executed based on this data, ensuring that the training task can continuously train and update parameters based on the training in the previous task segments.

[0055] Thirdly, embodiments of this application provide a resource scheduling apparatus, which includes a unit for performing the method described in the first aspect or any possible implementation thereof.

[0056] Fourthly, embodiments of this application provide a resource scheduling apparatus, which includes a processor and a memory. The memory stores a computer program, and the processor invokes the computer program to execute the method described in the first aspect or any possible implementation thereof.

[0057] Fifthly, embodiments of this application provide a computer-readable storage medium for storing a computer program that, when executed on a processor, implements the method described in the first aspect or any possible implementation thereof. Attached Figure Description

[0058] The accompanying drawings used in the embodiments of this application are described below.

[0059] Figure 1 This is a schematic diagram of a system architecture based on serverless services provided in an embodiment of this application;

[0060] Figure 2 This is a flowchart illustrating the overall approach to serverless task scheduling provided in this application embodiment;

[0061] Figure 3 This is a flowchart of a Serverless scheduling method for ensuring consistent task experience provided in an embodiment of this application;

[0062] Figure 4 This is a flowchart of another Serverless scheduling method for ensuring consistent task experience provided in the embodiments of this application;

[0063] Figure 5 This is a schematic diagram of the structure of a resource scheduling device 50 provided in an embodiment of this application;

[0064] Figure 6 This is a schematic diagram of the structure of a resource scheduling device 60 provided in an embodiment of this application. Detailed Implementation

[0065] The embodiments of this application are described below with reference to the accompanying drawings.

[0066] In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; the word "and / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more. Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Therefore, features defined with "first" and "second" can explicitly or implicitly include one or more of that feature. In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0067] The following section introduces the relevant technical background and concepts involved in the embodiments of this application.

[0068] 1) Consistency of Task Experience

[0069] This ensures consistent results and performance across different users or the same user performing the same task at different times. It aims to improve user satisfaction, reduce learning costs, and increase efficiency. In other words, regardless of when a user stops training, the system ensures that the task's progress and efficiency will not significantly change due to hardware switching or other factors when training resumes. This means the user's training experience remains similar across different points in time, without fluctuations in training effectiveness or time caused by scheduling methods.

[0070] 2) Kubernetes

[0071] Kubernetes, or K8s for short, is an open-source application management tool for managing containerized applications across multiple hosts in a cloud platform. Kubernetes aims to make deploying containerized applications simple and efficient, providing a complete set of mechanisms for application deployment, planning, updating, and maintenance. It is an industry-standard and widely used container management platform. The core concepts of Kubernetes are introduced below:

[0072] Container group (Pod): Pod is the smallest deployable unit in Kubernetes. A Pod can contain one or more containers. These containers share resources such as storage and network and are always scheduled together on the same machine.

[0073] Node: A Node is a worker node in Kubernetes, which can be a physical machine or a virtual machine, and is responsible for running Pods.

[0074] Deployment: Deployment is used to manage the creation, updating, scaling up and down of Pods to ensure high availability of applications.

[0075] Service: A Service defines a way to access a group of Pods, providing a stable network access point for the group of Pods and enabling load balancing and service discovery.

[0076] Namespace: Namespaces are used to isolate different users, projects, or environments, ensuring resource isolation and security. They allow multiple users or teams to use different resources within the same cluster without interfering with each other.

[0077] 3) Checkpoint

[0078] A checkpoint is a snapshot of an AI model's state at a specific point in time during training, used to save the model's intermediate state during training. By periodically saving checkpoints, which typically include the model's parameters (weights and biases), optimizer state, and other relevant training information, the current state of the model can be preserved periodically during training, avoiding data loss due to training interruptions or failures. It also supports the recovery and debugging of training tasks and can be used for model evaluation and inference.

[0079] 4) Batch

[0080] In AI training tasks, large amounts of training data are typically used to adjust model parameters (e.g., neural network weights). Loading the entire dataset into memory at once can exceed hardware resource limitations, especially when the dataset is very large. Therefore, instead of feeding all the data into the model at once, the data is usually divided into multiple small batches for training. Using batches instead of individual data samples can improve computational efficiency, striking a balance between training speed and computational resources. Furthermore, using batches to estimate gradients can make the training process more stable. The number of data samples in each batch is called the batch size. For example, if the training dataset contains 1000 data samples and the batch size is 32, then each forward and backward propagation will be performed using 32 samples, and this process will repeat multiple times until all samples have been processed. In AI training tasks, the value of the batch size has a significant impact on the training results, and it is generally a power of 2.

[0081] 5) Epoch

[0082] An epoch represents the number of times the entire training dataset has been used to train and update the model's parameters during training. In other words, an epoch is the process of the entire training set passing through the neural network once. One epoch may not be enough for the model to learn useful features or patterns from the data, so multiple epochs are usually needed to train the model until the loss function stabilizes or reaches a preset stopping criterion. Therefore, in actual training, the model needs to traverse the entire dataset multiple times to gradually adjust its parameters, especially when the dataset is very complex. After each epoch, the model is updated, which usually manifests as a gradual decrease in the loss function and a gradual increase in accuracy. The number of epochs is also very important for the training results. Too few epochs will result in inadequate model training and failure to achieve optimal results, while too many epochs will lead to overfitting and a decline in the model's generalization performance.

[0083] 6) Serverless

[0084] Serverless is a cloud computing execution model that allows developers to focus solely on implementing business logic, while third-party cloud platforms handle the complexities of server management, resource scheduling, and infrastructure maintenance. This service enables developers to run their applications correctly on various hardware configurations, accelerating development, reducing operating costs, minimizing maintenance workload, and supporting rapid elastic scaling. Although called "Serverless," servers still run in the background; users simply don't need to worry about or manage their configuration, maintenance, or scaling.

[0085] Currently, serverless services for large-scale model training are still under development by various vendors, and no clear and definitive solutions have been found. For small-scale inference tasks, major vendors have provided related serverless services. These services use less GPU memory and do not guarantee performance consistency, but have a smaller impact on the user experience. The following are the practices of several mainstream vendors in serverless model inference services:

[0086] The FunctionGraph service provides serverless GPU services designed specifically for AI inference workloads. By offering on-demand GPU computing resources, it effectively addresses pain points such as low resource utilization, high usage costs, and low elasticity caused by the traditional persistent GPU usage. Applications are limited to a vGPU memory limit of 16GB, primarily for supporting AI model inference tasks.

[0087] The PAI-Elastic Algorithm Service (PAI-EAS) allows users to deploy models as online inference services or AI-Web applications with a single click. It is designed for scenarios with dispersed model service calls and flexible usage, providing a serverless model deployment method, such as AI painting.

[0088] SageMaker Serverless Inference is a new inference service that allows you to easily deploy machine learning models for inference without configuring or managing the underlying infrastructure. Applications are limited to a maximum of 6GB of vGPU memory. When deploying a machine learning model, simply select the serverless option, and SageMaker will automatically provision, scale, and shut down compute capacity based on the number of inference requests. Users only pay for the duration of the inference code running and the amount of data processed, not for idle time.

[0089] In these serverless services, the use of GPU cards only requires a single allocation and scheduling process, typically using Kubernetes' general container task scheduling. Based on a series of rules and policies, Pods are assigned to suitable nodes. The scheduler considers factors such as node resource availability, Pod resource requirements, and node affinity / anti-affinity.

[0090] like Figure 1 As shown, Figure 1 A schematic diagram of a system architecture based on serverless services is provided. System 100 includes a user platform 101, a serverless platform 102, and hardware resources 103.

[0091] User platform 101 is user-facing, allowing users to perform various operations. For example, users can access the Serverless platform 102's interface via a webpage or app, gaining access and control. They can then select and execute a specific model (such as a large model) on the Serverless platform 102. User platform 101 can be a computer, a mobile phone, or an application on a computer.

[0092] Serverless platform 102 provides serverless services. After logging into serverless platform 102 through user platform 101, users can write and deploy code (such as models) without needing to concern themselves with the underlying hardware resources 103. Serverless platform 102 can allocate and release computing resources through an abstraction layer (such as AWS Lambda), deploying the user-selected models to hardware resources 103 for execution. The operation of serverless platform 102 does not mean there are no servers, but rather that servers run in the background. Users do not need to worry about or manage the configuration, operation, and expansion of these servers. These servers can be single servers or server clusters composed of multiple servers, and can be deployed in the cloud.

[0093] Hardware resource 103 is typically a resource pool consisting of multiple physical or virtual servers. For example, hardware resource 103 may include GPUs, TPUs, CPUs, etc., capable of performing corresponding computing tasks, such as executing corresponding models. Optionally, when executing the corresponding model, the allocation strategy of various resources in hardware resource 103 can be determined by the serverless platform 102.

[0094] After the corresponding model or code has been executed, the user can see the execution result on the Serverless platform 102, and the Serverless platform 102 will release and reclaim the allocated hardware resources.

[0095] The aforementioned Serverless services lack scheduling design and implementation related to different hardware resources, failing to provide users with a consistent performance experience. For example, some Serverless services mix servers of different generations, such as C7 and C6, with different CPU chips and network connection methods leading to inconsistent final performance. The GPU cards used for training or inference also vary, such as combinations of H100, P100, and A100 models, or combinations of 910B and 910C models, and the network architecture for each generation of computing cards is usually different. Currently, mainstream vendors have not yet solved this problem because, in the traditional computing resource field, compared to the development and delivery speed and cost advantages of Serverless, customers are less concerned about Quality of Service (QoS). The differences in Serverless service experience are mainly reflected in development efficiency, cost control, simplified operation and maintenance, and user experience, in which users generally recognize the advantages of Serverless.

[0096] The method proposed in the embodiments of this application can be applied to... Figure 1 In the system architecture based on serverless services shown, to address the issue of consistent user experience for AI training tasks in serverless scenarios, cloud operators can use multiple computing devices of different generations to centrally schedule and arrange user tasks. By designing a dynamic scheduling method combined with AI training checkpoints, it ensures that different types of hardware resources are utilized in a balanced manner, providing a unified performance experience.

[0097] like Figure 2 As shown, Figure 2 This application provides a flowchart illustrating the overall approach to serverless task scheduling, which can be based on... Figure 1 The Serverless platform 102 in the system architecture shown first refreshes the proportion of various resources in the Serverless platform resource pool for each new training task arriving on the Serverless platform. Then, it pre-allocates resources according to the task parameters (such as the number of checkpoints) and the proportion of various resources in the resource pool. The training is carried out in a balanced manner by scheduling at the checkpoint node. After the task is completed, the allocated computing resources are released.

[0098] The method proposed in this application can be implemented in various ways. Several methods are listed below for detailed explanation.

[0099] Please see Figure 3 , Figure 3 This is a flowchart of a Serverless scheduling method for ensuring consistent task experience proposed in an embodiment of this application. This method can be based on... Figure 1 The Serverless platform implementation in the system architecture shown can also be implemented based on other Serverless platforms or service platforms with similar functions. This method is illustrated based on the following assumptions, including but not limited to the following steps:

[0100] Assumption 1: The serverless platform's resource pool contains two types (there can be multiple types, but this example uses two simple types) of GPU hardware resources, with low-performance GPU model A and high-performance GPU model B; model A has 3 GPU cards and model B has 1 GPU card, with the resource quantity ratio of A to B being 3:1; model A has 16GB of video memory per card and model B has 32GB of video memory per card, with the video memory capacity ratio of A to B being 1:2.

[0101] Step S300: The user submits the first task to the Serverless platform.

[0102] Specifically, the first task can include large model training tasks or AI training tasks. Users can submit training requests to the serverless platform through APIs, web interfaces, or other triggering mechanisms to execute the first task. The training request can include training data (e.g., datasets), training models (e.g., deep learning models), training parameters (e.g., learning rate, batch size, loss function, number of checkpoints, etc.), training resource requirements (e.g., GPU memory requirements), and task configuration information (e.g., training cycle) and other AI training-related information.

[0103] Assumption 2: A user submits a large model training task to the Serverless platform, which requires 16GB of video memory, and sets the number of checkpoints to 4.

[0104] Assumption 3: A user submits a large model training task to the Serverless platform, which requires 16GB of video memory, and sets the number of checkpoints to 5.

[0105] Correspondingly, when the Serverless platform receives a request for a large model training task, it can treat the request as an event and trigger the relevant computing task.

[0106] Step S301: The Serverless platform refreshes the proportions of various resources in the resource pool.

[0107] Specifically, when a new training task is added to the Serverless platform, it is necessary to calculate the total amount of various hardware resources (such as GPUs of different generations) in the platform's resource pool, and then calculate the ratio of each type of hardware resource (such as video memory), that is, the ratio of the total video memory of each type of hardware resource to the total video memory in the resource pool. Since the performance and resource quantity of GPU hardware resources of different generations are different, on cloud servers, GPU resources are usually purchased according to GPU model and video memory quantity. However, in Serverless services, users only need to select the amount of resources (such as video memory) required by the task, without specifying the model and quantity of resources. After the Serverless platform refreshes and calculates the current ratio of various hardware resources, it pre-allocates the time for the training task to use various resources, thus making balanced use of various resources in the resource pool.

[0108] For example, the resource pool contains three types of GPU hardware resources, with type a accounting for 30%, type b accounting for 50%, and type c accounting for 20%. For training tasks submitted to the Serverless platform, it is desired that the task uses resource a for 30% of the entire training process, resource b for 50% of the time, and resource c for 20% of the time, thereby achieving fair resource allocation and consistent performance of the trained model.

[0109] Under assumption 1, after the refresh, the total video memory of Class A resources is 48GB, the total video memory of Class B resources is 32GB, and the total video memory in the resource pool is 80GB. Calculations show that Class A resources account for 3 / 5 and Class B resources account for 2 / 5. Therefore, the Serverless platform expects the task to use Class A resources for 3 / 5 of the training time and Class B resources for 2 / 5 of the training time.

[0110] To ensure that training tasks are conducted using pre-allocated resources, the following technical means can be employed.

[0111] Step S302: The Serverless platform divides the first task into multiple task segments.

[0112] Specifically, to ensure that the primary task (such as a large model training task) makes balanced use of various hardware resources in the platform's resource pool, the system first divides the primary task (such as a large model training task) into multiple task segments based on its primary parameters. These primary parameters include nodes representing the execution phases of the primary task. For example, a serverless platform might divide the primary task into multiple task segments based on checkpoints. Users typically set a checkpoint parameter when submitting a task to store the results and parameters of intermediate nodes. Usually, a checkpoint is saved after each fixed number of batches or each epoch. Checkpoints can be used for further training or inference testing. The batch size for a typical model is usually between 8 and 256, and the number of epochs is usually around 10 to 100. In practice, the number of checkpoints can range from ten to hundreds. For example, the system could divide the primary task into multiple task segments based on the number of checkpoints (N), where the number of checkpoints (N) and the number of task segments (M) satisfy: M = N + 1. Alternatively, the primary task could be divided into multiple task segments based on other parameters.

[0113] For example, when the number of checkpoints N=4, the number of task segments M=5, that is, it can be divided into 5 task segments with the same training time.

[0114] Step S303: The Serverless platform pre-allocates the number of task segments that use various types of hardware resources.

[0115] Specifically, after segmenting the first task, the Serverless platform can pre-allocate the number of task segments using each hardware resource based on the fourth quantity (the number of task segments) and the second resource proportion (the proportion of each type of resource in the resource pool). The goal is for the training task to use the proportion of time spent using each type of hardware resource equal to or close to the proportion of the amount of each type of resource. In other words, the Serverless platform aims to ensure that the difference between the first resource proportion and the second resource proportion is less than a preset threshold during the entire execution of the first task. The first resource proportion is the ratio of the third quantity to the fourth quantity. The third quantity is the number of task segments selecting the first hardware resource from the multiple task segments. The fourth quantity is the total number of task segments among the multiple task segments. The second resource proportion is the proportion of the first hardware resource in the total hardware resources included in the resource pool. This preset threshold can be a small value less than 1 pre-set by the Serverless platform to limit the first resource proportion from being equal to or close to the second resource proportion. The preset threshold can be set by the platform developers or by the user when submitting the task on the Serverless platform.

[0116] Under assumptions 1 and 2, the number of checkpoints N = 4, and the number of task segments M = 5 (the fourth number). At this point, the number of task segments can be divided by the proportion of each type of resource (for example, 3 / 5 * 5 = 3 is an integer). If the Serverless platform expects the first task to use 3 / 5 of the training time with type A resources and 2 / 5 of the training time with type B resources, then it is expected that 3 task segments in the first task will use type A resources for training (i.e., the third number is 3 for type A resources) and 2 task segments will use type B resources for training (i.e., the third number is 2 for type B resources).

[0117] Under assumptions 1 and 3, with N = 5 checkpoints, the number of task segments M = 6 (the fourth number). Since the number of task segments cannot be divided evenly by the proportion of each resource type (e.g., 3 / 5 * 6 = 3.6, which is not an integer), the Serverless platform expects the ratio of time spent using Class A resources to time spent using Class B resources in the first task to be close to their respective proportions in the resource pool. It prioritizes using hardware resources with the larger proportion to execute the task. For example, if the proportion of Class A resources is greater than that of Class B resources, then it is expected that 4 task segments in the first task will use Class A resources for training (the third number for Class A resources is 4), and 2 task segments will use Class B resources for training (the third number for Class B resources is 2).

[0118] To achieve the first task of training according to the pre-allocated resources of the system, it can be done through step S304.

[0119] Step S304: The Serverless platform selects hardware resources for each task segment based on the utilization rate of various resources.

[0120] Specifically, after dividing the first task (such as a large model training task) into multiple task segments, in order to ensure that the multiple task segments can make full and reasonable use of various hardware resources in the system's resource pool, so that the ratio of the time used by the entire first task to use various hardware resources is equal to or close to the ratio of various hardware resources pre-allocated by the system for the first task, hardware resources can be selected individually for each task segment. The utilization rate of various hardware resources in the Serverless platform changes dynamically as the task segments progress. Therefore, the utilization rate of various hardware resources in the resource pool can be recalculated before selecting hardware resources for each task segment. This utilization rate is the ratio of a first quantity to a second quantity. The first quantity is the number of task segments that have used the first hardware resource, and the second quantity is the number of task segments that have been pre-allocated to use the first hardware resource. The first hardware resource can be any type of hardware resource in the system's resource pool. Then, based on the utilization rate, the task is scheduled to be executed on the hardware resource with the lowest current utilization rate, ensuring that the training task uses various resources in the resource pool in a balanced manner according to the ratio pre-allocated by the system. When at least two types of hardware resources with the lowest utilization exist in the resource pool, the task is scheduled to be executed on one of these two types of hardware resources according to a preset strategy. This preset strategy can be random selection, selection based on the idle rate or proportion of a type of resource in the resource pool (e.g., prioritizing the type of resource with a higher proportion in the resource pool), or other strategies. In this way, the system can dynamically adjust resource allocation during task execution, ensuring that the usage time ratio of different types of resources matches the pre-allocated target. Furthermore, the training process of an AI training task can utilize various types of resources in the resource pool in a balanced manner, effectively reducing the problem of inconsistent training or inference results on different hardware resources, thereby achieving performance consistency. Specific scheduling methods include at least the following scenarios:

[0121] In scenario one, when the current training task segment is the starting task segment, the system will schedule the training task to be executed on the hardware resource with the lowest utilization rate selected for the current task segment.

[0122] In scenario two, if the current task segment is not the starting task segment, the checkpoint data saved from the execution of the previous task segment is scheduled and loaded onto the least used hardware resource selected for the current task segment. Then, the task segment and the loaded data are executed. The checkpoint stores the training parameters and results of the previous task segment. Therefore, the task continuity can be ensured by scheduling checkpoint nodes.

[0123] For example, the specific implementation process is described as follows:

[0124] 1) Premise: Under the conditions of assumption 1 and assumption 2, it is expected that the three task segments in the first task will be trained using Class A resources and the two task segments will be trained using Class B resources.

[0125] Time 0 (Task Segment 1): At the start of the task, since training has not yet begun, the system calculates that the utilization rate of both Class A and Class B resources is 0%. Since both Class A and Class B resources have the lowest utilization rates, a pre-defined strategy can be used to select one type of resource. For example, one type of resource can be randomly selected, taking the random selection of Class A resources as an example. Another possible implementation is to select based on the proportion of the two types of resources in the resource pool. Since the proportion of Class A resources in the resource pool is greater than that of Class B resources, Class A resources are selected. After selecting Class A resources for Task Segment 1, the first task is scheduled to be executed on Class A resources, using 16GB of memory.

[0126] Time 1 (Task Segment 2): After Task Segment 1 ends, the training data is saved. Before Task Segment 2 begins, the system will recalculate the utilization rate of various hardware resources. At this time, there is 1 task segment using Class A resources (3 task segments pre-allocated), and 0 task segments using Class B resources (2 task segments pre-allocated). The system calculates that the utilization rate of Class A resources is 1 / 3 = 33.3%, and the utilization rate of Class B resources is also 0%. Since the utilization rate of Class B resources is less than that of Class A resources, Class B resources are selected. That is, the data saved in Task Segment 1 and the first task are scheduled to Class B resources, and Task Segment 2 is executed using 16GB.

[0127] Time 2 (Task Segment 3): After Task Segment 2 ends, the training data is saved. Before Task Segment 3 begins, the system recalculates the utilization rate of various hardware resources. At this time, one task segment is using Class A resources (3 task segments are pre-allocated), and one task segment is using Class B resources (2 task segments are pre-allocated). The system calculates that the utilization rate of Class A resources is 1 / 3 = 33.3%, and the utilization rate of Class B resources is 1 / 2 = 50%. Since the utilization rate of Class A resources is less than that of Class B resources, Class A resources are selected. That is, the data saved in Task Segment 2 and the first task are scheduled to Class A resources, and Task Segment 3 is executed using 16GB.

[0128] Time 3 (Task Segment 4): After Task Segment 3 ends, the training data is saved. Before Task Segment 4 begins, the system recalculates the utilization rate of various hardware resources. At this time, two task segments are using Class A resources (pre-allocated 3 task segments), and one task segment is using Class B resources (pre-allocated 2 task segments). The system calculates that the utilization rate of Class A resources is 2 / 3 = 66.6%, and the utilization rate of Class B resources is 1 / 2 = 50%. Since the utilization rate of Class B resources is less than that of Class A resources, Class B resources are selected. That is, the data saved in Task Segment 3 and the first task are scheduled to Class B resources, and Task Segment 4 is executed using 16GB.

[0129] Time 4 (Task Segment 5): After Task Segment 4 ends, the training data is saved. Before Task Segment 5 begins, the system recalculates the utilization rate of various hardware resources. At this point, two task segments are using Class A resources (pre-allocated 3 task segments), and two task segments are using Class B resources (pre-allocated 2 task segments). The system calculates that the utilization rate of Class A resources is 2 / 3 = 66.6%, and the utilization rate of Class B resources is also 2 / 2 = 100%. Since the utilization rate of Class A resources is less than that of Class B resources, Class A resources are selected. That is, the data saved in Task Segment 4 and the first task are scheduled to use Class A resources, and Task Segment 5 is executed using 16GB. Task Segment 5 is the last task segment. After executing Task Segment 5 using Class A resources, the utilization rate of Class A resources becomes 3 / 3 = 100%, meaning that all the resources pre-allocated by the system for the first task are used. The training of the first task ends, and the final result of the training of the first task is saved after Task Segment 5 ends.

[0130] 2) Premise: Under the conditions of assumptions 1 and 3, it is expected that the four task segments in the first task will be trained using Class A resources and the two task segments will be trained using Class B resources.

[0131] Under premise 2), the method by which the system selects hardware resources for each task segment can refer to the method in premise 1) above.

[0132] Time 0 (Task Segment 1): At the start of the task, the system calculates that the utilization rate of both Class A and Class B resources is 0%. Since both Class A and Class B resources have the lowest utilization rates, a pre-defined strategy can be used to select one type of resource. For example, one type of resource can be randomly selected, taking Class A as an example. Another possible implementation is to select based on the proportion of the two types of resources in the resource pool. Since the proportion of Class A resources in the resource pool is greater than that of Class B resources, Class A resources are selected. After selecting Class A resources for Task Segment 1, the first task is scheduled to run on Class A resources, using 16GB of memory to execute Task Segment 1.

[0133] Time 1 (Task Segment 2): At this time, there is already 1 task segment using Class A resources (4 task segments pre-allocated), and 0 task segments using Class B resources (2 task segments pre-allocated). The system calculates that the utilization rate of Class A resources is 1 / 4 = 25%, and the utilization rate of Class B resources is 0%. Since the utilization rate of Class B resources is less than that of Class A resources, Class B resources are selected. That is, the data saved in Task Segment 1 and the first task are scheduled to Class B resources, and Task Segment 2 is executed using 16GB.

[0134] Time 2 (Task Segment 3): At this point, one task segment is using Class A resources (4 task segments pre-allocated), and one task segment is using Class B resources (2 task segments pre-allocated). The system calculates that the utilization rate of Class A resources is 1 / 4 = 25%, and the utilization rate of Class B resources is 1 / 2 = 50%. Since the utilization rate of Class A resources is less than that of Class B resources, Class A resources are selected. That is, the data saved in Task Segment 2 and the first task are scheduled to Class A resources, and Task Segment 3 is executed using 16GB.

[0135] Time 3 (Task Segment 4): At this point, two task segments are using Class A resources (4 task segments pre-allocated), and one task segment is using Class B resources (2 task segments pre-allocated). The system calculates that the utilization rate of Class A resources is 2 / 4 = 50%, and the utilization rate of Class B resources is also 1 / 2 = 50%. Therefore, a resource type can be selected according to a preset strategy. For example, one type of resource can be randomly selected, taking Class A resources as an example. Another possible implementation is to select based on the proportion of the two types of resources in the resource pool. Since the proportion of Class A resources in the resource pool is greater than that of Class B resources, Class A resources are selected. After selecting Class A resources for Task Segment 4, the data saved in Task Segment 3 and the first task are scheduled onto Class A resources, and Task Segment 4 is executed using 16GB.

[0136] Time 4 (Task Segment 5): At this point, 3 task segments are using Class A resources (4 task segments pre-allocated), and 1 task segment is using Class B resources (2 task segments pre-allocated). The system calculates that the utilization rate of Class A resources is 3 / 4 = 75%, and the utilization rate of Class B resources is 1 / 2 = 50%. Since the utilization rate of Class B resources is less than that of Class A resources, Class B resources are selected. That is, the data saved in Task Segment 4 and the first task are scheduled to Class B resources, and Task Segment 5 is executed using 16GB.

[0137] Time 5 (Task Segment 6): At this point, 3 task segments are using Class A resources (4 task segments pre-allocated), and 2 task segments are using Class B resources (2 task segments pre-allocated). The system calculates that the utilization rate of Class A resources is 3 / 4 = 75%, and the utilization rate of Class B resources is 2 / 2 = 100%. Since the utilization rate of Class A resources is less than that of Class B resources, Class A resources are selected. That is, the data saved in Task Segment 5 and the first task are scheduled to be executed on Class A resources, using 16GB to execute Task Segment 6. Task Segment 6 is the last task segment. After executing Task Segment 6 using Class A resources, the utilization rate of Class A resources becomes 4 / 4 = 100%, meaning that all the resources pre-allocated by the system for the first task are used. The training of the first task ends, and the final result of the training of the first task is saved after Task Segment 6 ends.

[0138] In the embodiments of this application, the steps listed above are only illustrative examples. Some steps may be omitted or replaced with other steps, or new steps may be added, which can also constitute a new feasible solution. In addition, the order of description of the steps above does not represent the order of execution of the steps. The order of execution of the steps may be the same as the order of description or may be different from the order of description (but the logic must be coherent and reasonable).

[0139] Using this method, for each large model training task submitted to the Serverless platform, the Serverless platform divides the task into multiple task segments, pre-allocates the number of task segments that use various types of hardware resources, and selects hardware resources for each task segment based on the utilization rate. At the same time, the scheduling method of switching between different hardware in each task segment in turn using Checkpoint can ensure that the training task can achieve the effect of fully and evenly utilizing various types of hardware resources in the serverless platform's resource pool, regardless of when the training task stops (e.g., stopping training midway or ending training), so as to achieve consistent training task performance and consistent user experience.

[0140] Please see Figure 4 , Figure 4 This is a flowchart of another Serverless scheduling method for ensuring consistent task experience proposed in this application embodiment. This method can be based on... Figure 1 The system architecture shown can also be implemented based on other serverless services or service system architectures with similar serverless service functions. The method is illustrated based on assumptions 1 and 3, including but not limited to the following steps:

[0141] Step S400: The user submits the first task to the Serverless platform.

[0142] The specific implementation details of this step can be found in the relevant description of step S300 above, and will not be repeated here.

[0143] Step S401: The Serverless platform refreshes the proportions of various resources in the resource pool.

[0144] The specific implementation details of this step can be found in the relevant description of step S301 above, and will not be repeated here.

[0145] Step S402: The Serverless platform divides the first task into multiple task segments.

[0146] The specific implementation details of this step can be found in the relevant description of step S302 above, and will not be repeated here.

[0147] Step S403: The Serverless platform pre-allocates the number of task segments that use various types of hardware resources.

[0148] The specific implementation details of this step can be found in the relevant description of step S303 above, and will not be repeated here.

[0149] Step S404: The Serverless platform allocates hardware resources to each task segment according to the first strategy.

[0150] Specifically, the Serverless platform divides the first task into multiple task segments. Then, based on the second resource ratio and the fourth quantity, it pre-allocates the number of task segments that use the first hardware resources for the first task. The hardware resources used by each task segment can be directly selected according to the first strategy. The first strategy can be a system-built-in preset or set by the user when submitting the first task.

[0151] For example, the first strategy could be to use different types of resources in the resource pool in rotation according to their proportions. Once the number of pre-allocated task segments for a certain type of resource is exhausted, that type of resource would no longer be used. Taking assumptions 1 and 3 as an example, if the serverless platform pre-allocates 4 task segments using type A resources and 2 task segments using type B resources, the hardware resource selection for multiple task segments is briefly described below:

[0152] Task Segment 1: If the proportion of each type of resource in the resource pool is greater than that of type B resources, then Task Segment 1 will prioritize the use of type A resources.

[0153] Task segment 2: Take turns, use Class B resources.

[0154] Task segment 3: Take turns, use Class A resources.

[0155] Task Segment 4: In turn, it is time to use Class B resources. At this point, the task segment that the system has pre-allocated to use Class B resources ends.

[0156] Then both task segment 5 and task segment 6 will use Class A resources until the task ends.

[0157] Optionally, in another possible implementation, the first strategy can also involve alternating the use of Class A resources and Class B resources according to the ratio of the number of task segments that pre-allocate to use Class A resources to the number of task segments that use Class B resources. Since the number of task segments that pre-allocate to use Class A resources is twice the number of task segments that use Class B resources, that is, after two task segments use Class A resources, one task segment uses Class B resources once, the hardware resource selection for multiple task segments is as follows:

[0158] Task segment 1 and Task segment 2: Use Class A resources twice first.

[0159] Task segment 3: Use Class B resources.

[0160] Tasks 4 and 5: Use Class A resources twice, alternating between them.

[0161] Task segment 6: Use Class B resources.

[0162] It is worth noting that the first strategy can also be other strategies to ensure that the first task can make balanced use of various hardware resources in the resource pool. Examples will not be given here.

[0163] In the embodiments of this application, the steps listed above are only illustrative examples. Some steps may be omitted or replaced with other steps, or new steps may be added, which can also constitute a new feasible solution. In addition, the order of description of the steps above does not represent the order of execution of the steps. The order of execution of the steps may be the same as the order of description or may be different from the order of description (but the logic must be coherent and reasonable).

[0164] Using this method, for each large model training task submitted to the Serverless platform, the platform divides the task into multiple task segments, pre-allocates the number of task segments that use various types of hardware resources, and allocates hardware resources to each task segment according to the first strategy (e.g., round-robin) to achieve the goal of balancing the use of various types of hardware resources in the resource pool and ensuring the consistency of the performance experience of the training task.

[0165] The methods of the embodiments of this application have been described in detail above, and the apparatus of the embodiments of this application is provided below.

[0166] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a resource scheduling device 50 provided in an embodiment of this application. The resource scheduling device 50 can be a device deployed on the Serverless platform in the above method embodiment or a server of the Serverless platform. The resource scheduling device 500 can include a splitting unit 501, a selection unit 502 and an execution unit 503, wherein each unit is described in detail below.

[0167] The segmentation unit 501 is used to divide the first task into multiple task segments according to the first parameter of the first task, wherein the first parameter includes nodes that characterize the execution stage of the first task, and the first task includes a large model training task.

[0168] Selection unit 502 is used to select hardware resources from the resource pool for the first task segment according to the utilization rate of various hardware resources. The resource pool includes multiple types of hardware resources, and the utilization rate is the ratio of a first quantity to a second quantity. The first quantity is the number of task segments that have used the first hardware resources among the multiple task segments, and the second quantity is the number of task segments that have been pre-allocated to use the first hardware resources among the multiple task segments. The first hardware resource is any one of the multiple types of hardware resources, and the first task segment is any one of the multiple task segments.

[0169] Execution unit 503 is used to execute the first task segment using the hardware resources selected for the first task segment.

[0170] In one possible implementation, the difference between the first resource percentage and the second resource percentage is less than a preset threshold. The first resource percentage is the ratio of the third quantity to the fourth quantity. The third quantity is the number of task segments that select the first hardware resource from multiple task segments. The fourth quantity is the total number of task segments in the multiple task segments. The second resource percentage is the percentage of the first hardware resource in the total hardware resources included in the resource pool.

[0171] In one possible implementation, the selection unit 502 selects the hardware resource with the lowest usage rate in the resource pool for the first task segment.

[0172] In one possible implementation, when there are at least two hardware resources with the lowest utilization, the selection unit 502 selects one of the hardware resources with the lowest utilization from the at least two hardware resources according to a preset strategy for the first task segment.

[0173] In one possible implementation, the device 50 further includes:

[0174] The computing unit is used to recalculate the utilization rate of various hardware resources in the resource pool before selecting hardware resources for each task segment.

[0175] In one possible implementation, the device 50 may further include:

[0176] The pre-allocation unit is used to pre-allocate the number of task segments that the first task uses the first hardware resources based on the second resource ratio and the fourth quantity.

[0177] In one possible implementation, the first parameter includes a checkpoint.

[0178] In one possible implementation, the number of checkpoints N and the number of multiple task segments M satisfy: M = N + 1, where M and N are both positive integers.

[0179] In one possible implementation, execution unit 503 executes the first task segment using hardware resources selected for the first task segment, including:

[0180] When the first task segment is the initial task segment, the execution unit 503 is used to execute the first task segment using the hardware resources selected for the first task segment;

[0181] If the first task segment is not the starting task segment, the execution unit 503 is used to execute the first task segment and the data generated after the execution of the previous task segment of the first task segment by using the hardware resources selected for the first task segment.

[0182] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a resource scheduling device 60 provided in an embodiment of this application. The resource scheduling device 60 includes a processor 601, a memory 602 and a communication interface 603, and the processor 601, memory 602 and communication interface 603 are interconnected through a bus.

[0183] The memory 602 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM), and is used for related computer programs and data. The communication interface 603 is used for receiving and sending data.

[0184] Processor 601 can be one or more central processing units (CPUs). If processor 601 is a CPU, the CPU can be a single-core CPU or a multi-core CPU.

[0185] The processor 601 in the resource scheduling device 60 is used to read the computer program code stored in the memory 602, and can be used to execute it. Figure 5 The operations performed by the segmentation unit 501, selection unit 502, and execution unit 503 in the illustrated device can, for example, perform the following operations:

[0186] The first task is divided into multiple task segments according to the first parameter of the first task, wherein the first parameter includes nodes that characterize the execution phase of the first task, and the first task includes a large model training task.

[0187] Hardware resources are selected for the first task segment from the resource pool based on the utilization rate of various hardware resources. The resource pool includes multiple types of hardware resources, and the utilization rate is the ratio of a first quantity to a second quantity. The first quantity is the number of task segments that have used the first hardware resources among the multiple task segments, and the second quantity is the number of task segments that have been pre-allocated to use the first hardware resources among the multiple task segments. The first hardware resource is any one of the multiple types of hardware resources, and the first task segment is any one of the multiple task segments.

[0188] The first task segment is executed using the hardware resources selected for the first task segment.

[0189] In one possible implementation, the processor 601 can also be used for:

[0190] Before selecting hardware resources for each task segment, the utilization rate of various hardware resources in the resource pool is recalculated.

[0191] In one possible implementation, the processor 601 can also be used for:

[0192] The number of task segments that the first task uses the first hardware resources are pre-allocated based on the second resource ratio and the fourth quantity.

[0193] In one possible implementation, the processor 601 selects the hardware resource with the lowest usage rate from the resource pool for the first task segment.

[0194] In one possible implementation, when there are at least two least used hardware resources, the processor 601 selects one of the at least two least used hardware resources for the first task segment according to a preset strategy.

[0195] In one possible implementation, the first parameter includes a checkpoint.

[0196] In one possible implementation, the number of checkpoints N and the number of multiple task segments M satisfy: M = N + 1, where M and N are both positive integers.

[0197] In one possible implementation, processor 601 executes the first task segment using hardware resources selected for the first task segment, including:

[0198] When the first task segment is the initial task segment, the processor 601 executes the first task segment using the hardware resources selected for the first task segment.

[0199] If the first task segment is not the starting task segment, the processor 601 executes the first task segment and the data generated after the execution of the previous task segment using the hardware resources selected for the first task segment.

[0200] It should be noted that the implementation and beneficial effects of each unit can be found by referring to [the relevant documentation / reference]. Figure 3 The corresponding description of the method embodiments shown.

[0201] It should be noted that any of the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the network devices or hosts provided in this application, the connection relationships between modules indicate that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0202] This application also provides a chip system, which includes at least one processor, a memory, and interface circuitry. The memory, the transceiver, and the at least one processor are interconnected via circuitry. The at least one memory stores a computer program. When the computer program is executed by the processor... Figure 3 The method and flow shown are thus implemented.

[0203] This application also provides a computer-readable storage medium storing a computer program that, when run on a processor... Figure 3 The method and flow shown are thus implemented.

[0204] This application also provides a computer program product, which, when run on a processor... Figure 3 The method and flow shown are thus implemented.

[0205] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program using computer program-related hardware. The computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing computer program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A resource scheduling method, characterized in that, include: The first task is divided into multiple task segments according to the first parameter of the first task, wherein the first parameter includes nodes that characterize the execution phase of the first task, and the first task includes a large model training task. Hardware resources are selected for the first task segment from the resource pool based on the utilization rate of various hardware resources. The resource pool includes multiple types of hardware resources. The utilization rate is the ratio of a first quantity to a second quantity. The first quantity is the number of task segments that have used the first hardware resources among the multiple task segments. The second quantity is the number of task segments that have been pre-allocated to use the first hardware resources among the multiple task segments. The first hardware resource is any one of the multiple types of hardware resources. The first task segment is any one of the multiple task segments. The first task segment is executed using the hardware resources selected for the first task segment.

2. The method according to claim 1, characterized in that, The difference between the first resource percentage and the second resource percentage is less than a preset threshold. The first resource percentage is the ratio of the third quantity to the fourth quantity. The third quantity is the number of task segments in which the first hardware resource is selected from the plurality of task segments. The fourth quantity is the total number of task segments in the plurality of task segments. The second resource percentage is the percentage of the first hardware resource in the total hardware resources contained in the resource pool.

3. The method according to claim 1 or 2, characterized in that, The hardware resources selected for the first task segment are the hardware resources with the lowest usage rate in the resource pool.

4. The method according to claim 3, characterized in that, When there are at least two hardware resources with the lowest utilization, the hardware resource selected for the first task segment is one of the at least two hardware resources with the lowest utilization, selected according to a preset strategy.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: Before selecting hardware resources for each task segment, the utilization rate of various hardware resources in the resource pool is recalculated.

6. The method according to any one of claims 2-5, characterized in that, The method further includes: The number of task segments that the first task uses the first hardware resources are pre-allocated based on the second resource ratio and the fourth quantity.

7. The method according to any one of claims 1-6, characterized in that, The first parameter includes checkpoints.

8. The method according to claim 7, characterized in that, The number of checkpoints N and the number of multiple task segments M satisfy: M = N + 1, where M and N are both positive integers.

9. The method according to any one of claims 1-8, characterized in that, The execution of the first task segment using the hardware resources selected for the first task segment includes: When the first task segment is the initial task segment, the first task segment is executed using the hardware resources selected for the first task segment; If the first task segment is not the starting task segment, the data generated after the execution of the first task segment and the previous task segment is executed by the hardware resources selected for the first task segment.

10. A resource scheduling device, characterized in that, include: A segmentation unit is used to divide the first task into multiple task segments according to the first parameter of the first task, wherein the first parameter includes nodes that characterize the execution phase of the first task, and the first task includes a large model training task. The selection unit is configured to select hardware resources from a resource pool for a first task segment based on the utilization rate of various types of hardware resources. The resource pool includes multiple types of hardware resources, and the utilization rate is the ratio of a first quantity to a second quantity. The first quantity is the number of task segments that have used the first hardware resources among the multiple task segments, and the second quantity is the number of task segments that have been pre-allocated to use the first hardware resources among the multiple task segments. The first hardware resource is any one of the multiple types of hardware resources, and the first task segment is any one of the multiple task segments. An execution unit is configured to execute the first task segment using hardware resources selected for the first task segment.

11. The apparatus according to claim 10, characterized in that, The difference between the first resource percentage and the second resource percentage is less than a preset threshold. The first resource percentage is the ratio of the third quantity to the fourth quantity. The third quantity is the number of task segments in which the first hardware resource is selected from the plurality of task segments. The fourth quantity is the total number of task segments in the plurality of task segments. The second resource percentage is the percentage of the first hardware resource in the total hardware resources contained in the resource pool.

12. The apparatus according to claim 10 or 11, characterized in that, The selection unit selects the hardware resource with the lowest usage rate from the resource pool for the first task segment.

13. The apparatus according to claim 12, characterized in that, When there are at least two hardware resources with the lowest utilization, the selection unit selects one of the two hardware resources with the lowest utilization according to a preset strategy for the first task segment.

14. The apparatus according to any one of claims 10-13, characterized in that, The device further includes: The computing unit is used to recalculate the utilization rate of various hardware resources in the resource pool before selecting hardware resources for each task segment.

15. The apparatus according to any one of claims 11-14, characterized in that, The device further includes: The pre-allocation unit is used to pre-allocate the number of task segments that the first task uses the first hardware resources based on the second resource ratio and the fourth quantity.

16. The apparatus according to any one of claims 10-15, characterized in that, The first parameter includes checkpoints.

17. The apparatus according to claim 16, characterized in that, The number of checkpoints N and the number of multiple task segments M satisfy: M = N + 1, where M and N are both positive integers.

18. The apparatus according to any one of claims 10-17, characterized in that, The execution unit is configured to execute the first task segment using hardware resources selected for the first task segment, including: When the first task segment is the initial task segment, the execution unit is used to execute the first task segment using the hardware resources selected for the first task segment; If the first task segment is not the starting task segment, the execution unit is used to execute the first task segment and the data generated after the execution of the previous task segment of the first task segment by using the hardware resources selected for the first task segment.

19. A resource scheduling device, characterized in that, Includes a unit for performing the method according to any one of claims 1-9.

20. A resource scheduling device, characterized in that, It includes a processor and a memory, the memory being used to store a computer program, and the processor being used to invoke the computer program to perform the method according to any one of claims 1-9.

21. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program that, when run on a processor, implements the method according to any one of claims 1-9.

Images