Business scheduling strategy determination method and device and related equipment

Abstract

The invention provides a service scheduling strategy determination method, which is used for scheduling and processing jobs in a large-scale cluster. The service scheduling strategy determination method comprises the following steps: inputting job information of a plurality of jobs to be scheduled into a first decision model; the first decision model selects at least one target job from the multiple to-be-scheduled jobs according to the job information of the multiple to-be-scheduled jobs; inputting the job information of the target job and the information of the computing resources of the computing cluster into a second decision model; the second decision model outputs a scheduling strategy of the target job according to the job information of the target job and the information of the computing resources, wherein the scheduling strategy comprises the information of the computing resources for executing the target job; and sending the scheduling strategy to a scheduler, and indicating the scheduler to schedule the target job to the corresponding computing resource for execution according to the scheduling strategy. In addition, the invention also provides a corresponding device, a computing device cluster, a computer readable storage medium and a computer program product.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, and related equipment for determining a service scheduling strategy. Background Art

[0002] For computing clusters processing jobs, the efficiency and quality of service scheduling directly impacts the efficiency and quality of the cluster's job processing. Service scheduling involves selecting appropriate nodes within the cluster to handle pending jobs. Optionally, the module responsible for service scheduling within the computing cluster can be called a scheduler. The scheduler schedules jobs for processing according to scheduling policies.

[0003] To improve job processing efficiency and quality, the computing cluster's scheduling strategy can be optimized. In some implementations, a model can be used to determine the scheduling strategy. Specifically, the model can determine the scheduling strategy based on the actual conditions of the computing cluster and the specific conditions of the jobs to be processed. This can optimize the scheduling strategy to a certain extent, thereby improving the efficiency and effectiveness of the cluster's job processing.

[0004] However, the amount of data that needs to be processed in large-scale computing clusters is large, while the processing capacity of the model is limited and often cannot meet the needs of large-scale computing clusters. As a result, the model-based scheduling method is only suitable for small-scale computing clusters and is not suitable for large-scale computing clusters. Summary of the Invention

[0005] In view of this, the present application provides a method for determining a business scheduling strategy for scheduling and processing jobs in a large-scale cluster. The present application also provides a corresponding apparatus, a computing device cluster, a computer-readable storage medium, and a computer program product.

[0006] In a first aspect, the present application provides a method for determining a service scheduling policy, which can address the problem that model-based service scheduling methods are unable to adapt to large-scale processing clusters. Specifically, during service scheduling, multiple jobs to be scheduled can first be input into a first decision model. Next, the first decision model can select at least one target job from the multiple jobs to be scheduled based on their job information. Next, the job information of the target job and information about the computing resources of the computing cluster can be input into a second decision model. Based on the job information and computing resource information of the target job, the second decision model can output a scheduling policy for the target job. The scheduling policy includes information about the computing resources that will execute the target job and is used to instruct the computing cluster to schedule the computing resources to process the target job. After obtaining the scheduling policy, the scheduling policy can be sent to a scheduler to instruct the scheduler to schedule the target job to the corresponding computing resources for execution according to the scheduling policy. In other words, during service scheduling, the first decision model can first select a job from the multiple jobs to be scheduled as the job to be processed. After the job to be processed is selected, the job and information about the computing resources of the computing cluster can be input into the second decision model, so that the second decision model selects an appropriate computing resource to process the target job based on the computing resource information. Therefore, the first decision model only needs to select one or more specific target jobs from multiple jobs, while the second decision model only needs to select specific computing resources from the computing cluster's resources. This reduces the model requirements by using different models to determine jobs and resources, rather than using a single model to determine both jobs and resources simultaneously. This makes it possible to determine business scheduling strategies based on models, resolving the issue that model-based business scheduling methods are not adaptable to large-scale processing clusters.

[0007] In some possible implementations, the determination of the target job may also refer to the status of the computing cluster. Specifically, in addition to the job information of the job to be scheduled, the input of the first decision model may also include the processing resource information of the computing cluster. The processing resource information of the computing cluster may, for example, include information about the computing resources, storage resources, and network resources of the computing cluster. When determining the target job, the processing resource information of the computing cluster may also be input into the first decision model so that the first decision model selects at least one target job from the multiple jobs to be scheduled based on the job information of the multiple jobs to be scheduled and the processing resource information of the computing cluster. In this way, the working status of the computing cluster is taken into consideration when determining the target job, which can avoid determining as the target job a job that the computing cluster cannot process.

[0008] In some possible implementations, the scheduling policy may also indicate other resources allocated to the target job, such as storage resources and network resources. Specifically, the information about the storage resources and network resources of the computing cluster may be input into the second decision model. The second decision model determines the information about the storage resources allocated to the target job based on the job information of the target job and the information about the storage resources of the computing cluster, and determines the information about the network resources allocated to the target job based on the job information of the target job and the information about the network resources of the computing cluster. In this way, not only can it be used to determine the computing resources for processing the target job, but it can also determine the storage resources and network resources used to process the target job. In a scenario where computing resources, storage resources, and network resources are decoupled, the specific resources required to process the target job can be determined to ensure the normal processing of the target job.

[0009] In some possible implementations, the first decision model and / or the second decision model can be optimized. Specifically, a first reward feedback for the first decision model and a second reward feedback for the second decision model can be generated based on the execution results of the scheduler's scheduling policy. The first reward feedback can then be input into the first decision model, and the second reward feedback can be input into the second decision model, thereby optimizing the decision model through the reward feedback. In this way, optimizing the decision model based on the execution results of the scheduling policy can improve the quality of the scheduling policy derived from the decision model.

[0010] In some possible implementations, reward feedback can be determined based on performance indicators. Specifically, multiple performance indicators can be first determined based on the execution results of the scheduler's scheduling policy. Then, based on the multiple performance indicators and the weight of each performance indicator, a first reward feedback and a second reward feedback are generated. The performance indicator used to calculate the first reward feedback and the performance indicator used to calculate the second reward feedback can be the same or different, and the weight used to calculate the first reward feedback and the weight used to calculate the second reward feedback can be the same or different.

[0011] In some possible implementations, performance indicators can be uniformly quantified. Specifically, performance indicators can include cluster resource utilization, scheduling fairness, and resource utilization balance. Accordingly, when calculating reward feedback, the three performance indicators of cluster resource utilization, scheduling fairness, and resource utilization balance can be uniformly quantified first, and then the first reward feedback and the second reward feedback are generated based on the uniformly quantified cluster resource utilization and the weight of cluster resource utilization, the uniformly quantified scheduling fairness and the weight of scheduling fairness, and the uniformly quantified resource utilization balance and the weight of resource utilization balance. In this way, the performance indicators after uniform quantification can relatively uniformly reflect the impact of the scheduling strategy on the computing cluster, thereby reasonably optimizing the decision model.

[0012] In some possible implementations, the optimization direction of the decision model can be adjusted by adjusting the weights of performance indicators. Specifically, if an optimization direction adjustment operation is obtained, the weights of the performance indicators can be adaptively updated based on the optimization direction adjustment operation. In this way, the weights of the computational reward feedback can be adjusted by triggering the optimization direction adjustment operation. After the weights are adjusted, even based on the same performance indicator, different reward feedback can be obtained, thereby updating the decision model in different directions, effectively changing the optimization direction of the computing cluster.

[0013] In the second aspect, the present application provides a business scheduling strategy determination device, which includes: a first determination unit, used to input the job information of multiple jobs to be scheduled into a first decision model; the first decision model selects at least one target job from the multiple jobs to be scheduled based on the job information of the multiple jobs to be scheduled; a second determination unit, used to input the job information of the target job and the information of the computing resources of the computing cluster into a second decision model; the second decision model outputs the scheduling policy of the target job based on the job information of the target job and the information of the computing resources, and the scheduling policy includes information about the computing resources for executing the target job; a scheduling unit, used to send the scheduling policy to the scheduler, instructing the scheduler to schedule the target job to the corresponding computing resources for execution according to the scheduling policy.

[0014] In some possible implementations, the device also includes an acquisition unit, which is used to obtain processing resource information of the computing cluster and input it into a first decision model; the first decision model is specifically used to select at least one target job from the multiple jobs to be scheduled based on the job information of the multiple jobs to be scheduled and the processing resource information of the computing cluster.

[0015] In some possible implementations, the second determination unit is specifically used to input information about the storage resources and network resources of the computing cluster into the second decision model; the scheduling strategy output by the second decision model also includes information about the storage resources and network resources for executing the target job.

[0016] In some possible implementations, the device also includes a decision model updating unit; the decision model updating unit is used to generate a first reward feedback for the first decision model and a second reward feedback for the second decision model according to the execution result of the scheduling strategy of the scheduler; and input the first reward feedback and the second reward feedback into the first decision model and the second decision model respectively to update the models.

[0017] In some possible implementations, the decision model updating unit is specifically used to determine multiple performance indicators based on the execution results of the scheduling policy of the scheduler; and generate the first reward feedback and the second reward feedback based on the multiple performance indicators and the weight of each performance indicator.

[0018] In some possible implementations, the performance indicators include cluster resource utilization, scheduling fairness, and resource utilization balance. The decision model updating unit is specifically used to uniformly quantify the cluster resource utilization, the scheduling fairness, and the resource utilization balance, respectively; and generate the first reward feedback and the second reward feedback based on the uniformly quantified cluster resource utilization and the weight of the cluster resource utilization, the uniformly quantified scheduling fairness and the weight of the scheduling fairness, and the uniformly quantified resource utilization balance and the weight of the resource utilization balance.

[0019] In some possible implementations, the decision model updating unit is further configured to update a weight of at least one of the performance indicators in response to an optimization direction adjustment operation.

[0020] In a third aspect, the present application provides a computing device cluster, wherein the computing device includes at least one computing device, and the at least one computing device includes at least one processor and at least one memory; the at least one memory is used to store instructions, and the at least one processor executes the instructions stored in the at least one memory, so that the computing device cluster executes the method in the above-mentioned first aspect or any possible implementation of the first aspect. It should be noted that the memory can be integrated into the processor or can be independent of the processor. The at least one computing device may also include a bus. The processor is connected to the memory via a bus. The memory may include a readable memory and a random access memory.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on at least one computing device, the at least one computing device executes the method described in the first aspect or any implementation of the first aspect.

[0022] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed on at least one computing device, enables the at least one computing device to execute the method described in the first aspect or any one of the implementations of the first aspect.

[0023] Based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0025] Figure 1a A schematic diagram of an application scenario provided in an embodiment of the present application;

[0026] Figure 1b Another schematic diagram of an application scenario provided in an embodiment of the present application;

[0027] Figure 2 A flow chart of a method for determining a service scheduling strategy provided in an embodiment of the present application;

[0028] Figure 3 Another flowchart of the method for determining a service scheduling strategy provided in an embodiment of the present application;

[0029] Figure 4 A flowchart of a decision model optimization method provided in an embodiment of the present application;

[0030] Figure 5 A schematic diagram of the structure of a device for determining a service scheduling strategy provided in an embodiment of the present application;

[0031] Figure 6 A schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0032] Figure 7 A schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0033] Figure 8 A schematic diagram of an implementation method of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The following will describe the solutions in the embodiments provided in this application in conjunction with the drawings in this application.

[0035] The terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate and are merely used to describe the manner in which objects with the same attributes are described in the embodiments of this application.

[0036] First, some of the nouns involved in this application are introduced.

[0037] Computing cluster: Also known as a processing cluster or just a cluster, it consists of at least one device with data processing capabilities. These devices can be physical or virtual. A computing cluster has the ability to process jobs. It can optionally process multiple jobs serially or in parallel.

[0038] Compute node: Also known as a node, it is a component of a computing cluster. A processing node can correspond to a physical device or a virtual device, used to process jobs. A compute node can process one or more jobs.

[0039] Processing resources, also known as resources, are the elements used to process jobs. Resources include, but are not limited to, compute resources, storage resources, and network resources. Each compute node can provide one or more processing resources. In a compute cluster where storage resources are not decoupled, compute resources, storage resources, and network resources are bundled together, and processing resources can be simply referred to as compute resources. In a compute cluster where storage resources are decoupled, compute resources, storage resources, and network resources are not bundled together.

[0040] Jobs: Jobs are tasks received by the cluster and awaiting processing. In the embodiments of this application, jobs can include various types of jobs, such as compute jobs and storage jobs. Jobs can be obtained based on user-initiated requests. A user-initiated request can correspond to one job or multiple jobs.

[0041] Business scheduling: also known as task scheduling or job scheduling. Business scheduling refers to selecting specific resources in a computing cluster to process specific jobs.

[0042] Scheduler: A software module within a computing cluster that performs business scheduling. Business scheduling involves selecting appropriate resources from the computing cluster to process jobs. The scheduler can perform business scheduling based on scheduling policies.

[0043] Scheduling policy: The principle followed by the scheduler during business scheduling, used to indicate the specific details of the business scheduling computing cluster. The scheduling policy can indicate the jobs to be processed and the resources used to process the jobs. The resources used to process jobs can include node resources, which represent the computing nodes performing the jobs, and can also include sub-resources such as computing resources, storage resources, and network resources, which represent the resources allocated by the scheduler to the jobs to be processed. If the computing cluster corresponds to multiple jobs to be processed, and the jobs are processed on a node-by-node basis, the scheduling policy can include job identifiers and node identifiers to indicate which node in the computing cluster is responsible for processing a specific job.

[0044] Scheduling Algorithm: An algorithm used to determine a scheduling strategy, also known as a service scheduling method. Optionally, a scheduling algorithm can include a model. When determining a scheduling strategy, the model can be used to select a scheduling strategy from multiple options.

[0045] Decision space: When making decisions, a model with decision-making capabilities can establish a decision space based on the attributes of the options. The decision space is a virtual one-dimensional or multi-dimensional space. Each point in the decision space corresponds to an option. The dimension of the decision space is related to the number of attributes of the options.

[0046] Decision Dimension: Decision Dimension refers to the number of attributes that can change in the model's decision-making options. For example, if a model is used to select a data item from multiple data items of category A, one data item from multiple data items of category B, and one data item from multiple data items of category C, resulting in the data combination "A, B, C," then the model has three attributes that can change in its options, and the decision dimension of the model is 3.

[0047] With the development of computer technology, computing clusters have gained widespread application. For example, in cloud computing scenarios, users can create computing requests through a client and send them to a computing cluster over the network. Based on the computing requests, the computing cluster determines one or more pending computing jobs and invokes compute nodes within the cluster to process these jobs.

[0048] In real-world scenarios, a computing cluster may contain multiple jobs to be processed. Therefore, the computing cluster may include a scheduler. The scheduler can call resources to process the jobs according to a scheduling policy determined by a scheduling algorithm.

[0049] For example, some implementations can use scheduling algorithms such as First Come First Serve (FCFS) and Shortest Job First (SJF) for service scheduling. However, these scheduling methods are difficult to adapt to the growth in the scale and complexity of computing clusters and the variability of workloads, and cannot meet the requirements for efficient scheduling.

[0050] To this end, in some implementations, business scheduling can be performed using a model. Specifically, the scheduling algorithm can include methods for invoking the model. When performing business scheduling, job-related and cluster-related information can be input into the model, which then generates a scheduling strategy. Specifically, the model can select one (or more) "job-resource" pairs from multiple "job-resource" pairs as the scheduling strategy. When performing business scheduling, the corresponding resources in the computing cluster can be invoked based on the "job-resource" pair to process the job.

[0051] Optionally, the above model may include an artificial intelligence (AI) model.

[0052] This example uses a reinforcement learning (RL) model, with a compute cluster processing jobs on a node-by-node basis. When scheduling tasks, the state information of each compute node in the compute cluster and the status information of the job to be processed can be obtained, and these two pieces of state information are used as state representations. The state representations can then be input into a pre-trained reinforcement learning model, which can then select one or more "job-node" pairs from multiple "job-node" pairs based on the state representations. The scheduling node then processes the job based on the selected "job-node" pair.

[0053] However, due to the limitations of the model's processing capabilities, the above model-based scheduling method may not be suitable for large-scale computing clusters such as data centers.

[0054] In large-scale computing clusters, the jobs that need to be processed can be complex and the number of jobs required can be relatively large. Furthermore, large-scale computing clusters may have a large number of resources available for processing jobs, and these resources may include multiple types of resources. Therefore, job scheduling requires a comprehensive analysis of job types and the resources available for processing them. However, current models struggle to meet this requirement and are therefore unsuitable for job scheduling in large-scale computing clusters.

[0055] For example, to determine the scheduling strategy, the model constructs a decision space based on the state representation and then uses this space to determine the scheduling strategy. The dimensionality of the decision space is determined by the attributes of the state representation. The more types of attribute information in the state representation, the more dimensional the decision space.

[0056] That is, if the model is required to select a job-node pair from multiple job-node pairs and perform the corresponding processing, the corresponding decision space of the model can have at least two decision dimensions, one of which corresponds to the job to be processed and the other corresponds to the compute nodes. If there are m jobs to be processed and n compute nodes (m and n are both positive integers), then the decision space constructed by the model includes m*n options (i.e., m*n job-node pairs). When scheduling jobs, the model needs to select one option from the m*n options as the job scheduling strategy.

[0057] For large-scale computing clusters, a large number of pending jobs may exist at any one time, resulting in a high number of pending jobs and a larger m. Furthermore, the computing cluster may include a large number of computing nodes, resulting in a larger n. Consequently, the number of options in the decision space (i.e., m*n) increases significantly. Existing models often struggle to meet the requirements for making decisions within such a vast decision space. Models that do meet this requirement often have large sizes, making them difficult to apply to computing clusters.

[0058] Based on this, an embodiment of the present application provides a method for determining a business scheduling strategy. The method for determining a business scheduling strategy can solve the problem that the above-mentioned model-based business scheduling method cannot adapt to large-scale processing clusters. Specifically, during the process of business scheduling, multiple jobs to be scheduled can first be input into a first decision model. Then, the first decision model can select at least one target job from the multiple jobs to be scheduled based on the job information of the multiple jobs to be scheduled. Then, the job information of the target job and the information of the computing resources of the computing cluster can be input into a second decision model. The second decision model can output a scheduling strategy for the target job based on the job information of the target job and the information of the computing resources. The scheduling strategy includes information about the computing resources that execute the target job, and is used to instruct the computing cluster to schedule the computing resources to process the target job. After obtaining the scheduling strategy, the scheduling strategy can be sent to the scheduler to instruct the scheduler to schedule the target job to the corresponding computing resources for execution according to the scheduling strategy. That is, when performing business scheduling, a job can first be selected from the multiple jobs to be scheduled as the job to be processed through the first decision model. After selecting the job to be processed, information about the job and the computing resources of the computing cluster can be input into the second decision model, so that the second decision model can select appropriate computing resources to process the target job based on the computing resource information. Therefore, the first decision model only needs to select one or more specific target jobs from multiple jobs, and the second decision model only needs to select specific computing resources from the computing resources of the computing cluster. In this way, compared to the solution of determining jobs and resources simultaneously through a single model, determining jobs and resources separately through different models reduces the requirements for the models. This makes it possible to determine business scheduling strategies through models, solving the problem that model-based business scheduling methods are unable to adapt to large-scale processing clusters.

[0059] For example, suppose a computing cluster has m jobs to be scheduled and n computing nodes. The computing cluster processes jobs on a node-by-node basis, and processes one job at a time. Then, the first decision model only needs to select one job from the m jobs to be scheduled as the target job, and the second decision model only needs to select one node from the n nodes to process the target job. Thus, the total number of options in the first and second decision models is m+n. Reducing the number of options from m*n to m+n reduces the requirements for the models. Clearly, by decoupling the business scheduling strategy and coordinating multiple models, the problem of adapting the model-based business scheduling method to large-scale processing clusters has been solved.

[0060] Next, various non-limiting specific implementations of the service scheduling strategy determination process are described in detail.

[0061] First, an exemplary application scenario is introduced.

[0062] See Figure 1a , Figure 1a A schematic diagram of an application scenario of the method for determining a service scheduling strategy provided in an embodiment of the present application. Figure 1a The application scenario shown includes user 11, user 12, client 21, client 22, and a computing cluster 30. The computing cluster 30 includes a scheduler 31, computing nodes 321, 322, and 323, and a service scheduling policy determination device 33. The client 21 can run on the terminal device used by user 11, and the client 22 can run on the terminal device used by user 12. The computing cluster 30 can include multiple devices with data processing capabilities. Optionally, the service scheduling policy determination device 33 and the scheduler 31 can be the same software module or different software modules. The service scheduling policy determination device 33 can call the first decision model and the second decision model.

[0063] For example, in a cloud computing scenario, client 21 may be a cloud computing client, such as a cloud computing software client running on a terminal device, or a cloud computing web client opened by a browser program on the terminal device. Computing cluster 30 may be a computing cluster of a cloud computing system, configured to process cloud computing requests initiated by users through the cloud computing client. Computing nodes 321, 322, and 323 may all be servers.

[0064] User 11 can send processing request A to computing cluster 30 via client 21, and user 12 can send processing request B to computing cluster 30 via client 22. Based on request A, computing cluster 30 can determine jobs a1 and a2 to be processed. Based on request B, computing cluster 30 can determine job b to be processed. In other words, the jobs to be scheduled may include jobs a1, a2, and b.

[0065] When determining the scheduling strategy, the service scheduling strategy determination device 33 may first input the job information of jobs a1, a2, and b into the first decision model. Based on the input job information, the first decision model may select a job (hereinafter assumed to be job a1) from jobs a1, a2, and b as the target job. Next, the service scheduling strategy determination device 33 may input the job information of job a1 and the computing resource information of the computing cluster 30 into the second decision model. The computing resource information may include, for example, the remaining computing resources of computing node 321, the remaining computing resources of computing node 322, and the remaining computing resources of computing node 333.

[0066] Based on the job information of job a1 and the computing resource information of computing cluster 30, the second decision model can select a node (hereafter assumed to be computing node 321) from computing node 321, computing node 322, and computing node 323 as the node to process job a, thereby obtaining a scheduling policy for job a1. Next, service scheduling policy determination device 33 can send the scheduling policy to scheduler 31, so that scheduler 31 schedules job a1 to computing node 321 for processing according to the scheduling policy.

[0067] The following is a detailed introduction to the specific implementation of the business scheduling strategy determination method.

[0068] See Figure 2 , Figure 2 A flow chart of a method for determining a service scheduling strategy provided in an embodiment of the present application. This method can be applied to Figure 1a The application scenario shown in FIG, or it can also be applied to other applicable application scenarios. Figure 1a The application scenario shown in FIG, then the method can be executed by the service scheduling strategy determination device 33. Figure 1a The application scenario shown is used as an example for explanation.

[0069] Specifically, Figure 2 The service scheduling strategy determination method shown may specifically include:

[0070] S201: Inputting job information of a plurality of jobs to be scheduled into a first decision model.

[0071] In an embodiment of the present application, a scheduling strategy can be determined by multiple models. In the process of determining the scheduling strategy by multiple models, the job to be scheduled can be first determined by the first decision model. Specifically, the business scheduling strategy determination device can input the job information of multiple jobs to be scheduled into the first decision model, so that the first decision model selects one or more jobs from the multiple jobs to be scheduled as the target job. For an introduction to the first decision model, please refer to S202 below and will not be repeated here.

[0072] Optionally, the scheduled job can be obtained based on a user request. A user request can correspond to one or more jobs. For example, in a cloud computing scenario, if a user requests a computing cluster to perform a certain cloud computing task, this cloud computing task can be processed as a single job. Alternatively, in other possible scenarios, the user request may be more complex, so multiple jobs can be determined based on the user request and processed separately.

[0073] Optionally, the computing cluster may include at least one job queue. The job queue is used to store pending jobs. After obtaining pending jobs, the pending jobs can be added to the job queue. Jobs in the job queue can be considered as pending jobs. During service scheduling, job information of the jobs in the job queue can be input into the first decision model.

[0074] In the embodiment of the present application, the job information of the to-be-scheduled job may include, for example, at least one of the following information: the job identifier, the job ownership information, the job static priority information, the job initiation time, the job size, the job type, the job status, and the job resource requirement. Each of these information is described below.

[0075] The job identifier can be used to uniquely identify the job to be processed; the job's attribution information is information related to the object that issued the job, and may include, for example, the user identifier and user level of the user who initiated the job; the job's static priority information may be, for example, the priority set by the user who issued the job; the job's initiation time may be the time when the processing cluster receives the job; the job's size may be the length of the message corresponding to the job; and the job's type indicates the type of data processing corresponding to the job. For example, assuming that the processing cluster provides two types of services, virtual machines and content distribution networks, the job's type may indicate whether the job is for a virtual machine service or a content distribution network service; the job's status may include, for example, the processing status and the waiting status of the job; and the job's resource requirement may be a pre-estimated amount of resources required to process the job. The job's resource requirement may, for example, indicate the minimum configuration of the node that processes the job, or it may indicate the amount of computing resources, network resources, and storage resources required to process the job.

[0076] It is understandable that the above job information is only used as an example. In actual application scenarios, more or less state representation information can be used according to the actual situation of the computing cluster.

[0077] In real-world scenarios, the capacity of a model's decision space is often pre-set when the model is configured, so the number of options that can be input into the model is limited. If too many options are input, the model may not work properly.

[0078] However, in some possible implementations, the number of jobs to be scheduled corresponding to the computing cluster may be large, exceeding the number that the first decision model can handle. For example, assume that the first decision model can select one job from a maximum of x jobs as the target job to be processed, that is, the capacity of the decision space of the first decision model is x (x is a positive integer greater than 1). However, during the actual operation of the processing cluster, the number of jobs to be processed may be greater than x. Assume that the number of jobs to be processed is y, and y is a positive integer greater than x. In this way, it is necessary to select one job from the y jobs to be processed as the target job to be processed. However, since the number of options (i.e., y) is greater than the capacity of the decision space of the first decision model (i.e., x), the first decision model does not have the ability to select one job from the y jobs to be processed as the target job to be processed.

[0079] To this end, in some possible implementations, jobs can be screened based on the capacity of the first decision model. Specifically, the screening can be performed based on the job information. For example, based on the static priority information or the job initiation time of the job, multiple jobs to be scheduled can be selected from multiple pending jobs, and the job information of the pending jobs can be input into the first decision model. For another example, based on the job's ownership information, jobs with a higher user level belonging to the user who initiated the job can be selected from multiple pending jobs as the jobs to be scheduled, and the job information of the pending jobs can be input into the first decision model.

[0080] Optionally, in some possible implementations, the scheduled jobs may be screened based on the operating status of the computing cluster. That is, in addition to the job information of the scheduled jobs, the input to the first decision model may also include information related to the computing cluster. Specifically, the computing cluster information input to the first decision model may include, for example, processing resource information of the computing cluster. The processing resource information of the computing cluster indicates the remaining processing resources in the computing cluster. The processing resources in the computing cluster may include computing resources of the computing cluster or other types of resources in the computing cluster.

[0081] Accordingly, when determining the target job, the first decision model can filter out jobs that don't match the compute cluster's processing resources from the multiple jobs to be scheduled based on the compute cluster's processing resource information, ensuring that the compute cluster has the capacity to process the target job. This allows jobs that the compute cluster cannot process to be filtered out, preventing them from being designated as target jobs, thereby ensuring that the target job is correctly processed.

[0082] Specifically, assume that at a certain moment, the computing cluster has 5 units of central processing unit (CPU) resources remaining, and there are two pending jobs in the computing cluster: job a1, job a2, and job a3. Processing job a1 requires 10 units of CPU resources, processing job a2 requires 2 units of CPU resources, and processing job a3 requires 4 units of CPU resources. Thus, the service scheduling policy determination device can input "5 units of CPU resources" as the computing cluster's processing resource information into the first decision model. The first decision model can filter jobs a1 and a2 based on the processing resource information, thereby filtering out job a1 that cannot be processed. Next, the first decision model can determine a target job from jobs a2 and a3. Because the combined CPU resources required by jobs a2 and a3 exceed the CPU resources available in the computing cluster, the first decision model can reject both jobs a2 and a3 as target jobs and instead select one job from between jobs a2 and a3 as the target job.

[0083] S202: The first decision model selects at least one target job from the multiple jobs to be scheduled according to the job information of the multiple jobs to be scheduled.

[0084] After job information of multiple jobs to be scheduled is input into a first decision model, the first decision model can select at least one job from the multiple jobs to be scheduled as a target job based on the job information, thereby scheduling computing cluster resources to process the target job. The first decision model is a pre-trained model that has the ability to select at least one job from the multiple jobs based on the job information.

[0085] The target jobs are the jobs that need to be processed in this scheduling. Optionally, the number of target jobs can be pre-set. That is, the first decision model can be pre-set to select n target jobs in each scheduling, thereby processing n target jobs in this business scheduling. Alternatively, the number of target jobs can be determined based on other information. For example, the number of target jobs to be processed in this business scheduling can be determined based on the total number of jobs to be scheduled.

[0086] In embodiments of the present application, the first decision model and the second decision model described below may be integrated into or invoked by the service scheduling policy determination device. For example, the first decision model may be run on a computing node in a computing cluster, and the service scheduling policy determination device may invoke the first decision model via communication within the computing cluster, thereby selecting at least one target job based on the first decision model.

[0087] Optionally, the first decision model may include a policy network and a value network. The policy network is used to make decisions, while the value network is used to evaluate the rationality of the decisions made by the decision network. Specifically, the policy network can be used to select at least one job to be scheduled from multiple jobs to be scheduled. The value network can determine whether the job to be scheduled selected by the policy network is suitable as a target job. If so, the first decision model can output the at least one job to be scheduled as the target job. If not, the first decision model can make a new selection using the policy network.

[0088] Optionally, the first decision model can be implemented based on an actor-critic structure. In addition, the first decision model can be pre-trained. For an introduction to training the first decision model, see Figure 4 , I will not go into details here.

[0089] S203: Inputting the job information of the target job and the information of the computing resources of the computing cluster into the second decision model.

[0090] After determining the target job, the computing resources to process the target job can be determined using the second decision model. Specifically, the service scheduling strategy determination device can obtain the target job returned by the first decision model and input the target job's job information and the computing resources of the computing cluster into the second decision model, thereby selecting the computing resources using the second decision model.

[0091] The computing resource information of the computing cluster represents the available resources of the computing cluster. For example, if business scheduling is performed using computing nodes as units, the computing resource information of the computing cluster may represent that the units in the computing cluster may be computing nodes. Accordingly, the computing resource information of the cluster may include relevant information about each computing node. The relevant information about the computing node may include, for example, the computing node's identifier, the computing node's operating status, and the computing node's resource usage. Based on the relevant information about the computing node, the second decision model may select a suitable computing node from multiple computing nodes to process the target job.

[0092] In some other possible implementations, the computing cluster can also perform service scheduling based on resource type. Specifically, assuming that processing a job requires multiple resources, the service scheduling strategy determination device can determine the relevant information for each resource separately, thereby scheduling the corresponding resources to process the target job based on the relevant information. For example, assuming that processing the target job requires three resources: computing resources, storage resources, and network resources. The service scheduling strategy determination device can then determine the computing resource information, storage resource information, and network resource information corresponding to the target job separately.

[0093] Optionally, the service scheduling strategy determining device may determine the information of multiple resources through a second decision model, or may determine the information of multiple resources through multiple decision models.

[0094] In a first implementation, the business scheduling strategy determination device can determine the information of multiple resources through a second decision model. For example, in a computing cluster where storage resources are decoupled, resources such as computing resources, storage resources, and network resources are not bound to each other. Only determining the computing resources of the target job cannot process the target job. Accordingly, the scheduling strategy includes not only the information of the computing resources of the target job, but also the information of other resources of the target job, such as the information of the storage resources and network resources of the target job. In this way, in a scenario where computing resources, storage resources, and network resources are decoupled, the specific resources required to process the target job can be determined to ensure the normal processing of the target job.

[0095] Accordingly, the information input into the second decision model by the service scheduling strategy determination device may include not only the target job's job information and the computing cluster's computing resource information, but also the computing cluster's storage resource information and network resource information. The second decision model may select appropriate computing resources, storage resources, and network resources to process the target job based on the target job's job information, the computing cluster's computing resource information, the storage resource information, and the network resource information.

[0096] In a second implementation, the service scheduling policy determination device may determine information about multiple resources using multiple decision models. That is, the service scheduling policy determination device may determine information about the target job's computing resources using a second decision model, may determine information about the target job's storage resources using a third decision model, and may determine information about the target job's network resources using a fourth decision model. The input to the third decision model may include job information about the target job and information about the storage resources of the computing cluster. The input to the fourth decision model may include job information about the target job and information about the network resources of the computing cluster.

[0097] Similar to the first decision model, the second decision model can be a pre-trained decision model for selecting one or more options from a plurality of options. In an embodiment of the present application, the second decision model can be used to select corresponding computing resources for each target job from the computing resources of the computing cluster. The structure of the second decision model can be found above and will not be repeated here.

[0098] Optionally, the second decision model and the first decision model may be the same model or different models. If the second decision model and the first decision model are the same model, the first decision model may include at least two decision heads. One decision head is used to determine the target job, and the other decision head is used to determine information about the computing resources corresponding to the target job.

[0099] S204: The second decision model outputs a scheduling strategy for the target job based on the job information of the target job and the information of computing resources.

[0100] After inputting the target job's job information and information about the computing cluster's resources into the second decision model, the second decision model can derive a scheduling policy for the target job based on the job information and computing resource information. The scheduling policy for the target job includes information about the computing resources that will execute the target job, indicating the computing resources within the computing cluster that will be used to process the target job. Based on the scheduling policy for the target job, the target job can be scheduled to the corresponding computing resource, which can then be used to process the target job. In other words, the second decision model can select appropriate computing resources from the computing cluster's resources to process the target job based on the processing requirements for the target job and the actual conditions of the computing cluster.

[0101] Optionally, if the first decision model determines multiple target jobs, the second decision model can determine a scheduling policy for each target job based on the job information of each target job. Each scheduling policy corresponds to a target job and is used to indicate the computing resources used to process the target job.

[0102] If the second decision model is also used to determine other types of resources, the scheduling policy output by the second decision model may also indicate information about these other resources. For example, if the service scheduling policy determination device inputs information about the computing cluster's storage resources and network resources into the second decision model, the second decision model may also select storage resources and network resources for the target job. Accordingly, the scheduling policy output by the second decision model may also include information about the storage resources and network resources used to execute the target job, respectively indicating the storage resources and network resources used to process the target job.

[0103] S205: Send the scheduling policy to the scheduler, instructing the scheduler to schedule the target job to the corresponding computing resource for execution according to the scheduling policy.

[0104] After determining the scheduling strategy through the second decision model, the service scheduling strategy determination device can send the strategy to the scheduler and instruct the scheduler to schedule the target job to the corresponding computing resource for execution according to the scheduling strategy, thereby completing the service scheduling process.

[0105] "Scheduling the target job to the corresponding computing resources for execution" means processing the target job using the computing resources indicated by the scheduling policy. For example, if a computing cluster processes jobs using compute nodes, the scheduler can send the target job to the compute nodes indicated by the scheduling policy, allowing the compute nodes to process the target job.

[0106] After the scheduler completes the execution of the computing resources corresponding to the target job scheduling value according to the scheduling strategy, if there are still jobs to be scheduled in the computing cluster, you can re-execute Figure 2 The service scheduling strategy determination method shown in the figure is used to perform the next round of service scheduling. In addition, in the i-th round (i is a positive integer) of service scheduling, the information of the computing resources of the computing cluster can be recorded, so that the second decision model can obtain the scheduling strategy based on this information in the i+1-th round of service scheduling. Accordingly, the specific scheduling process can be as follows: Figure 3 shown.

[0107] As can be seen, in the embodiment of the present application, the first decision model only needs to select one or more specific target jobs from multiple jobs, and the second decision model only needs to select specific computing resources from the computing resources of the computing cluster. In this way, compared to the solution of determining jobs and resources simultaneously through a single model, determining jobs and resources separately through different models reduces the requirements for the models. This makes it possible to determine business scheduling strategies through models, solving the problem that model-based business scheduling methods cannot adapt to large-scale processing clusters.

[0108] Continued use Figure 1a The implementation shown in FIG. 1 is illustrated. Assume that, during job scheduling, scheduler 31 can assign one job at a time to a specific compute node for processing. Then, the first decision model can be used to select a job as the target job from among jobs a1, a2, and b, and the second decision model can be used to select a compute node from among compute nodes 321, 322, and 323 as the node to process the target job.

[0109] If a decision model is used to directly determine the "job-node" pair, since jobs and computing nodes can be freely combined, there are 3*3=9 options in the decision space of the decision model.

[0110] And in Figure 2 In the provided method for determining a business scheduling strategy, jobs and computing nodes can be determined separately through two decision models. The number of options in the decision space of the first decision model is 3 (corresponding to jobs a1, a2, and b, respectively), and the number of options in the decision space of the second decision model is 3 (corresponding to computing nodes 321, 322, and 323, respectively). Therefore, the total number of options in the decision space of the decision model used to determine the scheduling strategy in the computing cluster is 3+3=6. It can be seen that first determining the target job through the first decision model, and then determining the scheduling strategy through the second decision model combined with the target job can reduce the demand for the capacity of the model's decision space.

[0111] In embodiments of the present application, a scheduling strategy can be determined using a decision model. The decision model can be pre-trained. However, in some application scenarios, there may be discrepancies between the decision model and actual scheduling requirements, resulting in the decision model not being able to meet the actual needs of service scheduling. To this end, in some possible implementations, the decision model can be updated. This is described in detail below with reference to the accompanying drawings.

[0112] See Figure 4 , Figure 4 A flowchart of a decision model updating method provided in an embodiment of the present application. The method can be specifically executed by a decision model updating unit. The decision model updating unit can be run on a computing cluster or on other devices outside the computing cluster. For example, a decision model updating unit can be pre-trained in a certain device (or a certain cluster). Figure 2 The first and second decision models in the illustrated implementation are deployed in a computing cluster for service scheduling. During service scheduling, the decision models can be further adjusted in the device (or cluster) used to train the decision models based on the actual conditions of the computing cluster to make them more consistent with the actual conditions of the computing cluster.

[0113] Alternatively, the method can be applied to the above Figure 1b The application scenario shown in FIG. 1 may also be applied to other applicable application scenarios. Figure 1a Based on the application scenario shown, Figure 1b In the illustrated application scenario, the service scheduling strategy determining device 33 further includes a decision model updating unit 331 .

[0114] Specifically, Figure 4 The decision model updating method shown may specifically include:

[0115] S401: Generate a first reward feedback of the first decision model and a second reward feedback of the second decision model according to the execution result of the scheduling strategy of the scheduler.

[0116] To update the first decision model and the second decision model, the reward feedback for the first decision model and the second decision model can first be determined based on the execution results of the scheduler's scheduling policy. In this embodiment of the present application, the reward feedback for the first decision model can be referred to as the first reward feedback, and the reward feedback for the second decision model can be referred to as the second reward feedback. In this way, by optimizing the decision model based on the execution results of the scheduling policy, the quality of the scheduling policy obtained by the decision model can be improved.

[0117] In the embodiments of the present application, the reward feedback of the decision model (including the first reward feedback and the second reward feedback) can be obtained based on the execution results of the scheduling policy. In other words, if the execution results of the scheduling policy are good, then the first decision model and / or the second decision model can be forward optimized; if the execution results of the scheduling policy are poor, then the first decision model and / or the second decision model can be reverse optimized.

[0118] Optionally, the execution results of the scheduling policy can be represented by performance indicators. Specifically, the decision model update unit determines the performance indicator based on the execution results of the scheduling policy and then first rewards feedback based on the performance indicator. The performance indicator of the computing cluster can reflect the impact of the scheduling policy on the operating conditions of the computing cluster. The performance indicator of the computing cluster can also be referred to as the computing cluster's operating parameter load parameter.

[0119] Optionally, the performance indicator may include multiple types of indicators, such as at least one of the following: resource utilization of the computing cluster, scheduling fairness of the computing cluster, and resource utilization balance of the computing cluster. The resource utilization of the computing cluster may indicate how much of each resource in the computing cluster is utilized and how much is idle. The scheduling fairness of the computing cluster may indicate the computing cluster's ability to fairly and evenly allocate resources to each job when processing it. The resource utilization balance of the computing cluster, also known as the resource utilization balance degree, may indicate the average degree to which various resources in the computing cluster are consumed.

[0120] Optionally, the aforementioned performance indicators, such as the computing cluster's resource utilization, scheduling fairness, and resource utilization balance, can be collected by the computing cluster while executing jobs according to the scheduling policy and sent by the computing cluster to the decision model update unit. Alternatively, the computing cluster can record the performance indicators in a work log while processing jobs. The decision model update unit obtains the performance indicators by reading the computing cluster's work log.

[0121] Optionally, the decision model update unit can update the decision model after each round of scheduling. That is to say, after each round of business scheduling is completed, the reward feedback can be determined based on the performance indicators of the computing cluster after this round of business scheduling, thereby updating the decision model. Alternatively, the decision model can be updated after an interval of n rounds (n is a positive integer greater than 1) of business scheduling. Alternatively, the decision model can be adjusted using the stored performance indicators after the optimization direction is adjusted. Then the performance indicators obtained by the decision model update unit can include the performance indicators of the computing cluster in the time period from the last adjustment of the optimization direction to the current adjustment of the optimization direction. For an introduction to the optimization direction, please refer to the following text and will not be repeated here.

[0122] After obtaining at least one performance indicator, reward feedback can be calculated based on the performance indicator to adjust the decision model using the reward feedback. Optionally, if the first decision model is a reinforcement learning model, the first reward decision can also be referred to as a first reward (Reward) or a first cumulative reward (Cumulative Reward).

[0123] If the decision model update unit obtains a single performance metric, it can be used as reward feedback to reinforce learning across multiple decision models. If the decision model update unit obtains multiple performance metrics, it can combine these metrics to calculate reward feedback, allowing for reinforcement learning across multiple decision models. Each performance metric may correspond to a different optimization direction. In practical scenarios, different performance metrics can be combined into a unified reward feedback loop based on the needs of the optimized computing cluster.

[0124] Optionally, if the decision model update unit obtains multiple performance indicators, a weight can be set for each performance indicator. The weight indicates the degree of influence of the performance indicator on the reward feedback. When calculating the reward feedback, each performance indicator can be multiplied by the weight of the performance indicator to obtain the corresponding reward feedback. Optionally, the weight used to calculate the first reward feedback and the weight used to calculate the second reward feedback can be the same or different.

[0125] In some possible implementations, the magnitudes of different performance indicators may have different impacts on the computing cluster. For example, for some performance indicators, larger values ​​may indicate better computing cluster performance; for other performance indicators, smaller values ​​may indicate better computing cluster performance; and for yet another performance indicator, closer values ​​to a threshold may indicate better computing cluster performance. Furthermore, even if the magnitudes of performance indicators have the same impact on the computing cluster, the magnitudes of different performance indicators may affect the computing cluster's performance to varying degrees. For example, the magnitudes of some performance indicators may vary linearly, and their magnitudes may be strongly correlated with the computing cluster's performance. Meanwhile, the magnitudes of other performance indicators may vary exponentially, and their magnitudes may be less correlated with the computing cluster's performance.

[0126] In this way, if the performance indicators are not preprocessed and the reward feedback is directly calculated using the product of the performance indicators and the weights, the optimization effect of the decision model may be poor.

[0127] To this end, some possible implementations can quantify performance indicators uniformly, and then use these uniformly quantified performance indicators to calculate reward feedback. Specifically, a uniformly quantified indicator can be calculated for each performance indicator separately, and then the reward feedback can be calculated using the uniformly quantified indicator for each performance indicator. For example, the uniformly quantified indicator corresponding to each performance indicator can be multiplied by the weight, and the sum of the products corresponding to each performance indicator can be used as the reward feedback.

[0128] In this way, the performance indicators after unified quantification can relatively uniformly reflect the impact of the scheduling strategy on the computing cluster, thereby reasonably optimizing the decision model. Optionally, the above-mentioned unified quantitative indicators can be obtained based on the rate of change of the performance indicators. For example, for a linearly changing performance indicator, its linear change rate can be calculated, and the unified quantitative indicator of the performance indicator can be determined based on the linear change rate of the performance indicator. For another example, for a performance indicator that changes exponentially, its multiple change rate can be calculated, and the unified quantitative indicator of the performance indicator can be determined based on the multiple change rate of the performance indicator. For another example, for an exponentially changing performance indicator, its exponential change rate can be calculated, and the unified quantitative indicator of the performance indicator can be determined based on the direct change rate of the performance indicator.

[0129] For example, assuming that the performance index A changes linearly, and the larger the performance index A is, the better the performance of the computing cluster is, then the unified quantitative index of the performance index A can be calculated by the following formula (1).

[0130]

[0131] Where a is the value of performance indicator A, t is the baseline value of performance indicator A, and IR(a) is the unified quantitative index of performance indicator A when the value of performance indicator A is a. Optionally, IR(a) can be expressed as IR(a,t), which represents the unified quantitative index of value a relative to the baseline value t. The baseline value t can be a preset baseline value or the value of performance indicator A collected last time.

[0132] For another example, assuming that the performance index B changes linearly, and the smaller the performance index B is, the better the performance of the computing cluster is, then the unified quantitative index of the performance index B can be calculated by the following formula (2).

[0133]

[0134] Where b is the value of performance indicator B, t is the baseline value of performance indicator B, and IR(b) is the unified quantitative index of performance indicator B when the value of performance indicator C is b. Optionally, IR(b) can be expressed as IR(b,t), which represents the unified quantitative index of value b relative to baseline value t. The baseline value t can be a preset baseline value or the value of performance indicator B collected last time.

[0135] For another example, assuming that the performance index C changes exponentially, and the larger the performance index C is, the better the performance of the computing cluster is, then the unified quantitative index of the performance index C can be calculated by the following formula (3).

[0136]

[0137] Where c is the value of performance indicator C, t is the baseline value of performance indicator C, and IR(c) is the unified quantitative index of performance indicator C when the performance indicator C value is c. Optionally, IR(c) can be expressed as IR(c, t), which represents the unified quantitative index of value c relative to the baseline value t. The baseline value t can be a preset baseline value or the value of performance indicator C collected last time.

[0138] For another example, assuming that the performance index D changes exponentially, and the larger the performance index D is, the better the performance of the computing cluster is, then the unified quantitative index of the performance index D can be calculated by the following formula (4).

[0139] IR(d)=ln(d)-t Formula (4)

[0140] Where d is the value of performance indicator D, t is the baseline value of performance indicator D, and IR(c) is the unified quantitative index of performance indicator D when the performance indicator D is at value d. Optionally, IR(c) can be expressed as IR(c, t), which represents the unified quantitative index of value d relative to the baseline value t. The baseline value t can be a preset baseline value or the logarithm of the last collected value of performance indicator D.

[0141] After the performance indicators are uniformly quantified, the first reward feedback and the second reward feedback can be calculated respectively according to the uniformly quantified performance indicators and the weights corresponding to each performance indicator.

[0142] That is to say, if the performance indicators include cluster resource utilization, scheduling fairness, and resource utilization balance, then when calculating the reward feedback, the cluster resource utilization, scheduling fairness, and resource utilization balance can be uniformly quantified respectively. Then, based on the uniformly quantified cluster resource utilization and the weight corresponding to the cluster resource utilization, the uniformly quantified scheduling fairness and the weight corresponding to scheduling fairness, and the uniformly quantified resource utilization balance and the weight of resource utilization balance, the first reward feedback and the second reward feedback can be obtained.

[0143] The performance indicator used to calculate the first reward feedback and the performance indicator used to calculate the second reward feedback can be the same or different. The weight used to calculate the first reward feedback and the weight used to calculate the second reward feedback can be the same or different.

[0144] S402: Input the first reward feedback and the second reward feedback into the first decision model and the second decision model respectively to update the models.

[0145] After obtaining the first reward feedback and the second reward feedback, the first decision model can be adjusted based on the first reward feedback, and the second decision model can be adjusted based on the second reward feedback. Specifically, the first reward feedback can be input into the first decision model, and the second reward feedback can be input into the second decision model. The first and second decision models can be optimized based on the input reward feedback.

[0146] Optionally, if the first decision model is implemented based on a reinforcement learning model, then the reward feedback can be used as a reward for the decision model, and the first decision model can be adjusted based on the reward mechanism of the reinforcement learning model. In addition, the performance indicators obtained by the decision model update unit may come from multiple rounds of business scheduling processes. Then, when reinforcement learning is performed based on the reward feedback, the reward feedback after each round of business scheduling can be calculated separately, and then the reward feedback can be used as the reward of the decision model to adjust the decision model. In this way, even if the computing cluster changes, the first decision model and the second decision model can be adaptively adjusted through reinforcement learning, so that the scheduling measures determined by the first decision model and the second decision model can adapt to the actual situation of the computing cluster.

[0147] For example, assuming that the decision model update unit obtains the performance indicators after the i-th round of business scheduling (i is a positive integer greater than 1), then when performing reinforcement learning on the first decision model, the job information of the scheduled job in the i-1th round of business scheduling can be used as the input of the first decision model, the target job determined in the i-th round of business scheduling can be used as the action determined by the first decision model, and the first reward feedback calculated based on the performance indicators in the i-th round of business scheduling can be used as the reward to update the first decision model.

[0148] For another example, suppose the decision model update unit obtains the performance indicators after the jth round of business scheduling (j is a positive integer greater than 1). Then, when performing reinforcement learning on the second decision model, the job information of the target job in the j-1th round of business scheduling and the computing cluster's computing resource information can be used as the input of the second decision model, the scheduling strategy determined in the jth round of business scheduling can be used as the action determined by the second decision model, and the second reward feedback calculated based on the performance indicators in the jth round of business scheduling can be used as the reward to update the second decision model.

[0149] exist Figure 2 In the implementation described above, performance indicators can be used to update and optimize multiple decision models to obtain more optimal service scheduling strategies and achieve the goal of optimizing the computing cluster. Different performance indicators often correspond to different optimization directions. In real-world scenarios, it is often necessary to optimize the decision model in a specific direction. Therefore, by setting the weight of each performance indicator (or a unified quantitative performance indicator), the specific optimization direction can be determined.

[0150] For example, assume that the performance indicators include cluster resource utilization, scheduling fairness, and resource utilization balance. If the optimization direction is to improve the resource utilization of the computing cluster, then in the weights used to calculate the reward feedback (first reward feedback and / or second reward feedback), the weight corresponding to cluster resource utilization can be greater than the weight corresponding to scheduling fairness, or greater than the weight corresponding to resource utilization balance.

[0151] In some possible application scenarios, the optimization direction may change. Accordingly, the decision model update unit can adaptively adjust the weight of the reward feedback according to the optimization direction.

[0152] For example, before time t, the administrator of the computing cluster may want to improve the resource utilization of the computing cluster. In this case, the weight corresponding to the resource utilization may be greater than the weight corresponding to scheduling fairness and greater than the weight corresponding to resource utilization balance. The decision model update unit updates the first decision model and the second decision model in the direction of improving the resource utilization of the computing cluster.

[0153] After time t, the computing cluster's administrators may wish to improve the cluster's scheduling fairness. This requires the decision model update unit to update multiple decision models in the direction of improving the cluster's scheduling fairness. Accordingly, the decision model update unit can adaptively adjust the weights of the computing reward indicators. Specifically, it can increase the weight corresponding to scheduling fairness and adaptively decrease the weights corresponding to resource utilization and / or resource utilization balance.

[0154] Specifically, if the optimization direction of the computing cluster needs to be changed, an optimization direction adjustment operation can be triggered on the decision model update client. The optimization direction adjustment operation is used to adjust the optimization direction of the computing cluster. The decision model update client can be a client corresponding to the decision model update unit, which is used to trigger the optimization direction adjustment operation. After obtaining the optimization direction adjustment operation, the decision model update client can send the optimization direction adjustment operation to the decision model update unit. The decision model update unit can adjust the weight according to the optimization direction adjustment operation. Optionally, the user can set the updated weight on the decision model update client. The decision model update unit can use the feedback adjustment set by the user instead of the original weight calculation reward feedback.

[0155] This allows you to adjust the weights of reward feedback by triggering an optimization direction adjustment. This adjustment allows you to get different reward feedback even based on the same performance metric, thereby updating the decision model in different directions and changing the optimization direction of the computing cluster.

[0156] The embodiment of the present application also provides a service scheduling strategy determination device. The service scheduling strategy determination device can be applied to Figure 1a or Figure 1b The computing cluster 30 in the implementation shown. Specifically, Figure 5 As shown, the service scheduling strategy determination device 500 includes:

[0157] A first determining unit 510 is configured to input job information of a plurality of jobs to be scheduled into a first decision model; the first decision model selects at least one target job from the plurality of jobs to be scheduled based on the job information of the plurality of jobs to be scheduled;

[0158] A second determining unit 520 is configured to input the job information of the target job and the information of the computing resources of the computing cluster into a second decision model; the second decision model outputs a scheduling policy for the target job based on the job information of the target job and the information of the computing resources, the scheduling policy including information of the computing resources for executing the target job;

[0159] The scheduling unit 530 is configured to send the scheduling policy to the scheduler, and instruct the scheduler to schedule the target job to the corresponding computing resource for execution according to the scheduling policy.

[0160] The first determination unit 510, the second determination unit 520, and the scheduling unit 530 can all be implemented in software or hardware. For example, the implementation of the scheduling unit 530 will be described below using the scheduling unit 530 as an example. Similarly, the implementation of the first determination unit 510 and the second determination unit 520 can refer to the implementation of the scheduling unit 530.

[0161] As an example of a software functional unit, the scheduling unit 530 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the scheduling unit 530 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region may include multiple AZs.

[0162] Similarly, multiple hosts / virtual machines / containers running the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0163] As an example of a hardware functional unit, the scheduling unit 530 may include at least one computing device, such as a server. Alternatively, the scheduling unit 530 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0164] The multiple computing devices included in the scheduling unit 530 can be distributed in the same region or in different regions. The multiple computing devices included in the scheduling unit 530 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the scheduling unit 530 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, and other computing devices.

[0165] It should be noted that, in other embodiments, the first determination unit 510 is used to execute any step in the service scheduling policy determination method, the second determination unit 520 is used to execute any step in the service scheduling policy determination method, and the scheduling unit 530 can be used to execute any step in the service scheduling policy determination method. The steps that the first determination unit 510, the second determination unit 520, and the scheduling unit 530 are responsible for implementing can be specified as needed. By implementing different steps in the service scheduling policy determination method based on the first determination unit 510, the second determination unit 520, and the scheduling unit 530, all functions of the service scheduling policy determination device are realized.

[0166] This application also provides a computing device. Figure 5 As shown, computing device 100 includes a bus 102, a processor 104, a memory 106, and a communication interface 108. Processor 104, memory 106, and communication interface 108 communicate with each other via bus 102. Computing device 600 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computing device 100.

[0167] The bus 102 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 The fact that only one line is used in the figure does not mean that there is only one bus or only one type of bus. Bus 102 may include a path for transmitting information between various components of computing device 100 (eg, memory 106, processor 104, communication interface 108).

[0168] The processor 104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0169] The memory 106 may include volatile memory, such as random access memory (RAM). The processor 104 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0170] The memory 106 stores executable program code, and the processor 104 executes the executable program code to implement the functions of the first determination unit 510, the second determination unit 520, and the scheduling unit 530, respectively, thereby implementing the service scheduling policy determination method. In other words, the memory 106 stores instructions for executing this stored method.

[0171] Alternatively, the memory 106 stores executable code, and the processor 104 executes the executable code to implement the functions of the aforementioned service scheduling policy determination device, thereby implementing the service scheduling policy determination method. That is, the memory 106 stores instructions for executing the service scheduling policy determination method.

[0172] The communication interface 108 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 100 and other devices or a communication network.

[0173] Embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0174] like Figure 7 As shown, the computing device cluster includes at least one computing device 100. The memory 106 in one or more computing devices 100 in the computing device cluster may store the same instructions for executing the service scheduling policy determination method.

[0175] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing the service scheduling policy determination method. In other words, the combination of one or more computing devices 100 can jointly execute instructions for executing the basic service scheduling policy determination method.

[0176] It should be noted that the memory 106 in different computing devices 100 in the computing device cluster may store different instructions, each for executing part of the functions of the service scheduling policy determination apparatus 500. That is, the instructions stored in the memory 106 in different computing devices 100 may implement the functions of one or more of the first determination module 510, the second determination module 520, and the scheduling module 530.

[0177] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network, which may be a wide area network or a local area network. Figure 8 A possible implementation is shown. Figure 8As shown, two computing devices 100A and 100B are connected via a network. Specifically, the connection to the network is achieved through a communication interface in each computing device. In this possible implementation, the memory 106 in computing device 100A stores instructions for executing the first determination module 510 and the second determination module 520. Simultaneously, the memory 106 in computing device 100B stores instructions for executing the functions of the scheduling module 530.

[0178] Figure 8 The connection between the computing device clusters shown can be based on the fact that the service scheduling policy determination method provided in this application can be divided into two parts: calling a decision model and interacting with the computing cluster. Therefore, it is considered that the function of calling the decision model is performed by computing device 100A, and the function of interacting with the computing cluster is performed by computing device 100B. It is understood that if the service scheduling policy determination apparatus further includes a decision model updating unit, the instructions of the decision model updating unit can be stored in the memory 106 of computing device 100B.

[0179] It should be understood that Figure 8 The functions of the computing device 100A shown in FIG. 1 may also be completed by multiple computing devices 100. Similarly, the functions of the computing device 100B may also be completed by multiple computing devices 100.

[0180] The present application embodiment also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similarly referred to as Figure 7 and Figure 8 The connection mode of the computing device cluster is different in that the memory 106 of one or more computing devices 100A in the computing device cluster may store the same instructions for executing the service scheduling strategy determination method.

[0181] In some possible implementations, the memory of one or more computing devices 100B in the computing device cluster may also store partial instructions for executing the service scheduling strategy determination method. In other words, the combination of one or more computing devices can jointly execute the instructions for executing the service scheduling strategy determination method.

[0182] It should be noted that the memory 106 in different computing devices 100A in the computing device cluster may store different instructions for executing part of the functions of the service scheduling policy determination apparatus. In other words, the instructions stored in the memory 106 in different computing devices 100A may implement the functions of one or more units in the service scheduling policy determination apparatus.

[0183] Embodiments of the present application also provide a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the service scheduling policy determination method.

[0184] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the service scheduling policy determination method, or instruct the computing device to execute the service scheduling policy determination method.

[0185] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for determining a service scheduling strategy, characterized in that: The method comprises: Inputting job information of a plurality of jobs to be scheduled into a first decision model; The first decision model selects at least one target job from the multiple jobs to be scheduled according to the job information of the multiple jobs to be scheduled; Inputting the job information of the target job and the information of computing resources of the computing cluster into a second decision model; The second decision model outputs a scheduling policy for the target job based on the job information of the target job and the information of the computing resources, wherein the scheduling policy includes information of the computing resources for executing the target job; The scheduling policy is sent to the scheduler, instructing the scheduler to schedule the target job to the corresponding computing resource for execution according to the scheduling policy.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining processing resource information of the computing cluster and inputting it into a first decision model; The first decision model selects at least one target job from the multiple jobs to be scheduled according to the job information of the multiple jobs to be scheduled, including: The first decision model selects at least one target job from the multiple jobs to be scheduled according to job information of the multiple jobs to be scheduled and processing resource information of the computing cluster.

3. The method according to claim 1 or 2, characterized in that The method further comprises: Inputting information about storage resources and network resources of the computing cluster into the second decision model; The scheduling strategy output by the second decision model also includes information about storage resources and network resources for executing the target job.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Generate a first reward feedback for the first decision model and a second reward feedback for the second decision model according to the execution result of the scheduling strategy of the scheduler; The first reward feedback and the second reward feedback are input into the first decision model and the second decision model respectively to update the models.

5. The method according to any one of claims 1 to 4, characterized in that Generating the first reward feedback of the first decision model and the second reward feedback of the second decision model according to the execution result of the scheduling strategy of the scheduler includes: Determining a plurality of performance indicators according to the execution results of the scheduling strategy of the scheduler; The first reward feedback and the second reward feedback are generated according to the multiple performance indicators and the weight of each performance indicator.

6. The method according to claim 5, characterized in that The performance indicators include cluster resource utilization, scheduling fairness, and resource utilization balance. The execution results of the scheduling strategy of the scheduler and the generation of the first reward feedback and the second reward feedback based on the multiple performance indicators and the scheduler include: Uniformly quantifying the cluster resource utilization, the scheduling fairness, and the resource utilization balance; The first reward feedback and the second reward feedback are generated according to the uniformly quantified cluster resource utilization and the weight of the cluster resource utilization, the uniformly quantified scheduling fairness and the weight of the scheduling fairness, and the uniformly quantified resource utilization balance and the weight of the resource utilization balance.

7. The method according to claim 5 or 6, characterized in that The method further comprises: In response to the optimization direction adjustment operation, the weight of at least one of the performance indicators is updated.

8. A device for determining a service scheduling strategy, characterized in that: The device comprises: A first determining unit is configured to input job information of a plurality of jobs to be scheduled into a first decision model; the first decision model selects at least one target job from the plurality of jobs to be scheduled based on the job information of the plurality of jobs to be scheduled; a second determining unit, configured to input the job information of the target job and the information of the computing resources of the computing cluster into a second decision model; the second decision model outputting a scheduling policy for the target job based on the job information of the target job and the information of the computing resources, the scheduling policy including information of the computing resources for executing the target job; The scheduling unit is used to send the scheduling policy to the scheduler, instructing the scheduler to schedule the target job to the corresponding computing resource for execution according to the scheduling policy.

9. The device according to claim 8, characterized in that The device further includes an acquisition unit, The acquisition unit is configured to acquire processing resource information of the computing cluster and input the information into a first decision model; The first decision model is specifically configured to select at least one target job from the multiple jobs to be scheduled according to job information of the multiple jobs to be scheduled and processing resource information of the computing cluster.

10. The device according to claim 8 or 9, characterized in that The second determination unit is specifically used to input the storage resource information and network resource information of the computing cluster into the second decision model; the scheduling strategy output by the second decision model also includes the storage resource information and network resource information for executing the target job.

11. The device according to any one of claims 8 to 10, characterized in that The device also includes a decision model updating unit; The decision model updating unit is configured to generate a first reward feedback of the first decision model and a second reward feedback of the second decision model according to the execution result of the scheduling policy of the scheduler; The first reward feedback and the second reward feedback are input into the first decision model and the second decision model respectively to update the models.

12. The device according to any one of claims 8 to 11, characterized in that The decision model updating unit is specifically configured to determine a plurality of performance indicators according to an execution result of the scheduling policy of the scheduler; and generate the first reward feedback and the second reward feedback according to the plurality of performance indicators and a weight of each of the performance indicators.

13. The device according to claim 12, characterized in that The performance indicators include cluster resource utilization, scheduling fairness and resource utilization balance, The decision model updating unit is specifically used to uniformly quantify the cluster resource utilization, the scheduling fairness, and the resource utilization balance respectively; and generate the first reward feedback and the second reward feedback according to the uniformly quantified cluster resource utilization and the weight of the cluster resource utilization, the uniformly quantified scheduling fairness and the weight of the scheduling fairness, and the uniformly quantified resource utilization balance and the weight of the resource utilization balance.

14. The device according to claim 12 or 13, characterized in that The decision model updating unit is further configured to update the weight of at least one of the performance indicators in response to an optimization direction adjustment operation.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, which, when executed on a computing device, enable the computing device to perform the method according to any one of claims 1 to 7.

16. A computer program product comprising instructions which, when executed on a computing device, cause the computing device to perform the method according to any one of claims 1 to 7.

Images