A task scheduling implementation method, system and computer readable medium

Abstract

The application provides a task scheduling implementation method, system and computer readable medium. The task scheduling implementation method comprises the following steps: in response to an access request containing a response task, a set of scheduling parameters is configured for a task scheduler, and in response to the access request, a task score is calculated, the task is determined to be in a task queue, a release order of the task is determined, and a resource definition parameter is configured; a listening operation is performed on the task to listen to the execution state corresponding to the task, the resource consumption of a Kubernetes cluster and the number of tasks in execution in the task scheduler are determined, when the execution condition is met, the task is released to the Kubernetes cluster based on the release order, a job resource is generated, and the Pod corresponding to the job resource is scheduled based on the preconfigured Pod scheduling strategy to execute the task. Through the application, accurate implementation of task scheduling and stable and efficient task execution are realized.

Description

Technical Field

[0001] This invention relates to the field of task scheduling technology, and in particular to a task scheduling implementation method, system, and computer-readable medium. Background Technology

[0002] Kubernetes (K8s) is a portable and scalable container orchestration and management tool designed for container services. Essentially, it's a cluster of servers where specific programs can run on each node to manage containers, automating resource management. Task scheduling, on the other hand, refers to the process by which a system automatically executes specific tasks at predetermined times. Task scheduling frees up human labor, allowing the system to automatically execute tasks, thus forming a task scheduling system.

[0003] Generally, there are two solutions for task scheduling in a Kubernetes cluster: one is to use third-party open-source task scheduling products (such as Quartz, XXL-JOB, Elastic-Job, etc.); the other is to develop a custom scheduling product. However, when developing third-party open-source task scheduling products, due to the diverse nature of user business systems, and the need to adapt to various user business systems, the products can only guarantee some common functions, resulting in poor adaptability to different user business systems. Furthermore, users cannot perform secondary development on open-source products based on the functions required by their own business systems, leading to insufficient scalability. On the other hand, developing a custom scheduling product involves high development costs, and task orchestration, sharding, retries, and data cleanup must all be handled manually, resulting in high operational costs.

[0004] In view of this, it is necessary to improve the existing task scheduling methods to solve the above problems. Summary of the Invention

[0005] The purpose of this invention is to solve the problems of insufficient compatibility and scalability with user business systems when implementing task scheduling through third-party open-source scheduling products, as well as the problem of high development and maintenance costs when implementing task scheduling through self-developed scheduling products.

[0006] To achieve the above objectives, in a first aspect, the present invention provides a task scheduling implementation method, which, in response to an access request containing a response task,

[0007] include:

[0008] Configure a set of scheduling parameters for the task scheduler. The task scheduler responds to access requests by calculating task scores based on the set of scheduling parameters, determining the order in which tasks are sent in the task queue of the task scheduler based on the task scores, and configuring the resource definition parameters corresponding to the tasks.

[0009] The task scheduler performs a listening operation on the task to monitor the execution status of the task and determines the resource consumption of the Kubernetes cluster and the number of tasks in execution in the task scheduler based on the execution status. When the resource consumption and the number of tasks in execution in the task scheduler meet the execution conditions, the task scheduler issues the task to the Kubernetes cluster according to the issuance order. The Kubernetes cluster generates job resources according to the resource definition parameters and schedules the Pods corresponding to the job resources according to the Pod scheduling policy pre-configured in the Kubernetes cluster to execute the task.

[0010] As a further improvement of the present invention, before calculating the task score based on the scheduling parameter set, the method further includes:

[0011] The task is decomposed to obtain multiple subtasks corresponding to the task, and task objects corresponding to the task and subtasks are generated respectively. The task objects are then stored in the storage unit.

[0012] As a further improvement of the present invention, the listening operation is implemented by establishing a long connection between the task scheduler and the task;

[0013] The task scheduler updates the execution status of the task to the storage unit.

[0014] As a further improvement of the present invention, the scheduling parameter set includes queue parameters, which include the scheduling queue length and / or the number of concurrent tasks.

[0015] As a further improvement of the present invention, the scheduling parameter set also includes task parameters, which include one or any combination of task priority, average task execution time and task complexity.

[0016] As a further improvement of the present invention, the resource consumption and the number of tasks being executed in the task scheduler satisfy the execution condition specifically as follows:

[0017] The Kubernetes cluster consists of nodes with CPU utilization <70%, memory utilization <80%, disk utilization <70%, and the number of tasks running in the task scheduler < the number of concurrent tasks.

[0018] As a further improvement of the present invention, the calculation of the task score based on the scheduling parameter set is achieved by weighting and summing the task priority score, the task average time score, and the task complexity score, specifically calculated by the following formula:

[0019] Task score = x * task priority score + y * average task time score + z

[0020] *Task complexity score;

[0021] Where x, y, and z are the weighting coefficients of task priority score, average task time score, and task complexity score, respectively, and x+y+z=100%.

[0022] As a further improvement of the present invention, the resource definition parameters include one or any combination of the following: the number of tasks to be completed, the number of tasks to be executed in parallel, the task execution timeout, the number of task retries, the automatic resource cleanup time after the job is completed, and the subtask index number.

[0023] Secondly, the present invention also provides a task scheduling implementation system, comprising:

[0024] One or more processors and storage devices;

[0025] The storage device is coupled to the one or more processors;

[0026] The storage device stores computer program instructions that are executed by one or more processors, the computer program instructions executing the task scheduling implementation method as described in any of the first aspects.

[0027] Thirdly, the present invention provides a computer-readable medium storing computer program instructions, which, when read and executed by a processor, perform the task scheduling implementation method as described in any one of the first aspects.

[0028] Compared with the prior art, the beneficial effects of the present invention are:

[0029] By configuring a set of scheduling parameters, a task score is calculated based on the parameter set. The task score determines the order in which tasks are sent in the task queue of the task scheduler. Resource definition parameters corresponding to the tasks are configured. The task scheduler performs a listening operation on the tasks to monitor their execution status. Based on the execution status, it determines the resource consumption of the Kubernetes cluster and the number of tasks in execution in the task scheduler. When the resource consumption of the Kubernetes cluster and the number of tasks in execution in the task scheduler meet the execution conditions, the task scheduler sends the tasks to the Kubernetes cluster according to the sending order, thereby improving the accuracy and real-time nature of task scheduling. The Kubernetes cluster generates job resources according to the resource definition parameters and schedules the Pods corresponding to the jobs according to the pre-configured Pod scheduling policy of the Kubernetes cluster to execute the tasks. Pod scheduling is implemented using the kube-scheduler component of the Kubernetes cluster, achieving stable and efficient task execution and solving the problem of high development and maintenance costs associated with developing scheduling products for task scheduling in existing technologies. Attached Figure Description

[0030] Figure 1 This is a schematic diagram illustrating the steps of a task scheduling implementation method according to the present invention;

[0031] Figure 2 A topology diagram for running the task scheduling implementation method shown in this invention in a scenario based on users, nodes, and Kubernetes clusters;

[0032] Figure 3 A schematic diagram of the scheduling parameter set;

[0033] Figure 4 This is a diagram illustrating the disassembly operation performed on the task.

[0034] Figure 5 Implement the system topology for task scheduling;

[0035] Figure 6 A topology diagram for computer-readable media. Detailed Implementation

[0036] The present invention will now be described in detail with reference to the embodiments shown in the accompanying drawings. However, it should be noted that these embodiments are not intended to limit the present invention. Equivalent changes or substitutions in function, method, or structure made by those skilled in the art based on these embodiments are all within the scope of protection of the present invention.

[0037] Please refer to Figures 1 to 4As shown, the present invention shows a specific implementation of a task scheduling implementation method. The application scenario of a task scheduling implementation method disclosed in this embodiment is as follows: Deploy a task scheduler 10 inside a node 100, configure a scheduling parameter set for the task scheduler 10, and the task scheduler 10 responds to an access request containing a response task, calculates a task score based on the scheduling parameters, determines the order of task distribution in the task queue of the task scheduler 10 according to the task score, and configures resource definition parameters corresponding to the task; The task scheduler 10 performs a listening operation on the task to monitor the execution status corresponding to the task, and determines the resource consumption of the Kubernetes cluster 200 and the number of tasks in execution in the task scheduler 10 according to the execution status. When the resource consumption and the number of tasks in execution meet the execution conditions, the task scheduler 10 distributes tasks to the Kubernetes cluster 200 based on the distribution order. The Kubernetes cluster 200 generates job resources according to the resource definition parameters and performs scheduling on the Pod corresponding to the job resource based on the pre-configured Pod scheduling policy of the Kubernetes cluster 200 to execute the task. By self-configuring the scheduling parameter set, determining the distribution order in the task queue according to the scheduling parameter set, and performing scheduling on the Pod corresponding to the job resource by the pre-configured Pod scheduling policy of the Kubernetes cluster 200, it is possible to solve the problems of lack of adaptability to the user business system and insufficient scalability existing in task scheduling through third-party scheduling open-source products in the prior art, as well as the problems of high development and maintenance costs existing in task scheduling through self-developed scheduling products.

[0038] As shown Figure 1 A task scheduling implementation method, in response to an access request containing a response task, at least includes the following steps S1 to step S2.

[0039] Step S1: Configure a scheduling parameter set for the task scheduler. The task scheduler responds to the access request, calculates a task score based on the scheduling parameter set, determines the order of task distribution in the task queue of the task scheduler according to the task score, and configures resource definition parameters corresponding to the task.

[0040] Exemplarily, as shown Figure 2As shown, a task scheduler 10 is deployed within node 100, which manages the Kubernetes cluster 200 consisting of control node 20 and business nodes 30. A Pod scheduler 201 is deployed within control node 20. Business nodes 30 are defined from business node-1 to business node-n. When a user sends an access request containing a response task to node 100, the task scheduler deployed on node 100 responds to the request and distributes the task to control node 20 of the Kubernetes cluster 200. The Pod scheduler 201 (i.e., the kube-scheduler component) deployed on control node 20 generates the job resource corresponding to the task and schedules the Pod corresponding to the job resource to execute the task. Before distributing the task to the Kubernetes cluster 200, the task scheduler 10 creates a corresponding task object for the task, stores the task object in storage unit 40, and performs a listening operation on the task to monitor its execution status and update the execution status in storage unit 40.

[0041] It should be noted that the task scheduler 10 manages the Kubernetes cluster 200. There can be one or multiple Kubernetes clusters 200. When multiple Kubernetes clusters 200 are deployed, the task scheduler 10 manages all of them simultaneously, distributing tasks to each cluster. Regarding the deployment location of the task scheduler 10, it is deployed on node 100. Node 100 can be independent of the Kubernetes cluster 200, or it can be a control node within the Kubernetes cluster 200. When node 100 is the control node of Kubernetes cluster 200, task scheduler 10 is deployed on control node 20, and Pod scheduler 201 is also deployed on control node 20. Task scheduler 10, deployed on control node 20, responds to user access requests containing response tasks, calculates task scores based on scheduling parameter sets, determines the task's delivery order in the task queue based on the task score, and delivers the task to Pod scheduler 201 according to the delivery order. Pod scheduler 201 generates job resources corresponding to the task and schedules the Pod corresponding to the job resources to business node 30 to execute the task. Storage unit 40 can be deployed independently of node 100 and Kubernetes cluster 200, or it can be deployed on node 100 or Kubernetes cluster 200, as long as it can store task objects and subsequently store the execution status of the tasks. This embodiment does not limit this.

[0042] Specifically, with Figure 2 The deployment location shown is used as an example for illustration. The user configures a set of scheduling parameters for the task scheduler 10 deployed on node 100. These parameters include queue parameters and task parameters. The queue parameters include the scheduling queue length (priorityQueueLength) and / or the number of concurrent tasks (concurrentTaskCount). The task parameters include one or any combination of task priority, average task execution time (ExecuteTime), and task complexity (Complexity). The user sends an access request containing the response task to node 100. The task scheduler 10 deployed on node 100 responds to this access request, performs a task decomposition operation, obtains multiple subtasks corresponding to the task, generates task objects corresponding to the task and its subtasks, and stores the task objects in storage unit 40. For example, parameters... Figure 4 As shown, a decomposition operation is performed on task 1, which is decomposed into subtask 11, subtask 12 and subtask 13, and task objects 1, 12 and 13 corresponding to task 1, subtask 11, subtask 12 and subtask 13 are generated respectively. The generated task objects 1, 12 and 13 are stored in storage unit 40.

[0043] The task score, calculated based on the scheduling parameter set, is achieved by weighting and summing the task priority score, average execution time score, and task complexity score. The task priority is set by the task provider, with priority levels ranging from 1 to 5, where level 5 is the highest. The task priority score is calculated as priority * 20, with a maximum score of 100. The average execution time is determined by the average execution time of the task type, for example, calculated based on the execution times of the most recent 100 tasks of that type, in seconds (s). Task types can include creation, deletion, update, and modification. A 10-second threshold is used: if the average execution time is greater than 10 seconds, the average execution time score is set to 50; if it is less than or equal to 10 seconds, the score is set to 100. The task complexity is determined by the number of subtasks. The task priority score is calculated as (number of subtasks - 1) * 20, with a maximum score of 100. The specific formula for calculating the task score by weighting and summing the task priority score, average execution time score, and task complexity score is as follows:

[0044] Task score = x * task priority score + y * average task time score + z

[0045] *Task complexity score;

[0046] Where x, y, and z are the weighting coefficients of task priority score, average task time score, and task complexity score, respectively, and x+y+z=100%.

[0047] For example, a task is broken down into 9 subtasks. The task submitter sets the task priority to level 3, and the average execution time for this task type is 20 seconds. Task priority score = priority * 20 = 3 * 20 = 60; the average execution time is 20 seconds, and 20 > 10, so the average time score is 50; Task priority score = (number of subtasks - 1) * 20 = (9 - 1) * 20 = 160. Since the highest score for task complexity is 100, and 160 > 100, the task priority score is directly set to 100. Taking 60% for x, 20% for y, and 20% for z to calculate the task score, we get: Task score = 60% * 60 + 20% * 50 + 20% * 100 = 66. Therefore, the task score is 66.

[0048] The task allocation order in the task queue of task scheduler 10 is determined based on the task score, from highest to lowest. The higher the task score, the earlier the task is allocated in the task queue. Meanwhile, the length of the task queue and the number of concurrent tasks are determined by the scheduling queue length and the number of concurrent tasks mentioned above. Generally, the scheduling queue length is set to 50, and the number of concurrent tasks is set to 30.

[0049] After determining the order in which tasks are dispatched in the task queue, the task scheduler 10 configures the resource definition parameters corresponding to the tasks. These parameters include one or any combination of the following: the number of tasks to be completed, the number of tasks to be executed in parallel, the task execution timeout, the number of task retries, the automatic resource cleanup time after the job is completed, and the subtask index number. For example, see the following code:

[0050] apiVersion:batch / v1

[0051] kind:Job

[0052] metadata:

[0053] name:'job-task-<task UUID>'

[0054] spec:

[0055] completions:<number of tasks> # Number of tasks to be completed

[0056] parallelism: <number of tasks> # The number of tasks to execute in parallel. The default value is the number of tasks, and the maximum value is 5. If the number of tasks is greater than 5, it will be set to 5.

[0057] activeDeadlineSeconds: <Maximum task execution time> # Task execution timeout, set as the maximum execution time for this type of task.

[0058] backoffLimit: 3 # Number of retries

[0059] ttlSecondsAfterFinished: Automatic resource cleanup time after job #1800 is completed, default 30 minutes.

[0060] completionMode:Indexed

[0061] template:

[0062] spec:

[0063] restartPolicy:OnFailure

[0064] containers:

[0065] -name:'worker'

[0066] image:'worker'

[0067] env:

[0068] -name:taskId

[0069] value:<task UUID> # The UUID of the task to be executed, passed as an environment variable.

[0070] volumeMounts:

[0071] -mountPath: / worker

[0072] name:subTaskInfo

[0073] volumes:

[0074] -name:subTaskInfo

[0075] downwardAPI:

[0076] tems:

[0077] -path:"subtaskIndex.txt"# Subtask index number, subtask id: <task UUID>-<subtask index number>

[0078] fieldRef:

[0079] fieldPath:metadata.annotations['batch.kubernetes.io / job-completion-index']

[0080] Step S2: The task scheduler performs a listening operation on the task to monitor the execution status of the task and determines the resource consumption of the Kubernetes cluster and the number of tasks in execution in the task scheduler based on the execution status. When the resource consumption and the number of tasks in execution in the task scheduler meet the execution conditions, the task scheduler sends the task to the Kubernetes cluster according to the distribution order. The Kubernetes cluster generates job resources according to the resource definition parameters and schedules the Pods corresponding to the job resources according to the Pod scheduling policy pre-configured in the Kubernetes cluster to execute the task.

[0081] Specifically, in combination Figure 2As shown, a long connection is established between the task scheduler 10 and the Kubernetes cluster 200 to enable the task scheduler 10 to perform a listening operation on tasks. The condition for triggering the task listening operation is that the task has been sent to the Kubernetes cluster 200, that is, the task scheduler 10 performs a listening operation on the task sent to the Kubernetes cluster 200 by the task scheduler 10. Through this listening operation, the execution status of the corresponding task is monitored. The execution status can be, for example, in progress, not executed, completed and successful, completed and failed, etc. At the same time, the task scheduler 10 updates the execution status of the monitored task to the storage unit 40. Based on the execution status of the task, the resource consumption of the Kubernetes cluster 200 and the number of tasks in execution in the task scheduler 10 are determined. It is then determined whether the resource consumption of the Kubernetes cluster 200 and the number of tasks in execution in the task scheduler 10 meet the execution conditions. If yes, the task scheduler 10 sends the task to the Kubernetes cluster 200 according to the sending order; if not, the operation of sending the task to the Kubernetes cluster 200 is not performed. Specifically, the resource consumption of Kubernetes cluster 200 and the number of tasks running in task scheduler 10 meet the following execution conditions: the CPU utilization of each node in Kubernetes cluster 200 (i.e., control node 20 and business node 30) is <70%, memory utilization is <80%, disk utilization is <70%, and the number of tasks running in task scheduler 10 is < the number of concurrent tasks. The number of tasks running in task scheduler 10 is the same as the number of tasks running in Kubernetes cluster 200. If multiple Kubernetes clusters are deployed, the number of tasks running in task scheduler 10 is the sum of the number of tasks running in all the Kubernetes clusters.

[0082] When the resource consumption of Kubernetes cluster 200 and the number of tasks being executed in task scheduler 10 meet the execution conditions, task scheduler 10 issues tasks to Kubernetes cluster 200 based on the issuance order. Kubernetes cluster 200 generates job resources corresponding to the tasks according to the resource definition parameters configured by task scheduler 10. Pod scheduler 201 deployed on control node 20 schedules the Pods corresponding to the job resources according to the Pod scheduling policy pre-configured by Kubernetes cluster 200, so as to schedule the Pods to the appropriate business nodes 30 to execute the tasks.

[0083] The pre-configured Pod scheduling strategy consists of a pre-selection algorithm and a selection algorithm. The pre-selection algorithm selects suitable business nodes in the Kubernetes cluster and filters out cluster nodes that do not meet the conditions, such as insufficient resources, abnormal status, and affinity mismatch. The selection algorithm scores the selected business nodes and selects the business node with the highest score as the target node for Pod scheduling. That is, the Pod is scheduled to the target node. Resources, load, cache, and the image required by the Pod may all affect the score of the business node.

[0084] In addition, the Kubernetes cluster 200 deploys the API server component, Controller Manager component, kuber-scheduler component, and kubelet component. The API server component stores job resources in etcd, the Controller Manager component analyzes the Pod resource objects corresponding to the job, the kuber-scheduler component schedules the Pod to the appropriate business node 30, and the kubelet component creates and maintains the Pod.

[0085] In this application, a scheduling parameter set is configured by the user, a task score is calculated based on the scheduling parameter set, the task order in the task queue of the task scheduler 10 is determined according to the task score, and the resource definition parameters corresponding to the task are configured. The task scheduler 10 performs a listening operation on the task to monitor the execution status of the task and determines the resource consumption of the Kubernetes cluster 200 and the number of tasks in execution in the task scheduler 10 based on the execution status. When the resource consumption of the Kubernetes cluster 200 and the number of tasks in execution in the task scheduler 10 meet the execution conditions, the task scheduler 10 sends the task to the Kubernetes cluster 200 according to the order of issuance. This improves the accuracy and real-time nature of task scheduling and solves the problems of insufficient adaptability and scalability of task scheduling implemented by third-party open-source scheduling products in the prior art. Kubernetes cluster 200 generates job resources based on resource definition parameters, and schedules the Pod corresponding to the job based on the Pod scheduling policy pre-configured by Kubernetes cluster 200 to execute the task. It uses the kube-scheduler component of Kubernetes cluster 200 (i.e., Pod scheduler 201) to implement Pod scheduling, thereby achieving stable and efficient task execution and solving the problem of high development and maintenance costs in the existing technology of developing scheduling products to implement task scheduling.

[0086] Combination Figure 5As shown in the figure, based on the technical solution of the task scheduling implementation method disclosed in the above embodiments, this embodiment also discloses a task scheduling implementation system 500.

[0087] A task scheduling implementation system 500 includes:

[0088] One or more processors 51; and a storage device 52, the storage device 52 being coupled to one or more processors 51;

[0089] The storage device 52 stores computer program instructions that are executed by one or more processors. These computer program instructions perform the steps in a task scheduling implementation method as disclosed in the above embodiments. Specifically, the storage device 52 may include one or more memories, namely memories 521 to 52j, where the parameter j is a positive integer greater than or equal to 2. The storage device 52 is coupled to one or more processors 51 to the system bus 53 to perform data / signal communication.

[0090] The task scheduling implementation system 500 can be considered as a data center, a physical machine cluster, a standalone physical machine, a wearable electronic device, or a cloud server. The aforementioned processor 51 can be a physical or virtual processor (e.g., a physical CPU or vCPU), and the aforementioned storage device 52 can be a physical or virtual storage device (e.g., a NAND storage device or a virtual disk).

[0091] The technical solutions in this embodiment that have the same parts as those in the above embodiments are described in the above embodiments and will not be repeated here.

[0092] Combination Figure 6 As shown in the figure, this embodiment also discloses a computer-readable medium 900.

[0093] The computer-readable medium 900 may be configured, wholly or partially, in a physical computer, server, cluster server, or data center.

[0094] In this embodiment, a computer-readable medium 900 stores computer program instructions 901. These instructions 901 are read and executed by a processor read 902 to perform the steps of a task scheduling implementation method as described in the foregoing embodiments. Optionally, the computer-readable medium 900 may be configured as a server, and the server may run on a physical device used to build a private cloud, hybrid cloud, or public cloud.

[0095] Furthermore, the computer-readable medium 900 can also be configured as a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The computer-readable medium 900 is used to store a program, and after receiving an execution instruction, the processor 902 executes the steps in the task scheduling implementation method disclosed in the above embodiments.

[0096] Meanwhile, the processor 902 disclosed in this embodiment may be an integrated circuit chip with signal processing capabilities. The processor 902 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. Technical solutions with the same parts as any of the above embodiments are described in the above embodiments and will not be repeated here.

[0097] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent embodiments or modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

[0098] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0099] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A task scheduling implementation method, responding to an access request containing a response task, Its features are, include: Configure a set of scheduling parameters for the task scheduler. The task scheduler responds to access requests by calculating task scores based on the set of scheduling parameters, determining the order in which tasks are sent in the task queue of the task scheduler based on the task scores, and configuring the resource definition parameters corresponding to the tasks. The task scheduler performs a listening operation on the task to monitor the execution status of the task and determines the resource consumption of the Kubernetes cluster and the number of tasks in execution in the task scheduler based on the execution status. When the resource consumption and the number of tasks in execution in the task scheduler meet the execution conditions, the task scheduler sends the task to the Kubernetes cluster based on the sending order. The Kubernetes cluster generates job resources according to the resource definition parameters and schedules the Pods corresponding to the job resources according to the Pod scheduling policy pre-configured in the Kubernetes cluster to execute the task. The formula for calculating the task score is as follows: x, y, and z are the weighting coefficients for the task priority score, the average task time score, and the task complexity score, respectively, and x + y + z = 100%; The resource definition parameters include one or any combination of the following: the number of tasks to be completed, the number of tasks to be executed in parallel, the task execution timeout, the number of task retries, the automatic resource cleanup time after the job is completed, and the subtask index number; The resource consumption and the number of tasks in execution in the task scheduler meet the execution conditions as follows: the CPU utilization of each node in the Kubernetes cluster is less than 70%, the memory utilization is less than 80%, the disk utilization is less than 70%, and the number of tasks in execution in the task scheduler is less than the number of concurrent tasks.

2. The task scheduling implementation method according to claim 1, characterized in that, Before calculating the task score based on the scheduling parameter set, the following is also included: The task is decomposed to obtain multiple subtasks corresponding to the task, and task objects corresponding to the task and subtasks are generated respectively. The task objects are then stored in the storage unit.

3. The task scheduling implementation method according to claim 2, characterized in that, The listening operation is implemented by establishing a long connection between the task scheduler and the task; The task scheduler updates the execution status of the task to the storage unit.

4. The task scheduling implementation method according to claim 1, characterized in that, The scheduling parameter set includes queue parameters, which include the scheduling queue length and / or the number of concurrent tasks.

5. The task scheduling implementation method according to claim 4, characterized in that, The scheduling parameter set also includes task parameters, which include one or any combination of task priority, average task execution time, and task complexity.

6. A task scheduling implementation system, characterized in that, include: One or more processors and storage devices; The storage device is coupled to the one or more processors; The storage device stores computer program instructions that are executed by one or more processors, and the computer program instructions execute the task scheduling implementation method as described in any one of claims 1 to 5.

7. A computer-readable medium, characterized in that, The computer-readable medium stores computer program instructions, which, when read and executed by a processor, perform the task scheduling implementation method as described in any one of claims 1 to 5.

Images