Task resource scheduling method and device, equipment and storage medium

Abstract

The invention relates to the technical field of financial science and technology, and discloses a task resource scheduling method and device, equipment and a storage medium. The method comprises the steps of obtaining a plurality of to-be-executed tasks; according to a preset task model, obtaining a task priority and a task dependency relationship corresponding to each task identifier; determining current available resource information according to a preset resource model; sorting the to-be-executed tasks according to the priority of each task and the dependency relationship of each task to obtain a target task queue; based on the target task queue, the current available resource information and a preset task scheduling strategy, performing resource allocation on each to-be-executed task to obtain a resource allocation result; executing each to-be-executed task according to the resource allocation result; when the execution of each to-be-executed task is finished, obtaining a performance evaluation index corresponding to each to-be-executed task; and adjusting the task scheduling strategy according to each performance evaluation index. According to the embodiment of the invention, the flexibility of resource allocation and task scheduling can be improved.

Description

Technical Field

[0001] The present application relates to the field of financial technology, and in particular to a task resource scheduling method, device, equipment and storage medium. Background Art

[0002] The financial business service orchestration system is a technology used in the field of information technology to automate and coordinate the scheduling and execution of information technology (IT) resources, and task scheduling and execution are key technical solutions.

[0003] Financial services usually involve multiple types of tasks, such as complex transaction processing, risk assessment, and customer service. Each task type has different task requirements and resource requirements. However, due to the limitations of specific application scenarios such as configuration management, application deployment, and orchestration tasks, existing service orchestration systems are usually difficult to flexibly schedule resources and execute tasks according to the dynamic changes of financial services. Summary of the invention

[0004] The embodiments of the present application provide a task resource scheduling method, apparatus, device and storage medium, which can improve the flexibility of resource allocation and task scheduling.

[0005] In a first aspect, an embodiment of the present application provides a task resource scheduling method, the method comprising:

[0006] Get multiple pending tasks with corresponding task identifiers;

[0007] Acquire the task priorities and task dependencies corresponding to the task identifiers according to the preset task model;

[0008] Determine currently available resource information based on a preset resource model;

[0009] Sorting the plurality of tasks to be executed according to the priority of each task and the dependency relationship of each task to obtain a target task queue;

[0010] Based on the target task queue, the currently available resource information and the preset task scheduling strategy, resources are allocated to each of the tasks to be executed to obtain a resource allocation result;

[0011] Execute each of the tasks to be executed according to the resource allocation result;

[0012] When the execution of each of the tasks to be executed is completed, obtaining the performance evaluation indicators corresponding to each of the tasks to be executed;

[0013] The task scheduling strategy is adjusted according to each of the performance evaluation indicators.

[0014] In a second aspect, an embodiment of the present application further provides a task resource scheduling device, which includes:

[0015] A first acquisition unit, used to acquire a plurality of to-be-executed tasks carrying corresponding task identifiers;

[0016] A second acquisition unit, configured to acquire, according to a preset task model, task priorities and task dependencies respectively corresponding to the task identifiers;

[0017] A first determining unit, configured to determine currently available resource information according to a preset resource model;

[0018] A sorting unit, used for sorting the plurality of tasks to be executed according to the priority of each task and the dependency relationship of each task to obtain a target task queue;

[0019] An allocation unit, configured to allocate resources to each of the tasks to be executed based on the target task queue, the currently available resource information and a preset task scheduling strategy, and obtain a resource allocation result;

[0020] An execution unit, configured to execute each of the tasks to be executed according to the resource allocation result;

[0021] A third acquisition unit is used to acquire the performance evaluation index corresponding to each of the tasks to be executed when the execution of each of the tasks to be executed is completed;

[0022] An adjustment unit is used to adjust the task scheduling strategy according to each of the performance evaluation indicators.

[0023] In a third aspect, an embodiment of the present application further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the computer program.

[0024] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0025] The embodiment of the present application provides a task resource scheduling method, device, equipment and storage medium. The method includes: obtaining multiple tasks to be executed with corresponding task identifiers; obtaining the task priorities and task dependencies corresponding to each of the task identifiers according to a preset task model; determining the currently available resource information according to a preset resource model; sorting the multiple tasks to be executed according to each of the task priorities and each of the task dependencies to obtain a target task queue; allocating resources to each of the tasks to be executed based on the target task queue, the currently available resource information and the preset task scheduling strategy to obtain a resource allocation result; executing each of the tasks to be executed according to the resource allocation result; when each of the tasks to be executed is completed, obtaining the performance evaluation indicators corresponding to each of the tasks to be executed; adjusting the task scheduling strategy according to each of the performance evaluation indicators. In the embodiment of the present application, based on the target task queue obtained by sorting multiple tasks to be executed according to the priority of each task to be executed and the task dependency relationship, resources are allocated to each task to be executed based on the target task queue, currently available resource information and a preset task scheduling strategy. After each task to be executed is executed according to the allocated resources, performance evaluation indicators corresponding to each task to be executed are obtained. The task scheduling strategy is adjusted according to each performance evaluation indicator, and resources are subsequently allocated through the adjusted task scheduling strategy, thereby improving the flexibility of resource allocation and task scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 A schematic diagram of an application scenario of the task resource scheduling method provided in an embodiment of the present application;

[0028] Figure 2 A schematic diagram of a process flow in a task resource scheduling method provided in an embodiment of the present application;

[0029] Figure 3 A schematic diagram of a sub-process of the task resource scheduling method provided in an embodiment of the present application;

[0030] Figure 4 A schematic block diagram of a task resource scheduling device provided in an embodiment of the present application; and

[0031] Figure 5 A schematic block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0034] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0035] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0036] Embodiments of the present application provide a task resource scheduling method, apparatus, device, and storage medium.

[0037] The executor of the task resource scheduling method may be the task resource scheduling device provided in the embodiment of the present application, or a computer device integrated with the task resource scheduling device, wherein the task resource scheduling device may be implemented in hardware or software, and the computer device may be a terminal or a server. The embodiment of the present application takes the computer device as a server as an example to explain in detail the task resource scheduling method provided in the present application.

[0038] See also Figure 1 , Figure 1 A schematic diagram of an application scenario of the task resource scheduling method provided in an embodiment of the present application. In one embodiment, the application scenario includes:

[0039] This task resource scheduling method is applied to Figure 1In the server in, in some embodiments, after the server obtains multiple tasks to be executed with corresponding task identifiers, it obtains the task priorities and task dependencies corresponding to each task identifier from a preset task model, and sorts the multiple tasks to be executed according to each task priority and each task dependency to obtain a target task queue; determines the current available resource information from a preset resource model; allocates resources to each task to be executed based on the target task queue, the current available resource information and the preset task scheduling strategy, and after each task to be executed is executed according to the allocated resources, the server obtains the performance evaluation indicators corresponding to each task to be executed, and adjusts the task scheduling strategy according to each performance evaluation indicator, and dynamically adjusts the task scheduling strategy according to each performance evaluation indicator to adapt to the needs of task execution, thereby improving the flexibility of resource allocation and task scheduling.

[0040] It should be noted that the application scenario of the above-mentioned task resource scheduling method is only used to illustrate the technical solution of the present application, and is not used to limit the technical solution of the present application. The above-mentioned connection relationship may also have other forms.

[0041] Figure 2 A flowchart of a task resource scheduling method provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the method includes the following steps S110-S180.

[0042] S110: Acquire multiple tasks to be executed that carry corresponding task identifiers.

[0043] Specifically, multiple tasks to be executed are obtained from a task queue or a database storing tasks to be executed, and each task to be executed carries a unique task identifier (such as a task code) corresponding to it, which facilitates subsequent identification and management of the tasks to be executed.

[0044] S120: Acquire task priorities and task dependencies corresponding to the task identifiers according to a preset task model.

[0045] Among them, task priority refers to a way of sorting tasks according to factors such as the importance, urgency, and scope of impact when multiple tasks exist at the same time, which determines the order of execution of tasks. Task dependency refers to the fact that the execution of a task depends on the completion of one or more tasks, which reflects the fixed order and logical relationship between tasks and ensures that tasks are executed in the correct order.

[0046] Specifically, a plurality of different types of tasks and task attributes are stored in the preset task model, and the task attributes include task identification, task description, task type (e.g., tasks to be executed, executed tasks, tasks being executed), task priority and task dependency, wherein the task identification is unique, and the task priority and task dependency of a task correspond to a task identification. In the preset task model, according to the correspondence between the task priority and the task dependency and the task identification, the task priority and task dependency corresponding to each task identification can be obtained through the task identification of each task to be executed. Among them, the dependency between tasks can be represented by a directed acyclic graph (DAG), a text description, a critical path method (CPM) and a Gantt chart, which can be selected according to actual needs.

[0047] The task priority and dependency are obtained through the preset task model. The fields and storage methods of the task model can be adjusted as needed. Its flexibility enables the resource scheduling system to better cope with changes, thereby improving the scalability of the resource scheduling system.

[0048] S130: Determine currently available resource information according to a preset resource model.

[0049] The resource model stores a structured framework or template for real-time acquisition of currently available resource information in the system. The currently available resource information includes: resource type (such as computing resources, storage resources, network resources, and software resources, etc.) and resource attributes (such as memory resources whose resource type is storage resources, whose attributes may include total capacity, used capacity, available capacity, etc.). The currently available resource information includes the specific conditions of various resources acquired by the resource model at the current time point, including resource types and the number of available resources corresponding to each resource type (such as memory resources, whose currently available resource information includes storage resources and available capacity, where storage resources are resource types and available capacity are the number of available resources).

[0050] Specifically, since different types of tasks have different resource requirements, it is necessary to clarify the resource requirements required for the type of task to be executed based on the type of task to be executed. For example, computationally intensive tasks (such as data analysis and image rendering) usually require a large amount of central processing unit (CPU) resources and memory resources. According to the resource requirements required for the task to be executed, the resource type and the amount of available resources in the resource requirements are determined from the preset resource model. By combining the task to be executed and the preset resource model, the accuracy of the currently available resource information can be improved to ensure the efficient execution of the task and the stable operation of the system.

[0051] S140 , sorting the plurality of tasks to be executed according to the priority of each task and the dependency relationship of each task to obtain a target task queue.

[0052] Among them, the target task queue refers to an ordered task set obtained by sorting multiple tasks to be executed according to their task priorities and task dependencies. The target task queue can be implemented using different data structures, such as linked lists, arrays, or priority queues. It usually contains task-related information, such as task identification, task description, task priority, task dependencies, etc., so that the system can effectively manage and monitor tasks. Through the target task queue, the system can better plan and schedule the execution of tasks and improve the overall performance and efficiency of the system.

[0053] Further, in some embodiments, S140 includes the following steps:

[0054] The plurality of tasks to be executed are initially sorted based on the priority of each task to obtain an initial task queue.

[0055] The initial task queue refers to an ordered task set obtained by sorting multiple tasks to be executed according to their priorities.

[0056] Specifically, for example, a preset task model stores task priorities and task dependencies corresponding to a plurality of different task identifiers:

[0057] Task ID 001: Task priority: high, Task dependency: none;

[0058] Task ID 002: Task priority: medium, Task dependency: executed after task ID 003;

[0059] Task ID 003: Task priority: low, task dependency: executed after task ID 001;

[0060] The task priority and task dependency corresponding to each task ID are obtained respectively through each task ID, that is, through the task ID "task ID 001", the "task priority: high" and "task dependency: none" corresponding to "task ID 001" are obtained; through the task ID "task ID 002", the "task priority: medium" and "task dependency: executed after task ID 003" corresponding to "task ID 002" are obtained; and through the task ID "task ID 003", the "task priority: low" and "task dependency: executed after task ID 001" corresponding to "task ID 003" are obtained. Then, the tasks corresponding to each task ID are sorted according to each task priority, and the initial task queue obtained is ["task ID 001", "task ID 002" and "task ID 003"].

[0061] The initial task queue is adjusted based on each of the task dependencies to obtain the target task queue.

[0062] Specifically, we continue to elaborate based on the example of "the following tasks to be executed are stored in a preset task model" mentioned above. On the basis of the initial task queue obtained by sorting the tasks corresponding to each task identifier according to each task priority, the initial task queue is adjusted according to each task dependency. Among them, since "task identifier 002" has a task dependency "executed after task identifier 003" (that is, the execution of "task identifier 002" depends on the completion of "task identifier 003"), "task identifier 003" has a task dependency "executed after task identifier 001" (that is, the execution of "task identifier 003" depends on the completion of "task identifier 001"), the execution order of "task identifier 001" is before "task 003", and the execution order of "task identifier 003" is before "task identifier 002". Therefore, the target task queue finally obtained is ["task identifier 001", "task identifier 003" and "task identifier 002"].

[0063] S150, performing resource allocation for each of the tasks to be executed based on the target task queue, the currently available resource information, and a preset task scheduling strategy to obtain a resource allocation result.

[0064] The task scheduling strategy refers to a method or set of rules for dynamically allocating resources among multiple tasks to be executed. Its purpose is to maximize the system's performance and resource utilization while meeting task requirements.

[0065] Specifically, based on the target task queue obtained, each task to be executed in the target task queue is dynamically allocated resources according to its relationship with the currently available resource information. In addition, a mutex or queue algorithm is used to solve the situation that a plurality of tasks to be executed compete for the same resource at the same time and resource allocation conflicts occur. At the same time, a dynamic resource allocation strategy is adopted to dynamically adjust the allocation of resources according to the real-time utilization of resources and task requirements, including dynamically adjusting resource priorities, task preemption and resource reservation, etc., so as to improve resource utilization and task execution efficiency.

[0066] Through the task scheduling strategy, on the one hand, resources can be allocated to the most needed tasks based on task requirements and resource availability, thereby maximizing the performance of the system; on the other hand, when the system load increases or resource requirements change, the task scheduling strategy can adjust resource allocation according to actual conditions to cope with different workloads and task requirements, thereby improving the scalability of the task scheduling system.

[0067] Specifically, in some embodiments, Figure 3 As shown, the currently available resource information includes multiple resource types and the available resource quantities corresponding to each resource type, and S150 includes the following steps:

[0068] S151: Determine the to-be-executed task that has not been resource-allocated and is at the top of the order in the target task queue as the target execution task.

[0069] Specifically, the target task queue is traversed and the resource allocation status of each task is checked, and the to-be-executed task in the target task queue that has not been resource allocated and is ranked first in the queue is used as the target execution task. Obtaining the target execution task in this way can ensure that the to-be-executed tasks are executed in order according to the task priority and task dependency, and at the same time helps to avoid confusion and omissions in the resource allocation process.

[0070] S152. For the target execution task, determine the target resource type corresponding to the target task identifier according to the correspondence between the task identifier and the resource type, where the target task identifier is the task identifier corresponding to the target execution task.

[0071] Specifically, the correspondence between the task identifier and the resource type can be determined through a pre-established task resource mapping table or a task resource database query. For example, assuming that the mapping table between the task identifier and the resource type is: {Task A: CPU, Task B: Storage, Task C: GPU}; where the target execution task is Task C, then the corresponding target resource type is GPU. Based on the correspondence between the task identifier and the resource type, the target resource type required for the target execution task is clarified so that the task requirements can be met more accurately during resource allocation to avoid unreasonable resource allocation.

[0072] S153: Determine a target available resource quantity according to the available resource quantity corresponding to the target resource type in the current available resource information.

[0073] Further, in some embodiments, S153 includes the following steps:

[0074] An estimated task execution time corresponding to the target task identifier is obtained from the task model.

[0075] The target available resource quantity is determined according to the estimated task execution time and the available resource quantity corresponding to the target resource type in the current available resource information.

[0076] The estimated task execution time refers to the time required to complete the task to be executed, which is predicted based on the characteristics of the task to be executed, available resource information and historical execution data before the task to be executed actually starts.

[0077] Specifically, first, analyze the impact of the estimated task execution time on resource requirements. If the task execution time is long, more resources may be required to ensure that the task to be executed is completed within a reasonable time. Then, the target available resource quantity is determined in combination with the available resource quantity corresponding to the target resource type in the current available resource information. Specifically, an algorithm can be used. When the estimated task execution time is short, resources can be allocated from the available resources according to a relatively low ratio to avoid resource waste; when the estimated task execution time is long, the allocated resource ratio can be appropriately increased to speed up task execution. In addition, by using prediction and optimization technology, historical data analysis, machine learning and other technologies can be used to predict and optimize task execution time, resource requirements, etc., so as to more accurately evaluate task execution time and resource requirements and optimize task scheduling.

[0078] For example, if there are 100 units of available resources, when the estimated task execution time is short, resources are allocated at a ratio of 30%, that is, the target number of available resources is 30 units; when the estimated task execution time is long, resources are allocated at a ratio of 60%, that is, the target number of available resources is 60 units.

[0079] Combining the estimated task execution time and resource allocation, firstly, it can avoid long waiting or failure of certain tasks due to insufficient resources, thereby improving the stability and reliability of the system; secondly, it can make the task scheduling system better adapt to the dynamic changes of tasks and resources, so as to improve the flexibility and adaptability of the task scheduling system.

[0080] Further, in some embodiments, determining the target available resource quantity according to the estimated task execution time and the available resource quantity corresponding to the target resource type in the current available resource information comprises the following steps:

[0081] Obtaining a target task amount corresponding to the target execution task;

[0082] Determine the current required resource quantity according to the target task quantity, the unit time processing task quantity corresponding to the target resource type, and the estimated task execution time;

[0083] If the currently required resource quantity is less than or equal to the available resource quantity corresponding to the target resource type, the currently required resource quantity is used as the target available resource quantity;

[0084] If the currently required resource quantity is greater than the available resource quantity corresponding to the target resource type, the available resource quantity corresponding to the target resource type is used as the target available resource quantity.

[0085] The amount of tasks processed per unit time corresponding to the target resource type refers to the number of tasks that a certain type of resource can process or complete within a specific time unit (such as per hour, per minute, etc.).

[0086] Specifically, after obtaining the target task volume corresponding to the target execution task according to the target task identifier of the target execution task, the current required resource volume corresponding to the target task volume is calculated according to the calculation method of current required resource volume = target task volume / (task volume processed per unit time × estimated task execution time). For example: the target task volume is to process 10,000 data records; the target resource type (assuming it is a specific CPU computing resource) corresponds to a processing task volume per unit time of 50 data records per second; the estimated task execution time is 20 seconds; then the current required resource volume = 10,000 / (50 × 20) = 10. This means that in this case, in order to complete the task of processing 10,000 data records within the estimated 20 seconds, 10 target resource types (the specific CPU computing resources) are required.

[0087] When the current required number of resources is less than or equal to the available number of resources corresponding to the target resource type, the current required number of resources is used as the target available number of resources; when the current required number of resources is greater than the available number of resources corresponding to the target resource type, the available number of resources corresponding to the target resource type is used as the target available number of resources. By comprehensively considering the target task volume, the number of tasks processed per unit time, and the estimated task execution time, the actual number of resources required for the task can be determined more accurately, so that resource allocation is more in line with the actual needs of the task, resource utilization efficiency is improved, and dynamic changes in task volume and resource allocation can be adapted.

[0088] S154. Allocate resources to the target execution task according to the target available resource quantity and a preset load balancing algorithm to obtain a resource allocation result corresponding to the target execution task.

[0089] Among them, load balancing algorithm refers to a strategy for distributing workloads among multiple resources (such as servers, computing nodes, or services) with the aim of optimizing resource usage, maximizing throughput, minimizing response time, and avoiding overloading of any single resource, so as to achieve efficient use of resources and improve system performance and reliability.

[0090] Specifically, the load balancing algorithm includes round-robin allocation, weighted round-robin allocation, etc., which can be set according to actual needs.

[0091] S155, updating the currently available resource information according to the resource model, and returning to the step of determining the to-be-executed task that has not been allocated resources and is at the front in the target task queue as the target execution task, until resources have been allocated to each of the to-be-executed tasks.

[0092] Specifically, for example, assume that there are three types of resources in a task scheduling system: computing resources, storage resources, and network resources; the currently available resource information is: computing resources have 10 CPUs, storage resources have 100 GigaByte (GB) hard disk space, and network resources have 100 bandwidth (Million bits per second, Mbps).

[0093] The target task queue contains [task ID 100, task ID 200, task ID 300], where "task ID 100", "task ID 200" and "task ID 300" are all pending tasks for which no resources have been allocated. The correspondence between task IDs and resource types is as follows: task ID 100 requires computing resources and storage resources; task ID 200 requires computing resources and network resources; task ID 300 requires storage resources and network resources.

[0094] First, according to the task scheduling strategy, task ID 100 is the target execution task. In S150, resources are allocated to task ID 100 according to the current available resource information and the task scheduling strategy. Assume that task ID 100 requires 4 CPUs and 30GB hard disk space. After allocation, the current available resource information is updated to: computing resources have 6 CPUs, storage resources have 70GB hard disk space, and network resources have 100Mbps bandwidth.

[0095] Then, in S151, task identifier 100 is determined as having been resource allocated, and then task identifier 200 is determined as the target execution task. In S152, the computing resources and network resources required by task identifier 200 are determined based on the correspondence between the task identifier and the resource type. In S153, based on the current available resource information, the target available resource quantity is determined to be 6 CPUs for computing resources and 100Mbps bandwidth for network resources. In S154, resources are allocated to task identifier 200 based on the load balancing algorithm, assuming that 3 CPUs and 50Mbps bandwidth are allocated. After allocation, the current available resource information is updated to: 3 CPUs for computing resources, 70GB hard disk space for storage resources, and 50Mbps bandwidth for network resources.

[0096] Next, the task identifier 300 is determined as the target execution task, and the above steps are repeated to allocate resources to the task identifier 300 until all tasks have been allocated resources.

[0097] S160: Execute each of the tasks to be executed according to the resource allocation result.

[0098] Specifically, the resource allocation result is optimized according to the task requirements and available resource information, so that the role of resources can be maximized and the execution speed and quality of the task can be improved. In addition, if the available resource information changes, the task can be re-executed according to the new resource allocation result to ensure that the task can be executed under the optimal resource configuration.

[0099] S170: When the execution of each of the tasks to be executed is completed, obtain the performance evaluation indicators corresponding to each of the tasks to be executed.

[0100] Among them, performance evaluation indicators are quantitative indicators used to measure the performance and effects of the tasks to be executed during the execution process, including the actual completion time of the task, resource utilization, etc.

[0101] Specifically, during the execution of the tasks to be executed, the performance evaluation index data can be collected through the log recording system, and then the log data corresponding to the performance evaluation index data can be parsed and aggregated through the log analysis tool to extract the performance evaluation index of each task to be executed. Through the performance evaluation index of each task to be executed, the task scheduling strategy can make intelligent decisions based on the real-time data and performance indicators of the tasks to be executed during execution, and use resource scheduling algorithm technology to predict task execution time, resource utilization, etc., so as to make better scheduling decisions.

[0102] S180: Adjust the task scheduling strategy according to each of the performance evaluation indicators.

[0103] Specifically, the task scheduling strategy is a strategy for allocating resources to the tasks to be executed. After each task to be executed is completed, each task to be executed will have a corresponding performance evaluation indicator. By deeply analyzing the performance data collected in the performance evaluation indicators, potential problems in the task scheduling strategy can be identified. For example, if it is found that the response time of a task is long, it may be necessary to optimize the processing logic of the task or allocate more resources to the task.

[0104] Specifically, in some embodiments, the performance evaluation index includes task execution completion time and resource utilization, and S180 includes the following steps:

[0105] For each of the performance evaluation indicators, if the task execution completion time is greater than the first preset duration threshold, and / or the resource utilization is outside the preset resource utilization threshold range, a policy adjustment instruction for the task scheduling policy is generated based on the task execution completion time and the resource utilization.

[0106] The task scheduling strategy is adjusted according to the strategy adjustment instruction.

[0107] Specifically, a performance monitoring tool is used or a timed checkpoint is set during the task execution process to obtain the value of the performance evaluation index corresponding to each task to be executed. The obtained task execution completion time and the first preset duration threshold and the resource utilization rate are respectively judged whether they are outside the preset resource utilization rate threshold range. When the task execution completion time is greater than the threshold, and / or the resource utilization rate is too high or too low (i.e., the resource utilization rate is outside the preset resource utilization rate threshold range), a policy adjustment instruction is generated, and the task scheduling policy is adjusted according to the policy adjustment instruction.

[0108] A reasonable task scheduling strategy can avoid excessive resource usage or idle resources in the system, thereby enhancing the flexibility and adaptability of the task scheduling strategy.

[0109] Furthermore, in some embodiments, generating a policy adjustment instruction of the task scheduling policy according to the task execution completion time and the resource utilization rate includes:

[0110] If the task execution completion time is greater than the first preset time threshold, and the resource utilization rate is greater than the maximum resource utilization rate in the resource utilization rate threshold range, then generating a resource expansion adjustment instruction;

[0111] If the task execution completion time is less than the second preset duration threshold, and the resource utilization is less than the minimum resource utilization in the resource utilization threshold range, a resource scaling adjustment instruction is generated, and the first preset duration threshold is greater than the second preset duration threshold.

[0112] Specifically, a performance monitoring tool or a system-built-in monitoring module is used to monitor the task execution completion time and resource utilization of each task. For each task being executed, its execution completion time is compared with a first preset duration threshold and a second preset duration threshold, and its resource utilization is compared with a resource utilization threshold range.

[0113] If the task execution completion time is greater than the first preset duration threshold and the resource utilization rate is greater than the maximum resource utilization rate in the resource utilization rate threshold range, it means that the task execution time is too long and the resources are tight, and a resource expansion adjustment instruction is generated at this time; if the task execution completion time is less than the second preset duration threshold and the resource utilization rate is less than the minimum resource utilization rate in the resource utilization rate threshold range, it means that the task is executed quickly and the resources are relatively idle, and a resource reduction adjustment instruction is generated at this time.

[0114] For example, suppose an online video processing system has multiple video transcoding tasks executed on different servers, wherein the first preset duration threshold is set to 60 minutes, the second preset duration threshold is set to 10 minutes, and the resource utilization threshold range is set to 40%-70%. Task ID x originally allocated certain computing resources for video transcoding. After a period of monitoring, it is found that the task execution completion time of task ID x has exceeded the first preset duration threshold of 60 minutes, and the resource utilization rate has reached 80%, which is higher than the maximum resource utilization rate in the resource utilization threshold range (40%-70%). At this time, a resource expansion adjustment instruction is generated, which can notify the administrator to increase server resources or temporarily allocate resources of other tasks to task ID x to speed up its transcoding speed.

[0115] Another task to be executed with task ID y also executes the video transcoding task. Monitoring shows that its execution completion time is only 5 minutes, and the resource utilization rate is 30%, which is less than the minimum resource utilization rate in the resource utilization rate threshold range. At this time, according to the resource scaling adjustment instruction, it is possible to consider reducing the resource allocation of task ID y, allocating the excess resources to other tasks that need the resources more, or releasing some resources to reduce costs.

[0116] Dynamically adjust resource allocation according to the actual execution of tasks to avoid resource waste and resource shortage. In addition, scaling down when resource utilization is low can reduce unnecessary resource consumption and reduce system operating costs.

[0117] To sum up, in the embodiment of the present application, on the basis of obtaining a target task queue by sorting multiple tasks to be executed according to the priority of each task to be executed and the task dependency relationship, resources are allocated to each task to be executed based on the target task queue, currently available resource information and a preset task scheduling strategy. After each task to be executed is executed according to the allocated resources, performance evaluation indicators corresponding to each task to be executed are obtained, and the task scheduling strategy is adjusted according to each performance evaluation indicator, thereby improving the flexibility of resource allocation and task scheduling.

[0118] Figure 4 is a schematic block diagram of a task resource scheduling device provided in an embodiment of the present application. Figure 4 As shown, corresponding to the above task resource scheduling method, the present invention also provides a task resource scheduling device 1000. The task resource scheduling device 1000 includes a unit for executing the above task resource scheduling method. Figure 4 The task resource scheduling device 1000 includes a first acquisition unit 1001, a second acquisition unit 1002, a first determination unit 1003, a sorting unit 1004, an allocation unit 1005, an execution unit 1006, a third acquisition unit 1007 and an adjustment unit 1008, wherein:

[0119] The first acquisition unit 1001 is used to acquire a plurality of to-be-executed tasks carrying corresponding task identifiers;

[0120] The second acquisition unit 1002 is used to acquire the task priority and task dependency relationship respectively corresponding to each of the task identifiers according to a preset task model;

[0121] The first determining unit 1003 is used to determine currently available resource information according to a preset resource model;

[0122] The sorting unit 1004 is used to sort the plurality of tasks to be executed according to the priority of each task and the dependency relationship of each task to obtain a target task queue;

[0123] The allocation unit 1005 is used to allocate resources to each of the tasks to be executed based on the target task queue, the currently available resource information and the preset task scheduling strategy to obtain a resource allocation result;

[0124] An execution unit 1006, configured to execute each of the tasks to be executed according to the resource allocation result;

[0125] The third acquisition unit 1007 is used to acquire the performance evaluation index corresponding to each of the tasks to be executed when the tasks to be executed are completed;

[0126] The adjusting unit 1008 is used to adjust the task scheduling strategy according to each of the performance evaluation indicators.

[0127] In some embodiments, the sorting unit 1004 is specifically used to:

[0128] Initially sorting the plurality of tasks to be executed based on the priority of each task to obtain an initial task queue;

[0129] The initial task queue is adjusted based on each of the task dependencies to obtain the target task queue.

[0130] In some embodiments, the currently available resource information includes multiple resource types and the available resource quantities corresponding to each resource type; the allocation unit 1005 includes:

[0131] A second determining unit is used to determine the to-be-executed task that has not been resource-allocated and is at the top of the order in the target task queue as a target execution task;

[0132] A third determining unit is used to determine, for the target execution task, a target resource type corresponding to the target task identifier according to a correspondence between the task identifier and the resource type, wherein the target task identifier is a task identifier corresponding to the target execution task;

[0133] a fourth determining unit, configured to determine a target available resource quantity according to the available resource quantity corresponding to the target resource type in the current available resource information;

[0134] An allocation unit, configured to allocate resources to the target execution task according to the target available resource quantity and a preset load balancing algorithm, and obtain a resource allocation result corresponding to the target execution task;

[0135] An updating unit is used to update the currently available resource information according to the resource model, and return to execute the step of determining the to-be-executed task that has not been allocated resources and is ranked first in the target task queue as the target execution task, until all the to-be-executed tasks have been allocated resources.

[0136] In some embodiments, the fourth determining unit includes:

[0137] A fourth acquisition unit, configured to acquire an estimated task execution time corresponding to the target task identifier from the task model;

[0138] The fifth determining unit is used to determine the target available resource quantity according to the estimated task execution time and the available resource quantity corresponding to the target resource type in the current available resource information.

[0139] In some embodiments, the fifth determining unit is specifically configured to:

[0140] Obtaining a target task amount corresponding to the target execution task;

[0141] Determine the current required resource quantity according to the target task quantity, the unit time processing task quantity corresponding to the target resource type, and the estimated task execution time;

[0142] If the currently required resource quantity is less than or equal to the available resource quantity corresponding to the target resource type, the currently required resource quantity is used as the target available resource quantity;

[0143] If the currently required resource quantity is greater than the available resource quantity corresponding to the target resource type, the available resource quantity corresponding to the target resource type is used as the target available resource quantity.

[0144] In some embodiments, the performance evaluation index includes task execution completion time and resource utilization; the adjustment unit 1008 includes:

[0145] A first generating unit is used for generating, for each of the performance evaluation indicators, a policy adjustment instruction of the task scheduling policy according to the task execution completion time and the resource utilization if the task execution completion time is greater than a first preset duration threshold and / or the resource utilization is outside a preset resource utilization threshold range;

[0146] The adjustment subunit is used to adjust the task scheduling strategy according to the strategy adjustment instruction.

[0147] In some embodiments, the first generating unit includes:

[0148] A second generating unit is used to generate a resource expansion adjustment instruction if the task execution completion time is greater than a first preset duration threshold and the resource utilization rate is greater than a maximum resource utilization rate in the resource utilization rate threshold range;

[0149] The third generating unit is used to generate a resource scaling adjustment instruction if the task execution completion time is less than a second preset duration threshold and the resource utilization is less than the minimum resource utilization in the resource utilization threshold range, and the first preset duration threshold is greater than the second preset duration threshold.

[0150] To sum up, the task resource scheduling device 1000 in the embodiment of the present application, on the basis of obtaining a target task queue by sorting multiple tasks to be executed according to the priority of each task to be executed and the task dependency relationship, allocates resources to each task to be executed based on the target task queue, the currently available resource information and the preset task scheduling strategy. After each task to be executed is executed according to the allocated resources, performance evaluation indicators corresponding to each task to be executed are obtained, and the task scheduling strategy is adjusted according to each performance evaluation indicator, thereby improving the flexibility of resource allocation and task scheduling.

[0151] The task resource scheduling device can be implemented in the form of a computer program. Figure 5 Runs on the computer device shown.

[0152] See also Figure 5 , Figure 5 800 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 800 is a terminal or a server, and a task resource scheduling platform is deployed in the computer device 800.

[0153] See also Figure 5 The computer device 800 includes a processor 802 , a memory and a network interface 805 connected via a system bus 801 , wherein the memory may include a non-volatile storage medium 803 and an internal memory 804 .

[0154] The non-volatile storage medium 803 may store an operating system 8031 ​​and a computer program 8032. The computer program 8032 includes program instructions, which, when executed, may cause the processor 802 to execute an order allocation method.

[0155] The processor 802 is used to provide computing and control capabilities to support the operation of the entire computer device 800 .

[0156] The internal memory 804 provides an environment for the operation of the computer program 8032 in the non-volatile storage medium 803. When the computer program 8032 is executed by the processor 802, the processor 802 can execute an order allocation method.

[0157] The network interface 805 is used to communicate with other devices over the network. Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 800 to which the solution of the present application is applied. The specific computer device 800 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0158] The processor 802 is used to run the computer program 8032 stored in the memory to implement the following steps:

[0159] Get multiple pending tasks with corresponding task identifiers;

[0160] Acquire the task priorities and task dependencies corresponding to the task identifiers according to the preset task model;

[0161] Determine currently available resource information based on a preset resource model;

[0162] Sorting the plurality of tasks to be executed according to the priority of each task and the dependency relationship of each task to obtain a target task queue;

[0163] Based on the target task queue, the currently available resource information and the preset task scheduling strategy, resources are allocated to each of the tasks to be executed to obtain a resource allocation result;

[0164] Execute each of the tasks to be executed according to the resource allocation result;

[0165] When the execution of each of the tasks to be executed is completed, obtaining the performance evaluation indicators corresponding to each of the tasks to be executed;

[0166] The task scheduling strategy is adjusted according to each of the performance evaluation indicators.

[0167] It should be understood that in the embodiment of the present application, the processor 802 may be a central processing unit (CPU), and the processor 802 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0168] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0169] Therefore, the present application also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the following steps:

[0170] Get multiple pending tasks with corresponding task identifiers;

[0171] Acquire the task priorities and task dependencies corresponding to the task identifiers according to the preset task model;

[0172] Determine currently available resource information based on a preset resource model;

[0173] Sorting the plurality of tasks to be executed according to the priority of each task and the dependency relationship of each task to obtain a target task queue;

[0174] Based on the target task queue, the currently available resource information and the preset task scheduling strategy, resources are allocated to each of the tasks to be executed to obtain a resource allocation result;

[0175] Execute each of the tasks to be executed according to the resource allocation result;

[0176] When the execution of each of the tasks to be executed is completed, obtaining the performance evaluation indicators corresponding to each of the tasks to be executed;

[0177] The task scheduling strategy is adjusted according to each of the performance evaluation indicators.

[0178] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0179] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0180] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0181] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present application can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0182] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0183] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A task resource scheduling method, characterized in that: include: Get multiple pending tasks with corresponding task identifiers; Acquire the task priorities and task dependencies corresponding to the task identifiers according to the preset task model; Determine currently available resource information based on a preset resource model; Sorting the plurality of tasks to be executed according to the priority of each task and the dependency relationship of each task to obtain a target task queue; Based on the target task queue, the currently available resource information and the preset task scheduling strategy, resources are allocated to each of the tasks to be executed to obtain a resource allocation result; Execute each of the tasks to be executed according to the resource allocation result; When the execution of each of the tasks to be executed is completed, obtaining the performance evaluation indicators corresponding to each of the tasks to be executed; The task scheduling strategy is adjusted according to each of the performance evaluation indicators.

2. The task resource scheduling method according to claim 1, characterized in that: The step of sorting the plurality of tasks to be executed according to the priority of each task and the dependency relationship of each task to obtain a target task queue includes: Initially sorting the plurality of tasks to be executed based on the priority of each task to obtain an initial task queue; The initial task queue is adjusted based on each of the task dependencies to obtain the target task queue.

3. The task resource scheduling method according to claim 1, characterized in that: The currently available resource information includes multiple resource types and the available resource quantities corresponding to each resource type; the resource allocation is performed on each of the tasks to be executed based on the target task queue, the currently available resource information and the preset task scheduling strategy to obtain a resource allocation result, including: Determine the to-be-executed task that has not been resource-allocated and is at the top of the order in the target task queue as the target execution task; For the target execution task, according to the correspondence between the task identifier and the resource type, determine the target resource type corresponding to the target task identifier, wherein the target task identifier is the task identifier corresponding to the target execution task; Determine a target available resource quantity according to the available resource quantity corresponding to the target resource type in the current available resource information; Allocate resources to the target execution task according to the target available resource quantity and a preset load balancing algorithm to obtain a resource allocation result corresponding to the target execution task; The currently available resource information is updated according to the resource model, and the step of determining the to-be-executed task that has not been allocated resources and is ranked first in the target task queue as the target execution task is returned to be executed until resources have been allocated to each of the to-be-executed tasks.

4. The task resource scheduling method according to claim 3, characterized in that: The determining the target available resource quantity according to the available resource quantity corresponding to the target resource type in the current available resource information includes: Obtaining an estimated task execution time corresponding to the target task identifier from the task model; The target available resource quantity is determined according to the estimated task execution time and the available resource quantity corresponding to the target resource type in the current available resource information.

5. The task resource scheduling method according to claim 4, characterized in that: The determining the target available resource quantity according to the estimated task execution time and the available resource quantity corresponding to the target resource type in the current available resource information includes: Obtaining a target task amount corresponding to the target execution task; Determine the current required resource quantity according to the target task quantity, the unit time processing task quantity corresponding to the target resource type, and the estimated task execution time; If the currently required resource quantity is less than or equal to the available resource quantity corresponding to the target resource type, the currently required resource quantity is used as the target available resource quantity; If the currently required resource quantity is greater than the available resource quantity corresponding to the target resource type, the available resource quantity corresponding to the target resource type is used as the target available resource quantity.

6. The task resource scheduling method according to claim 1, characterized in that: The performance evaluation indicators include task execution completion time and resource utilization; and adjusting the task scheduling strategy according to each of the performance evaluation indicators includes: For each of the performance evaluation indicators, if the task execution completion time is greater than a first preset duration threshold, and / or the resource utilization is outside a preset resource utilization threshold range, a policy adjustment instruction for the task scheduling policy is generated according to the task execution completion time and the resource utilization; The task scheduling strategy is adjusted according to the strategy adjustment instruction.

7. The task resource scheduling method according to claim 6, characterized in that: The generating of the policy adjustment instruction of the task scheduling policy according to the task execution completion time and the resource utilization rate includes: If the task execution completion time is greater than the first preset time threshold, and the resource utilization rate is greater than the maximum resource utilization rate in the resource utilization rate threshold range, then generating a resource expansion adjustment instruction; If the task execution completion time is less than the second preset duration threshold, and the resource utilization is less than the minimum resource utilization in the resource utilization threshold range, a resource scaling adjustment instruction is generated, and the first preset duration threshold is greater than the second preset duration threshold.

8. A task resource scheduling device, characterized in that: include: A first acquisition unit, used to acquire a plurality of to-be-executed tasks carrying corresponding task identifiers; A second acquisition unit, configured to acquire, according to a preset task model, task priorities and task dependencies respectively corresponding to the task identifiers; A first determining unit, configured to determine currently available resource information according to a preset resource model; A sorting unit, used for sorting the plurality of tasks to be executed according to the priority of each task and the dependency relationship of each task to obtain a target task queue; An allocation unit, configured to allocate resources to each of the tasks to be executed based on the target task queue, the currently available resource information and a preset task scheduling strategy, and obtain a resource allocation result; An execution unit, configured to execute each of the tasks to be executed according to the resource allocation result; A third acquisition unit is used to acquire the performance evaluation index corresponding to each of the tasks to be executed when the execution of each of the tasks to be executed is completed; An adjustment unit is used to adjust the task scheduling strategy according to each of the performance evaluation indicators.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the task resource scheduling method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the task resource scheduling method according to any one of claims 1 to 7 is implemented.

Images