Task processing method and device, equipment, medium and program product

By configuring task priorities and processing tasks based on dependencies and resource requirements, the problems of slow task execution and resource waste are solved, achieving efficient and fast task execution and resource optimization.

CN119045998APending Publication Date: 2024-11-29CHINA CONSTRUCTION BANK +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411102159.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-12
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

Existing task processing methods result in slow execution of data verification tasks, failing to guarantee efficient and rapid completion, and underutilizing resources, leading to uneven task execution time and resource waste.

Method used

By pre-configuring priorities for each task to be executed and determining whether a task can be executed based on the dependencies and resource requirements between tasks, parallel processing is performed using the remaining system resources, and the task order and resource requirements are dynamically adjusted.

Benefits of technology

It significantly improved task execution efficiency, reduced overall execution time, maximized the use of system resources, optimized resource utilization, and enhanced system performance and stability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119045998A_ABST
    Figure CN119045998A_ABST
Patent Text Reader

Abstract

The invention provides a task processing method and device, equipment, a medium and a program product, and is applied to the technical field of resource management, the method comprises the following steps: obtaining a plurality of to-be-executed tasks, each to-be-executed task being pre-configured with a priority; sequentially reading all the to-be-executed tasks according to the descending order of the priorities; when each to-be-executed task is read, whether the to-be-executed task can be executed or not is determined according to the dependency relationship among different to-be-executed tasks, the resource demand of the to-be-executed task and the current remaining resources, and the resource demand comprises the number of processors; and when it is determined that the to-be-executed task can be executed, executing the to-be-executed task. According to the method, rapid task processing is achieved, and the task processing speed and efficiency are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of resource management technology, and in particular to a task processing method, device, equipment, medium and program product. Background Art

[0002] In modern data processing and various project management, dependencies between tasks are a common and critical feature. Task dependency means that the execution of one task depends on the execution result of another task. This dependency affects the execution order of tasks, so in order to ensure that tasks are executed in the correct order and the results of task execution are correct, it is crucial to use appropriate task processing methods.

[0003] The general task processing method is to execute tasks by dividing the directed acyclic graph (DAG). The specific process is: execute tasks in sequence according to the dependency relationship. For example, task 2 depends on task 1, task 3 depends on task 2, task 4 depends on task 3, and task 5 depends on task 4. Then the five tasks need to be sorted according to the DAG graph and executed in sequence. And when there are fewer dependencies in other tasks, they will be processed first before tasks with more dependencies, so as to quickly complete the data verification task.

[0004] However, the common task processing methods in the prior art have the problem of slow execution of data verification tasks, making it difficult to ensure efficient and rapid completion of data verification tasks. Summary of the invention

[0005] The present application provides a task processing method, device, equipment, medium and program product to solve the problem that the task processing speed in the prior art is slow and cannot ensure the efficient and rapid completion of data verification tasks.

[0006] In a first aspect, the present application provides a task processing method, comprising:

[0007] Get multiple tasks to be executed, each of which is pre-configured with a priority;

[0008] Read all pending tasks in descending order of priority;

[0009] When reading each task to be executed, determining whether the task to be executed can be executed according to the dependencies between different tasks to be executed, the resource requirements of the tasks to be executed, and the current remaining resources, wherein the resource requirements include the number of processors;

[0010] When it is determined that the task to be executed can be executed, the task to be executed is executed.

[0011] Optionally, when reading each task to be executed, determining whether the task to be executed can be executed according to the dependencies between different tasks to be executed, the resource requirements of the tasks to be executed, and the currently remaining resources includes:

[0012] When reading each task to be executed, determining the predecessor task information of the task to be executed according to the dependency relationship between different tasks to be executed; wherein the predecessor task information is used to indicate whether there is a predecessor task for the task to be executed, and the execution status of the predecessor task;

[0013] If the predecessor task information indicates that the task to be executed does not have the predecessor task, determining whether the resource requirement of the task to be executed is greater than or equal to the current remaining resources;

[0014] If the predecessor task information indicates that the task to be executed has a predecessor task and the predecessor task has been completed, determining whether the resource requirement of the task to be executed is greater than or equal to the current remaining resources;

[0015] If the resource demand is greater than or equal to the current remaining resources, determining that the task to be executed can be executed;

[0016] If the resource requirement is less than the current remaining resources, it is determined that the task to be executed cannot be executed.

[0017] Optionally, when reading each task to be executed, after determining the predecessor task information of the task to be executed according to the dependency relationship between different tasks to be executed, the method further includes:

[0018] If the predecessor task information indicates that the task to be executed has the predecessor task, and the predecessor task has not been completed, it is determined that the task to be executed cannot be executed.

[0019] Optionally, the method further includes:

[0020] Inputting the execution status of the task to be executed into a task status information database;

[0021] Accordingly, determining the predecessor task information of the task to be executed according to the dependency relationship between different tasks to be executed includes:

[0022] Determine whether there is a predecessor task for the task to be executed according to the dependency relationship between the tasks to be executed;

[0023] If the predecessor task exists, the execution status of the predecessor task is obtained from the task status information database.

[0024] Optionally, the method further includes:

[0025] For each task to be executed, adjusting the resource requirement of the task to be executed in a third time period according to the execution information of the task to be executed in the first time period and the execution information of the task to be executed in the second time period;

[0026] The second time period is later than the first time period and earlier than the third time period, and the execution information includes resource requirements and execution duration.

[0027] Optionally, for each task to be executed, adjusting the resource requirement of the task to be executed in a third time period according to the execution information of the task to be executed in the first time period and the execution information of the task to be executed in the second time period includes:

[0028] For each task to be executed, if the resource requirement of the first time period is less than the resource requirement of the second time period, and the difference between the execution time of the first time period and the execution time of the second time period exceeds a preset difference, then the resource requirement of the task to be executed in the third time period is increased;

[0029] If the resource demand of the first time period is less than the resource demand of the second time period, and the duration difference does not exceed a preset duration difference, the resource demand of the third time period is reduced.

[0030] Optionally, the method further includes:

[0031] After all pending tasks are completed, determine the execution order of each pending task;

[0032] For each task to be executed, re-determine a new execution order of the task to be executed according to the execution order of the task to be executed and the historical execution order;

[0033] According to the new execution order, the priorities of all tasks to be executed are updated to obtain a new priority of each task to be executed.

[0034] Optionally, before acquiring a plurality of tasks to be executed, the method further includes:

[0035] According to the dependency relationship between different executable tasks, a priority is configured for each executable task in the plurality of executable tasks; wherein the plurality of tasks to be executed belong to the plurality of executable tasks.

[0036] Optionally, configuring a priority for each executable task in the multiple executable tasks according to the dependency relationship between different executable tasks; wherein the multiple tasks to be executed belong to the multiple executable tasks, including:

[0037] randomly assigning an initial priority to each of the plurality of executable tasks;

[0038] Read each executable task in order from the highest to the lowest initial priority;

[0039] When reading each executable task, determining whether the executable task has a preceding executable task according to the dependency relationship between different executable tasks;

[0040] If the executable task has the predecessor executable task, the initial priority of the executable task is adjusted according to the current priority of the predecessor executable task so that the priority of the executable task is less than the current priority of the predecessor executable task, and the priority of the executable task is obtained.

[0041] In a second aspect, the present application provides a task processing device, comprising:

[0042] An acquisition module is used to acquire multiple tasks to be executed, each of which is pre-configured with a priority;

[0043] The reading module is used to read all pending tasks in descending order of priority;

[0044] A determination module, configured to determine whether each task to be executed can be executed according to the dependencies between different tasks to be executed, the resource requirements of the tasks to be executed, and the current remaining resources when reading each task to be executed, wherein the resource requirements include the number of processors;

[0045] The execution module is used to execute the task to be executed when it is determined that the task to be executed can be executed.

[0046] Optionally, the determining module is specifically used to:

[0047] When reading each task to be executed, determining the predecessor task information of the task to be executed according to the dependency relationship between different tasks to be executed; wherein the predecessor task information is used to indicate whether there is a predecessor task for the task to be executed, and the execution status of the predecessor task;

[0048] If the predecessor task information indicates that the task to be executed does not have the predecessor task, determining whether the resource requirement of the task to be executed is greater than or equal to the current remaining resources;

[0049] If the predecessor task information indicates that the task to be executed has a predecessor task and the predecessor task has been completed, determining whether the resource requirement of the task to be executed is greater than or equal to the current remaining resources;

[0050] If the resource demand is greater than or equal to the current remaining resources, determining that the task to be executed can be executed;

[0051] If the resource requirement is less than the current remaining resources, it is determined that the task to be executed cannot be executed.

[0052] Optionally, when reading each task to be executed, after determining the predecessor task information of the task to be executed according to the dependency relationship between different tasks to be executed, the determining module is further used to:

[0053] If the predecessor task information indicates that the task to be executed has the predecessor task, and the predecessor task has not been completed, it is determined that the task to be executed cannot be executed.

[0054] Optionally, the determining module is further used to:

[0055] Inputting the execution status of the task to be executed into a task status information database;

[0056] Accordingly, the determining module is specifically used for:

[0057] Determine whether there is a predecessor task for the task to be executed according to the dependency relationship between the tasks to be executed;

[0058] If the predecessor task exists, the execution status of the predecessor task is obtained from the task status information database.

[0059] Optionally, the task processing device further includes an adjustment module, wherein the adjustment module is used to:

[0060] For each task to be executed, adjusting the resource requirement of the task to be executed in a third time period according to the execution information of the task to be executed in the first time period and the execution information of the task to be executed in the second time period;

[0061] The second time period is later than the first time period and earlier than the third time period, and the execution information includes resource requirements and execution duration.

[0062] Optionally, the adjustment module is specifically used to:

[0063] For each task to be executed, if the resource requirement of the first time period is less than the resource requirement of the second time period, and the difference between the execution time of the first time period and the execution time of the second time period exceeds a preset difference, then the resource requirement of the task to be executed in the third time period is increased;

[0064] If the resource demand of the first time period is less than the resource demand of the second time period, and the duration difference does not exceed a preset duration difference, the resource demand of the third time period is reduced.

[0065] Optionally, the determining module is further used to:

[0066] After all pending tasks are completed, determine the execution order of each pending task;

[0067] For each task to be executed, re-determine a new execution order of the task to be executed according to the execution order of the task to be executed and the historical execution order;

[0068] According to the new execution order, the priorities of all tasks to be executed are updated to obtain a new priority of each task to be executed.

[0069] Optionally, before acquiring the plurality of tasks to be executed, the acquisition module is further configured to:

[0070] According to the dependency relationship between different executable tasks, a priority is configured for each executable task in the plurality of executable tasks; wherein the plurality of tasks to be executed belong to the plurality of executable tasks.

[0071] Optionally, the execution module is specifically used to:

[0072] randomly assigning an initial priority to each of the plurality of executable tasks;

[0073] Read each executable task in order from the highest to the lowest initial priority;

[0074] When reading each executable task, determining whether the executable task has a preceding executable task according to the dependency relationship between different executable tasks;

[0075] If the executable task has the predecessor executable task, the initial priority of the executable task is adjusted according to the current priority of the predecessor executable task so that the priority of the executable task is less than the current priority of the predecessor executable task, and the priority of the executable task is obtained.

[0076] In a third aspect, an embodiment of the present application provides an electronic device, comprising: at least one processor and a memory; the memory stores computer execution instructions; the at least one processor executes the computer execution instructions stored in the memory, so that the at least one processor performs the task processing method described in the first aspect and various possible designs of the first aspect.

[0077] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the task processing method described in the first aspect and various possible designs of the first aspect is implemented.

[0078] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the task processing method described in the first aspect and various possible designs of the first aspect.

[0079] The task processing method, device, equipment, medium and program product provided by the present application, in which a plurality of tasks to be executed are first obtained, each of which is pre-configured with a priority; then, all the tasks to be executed are read in order of priority from large to small; then, when reading each task to be executed, whether the task to be executed can be executed is determined according to the dependency relationship between different tasks to be executed, the resource requirements of the tasks to be executed and the current remaining resources, wherein the resource requirements include the number of processors; finally, when it is determined that the task to be executed can be executed, the task to be executed is executed. The method executes multiple tasks to be executed in parallel by reading each task to be executed in turn according to the priority, and by judging whether the tasks to be executed have a dependency relationship and whether the current remaining resources meet the resource requirements of the tasks to be executed, thereby significantly improving the task execution efficiency, reducing the overall execution time, and maximizing the use of system resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0081] Figure 1 Schematic diagram of the task processing method provided in the embodiment of the present application Figure 1 ;

[0082] Figure 2 Schematic diagram of the task processing method provided in the embodiment of the present application Figure 2 ;

[0083] Figure 3 Schematic diagram of the task processing method provided in the embodiment of the present application Figure 3 ;

[0084] Figure 4 A schematic diagram of the structure of a task processing device provided in an embodiment of the present application;

[0085] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0086] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0087] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0088] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of information such as asset data or user data involved shall comply with the provisions of relevant laws and regulations and shall not violate public order and good morals.

[0089] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0090] In modern data processing and various project management, the dependency between tasks is a common and critical feature. For example, as a platform for unified management of architecture asset data, the architecture asset management system needs to support each project team to obtain the architecture asset information released by each system to achieve asset reuse and sharing, resulting in dependencies between a large number of tasks, and these tasks can not only be data comparison tasks, but also data preprocessing tasks (i.e., integrating and processing different data tables). As for the data comparison task, the architecture asset quality verification module in the architecture asset management system performs consistency comparison on the design asset data and production asset data stored in the architecture asset management system. The consistency comparison between the design asset data and the production asset data is based on the built data warehouse and big data platform. In this comparison process, due to the large number of access data sources and the large amount of data. Therefore, in order to be able to process the complex logic in these data verification tasks in real time, quickly and correctly, it is crucial to use appropriate task processing methods.

[0091] In the prior art, the general task processing method is to execute tasks by dividing the DAG graph. The specific process is: execute tasks in sequence according to the dependency relationship. For example, task 2 depends on task 1, task 3 depends on task 2, task 4 depends on task 3, and task 5 depends on task 4. Then it is necessary to sort the 5 tasks according to the DAG graph and execute them in sequence. In addition, when there are fewer dependencies in other tasks, they will be processed first before tasks with more dependencies. For example, if only task 6 depends on task 7, then tasks 6 and 7 will be arranged to be executed before tasks 1 to 5, so as to quickly complete the data verification task.

[0092] However, in the prior art, all tasks must be executed strictly in the order of dependencies, and the execution efficiency of tasks is low. For tasks with fewer dependencies, although they will be executed first, tasks with more dependencies will be executed later. This method will lead to an imbalance in the execution time of tasks, and tasks with more dependencies will have a longer waiting time. In addition, since the number of dependencies has no direct relationship with the resources used for task execution, and there is no corresponding resource adjustment method, it may cause tasks with fewer dependencies to occupy a large amount of resources or tasks with less resource requirements to occupy a large amount of resources, resulting in a long waiting time for tasks and a waste of resources, reducing the efficiency of task execution. Finally, when tasks or dependencies change, the DAG graph needs to be re-divided and re-sorted, which increases the maintenance cost and complexity of the system.

[0093] Based on this, the present application provides a task processing method, device, equipment, medium and program product. Since not all tasks to be executed have dependencies, for example, task 1 exists independently, task 2 depends on task 1, task 3 depends on task 2, task 4 depends on task 3, and task 5 depends on task 4, but task 6 exists independently, and task 7 depends on task 6. Therefore, when a task to be executed is being processed, it is possible to obtain the execution status of other tasks to be executed to determine whether the task to be executed has a predecessor task and whether the current system server resources meet the resource requirements of the task to be executed. If both are satisfied, other idle server resources can be used to process the tasks to be executed in parallel, that is, task 6 is processed while processing task 1. In this way, the system can use idle resources to process multiple tasks simultaneously based on the judgment of the dependencies and resource requirements of the tasks to be executed. This method significantly improves the efficiency of task execution, reduces the overall execution time of tasks, and maximizes the use of system resources. Especially when the dependencies between tasks are complex and the system resources are sufficient, this method can give full play to the performance advantages of the system and improve the overall effect of task processing.

[0094] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0095] Figure 1 Schematic diagram of the task processing method provided in the embodiment of the present application Figure 1 .like Figure 1 As shown, the method includes:

[0096] S101. Acquire a plurality of tasks to be executed, each of which is pre-configured with a priority.

[0097] The execution subject of the embodiment of the present application is an electronic device, which can be a terminal device, such as a laptop, a desktop computer, a tablet computer, etc., or a server. In practical applications, whether the electronic device is a terminal device or a server can be determined according to actual conditions, and no specific limitation is imposed on this.

[0098] Each task to be executed is pre-configured with a priority so that all tasks to be executed can be read in a certain order later.

[0099] It is understandable that when a technical verification personnel needs to perform a data comparison task, they can send a data verification request to the server through a terminal device, and the data verification request can be used to indicate whether the design data is consistent with the R&D data. After that, the server obtains multiple data verification requests, that is, multiple tasks to be executed.

[0100] Optionally, before step S101, it is also necessary to pre-configure the priority of each task to be executed, and the specific method includes:

[0101] According to the dependency relationship between different executable tasks, a priority is configured for each executable task in the multiple executable tasks.

[0102] Among them, multiple tasks to be executed belong to multiple executable tasks. A dependency relationship means that the execution of an executable task needs to depend on the execution result of another executable task. For example, assuming there are executable tasks A and executable tasks B, and the execution of executable task B needs to depend on the execution result of executable task A, then executable task B has a dependency relationship with executable task A.

[0103] It is understandable that, according to the dependency relationship between all executable tasks, a corresponding priority is configured for each of all executable tasks, so that each subsequent executable task has a pre-configured priority after becoming a task to be executed.

[0104] Furthermore, the method of configuring the priority of each executable task among the multiple executable tasks is as follows:

[0105] First, an initial priority is randomly assigned to each of the multiple executable tasks; then, each executable task is read in order from the largest to the smallest initial priority; then, when reading each executable task, it is determined whether the executable task has a predecessor executable task based on the dependency relationship between different executable tasks; finally, if the executable task has a predecessor executable task, the initial priority of the executable task is adjusted based on the current priority of the predecessor executable task, so that the priority of the executable task is less than the current priority of the predecessor executable task, and the priority of the executable task is obtained.

[0106] The current priority of the executable task can be the initial priority randomly assigned to the executable task, or can be the priority after the initial priority of the executable task is adjusted according to the dependency relationship between different executable tasks. In practical applications, it can be determined according to the dependency relationship of the executable tasks, and the embodiments of the present application do not make specific restrictions here.

[0107] For example, if there are executable tasks A, B, and C, the dependency relationship between them is that C has a dependency on A, and B has a dependency on C. The initial priorities are: the priority of A is higher than that of B, and the priority of B is higher than that of C; when reading each executable task, by obtaining the dependency relationship between them, it is determined that B has a predecessor, C, and C has a predecessor, A. Since the current priority of A is higher than that of C, the current priority of C does not need to be adjusted, but the current priority of C is lower than that of B. Therefore, the initial priority of B needs to be adjusted so that the current priority of B is lower than that of C. The final priority order is: the priority of A is higher than that of C, and the priority of C is higher than that of B.

[0108] It can be understood that in the process of assigning priorities to multiple executable tasks, by reading tasks according to priorities and dynamically adjusting priorities according to dependencies, the priority completion of the preceding executable tasks is ensured, thereby avoiding dependency conflicts. This method of dynamically adjusting priorities ensures the orderly completion of each executable task, reduces waiting time, and improves the efficiency of task execution and the stability of the system.

[0109] S102. Read all tasks to be executed in order from highest priority to lowest priority.

[0110] It is understandable that, according to the pre-configured priority of each task to be executed, all tasks to be executed are read in order from high to low priority, so that the maximum system resources can be used to perform parallel processing of multiple tasks to be executed, thereby improving task processing efficiency.

[0111] S103 . When reading each task to be executed, determine whether the task to be executed can be executed according to the dependency relationship between different tasks to be executed, the resource requirements of the tasks to be executed, and the current remaining resources.

[0112] The resource requirement of the task to be executed refers to the number of processors required when the task to be executed is executed. The current remaining resources refer to the number of processors that are not used in the current state of the system.

[0113] It can be understood that when reading each task to be executed, by comprehensively considering the dependencies between different tasks to be executed, the resource requirements of the tasks to be executed, and the current remaining resources, multiple tasks to be executed without predecessor tasks can be processed in parallel if resources permit, ensuring that the remaining resources are fully utilized and avoiding idle resources, further reducing the overall execution time of multiple tasks to be executed, and improving task execution efficiency.

[0114] S104: When it is determined that the task to be executed can be executed, execute the task to be executed.

[0115] It is understandable that a task to be executed can be executed only when it is determined that the task to be executed can be executed based on the dependency relationship between different tasks to be executed, the resource requirements of the tasks to be executed, and the current remaining resources.

[0116] The present application provides a task processing method, which first obtains multiple tasks to be executed, each of which is pre-configured with a priority; then, reads all the tasks to be executed in order of priority from large to small; then, when reading each task to be executed, determines whether the task to be executed can be executed according to the dependency relationship between different tasks to be executed, the resource requirements of the tasks to be executed, and the current remaining resources, and the resource requirements include the number of processors; finally, when it is determined that the task to be executed can be executed, the task to be executed is executed. The method reads each task to be executed in turn according to the priority, and determines whether to execute the task to be executed by judging whether the task to be executed has a dependency relationship and whether the current remaining resources meet the resource requirements of the task to be executed, thereby realizing the parallel execution of multiple tasks to be executed that do not have a dependency relationship, significantly improving the task execution efficiency, reducing the overall execution time, and maximizing the use of system resources.

[0117] Furthermore, when determining whether a task to be executed can be executed based on the dependencies between different tasks to be executed, the resource requirements of the tasks to be executed, and the current remaining resources, it is necessary to judge the dependencies of the tasks to be executed, the resource requirements of the tasks to be executed, and the current remaining resources respectively. The specific process is as follows:

[0118] First, when reading each task to be executed, determine the predecessor task information of the task to be executed based on the dependency relationship between different tasks to be executed; then, if the predecessor task information indicates that there is no predecessor task for the task to be executed, determine whether the resource demand of the task to be executed is greater than or equal to the current remaining resources; accordingly, if the predecessor task information indicates that there is a predecessor task for the task to be executed and the predecessor task has been executed, continue to determine whether the resource demand of the task to be executed is greater than or equal to the current remaining resources; finally, if the resource demand is greater than or equal to the current remaining resources, determine that the task to be executed can be executed, and accordingly, if the resource demand is less than the current remaining resources, determine that the task to be executed cannot be executed.

[0119] The predecessor task information is used to indicate whether there is a predecessor task for the task to be executed, and the execution status of the predecessor task.

[0120] For example, if there are pending tasks A, B and C, the dependency relationship between them is that task C is dependent on task B; the total system resources are 5 processors, the resource requirement of task A is 2 processors, the resource requirement of task B is 2 processors, and the resource requirement of task C is 3 processors. Then, when reading each task to be executed, since the predecessor task information indicates that there is no predecessor task to be executed for task A to be executed, and the current remaining resources are 5 processors to meet the resource requirements of task A to be executed, task A to be executed can be executed; further, continue to read task B to be executed, since there is no predecessor task to be executed for task B to be executed, and the current remaining resources are 3 processors to meet the resource requirements of task B to be executed, task B to be executed can also be executed, and accordingly, continue to read task C to be executed, since there is a predecessor task to be executed for task C to be executed, but it is obtained that task B to be executed has been executed and 2 processor resources have been released, then at this time, since the predecessor task to be executed B has been completed and the current remaining resources are 3 processors, it is determined that task C to be executed is executable; accordingly, if the resource requirement of task C to be executed is 5 processors, although the predecessor task to be executed B has been completed, the current remaining resources are 3 processors, which is less than the resource requirement of task C to be executed, it is determined that task C to be executed cannot be executed.

[0121] It is understandable that when reading each task to be executed, by comprehensively considering the dependencies between different tasks to be executed, the resource requirements of the tasks to be executed, and the current remaining resources, the tasks to be executed without predecessor tasks (A and B) are executed immediately if resources permit, so that multiple tasks to be executed without predecessor tasks can be processed in parallel, ensuring that the remaining resources are fully utilized, avoiding idle resources, reducing the overall execution time of multiple tasks to be executed, and improving the efficiency of task execution. For tasks to be executed with predecessor tasks, they need to wait until their predecessor tasks are completed and have sufficient remaining resources before they can be executed, ensuring that the dependencies between different tasks to be executed are maintained and avoiding execution errors caused by unsatisfied dependencies.

[0122] Further, if the predecessor task information indicates that the task to be executed has a predecessor task, and the predecessor task has not been completed, it is determined that the task to be executed cannot be executed.

[0123] Continuing with the above example, since there is a previous pending task B for pending task C, but it is obtained that pending task B has not been completed, even if the current remaining resources are greater than or equal to the resource requirements of pending task C, it is still determined that pending task C cannot be executed.

[0124] It can be understood that using the dependency relationship between different pending tasks as the first criterion for determining whether a pending task can be executed ensures that all pending tasks can be executed in an orderly and correct manner, effectively avoids the possibility of task execution errors, and improves the stability of the system.

[0125] Furthermore, after determining whether the task to be executed can be executed, the status of the task to be executed needs to be updated. The specific process is as follows:

[0126] The execution status of the task to be executed is input into the task status information database.

[0127] The status information database is used to store the execution status of tasks to be executed.

[0128] It is understandable that after determining whether a pending task can be executed, in order to facilitate subsequent determination of whether other pending tasks with predecessor tasks can be executed, the execution status of the current pending task needs to be input into the task status information database for storage.

[0129] Optionally, the execution status of the task to be executed may be not yet started, being executed, or completed, etc. The specific execution status of the task to be executed is determined according to the execution status of the task to be executed in actual situations.

[0130] Optionally, by utilizing a status information database to save the execution status of pending tasks, the user can be notified by email or other means after a pending task fails. After the problem is fixed, the status information database can be re-read to quickly and accurately confirm and re-process the pending task that failed last time, without re-executing the pending task that has already been completed, thereby saving resources.

[0131] Accordingly, according to the dependency relationship between different tasks to be executed, the predecessor task information of the task to be executed can be determined as follows:

[0132] According to the dependency relationship between the tasks to be executed, it is determined whether there is a predecessor task for the task to be executed. If there is a predecessor task, the execution status of the predecessor task is obtained from the task status information database.

[0133] It is understandable that when reading the tasks to be executed, it is determined whether there is a predecessor task for the tasks to be executed based on the dependency relationship between the tasks to be executed. If it is determined that there is a predecessor task for the tasks to be executed, it is necessary to obtain the execution status of the predecessor task from the task status information database. The tasks to be executed can only be executed when the execution status of the predecessor task is completed. Otherwise, the tasks to be executed cannot be executed. This ensures that the dependency relationship between different tasks to be executed is maintained, avoiding execution errors caused by unsatisfied dependencies.

[0134] The specific implementation process is illustrated in the above embodiments, and the embodiments of the present application will not be described in detail here.

[0135] Figure 2 Schematic diagram of the task processing method provided in the embodiment of the present application Figure 2 This embodiment is in Figure 1 Based on the embodiment, the process of optimizing and adjusting after all pending tasks are completed is described in detail, such as Figure 2 As shown, the method includes:

[0136] S201. After all pending tasks are completed, determine the execution order of each pending task.

[0137] It is understandable that after all the tasks to be executed are completed, the execution order of each task to be executed is sorted and counted to determine the execution order of each task to be executed.

[0138] S202: for each task to be executed, re-determine a new execution order of the task to be executed according to the execution order of the task to be executed and the historical execution order.

[0139] The historical execution order refers to the multiple execution orders of the task to be executed in the past period of time.

[0140] Optionally, based on the execution order of the tasks to be executed and the historical execution order, the new execution order of the tasks to be executed can be re-determined based on the execution order of the tasks to be executed and the average value of multiple execution orders of the tasks to be executed in the past period of time; it can also be based on the minimum value of multiple execution orders of the tasks to be executed in the past period of time; it can also be based on the maximum value of multiple execution orders of the tasks to be executed in the past period of time and other information to re-determine the new execution order of the tasks to be executed. In actual applications, the method of re-determining the new execution order of the tasks to be executed is determined according to the specific circumstances, and the embodiments of the present application do not make specific restrictions here.

[0141] It can be understood that redetermining the new execution order of tasks to be executed based on the execution order of tasks to be executed and the corresponding historical execution order can better balance the system load, reduce the waiting time between tasks to be executed, and thus improve the overall efficiency of task execution; it can also enable the system to adapt to different task characteristics and changing work resource loads, thereby improving the adaptability and flexibility of the system.

[0142] S203: Update the priorities of all tasks to be executed according to the new execution order, and obtain a new priority of each task to be executed.

[0143] It is understandable that updating the priority of the task to be executed according to the new execution order can make the system more efficient and flexible when executing the task to be executed later, further reducing the execution time of the overall task.

[0144] Optionally, in addition to readjusting the priorities of tasks to be executed, the resource requirements of tasks to be executed can also be dynamically adjusted according to historical execution information to further maximize the use of system resources, reduce the execution time of tasks to be executed, and improve task processing efficiency.

[0145] In a possible implementation, the method for adjusting the resource requirements of the task to be executed is as follows:

[0146] For each task to be executed, adjusting the resource demand of the task to be executed in the third time period according to the execution information of the task to be executed in the first time period and the execution information of the second time period;

[0147] The second period is later than the first period and earlier than the third period, and the execution information includes resource requirements and execution duration. For example, the first period may be yesterday, the second period may be today, and the third period may be tomorrow.

[0148] It is understandable that by analyzing the resource requirements and execution duration of the tasks to be executed in the second time period and the first time period to adjust the resource requirements of the tasks to be executed in the third time period, the problem of excessive or insufficient resource allocation can be effectively avoided.

[0149] Furthermore, in a possible implementation, the resource requirement of the task to be executed in the third period is adjusted as follows:

[0150] For each task to be executed, if the resource demand in the first time period is less than the resource demand in the second time period, and the execution time difference between the first time period and the execution time of the second time period exceeds the preset time difference, the resource demand of the task to be executed in the third time period will be increased; if the resource demand in the first time period is less than the resource demand in the second time period, and the time difference does not exceed the preset time difference, the resource demand in the third time period will be reduced.

[0151] For example, suppose there is a task A to be executed, and its resource requirements and execution time in different time periods are as follows: in the first time period, the resource requirements of task A to be executed are 2 processors, and the execution time is 10 minutes; in the second time period, the resource requirements of task A to be executed are 4 processors, and the execution time is 6 minutes; and the preset time difference is 2 minutes. If the difference between the execution time of the first time period and the execution time of the second time period is 4 minutes, and the difference is greater than the preset time difference of 3 minutes, it means that the execution time of the second time period is significantly reduced after increasing the resource requirements, so it is necessary to increase the resource requirements of the third time period to further optimize the execution efficiency. Correspondingly, if in the first time period, the resource requirements of task A to be executed are 2 processors, and the execution time is 10 minutes; in the second time period, the resource requirements of task A to be executed are 4 processors, and the execution time is still 10 minutes; and the preset time difference is 2 minutes. If the difference between the execution time of the first period and the execution time of the second period is 0 minutes, and the difference is less than the preset time difference of 3 minutes, it means that increasing the resource demand has not significantly reduced the execution time. Therefore, the resource demand of the third period needs to be reduced to avoid resource waste.

[0152] It should be noted that when the resource demand in the third time period is adjusted, the resource demand can be adjusted to the resource demand in the first time period, or the resource demand can be adjusted within the range between the resource demand in the first time period and the resource demand in the second time period. The actual resource demand adjustment size is not specifically limited in the embodiments of this application.

[0153] It is understandable that by comparing the execution time of pending tasks caused by changes in resource requirements in different time periods, the resource requirements of pending tasks in the third period can be dynamically adjusted, which can improve the utilization rate of remaining system resources while ensuring task execution efficiency.

[0154] The present application provides a task processing method, which can significantly optimize resource utilization, improve task scheduling and system performance, reduce task waiting time, enhance system adaptability and reliability, and thus achieve more efficient and stable task execution and system operation by redetermining the new execution order of tasks to be executed and adjusting the resource requirements of tasks to be executed in a third time period after all tasks to be executed are completed.

[0155] Figure 3 Schematic diagram of the task processing method provided in the embodiment of the present application Figure 3 ; This embodiment is in Figure 1 , Figure 2 Based on the embodiment, the overall process of the task processing method is described as follows: Figure 3 As shown, the method includes:

[0156] Step 1: Get multiple tasks to be executed.

[0157] Step 2: Read all pending tasks in descending order of priority.

[0158] Step 3: When reading the tasks to be executed, determine whether there is a predecessor task for the tasks to be executed based on the dependency relationship between the tasks to be executed. If it is determined that there is a predecessor task for the tasks to be executed, execute step 4; otherwise, execute step 6.

[0159] Step 4: Obtain the execution status of the predecessor task from the task status information database;

[0160] Step 5: Determine whether the execution status of the predecessor task is completed. If it is completed, execute step 6; otherwise, execute step 7.

[0161] Step 6: Determine whether the resource requirement of the task to be executed is greater than or equal to the current remaining resources. If the resource requirement is greater than or equal to the current remaining resources, execute step 7; otherwise, execute step 8.

[0162] Step 7: Determine that the task to be executed cannot be executed, and re-execute step 2.

[0163] Step 8: Determine that the pending tasks can be executed, and execute the pending tasks.

[0164] Step 9: Input the execution status of the task to be executed into the task status information database.

[0165] Step 10: After all pending tasks are completed, a new execution order of the pending tasks is re-determined and updated.

[0166] Step 11: For each task to be executed, according to the execution information of the task to be executed in the first time period and the execution information of the second time period, the resource demand of the task to be executed in the third time period is adjusted.

[0167] It should be understood that the specific implementation methods in each step are already described in Figure 1 as well as Figure 2 The corresponding embodiments are described in detail, and the embodiments of the present application will not be described in detail here.

[0168] It should be noted that before obtaining the tasks to be executed and during the entire process of completing the tasks to be executed, multiple tools and components are needed to implement them. The specific processing tools, components and implementation methods used are as follows:

[0169] It runs in an existing data warehouse, such as a distributed file system (Hadoop Distributed FileSystem, HDFS) and a beehive (HIVE) data warehouse, which is used to store raw data, intermediate data and result data. It requires distributed computing (HADOOP) and other resource coordinators (Yet Another Resource Negotiator, YARN) to schedule server resources for data processing. It requires DataX tools and shell and Python scripts to read in raw data and write target data to external systems. It requires a map-reduce (MAPREDUCE) programming model and a SPARK engine to process data. It requires a structured query language (MySQL) database to store the execution status and resource requirements of tasks to be executed. It accesses the MySQL database through shell scripts to query the execution status and resource requirements of the predecessor tasks, update the execution status and resource requirements of the current tasks to be executed, and sequentially schedule the execution of the tasks to be executed stored in the text (txt) file. It requires the Dolphin tool or the CRONTAB command to schedule the tasks to be executed. If these prerequisite dependent environments are not available, they need to be installed before performing the embodiments of the present application.

[0170] Create a path on the server to store the log (date, dat) files provided by the upstream system. Use tools such as DataX or shell to generate and write dat files from your own database to the folder. Save the dat file and the information of the corresponding original data layer (Operational Data Store, ODS) table in HIVE in a txt file. Read the file in txt through a shell script to write the dat file to the corresponding table of HIVE. Save the name of the task to be executed, the name of the predecessor task required for the execution of the task to be executed, and the resource requirement information required by the task in the txt file. Use a shell script to read the task information in txt in a loop and query the mysql database table at the same time. If the predecessor tasks have been completed and there are enough remaining resources to execute the current task to be executed, schedule the task to be executed through MAPREDUCE or SPARK. At the same time, write the execution status and resource requirements of the task to be executed into mysql data to generate the detailed data layer (Data Warehouse Detail, DWD) and the application data layer (Application Data Store, ADS) in the processing table and result table information and temporary table, wait until all pending tasks in the txt file are completed, end the pending task, if the pending task fails, restart, if the restart also fails, the pending task stops executing, and notify the user through email and other means. After the problem is fixed, the shell can also read the execution status in the MySQL database and continue to execute the pending task; create a path on the server to store files that supply data to the downstream, and use tools such as DataX to write data directly to the database; after all pending tasks are executed, read the task execution status of the previous few days in the MySQL database, automatically optimize the task priority and task configuration resource requirements, and then write to the txt configuration file.

[0171] It should be further explained that different tools require different information to be written. For example:

[0172] The file information of the hive data is shown in Table 1:

[0173] Table 1 File Information Table

[0174]

[0175] The field type Varchar is a data type used to store variable-length character strings in a database.

[0176] The adjustment information of the tasks to be executed is shown in Table 2:

[0177] Table 2 Task adjustment information table to be executed

[0178] name Field Type Name meaning Priority Integer Priority of tasks Job_name Varchar Task Name Pro_catagory Varchar Predecessor task assignment Pro_job_name Varchar Pre-configuration Tasks Ingeer varchar Engine type, default value if not filled Thread Varchar Parallelism, default value if not filled Memory Varchar Memory usage, default value if not filled Is_Optimize Boolean Continue to optimize

[0179] The field type Integer is a numeric data type that does not contain a decimal part. The field type Boolean is a data type used to represent a logical state.

[0180] The data of the task status information to be executed is shown in Table 3:

[0181] Table 3 Status information table of tasks to be executed

[0182] name Field Type Name Explanation Hdfs_data_dt Varchar Execution date Job_type Varchar Task category, such as shell, hive, dataX, etc. Job_catagory Varchar Tasks assigned to ODS, DWD, ADS, etc. Job_name Varchar Task Name Start_time Datatime Start time End_time Datatime End Time Timeslong Double Execution time Status Varchar Task status 1 is completed; 2 is failed; 0 is in progress

[0183] The field type Datatime is a field type used to store date and time data; the field type Double is a large-precision data type used to store floating-point numbers.

[0184] Task resource requirement information is shown in Table 4:

[0185] Table 4 Task resource requirement information table

[0186] name Field Type Name meaning Job_type Varchar Task Category Job_catagory Varchar Task Attribution Job_name Varchar Task Name Ingeer_type varchar Engine type, MAPREDUCE, SPARK Threads Varchar Parallelism Memory Varchar Memory usage

[0187] When writing the task information to be executed into the file information of the hive data and querying the predecessor task information, the specific data information is shown in Table 5:

[0188] Table 5

[0189]

[0190] Among them, insert_log_mysql refers to inserting log information into the MySQL database; update_log_mysql refers to updating log information in the MySQL database; update_error_mysql: refers to updating error records in the MySQL database; wait_table_mysql refers to waiting for the execution result of the task.

[0191] When writing data to the MySQL database through the above task, the specific data information is shown in Table 6:

[0192] Table 6

[0193]

[0194] Among them, Start_script refers to the task script used to start the task to be executed; insert_log_mysql refers to inserting log information into the MySQL database; update_log_mysql refers to updating log information in the MySQL database; update_error_mysql: refers to updating error records in the MySQL database; wait_table_mysql refers to waiting for the execution result of the task.

[0195] The present application provides a task processing method, which determines whether to execute each task to be executed by reading each task to be executed in sequence according to the priority, and by judging whether there is a dependency relationship between the tasks to be executed and whether the current remaining resources meet the resource requirements of the tasks to be executed. After all the tasks to be executed are completed, the new execution order of the tasks to be executed is re-determined and the resource requirements of the tasks to be executed in the third time period are adjusted. This can significantly optimize resource utilization, improve task scheduling and system performance, reduce task waiting time, and enhance system adaptability and reliability, thereby achieving more efficient and stable task execution and system operation.

[0196] Figure 4 This is a schematic diagram of the structure of the task processing device provided in the embodiment of the present application. Figure 4 As shown, the task processing device 40 includes:

[0197] The acquisition module 401 is used to acquire a plurality of tasks to be executed, each of which is pre-configured with a priority;

[0198] The reading module 402 is used to read all the tasks to be executed in order from the highest priority to the lowest priority;

[0199] The determination module 403 is used to determine whether the task to be executed can be executed according to the dependency relationship between different tasks to be executed, the resource requirements of the tasks to be executed and the current remaining resources when reading each task to be executed, and the resource requirements include the number of processors;

[0200] The execution module 404 is used to execute the task to be executed when it is determined that the task to be executed can be executed.

[0201] In one achievable manner, the determination module 403 is specifically configured to:

[0202] When reading each task to be executed, the predecessor task information of the task to be executed is determined according to the dependency relationship between different tasks to be executed; wherein the predecessor task information is used to indicate whether there is a predecessor task for the task to be executed, and the execution status of the predecessor task;

[0203] If the predecessor task information indicates that the task to be executed does not have a predecessor task, then determining whether the resource requirement of the task to be executed is greater than or equal to the current remaining resources;

[0204] If the predecessor task information indicates that the task to be executed has a predecessor task and the predecessor task has been completed, then determine whether the resource requirement of the task to be executed is greater than or equal to the current remaining resources;

[0205] If the resource demand is greater than or equal to the current remaining resources, it is determined that the task to be executed can be executed;

[0206] If the resource requirement is less than the current remaining resources, it is determined that the task to be executed cannot be executed.

[0207] In one achievable manner, when reading each task to be executed, after determining the predecessor task information of the task to be executed according to the dependency relationship between different tasks to be executed, the determination module 403 is further used to:

[0208] If the predecessor task information indicates that the to-be-executed task has a predecessor task and the predecessor task has not been completely executed, it is determined that the to-be-executed task cannot be executed.

[0209] In one achievable manner, the determination module 403 is further configured to:

[0210] Inputting the execution status of the task to be executed into the task status information database;

[0211] Accordingly, the determination module 403 is specifically configured to:

[0212] According to the dependency relationship between the tasks to be executed, determine whether there are predecessor tasks for the tasks to be executed;

[0213] If there is a predecessor task, the execution status of the predecessor task is obtained from the task status information database.

[0214] In one achievable manner, the task processing device further includes an adjustment module, the adjustment module being configured to:

[0215] For each task to be executed, adjusting the resource demand of the task to be executed in the third time period according to the execution information of the task to be executed in the first time period and the execution information of the second time period;

[0216] The second time period is later than the first time period and earlier than the third time period, and the execution information includes resource requirements and execution duration.

[0217] In one achievable manner, the adjustment module is specifically used to:

[0218] For each task to be executed, if the resource requirement in the first time period is less than the resource requirement in the second time period, and the difference between the execution time of the first time period and the execution time of the second time period exceeds the preset time difference, the resource requirement of the task to be executed in the third time period is increased;

[0219] If the resource demand of the first time period is less than the resource demand of the second time period, and the duration difference does not exceed the preset duration difference, the resource demand of the third time period is reduced.

[0220] In one achievable manner, the determination module 403 is further configured to:

[0221] After all pending tasks are completed, determine the execution order of each pending task;

[0222] For each task to be executed, a new execution order of the task to be executed is re-determined according to the execution order of the task to be executed and the historical execution order;

[0223] According to the new execution order, the priorities of all tasks to be executed are updated to obtain the new priority of each task to be executed.

[0224] In one achievable manner, before acquiring a plurality of tasks to be executed, the acquisition module 401 is further configured to:

[0225] According to the dependency relationship between different executable tasks, a priority is configured for each executable task in the multiple executable tasks; wherein the multiple tasks to be executed belong to the multiple executable tasks.

[0226] In one achievable manner, the execution module 404 is specifically configured to

[0227] randomly assigning an initial priority to each of the plurality of executable tasks;

[0228] Read each executable task in order from the highest to the lowest initial priority;

[0229] When reading each executable task, determine whether the executable task has a predecessor executable task based on the dependency relationship between different executable tasks;

[0230] If the executable task has a predecessor executable task, the initial priority of the executable task is adjusted according to the current priority of the predecessor executable task so that the priority of the executable task is less than the current priority of the predecessor executable task, and the priority of the executable task is obtained.

[0231] The task processing device provided in the embodiment of the present application can be used to execute the technical solution of the task processing method in the above embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0232] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the determination module 403 can be a separately established processing element, or it can be integrated in a chip of the above device for implementation. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The function of the above determination module 403. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0233] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device 50 may include: at least one processor 501 and a memory 502. The electronic device 50 also includes a communication component 503. The processor 501, the memory 502 and the communication component 503 are connected via a bus 504.

[0234] In a specific implementation process, at least one processor 501 executes the computer execution instructions stored in the memory 502, so that at least one processor 501 performs the above-mentioned task processing method.

[0235] The specific implementation process of the processor 501 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0236] The electronic device provided in the embodiment of the present application can be used to execute the task processing method provided in any of the above method embodiments. Its implementation principles and technical effects are similar and will not be repeated here.

[0237] In the above Figure 5In the illustrated embodiment, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0238] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0239] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory.

[0240] An embodiment of the present application further provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the technical solution of the task processing method in the above embodiment.

[0241] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, the technical solution of the task processing method in the above embodiment can be implemented.

[0242] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0243] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

[0244] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The above-mentioned module-composed unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0245] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the methods of various embodiments of the present application.

[0246] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor.

[0247] The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.

[0248] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0249] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0250] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic control unit or a main control device.

[0251] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0252] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A task processing method, characterized in that: include: Get multiple tasks to be executed, each of which is pre-configured with a priority; Read all pending tasks in descending order of priority; When reading each task to be executed, determining whether the task to be executed can be executed according to the dependencies between different tasks to be executed, the resource requirements of the tasks to be executed, and the current remaining resources, wherein the resource requirements include the number of processors; When it is determined that the task to be executed can be executed, the task to be executed is executed.

2. The method according to claim 1, characterized in that: When reading each task to be executed, determining whether the task to be executed can be executed according to the dependency relationship between different tasks to be executed, the resource requirements of the tasks to be executed, and the current remaining resources includes: When reading each task to be executed, determining the predecessor task information of the task to be executed according to the dependency relationship between different tasks to be executed; wherein the predecessor task information is used to indicate whether there is a predecessor task for the task to be executed, and the execution status of the predecessor task; If the predecessor task information indicates that the task to be executed does not have the predecessor task, determining whether the resource requirement of the task to be executed is greater than or equal to the current remaining resources; If the predecessor task information indicates that the task to be executed has a predecessor task and the predecessor task has been completed, determining whether the resource requirement of the task to be executed is greater than or equal to the current remaining resources; If the resource demand is greater than or equal to the current remaining resources, determining that the task to be executed can be executed; If the resource requirement is less than the current remaining resources, it is determined that the task to be executed cannot be executed.

3. The method according to claim 2, characterized in that After determining the predecessor task information of each task to be executed according to the dependency relationship between different tasks to be executed when reading each task to be executed, the method further includes: If the predecessor task information indicates that the task to be executed has the predecessor task, and the predecessor task has not been completed, it is determined that the task to be executed cannot be executed.

4. The method according to claim 2 or 3, characterized in that: The method further comprises: Inputting the execution status of the task to be executed into a task status information database; Accordingly, determining the predecessor task information of the task to be executed according to the dependency relationship between different tasks to be executed includes: Determine whether there is a predecessor task for the task to be executed according to the dependency relationship between the tasks to be executed; If the predecessor task exists, the execution status of the predecessor task is obtained from the task status information database.

5. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: For each task to be executed, adjusting the resource requirement of the task to be executed in a third time period according to the execution information of the task to be executed in the first time period and the execution information of the task to be executed in the second time period; The second time period is later than the first time period and earlier than the third time period, and the execution information includes resource requirements and execution duration.

6. The method according to claim 5, characterized in that The step of adjusting, for each task to be executed, the resource requirement of the task to be executed in the third time period according to the execution information of the task to be executed in the first time period and the execution information of the task to be executed in the second time period includes: For each task to be executed, if the resource requirement of the first time period is less than the resource requirement of the second time period, and the duration difference between the execution duration of the first time period and the execution duration of the second time period exceeds a preset duration difference, then the resource requirement of the task to be executed in the third time period is increased; If the resource demand of the first time period is less than the resource demand of the second time period, and the duration difference does not exceed a preset duration difference, the resource demand of the third time period is reduced.

7. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: After all pending tasks are completed, determine the execution order of each pending task; For each task to be executed, re-determine a new execution order of the task to be executed according to the execution order of the task to be executed and the historical execution order; According to the new execution order, the priorities of all tasks to be executed are updated to obtain the new priority of each task to be executed.

8. The method according to any one of claims 1 to 3, characterized in that: Before acquiring a plurality of tasks to be executed, the method further includes: According to the dependency relationship between different executable tasks, a priority is configured for each executable task in the plurality of executable tasks; wherein the plurality of tasks to be executed belong to the plurality of executable tasks.

9. The method according to claim 8, characterized in that The step of configuring a priority for each of the multiple executable tasks according to the dependency relationship between different executable tasks; wherein the multiple tasks to be executed belong to the multiple executable tasks, including: randomly assigning an initial priority to each of the plurality of executable tasks; Read each executable task in order from the highest to the lowest initial priority; When reading each executable task, determining whether the executable task has a preceding executable task according to the dependency relationship between different executable tasks; If the executable task has the predecessor executable task, the initial priority of the executable task is adjusted according to the current priority of the predecessor executable task so that the priority of the executable task is less than the current priority of the predecessor executable task, and the priority of the executable task is obtained.

10. A task processing device, characterized in that: include: An acquisition module is used to acquire multiple tasks to be executed, each of which is pre-configured with a priority; The reading module is used to read all pending tasks in descending order of priority; A determination module, configured to determine whether each task to be executed can be executed according to the dependencies between different tasks to be executed, the resource requirements of the tasks to be executed, and the current remaining resources when reading each task to be executed, wherein the resource requirements include the number of processors; The execution module is used to execute the task to be executed when it is determined that the task to be executed can be executed.

11. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 9 when executed by a processor.

13. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 9 when being executed by a processor.