Task scheduling method, electronic device and computer program product

Abstract

The invention discloses a task scheduling method, electronic equipment and a computer program product, and relates to the technical field of data processing.The method comprises the steps that reusable procedures are determined from data processing tasks to be scheduled, the data processing tasks are composed of at least one atomization procedure, and the reusable procedures are reusable atomization procedures; and scheduling the data processing task through a dynamic planning algorithm, and multiplexing the reusable process in the scheduling process. The resource utilization rate and the task scheduling efficiency in the complex data processing task scheduling process can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a task scheduling method, an electronic device, and a computer program product. Background Art

[0002] Data processing tasks refer to tasks that perform a series of processing and transformations on raw data to generate useful information or insights. They are widely used in fields such as big data analysis, machine learning, and data warehouse management. They are characterized by multi-stage processing, computational intensiveness, data dependence, and diverse resource requirements. In data processing tasks, reasonable task scheduling is crucial to improving the overall performance of the system.

[0003] However, traditional task scheduling methods such as first-in-first-out and shortest job first have unreasonable allocation of computing resources and low utilization when processing complex data processing tasks, resulting in low task scheduling efficiency. Summary of the invention

[0004] The main purpose of this application is to provide a task scheduling method, electronic equipment and computer program product, aiming to solve the technical problems of low resource utilization and task scheduling efficiency in traditional task scheduling methods when processing complex data processing tasks.

[0005] To achieve the above object, the present application provides a task scheduling method, the method comprising:

[0006] Determine a reusable process from the data processing tasks to be scheduled, wherein the data processing task is composed of at least one atomic process, and the reusable process is a reusable atomic process;

[0007] The data processing tasks are scheduled through a dynamic programming algorithm, and during the scheduling process, reusable processes are reused.

[0008] In one embodiment, the step of determining a reusable process from the data processing tasks to be scheduled includes:

[0009] Atomize and decompose the data processing task to be scheduled to obtain an atomic process list consisting of the atomic processes of the data processing task;

[0010] Parsing the atomization processes in the atomization process list to obtain process input information and process operation information of each atomization process in the atomization process list;

[0011] Based on the process input information and the process operation information, a reusable process is determined from the atomic process list.

[0012] In one embodiment, the process input information includes input source information and input condition information;

[0013] The step of determining a reusable process from the atomic process list based on the process input information and the process operation information includes:

[0014] From the atomic process list, atomic processes having the same input source information, input condition information and process operation information are determined as reusable processes.

[0015] In one embodiment, the method further comprises:

[0016] When the data sources pointed to by the input source information are the same, it is determined that the input source information is the same.

[0017] In one embodiment, the method further comprises:

[0018] When the data sources are the same reusable process, make sure that the data sources are the same.

[0019] In one embodiment, the number of data processing tasks to be scheduled is greater than one, and the step of scheduling the data processing tasks by using a dynamic programming algorithm includes:

[0020] Based on the process flow chart of each of the data processing tasks, the atomic processes of each of the data processing tasks are scheduled through a dynamic programming algorithm, wherein the process flow chart is a directed acyclic graph, the nodes in the directed acyclic graph represent the atomic processes, and the directed edges in the directed acyclic graph represent the dependency relationships between the atomic processes.

[0021] In one embodiment, the method further comprises:

[0022] Atomizing and decomposing each of the data processing tasks to obtain the atomic steps of each of the data processing tasks and the dependency relationship between the atomic steps;

[0023] Based on the atomic processes of each of the data processing tasks and the dependencies between the atomic processes, a process flow chart of each of the data processing tasks is constructed.

[0024] In one embodiment, the step of reusing the reusable process during the scheduling process includes:

[0025] During the scheduling process, the execution results of the reusable process are cached, and when the reusable process is executed again, the cached execution results are output accordingly to reuse the reusable process.

[0026] In addition, to achieve the above-mentioned purpose, the present application also provides an electronic device, which includes: a memory, a processor, and a task scheduling program stored in the memory and executable on the processor, and the task scheduling program implements the steps of the task scheduling method as described above when executed by the processor.

[0027] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, on which a task scheduling program is stored, and when the task scheduling program is executed by a processor, the steps of the task scheduling method as described above are implemented.

[0028] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a task scheduling program, and when the task scheduling program is executed by a processor, the steps of the task scheduling method as described above are implemented.

[0029] The embodiment of the present application provides a task scheduling method, electronic device and computer program product, which relate to the field of data processing technology. The embodiment of the present application determines a reusable process from the data processing task to be scheduled, wherein the data processing task is composed of at least one atomic process, and the reusable process is a reusable atomic process; the data processing task is scheduled through a dynamic programming algorithm, and the reusable process is reused during the scheduling process, so that the embodiment of the present application can decompose the complex data processing task into indivisible basic operation units, namely atomic processes, so as to manage and schedule tasks in a more fine-grained manner and improve resource utilization.

[0030] In addition, the embodiment of the present application also marks the repeated atomic processes in complex data processing tasks as reusable processes, and reuses the reusable processes during the scheduling process, thereby significantly reducing repeated calculations and improving the execution efficiency of tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0032] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0033] Figure 1 A flowchart of the first embodiment of the task scheduling method of the present application is provided;

[0034] Figure 2 A schematic diagram of a process for determining reusable processes in the task scheduling method of the present application;

[0035] Figure 3 A schematic diagram of the process of scheduling reuse for dynamic planning in the task scheduling method of this application;

[0036] Figure 4 A schematic diagram of the construction process flow chart of the task scheduling method of this application;

[0037] Figure 5 A schematic diagram of the device structure of the hardware operating environment involved in the task scheduling method in the embodiment of the present application.

[0038] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0039] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0040] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0041] The main solution of the embodiment of the present application is: determine the reusable process from the data processing tasks to be scheduled, wherein the data processing task is composed of at least one atomic process, and the reusable process is a reusable atomic process; schedule the data processing tasks through a dynamic programming algorithm, and reuse the reusable process during the scheduling process.

[0042] The embodiments of the present application can decompose complex data processing tasks into indivisible basic operation units, namely atomic processes, so as to manage and schedule tasks in a more fine-grained manner and improve resource utilization.

[0043] In addition, the embodiment of the present application also marks the repeated atomic processes in complex data processing tasks as reusable processes, and reuses the reusable processes during the scheduling process, thereby significantly reducing repeated calculations and improving the execution efficiency of tasks.

[0044] The execution subject of the embodiments of the present application is an electronic device, which may include but is not limited to mobile terminals such as mobile phones (such as mobile phones), laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs (Televisions), desktop computers, etc., or any electronic device that can achieve the above functions, and the embodiments of the present application do not specifically limit this. The following uses electronic devices as the execution subject as an example to illustrate the following embodiments of the present application.

[0045] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0046] This application proposes a task scheduling method of a first embodiment.

[0047] Please refer to Figure 1 , Figure 1 A flowchart diagram of the first embodiment of the task scheduling method of the present application is provided.

[0048] In this embodiment, the task scheduling method includes steps S100 to S200:

[0049] Step S100, determining a reusable process from the data processing tasks to be scheduled, wherein the data processing task is composed of at least one atomic process, and the reusable process is a reusable atomic process;

[0050] Those skilled in the art will know that data processing tasks refer to tasks that perform a series of processing and conversions on raw data to generate useful information or insights, which usually involve multiple processing stages, each of which may include different operations, such as data cleaning, data integration, format conversion, data classification, data evaluation, etc. Taking the generation and processing of data components as an example, this data processing task mainly involves five processing stages: raw data collection, raw data cleaning, basic data integration, data component development, and data component circulation, trading, and maintenance. Each processing stage includes multiple indivisible basic operations.

[0051] In this embodiment, an atomic process refers to an indivisible basic operation unit in a data processing task. Each atomic process is an independent small task with clear input, output and operation. A reusable process refers to an atomic process with the same input, output and operation.

[0052] Exemplarily, the input, output and operation of atomization processes A and B are the same, the input, output and operation of atomization processes C and D are the same, and the inputs of atomization processes A and C are different. In this case, A and B can be reused with each other, C and D can be reused with each other, A and B are the same reusable process, C and D are the same reusable process, but A and B cannot be reused with C and D.

[0053] It is understandable that when the input and operation of two atomization processes are the same, their outputs are the same. Therefore, whether the two atomization processes are the same reusable process can be determined by whether the input and operation of the atomization processes are the same.

[0054] In this embodiment, when processing complex data processing tasks, the electronic device will atomize the data processing tasks to be scheduled, decompose them into multiple atomic processes, and determine the input and operation of each atomic process, so as to identify which atomic processes can be reused with each other based on the input and operation, and then identify the reusable processes.

[0055] For example, Figure 2 As shown, in a feasible implementation manner, step S100 may include steps S110 to S130:

[0056] Step S110, atomically decomposing the data processing task to be scheduled to obtain an atomic process list consisting of the atomic processes of the data processing task;

[0057] It should be noted that atomic decomposition refers to decomposing a complex data processing task into multiple indivisible basic operation units, namely, atomic processes.

[0058] In this implementation, all data processing tasks to be scheduled are atomically decomposed to obtain atomic steps corresponding to each data processing task.

[0059] This implementation method can establish an exclusive atomic process list for each data processing task based on the data processing task, specifically storing the atomic processes obtained after the atomic decomposition of the data processing task, or establish a general atomic process list to store the atomic processes of all data processing tasks.

[0060] It should be noted that, after the atomic processes of each data processing task are disassembled, the present embodiment needs to mark all the atomic processes in order to identify different atomic processes and confirm the data processing task to which each atomic process belongs.

[0061] Step S120, parsing the atomization processes in the atomization process list to obtain process input information and process operation information of each atomization process in the atomization process list;

[0062] It should be noted that process input information refers to the description of the input data required for the execution of the atomic process, usually including the source, format and content of the input data. Process operation information refers to the description of the specific operation of the atomic process, usually including the steps, logic and methods of the operation.

[0063] After establishing the atomic process list, this embodiment also needs to parse each atomic process in the atomic process list to determine its process input information and process operation information in order to identify reusable processes.

[0064] Step S130, based on the process input information and the process operation information, determine the reusable process from the atomic process list.

[0065] In this embodiment, whether the input of the atomic process is the same can be determined through the process input information, and whether the operation of the atomic process is the same can be determined through the process operation information, so that the atomic processes with the same input and operation are determined as the same reusable process and marked, so that the same reusable process can be reused during the scheduling process to avoid repeated execution.

[0066] Exemplarily, in a feasible implementation, the process input information includes input source information and input condition information, and step S130 may include step S131;

[0067] Step S131, from the atomic process list, determine the atomic processes with the same input source information, input condition information and process operation information as reusable processes.

[0068] It should be noted that, in this embodiment, the input source information indicates the source of the input data, that is, the data source, and the input condition information refers to the conditions that the input data needs to meet, such as the time range of the data, the format requirements of the data, the screening conditions of the data, etc.

[0069] In this embodiment, when the input source information and the input condition information are the same, it can be determined that the input of the atomic process is the same, so that when the process operation information is the same, it is determined that the output results can be reused with each other and only need to be executed once without repeated execution of the reusable process.

[0070] In a feasible implementation manner, the task scheduling method further includes step A10:

[0071] Step A10, when the data sources pointed to by the input source information are the same, it is determined that the input source information is the same.

[0072] In this embodiment, even if there are slight differences in the input source information of two atomic processes (such as different descriptions, names or numbers of the same data source), as long as the data sources they point to are the same, the input source information of the two atomic processes are considered to be the same.

[0073] It should be noted that in this embodiment, the data sources pointed to by the input source information of the atomic process are divided into two types, one is a preset database, data table, etc., and the other is other atomic processes that need to be executed before the atomic process. That is, the input data of the atomic process can be obtained from the output results of other atomic processes, or from a preset database, data table, or part of the output results of other atomic processes, and then another part is obtained from the preset database, data table, together forming the complete input data.

[0074] In a feasible implementation manner, the task scheduling method further includes step B10:

[0075] Step B10, when the data sources are the same reusable process, determine that the data sources are the same.

[0076] In this embodiment, when the data sources pointed to by the input source information of two atomic processes are other atomic processes and are not the same atomic process, it is necessary to first determine whether the other atomic processes pointed to are reusable processes. If the other atomic processes pointed to are atomic processes that can be reused with each other, that is, the same reusable process, then the data sources are still determined to be the same.

[0077] It is not difficult to understand that when the input source information points to multiple data sources, we can first determine whether the number of data sources is consistent, and then determine whether each data source is the same if they are consistent.

[0078] It is worth mentioning that in addition to determining reusable processes from the atomic process list in the above-mentioned manner, after the atomic processes of each data processing task are obtained by atomic decomposition, the atomic processes can be first analyzed to determine the process operation information and process input information of each atomic process, and then the atomic processes can be classified according to the process operation information. An atomic process list is generated for each type of atomic process. Finally, based on the process operation information, the atomic processes belonging to the same category in the atomic process list are judged for input consistency, thereby determining the atomic processes that can be reused with each other under each category.

[0079] It can be understood that after determining the reusable processes, this embodiment also needs to classify and number the reusable processes, and assign the same number or identifier to the atomic processes with the same input and operation, to ensure that during the scheduling process, it is accurate to identify whether the atomic process currently to be executed can be reused.

[0080] Step S200, scheduling data processing tasks through a dynamic programming algorithm, and reusing reusable processes during the scheduling process.

[0081] It should be noted that this embodiment can be scheduled in units of data processing tasks or in units of atomic processes. When scheduling is performed in units of data processing tasks, after each thread of the electronic device completes a data processing task, it will execute the next data processing task in sequence from the remaining data processing tasks to be scheduled according to the data processing task execution order calculated by the dynamic programming algorithm. When scheduling is performed in units of atomic processes, after each thread of the electronic device completes an atomic process, it will execute the next atomic process in sequence from the remaining atomic processes according to the atomic process execution order calculated by the dynamic programming algorithm.

[0082] It is worth mentioning that, when scheduling data processing tasks through a dynamic programming algorithm, this embodiment will take into account the impact of reusable process reuse on task scheduling.

[0083] Exemplarily, when scheduling is performed in units of data processing tasks, the execution order of the atomic processes in data processing task A is atomic processes A1, A2, A3 and A4, where A1 and A3 are atomic processes that can be reused with each other, the computing time required for A1 is 3, the computing time required for A2 is 2, the computing time required for A3 is 3, and the computing time required for A4 is 4. In the traditional task scheduling algorithm, the total computing time required for A is 12, while in the task scheduling method provided in this embodiment, A1 and A3 are reused, A1 is executed only once, and A3 directly calls the output result of A1, so the dynamic programming algorithm calculates that the total computing time required for data processing task A is 9.

[0084] Correspondingly, when scheduling in units of atomic processes, the impact of reusable processes on task scheduling will also be taken into consideration.

[0085] For example, Figure 3 As shown, the number of data processing tasks to be scheduled is greater than one, and the step of scheduling the data processing tasks by a dynamic programming algorithm in step S200 includes step S210;

[0086] Step S210, based on the process flow chart of each data processing task, the atomic process of each data processing task is scheduled through a dynamic programming algorithm, wherein the process flow chart is a directed acyclic graph, the nodes in the directed acyclic graph represent the atomic processes, and the directed edges in the directed acyclic graph represent the dependency relationship between the atomic processes.

[0087] It should be noted that in this embodiment, the process flow chart refers to a directed acyclic graph that describes each atomic process in a data processing task and its dependencies. Dependency refers to the order of the atomic processes in a data processing task. Dependency determines which atomic processes must be completed before other atomic processes to ensure the correctness and effectiveness of the entire task.

[0088] In this embodiment, a corresponding process flow chart is constructed in advance for each data processing task to be scheduled, so that the atomic processes of each data processing task are scheduled through a dynamic programming algorithm to determine the execution order of each atomic process.

[0089] For example, Figure 4 As shown, in a feasible implementation manner, the task scheduling method further includes steps S300 to S400:

[0090] Step S300, atomically decompose each data processing task to obtain the atomic steps of each data processing task and the dependency relationship between the atomic steps;

[0091] Step S400: construct a process flow chart of each data processing task based on the atomic processes of each data processing task and the dependency relationship between the atomic processes.

[0092] This implementation manner can atomically decompose the data processing tasks to obtain the atomic processes of each data processing task and the dependencies between the atomic processes, thereby constructing a process flow chart for each data processing task.

[0093] In this embodiment, if Figure 3 As shown, in step S200, the step of reusing the reusable process during the scheduling process includes step S220:

[0094] Step S220, during the scheduling process, the execution result of the reusable process is cached, and when the reusable process is executed again, the cached execution result is output accordingly to reuse the reusable process.

[0095] In this embodiment, the reuse of the reusable process is achieved by caching the execution result when the reusable process is executed for the first time, and then directly calling the cached execution result when the reusable process needs to be executed next time.

[0096] Through the above method, each reusable process only needs to be executed once. There is no need to execute it next time it is encountered, and the corresponding execution result can be directly obtained from the cache.

[0097] This embodiment decomposes complex data processing tasks into atomic processes, identifies the reusable parts, i.e., reusable processes, and then uses a dynamic programming algorithm for efficient scheduling, thereby allocating computing resources more reasonably and avoiding resource waste. At the same time, by caching the execution results of reusable processes, unnecessary repeated calculations are reduced, the speed and efficiency of task execution are improved, more tasks can be processed at the same time, and the system throughput is improved.

[0098] It is worth mentioning that in this embodiment, when scheduling data processing tasks through a dynamic programming algorithm, the user can set a suitable objective function according to actual needs, such as the shortest expected execution time, the least resource consumption, the maximum number of reuses of reusable processes, etc., or comprehensively measure multiple indicators through preset weight parameters.

[0099] Exemplarily, a priority can be set in advance for each data processing task to be scheduled, and the number of reusable processes in each data processing task can be counted. Through preset weight parameters, the two indicators of priority and the number of reusable processes are comprehensively considered to form an objective function of the dynamic programming algorithm, and the data processing tasks are scheduled based on the objective function to ensure that the scheduling results comprehensively consider priority and atomic process reuse, thereby ensuring that high-priority data processing tasks are given priority and the overall task scheduling efficiency is ensured.

[0100] It should also be noted that, after each round of scheduling is completed, this embodiment will generate a scheduling task execution log for staff to view.

[0101] It can be understood that when scheduling is performed through a dynamic programming algorithm, due to the limited resources of the electronic device, the scheduling of all data processing tasks can be completed in multiple rounds, and each round only selects a fixed number of data processing tasks or data that the computing resources of the electronic device can bear for execution.

[0102] This embodiment determines reusable processes from the data processing tasks to be scheduled, wherein the data processing tasks are composed of at least one atomic process and the reusable processes are reusable atomic processes; the data processing tasks are scheduled through a dynamic programming algorithm, and the reusable processes are reused during the scheduling process, so that this embodiment can decompose complex data processing tasks into indivisible basic operation units, namely atomic processes, so as to manage and schedule tasks in a finer granularity and improve resource utilization.

[0103] In addition, this embodiment also marks repeated atomic processes in complex data processing tasks as reusable processes, and reuses the reusable processes during the scheduling process, thereby significantly reducing repeated calculations and improving task execution efficiency.

[0104] In order to facilitate understanding of the core concept of the task scheduling method provided in the above embodiment, a specific embodiment is given:

[0105] In this specific embodiment, all data processing tasks to be scheduled that meet the task scheduling conditions can be found based on the priority queuing scheduling method, wherein the task scheduling condition is a condition that triggers task scheduling, which can be time or other preconditions.

[0106] Taking time as an example, if a data processing task is set to be scheduled at a certain time point, then these tasks need to be scheduled at that time point. For example, if there are 10 tasks to be executed at 18:00, then these 10 tasks are the data processing tasks to be scheduled.

[0107] Taking the predecessor task as an example, a data processing task to be scheduled when a predecessor task is completed is set. When the predecessor task is completed, this part of the task needs to be scheduled. For example, if a predecessor task is the input of data resource table A, then when the predecessor task of data resource table A input is completed, it will trigger the data processing task that needs to use data resource table A to start scheduling and running.

[0108] In this specific implementation, in addition to the task scheduling conditions, each data processing task can also set a priority.

[0109] In this specific embodiment, when the number of tasks that meet the task scheduling conditions exceeds the preset maximum number, they can be divided according to priority and executed in batches, which can avoid congestion or failure caused by a large number of data processing tasks. When the number of tasks that meet the task scheduling conditions is below the preset maximum number, they can all be executed without considering the priority.

[0110] In this specific embodiment, for data processing tasks that meet the task scheduling conditions, all the processes included therein are analyzed, and the atomic processes are parsed to obtain reusable atomic processes, and a process flow chart is constructed.

[0111] Generally speaking, data processing tasks mainly include processes such as reading data, calculation, output, safety detection operators, standard operators, and quality inspection operators. By disassembling each process, we can get atomic processes, so as to determine the input data and computing operations required for each data processing task in each atomic process, and then determine which atomic processes can be reused.

[0112] In this specific embodiment, a dynamic programming method is used to perform dynamic programming analysis based on a directed acyclic graph of task execution to find common nodes and optimal methods, so that tasks that can shorten computing tasks and time are included in the current round of scheduling pool.

[0113] This specific embodiment coordinates all tasks to be scheduled and finds the optimal method to execute these tasks faster and more efficiently, for example, the shortest path length, the least resource consumption, and the shortest expected execution time. It can be evaluated by one of the indicators or by a comprehensive evaluation of multiple indicators. For example, a comprehensive score can be achieved through weighted average.

[0114] After each process is atomized, all tasks to be scheduled are planned in a coordinated manner to find the same nodes that can be reused by the tasks to be scheduled. That is, the input and output of multiple tasks that can reuse the node are the same, so that reuse is guaranteed. In the node that can be reused, the output result obtained after executing the node can be stored in the memory, so when other tasks reuse the node, they can directly read the result in the memory without repeated execution.

[0115] In this specific embodiment, after dynamic planning, the task execution begins. The scheduling executor executes the calculation process according to the result of dynamic planning, and the same atomic process is only executed once, and as much as possible, more calculation tasks are covered by one execution.

[0116] Since the scheduling order has been planned, and we know which nodes of which tasks can be executed only once, we can reuse the tasks of the nodes later. Instead of executing them, we can directly use the corresponding results. This saves computing resources and time, and covers more computing tasks at the same time. Due to the bottleneck of I / O read and write efficiency, if each task needs to read and write data separately, the efficiency will be greatly reduced. However, since the required input data and results of the reused nodes are stored in the memory, repeated reading and writing and calculation are avoided, which can greatly reduce the time.

[0117] Finally, this specific embodiment records the scheduling task execution log and starts the next round of scheduling analysis.

[0118] This specific implementation atomizes the complex processes of data processing tasks, and adopts dynamic programming methods combined with intelligent computing resource evaluation and queuing methods to calculate the sub-processes (i.e., reusable processes) at one time. This can increase the number of tasks scheduled in parallel and the execution efficiency of the number of tasks scheduled in parallel, make data reading and writing processing faster, and enable a larger amount of parallel scheduled tasks.

[0119] In addition, please refer to Figure 5 , Figure 5 A schematic diagram of the device structure of the hardware operating environment involved in the task scheduling method in the embodiment of the present application.

[0120] The present application also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the steps of the task scheduling method in the above-mentioned embodiment.

[0121] Reference below Figure 5 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present application. The electronic devices in the embodiments of the present application may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs (Televisions), desktop computers, etc., or any electronic device capable of implementing the above functions. Figure 5 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0122] like Figure 5As shown, the electronic device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0123] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0124] The electronic device provided by the present application adopts the task scheduling method in the above embodiment, which can solve the technical problem that the traditional task scheduling method has low resource utilization and task scheduling efficiency when processing complex data processing tasks. Compared with the prior art, the beneficial effects of the electronic device provided by the present application are the same as the beneficial effects of the task scheduling method provided by the above embodiment, and the other technical features in the electronic device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0125] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0126] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the above claims.

[0127] In addition, the present application also provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the steps of the task scheduling method in the above-mentioned embodiment.

[0128] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0129] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0130] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by an electronic device, the electronic device: determines a reusable process from the data processing tasks to be scheduled, wherein the data processing task is composed of at least one atomic process, and the reusable process is a reusable atomic process; schedules the data processing tasks through a dynamic programming algorithm, and reuses the reusable process during the scheduling process.

[0131] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0132] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0133] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0134] The computer-readable storage medium provided by the present application stores computer-readable program instructions (i.e., computer programs) for executing the steps of the above-mentioned task scheduling method, which can solve the technical problems of low resource utilization and task scheduling efficiency in traditional task scheduling methods when processing complex data processing tasks. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the task scheduling method provided by the above-mentioned embodiment, and will not be described in detail here.

[0135] In addition, an embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the task scheduling method in the above embodiment.

[0136] The computer program product provided by the present application can solve the technical problem that the traditional task scheduling method has low resource utilization and task scheduling efficiency when processing complex data processing tasks. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present application are the same as the beneficial effects of the task scheduling method provided by the above embodiment, which will not be repeated here.

[0137] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A task scheduling method, characterized in that: The method comprises: Determine a reusable process from the data processing tasks to be scheduled, wherein the data processing task is composed of at least one atomic process, and the reusable process is a reusable atomic process; The data processing tasks are scheduled through a dynamic programming algorithm, and during the scheduling process, reusable processes are reused.

2. The task scheduling method according to claim 1, characterized in that: The step of determining the reusable process from the data processing tasks to be scheduled includes: Atomize and decompose the data processing task to be scheduled to obtain an atomic process list consisting of the atomic processes of the data processing task; Parsing the atomization processes in the atomization process list to obtain process input information and process operation information of each atomization process in the atomization process list; Based on the process input information and the process operation information, a reusable process is determined from the atomic process list.

3. The task scheduling method according to claim 2, characterized in that: The process input information includes input source information and input condition information. The step of determining a reusable process from the atomic process list based on the process input information and the process operation information includes: From the atomic process list, atomic processes having the same input source information, input condition information and process operation information are determined as reusable processes.

4. The task scheduling method according to claim 3, characterized in that: The method further comprises: When the data sources pointed to by the input source information are the same, it is determined that the input source information is the same.

5. The task scheduling method according to claim 4, characterized in that: The method further comprises: When the data sources are the same reusable process, make sure that the data sources are the same.

6. The task scheduling method according to claim 1, characterized in that: The number of data processing tasks to be scheduled is greater than one, and the step of scheduling the data processing tasks by using a dynamic programming algorithm includes: Based on the process flow chart of each of the data processing tasks, the atomic processes of each of the data processing tasks are scheduled through a dynamic programming algorithm, wherein the process flow chart is a directed acyclic graph, the nodes in the directed acyclic graph represent the atomic processes, and the directed edges in the directed acyclic graph represent the dependency relationships between the atomic processes.

7. The task scheduling method according to claim 6, characterized in that: The method further comprises: Atomizing and decomposing each of the data processing tasks to obtain the atomic steps of each of the data processing tasks and the dependency relationship between the atomic steps; Based on the atomic processes of each of the data processing tasks and the dependencies between the atomic processes, a process flow chart of each of the data processing tasks is constructed.

8. The task scheduling method according to claim 1, characterized in that: The step of reusing the reusable process during the scheduling process includes: During the scheduling process, the execution results of the reusable process are cached, and when the reusable process is executed again, the cached execution results are output accordingly to reuse the reusable process.

9. An electronic device, characterized in that: include: A memory, a processor, and a task scheduling program stored in the memory and executable on the processor, wherein the task scheduling program implements the steps of the task scheduling method according to any one of claims 1 to 8 when executed by the processor.

10. A computer program product, characterized in that The computer program product comprises a task scheduling program, and when the task scheduling program is executed by a processor, the steps of the task scheduling method according to any one of claims 1 to 8 are implemented.

Images