Task scheduling method and terminal

Abstract

The present invention discloses a task scheduling method and terminal. After receiving a task, the task data volume in each computer room is first counted, and the task scheduling is performed based on the computer room where the data is located. Specifically, when the task data are all in the same computer room, the task is directly assigned to the computer room if the computer room resources are sufficient. When the resources are insufficient, the corresponding computer room is selected based on the priority of the task. When the task data are not in the same computer room, the task is scheduled based on the resource situation of the computer room with the largest amount of data required by the task. Therefore, by allocating tasks based on the amount of data required by the task and the location of the data, the IO traffic between computer rooms can be reduced, and the rationality of task scheduling and the efficiency of task processing can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of task scheduling, and in particular to a task scheduling method and a terminal. Background Art

[0002] Currently, there are many big data query tasks. Usually, such big data query tasks are scheduled and allocated by a management center based on the remaining resources of each sub-node. When a node has sufficient resources, the task is scheduled first.

[0003] However, currently many sub-nodes may be located in different locations or computer rooms. When a task is scheduled to a certain computer room, other required data may need to be transmitted and obtained from other computer rooms, resulting in low task scheduling efficiency. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a task scheduling method and a terminal, which can improve the rationality of task scheduling and the efficiency of task processing.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A task scheduling method comprises the steps of:

[0007] Receive tasks and count the amount of data for the tasks in each computer room;

[0008] If the data required for the task is located in the same computer room, determine whether the remaining resources of the computer room exceed the data volume; if so, assign the task to the computer room; if not, select a computer room to be assigned according to the priority of the task;

[0009] If the data required for the task is not located in the same computer room, determine whether the remaining resources of the computer room with the largest data volume exceed the data volume. If so, assign the task to the computer room with the largest data volume. If not, split the task and assign it to multiple computer rooms.

[0010] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0011] A task scheduling terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0012] Receive tasks and count the amount of data for the tasks in each computer room;

[0013] If the data required for the task is located in the same computer room, determine whether the remaining resources of the computer room exceed the data volume; if so, assign the task to the computer room; if not, select a computer room to be assigned according to the priority of the task;

[0014] If the data required for the task is not located in the same computer room, determine whether the remaining resources of the computer room with the largest data volume exceed the data volume. If so, assign the task to the computer room with the largest data volume. If not, split the task and assign it to multiple computer rooms.

[0015] The beneficial effects of the present invention are as follows: after receiving a task, the amount of data generated by the task in each computer room is first counted, and the task is scheduled based on the computer room where the data is located. Specifically, when the task data is all in the same computer room, the task is directly assigned to that computer room if the computer room resources are sufficient. If the resources are insufficient, the corresponding computer room is selected based on the priority of the task. When the task data is not in the same computer room, the task is scheduled based on the resource situation of the computer room with the largest amount of data required by the task. Therefore, by allocating tasks based on the amount of data required by the task and the location of the data, the I / O traffic between computer rooms can be reduced, and the rationality of task scheduling and the efficiency of task processing can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flowchart of a task scheduling method according to an embodiment of the present invention;

[0017] Figure 2 A schematic diagram of a task scheduling terminal according to an embodiment of the present invention;

[0018] Description of labels:

[0019] 1. A task scheduling terminal; 2. Memory; 3. Processor. DETAILED DESCRIPTION

[0020] To illustrate the technical content, achieved objectives and effects of the present invention in detail, the following description is given in conjunction with the embodiments and accompanying drawings.

[0021] Please refer to Figure 1 , an embodiment of the present invention provides a task scheduling method, comprising the steps of:

[0022] Receive tasks and count the amount of data for the tasks in each computer room;

[0023] If the data required for the task is located in the same computer room, determine whether the remaining resources of the computer room exceed the data volume; if so, assign the task to the computer room; if not, select a computer room to be assigned according to the priority of the task;

[0024] If the data required for the task is not located in the same computer room, determine whether the remaining resources of the computer room with the largest data volume exceed the data volume. If so, assign the task to the computer room with the largest data volume. If not, split the task and assign it to multiple computer rooms.

[0025] As can be seen from the above description, the beneficial effects of the present invention are as follows: after receiving a task, the amount of data in each computer room is first counted, and the task is scheduled in combination with the computer room where the data is located; specifically, when the task data are all in the same computer room, the task is directly assigned to the computer room if the computer room resources are sufficient, and when the resources are insufficient, the corresponding computer room needs to be selected based on the priority of the task; and when the task data are not in the same computer room, the task is scheduled based on the resource situation of the computer room with the largest amount of data required for the task. Therefore, by allocating tasks based on the amount of data required for the task and the location of the data, the IO traffic between computer rooms can be reduced, and the rationality of task scheduling and the efficiency of task processing can be improved.

[0026] Furthermore, the receiving of tasks and counting the amount of data of the tasks in each computer room includes:

[0027] Receive a task and obtain the task information carried by the task, the task information including the name of the database table where the required data is located and its query conditions;

[0028] The amount of data of the task in each database table is counted according to the database table name and its query condition, and the amount of data of the task in each computer room is counted according to the positional relationship between the database table and the computer room.

[0029] From the above description, it can be seen that by performing statistics on the amount of data in each computer room according to the database table name and query conditions carried by the task, the required amount of data in each computer room can be obtained quickly and accurately.

[0030] Furthermore, after receiving the task and counting the data volume of the task in each computer room, the method further includes:

[0031] Determine the computer room corresponding to the data required for the task and monitor the resources of the computer room.

[0032] From the above description, it can be seen that resource monitoring of the computer room where the data required for the task is located facilitates subsequent task scheduling between computer rooms and improves the efficiency of task processing.

[0033] Furthermore, if the number of tasks is not exceeded, selecting a computer room to be assigned according to the priority of the task includes:

[0034] If the task is of high priority, the task is assigned to the computer room where the data is located and is processed first;

[0035] Otherwise, the task is assigned to other computer rooms. When the remaining resources of the computer room where the data is located reach the data volume, the task is assigned to the computer room where the data is located.

[0036] From the above description, it can be seen that when a task has a high priority, it can be processed first in the computer room where the data is located. Otherwise, the task will be assigned to other computer rooms. When the remaining resources in the computer room where the data is located are sufficient, the task will be assigned back to the computer room where the data is located, thereby improving the rationality of task scheduling.

[0037] Furthermore, if the number of tasks is not exceeded, splitting the tasks and distributing them to multiple computer rooms includes:

[0038] Split the task into subtasks corresponding to the computer room where the data required for the task is located;

[0039] The subtask is pushed to the corresponding computer room, and when the remaining resources of the computer room exceed the data volume of the subtask, the subtask is processed.

[0040] From the above description, we can see that tasks are split according to the computer room where the data is located. After being split into subtasks according to the computer room, the subtasks are pushed to the corresponding computer room. When the remaining resources of the computer room exceed the data volume of the subtask, the subtask is processed, thereby further reducing the IO traffic between computer rooms.

[0041] Please refer to Figure 2 Another embodiment of the present invention provides a task scheduling terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0042] Receive tasks and count the amount of data for the tasks in each computer room;

[0043] If the data required for the task is located in the same computer room, determine whether the remaining resources of the computer room exceed the data volume; if so, assign the task to the computer room; if not, select a computer room to be assigned according to the priority of the task;

[0044] If the data required for the task is not located in the same computer room, determine whether the remaining resources of the computer room with the largest data volume exceed the data volume. If so, assign the task to the computer room with the largest data volume. If not, split the task and assign it to multiple computer rooms.

[0045] As can be seen from the above description, the beneficial effects of the present invention are as follows: after receiving a task, the amount of data in each computer room is first counted, and the task is scheduled in combination with the computer room where the data is located; specifically, when the task data are all in the same computer room, the task is directly assigned to the computer room if the computer room resources are sufficient, and when the resources are insufficient, the corresponding computer room needs to be selected based on the priority of the task; and when the task data are not in the same computer room, the task is scheduled based on the resource situation of the computer room with the largest amount of data required for the task. Therefore, by allocating tasks based on the amount of data required for the task and the location of the data, the IO traffic between computer rooms can be reduced, and the rationality of task scheduling and the efficiency of task processing can be improved.

[0046] Furthermore, the receiving of tasks and counting the amount of data of the tasks in each computer room includes:

[0047] Receive a task and obtain the task information carried by the task, the task information including the name of the database table where the required data is located and its query conditions;

[0048] The amount of data of the task in each database table is counted according to the database table name and its query condition, and the amount of data of the task in each computer room is counted according to the positional relationship between the database table and the computer room.

[0049] From the above description, it can be seen that by performing statistics on the amount of data in each computer room according to the database table name and query conditions carried by the task, the required amount of data in each computer room can be obtained quickly and accurately.

[0050] Furthermore, after receiving the task and counting the data volume of the task in each computer room, the method further includes:

[0051] Determine the computer room corresponding to the data required for the task and monitor the resources of the computer room.

[0052] From the above description, it can be seen that resource monitoring of the computer room where the data required for the task is located facilitates subsequent task scheduling between computer rooms and improves the efficiency of task processing.

[0053] Furthermore, if the number of tasks is not exceeded, selecting a computer room to be assigned according to the priority of the task includes:

[0054] If the task is of high priority, the task is assigned to the computer room where the data is located and is processed first;

[0055] Otherwise, the task is assigned to other computer rooms. When the remaining resources of the computer room where the data is located reach the data volume, the task is assigned to the computer room where the data is located.

[0056] From the above description, it can be seen that when a task has a high priority, it can be processed first in the computer room where the data is located. Otherwise, the task will be assigned to other computer rooms. When the remaining resources in the computer room where the data is located are sufficient, the task will be assigned back to the computer room where the data is located, thereby improving the rationality of task scheduling.

[0057] Furthermore, if the number of tasks is not exceeded, splitting the tasks and distributing them to multiple computer rooms includes:

[0058] Split the task into subtasks corresponding to the computer room where the data required for the task is located;

[0059] The subtask is pushed to the corresponding computer room, and when the remaining resources of the computer room exceed the data volume of the subtask, the subtask is processed.

[0060] From the above description, we can see that tasks are split according to the computer room where the data is located. After being split into subtasks according to the computer room, the subtasks are pushed to the corresponding computer room. When the remaining resources of the computer room exceed the data volume of the subtask, the subtask is processed, thereby further reducing the IO traffic between computer rooms.

[0061] The task scheduling method and terminal described above are applicable to task scheduling projects, reducing IO traffic between computer rooms, improving the rationality of task scheduling and the efficiency of task processing, and are described below through specific implementation methods:

[0062] Example 1

[0063] Please refer to Figure 1 , a task scheduling method, comprising the steps of:

[0064] S1. Receive tasks and count the amount of data for the tasks in each computer room.

[0065] S11. Receive a task and obtain task information carried by the task, wherein the task information includes the name of the database table where the required data is located and its query conditions.

[0066] Specifically, the task information carried by each task needs to pass some additional parameters, including the names of the libraries and tables involved in this task, as well as the conditions involved in each library and table.

[0067] S12. Count the amount of data of the task in each database table according to the database table name and its query condition, and count the amount of data of the task in each computer room according to the positional relationship between the database table and the computer room.

[0068] Specifically, when the task analysis module receives a new task, it can preliminarily count the amount of data of this task in each database table based on the library table name and query conditions involved in this task, and count the amount of data of the task in each computer room based on the location relationship between the database table and the computer room.

[0069] S13: Determine the computer room corresponding to the data required for the task, and monitor the resources of the computer room.

[0070] Specifically, there is a monitoring program that can monitor the occupancy of various resources in each computer room in real time.

[0071] S2. If the data required for the task is located in the same computer room, determine whether the remaining resources of the computer room exceed the data volume. If so, assign the task to the computer room. If not, select the computer room to be assigned according to the priority of the task.

[0072] The step of selecting a computer room to be assigned according to the priority of the task includes:

[0073] If the task is of high priority, the task is assigned to the computer room where the data is located and is processed first;

[0074] Otherwise, the task is assigned to other computer rooms. When the remaining resources of the computer room where the data is located reach the data volume, the task is assigned to the computer room where the data is located.

[0075] Specifically, when the data in a task are all in the same computer room, if the resources of the computer room are sufficient, the task is directly assigned to this computer room.

[0076] If the resources in this computer room are insufficient, special processing will be performed based on the priority of the task. If it is a high priority, it will be assigned to this computer room and the task will be queued.

[0077] If it is a low priority, there are two options: one is to assign it to this computer room and wait for subsequent processing; the other is to assign it to another computer room and wait for processing when the IO resources of this computer room are sufficient.

[0078] S3. If the data required for the task are not located in the same computer room, determine whether the remaining resources of the computer room with the largest data volume exceed the data volume. If so, allocate the task to the computer room with the largest data volume. If not, split the task and allocate it to multiple computer rooms.

[0079] The tasks are split and distributed to multiple computer rooms, including:

[0080] Split the task into subtasks corresponding to the computer room where the data required for the task is located;

[0081] The subtask is pushed to the corresponding computer room, and when the remaining resources of the computer room exceed the data volume of the subtask, the subtask is processed.

[0082] Specifically, when the data in the task belongs to different computer rooms, a judgment needs to be made based on the amount of data stored in each computer room that will be used by this task, and the task is assigned to the computer room with the largest amount of data. If the computer room has sufficient resources, the task is executed directly.

[0083] If resources are insufficient, this task will also be assigned to this computer room, but the task will be split into sub-query statements and associated statements outside this computer room into sub-tasks. The sub-tasks will be pushed to the corresponding computer room for preliminary data statistics and returned to this computer room. Tasks will be allocated based on the remaining resources of each computer room and the amount of data for the sub-tasks.

[0084] Therefore, in this embodiment, tasks are assigned based on the estimated data volume of the task, the data volume, and the remaining resources in the computer room. When a task is assigned to a computer room cluster with a large data volume and sufficient resources, each subtask no longer needs to pull data from other computer rooms for calculation. Even if data is pulled, it is only a small amount of data. The purpose is that each task only needs to obtain data from the computer room cluster where the task is located, which can effectively reduce the amount of data pulled by each task and improve the effective utilization of resources.

[0085] Example 2

[0086] Please refer to Figure 2 A task scheduling terminal 1 includes a memory 2, a processor 3, and a computer program stored in the memory 2 and executable on the processor 3. When the processor 3 executes the computer program, each step of a task scheduling method of embodiment 1 is implemented.

[0087] In summary, the present invention provides a task scheduling method and terminal. After receiving a task, the task data volume in each computer room is first counted, and the task scheduling is performed based on the computer room where the data is located. Specifically, when the task data are all in the same computer room, the task is directly assigned to the computer room if the computer room resources are sufficient. If the resources are insufficient, the corresponding computer room needs to be selected based on the priority of the task. When the task data are not in the same computer room, the task is scheduled based on the resource situation of the computer room with the largest amount of data required for the task. Therefore, by allocating tasks based on the amount of data required for the task and the location of the data, the IO traffic between computer rooms can be reduced, and the rationality of task scheduling and the efficiency of task processing can be improved.

[0088] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's description and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A task scheduling method, characterized in that: Including steps: Receive tasks and count the amount of data for the tasks in each computer room; If the data required for the task is located in the same computer room, determine whether the remaining resources of the computer room exceed the data volume; if so, assign the task to the computer room; if not, select a computer room to be assigned according to the priority of the task; If the data required for the task are not located in the same computer room, determine whether the remaining resources of the computer room with the largest data volume exceed the data volume. If so, assign the task to the computer room with the largest data volume. If not, split the task and assign it to multiple computer rooms. If the number of tasks is not exceeded, selecting a computer room to be assigned according to the priority of the task includes: If the task is of high priority, the task is assigned to the computer room where the data is located and is processed first; Otherwise, the task is assigned to other computer rooms. When the remaining resources of the computer room where the data is located reach the data volume, the task is assigned to the computer room where the data is located.

2. A task scheduling method according to claim 1, characterized in that: The receiving of tasks and counting the amount of data of the tasks in each computer room includes: Receive a task and obtain the task information carried by the task, the task information including the name of the database table where the required data is located and its query conditions; The amount of data of the task in each database table is counted according to the database table name and its query condition, and the amount of data of the task in each computer room is counted according to the positional relationship between the database table and the computer room.

3. A task scheduling method according to claim 2, characterized in that: After receiving the task and counting the data volume of the task in each computer room, the following steps are included: Determine the computer room corresponding to the data required for the task and monitor the resources of the computer room.

4. A task scheduling method according to claim 1, characterized in that: If the number of tasks is not exceeded, the tasks are split and distributed to multiple computer rooms, including: Split the task into subtasks corresponding to the computer room where the data required for the task is located; The subtask is pushed to the corresponding computer room, and when the remaining resources of the computer room exceed the data volume of the subtask, the subtask is processed.

5. A task scheduling terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the following steps are implemented: Receive tasks and count the amount of data for the tasks in each computer room; If the data required for the task is located in the same computer room, determine whether the remaining resources of the computer room exceed the data volume; if so, assign the task to the computer room; if not, select a computer room to be assigned according to the priority of the task; If the data required for the task are not located in the same computer room, determine whether the remaining resources of the computer room with the largest data volume exceed the data volume. If so, assign the task to the computer room with the largest data volume. If not, split the task and assign it to multiple computer rooms. If the number of tasks is not exceeded, selecting a computer room to be assigned according to the priority of the task includes: If the task is of high priority, the task is assigned to the computer room where the data is located and is processed first; Otherwise, the task is assigned to other computer rooms. When the remaining resources of the computer room where the data is located reach the data volume, the task is assigned to the computer room where the data is located.

6. A task scheduling terminal according to claim 5, characterized in that: The receiving of tasks and counting the amount of data of the tasks in each computer room includes: Receive a task and obtain the task information carried by the task, the task information including the name of the database table where the required data is located and its query conditions; The amount of data of the task in each database table is counted according to the database table name and its query condition, and the amount of data of the task in each computer room is counted according to the positional relationship between the database table and the computer room.

7. A task scheduling terminal according to claim 6, characterized in that: After receiving the task and counting the data volume of the task in each computer room, the following steps are included: Determine the computer room corresponding to the data required for the task and monitor the resources of the computer room.

8. The task scheduling terminal according to claim 5, characterized in that: If the number of tasks is not exceeded, the tasks are split and distributed to multiple computer rooms, including: Split the task into subtasks corresponding to the computer room where the data required for the task is located; The subtask is pushed to the corresponding computer room, and when the remaining resources of the computer room exceed the data volume of the subtask, the subtask is processed.

Images