Node task scheduling method and device, equipment and storage medium
By adjusting the task priority and execution order according to the node resource status in a distributed system, the problems of slow and low-efficiency tasks caused by insufficient resources in traditional methods are solved, and the rational utilization of resources and the improvement of task execution efficiency are achieved.
Patent Information
- Application Number
- CN202411115523.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-12-10
AI Technical Summary
Traditional node task scheduling methods cannot adjust the task execution order when data processing resources are insufficient, resulting in slow task execution, reduced efficiency, and unreasonable allocation and utilization of resources.
By receiving the pending tasks assigned by the task scheduling module in the distributed system, sorting them according to priority, and determining whether to perform tasks based on the remaining data processing resources of the target node. If it is insufficient, lower the priority and reorder them to ensure reasonable use of resources.
Execute tasks according to priority when resources are sufficient, adjust priorities when resources are insufficient, improve task execution fluency and resource utilization, and improve nodes' efficiency in processing tasks.
Smart Images
Figure CN119105850A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a node task scheduling method, device, equipment and storage medium. Background Art
[0002] In a distributed system, task scheduling within a node determines the allocation of data processing resources in the node and the order of task execution. Traditional node task scheduling methods are often based on fixed task priorities or polling methods. When the data processing resources of a node are insufficient, the execution order of tasks cannot be adjusted, resulting in slow or even stagnant task execution, reduced efficiency of node task processing, and inability to reasonably allocate and utilize data processing resources in the node, resulting in a waste of node data processing resources. Summary of the invention
[0003] In order to solve the above technical problems, the present disclosure provides a node task scheduling method, device, equipment and storage medium.
[0004] A first aspect of the present disclosure provides a node task scheduling method, which is applicable to any target node in a distributed system, and the method includes:
[0005] Receiving multiple tasks to be processed assigned by a task scheduling module in a distributed system;
[0006] Sorting the multiple pending tasks in descending order of priority of each pending task to obtain a task sequence, and executing the multiple pending tasks in the task sequence in sequence starting from the first pending task in the task sequence;
[0007] For the current pending task in the task sequence, determine whether the remaining data processing resources of the target node meet the data processing resources required by the current pending task;
[0008] If so, execute the current pending task;
[0009] Otherwise, the priority of the current task to be processed is downgraded to obtain the target priority of the current task to be processed, and the target priority is smaller than the priority of at least one task to be processed other than the current task to be processed in the task sequence, and the steps of sorting the multiple tasks to be processed and the subsequent steps are re-executed in descending order of priority of each task to be processed.
[0010] A second aspect of the present disclosure provides a node task scheduling device, which is applicable to any target node in a distributed system, and includes:
[0011] A receiving module, used for receiving a plurality of tasks to be processed assigned by a task scheduling module in a distributed system;
[0012] The first sorting module is used to sort the multiple tasks to be processed in descending order of priority of each task to be processed to obtain a task sequence, and execute the multiple tasks to be processed in the task sequence in sequence starting from the first task to be processed in the task sequence;
[0013] A judgment module is used to judge whether the remaining data processing resources of the target node meet the data processing resources required by the current task to be processed in the task sequence;
[0014] A first execution module, configured to execute the current task to be processed if yes;
[0015] The second execution module is used to downgrade the priority of the current task to be processed, obtain the target priority of the current task to be processed, and re-execute the steps of sorting multiple tasks to be processed in descending order of priority of each task to be processed and the subsequent steps.
[0016] A third aspect of the present disclosure provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the node task scheduling method of the first aspect can be implemented.
[0017] A fourth aspect of the present disclosure provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the node task scheduling method of the first aspect can be implemented.
[0018] Compared with the prior art, the technical solution provided by the present invention has the following advantages:
[0019] The present invention receives a plurality of pending tasks assigned by a task scheduling module in a distributed system through any target node in the distributed system; sorts the plurality of pending tasks in descending order of priority of each pending task to obtain a task sequence, and executes the plurality of pending tasks in the task sequence in sequence starting from the first pending task in the task sequence; for a current pending task in the task sequence, determines whether the remaining data processing resources of the target node meet the data processing resources required by the current pending task; if so, executes the current pending task; otherwise, downgrades the priority of the current pending task to obtain a target priority of the current pending task, the target priority being less than the priority of at least one pending task other than the current pending task in the task sequence, and re-executes the steps of sorting the plurality of pending tasks in descending order of priority of each of the pending tasks and the steps thereafter. The present invention can execute tasks according to the priority of the tasks when the data processing resources of the node are sufficient, and timely adjust the priority of the tasks to be executed by the node when the data processing resources of the node are insufficient, so that the data processing resources of the node are reasonably utilized, the fluency of the node in executing tasks is improved, the order of tasks executed by the node is dynamically adjusted according to the resource status of the node, the data processing resources of the node are reasonably allocated and utilized, the utilization rate of the data processing resources of the node is improved, and the efficiency of the node in processing tasks is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0022] Figure 1 is a flowchart of a node task scheduling method provided by an embodiment of the present disclosure;
[0023] Figure 2 is a flowchart of a node task scheduling method provided by an embodiment of the present disclosure;
[0024] Figure 3 It is a structural diagram of a node task scheduling device provided by an embodiment of the present disclosure;
[0025] Figure 4 It is a structural diagram of a computer device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0026] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.
[0027] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0028] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.
[0029] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0030] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0031] The node task scheduling method provided by the embodiment of the present disclosure is applicable to any target node in a distributed system. The target node can be understood as a computer device, which can be understood as any device with processing and computing capabilities, including but not limited to electronic devices such as servers and computers.
[0032] In order to better understand the inventive concept of the embodiments of the present disclosure, the technical solutions of the embodiments of the present disclosure are described below in conjunction with exemplary embodiments.
[0033] Figure 11 is a flowchart of a node task scheduling method provided by an embodiment of the present disclosure. The method is applicable to any target node in a distributed system. The target node can be understood as a computer device. Figure 1 As shown, the node task scheduling method provided in this embodiment includes the following steps:
[0034] Step 110: Receive a plurality of tasks to be processed assigned by a task scheduling module in a distributed system.
[0035] In the disclosed embodiment, the distributed system includes a task scheduling module, which can use a load balancing algorithm to assign pending tasks to multiple nodes for processing. For example, tasks can be assigned to nodes with lower loads to improve the overall performance of the system.
[0036] For any target node in the distributed system, the executor in the target node can receive a plurality of tasks to be processed assigned by the task scheduling module in the distributed system.
[0037] Step 120 , sorting the multiple tasks to be processed in descending order of priority of each task to be processed to obtain a task sequence, and executing the multiple tasks to be processed in the task sequence in sequence starting from the first task to be processed in the task sequence.
[0038] In the disclosed embodiment, after receiving multiple pending tasks, the executor in the target node can obtain the priority of the pending tasks from the attribute data of each pending task, and then sort the multiple pending tasks in descending order of priority of each pending task to obtain a task sequence, and start from the first pending task in the task sequence and execute the multiple pending tasks in the task sequence in sequence.
[0039] The priority of the task to be processed can be pre-set by the user as needed.
[0040] Step 130: for the current task to be processed in the task sequence, determine whether the remaining data processing resources of the target node meet the data processing resources required by the current task to be processed.
[0041] In the disclosed embodiment, for the current pending task in the task sequence, the executor in the target node may determine whether the remaining data processing resources of the target node meet the data processing resources required by the current pending task.
[0042] The remaining data processing resources of the target node can be understood as unused computing resources in the target node. The remaining data processing resources of the target node can include at least one of the remaining computing resources of the central processing unit (CPU) of the target node, the remaining memory space of the target node, the network bandwidth of the target node, and the remaining disk space of the target node.
[0043] The data processing resources required for the current task to be processed can be understood as the minimum computing resources required to execute the current task to be processed.
[0044] Step 140: If the remaining data processing resources of the target node meet the data processing resources required by the current task to be processed, the current task to be processed is executed.
[0045] In the embodiment of the present disclosure, if the remaining data processing resources of the target node meet the data processing resources required by the current task to be processed, the executor in the target node can execute the current task to be processed.
[0046] Step 150: If the remaining data processing resources of the target node do not meet the data processing resources required by the current task to be processed, the priority of the current task to be processed is downgraded to obtain the target priority of the current task to be processed. The target priority is smaller than the priority of at least one task to be processed other than the current task to be processed in the task sequence. The steps of sorting the multiple tasks to be processed and the subsequent steps are re-executed in descending order of priority of each task to be processed.
[0047] In the disclosed embodiment, if the remaining data processing resources of the target node do not meet the data processing resources required by the current pending task, the executor in the target node may downgrade the priority of the current pending task to obtain the target priority of the current pending task, which is lower than the priority of at least one pending task other than the current pending task in the task sequence, and then re-execute the above steps 120 to 150. Specifically, the executor in the target node may sort the multiple remaining pending tasks in descending order of priority of each remaining pending task to obtain an updated task sequence, and re-execute step 120, starting from the first pending task in the task sequence, and sequentially execute the multiple pending tasks in the task sequence and the steps thereafter.
[0048] In some embodiments, the executor in the target node may replace the priority of the current task to be processed with the priority of the next task to be processed of the current task to be processed in the task sequence to obtain the target priority of the current task to be processed.
[0049] For example, task sequence A includes five pending tasks, namely A, B, C, D, and E. The priority of task A is 5, the priority of task B is 4, the priority of task C is 3, the priority of task D is 2, and the priority of task E is 1. The priority order is ABCDE. If the target node has completed execution of A and B, C is the current pending task, and the next pending task of C is D. If the remaining data processing resources of the target node do not meet the data processing resources required by the current pending task C, the target node can replace the priority of C with the priority of D, and the target priority of C is 2, and the target priority of D is 3, that is, the priority of C is reduced from 3 to 2, and the priority of D is increased from 2 to 3. At this time, the priority order of the three remaining pending tasks C, D, and E is DCE, and the composed task sequence B is D, C, and E. The target node can use D as the current pending task and execute the above-mentioned steps of judging whether the remaining data processing resources of the target node meet the data processing resources required by the current pending task and the subsequent steps.
[0050] In other embodiments, the executor in the target node may obtain the priority of the last pending task in the task sequence; and adjust the priority of the current pending task to the target priority, where the target priority is less than or equal to the priority of the last pending task.
[0051] For example, task sequence A includes five pending tasks, A, B, C, D, and E. The priority of pending task A is 5, the priority of pending task B is 4, the priority of pending task C is 3, the priority of pending task D is 2, and the priority of pending task E is 1. The priority order is ABCDE. If the target node has completed A and B, C is the current pending task, and the tail pending task in task sequence A is E. If the remaining data processing resources of the target node do not meet the data processing resources required by the current pending task C, the target node can obtain the priority of E, and then adjust the priority of C to the priority of E to obtain the target priority of C, that is, the target priority of C is 1.
[0052] An embodiment of the present disclosure receives a plurality of pending tasks assigned by a task scheduling module in a distributed system through any target node in the distributed system; sorts the plurality of pending tasks in descending order of priority of each pending task to obtain a task sequence, and executes the plurality of pending tasks in the task sequence in sequence starting from the first pending task in the task sequence; for a current pending task in the task sequence, determines whether the remaining data processing resources of the target node meet the data processing resources required by the current pending task; if so, executes the current pending task; otherwise, downgrades the priority of the current pending task to obtain a target priority of the current pending task, the target priority being less than the priority of at least one pending task other than the current pending task in the task sequence, and re-executes the steps of sorting the plurality of pending tasks in descending order of priority of each pending task and the steps thereafter. The present invention can execute tasks according to the priority of the tasks when the data processing resources of the node are sufficient, and timely adjust the priority of the tasks to be executed by the node when the data processing resources of the node are insufficient, so that the data processing resources of the node are reasonably utilized, the fluency of the node in executing tasks is improved, the order of tasks executed by the node is dynamically adjusted according to the resource status of the node, the data processing resources of the node are reasonably allocated and utilized, the utilization rate of the data processing resources of the node is improved, and the efficiency of the node in processing tasks is improved.
[0053] In some embodiments of the present disclosure, the target node may also execute during the process of performing the above steps: Figure 2 A flowchart of a node task scheduling method is provided, such as Figure 2 As shown, the node task scheduling method provided in this embodiment includes the following steps:
[0054] Step 210: Monitor whether the remaining data processing resources of the target node are less than or equal to a preset processing resource threshold.
[0055] In the disclosed embodiment, when the target node is executing the tasks to be processed, the data processing resources of the target node may be increasingly occupied by the tasks to be processed. Therefore, the executor in the target node can monitor whether the remaining data processing resources of the target node are less than or equal to a preset processing resource threshold.
[0056] The preset processing resource threshold can be understood as the minimum data processing resource to ensure that the target node can run stably, and can be set as needed, without limitation here.
[0057] Step 220: When the remaining data processing resources are less than or equal to a preset processing resource threshold, suspend the execution of at least one ongoing task to be processed, and release the data processing resources occupied by the at least one task to be processed.
[0058] In an embodiment of the present disclosure, when the remaining data processing resources of the target node are less than or equal to a preset processing resource threshold, it means that the remaining data processing resources of the target node can no longer ensure that the target node can operate stably. At this time, the target node can suspend the execution of at least one ongoing task to be processed and release the data processing resources of the target node occupied by at least one task to be processed.
[0059] In some embodiments, the executor in the target node can obtain the resource size of the target node occupied by each pending task in at least one pending task being performed by the target node; then determine the highest resource size among the resource sizes; suspend execution of the target pending task corresponding to the highest resource size, and release the data processing resources occupied by the target pending task.
[0060] Therefore, tasks with high resource usage can be suspended first to ensure that the target node can run stably.
[0061] Step 230: When the remaining data processing resources of the target node are greater than the preset processing resource threshold, continue to execute the current task to be processed in the task sequence to determine whether the remaining data processing resources of the target node meet the data processing resources required by the current task to be processed and the subsequent steps.
[0062] When the remaining data processing resources of the target node are greater than the preset processing resource threshold, it means that the remaining data processing resources of the target node can ensure that the target node can operate stably. At this time, the target node can continue to execute the current pending task in the task sequence to determine whether the remaining data processing resources of the target node meet the data processing resources required for the current pending task and its subsequent steps.
[0063] In this way, the data processing resources of the target node can be released in time to ensure that the target node can operate stably.
[0064] In some embodiments of the present disclosure, after the target node starts from the first pending task in the task sequence and executes multiple pending tasks in the task sequence in sequence, it can obtain the execution progress of the multiple pending tasks and the remaining data processing resources of the target node; then send the execution progress and the remaining data processing resources to the task scheduling module, so that the task scheduling module determines the target speed of allocating the pending tasks to the target node based on the execution progress of the multiple pending tasks and determines the target number of pending tasks to be allocated to the target node based on the remaining data processing resources.
[0065] Among them, the target speed is positively correlated with the execution progress, and the target quantity is positively correlated with the remaining data processing resources. Specifically, the faster the execution progress of the target node for multiple pending tasks, the faster the target speed of the task scheduling module to allocate pending tasks to the target node, the more remaining data processing resources the target node has, and the greater the target quantity of pending tasks the task scheduling module allocates to the target node.
[0066] In some embodiments of the present disclosure, after the target node sorts a plurality of pending tasks in descending order of priority of each pending task to obtain a task sequence, it can determine the priority of at least one pending task in the task sequence in response to a user's priority adjustment operation on at least one pending task in the task sequence; and then re-sort the plurality of pending tasks in descending order of priority of each pending task to obtain an updated task sequence.
[0067] Therefore, the priority of each task to be processed can be adjusted in time according to the actual needs of the user, so that the priority of each task to be processed meets the actual needs, which can improve the user experience.
[0068] Figure 3 is a schematic diagram of the structure of a node task scheduling device provided by an embodiment of the present disclosure, and the device can be understood as the above-mentioned target node or some functional modules in the above-mentioned target node. Figure 3 As shown, the node task scheduling device 300 is applicable to any target node in a distributed system, and the device includes:
[0069] A receiving module 310, configured to receive a plurality of tasks to be processed assigned by a task scheduling module in a distributed system;
[0070] The first sorting module 320 is used to sort the multiple tasks to be processed in descending order of priority of each task to be processed to obtain a task sequence, and execute the multiple tasks to be processed in the task sequence in sequence starting from the first task to be processed in the task sequence;
[0071] The judgment module 330 is used to judge whether the remaining data processing resources of the target node meet the data processing resources required by the current task to be processed in the task sequence;
[0072] A first execution module 340, configured to execute the current task to be processed if yes;
[0073] The second execution module 350 is used to downgrade the priority of the current task to be processed, obtain the target priority of the current task to be processed, and the target priority is smaller than the priority of at least one task to be processed other than the current task to be processed in the task sequence, and re-execute the steps of sorting the multiple tasks to be processed in descending order of priority of each task to be processed and the steps thereafter.
[0074] Optionally, the second execution module includes:
[0075] The replacement submodule is used to replace the priority of the current pending task with the priority of the next pending task of the current pending task in the task sequence to obtain the target priority of the current pending task.
[0076] Optionally, the second execution module includes:
[0077] The first acquisition submodule is used to acquire the priority of the tail task to be processed in the task sequence;
[0078] The adjustment submodule is used to adjust the priority of the current task to be processed to a target priority, where the target priority is less than or equal to the priority of the tail task to be processed.
[0079] Optionally, the node task scheduling device includes:
[0080] A monitoring module, used to monitor whether the remaining data processing resources of the target node are less than or equal to a preset processing resource threshold;
[0081] A pause module, used to pause the execution of at least one ongoing task to be processed and release the data processing resources occupied by at least one task to be processed when the remaining data processing resources are less than or equal to a preset processing resource threshold;
[0082] The third execution module is used to continue executing the current task to be processed in the task sequence when the remaining data processing resources of the target node are greater than a preset processing resource threshold, and to determine whether the remaining data processing resources of the target node meet the data processing resources required by the current task to be processed and the subsequent steps.
[0083] Optionally, the above-mentioned pause module includes:
[0084] The second acquisition submodule is used to acquire the resource size of the target node occupied by each pending task in at least one pending task being processed;
[0085] A determination submodule, used for determining the highest resource size among various resource sizes;
[0086] The pause submodule is used to pause the execution of the target pending task corresponding to the maximum resource size and release the data processing resources occupied by the target pending task.
[0087] Optionally, the node task scheduling device includes:
[0088] An acquisition module is used to obtain the execution progress of multiple pending tasks and the remaining data processing resources of the target node;
[0089] The sending module is used to send the execution progress and the remaining data processing resources to the task scheduling module, so that the task scheduling module determines the target speed of allocating the pending tasks to the target node based on the execution progress and determines the target number of the pending tasks to be allocated to the target node based on the remaining data processing resources, the target speed is positively correlated with the execution progress, and the target number is positively correlated with the remaining data processing resources.
[0090] Optionally, the node task scheduling device includes:
[0091] A determination module, configured to determine the priority of at least one task to be processed in the task sequence in response to a user's priority adjustment operation on at least one task to be processed in the task sequence;
[0092] The second sorting module is used to re-sort the multiple tasks to be processed in descending order of priority of each task to be processed to obtain an updated task sequence.
[0093] The device provided in the embodiments of the present disclosure can implement the method of any of the above embodiments, and its execution method and beneficial effects are similar, which will not be repeated here.
[0094] The embodiments of the present disclosure also provide a computer device, which can be understood as the above-mentioned target node. The computer device includes a processor and a memory, wherein a computer program is stored in the memory. When the computer program is executed by the processor, the method of any of the above-mentioned embodiments can be implemented. The execution method and beneficial effects are similar and will not be repeated here.
[0095] Figure 4 is a schematic diagram of the structure of a computer device provided by an embodiment of the present disclosure, such as Figure 4 As shown, the computer device 400 may include a processor 410 and a memory 420, wherein the memory 420 stores a computer program 421, and when the computer program 421 is executed by the processor 410, the method provided by any of the above embodiments can be implemented, and its execution method and beneficial effects are similar and will not be repeated here.
[0096] Of course, to simplify, Figure 4Only some of the components related to the present invention in the computer device 400 are shown, and components such as a bus, an input / output interface, an input device, and an output device are omitted. In addition, according to specific application conditions, the computer device 400 may also include any other appropriate components.
[0097] An embodiment of the present disclosure provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method of any of the above embodiments can be implemented. The execution method and beneficial effects are similar and will not be repeated here.
[0098] The above-mentioned computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0099] The computer program may be written in any combination of one or more programming languages to write program codes for performing the operations of the disclosed embodiments, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer device, partially on the user's device, as a separate software package, partially on the user's computer device and partially on a remote computer device, or entirely on a remote computer device or server.
[0100] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.
[0101] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.
[0102] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A node task scheduling method, characterized in that: Applicable to any target node in a distributed system, the method comprises: Receiving a plurality of tasks to be processed assigned by a task scheduling module in the distributed system; Sorting the plurality of tasks to be processed in descending order of priority of each of the tasks to be processed to obtain a task sequence, and executing the plurality of tasks to be processed in the task sequence in sequence starting from the first task to be processed in the task sequence; For the current task to be processed in the task sequence, determining whether the remaining data processing resources of the target node meet the data processing resources required by the current task to be processed; If yes, execute the current pending task; Otherwise, the priority of the current task to be processed is downgraded to obtain the target priority of the current task to be processed, and the target priority is smaller than the priority of at least one task to be processed in the task sequence other than the current task to be processed. The steps of sorting the multiple tasks to be processed in descending order of priority of each task to be processed and subsequent steps are re-executed.
2. The method according to claim 1, characterized in that The step of downgrading the priority of the current task to be processed to obtain the target priority of the current task to be processed includes: The priority of the current task to be processed is replaced with the priority of the next task to be processed of the current task to be processed in the task sequence to obtain the target priority of the current task to be processed.
3. The method according to claim 1, characterized in that The step of downgrading the priority of the current task to be processed to obtain the target priority of the current task to be processed includes: Obtaining the priority of the last task to be processed in the task sequence; The priority of the current task to be processed is adjusted to a target priority, where the target priority is less than or equal to the priority of the tail task to be processed.
4. The method according to claim 1, characterized in that: The method further comprises: Monitoring whether the remaining data processing resources of the target node are less than or equal to a preset processing resource threshold; When the remaining data processing resources are less than or equal to a preset processing resource threshold, suspending execution of at least one ongoing task to be processed, and releasing the data processing resources occupied by the at least one task to be processed; When the remaining data processing resources of the target node are greater than a preset processing resource threshold, continue to execute the current task to be processed in the task sequence, and determine whether the remaining data processing resources of the target node meet the data processing resources required by the current task to be processed and the subsequent steps.
5. The method according to claim 4, characterized in that The pausing execution of at least one ongoing task to be processed and releasing data processing resources occupied by the at least one task to be processed includes: Obtaining the resource size of the target node occupied by each of at least one ongoing task to be processed; determining a highest resource size among the resource sizes; The execution of the target task to be processed corresponding to the maximum resource size is suspended, and the data processing resources occupied by the target task to be processed are released.
6. The method according to claim 1, characterized in that After starting from the first task to be processed in the task sequence and sequentially executing multiple tasks to be processed in the task sequence, the method further includes: Obtaining the execution progress of the plurality of tasks to be processed and the remaining data processing resources of the target node; The execution progress and the remaining data processing resources are sent to the task scheduling module, so that the task scheduling module determines a target speed for allocating pending tasks to the target node based on the execution progress and determines a target number of pending tasks to be allocated to the target node based on the remaining data processing resources, wherein the target speed is positively correlated with the execution progress, and the target number is positively correlated with the remaining data processing resources.
7. The method according to claim 1, characterized in that After the plurality of tasks to be processed are sorted in descending order of priority of each task to be processed to obtain a task sequence, the method further includes: In response to a user's priority adjustment operation on at least one task to be processed in the task sequence, determining the priority of at least one task to be processed in the task sequence; The plurality of tasks to be processed are re-sorted in descending order of priority of each of the tasks to be processed to obtain an updated task sequence.
8. A node task scheduling device, characterized in that: Applicable to any target node in a distributed system, the device comprises: A receiving module, used for receiving a plurality of tasks to be processed assigned by a task scheduling module in the distributed system; A first sorting module is used to sort the plurality of tasks to be processed in descending order of priority of each of the tasks to be processed to obtain a task sequence, and to execute the plurality of tasks to be processed in the task sequence in sequence starting from the first task to be processed in the task sequence; A judgment module, used for judging, with respect to a current task to be processed in the task sequence, whether the remaining data processing resources of the target node meet the data processing resources required by the current task to be processed; A first execution module, configured to execute the current task to be processed if yes; The second execution module is used to downgrade the priority of the current task to be processed, obtain the target priority of the current task to be processed, and the target priority is smaller than the priority of at least one task to be processed other than the current task to be processed in the task sequence, and re-execute the steps of sorting the multiple tasks to be processed in descending order of priority of each task to be processed and the subsequent steps.
9. A computer device, characterized in that: include: A memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the node task scheduling method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the node task scheduling method according to any one of claims 1 to 7 is implemented.