Task scheduling method, system, equipment, medium and product

Abstract

The invention discloses a task scheduling method, system and device, a medium and a product, and the method comprises the steps: screening a management node from available computing nodes in response to a project task set, so as to enable the management node to cooperate with the remaining available computing nodes to form a computing resource group; receiving a group cooperation efficiency improvement coefficient of the computing resource group uploaded by the management node, and obtaining the efficiency workload of the computing resource group according to the group cooperation efficiency improvement coefficient; according to the efficiency workload, assembling the project task set into batch task groups, and issuing the batch task groups to the management node, so that the management node assigns and executes the batch task groups; and in response to an assignment completion notification of the management node, updating the available computing nodes. By adopting the embodiment of the invention, the project task can be autonomously dispatched and executed in combination with the collaboration capability of the computing node, and the task processing efficiency is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of task scheduling, in particular to a task scheduling method, system, device, medium and product. BACKGROUND

[0002] The current mainstream task scheduling scheme is that the task scheduling center dispatches tasks to the computing nodes, the computing nodes autonomously execute after receiving the task instructions, and the execution completion information is fed back to the task scheduling center.

[0003] However, the task scheduling center currently directly dispatches tasks to the computing nodes based on the online state of the computing nodes, and the computing nodes can only passively accept the execution tasks and cannot actively participate in the task dispatching according to the state of the computing nodes. SUMMARY

[0004] Embodiments of the present application aim to provide a task scheduling method, system, device, medium and product, which can combine the collaboration capabilities of computing nodes to autonomously assign and execute project tasks, and effectively improve the task processing efficiency.

[0005] In a first aspect, the embodiments of the present application provide a task scheduling method, comprising: In response to a project task set, a management node is selected from available computing nodes, so that the management node forms a computing resource group with the remaining available computing nodes; A group collaboration efficiency improvement coefficient of the computing resource group uploaded by the management node is received, and an efficiency workload of the computing resource group is obtained according to the group collaboration efficiency improvement coefficient; According to the efficiency workload, the project task set is assembled into a batch task group, and the batch task group is issued to the management node, so that the management node dispatches and executes the batch task group; In response to the dispatch completion notification of the management node, the available computing nodes are updated.

[0006] As an improvement of the above-mentioned scheme, before the task scheduling method responds to the project task set and selects the management node from the available computing nodes, so that the management node forms the computing resource group with the remaining available computing nodes, the task scheduling method further comprises: In response to an initial task, the initial task is divided into at least one subtask according to the associated project and content of the initial task; the subtask includes project functions and service call relationships; All task dependency relationships of the subtasks are obtained by traversing the subtasks, and a project task dependency relationship graph is constructed; The initial task is grouped according to the connectivity of the project task dependency relationship graph, and a project task set is obtained.

[0007] As an improvement of the above scheme, the management node is screened from the available computing nodes in response to the project task set, so that the management node forms a computing resource group with the remaining available computing nodes, comprising: In response to the project task set, the number ratio of the project tasks in the project task set to the available computing nodes is calculated; According to the number ratio, a detection strategy is issued to the available computing nodes; the detection strategy includes a participation state detection strategy and a computing capacity detection strategy; The detection results uploaded by the available computing nodes based on the detection strategy are received; According to the detection results and the number ratio, a management node is screened from the available computing nodes, so that the management node forms a computing resource group with the remaining available computing nodes.

[0008] As an improvement of the above scheme, the detection strategy is issued to the available computing nodes according to the number ratio, comprising: If the number ratio meets a preset first number ratio threshold, a first detection strategy is issued to the available computing nodes; the first detection strategy sets the participation state to participation, and obtains the computing capacity according to the hardware configuration and running state of the available computing nodes; If the number ratio meets a preset second number ratio threshold, a second detection strategy is issued to the available computing nodes; the second detection strategy obtains the participation state according to the usage rate of the available computing nodes and a preset minimum available threshold coefficient, and obtains the computing capacity according to the hardware configuration and running state of the available computing nodes; If the number ratio meets a preset third number ratio threshold, a third detection strategy is issued to the available computing nodes; the third detection strategy obtains the participation state according to the usage rate of the available computing nodes, and obtains the computing capacity according to the hardware configuration and running state of the available computing nodes.

[0009] As an improvement of the above scheme, the management node is screened from the available computing nodes in response to the project task set, so that the management node forms a computing resource group with the remaining available computing nodes, comprising: If the number ratio meets a preset first number ratio threshold, the available computing node with the smallest computing capacity is screened as the management node according to the detection results; If the number ratio meets a preset second number ratio threshold, the available computing node with the participation state of participation and the computing capacity of medium is screened as the management node according to the detection results; If the quantity ratio meets a preset third quantity ratio threshold, a computing node with the largest computing capacity and a participation state of participation is selected as the management node according to the detection result. Information of the management node is sent to the available computing nodes, so that the management node forms a computing resource group with the remaining available computing nodes.

[0010] As an improvement of the above scheme, the computing resource group includes the management node and a common computing node. The common computing node is selected by the management node from the remaining available computing nodes according to a computing capacity and a network performance of the management node and the remaining available computing nodes when the management node cooperates with the remaining available computing nodes, and a group cooperation efficiency improvement coefficient of the management node and the common computing node is greater than a preset efficiency threshold.

[0011] As an improvement of the above scheme, the group cooperation efficiency improvement coefficient of the computing resource group uploaded by the management node is received, and an efficiency workload of the computing resource group is obtained according to the group cooperation efficiency improvement coefficient, including: The group cooperation efficiency improvement coefficient of the computing resource group uploaded by the management node is received. The computing capacity of each available computing node in the computing resource group is obtained. The efficiency workload of all the available computing nodes in the computing resource group and the management node after cooperation is obtained according to the group cooperation efficiency improvement coefficient and the computing capacity.

[0012] As an improvement of the above scheme, the project task set is assembled into a batch task group according to the efficiency workload, and the batch task group is sent to the management node, so that the management node dispatches and executes the batch task group, including: The task workload of each project task in the project task set is obtained. If the task workload of the current project task is less than the efficiency workload, the project task set is traversed, and the current project task is updated. If the task workload of the current project task is greater than or equal to the efficiency workload, the current project task is assembled into a batch task group, and the batch task group is sent to the management node, so that the management node dispatches and executes the batch task group. The unexecuted project task in the project task set is updated according to the execution of the project task.

[0013] As an improvement of the above scheme, the management node segments the batch task group, and assigns the segmented execution units to available computing nodes in the computing resource group according to the execution capability of the segmented execution units and the available computing nodes in the computing resource group.

[0014] As an improvement of the above scheme, the available computing nodes are updated in response to the assignment completion notification of the management node, including: In response to the assignment completion notification of the management node, a polling instruction is issued to the management node at a preset interval; The task progress information, load information and idle node list information uploaded by the management node based on the polling instruction are received, and it is judged whether the computing resource group meets the release condition; If the release condition is met, the to-be-released node is obtained, and it is checked whether the supplement condition is met in other computing resource groups; When the supplement condition is met, the to-be-released node is supplemented to other computing resource groups; If the supplement condition is not met, the to-be-released node is directly released.

[0015] As an improvement of the above scheme, the task progress information, load information and idle node list information uploaded by the management node based on the polling instruction are received, and it is judged whether the computing resource group meets the release condition, including: The first task progress information, first load information and first idle node list information uploaded by the management node based on the polling instruction are received; A preset maximum progress load idle node release coefficient is obtained, and the first task progress information and first load information are combined to obtain the first stress load degree of the computing resource group; If the first stress load degree is less than a preset stress load degree threshold, and the first idle node list information is not empty, it is considered that the available computing nodes in the computing resource group meet the release condition, otherwise it is considered that the release condition is not met.

[0016] As an improvement of the above scheme, if the release condition is met, the to-be-released node is obtained, and it is checked whether the supplement condition is met in other computing resource groups, including: If the release condition is met, the idle nodes in the computing resource group are selected as the to-be-released nodes; The second stress load degree, second idle node list information and execution time of other computing resource groups are obtained; If the second pressure load degree is less than a preset pressure load degree threshold, the second idle node list information is empty, and the execution time is not less than a preset execution time threshold, it is considered that the computing resource group satisfies the supplement condition, otherwise it is considered that the supplement condition is not satisfied.

[0017] In a second aspect, an embodiment of the present application provides a task scheduling system, comprising: A computing node grouping module is configured to filter a management node from available computing nodes in response to a project task set, so that the management node forms a computing resource group with the remaining available computing nodes; An efficiency workload calculation module is configured to receive a grouping cooperation efficiency promotion coefficient of the computing resource group uploaded by the management node, and obtain an efficiency workload of the computing resource group according to the grouping cooperation efficiency promotion coefficient; A task issuing module is configured to assemble the project task set into a batch task group according to the efficiency workload, and issue the batch task group to the management node, so that the management node dispatches and executes the batch task group; A node updating module is configured to update the available computing nodes in response to a dispatch completion notification of the management node.

[0018] In a third aspect, an embodiment of the present application provides a task scheduling device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the task scheduling method as described above.

[0019] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, comprising a stored computer program, wherein the computer program controls a device where the computer readable storage medium is located to execute the task scheduling method as described above when the computer program runs.

[0020] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising a computer program or computer instructions, wherein the computer program or the computer instructions are executed by a processor to execute the task scheduling method as described above.

[0021] Compared with the prior art, the task scheduling method, system, device, medium and product provided by the present application can filter a management node from available computing nodes in response to a project task set, so that the management node forms a computing resource group with the remaining available computing nodes; receive a group cooperation efficiency improvement coefficient of the computing resource group uploaded by the management node, and obtain an efficiency workload of the computing resource group according to the group cooperation efficiency improvement coefficient; assemble the project task set into a batch task group according to the efficiency workload, and distribute the batch task group to the management node, so that the management node dispatches and executes the batch task group; and update the available computing nodes in response to a dispatch completion notification of the management node. By using the embodiment of the present application, the autonomous dispatching and execution of project tasks can be combined with the cooperation capability of computing nodes, and the task processing efficiency is effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 is a step flow diagram of a task scheduling method provided by an embodiment of the present application; Figure 2 is a structural diagram of a task scheduling system provided by an embodiment of the present application; Figure 3 is a structural diagram of a task scheduling device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0024] In the description and claims of the specification, it is to be understood that the terms first, second, etc. are used only for the purpose of description and are not to be construed as indicating or implying relative importance or an indicated number of features. They are also not necessarily used to describe a sequence or an order, unless explicitly stated otherwise. The terms are interchangeable under appropriate circumstances. Thus, a feature described as "first" can be construed as comprising at least one of the feature.

[0025] In the distributed computing and large-scale task processing scenarios, in order to achieve efficient decomposition and execution of complex projects, the current mainstream technical solution generally adopts a centralized scheduling mode with a task scheduling center as the core.

[0026] In this mode, the task scheduling center directly allocates a single task to an online available computing node. The computing node, as an execution carrier, passively receives the scheduling instruction and independently completes the task, and feeds back the execution result to the scheduling center, and finally the scheduling center collects and analyzes to confirm the completion status of the total task.

[0027] However, as the complexity of computing tasks increases and the scale of computing power expands, due to the island state of computing nodes in receiving and executing tasks in the existing mode, the computing nodes cannot actively participate in task allocation decisions, resulting in frequent mismatch between supply and demand, which seriously restricts the utilization rate of cluster resources.

[0028] Based on the above considerations, the embodiments of the present application provide a task scheduling method applied to a task scheduling center. It should be noted that the task scheduling center can interact with any computing node in the computing node cluster, and the computing nodes in the computing node cluster can also interact with each other.

[0029] Please refer to Figure 1 In this embodiment, the task scheduling method is specifically executed by steps S1 to S4 in the task scheduling center: S1, in response to a project task set, a management node is selected from available computing nodes, so that the management node forms a computing resource group with the remaining available computing nodes.

[0030] It should be noted that the management node, as a representative of the computing resource group, directly interacts with the task scheduling center, and undertakes the function of uniting other computing nodes in the computing resource group to participate in task cooperation.

[0031] S2, receiving the group team cooperation efficiency improvement coefficient of the computing resource group uploaded by the management node, and obtaining the efficiency workload of the computing resource group according to the group team cooperation efficiency improvement coefficient.

[0032] It should be noted that the group team cooperation efficiency improvement coefficient reflects the efficiency gain proportion of each computing node in the computing resource group after cooperating with the management node, and the higher the coefficient value, the stronger the synergistic effect of node cooperation.

[0033] S3, according to the efficiency workload, the project task set is assembled into a batch task group, and the batch task group is issued to the management node, so that the management node dispatches and executes the batch task group.

[0034] In the embodiments of the present application, the batch task groups are assembled according to the efficiency workload of the computing resource groups, which can match the collaboration ability of the computing resource groups. In this process, the task scheduling center globally schedules the project tasks, and then the management node further disassembles and assigns the batch task groups to specific computing nodes for self-execution.

[0035] S4, updating the available computing nodes in response to the assignment completion notification of the management node.

[0036] In some preferred embodiments, the task scheduling center has screened and marked the management nodes in the previous task execution process or in the initialization process, and the management nodes manage the computing resource groups. At this time, the task scheduling center can directly select the existing management nodes when screening the management nodes, or can re-screen the management nodes according to business requirements.

[0037] In other preferred embodiments, the task scheduling center has not completed the initialization, and at this time the task scheduling center needs to screen the management nodes from all available computing nodes and receive the related information of the computing resource groups formed by the self-organization of the management nodes.

[0038] It should be noted that the available computing nodes and the management nodes in the computing node cluster are dynamic, which will be iteratively updated according to the task execution. Of course, the computing nodes in the computing resource groups managed by the management nodes are also dynamic.

[0039] In the above scheme, by screening the management nodes and combining the internal node states of the computing resource groups, an effective path for computing nodes to actively participate in scheduling decisions is built, breaking the limitation of computing nodes in traditional centralized scheduling that can only passively receive tasks. At the same time, by introducing the group cooperation efficiency improvement coefficient to calculate the efficiency workload, the task allocation logic is upgraded from the traditional dependence on the online state of the computing nodes to the deep combination of the actual collaboration ability of the computing resource groups. Further, the strategy of batch task grouping based on the efficiency workload can make the task allocation more suitable for the collaboration level of the resource groups, fully exert the computing power advantage of the computing resource group collaboration, and significantly improve the overall task processing efficiency.

[0040] As a preferred implementation, before step S1, the task scheduling method further includes steps S01-S03: S01, in response to an initial task, dividing the initial task into at least one subtask according to the associated project and content of the initial task; the subtask includes project functions and service call relationship; S02, traversing the subtasks to obtain the task dependency relationship of all the subtasks, and constructing a project task dependency relationship graph; S03, grouping the initial tasks according to the connectivity of the project task dependency graph, to obtain a project task set.

[0041] It should be noted that the initial task is triggered automatically by a management system or manually by a user, and the management system is an upstream application of the task scheduling center. The initial task is the source task of the task scheduling process, which can be a transformation task, a new function development, or data governance, etc. The initial task can cover multiple associated projects or even the whole system.

[0042] Exemplarily, the initial task is represented as , wherein, is the total number of associated projects. For any , there is , wherein, is the associated project, is the task content.

[0043] In step S01, when the task scheduling center receives the initial task, it starts sub-task disassembly and divides the initial task into . For any sub-task , there is , wherein, is the service call range list.

[0044] In some preferred embodiments, the service call range list is obtained by jointly querying the associated project and the task content in the service call association relationship platform. The jth service call record in the service call range list is represented as , wherein, is the service providing project, is the service consuming project, , is the function point, .

[0045] Preferably, when step S02 is performed, the task scheduling center traverses each sub-task, takes the project function as the node, takes each record in the service call unit list as the edge, links the nodes through the directed edge, and the direction of the edge is from the service providing project to the service consuming project, and finally constructs the project task dependency graph G.

[0046] Further, when step S03 is performed, according to the connectivity of the project task dependency graph, the project task dependency graph G is split into connected components, and the obtained connected subgraph is taken as the project task set .

[0047] In the above scheme, by responding to the initial task and disassembling it into sub-tasks containing project functions and service call relationships, a fine basis is laid for subsequent task dependency analysis; then, based on the project dependency relationship, a project task dependency graph is constructed, and the initial task is grouped according to the connectivity of the project task dependency graph, which can aggregate sub-tasks with strong dependencies into the same project task set, ensuring efficient collaboration of tasks within the computing resource group; at the same time, parallel scheduling across resource groups between different project task sets can be supported, improving the parallel execution degree of the overall task.

[0048] As a preferred embodiment, step S1, in response to the project task set, a management node is selected from the available computing nodes, so that the management node forms a computing resource group with the remaining available computing nodes, and steps S11-S14 are executed: S11, in response to the project task set, the number ratio of the project tasks in the project task set to the number of available computing nodes is calculated; S12, according to the number ratio, a detection strategy is issued to the available computing nodes; the detection strategy includes a participation state detection strategy and a computing power detection strategy; S13, receiving the detection results uploaded by the available computing nodes based on the detection strategy; S14, according to the detection results and the number ratio, a management node is selected from the available computing nodes, so that the management node forms a computing resource group with the remaining available computing nodes.

[0049] It should be noted that the number ratio of the project task and the available computing node reflects the creation pressure of the project task, and based on the number ratio, the detection strategy can be issued to realize the dynamic adaptation of the detection logic and the task pressure, which can avoid excessive detection under low pressure and ensure that the truly adaptive nodes can be selected under high pressure, so as to balance the efficiency and accuracy in the detection process.

[0050] Under different creation pressures, the core needs of the management node are also different, and the embodiments of the present application select the available computing nodes that meet the computing power demand corresponding to the creation pressure as the management node based on the detection results and the number ratio, which can maximize the resource utilization rate.

[0051] In the above scheme, the number ratio of the project task and the available computing node is taken as the core, the creation pressure scene of the project task is accurately perceived, and the differentiated detection strategy can be dynamically matched, and finally the management node is selected and the computing resource group is formed, which can flexibly and accurately match the task demand, and further improve the resource utilization rate and collaborative execution efficiency of the entire task scheduling system under different pressure scenes.

[0052] Further, preferably, step S12, according to the quantity ratio, issues a detection strategy to the available computing nodes, including: If the quantity ratio meets a preset first quantity ratio threshold, a first detection strategy is issued to the available computing nodes; the first detection strategy sets the participation state as participation, and obtains the computing capability according to the hardware configuration and running state of the available computing node; If the quantity ratio meets a preset second quantity ratio threshold, a second detection strategy is issued to the available computing nodes; the second detection strategy obtains the participation state according to the usage rate of the available computing node and a preset minimum available threshold coefficient, and obtains the computing capability according to the hardware configuration and running state of the available computing node; If the quantity ratio meets a preset third quantity ratio threshold, a third detection strategy is issued to the available computing nodes; the third detection strategy obtains the participation state according to the usage rate of the available computing node, and obtains the computing capability according to the hardware configuration and running state of the available computing node.

[0053] It should be noted that, in the embodiments of the present application, when the quantity ratio meets the preset first quantity ratio threshold, the creation pressure of the project task is a low pressure scenario; when the quantity ratio meets the preset second quantity ratio threshold, the creation pressure of the project task is a medium pressure scenario; and when the quantity ratio meets the preset third quantity ratio threshold, the creation pressure of the project task is a high pressure scenario.

[0054] Exemplarily, the number of project tasks in the project task set is , and the number of available computing nodes is When , it is considered that the preset first quantity ratio threshold is met; when , it is considered that the preset second quantity ratio threshold is met; and when , it is considered that the preset third quantity ratio threshold is met.

[0055] In some preferred embodiments, the computing capability of the available computing node is represented as: (1); wherein, is the number of central processor threads, is the usage rate of the kth central processor, is the frequency of the kth central processor, is the central processor process, is the memory capacity, is the memory remaining ratio, is the memory frequency, and ​For the hard disk reading speed, For the coordination task algorithm power consumption matching coefficient, For the coordination task memory consumption matching coefficient, For the coordination task disk consumption matching coefficient, For the total efficiency of the central processor one two three cache.

[0056] It should be noted that the coordination task algorithm power consumption matching coefficient is the proportion weight of the calculation node itself processor algorithm power consumption in the overall processor resource evaluation; the coordination task memory consumption matching coefficient is the proportion weight of the calculation node itself memory consumption in the overall processor resource evaluation.

[0057] In some preferred embodiments, the participation state of the available calculation node in the third detection strategy is represented as: (2); Wherein, is the usage rate of the kth central processor, is the maximum usage rate coefficient of the available calculation node, represents whether x is greater than or equal to y, and if greater than or equal to y, 1 is taken, otherwise 0 is taken.

[0058] On the basis of formula (2), the participation state of the available calculation node in the second detection strategy is represented as: (3); Wherein, is a preset minimum available threshold coefficient. In the second detection strategy, when the standard participation state of the available calculation node is greater than the minimum available threshold coefficient, the participation state is set to 1, indicating participation, otherwise it is set to 0, indicating non-participation.

[0059] In the embodiment of the application, in the low pressure scene, the participation state of the available calculation node is directly set to participation, which can focus on the computing power of the calculation node to quickly complete the detection, shorten the networking period and guarantee the efficiency of the group; in the medium pressure scene, the participation state is judged by the minimum available threshold coefficient, which can ensure the availability of the participating node, and at the same time provide sufficient quality reference for subsequent screening of management nodes; in the high pressure scene, the requirement for node effectiveness is the highest, and it is necessary to preferentially screen out nodes that can undertake tasks to avoid invalid nodes occupying collaborative resources, so that the participation state is only judged by the usage rate to ensure that the node load is low and the ability is reliable.

[0060] ​​Preferably, step S14 screens a management node from the available computing nodes according to the detection result and the number ratio, so that the management node forms a computing resource group with the remaining available computing nodes, including: If the number ratio meets a preset first number ratio threshold, the available computing node with the minimum computing capability is screened as the management node according to the detection result. If the number ratio meets a preset second number ratio threshold, the available computing node with the participation state of participation and the medium computing capability is screened as the management node according to the detection result. If the number ratio meets a preset third number ratio threshold, the available computing node with the participation state of participation and the maximum computing capability is screened as the management node according to the detection result. Information of the management node is sent to the available computing nodes, so that the management node forms a computing resource group with the remaining available computing nodes.

[0061] It can be understood that when the number ratio meets the preset first number ratio threshold, the task processing pressure is small, at this time, the core function of the management node is basic coordination, and it is unnecessary to occupy a high computing power node, and in the embodiment of the application, the available computing node with the minimum computing capability is selected; when the number ratio meets the preset second number ratio threshold, the task processing pressure is medium, the management node needs to coordinate the efficiency, and avoid that resources are idle due to excessive capability or coordination is stalled due to insufficient capability, and in the embodiment of the application, the available computing node with the participation state and the medium computing capability is selected; when the number ratio meets the preset third number ratio threshold, the task processing pressure is large, the management node needs to bear more complex decisions such as load balancing and conflict adjustment, and needs strong computing capability support, and in the embodiment of the application, the available computing node with the participation state and the maximum computing capability is selected.

[0062] As a preferred embodiment, the computing resource group includes a management node and a common computing node. The common computing node is selected by the management node from the remaining available computing nodes according to a computing capability and network performance of the management node and the remaining available computing nodes when they cooperate, and a group cooperation efficiency improvement coefficient; the group cooperation efficiency improvement coefficient of the management node and the common computing node is greater than a preset efficiency threshold.

[0063] It should be noted that the management node can detect information of other computing nodes in the computing node cluster where the management node is located. For the computing node, when it receives an instruction from the task scheduling center to designate it as a management node, it detects the computing capability and network performance of the remaining available computing nodes when cooperating, further calculates the group cooperation efficiency promotion coefficient of each available computing node to select the available computing node as a general computing node for grouping.

[0064] In some preferred embodiments, the method for the management node to cooperate with the remaining available computing nodes to form a computing resource group comprises: sequentially calculating the group assistance comprehensive promotion effective capability that the available computing node can provide when cooperating with all the remaining available computing nodes; calculating the network performance difference between the available computing node and the remaining available computing nodes; calculating the group cooperation efficiency promotion coefficient of each available computing node according to the group assistance comprehensive promotion effective capability and the network performance difference; adding the available computing node with the group cooperation efficiency promotion coefficient greater than a preset efficiency threshold to the computing resource group as a general computing node.

[0065] Further, preferably, the group assistance comprehensive promotion effective capability comprises data operation capability, cache efficiency, cache space, parallel capability and hardware encoding and decoding format support list.

[0066] Further, in some preferred embodiments, the data operation capability is calculated by the computing capability of the available computing node and the number of operations that can be directly performed by using the hardware instruction set in the task creation logic.

[0067] Exemplarily, the data operation capability of the remaining i-th available computing node is represented as: (4); wherein, is the computing capability of the i-th available computing node, is the number of operations that can be directly performed by using the hardware instruction set in the task creation logic.

[0068] In some preferred embodiments, the cache efficiency is calculated according to the latency and cache conversion efficiency of the available computing node.

[0069] Exemplarily, the cache efficiency of the remaining i-th available computing node is represented as: (5); wherein, ​​​the latency for reading from the cache center, the cache conversion efficiency.

[0070] In some preferred embodiments, the cache space is calculated according to the total amount of CPU L1, L2 and L3 cache efficiency of the available computing nodes, the maximum cache size of the cache center, the latency, the network bandwidth speed between the cache centers and the cache conversion efficiency.

[0071] Exemplarily, the cache space of the remaining i-th available computing node is represented as: (6); wherein, the total amount of CPU L1, L2 and L3 cache efficiency, the maximum cache size of the cache center, the latency for reading from the cache center, the network bandwidth speed between the cache centers, the cache conversion efficiency.

[0072] In some preferred embodiments, the parallel capability is calculated according to the number of CPU threads of the available computing nodes, the usage rate of the CPU, the preset usage rate effectiveness selection threshold, the number of GPU computing units and the usage rate of the GPU.

[0073] Exemplarily, the parallel capability of the remaining i-th available computing node is represented as: (7); wherein, nc is the number of CPU threads, the usage rate of the k-th CPU, the usage rate effectiveness selection threshold, ngp is the number of GPU computing units, the usage rate of the k-th GPU, s(x) represents the sign of x, i.e. , represents the GPU operation adaptation coefficient, the minimum is 0 and the maximum is 100, and the setting basis is the proportion of control logic and numerical calculation logic in the algorithm, which is 100 when all of them are numerical calculation logic and 0 when all of them are control logic.

[0074] It should be noted that the hardware encoding and decoding format support list is read through instruction set identification.

[0075] As a preferred embodiment, the management node traverses the remaining available computing nodes, sends a network performance probe consistent request to each available, the content of the network performance probe consistent request is fixed, indicating that the available computing node directly returns the original text. After the management node receives the response, according to the size of the request message and the request sending and response time, the network performance of the management node and the available computing node is calculated, and then the group cooperation efficiency improvement coefficient is calculated according to the network performance and the effective capacity of the group assistance comprehensive improvement.

[0076] Exemplarily, the management node traverses the available computing nodes to obtain the network performance of the management node and each available computing node. (8). Wherein, is the message size of the request , and is the request sending and response time.

[0077] Further, the management node traverses the available computing nodes to obtain the group cooperation efficiency improvement coefficient of the management node and each available computing node. (9).

[0078] After traversal is completed, the group cooperation efficiency improvement coefficient of the management node and all remaining available computing node sets is obtained. Wherein, nn is the number of available computing nodes in the remaining available computing node set.

[0079] In some preferred embodiments, the efficiency threshold includes a fixed efficiency threshold and a floating efficiency threshold. The fixed efficiency threshold is a pre-set static standard, and the floating efficiency threshold is determined by the overall group cooperation efficiency improvement coefficient of the remaining available computing nodes.

[0080] Exemplarily, when the available computing node corresponds to and , it is added to the computing resource group; wherein, is the fixed efficiency threshold, .

[0081] Through the above embodiments, the computing resource group is finally obtained, wherein nbn is the group cooperation efficiency improvement coefficient of the remaining available computing nodes in the computing resource group , which is greater than the fixed efficiency threshold.and the efficiency improvement coefficient the total number of computing nodes greater than or equal to half of the average of all positive efficiency improvement coefficients.

[0082] In the embodiments of the present application, the management node can autonomously form a group according to the group cooperation efficiency improvement coefficient of cooperation with other available computing nodes, ensuring that the cooperation of the ordinary computing nodes and the management node can produce substantial efficiency improvement.

[0083] As a preferred embodiment, step S2, receiving the group cooperation efficiency improvement coefficient of the computing resource group uploaded by the management node, obtaining the efficiency workload of the computing resource group according to the group cooperation efficiency improvement coefficient, comprises: receiving the group cooperation efficiency improvement coefficient of the computing resource group uploaded by the management node; obtaining the computing capacity of each available computing node in the computing resource group; obtaining the efficiency workload of all the available computing nodes in the computing resource group after cooperating with the management node according to the group cooperation efficiency improvement coefficient and the computing capacity.

[0084] In the embodiments of the present application, the group cooperation efficiency improvement coefficient is calculated by formula (4)-(9); and the computing capacity of each available computing node is calculated by formula (1).

[0085] Further, the efficiency workload of the computing resource group NB is expressed as: (10); wherein, the group cooperation efficiency improvement coefficient of the management node cooperating with the kth ordinary computing node , is denoted as the computing capacity of the kth ordinary computing node .

[0086] In the above scheme, the efficiency workload is determined by combining the group cooperation efficiency improvement coefficient uploaded by the management node and the computing capacity of each node in the computing resource group, reflecting the upper limit of the task amount that the computing resource group can carry under the current cooperation state, and providing a basis for subsequent batch task grouping based on the efficiency workload.

[0087] As a preferred embodiment, step S3, assembling the project task set into a batch task grouping according to the efficiency workload, and issuing the batch task grouping to the management node, so that the management node dispatches and executes the batch task grouping, comprises: obtaining the task workload of each project task in the project task set; If the workload of the current project task is less than the efficiency workload, then traverse the project task set and update the current project task. If the workload of the current project task is greater than or equal to the efficiency workload, the current project task is assembled into a batch task group, and the batch task group is sent to the management node so that the management node can assign the batch task group for execution. Update the unexecuted project tasks in the project task set based on the execution status of the project tasks.

[0088] In this embodiment of the invention, the task scheduling center utilizes the entire computing resource pool. Using resources as the unit, the efficiency improvement coefficient of information and group collaboration within the resource group NB is calculated. The efficiency workload of group NB is calculated jointly using equation (10). .

[0089] Next, from the project task set The set of unexecuted project tasks is obtained by filtering. The workload is created by using a forecasting task that summarizes data through iterative loops. By adopting a workload variance minimization approach, the computing resource groups are included. Group the project tasks to form batch tasks. Batch tasks will be issued later. For computing resource groups Management node in .

[0090] If the workload of the current project task is less than the efficiency workload, it indicates a potential resource idleness issue. In this embodiment of the invention, the set of unexecuted project tasks is further traversed. In order to try to Add to the batch task group that maximizes workload middle, Simultaneously update , The total workload of the set of unexecuted project tasks. The workload for the added project tasks. Before adding for the first time, set... .

[0091] If the current workload is not less than the efficiency mentioned above, it means that the scale of the current project task matches the team's collaborative ability. After the current project task is distributed, the computing resource team can execute the task in the best collaborative state.

[0092] For example, if the workload of the current project task Efficiency workload greater than or equal to the number of computing resource groups NB Only the current project task is assigned to the computing resource group NB at this time Batch task grouping .

[0093] In the above scheme, the batch task grouping can be assembled in a way that minimizes the difference in workload, avoiding the idle of resources caused by the task quantity far lower than the efficiency workload, so that the computing capacity of the computing resource group can be fully utilized, and the project task set can be completed with higher efficiency and better resource allocation.

[0094] Further, preferably, the management node segments the batch task grouping, and according to the execution capability of the segmented execution unit and the available computing nodes in the computing resource group, the segmented execution unit is dispatched to the available computing nodes in the computing resource group for execution.

[0095] It should be noted that the batch task grouping is a task set containing several execution units, and each execution unit is the smallest executable particle of the task.

[0096] As a preferred embodiment, the method for the management node to dispatch the batch task grouping includes: According to the task creation logic pipeline, the batch task grouping is split into several segmented execution units; Calculate the resource consumption bias of the segmented execution unit; Obtain the segmented execution advantage of the ordinary computing node; According to the resource consumption bias and the segmented execution advantage, the segmented execution unit is dispatched to the ordinary computing unit.

[0097] Exemplarily, the management node The batch task grouping is segmented according to the task creation logic pipeline, and every 1000 execution units are taken as a grouping to obtain the segmented execution units of the batch task grouping , wherein np is the total number of segments.

[0098] Further, preferably, the resource consumption bias is calculated according to the number of computing logic instructions, the number of data interaction instructions, and the number of non-computing and logic instructions in the instruction set in the segmented execution unit.

[0099] Exemplarily, the resource consumption bias in the segmented is calculated as: (11); ​​Here, mm(x,y,z) means to get the minimum absolute value among x, y, and z, and multiply it by the position of the minimum value (the position can be 1, 2, or 3). Calculate the number of logical instructions for the numerical values ​​in the segment. This represents the number of data interaction instructions in the segment. This represents the number of non-computational and logical instructions in the instruction set within the segment.

[0100] Preferably, the segmented execution advantage of a regular computing node is calculated based on cache efficiency, cache space, parallel capability, hardware codec format support list, and the number of hardware codec instructions used in the segment.

[0101] For example, computing resource groups Idle ordinary computing nodes The advantages of segmented execution Represented as: (12); in, For data processing capabilities, For caching efficiency, For cache space, For parallel capabilities, List of supported hardware codec formats. This indicates the number of hardware encoding / decoding instructions used in this segment, and fi(x,y) represents the number of operations performed directly using the hardware instruction set x.

[0102] Further, iterate through and select | |The smallest ordinary computing node Execute segmented execution units When a regular computing node receives an assigned segmented execution unit, it executes the segment itself and sends an execution completion notification to the management node after completion. The management node then continues to assign segmented execution units until all segmented execution units have been assigned.

[0103] In the above scheme, batch tasks are grouped by combining management nodes with ordinary computing nodes. By segmenting the execution into minimum batch units, and considering the resource consumption of each segment and the segmented execution advantages of ordinary computing nodes, execution nodes can be assigned to each segment. Each computing node then executes its segmented task independently and notifies the management node upon completion. .

[0104] In a preferred implementation, step S4, in response to the assignment completion notification from the management node, updates the available computing nodes, and is executed through steps S41-S45: S41. In response to the assignment completion notification from the management node, a polling instruction is sent to the management node at a preset interval. S42. Receive the task progress information, load information and idle node list information uploaded by the management node based on the polling instruction, and determine whether the computing resource community meets the release conditions. S43. If the release condition is met, obtain the node to be released and check whether the supplementary condition is met in other computing resource communities. S44. When the supplementation conditions are met, the node to be released is supplemented to other computing resource communities; S45. If the supplementary conditions are not met, the node to be released shall be released directly.

[0105] In the above scheme, the status of computing resources is tracked in real time through a periodic polling mechanism to dynamically adjust available computing nodes, enabling computing resources to flow on demand globally. This ensures the flexibility of the resource pool, fundamentally reduces waste caused by long-term resource idleness, and improves the system's response speed to dynamic task demands.

[0106] Further, preferably, step S42, receiving the task progress information, load information, and idle node list information uploaded by the management node based on the polling instruction, and determining whether the computing resource community meets the release conditions, includes: Receive the first task progress information, the first load information, and the first idle node list information uploaded by the management node based on the polling instruction; Obtain the preset maximum progress load idle node release coefficient, and combine it with the first task progress information and the first load information to obtain the first pressure load of the computing resource group; If the first pressure load is less than the preset pressure load threshold and the first idle node list information is not empty, then the available computing nodes in the computing resource community are considered to meet the release conditions; otherwise, they are considered not to meet the release conditions.

[0107] For example, the first stress load of the computing resource community is expressed as ,when and At that time, it is considered that the computing resource group meets the release conditions. Among them, This is the first task progress information for the computational resource group. This is the first load information for the computing resource community. The preset maximum progress load idle node release coefficient. This provides information on the first list of free nodes for the computing resource community.

[0108] In the above scheme, the decision on whether to release a node is made by combining the node's idle status and stress load. This allows for the flexible release of computing nodes at different stages of task execution, avoiding the long-term ineffective occupation of computing nodes by computing resource groups.

[0109] Preferably, step S43, if the release condition is met, then the node to be released is obtained, and the supplementary conditions are checked in other computing resource communities, including: If the release conditions are met, then select idle nodes from the computing resource community as nodes to be released; Obtain the second stress load, second idle node list information, and execution time of other computing resource groups; If the second pressure load is less than the preset pressure load threshold, the second idle node list information is empty, and the execution time is not less than the preset execution time threshold, then the computing resource community is considered to meet the supplementary conditions; otherwise, it is considered not to meet the supplementary conditions.

[0110] For example, other computing resource groups The second pressure load is expressed as ,when , and At that time, it was believed that the computing resource group 'The supplementary conditions are met. Among them,' For computing resource groups The second task progress information, For computing resource groups 'Second load information, For computing resource groups 'Execution time, The average lifetime of all computing resource communities. For computing resource groups The second list of free nodes.

[0111] When the supplementary conditions are met, the task scheduling center will compute resource groups. Compute nodes in the first list of available nodes are added to the compute resource community. In China, as a computing resource group 'Ordinary computing nodes. When the supplementary conditions are not met, the task scheduling center will allocate computing resources to groups.' The compute nodes in the first list of idle nodes are reclaimed and released into the compute resource cluster.

[0112] In the above scheme, the groups of computing resources that need to be supplemented are accurately identified by stress load, execution time and idle nodes, and the nodes to be released are accurately supplemented to the groups of computing resources that have continuous demand and can effectively utilize computing nodes, thereby improving the overall resource utilization rate from the source of resource allocation.

[0113] The task scheduling method provided in this invention establishes an effective path for computing nodes to actively participate in scheduling decisions by filtering management nodes and combining the node status within the computing resource group. This breaks the limitation of computing nodes passively receiving tasks in traditional centralized scheduling. Simultaneously, by introducing a group collaboration efficiency improvement coefficient to optimize workload, the task allocation logic is upgraded from solely relying on the online status of computing nodes to deeply integrating the actual collaborative capabilities of the computing resource group. Furthermore, the strategy of batch task grouping based on efficiency workload allows task allocation to better align with the collaboration level of the resource group, fully leveraging the computing power advantages of the collaborative computing resource group and significantly improving overall task processing efficiency.

[0114] This invention provides a task scheduling system. Please refer to [link to relevant documentation]. Figure 2 The task scheduling system includes a computing node grouping module 11, an efficiency workload calculation module 12, a task distribution module 13, and a node update module 14, wherein: The computing node grouping module 11 is used to filter management nodes from available computing nodes in response to the project task set, so that the management nodes can coordinate with the remaining available computing nodes to form a computing resource group. The efficiency workload calculation module 12 is used to receive the group collaboration efficiency improvement coefficient of the computing resource group uploaded by the management node, and obtain the efficiency workload of the computing resource group based on the group collaboration efficiency improvement coefficient. The task assignment module 13 is used to assemble the project task set into batch task groups according to the efficiency workload, and to send the batch task groups to the management node so that the management node can assign the batch task groups for execution. The node update module 14 is used to update the available computing nodes in response to the assignment completion notification from the management node.

[0115] In a preferred embodiment, the task scheduling system further includes an initial task grouping module, used for: In response to an initial task, the initial task is divided into at least one subtask based on the associated projects and content of the initial task; the subtask includes project functions and service call relationships; Traverse the subtasks to obtain the task dependencies of all subtasks and construct the project task dependency graph; Based on the connectivity of the project task dependency graph, the initial tasks are grouped to obtain a project task set.

[0116] In a preferred embodiment, the computing node cluster module 11 includes: A quantity ratio calculation unit is used to calculate the quantity ratio of project tasks in the project task set to the number of available computing nodes in response to the project task set. The detection strategy distribution unit is used to distribute detection strategies to the available computing nodes according to the quantity ratio; the detection strategies include participation status detection strategies and computing capability detection strategies; The detection result receiving unit is used to receive the detection results uploaded by the available computing nodes based on the detection strategy; The management node filtering unit is used to filter management nodes from the available computing nodes based on the detection results and the quantity ratio, so that the management nodes can work with the remaining available computing nodes to form a computing resource group.

[0117] Further, preferably, the detection strategy dissemination unit is specifically used for: If the quantity ratio meets the preset first quantity ratio threshold, then a first detection strategy is issued to the available computing nodes; the first detection strategy is to set the participation status to participation and obtain the computing power based on the hardware configuration and running status of the available computing nodes. If the quantity ratio meets the preset second quantity ratio threshold, then a second detection strategy is issued to the available computing nodes; the second detection strategy is to obtain the participation status based on the utilization rate of the available computing nodes and the preset minimum availability threshold coefficient, and to obtain the computing power based on the hardware configuration and operating status of the available computing nodes. If the quantity ratio meets the preset third quantity ratio threshold, then a third detection strategy is issued to the available computing nodes; the third detection strategy is to obtain the participation status based on the utilization rate of the available computing nodes, and to obtain the computing power based on the hardware configuration and operating status of the available computing nodes.

[0118] Preferably, the management node filtering unit is specifically used for: If the quantity ratio meets the preset first quantity ratio threshold, then based on the detection results, the available computing node with the smallest computing power is selected as the management node. If the quantity ratio meets the preset second quantity ratio threshold, then based on the detection results, available computing nodes with a participation status of "participating" and medium computing power are selected as management nodes. If the quantity ratio meets the preset third quantity ratio threshold, then based on the detection results, select the available computing nodes with the highest participation status and computing power as management nodes. The information of the management node is sent to the available computing nodes so that the management node can work with the remaining available computing nodes to form a computing resource group.

[0119] In a preferred embodiment, the computing resource community includes management nodes and ordinary computing nodes; The ordinary computing nodes are selected by the management node from the remaining available computing nodes based on the computing power and network performance when the management node collaborates with the remaining available computing nodes, and the efficiency improvement coefficient of the group collaboration is calculated; the efficiency improvement coefficient of the group collaboration between the management node and the ordinary computing nodes is greater than a preset efficiency threshold.

[0120] Further, preferably, the efficiency workload calculation module 12 is specifically used for: Receive the efficiency improvement coefficient of the group collaboration of computing resources uploaded by the management node; To acquire the computing power of each available computing node in the computing resource community; Based on the group collaboration efficiency improvement coefficient and the computing power, the efficiency workload after all available computing nodes in the computing resource group collaborate with the management node is obtained.

[0121] In a preferred embodiment, the task distribution module 13 is specifically used for: Get the workload of each task in the project task set; If the workload of the current project task is less than the efficiency workload, then traverse the project task set and update the current project task. If the workload of the current project task is greater than or equal to the efficiency workload, the current project task is assembled into a batch task group, and the batch task group is sent to the management node so that the management node can assign the batch task group for execution. Update the unexecuted project tasks in the project task set based on the execution status of the project tasks.

[0122] Furthermore, preferably, the management node segments the batch task group and, based on the segmented execution units and the execution capabilities of the available computing nodes in the computing resource group, assigns the segmented execution units to the available computing nodes in the computing resource group for execution.

[0123] In a preferred embodiment, the node update module 14 includes: The polling instruction issuing unit is used to issue polling instructions to the management node at preset intervals in response to the assignment completion notification from the management node. The release condition judgment unit is used to receive task progress information, load information and idle node list information uploaded by the management node based on the polling instruction, and to determine whether the computing resource community meets the release condition. The supplementary condition judgment unit is used to obtain the node to be released if the release condition is met, and to check whether the supplementary condition is met in other computing resource communities. The supplement control unit is used to supplement the node to be released to other computing resource communities when the supplement conditions are met; The release control unit is used to directly release the node to be released when the supplementary conditions are not met.

[0124] Further, preferably, the release condition determination unit is specifically used for: Receive the first task progress information, the first load information, and the first idle node list information uploaded by the management node based on the polling instruction; Obtain the preset maximum progress load idle node release coefficient, and combine it with the first task progress information and the first load information to obtain the first pressure load of the computing resource group; If the first pressure load is less than the preset pressure load threshold and the first idle node list information is not empty, then the available computing nodes in the computing resource community are considered to meet the release conditions; otherwise, they are considered not to meet the release conditions.

[0125] Preferably, the supplementary condition judgment unit is specifically used for: If the release conditions are met, then select idle nodes from the computing resource community as nodes to be released; Obtain the second stress load, second idle node list information, and execution time of other computing resource groups; If the second pressure load is less than the preset pressure load threshold, the second idle node list information is empty, and the execution time is not less than the preset execution time threshold, then the computing resource community is considered to meet the supplementary conditions; otherwise, it is considered not to meet the supplementary conditions.

[0126] The task scheduling system provided in this invention, by filtering management nodes and combining the node status within the computing resource group, enables computing nodes to actively participate in scheduling decisions, breaking the limitation of traditional centralized scheduling where computing nodes can only passively receive tasks. Simultaneously, by introducing a group collaboration efficiency improvement coefficient to optimize workload, the task allocation logic is upgraded from solely relying on the online status of computing nodes to deeply integrating the actual collaborative capabilities of the computing resource group. Furthermore, the strategy of batch task grouping based on efficiency workload allows task allocation to better align with the collaboration level of the resource group, fully leveraging the computing power advantages of the collaborative computing resource group and significantly improving overall task processing efficiency.

[0127] Please see Figure 3 , Figure 3 This is a structural block diagram of a task scheduling device provided in an embodiment of the present invention. The task scheduling device includes a processor 31, a memory 32, and a computer program stored in the memory 32 and executable on the processor 31. When the processor 31 executes the computer program, it implements the steps in the above-described task scheduling method embodiments, such as steps S1 to S4.

[0128] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 32 and executed by the processor 31 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the task scheduling device.

[0129] The task scheduling device may include, but is not limited to, a processor 31 and a memory 32. Those skilled in the art will understand that the schematic diagram is merely an example of a task scheduling device and does not constitute a limitation on the task scheduling device. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the task scheduling device may also include input / output devices, network access devices, buses, etc.

[0130] The processor 31 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 31 is the control center of the task scheduling device, connecting various parts of the entire task scheduling device through various interfaces and lines.

[0131] The memory 32 can be used to store the computer programs and / or modules. The processor 31 implements various functions of the task scheduling device by running or executing the computer programs and / or modules stored in the memory 32 and calling the data stored in the memory 32. The memory 32 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 32 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0132] If the modules / units integrated into the task scheduling device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 31, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0133] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A task scheduling method, characterized by, The method comprises the following steps: in response to a project task set, screening a management node from available computing nodes, so that the management node forms a computing resource team with the remaining available computing nodes; receiving a group team cooperation efficiency improvement coefficient of the computing resource team uploaded by the management node, and obtaining an efficiency workload of the computing resource team according to the group team cooperation efficiency improvement coefficient; according to the efficiency workload, assembling the project task set into a batch task group, and issuing the batch task group to the management node, so that the management node dispatches and executes the batch task group; in response to the dispatch completion notification of the management node, updating the available computing nodes.

2. The task scheduling method of claim 1, wherein, Before the step of responding to a project task set, screening a management node from available computing nodes, so that the management node forms a computing resource team with the remaining available computing nodes, the method further comprises the following steps: in response to an initial task, dividing the initial task into at least one subtask according to the associated project and content of the initial task; the subtask includes project function and service call relationship; traversing the subtask to obtain the task dependency relationship of all the subtasks, and constructing a project task dependency relationship graph; grouping the initial task according to the connectivity of the project task dependency relationship graph to obtain a project task set.

3. The task scheduling method of claim 1, wherein, The step of responding to a project task set, screening a management node from available computing nodes, so that the management node forms a computing resource team with the remaining available computing nodes, comprises the following steps: in response to a project task set, calculating the number ratio of the project task in the project task set to the available computing nodes; issuing a detection strategy to the available computing nodes according to the number ratio; the detection strategy includes a participation state detection strategy and a computing capacity detection strategy; receiving the detection result uploaded by the available computing nodes based on the detection strategy; screening a management node from the available computing nodes according to the detection result and the number ratio, so that the management node forms a computing resource team with the remaining available computing nodes.

4. The task scheduling method of claim 3, wherein, The step of issuing a detection strategy to the available computing nodes according to the number ratio, comprises the following steps: if the number ratio meets a preset first number ratio threshold, issuing a first detection strategy to the available computing nodes; the first detection strategy sets the participation state as participation, and obtains the computing capacity according to the hardware configuration and running state of the available computing nodes; if the number ratio meets a preset second number ratio threshold, issuing a second detection strategy to the available computing nodes; the second detection strategy obtains the participation state according to the usage rate of the available computing nodes and a preset minimum available threshold coefficient, and obtains the computing capacity according to the hardware configuration and running state of the available computing nodes; if the number ratio meets a preset third number ratio threshold, issuing a third detection strategy to the available computing nodes; the third detection strategy obtains the participation state according to the usage rate of the available computing nodes, and obtains the computing capacity according to the hardware configuration and running state of the available computing nodes.

5. A task scheduling method as claimed in claim 3 or 4, characterized in that, The management node is selected from the available computing nodes according to the detection result and the quantity ratio, so that the management node cooperates with the remaining available computing nodes to form a computing resource group, and the method comprises the following steps of: If the quantity ratio meets a preset first quantity ratio threshold, the available computing node with the minimum computing capacity is selected as the management node according to the detection result; If the quantity ratio meets a preset second quantity ratio threshold, the available computing node with the medium computing capacity and the participation state of participation is selected as the management node according to the detection result; If the quantity ratio meets a preset third quantity ratio threshold, the available computing node with the maximum computing capacity and the participation state of participation is selected as the management node according to the detection result; The information of the management node is sent to the available computing nodes, so that the management node cooperates with the remaining available computing nodes to form a computing resource group.

6. The method of claim 1, wherein, The computing resource group comprises the management node and the common computing node; The common computing node is selected from the remaining available computing nodes by the management node according to the computing capacity and the network performance of the management node and the remaining available computing nodes when they cooperate. The group cooperation efficiency promotion coefficient of the management node and the common computing node is greater than a preset efficiency threshold.

7. The task scheduling method of claim 1 or 6, wherein, The group cooperation efficiency promotion coefficient of the computing resource group uploaded by the management node is received, and the efficiency workload of the computing resource group is obtained according to the group cooperation efficiency promotion coefficient, and the method comprises the following steps of: The group cooperation efficiency promotion coefficient of the computing resource group uploaded by the management node is received. The computing capacity of each available computing node in the computing resource group is obtained. The efficiency workload of all the available computing nodes and the management node in the computing resource group after cooperation is obtained according to the group cooperation efficiency promotion coefficient and the computing capacity.

8. The task scheduling method of claim 1, wherein, The project task set is assembled into a batch task group according to the efficiency workload, and the batch task group is sent to the management node, so that the management node dispatches and executes the batch task group, and the method comprises the following steps of: The task workload of each project task in the project task set is obtained. If the task workload of the current project task is less than the efficiency workload, the project task set is traversed, and the current project task is updated. If the task workload of the current project task is greater than or equal to the efficiency workload, the current project task is assembled into a batch task group, and the batch task group is sent to the management node, so that the management node dispatches and executes the batch task group. The unexecuted project task in the project task set is updated according to the execution situation of the project task.

9. The task scheduling method of claim 1 or 8, wherein, The management node segments the batch task group, and dispatches the segmented execution unit to the available computing nodes in the computing resource group for execution according to the execution capacity of the available computing nodes in the computing resource group.

10. The method of claim 1, wherein, The available computing nodes are updated in response to the dispatch completion notification of the management node, and the method comprises the following steps of: in response to a dispatch completion notification of the management node, issuing a polling instruction to the management node at a preset interval time; receiving task progress information, load information and idle node list information uploaded by the management node based on the polling instruction, and judging whether the computing resource group meets a release condition; if the release condition is met, obtaining a to-be-released node and checking whether a supplement condition is met in other computing resource groups; if the supplement condition is met, supplementing the to-be-released node to other computing resource groups; if the supplement condition is not met, directly releasing the to-be-released node.

11. The task scheduling method of claim 10, wherein, The receiving task progress information, load information and idle node list information uploaded by the management node based on the polling instruction, and judging whether the computing resource group meets a release condition, comprises: receiving first task progress information, first load information and first idle node list information uploaded by the management node based on the polling instruction; obtaining a preset maximum progress load idle node release coefficient, combining the first task progress information, the first load information, to obtain a first stress load degree of the computing resource group; if the first stress load degree is less than a preset stress load degree threshold, and the first idle node list information is not empty, it is considered that the available computing nodes in the computing resource group meet the release condition, otherwise it is considered that the release condition is not met.

12. The task scheduling method of claim 10, wherein, If the release condition is met, the to-be-released node is obtained from the computing resource group, and it is checked whether a supplement condition is met in other computing resource groups, comprising: if the release condition is met, filtering an idle node from the computing resource group as the to-be-released node; obtaining a second stress load degree, second idle node list information and execution time of other computing resource groups; if the second stress load degree is less than a preset stress load degree threshold, the second idle node list information is empty, and the execution time is not less than a preset execution time threshold, it is considered that the computing resource group meets the supplement condition, otherwise it is considered that the supplement condition is not met.

13. A task scheduling system, characterized by comprising: a computing node grouping module, configured to filter a management node from available computing nodes in response to a project task set, so that the management node forms a computing resource group in cooperation with the remaining available computing nodes; an efficiency workload calculation module, configured to receive a group cooperation efficiency improvement coefficient of the computing resource group uploaded by the management node, and obtain an efficiency workload of the computing resource group according to the group cooperation efficiency improvement coefficient; a task issuing module, configured to assemble the project task set into a batch task group according to the efficiency workload, and issue the batch task group to the management node, so that the management node dispatches and executes the batch task group; a node updating module, configured to update the available computing nodes in response to a dispatch completion notification of the management node.

14. A task scheduling device, characterized by comprising a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, when the processor executes the computer program, the task scheduling method in any one of claims 1 to 12 is realized. comprising a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, when the processor executes the computer program, the task scheduling method in any one of claims 1 to 12 is realized.

15. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises a stored computer program, wherein the computer readable storage medium controls a device where the computer readable storage medium is located to perform the task scheduling method as claimed in any one of claims 1 to 12 when the computer program is running.

16. A computer program product, characterised in that, The computer program product comprises a computer program or computer instructions, which are executed by a processor to perform the task scheduling method as claimed in any one of claims 1 to 12.

Images