Calculation power resource scheduling method and device, electronic equipment and storage medium

By obtaining the computing power information of the task processing node and generating scheduling instructions, dynamically scheduling the task processing nodes is solved, and the task processing efficiency and system performance are improved.

CN118939397APending Publication Date: 2024-11-12CHINA MOBILE (SUZHOU) SOFTWARE TECH CO LTD +1
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202410987386.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In the prior art, computing power resource scheduling efficiency is low, resulting in serious impact on the performance of computer systems when nodes are busy or failures.

Method used

By obtaining the computing power information of the task processing node, generating preprocessing computing power, and generating scheduling instructions based on the available computing power and preprocessing computing power, dynamically scheduling task processing nodes to achieve flexible allocation and full utilization of resources.

Benefits of technology

It improves task processing efficiency, enhances the reliability and performance of the system, and realizes flexible task allocation in multi-processing node scenarios through real-time resource monitoring and dynamic resource scheduling algorithms.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118939397A_ABST
    Figure CN118939397A_ABST
Patent Text Reader

Abstract

The invention provides a computing power resource scheduling method and device, electronic equipment and a storage medium, and relates to the technical field of resource scheduling, the method comprises the following steps: acquiring computing power information of a computing power task and at least one task processing node, the computing power information representing available computing power of each task processing node; generating preprocessing computing power of the computing power task under the condition that the at least one task processing node meets a preset condition to determine a first target task processing node; generating a first scheduling instruction according to the available computing power of the first target task processing node and the preprocessing computing power, and scheduling the first target task processing node to process the computing power task according to the first scheduling instruction; wherein the preset condition is that the available computing power of the task processing node is greater than the preset computing power. According to the invention, the efficiency of computing power resource scheduling can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of resource scheduling, and in particular to a scheduling method, device, electronic device and storage medium for computing resources. Background Art

[0002] With the development of science and technology, especially in modern computer systems, the distribution and processing of tasks are very critical. Traditional task distribution and processing systems usually adopt static resource allocation strategies, and the distribution of tasks between different nodes is fixed. When some nodes are busy or fail, the performance of the entire system will be seriously affected. Therefore, how to achieve dynamic resource allocation and task scheduling to improve the reliability and performance of the system is a hot topic in computer system research.

[0003] In the prior art, task scheduling usually adopts a manual allocation method, where a task processing node is manually selected to process the target task, or automatically allocates tasks among multiple task processing nodes, that is, automatically allocates one or more task processing nodes to process the target task.

[0004] It can be seen that the existing technology has the problem of low efficiency in scheduling computing resources. Summary of the invention

[0005] Embodiments of the present invention provide a method, device, electronic device and storage medium for scheduling computing resources to solve the problem of low efficiency in scheduling computing resources in the prior art.

[0006] In a first aspect, an embodiment of the present invention provides a method for scheduling computing resources, including:

[0007] Obtain computing power information of a computing power task and at least one task processing node, wherein the computing power information indicates available computing power of each task processing node;

[0008] In the case where the at least one task processing node satisfies a preset condition to determine the first target task processing node, generating pre-processing computing power of the computing power task;

[0009] Generate a first scheduling instruction according to the available computing power of the first target task processing node and the pre-processing computing power, and schedule the first target task processing node to process the computing power task according to the first scheduling instruction;

[0010] Among them, the preset condition is that the available computing power of the task processing node is greater than the preset computing power.

[0011] Optionally, the available computing power of each task processing node is obtained by the following rule:

[0012] Obtain computing resource information and computing resource utilization rate of each task processing node, wherein the computing resource information indicates the total computing resource, and the computing resource utilization rate indicates the real-time utilization rate of the computing resource;

[0013] Generate the available computing power of each task processing node according to the computing power resource information and the computing power resource utilization rate;

[0014] The computing resources include at least one of the following: central processing unit (CPU), memory, disk and network bandwidth.

[0015] Optionally, the generating of the preprocessing computing power of the computing power task includes:

[0016] When the available computing power of the first target task processing node is greater than the preset computing power, obtaining the task type of the computing power task;

[0017] When the first target task processing node has executed the computing power task of the task type a number of times greater than or equal to 1, the preprocessing computing power is calculated by the distributed gradient boosting library XGBoost.

[0018] Optionally, after generating the preprocessing computing power of the computing power task, the method further includes:

[0019] Within a preset time period, when the available computing power represented by the computing power information is less than the pre-processing computing power, determining the maximum available computing power of the at least one task processing node;

[0020] Determine a second target task processing node according to the maximum available computing power, and generate a second scheduling instruction, wherein the second target task processing node is a task processing node with the largest available computing power among the at least one task processing node;

[0021] The second target task processing node is scheduled to process the computing task according to the second scheduling instruction.

[0022] Optionally, generating a first scheduling instruction according to the available computing power of the first target task processing node and the pre-processing computing power, and scheduling the first target task processing node to process the computing power task according to the first scheduling instruction includes:

[0023] Determine the size of the available computing power of the first target task processing node and the preprocessing computing power;

[0024] When the available computing power of the first target task processing node is greater than or equal to the pre-processing computing power, generating the first scheduling instruction;

[0025] The computing task is allocated to the first target task processing node according to the first scheduling instruction, and the computing task is processed by the first target task processing node.

[0026] Optionally, when multiple computing tasks are acquired at the same time, the multiple computing tasks are processed according to the following rules:

[0027] Get the priority information of each computing task;

[0028] Sort each computing task according to the priority information and perform scheduling processing of computing resources in sequence;

[0029] Among them, the priority information is the priority for executing computing resource scheduling processing.

[0030] Optionally, after generating a first scheduling instruction according to the available computing power of the first target task processing node and the pre-processing computing power, and scheduling the first target task processing node to process the computing power task according to the first scheduling instruction, the method further includes:

[0031] Obtaining a task execution result sent by the first target task processing node;

[0032] When the task execution result is abnormal, determining the abnormal type of the task execution result;

[0033] When the exception type is repeatable, the computing task is re-sent to the at least one task processing node for task processing.

[0034] In a second aspect, an embodiment of the present invention further provides a computing resource scheduling device, the device comprising:

[0035] A first acquisition module, used to acquire computing power tasks and computing power information of at least one task processing node, wherein the computing power information represents the available computing power of each task processing node;

[0036] A first generating module, configured to generate pre-processing computing power for the computing power task when the at least one task processing node satisfies a preset condition to determine a first target task processing node;

[0037] A second generating module is used to generate a first scheduling instruction according to the available computing power of the first target task processing node and the pre-processing computing power, and schedule the first target task processing node to process the computing power task according to the first scheduling instruction;

[0038] Among them, the preset condition is that the available computing power of the task processing node is greater than the preset computing power.

[0039] Optionally, the available computing power of each task processing node is obtained by the following rule:

[0040] Obtain computing resource information and computing resource utilization rate of each task processing node, wherein the computing resource information indicates the total computing resource, and the computing resource utilization rate indicates the real-time utilization rate of the computing resource;

[0041] Generate the available computing power of each task processing node according to the computing power resource information and the computing power resource utilization rate;

[0042] The computing resources include at least one of the following: central processing unit (CPU), memory, disk and network bandwidth.

[0043] Optionally, the first generating module includes:

[0044] an acquisition unit, configured to acquire the task type of the computing power task when the available computing power of the first target task processing node is greater than the preset computing power;

[0045] The first generating unit is used to calculate the preprocessing computing power through the distributed gradient boosting library XGBoost when the first target task processing node has executed the computing power task of the task type a number of times greater than or equal to 1.

[0046] Optionally, the device further comprises:

[0047] A first processing module, configured to determine the maximum available computing power of the at least one task processing node within a preset time period and when the available computing power represented by the computing power information is less than the pre-processing computing power;

[0048] A third generating module is used to determine a second target task processing node according to the maximum available computing power, and generate a second scheduling instruction, wherein the second target task processing node is a task processing node with the largest available computing power among the at least one task processing node;

[0049] The second processing module is used to schedule the second target task processing node to process the computing task according to the second scheduling instruction.

[0050] Optionally, the second generating module includes:

[0051] A processing unit, used to determine the size of the available computing power of the first target task processing node and the pre-processing computing power;

[0052] A second generating unit, configured to generate the first scheduling instruction when the available computing power of the first target task processing node is greater than or equal to the pre-processing computing power;

[0053] An allocating unit is used to allocate the computing task to the first target task processing node according to the first scheduling instruction, and the first target task processing node processes the computing task.

[0054] Optionally, when multiple computing tasks are acquired at the same time, the multiple computing tasks are processed according to the following rules:

[0055] Get the priority information of each computing task;

[0056] Sort each computing task according to the priority information and perform scheduling processing of computing resources in sequence;

[0057] Among them, the priority information is the priority for executing computing resource scheduling processing.

[0058] Optionally, the device further comprises:

[0059] A second acquisition module, used to acquire the task execution result sent by the first target task processing node;

[0060] A determination module, used for determining the abnormal type of the task execution result when the task execution result is abnormal;

[0061] The distribution module is used to re-distribute the computing power task to the at least one task processing node for task processing when the exception type is repeatable.

[0062] In a third aspect, an embodiment of the present invention provides an electronic device, including a transceiver and a processor.

[0063] The transceiver is used to obtain computing power tasks and computing power information of at least one task processing node, wherein the computing power information represents the available computing power of each task processing node;

[0064] The processor is used to generate pre-processing computing power of the computing power task when the at least one task processing node meets a preset condition to determine the first target task processing node;

[0065] The processor is further configured to generate a first scheduling instruction according to the available computing power of the first target task processing node and the pre-processing computing power, and schedule the first target task processing node to process the computing power task according to the first scheduling instruction;

[0066] Among them, the preset condition is that the available computing power of the task processing node is greater than the preset computing power.

[0067] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the method for scheduling computing resources as described in the first aspect are implemented.

[0068] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for scheduling computing resources as described in the first aspect are implemented.

[0069] In a sixth aspect, an embodiment of the present invention provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of the method for scheduling computing resources as described in the first aspect.

[0070] In an embodiment of the present invention, firstly, a computing power task and computing power information of at least one task processing node are obtained, wherein the computing power information represents the available computing power of each task processing node, and then a task processing node whose available computing power is greater than a preset computing power is selected from the at least one task processing node to obtain a first target task processing node, and then a preprocessing computing power of the computing power task is generated, and finally a first scheduling instruction is generated based on the available computing power of the first target task processing node and the preprocessing computing power, and the first target task processing node is scheduled to process the computing power task according to the first scheduling instruction. Through this method, through real-time resource monitoring and management, according to a dynamic resource scheduling algorithm, flexible task allocation in a multi-processing node scenario is achieved, and a task processing node whose computing power can match the computing power task can be selected from at least one task processing node, thereby making full use of resource computing power and improving task processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0072] Figure 1 It is a flowchart of a method for scheduling computing resources provided by an embodiment of the present invention;

[0073] Figure 2 It is a structural diagram of a scheduling system for computing resources provided by an embodiment of the present invention;

[0074] Figure 3 It is a flowchart of another method for scheduling computing resources provided by an embodiment of the present invention;

[0075] Figure 4 It is a structural schematic diagram of a scheduling device for computing resources provided by an embodiment of the present invention;

[0076] Figure 5 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0077] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0078] The terms "first", "second", etc. in the embodiments of the present invention are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0079] See also Figure 1 , Figure 1 is a flow chart of a method for scheduling computing resources provided by an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0080] Step 101: Obtain computing power tasks and computing power information of at least one task processing node, where the computing power information indicates available computing power of each task processing node;

[0081] Step 102: generating preprocessing computing power of the computing power task when the at least one task processing node satisfies a preset condition to determine the first target task processing node;

[0082] Step 103: Generate a first scheduling instruction according to the available computing power of the first target task processing node and the pre-processing computing power, and schedule the first target task processing node to process the computing power task according to the first scheduling instruction;

[0083] Among them, the preset condition is that the available computing power of the task processing node is greater than the preset computing power.

[0084] The computing resource scheduling method provided in the embodiment of the present invention may be executed by an electronic device, such as a computer device, etc., and is not limited in the embodiment of the present invention.

[0085] In step 101, the computing task may be a task request sent by an execution device or other device, for example, copying or pasting a target file, migrating a target file, and deleting a target file, which can all be understood as a computing task.

[0086] In addition, the above-mentioned task processing node can be understood as the executor of the above-mentioned computing power task, which is responsible for the execution of the specific computing power task, wherein each task processing node can include the above-mentioned computing power information, and the above-mentioned computing power information represents the available computing power of the task processing node.

[0087] It should be understood that each task processing node also includes a theoretical peak computing power. The peak computing power can better evaluate the task processing capability of each task processing node. The peak computing power can be calculated based on the processor model and the following formula:

[0088] S max =CPUa×CPUb;

[0089] Among them, S max It represents the peak computing power of the task processing node, CPUa represents the main frequency of the central processing unit (CPU), and CPUb represents the number of floating-point operations performed by the CPU in each clock cycle.

[0090] It should be noted that a task processing node may include multiple processors, so the peak computing power of the task processing node can be obtained by adding the peak computing power of each processor, that is:

[0091]

[0092] Among them, S max Indicates the peak computing power of the task processing node, represents the peak computing power of the first processor, represents the peak computing power of the second processor, Indicates the peak computing power of the nth processor.

[0093] In step 102, a task processing node whose available computing power is greater than the preset computing power is screened out from the at least one task processing node, that is, the first target task processing node is obtained. The first target task processing node may be a set including at least one task processing node. If the first target task processing node only includes one task processing node, then it means that there is only one task processing node whose available computing power is greater than the preset computing power among the at least one task processing node.

[0094] In addition, the above-mentioned preprocessing computing power can be understood as the expected processing computing power of the above-mentioned computing power tasks. In addition to considering the available computing power of the task processing node, it is also necessary to consider whether the task processing node has processed tasks of the same type as the above-mentioned computing power tasks. If the task processing node has not processed tasks of the same task type, it may affect the final computing power. Therefore, in this step, it is necessary to generate the preprocessing computing power of the above-mentioned computing power nodes to achieve an accurate assessment of the computing power of the above-mentioned first target task processing node, and further achieve better task processing effects.

[0095] In step 103, a first scheduling instruction is generated based on the available computing power of the above-mentioned first target task processing node and the above-mentioned preprocessing computing power, so as to schedule the above-mentioned first target task processing node to process the above-mentioned computing power task, wherein the relationship between the available computing power and the preprocessing computing power of the above-mentioned first target task processing node can be obtained, for example: the available computing power and the preprocessing computing power of the above-mentioned first target task processing node can be compared in size, and when the above-mentioned preprocessing computing power is greater than the above-mentioned available computing power, the above-mentioned first scheduling instruction is generated.

[0096] In this implementation scheme, firstly, the computing power task and the computing power information of at least one task processing node are obtained, wherein the computing power information represents the available computing power of each task processing node, and then a task processing node whose available computing power is greater than a preset computing power is selected from the at least one task processing node to obtain a first target task processing node, based on which a preprocessing computing power of the computing power task is generated, and finally, according to the available computing power of the first target task processing node and the preprocessing computing power, a first scheduling instruction is generated, and according to the first scheduling instruction, the first target task processing node is scheduled to process the computing power task. Through this method, through real-time resource monitoring and management and based on a dynamic resource scheduling algorithm, flexible task allocation in a multi-processing node scenario is achieved, and a task processing node whose computing power can match the computing power task can be selected from at least one task processing node, thereby making full use of resource computing power and improving task processing efficiency.

[0097] As an optional implementation, based on the method for scheduling computing resources provided in the embodiment of the present invention, the present invention also provides a scheduling system for computing resources. Figure 2 , Figure 2 It is a structural diagram of the scheduling system of computing resources.

[0098] like Figure 2 As shown, the computing resource scheduling system is provided with a task scheduling center and multiple task processing nodes, wherein the task scheduling center can be understood as the core part, which is used for the centralized management of computing tasks and the management of task processing nodes, and includes the following functions:

[0099] 1. Management of task processing nodes: The task scheduling center provides a unified registration center for each processing node to realize computing power monitoring, resource monitoring, abnormal status monitoring, task number monitoring, etc. of the processing nodes, and provide data support for task allocation and scheduling.

[0100] 2. Allocation and scheduling of computing tasks: The task scheduling center can automatically allocate tasks to users and automatically schedule tasks based on factors such as task type, priority, and urgency, thereby achieving dynamic allocation of tasks.

[0101] 3. Management of computing tasks: The task scheduling center can manage tasks in a unified manner, including task status, priority, resource requirements and other information.

[0102] 4. Monitoring of computing tasks: The task scheduling center can monitor the execution of tasks in real time, including task execution time, completion status and other information.

[0103] In addition, the task processing node can be understood as the execution subject of the computing task, which is responsible for the execution of specific computing tasks and includes the following functions:

[0104] 1. Execution management of computing tasks: Nodes can implement specific tasks assigned by the task scheduling center and achieve stable and accurate operation of tasks based on the node's resource conditions.

[0105] 2. Management of the execution status of computing tasks: The node manages the status of its own task execution and reports the execution results of the task. For normal tasks, after the task execution is completed, the results are directly reported to the task scheduling center. For tasks with abnormal execution, it is determined whether the execution can be retried based on different abnormality types. For tasks that can be retried, they are re-sent to the executor. For tasks that cannot be retried or have been retried too many times, the abnormal results are reported to the task scheduling center.

[0106] 3. Status monitoring of task processing nodes: The node monitors its own status and reports the current status of the node through a regular reporting mechanism, including: peak calculation examples, resource usage status, whether it is abnormal, number of tasks, and other information.

[0107] It should be noted that in the daily processing of the program, it generally includes synchronous operations and asynchronous operations, among which asynchronous operations require the execution of tasks through another task system. After the program receives a request for asynchronous task processing, it can be sent to this system in a unified manner. This system realizes the intelligent allocation and scheduling of tasks through a dynamic resource scheduling algorithm, and can flexibly adjust the task distribution according to the dynamic load of the node, thereby greatly improving the efficiency of task processing. In addition, the system can also realize real-time monitoring and management of node resources, including monitoring and scheduling of resources such as CPU, memory, and disk, while ensuring task processing, it can also ensure the stability and reliability of the system.

[0108] Optionally, the available computing power of each task processing node is obtained by the following rule:

[0109] Obtain computing resource information and computing resource utilization rate of each task processing node, wherein the computing resource information indicates the total computing resource, and the computing resource utilization rate indicates the real-time utilization rate of the computing resource;

[0110] Generate the available computing power of each task processing node according to the computing power resource information and the computing power resource utilization rate;

[0111] The computing resources include at least one of the following: central processing unit (CPU), memory, disk and network bandwidth.

[0112] In this implementation scheme, the computing power resource information and computing power resource utilization rate of each of the above-mentioned task processing nodes are first obtained, so that the available computing power of each task processing node can be determined based on the above-mentioned computing power resource information and the above-mentioned computing power resource utilization rate. Through this method, the available computing power of each task processing node can be accurately confirmed, taking into account the real-time usage status of the task processing node, avoiding uneven distribution of computing power tasks, thereby realizing dynamic management of task processing nodes, efficient use of resources and rapid processing of tasks.

[0113] For ease of understanding, the above computing resources may include parameters such as central processing unit CPU, memory, disk, and network bandwidth. For details, please refer to Table 1:

[0114] Table 1

[0115]

[0116] Among them, the current resource utilization rate reported by the task processing node can be represented by T1, and the utilization rate of each task processing node can form a computing resource group, which can be represented by Z1.

[0117] As a preferred implementation, the above computing resource utilization rate can be brought into a preset model to calculate the available computing power of the task processing node. For details, see the following formula:

[0118]

[0119]

[0120] Among them, S usable represents the available computing power of the task processing node, n represents the number of logic operation chips in the task processing node, represents the current CPU usage of the task processing node, f(a i ) represents the computing power value of the logic chip in the task processing node, q(TOPS) represents the redundant computing power of the CPU in the task processing node, Indicates the current memory usage of the task processing node, R i Represents the total memory of the task processing node, Indicates the current disk usage of the task processing node, S i Represents the total disk of the task processing node, Indicates the current network usage rate of the task processing node, N i Indicates the total network bandwidth of the task processing node.

[0121] It should be noted that the redundant computing power of the CPU in the above-mentioned task processing node can be understood as the computing power indicator in the CPU specification that is higher than the externally advertised value, which is generally a redundant design in hardware design. For example: the CPU specification measured computing power is 100TOPS, but the actual computing power of the CPU may reach 120TOPS, of which 20TOPS is the redundant computing power.

[0122] In addition, the above computing power resources can be added according to actual working conditions or parameters, and the above preset model can be adjusted accordingly to obtain more accurate available computing power, which is not limited in this embodiment of the present invention.

[0123] Based on the above computing resource utilization rate and through the above preset model, taking into account the impact of each computing resource on the available computing power of the task processing node, the available computing power of each task processing node can be accurately obtained, thereby realizing dynamic management of task processing nodes, efficient utilization of resources and rapid processing of tasks.

[0124] Optionally, the generating of the preprocessing computing power of the computing power task includes:

[0125] When the available computing power of the first target task processing node is greater than the preset computing power, obtaining the task type of the computing power task;

[0126] When the first target task processing node has executed the computing power task of the task type a number of times greater than or equal to 1, the preprocessing computing power is calculated by the distributed gradient boosting library XGBoost.

[0127] In this implementation scheme, when the available computing power of the first target task processing node is greater than the preset computing power, the task type of the computing power task is first obtained, and then it is confirmed whether the first target task processing node has processed this type of computing power task. When the number of times the first target task processing node has processed this type of task is greater than or equal to 1, the specific pre-processing computing power is obtained by XGBoost calculation. Through this method, the task type of the computing power task is taken into consideration to plan the task allocation of the task processing unit, and the computing power requirements required to process the computing power task are further confirmed, thereby accurately realizing the scheduling of computing power resources.

[0128] It should be understood that the above-mentioned standard for setting computing power to allow scheduling can be set to S0, when the available computing power of the task processing node is S usable If it is greater than S0, it is marked that the task processing node is allowed to be scheduled, that is, it is confirmed as the first target task processing node mentioned above, otherwise it is marked as not allowed to be scheduled.

[0129] The above-mentioned set computing power can be set by the user according to actual needs or actual working conditions, and is not limited in this embodiment of the present invention.

[0130] For ease of understanding, the following example illustrates the method of calculating the above preprocessing computing power: After obtaining the task type of the above computing power task, if the task of this type is executed for the first time or has no successful execution record (the first time executed in at least one of the above task processing nodes or has no successful execution record), the preprocessing computing power S of the above computing power task can be set. pre If the computing task of this task type has been successfully executed more than or equal to 1 times, the specific pre-processing computing power S is calculated by XGBoost. pre .

[0131] It should be noted that XGBoost is a prediction model that mainly predicts the above preprocessing computing power through the GBDT algorithm.

[0132] Optionally, after generating the preprocessing computing power of the computing power task, the method further includes:

[0133] Within a preset time period, when the available computing power represented by the computing power information is less than the pre-processing computing power, determining the maximum available computing power of the at least one task processing node;

[0134] Determine a second target task processing node according to the maximum available computing power, and generate a second scheduling instruction, wherein the second target task processing node is a task processing node with the largest available computing power among the at least one task processing node;

[0135] The second target task processing node is scheduled to process the computing task according to the second scheduling instruction.

[0136] In this implementation scheme, within a preset time period, if the available computing power represented by the computing power information is less than the pre-processing computing power, it can be understood that the above-mentioned computing power task has not obtained a task processing node for task execution for a long time. Specifically, the pre-processing computing power of the above-mentioned computing power task is greater than the available computing power of all task processing nodes within the preset time period. Then, the maximum available computing power in the above-mentioned at least one task processing node can be determined, and the above-mentioned second target task processing node can be determined, that is, the task processing node with the highest available computing power is selected to execute the computing power task, and finally the above-mentioned second scheduling instruction is generated, and the above-mentioned second target task processing node is scheduled to process the above-mentioned computing power task according to the above-mentioned second scheduling instruction. Through this method, based on the dynamic resource scheduling algorithm, flexible task allocation in a multi-processing node scenario is realized, and a task processing node whose computing power can match the computing power task can be selected from at least one task processing node, thereby making full use of resource computing power and improving task processing efficiency.

[0137] It should be noted that the above-mentioned preset time period can be freely set by the user, and can be set to 10 minutes, 30 minutes or 60 minutes, etc. The specific time can be set according to actual needs, and this embodiment of the present invention is not limited to this.

[0138] Optionally, generating a first scheduling instruction according to the available computing power of the first target task processing node and the pre-processing computing power, and scheduling the first target task processing node to process the computing power task according to the first scheduling instruction includes:

[0139] Determine the size of the available computing power of the first target task processing node and the preprocessing computing power;

[0140] When the available computing power of the first target task processing node is greater than or equal to the pre-processing computing power, generating the first scheduling instruction;

[0141] The computing task is allocated to the first target task processing node according to the first scheduling instruction, and the computing task is processed by the first target task processing node.

[0142] In this implementation scheme, by judging the available computing power of the first target task processing node and the size of the pre-processing computing power, it is determined whether the first target task processing node can execute the computing power task. When the available computing power of the first target task processing node is greater than the pre-processing computing power, it can be understood that the real-time available computing power of the first target task processing node meets the computing power requirements of the computing power task and can complete the execution of the computing power task. Through this method, the task processing node that can meet the computing power task requirements is scheduled in at least one of the above-mentioned task processing nodes to perform task processing, so as to realize flexible task allocation in a multi-processing node scenario, and select a task processing node whose computing power can match the computing power task in at least one task processing node, so as to make full use of resource computing power and improve task processing efficiency.

[0143] As an alternative implementation, see Figure 3 , Figure 3 is another structural diagram of a method for scheduling computing resources provided by an embodiment of the present invention, such as Figure 3 As shown, first, the computing power reported by the processing node can be obtained, wherein the reported computing power can be understood as the peak computing power corresponding to the task processing node. The purpose of this step is to obtain the approximate computing power of each task processing node on the one hand, and on the other hand, each task processing node can be grouped according to the peak computing power to obtain several task processing nodes with high computing power. Then the processing node reports the real-time utilization rate of each resource. This step paves the way for the next step of calculating the available computing power. Finally, it is determined whether the computing power node is allowed to be scheduled, specifically, the size of the available computing power of the computing power node and the pre-processing computing power of the computing power task is determined. When the available computing power is greater than the pre-processing computing power, it is indicated that the task processing node can be scheduled to process the computing power task. Otherwise, it means that the computing power task cannot be processed and the entire process ends.

[0144] Optionally, when multiple computing tasks are acquired at the same time, the multiple computing tasks are processed according to the following rules:

[0145] Get the priority information of each computing task;

[0146] Sort each computing task according to the priority information and perform scheduling processing of computing resources in sequence;

[0147] Among them, the priority information is the priority for executing computing resource scheduling processing.

[0148] In this implementation, when multiple computing tasks are received, priority information of each computing task is obtained. The priority information can be used to indicate the urgency of the multiple computing tasks and the level of priority processing. The multiple computing tasks are arranged in order from high to low priority, and the scheduling of computing resources is performed in sequence, that is, task processing nodes are assigned one by one to process the computing tasks. Through this method, computing tasks that need urgent processing can be given priority, user needs are met, and the allocation effect of computing resources is improved.

[0149] For ease of understanding, for a computing task, if the target data center that matches the computing resources is not successfully matched, a failure to match prompt needs to be fed back to the system account that submitted the computing task. Based on the returned failure to match prompt, the user can choose to cancel the computing task or wait. If the user chooses to cancel the computing task, the user can send a task cancellation instruction to the scheduling center, and the scheduling center cancels the computing task in response to the task cancellation instruction from the user; if the user chooses to wait for the computing task to be successfully matched, the user can send a task waiting instruction to the scheduling center through the system account, and the scheduling center increases the priority information of the computing task in response to the task waiting instruction from the system account, and the computing task enters the queue state.

[0150] Optionally, after generating a first scheduling instruction according to the available computing power of the first target task processing node and the pre-processing computing power, and scheduling the first target task processing node to process the computing power task according to the first scheduling instruction, the method further includes:

[0151] Obtaining a task execution result sent by the first target task processing node;

[0152] When the task execution result is abnormal, determining the abnormal type of the task execution result;

[0153] When the exception type is repeatable, the computing task is re-sent to the at least one task processing node for task processing.

[0154] In this implementation scheme, the task execution result sent by the first target task processing node is obtained after a certain period of time. The task execution result may include successful execution and abnormal execution. In the case that the task execution result is abnormal, it is determined whether the computing task can be executed repeatedly. Considering that the special computing task cannot continue to be executed, but in the case that the computing task can be executed repeatedly, the computing task is reallocated according to the above scheme, and the task is reprocessed by the task processing node. In this scheme, the fault tolerance rate of computing task processing is improved, and the unprocessed computing task is avoided after special circumstances occur. The process of computing task processing is improved, and the scheduling effect of computing resources is improved.

[0155] See also Figure 4 , Figure 4 is a schematic diagram of a computing resource scheduling device provided by an embodiment of the present invention, such as Figure 4 As shown, the computing resource scheduling device 400 includes:

[0156] A first acquisition module 401 is used to acquire computing power tasks and computing power information of at least one task processing node, where the computing power information represents the available computing power of each task processing node;

[0157] A first generating module 402, configured to generate pre-processing computing power for the computing power task when the at least one task processing node satisfies a preset condition to determine a first target task processing node;

[0158] A second generating module 403 is used to generate a first scheduling instruction according to the available computing power of the first target task processing node and the pre-processing computing power, and schedule the first target task processing node to process the computing power task according to the first scheduling instruction;

[0159] Among them, the preset condition is that the available computing power of the task processing node is greater than the preset computing power.

[0160] Optionally, the available computing power of each task processing node is obtained by the following rule:

[0161] Obtain computing resource information and computing resource utilization rate of each task processing node, wherein the computing resource information indicates the total computing resource, and the computing resource utilization rate indicates the real-time utilization rate of the computing resource;

[0162] Generate the available computing power of each task processing node according to the computing power resource information and the computing power resource utilization rate;

[0163] The computing resources include at least one of the following: central processing unit (CPU), memory, disk and network bandwidth.

[0164] Optionally, the first generating module includes:

[0165] an acquisition unit, configured to acquire the task type of the computing power task when the available computing power of the first target task processing node is greater than the preset computing power;

[0166] The first generating unit is used to calculate the preprocessing computing power through the distributed gradient boosting library XGBoost when the first target task processing node has executed the computing power task of the task type a number of times greater than or equal to 1.

[0167] Optionally, the device further comprises:

[0168] A first processing module, configured to determine the maximum available computing power of the at least one task processing node within a preset time period and when the available computing power represented by the computing power information is less than the pre-processing computing power;

[0169] A third generating module is used to determine a second target task processing node according to the maximum available computing power, and generate a second scheduling instruction, wherein the second target task processing node is a task processing node with the largest available computing power among the at least one task processing node;

[0170] The second processing module is used to schedule the second target task processing node to process the computing task according to the second scheduling instruction.

[0171] Optionally, the second generating module includes:

[0172] A processing unit, used to determine the size of the available computing power of the first target task processing node and the pre-processing computing power;

[0173] A second generating unit, configured to generate the first scheduling instruction when the available computing power of the first target task processing node is greater than or equal to the pre-processing computing power;

[0174] An allocating unit is used to allocate the computing task to the first target task processing node according to the first scheduling instruction, and the first target task processing node processes the computing task.

[0175] Optionally, when multiple computing tasks are acquired at the same time, the multiple computing tasks are processed according to the following rules:

[0176] Get the priority information of each computing task;

[0177] Sort each computing task according to the priority information and perform scheduling processing of computing resources in sequence;

[0178] Among them, the priority information is the priority for executing computing resource scheduling processing.

[0179] Optionally, the device further comprises:

[0180] A second acquisition module, used to acquire the task execution result sent by the first target task processing node;

[0181] A determination module, used for determining the abnormal type of the task execution result when the task execution result is abnormal;

[0182] The distribution module is used to re-distribute the computing power task to the at least one task processing node for task processing when the exception type is repeatable.

[0183] Specifically, see Figure 5 An embodiment of the present invention further provides an electronic device, including a bus 501, a transceiver 502, an antenna 503, a bus interface 504, a processor 505 and a memory 506.

[0184] The transceiver 502 is used to obtain computing power tasks and computing power information of at least one task processing node, wherein the computing power information indicates the available computing power of each task processing node;

[0185] Processor 505, configured to generate pre-processing computing power of the computing power task when the at least one task processing node satisfies a preset condition to determine the first target task processing node;

[0186] The processor 505 is further configured to generate a first scheduling instruction according to the available computing power of the first target task processing node and the pre-processing computing power, and schedule the first target task processing node to process the computing power task according to the first scheduling instruction;

[0187] Among them, the preset condition is that the available computing power of the task processing node is greater than the preset computing power.

[0188] exist Figure 5In the embodiment, the bus architecture (represented by bus 501) is shown, and bus 501 may include any number of interconnected buses and bridges, and bus 501 links various circuits including one or more processors represented by processor 505 and memory represented by memory 506. Bus 501 may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. Bus interface 504 provides an interface between bus 501 and transceiver 502. Transceiver 502 may be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. Data processed by processor 505 is transmitted on a wireless medium via antenna 503, and further, antenna 503 also receives data and transmits the data to processor 505.

[0189] The processor 505 is responsible for managing the bus 501 and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management and other control functions. The memory 506 can be used to store data used by the processor 505 when performing operations.

[0190] Optionally, the processor 505 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD).

[0191] An embodiment of the present invention also provides an electronic device, comprising: a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, the various processes of the above-mentioned indication method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0192] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned policy management method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0193] An embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the method embodiment shown in Figure A or Figure B above are implemented, and the same technical effect can be achieved. To avoid repetition, they are not repeated here.

[0194] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0195] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0196] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for scheduling computing resources, characterized in that: include: Obtain computing power information of a computing power task and at least one task processing node, wherein the computing power information indicates available computing power of each task processing node; In the case where the at least one task processing node satisfies a preset condition to determine the first target task processing node, generating pre-processing computing power of the computing power task; Generate a first scheduling instruction according to the available computing power of the first target task processing node and the pre-processing computing power, and schedule the first target task processing node to process the computing power task according to the first scheduling instruction; Among them, the preset condition is that the available computing power of the task processing node is greater than the preset computing power.

2. The method for scheduling computing resources according to claim 1, characterized in that: The available computing power of each task processing node is obtained by the following rules: Obtain computing resource information and computing resource utilization rate of each task processing node, wherein the computing resource information indicates the total computing resource, and the computing resource utilization rate indicates the real-time utilization rate of the computing resource; Generate the available computing power of each task processing node according to the computing power resource information and the computing power resource utilization rate; The computing resources include at least one of the following: central processing unit (CPU), memory, disk and network bandwidth.

3. The method for scheduling computing resources according to claim 2, characterized in that: The pre-processing computing power for generating the computing power task includes: When the available computing power of the first target task processing node is greater than the preset computing power, obtaining the task type of the computing power task; When the first target task processing node has executed the computing power task of the task type a number of times greater than or equal to 1, the preprocessing computing power is calculated by the distributed gradient boosting library XGBoost.

4. The method for scheduling computing resources according to claim 3, characterized in that: After generating the preprocessing computing power of the computing power task, the method further includes: Within a preset time period, when the available computing power represented by the computing power information is less than the pre-processing computing power, determining the maximum available computing power of the at least one task processing node; Determine a second target task processing node according to the maximum available computing power, and generate a second scheduling instruction, wherein the second target task processing node is a task processing node with the largest available computing power among the at least one task processing node; The second target task processing node is scheduled to process the computing task according to the second scheduling instruction.

5. The method for scheduling computing resources according to claim 3, characterized in that: The generating a first scheduling instruction according to the available computing power of the first target task processing node and the pre-processing computing power, and scheduling the first target task processing node to process the computing power task according to the first scheduling instruction, includes: Determine the size of the available computing power of the first target task processing node and the preprocessing computing power; When the available computing power of the first target task processing node is greater than or equal to the pre-processing computing power, generating the first scheduling instruction; The computing task is allocated to the first target task processing node according to the first scheduling instruction, and the computing task is processed by the first target task processing node.

6. The method for scheduling computing resources according to claim 2, characterized in that: When multiple computing tasks are acquired at the same time, the multiple computing tasks are processed according to the following rules: Get the priority information of each computing task; Sort each computing task according to the priority information and perform scheduling processing of computing resources in sequence; Among them, the priority information is the priority for executing computing resource scheduling processing.

7. The method for scheduling computing resources according to claim 6, characterized in that: After generating a first scheduling instruction according to the available computing power of the first target task processing node and the pre-processing computing power, and scheduling the first target task processing node to process the computing power task according to the first scheduling instruction, the method further includes: Obtaining a task execution result sent by the first target task processing node; When the task execution result is abnormal, determining the abnormal type of the task execution result; When the exception type is repeatable, the computing task is re-sent to the at least one task processing node for task processing.

8. A computing resource scheduling device, characterized in that: include: A first acquisition module, used to acquire computing power tasks and computing power information of at least one task processing node, wherein the computing power information represents the available computing power of each task processing node; A first generating module, configured to generate pre-processing computing power for the computing power task when the at least one task processing node satisfies a preset condition to determine a first target task processing node; A second generating module is used to generate a first scheduling instruction according to the available computing power of the first target task processing node and the pre-processing computing power, and schedule the first target task processing node to process the computing power task according to the first scheduling instruction; Among them, the preset condition is that the available computing power of the task processing node is greater than the preset computing power.

9. The computing resource scheduling device according to claim 8, characterized in that: The available computing power of each task processing node is obtained by the following rules: Obtain computing resource information and computing resource utilization rate of each task processing node, wherein the computing resource information indicates the total computing resource, and the computing resource utilization rate indicates the real-time utilization rate of the computing resource; Generate the available computing power of each task processing node according to the computing power resource information and the computing power resource utilization rate; The computing resources include at least one of the following: central processing unit (CPU), memory, disk and network bandwidth.

10. The computing resource scheduling device according to claim 9, characterized in that: The first generation module comprises: an acquisition unit, configured to acquire the task type of the computing power task when the available computing power of the first target task processing node is greater than the preset computing power; The first generating unit is used to calculate the preprocessing computing power through the distributed gradient boosting library XGBoost when the first target task processing node has executed the computing power task of the task type a number of times greater than or equal to 1.

11. The computing resource scheduling device according to claim 10, characterized in that: The device also includes: A first processing module, configured to determine the maximum available computing power of the at least one task processing node within a preset time period and when the available computing power represented by the computing power information is less than the pre-processing computing power; A third generating module is used to determine a second target task processing node according to the maximum available computing power, and generate a second scheduling instruction, wherein the second target task processing node is a task processing node with the largest available computing power among the at least one task processing node; The second processing module is used to schedule the second target task processing node to process the computing task according to the second scheduling instruction.

12. The computing resource scheduling device according to claim 10, characterized in that: The second generation module comprises: A processing unit, used to determine the size of the available computing power of the first target task processing node and the pre-processing computing power; A second generating unit, configured to generate the first scheduling instruction when the available computing power of the first target task processing node is greater than or equal to the pre-processing computing power; An allocating unit is used to allocate the computing task to the first target task processing node according to the first scheduling instruction, and the first target task processing node processes the computing task.

13. The computing resource scheduling device according to claim 9, characterized in that: When multiple computing tasks are acquired at the same time, the multiple computing tasks are processed according to the following rules: Get the priority information of each computing task; Sort each computing task according to the priority information and perform scheduling processing of computing resources in sequence; Among them, the priority information is the priority for executing computing resource scheduling processing.

14. The computing resource scheduling device according to claim 13, characterized in that: The device also includes: A second acquisition module, used to acquire the task execution result sent by the first target task processing node; A determination module, used for determining the abnormal type of the task execution result when the task execution result is abnormal; The distribution module is used to re-distribute the computing power task to the at least one task processing node for task processing when the exception type is repeatable.

15. An electronic device, characterized in that: Including transceiver and processor, The transceiver is used to obtain computing power tasks and computing power information of at least one task processing node, wherein the computing power information represents the available computing power of each task processing node; The processor is used to generate pre-processing computing power of the computing power task when the at least one task processing node meets a preset condition to determine the first target task processing node; The processor is further configured to generate a first scheduling instruction according to the available computing power of the first target task processing node and the pre-processing computing power, and schedule the first target task processing node to process the computing power task according to the first scheduling instruction; Among them, the preset condition is that the available computing power of the task processing node is greater than the preset computing power.

16. An electronic device, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the method for scheduling computing resources as described in any one of claims 1 to 7 are implemented.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for scheduling computing resources as described in any one of claims 1 to 7 are implemented.

18. A computer program product, characterized in that It comprises computer instructions, which, when executed by a processor, implement the steps of the method for scheduling computing resources as described in any one of claims 1 to 7.

Citation Information

Cited By

  • Standard updating method and device for computing power resources

    CN119621346A

  • A standard update method and device for computing power resources

    CN119621346B

  • Task allocation method, computer-readable storage medium, electronic apparatus and computer program product

    WO2026144605A1