Calculation power center calculation power scheduling method based on multi-dimensional weight algorithm and storage medium

Abstract

The invention provides a computing power center computing power scheduling method based on a multi-dimensional weight algorithm and a storage medium, and the scheduling method comprises the following steps: S1, obtaining a submitted task of a user through a scheduling portal, forwarding the task to a scheduling management service, and obtaining metadata information such as a resource demand, a priority and deadline of the task by the scheduling management service; s2, the resource management service system obtains the newest NodeMetric of each computing node through a unified monitoring service, task types are divided according to the task resource demand characteristics, and the NodeMetric is a node real-time index; and S3, the scheduling management service system generates Wfinal according to a multi-dimensional index based on the task type through a weight calculator, and sorts the candidate nodes from high to low according to the Wfinal.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer system scheduling, and in particular to a computing power center computing power scheduling method based on a multi-dimensional weight algorithm and a storage medium. BACKGROUND

[0002] With the rapid development of cloud computing, big data, artificial intelligence and other technologies, computing power centers have become a core component of modern information infrastructure. As the core technology of computing power center operation, computing power center computing power scheduling is increasingly important.

[0003] Computing power center computing power scheduling refers to the process of assigning tasks to appropriate computing nodes for processing based on the resource demand characteristics of the tasks and the current state of the system after receiving a computing task request. An effective scheduling strategy not only improves resource utilization, but also significantly improves task response time and overall system performance.

[0004] Current computing power center computing power scheduling technology mainly has the following problems: 1. Limitations of traditional scheduling algorithms: existing First Come First Served (FCFS), Shortest Job First (SJF), Round Robin and other scheduling algorithms are inefficient in handling diverse task demands, and cannot fully utilize the computing resources of the computing power center 2. Imbalanced resource utilization: Traditional scheduling methods often result in some computing nodes being overloaded while others are idle, causing resource waste and performance bottlenecks.

[0005] 3. Lack of real-time adaptability: Existing scheduling systems are difficult to dynamically adjust according to real-time system state and load changes, and cannot effectively respond to sudden loads and system failures.

[0006] Insufficient consideration of multi-dimensional resources: Existing methods usually only consider a single resource dimension such as CPU or memory, lacking comprehensive consideration of multi-dimensional resources such as GPU, storage I / O, and task priority. SUMMARY

[0007] The purpose of the present application is to solve the above problems and provide a computing power center computing power scheduling method based on a multi-dimensional weight algorithm and a storage medium.

[0008] The technical solution of the present application is as follows: In a first aspect, the present application provides a computing power center computing power scheduling method based on a multi-dimensional weight algorithm, which includes the following steps: S1, obtaining a user's submitted task through a scheduling portal, forwarding the task to a scheduling management service, and the scheduling management service obtaining metadata information such as resource requirements, priority, and deadline of the task; S2, a resource management service system obtains the latest NodeMetrics of each computing node through a unified monitoring service, and divides the task type according to the task resource requirement characteristics, wherein the NodeMetrics is a node real-time index; S3, the scheduling management service system generates W_final based on the task type according to the multi-dimensional index through a weight calculator, and sorts the candidate nodes according to W_final from high to low; Wherein, the W_final is the maximum weight value obtained after the multi-dimensional weight calculation of each node, which is used for the node sorting of the scheduler; The multi-dimensional index includes CPU, memory, GPU, storage and queue, and the queue is the task queue to be run on the corresponding computing node; S4, after selecting the first node, performing resource reservation, task distribution and task real-time monitoring, if the allocation fails, backtracking and trying the next node; S5, the computing node periodically reports the node state and the task running state, and the unified monitoring system writes the index database for subsequent weight iteration; S6, if successful, release the resources and update the statistics, if failed or timeout, trigger the rescheduling.

[0009] In a second aspect, the application also provides a storage medium.

[0010] The storage medium is used for storing the scheduling system as described above, to realize the computing power center computing power scheduling method based on the multi-dimensional weight algorithm as described above.

[0011] The advantages or beneficial effects of the above technical solutions at least include: The application improves the utilization efficiency of computing power center computing power resources through multi-dimensional weight algorithm and intelligent scheduling strategy. The system can accurately schedule according to the resource requirement characteristics of the task and the real-time state of the node, effectively avoiding the common resource waste phenomenon in the traditional scheduling method.

[0012] Through dynamic weight calculation, the system can more reasonably allocate CPU, memory, GPU and storage resources, so that the utilization rate of various resources is improved. Compared with the traditional static scheduling method, the adaptive scheduling method of the application can dynamically adjust the resource allocation strategy according to the real-time load condition, so as to maximize the use efficiency of resources.

[0013] The scheduling method of the present application can automatically adapt to different types of workloads and changing system environments. Through real-time monitoring and analysis, the system can quickly identify task types and select the most suitable scheduling strategy, achieving intelligent task allocation without human intervention.

[0014] The dynamic weight adjustment mechanism enables the system to flexibly adjust scheduling decisions based on historical performance, current load, task priority, and other factors. This adaptive capability enables the system to maintain optimal performance in various complex operating environments, effectively responding to sudden load changes and system anomalies. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings illustrate exemplary embodiments of the present application and, together with the general description of the application given above, and the detailed description of the embodiments given below, serve to explain the principles of the present application. These drawings should be understood as being merely illustrative, as the present application can be implemented in various forms.

[0016] Figure 1 A hierarchical architecture diagram of an embodiment of the present application is shown; Figure 2 A scheduling algorithm flowchart of an embodiment of the present application is shown; Figure 3 A service component interaction timing diagram of an embodiment of the present application is shown; Figure 4 A weight calculation process diagram of an embodiment of the present application is shown; DETAILED DESCRIPTION

[0017] Embodiments of the present application will be described in more detail by referring to the accompanying drawings. Although certain embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be interpreted as being limited to the embodiments set forth herein, but rather these embodiments are provided to more thoroughly and completely understand the present application. It should be understood that the drawings and embodiments of the present application are for illustrative purposes only and are not intended to limit the scope of protection of the present application.

[0018] It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0019] It should be understood that the term "include" and variations of "include" used herein are open-ended and mean "including, but not limited to". The term "based on" means "based, at least in part, on". The term "one embodiment" means "at least one embodiment". The term "another embodiment" means "at least one additional embodiment". The term "some embodiments" means "at least some embodiments". Related definitions are given throughout the detailed description. It is noted that the "first", "second", and "third" concepts mentioned in the present application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0020] It should be noted that the modification of "one" or "multiple" mentioned in the present application is illustrative rather than limiting, and those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more".

[0021] The names of the messages or information exchanged between the devices in the embodiments of the present application are only for illustrative purposes, and are not used to limit the scope of the messages or information.

[0022] A computing power center computing power scheduling method based on a multi-dimensional weight algorithm is used to solve the technical problems of unbalanced resource utilization, low scheduling efficiency, lack of real-time adaptability and the like in the prior art, and to realize efficient, balanced and dynamic scheduling of computing power center computing power resources. The scheduling method includes the following key components: The scheduling management layer includes a scheduling manager, a policy engine and the like. The scheduling manager is responsible for receiving and analyzing resource requirements and specifying optimal scheduling decisions. The policy engine selects appropriate scheduling strategies according to preset rules and real-time states.

[0023] The resource management layer is responsible for unified management and monitoring of computing power center resources, and realizes discovery, registration, allocation and release of resources. This layer maintains a list of completed resources, monitors the health status of resources, and provides accurate resource information for upper-layer scheduling decisions.

[0024] The computing node layer provides actual computing power and task execution environment, and includes various computing nodes distributed in the computing power center. Each node is equipped with a task execution service and a resource monitoring service, and can independently execute assigned tasks and report execution status and node resource status in real time.

[0025] The persistence layer provides persistent storage services for the entire system, including storage of task queues, system index databases and configuration information, to ensure the persistence of system state and the consistency of data.

[0026] Based on the above key components, through multi-dimensional weight calculation, the comprehensive consideration of CPU, memory, GPU, storage and other resources is realized, and the waste of resources caused by single resource dimension is avoided; at the same time, through dynamic scheduling mechanism, the scheduling strategy is adjusted according to the real-time state and load change of the system, and the scheduling efficiency is improved.

[0027] Based on the above, the scheduling method comprises the following steps: S1, through the scheduling portal, the user's submitted task is obtained, and the task is forwarded to the scheduling management service, and the scheduling management service obtains the metadata information of the task resource demand, priority and deadline; S2, the resource management service system obtains the latest NodeMetrics of each computing node through the unified monitoring service, and divides the task type according to the task resource demand characteristics, wherein the NodeMetrics is the node real-time index, such as cpu_util, mem_util, gpu_util, io_util, queue_len, uptime and the like; S3, the scheduling management service system generates W_final based on the task type according to multi-dimensional indexes through the weight calculator, and sorts the candidate nodes according to W_final from high to low; Wherein, the W_final is the final weight value obtained after multi-dimensional weight calculation of each node, which is used for node sorting by the scheduler; The multi-dimensional indexes include CPU, memory, GPU, storage and queue, and the queue is the task queue to be run on the corresponding computing node; S4, after selecting the first node, resource reservation, task issuing and task real-time monitoring are performed, and if the allocation fails, the next node is tried; S5, the computing node periodically reports the node state and the task running state, and the unified monitoring system writes the index database for subsequent weight iteration; S6, if successful, release the resources and update the statistics, and if failed or timeout, trigger the rescheduling; S7, scheduling main loop Responsible for continuously taking out the tasks to be processed from the task queue, combining the real-time node monitoring information, calculating the weight, selecting the optimal node and completing the task issuing and tracking; the loop is continuously executed until the scheduling service is stopped, and the process is shown in the following pseudo code: .

[0028] The task classification process in step S2 will classify the tasks according to the following pseudo code logic, and the classification result will drive the adjustment of the multi-dimensional weight coefficient, so that the scheduler will significantly amplify the resource richness of the resources most relevant to the task in the node weight calculation stage After the task is completed or failed, the resources of the corresponding node need to be released, and the process is shown in the following pseudo code: Among them, the multi-dimensional weight calculation method is responsible for converting the current state of the node and the real-time demand of the task into a comprehensive score W_final between 0 and 1, which is used by the scheduler for sorting and taking. The following describes the calculation process with five dimensions as an example. The calculation process is divided into four steps: index normalization, basic weight, dynamic adjustment, and final weight: S101, index normalization: The index normalization is performed by the following formula: ; ; The normalized score of each resource dimension is used to measure the idle degree and availability of the node in different resource dimensions. The higher the score , the more idle it is; wherein, each resource dimension includes cpu, mem, gpu and io, cpu is central processing unit, mem is memory, gpu is graphics processing unit, and io is storage read and write; util is the utilization rate, which is 0~1, the lower the utilization rate, the more remaining capacity, and the calculation normalization formula is calculated by taking the inverse value.

[0029] The queue dimension score in the multi-dimensional weight calculation is used to measure the current task queuing load of the node; this index reflects the "instant availability" of the node, the fewer the queued tasks, the easier it is for the node to process new tasks immediately. queue_len is the queue length, which uses exponential decay (for example, 0.1, which can be adjusted according to the business), so that the score is smoothly reduced, preventing the extreme case of completely excluding the node due to a small amount of queued tasks S102, basic weight calculation: The calculation formula of the basic weight calculation is as follows: ; ; Among them, is the basic weight value, is the weight coefficient of each dimension, which is adjusted according to the task type, , 、 、 and is calculated by step S101.

[0030] S103, calculate the dynamic factor: The calculation formula of the dynamic adjustment factor is as follows: ; is the dynamic adjustment factor; is the task priority, obtained through the scheduling management service, and the value is 0.0-1.0; is the node past 24-hour completion rate and response stability; wherein, 1 in the above formula is used to ensure that the dynamic adjustment factor is not 0; S104, calculate the final weight: The calculation formula of W_final is as follows: ; wherein, is the final weight value, is the base weight, used to obtain the static bearable capacity of the node, is the dynamic adjustment factor.

[0031] Based on the above method, an example is given as follows: Based on the overall scheduling process (including the steps of task entering, resource state acquisition, node weight calculation, strategy selection and node sorting, task allocation and execution, execution period monitoring and feedback, task completion / failure processing, etc.), a specific example is given. Assuming that the data center has 5 computing nodes (Node-A to Node-E), and there are currently 3 tasks to be scheduled (T001, T002, T003). The example focuses on the single round execution of the scheduling main loop, and shows how to process these tasks.

[0032] Example scenario setting Node information (real-time indicators, assuming the current collection time is July 21, 2025, 11:00 AM CST): Task information: System parameters: scheduling interval = 10s, retry upper limit = 3 times, weight coefficient threshold as the aforementioned standard (for example, CPU-intensive α = 0.50, etc.).

[0033] Step 1: Task entering and taking out Task queue contains T001, T002, T003 (sorted by priority: high -> normal -> low).

[0034] Main loop takes T001 (highest priority) from the head of the queue.

[0035] Step 2: Resource state acquisition Acquire 5 available nodes (all nodes are in ACTIVE state) and corresponding real-time indicators (as shown in the table above) through the resource management service.

[0036] Step 3: Node weight calculation Task classification: T001 is CPU-intensive, with basic weight coefficients α=0.50, β=0.20, γ=0.10, δ=0.10, and ε=0.10.

[0037] Calculate W_final for each node (example calculation for Node-A): Normalized score: Basic weight calculation: Dynamic adjustment: Priority factor is Historical performance factor is Final weight, the node with the highest final weight is 1, and if it exceeds 1, it means that the node's related resources are abundant. If there are multiple nodes with a final weight of 1, a node is randomly selected during scheduling: Similarly, calculate the weights of other nodes: Node-A=1.00, Node-B=0.832, Node-C=0.994, Node-D=1.00, Node-E=0.900.

[0038] Resource adaptation check: all node resources meet T001 requirements.

[0039] Step 4: Strategy selection and node sorting Sorting result: Node-A (1.00) > Node-D (1.00) > Node-C (0.994) > Node-E (0.900) > Node-B (0.832).

[0040] Step 5: Task allocation and execution Select Node-A, try resource reservation (lock 8-core CPU, etc.).

[0041] Task assignment succeeds, T001 starts executing on Node-A.

[0042] Step 6: Monitoring and feedback during execution Node-A reports relevant indicators to the monitoring system every 5s.

[0043] Step 7: Task completion / failure processing Suppose T001 completes within 4min: release resources, update statistics (success rate +1), and record logs.

[0044] The loop continues to process T002 and T003 (similar steps, but the basic weight coefficient is adjusted according to the task type, etc.).

[0045] The above scheduling method is implemented through a scheduling system; The scheduling system comprises: a scheduling management layer, which is configured to receive and analyze resource requirements and specify optimal scheduling decisions, and a policy engine configured to select appropriate scheduling policies according to preset rules and real-time states, wherein the scheduling management layer comprises a scheduling manager and a policy engine; a resource management layer, configured to uniformly manage and monitor the resources of the computing power center, and further configured to discover, register, allocate and release resources, wherein the resource management layer maintains a list of completed resources, monitors the health status of the resources, and provides resource information for the upper-layer scheduling decisions; a computing node layer, configured to provide actual computing power and task execution environment, and comprising various computing nodes distributed in the computing power center, each node being equipped with a task execution service and a resource monitoring service, and being capable of independently executing assigned tasks and reporting execution status and node resource status in real time; a persistent layer, configured to provide persistent storage services for the entire system, including task queue, system indicator database and configuration information storage, and ensuring the persistence of system state and the consistency of data.

[0046] Embodiments of the present application also provide a storage medium for storing the scheduling system as described above, so as to implement the computing power center computing power scheduling method based on the multi-dimensional weight algorithm as described above.

[0047] In the description of the application, it needs to be explained that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the application.

[0048] Those skilled in the art should understand that the above embodiments are only for clearly illustrating the application, and do not limit the scope of the application. Based on the above disclosure, other changes or modifications can also be made by those skilled in the art, and these changes or modifications are still within the scope of the application.

Claims

1. A computing power center computing power scheduling method based on a multi-dimensional weight algorithm, characterized in that: The scheduling method comprises the following steps: S1, obtaining the submitted task of the user through the scheduling portal, and forwarding the task to the scheduling management service, wherein the scheduling management service obtains metadata information such as resource requirements, priority and deadline of the task; S2, the resource management service system obtains the latest NodeMetrics of each computing node through the unified monitoring service, and divides the task type according to the task resource requirement characteristics, wherein the NodeMetrics is a node real-time index; S3, the scheduling management service system calculates the weight through the weight calculator, Based on the task type W_final is generated according to multi-dimensional indexes, and the candidate nodes are sorted according to W_final from high to low. Wherein, the W_final is the final weight value obtained after the multi-dimensional weight calculation of each node, which is used for the node sorting of the scheduler; The multi-dimensional index includes CPU, memory, GPU, storage and queue, and the queue is the task queue to be run on the corresponding computing node; S4, after selecting the first node, performing resource reservation, task issuing and real-time monitoring of the task, if the allocation fails, backtracking and trying the next node; S5, the computing node periodically reports the node state and the task running state, and the unified monitoring system writes the index database for subsequent weight iteration; S6, if successful, release the resources and update the statistics, if failed or timed out, trigger the rescheduling.

2. The computing power center computing power scheduling method based on the multi-dimension weight algorithm of claim 1, characterized in that: The calculation method of the W_final comprises the following steps: S101, index normalization: The index normalization is performed by the following formula: ； ； a normalized score for each resource dimension, for measuring the idle degree and availability of the node in different resource dimensions, the score The higher the more idle; wherein, each resource dimension includes cpu, mem, gpu and io, cpu is central processing unit, mem is memory, gpu is graphics processing unit, and io is read and write of storage. util is the utilization rate, which is 0~1, the lower the utilization rate, the more the remaining capacity, and the reverse value is calculated by the normalization formula; For the queue dimension score in multi-dimension weight calculation, used to measure the current task queuing load of the node; S102, basic weight calculation: The calculation formula of the basic weight calculation is as follows: ； ； wherein, is a base weight value, is a weight coefficient for each dimension, adjusted by task type, , , , and is calculated by step S101; S103, calculation of dynamic factor: The calculation formula of the dynamic adjustment factor is as follows: ； is a dynamic adjustment factor; For task priority, it is obtained through the scheduling management service and takes a value of 0.0-1.0; Node past 24 hour completion rate and response stability; S104, calculation of the final weight: The calculation formula of the W_final is as follows: ； wherein, is the most weighted value, is the base weight for obtaining the static bearable capacity of the node, is the dynamic adjustment factor.

3. The computing power center computing power scheduling method based on the multi-dimensional weight algorithm according to claim 2, characterized in that: The scheduling method further comprises step S7: S7, scheduling main loop Responsible for continuously taking out the to-be-processed task from the task queue, combining the real-time node monitoring information, calculating the weight, selecting the optimal node and completing the task issuing and tracking; the loop is executed continuously until the scheduling service is stopped.

4. The computing power center computing power scheduling method based on the multi-dimensional weight algorithm according to claim 2, characterized in that: In step S102, If the task type is a hybrid application, then are each set to 0.2; If CPU intensive applications are set to Finally, all the weight coefficients add up to 1.

5. The computing power center computing power scheduling method based on the multi-dimensional weight algorithm according to claim 4, characterized in that: The first 0.5 in the dynamic adjustment factor formula is used to prevent the node from being completely excluded and the performance fluctuation in the short term from causing the node to be excluded for a long time; the second 0.5 coefficient is the historical performance weight, which can be adjusted according to business requirements.

6. The computing power center computing power scheduling method based on the multi-dimension weight algorithm of claim 1, wherein: The scheduling method is realized by a scheduling system; The scheduling system comprises: A scheduling management layer, wherein the scheduling manager is used for receiving and analyzing resource requirements, and specifying the optimal scheduling decision, and a policy engine is used for selecting a suitable scheduling policy according to preset rules and real-time state, and the scheduling management layer comprises a scheduling manager and a policy engine; A resource management layer is configured to uniformly manage and monitor resources of the computing power center, and is further configured to discover, register, allocate and release the resources. The resource management layer maintains a list of completed resources, monitors health states of the resources, and provides resource information for upper-layer scheduling decisions. A computing node layer is configured to provide actual computing capability and a task execution environment, and includes various computing nodes distributed in the computing power center. Each node is equipped with a task execution service and a resource monitoring service, and is capable of independently executing an assigned task and reporting execution states and node resource states in real time. A persistence layer is configured to provide a persistent storage service for the entire system, including storage of a task queue, system index database and configuration information, and ensures persistence of system states and consistency of data.

7. A storage medium characterized by: The storage medium is configured to store the scheduling system of claim 6, so as to implement the computing power center computing power scheduling method based on the multi-dimensional weight algorithm of any one of claims 1-5.

Images