A k8s-based cpu and npu computing power distribution scheduling system

Abstract

The application provides a CPU and NPU computing power distribution scheduling system based on K8s, which comprises a K8s cluster and a scheduling optimization module, the K8s cluster comprises a master node, a plurality of worker nodes and a task characteristic analysis module, each worker node comprises at least one CPU and at least one NPU and is deployed with a dynamic computing power perception module, the dynamic computing power perception module is used for monitoring the real-time resource state of the corresponding worker node, and the real-time resource state comprises a CPU resource state, an NPU resource state and a memory resource state. When a user submits a task, the master node analyzes the task characteristics by calling the task characteristic analysis module, the scheduling optimization module determines a scheduling scheme by calling a built-in distributed optimization algorithm and a node computing power model based on the real-time resource state monitored by the dynamic computing power perception module and the task characteristics, and sends the scheduling scheme to the master node, and finally the master node schedules the task according to the scheduling scheme. The scheduling scheme comprises the worker node, the CPU core number and the NPU number of the task.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of task scheduling, and particularly relates to a CPU and NPU computing power distribution scheduling system based on K8s. BACKGROUND

[0002] In modern cloud computing and edge computing environments, K8s (Kubernetes, a container orchestration engine) has become the preferred orchestration platform for containerized applications. However, with the widespread application of deep learning and high-performance computing, NPU (Neural Processing Unit) has gradually become an important part of the heterogeneous computing environment. The existing K8s scheduling framework mainly targets CPU (Central Processing Unit) and memory resources, and lacks support for NPU. And for task allocation in a heterogeneous computing environment, most methods use static rules or single-objective optimization, which is difficult to dynamically adapt to changes in resource utilization and computing power demand, resulting in resource waste and task performance degradation. Although there have been some researches trying to improve the allocation strategy of heterogeneous resources, these methods are mainly based on hardware-level collaborative optimization. SUMMARY

[0003] The present application provides a CPU and NPU computing power distribution scheduling system based on K8s to at least solve the above technical problems in the prior art.

[0004] According to a first aspect of the present application, a CPU and NPU computing power distribution scheduling system based on K8s is provided, the system comprising a K8s cluster and a scheduling optimization module, the K8s cluster comprising a master node, a plurality of worker nodes and a task characteristic analysis module, each worker node comprising at least one CPU and at least one NPU and deploying a dynamic computing power perception module, each dynamic computing power perception module being connected with the scheduling optimization module.

[0005] The master node is configured to receive a task submitted by a user.

[0006] The task characteristic analysis module is configured to obtain the task characteristics of the task, and send the task and the task characteristics to the scheduling optimization module.

[0007] The dynamic computing power perception module is configured to monitor the real-time resource state of the corresponding worker node, the real-time resource state comprising a CPU resource state, an NPU resource state and a memory resource state.

[0008] The scheduling optimization module is configured to receive the task and the task characteristics sent by the task characteristics analysis module, determine a scheduling scheme of the task according to a distributed optimization algorithm, the task characteristics, a node computing power model and real-time resource states of each node, and send the scheduling scheme to the master node, wherein the scheduling scheme comprises a working node, a CPU core number and an NPU number of the task.

[0009] The master node is further configured to schedule the task according to the scheduling scheme.

[0010] According to an embodiment of the present application, the task characteristics comprise a task type, a computing resource requirement, a time delay requirement, a priority, a dependency relationship and a data location.

[0011] The computing resource requirement comprises a requirement size of CPU, NPU, memory and storage.

[0012] According to an embodiment of the present application, the node computing power model comprises a resource utilization rate model, a resource performance model, an energy consumption model and a computing power balance model.

[0013] According to an embodiment of the present application, the determination of the scheduling scheme of the task according to the distributed optimization algorithm, the task characteristics, the node computing power model and the real-time resource states of each node comprises:

[0014] The real-time resource states of each working node are obtained based on each dynamic computing power perception module;

[0015] A plurality of candidate working nodes are selected from the plurality of working nodes according to the computing resource requirement shown by the task characteristics and the real-time resource states of each working node;

[0016] A scheduling objective function and a constraint condition of the task are constructed;

[0017] The scheduling scheme is determined based on the plurality of candidate working nodes, the distributed optimization algorithm, the scheduling objective function, the constraint condition and the node computing power model.

[0018] According to an embodiment of the present application, the scheduling objective function is:

[0019]

[0020] UCPU and UNPU are utilization rates of CPU and NPU, t is a time delay, and a is a weight factor, which is used to control the balance between t and (UCPU, UNPU).

[0021] UCPU and UNPU are predicted based on a resource utilization rate model in the node computing power model;

[0022] t is calculated based on a resource performance model in the node computing power model.

[0023] According to an embodiment of the present application, the constraint condition comprises at least one of:

[0024] the utilized resource of the worker node does not exceed the node limit resource;

[0025] the task time delay does not exceed the time delay requirement;

[0026] the distance between the worker node assigned to the task and the data location of the task does not exceed a set distance.

[0027] According to an embodiment of the present application, the system further comprises a task optimization migration module, configured to monitor the performance of the task, and migrate the task from the current node to another node when the performance does not meet a set condition, wherein the performance comprises a plurality of performance parameters of the task when running on its corresponding node.

[0028] According to an embodiment of the present application, the performance parameters comprise task execution time delay, resource utilization rate and throughput.

[0029] the set condition is that the performance parameter is lower than a set performance parameter threshold.

[0030] According to an embodiment of the present application, the migrating the task from the current node to another node comprises:

[0031] obtaining running state data of the task;

[0032] determining an updated worker node according to the running state data of the task, the distributed optimization algorithm, the task characteristics, the node computing power model and the real-time resource state of the other node;

[0033] transmitting the task and the running state data to the updated worker node.

[0034] According to an embodiment of the present application, the distributed optimization algorithm comprises one of: genetic algorithm, ant colony optimization algorithm, particle swarm optimization algorithm, distributed reinforcement learning and heuristic scheduling algorithm.

[0035] The embodiment of the application provides a CPU and NPU computing power distribution scheduling system based on K8s, the system comprises a K8s cluster and a scheduling optimization module, the K8s cluster comprises a master node, a plurality of worker nodes and a task characteristic analysis module, each worker node comprises at least one CPU and at least one NPU and is provided with a dynamic computing power perception module, and each dynamic computing power perception module is connected with the scheduling optimization module; the master node is used for receiving a task submitted by a user; the task characteristic analysis module is used for obtaining a task characteristic of the task and sending the task and the task characteristic to the scheduling optimization module; the dynamic computing power perception module is used for monitoring a real-time resource state of the corresponding worker node, wherein the real-time resource state comprises a CPU resource state, an NPU resource state and a memory resource state; the scheduling optimization module is used for receiving the task and the task characteristic sent by the task characteristic analysis module, determining a scheduling scheme of the task according to a distributed optimization algorithm, the task characteristic, a node computing power model and a real-time resource state of each node, and sending the scheduling scheme to the master node, wherein the scheduling scheme comprises a worker node of the task, a CPU core number and an NPU number; and the master node is also used for scheduling the task according to the scheduling scheme. By deploying the dynamic computing power perception module on each worker node of the K8s cluster, the states of the CPU and the NPU in the worker node are monitored in real time, and the dynamic scheduling of the CPU and the NPU is realized in combination with the distributed optimization algorithm and the node computing power model. Under the condition of avoiding hardware modification, the support for the NPU and the dynamic scheduling of the task adapting to resource demand and computing power demand in a heterogeneous computing power environment are realized.

[0036] It should be understood that the matters described in this section are not intended to identify key or important features of the embodiments of the application, nor are they used to limit the scope of the application. Other features of the application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0037] The above and other objects, features and advantages of the example embodiments of the application will be more apparent from the following detailed description read in conjunction with the accompanying drawings, in which:

[0038] In the drawings, identical or corresponding reference signs refer to identical or corresponding parts.

[0039] Figure 1 A structure schematic diagram of the CPU and NPU computing power distribution scheduling system based on K8s provided by the embodiment of the application is shown. DETAILED DESCRIPTION

[0040] In order to make the purposes, characteristics and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0041] Figure 1 The structure of the K8s-based CPU and NPU computing power distribution scheduling system provided by the embodiments of the present application is shown.

[0042] Reference Figure 1 The embodiments of the present application provide a K8s-based CPU and NPU computing power distribution scheduling system, which comprises a K8s cluster 10 and a scheduling optimization module 20. The K8s cluster 10 comprises a master node 101, a plurality of worker nodes 102 and a task characteristic analysis module 103. Each worker node comprises at least one CPU and at least one NPU and is deployed with a dynamic computing power perception module 1021. Each dynamic computing power perception module 1021 is connected with the scheduling optimization module 20.

[0043] The master node 101 is configured to receive a task submitted by a user.

[0044] The master node 101 is also called the control plane of the K8s cluster 10. When a user needs to submit a task, the task can be submitted to the master node 101 of the K8s cluster 10.

[0045] In an embodiment of the present application, the task comprises a plurality of subtasks.

[0046] The task characteristic analysis module 103 is configured to obtain the task characteristics of the task and send the task and the task characteristics to the scheduling optimization module 20.

[0047] When a task is submitted, it usually carries task information, which is usually described in the form of Job or custom resources (such as VCJob). The task information usually includes the following contents: 1) task basic information, such as task name (unique identifier of the task); 2) task type, such as training task, inference task, data preprocessing task, task submission time; 3) computing resource requirement; 4) task priority related information.

[0048] The computing resource requirement includes CPU requirement, NPU requirement, memory requirement and storage requirement. The CPU requirement refers to how many CPU cores or computing time the task needs, the NPU requirement refers to the requirement of the task for a specific hardware accelerator (such as the number of NPU used or the computing power requirement), the memory requirement refers to the amount of memory required by the task (such as GB), and the storage requirement includes the disk space required by the task or the temporary data storage requirement.

[0049] The task priority related information includes service priority, task importance and resource sensitivity. The service priority refers to the task priority specified by the user or system configuration (such as high priority tasks requiring faster resource allocation), the task importance refers to the importance of the task in the overall scheduling (such as online inference tasks taking precedence over offline training tasks), and the resource sensitivity refers to the sensitivity of the task to different resources (such as inference tasks may rely more on NPU, while data processing tasks rely more on CPU and memory).

[0050] After receiving the task, the master node 101 calls the task characteristic analysis module 103, which extracts the task characteristics of the task by analyzing the task information.

[0051] The task characteristic analysis module 103 can extract the task characteristics by extracting key fields.

[0052] In an embodiment of the present application, the task characteristics include task type, computing resource requirement, latency requirement, priority, dependency relationship and data location. Among them, the computing resource requirement includes the requirement size of CPU, NPU, memory and storage.

[0053] Task characteristics refer to the properties and requirements of the task itself, including the following contents: task type: inference task, training task, data preprocessing task, etc.; resource requirement: the requirement size of the task for CPU, NPU, memory and storage; latency requirement: the time limit for completing the task (real-time requirement); priority: the scheduling priority allocated by the system, which determines the importance of the task; dependency relationship: whether the task depends on the results of other tasks, or whether it can be executed in parallel; data location: the storage location and distribution of input data (such as whether it is close to a certain node).

[0054] The dynamic computing power perception module 1021 is used to monitor the real-time resource state of the corresponding worker node 102, and the real-time resource state includes CPU resource state, NPU resource state and memory resource state.

[0055] To support and dynamically schedule the CPU and NPU, a dynamic computing power perception module 1021 is deployed on each worker node 102, which can collect the real-time resource status of the worker node 102 in real time. The real-time resource status includes the CPU resource status, the NPU resource status, and the memory resource status. After deploying the dynamic computing power perception module 1021, DaemonSet is used to ensure that each worker node 102 is covered.

[0056] The CPU resource status includes: total CPU resource capacity, including the current total number of CPU cores and computing power; CPU usage, including the CPU usage resource consumed by running pods and system processes; CPU remaining amount, which is the CPU resource that can be allocated to new tasks; load balancing status, including the current load of CPU, CPU usage rate, or load average.

[0057] The dynamic computing power perception module 1021 can collect the CPU resource status in the following ways: using the built-in resource monitoring tools of k8s, such as kubelet or Metrics Server, to collect the CPU usage rate and occupation from / proc / stat or cAdvisor.

[0058] The NPU resource status includes: total NPU capacity, including the number of NPU or computing power (such as the TOPS performance supported by each NPU) contained in the current node; NPU usage, including the usage rate of NPU, allocated memory, and remaining memory of the currently running model task; NPU remaining computing power, which is the idle computing power resource that can be allocated to new tasks; running status, which detects whether the NPU is in a normal working state.

[0059] The dynamic computing power perception module 1021 can collect the NPU resource status in the following ways: using the device management tools provided by Ascend (such as DCMI or Ascend Device Plugin) to obtain computing power and resource usage information; and reporting the NPU resource status through the k8s scheduling extension plug-in (such as Device Plugin).

[0060] The memory resource status includes: total memory capacity, which is the total amount of physical memory of the node; used memory, which is the total amount of memory occupied by all processes and pods; and remaining available memory, which is the idle memory that can be allocated.

[0061] The memory resource status can be obtained using the Metrics Server of Kubernetes or from the / proc / meminfo file.

[0062] The scheduling optimization module 20 is configured to receive the task and the task characteristics sent by the task characteristics analysis module 103, determine the scheduling scheme of the task according to the distributed optimization algorithm, the task characteristics, the node computing power model and the real-time resource state of each node, and send the scheduling scheme to the master node 101, wherein the scheduling scheme includes the working node 102, the CPU core number and the NPU number of the task.

[0063] The scheduling optimization module 20 is seamlessly connected with the K8s cluster 10 through an interface, and the K8s cluster 10 sends the task and the task characteristics to the scheduling optimization module 20 after receiving the task and obtaining the task characteristics. The dynamic computing power perception module 1021 uploads the monitored real-time resource state of the working node 102 to the scheduling optimization module 20 in real time.

[0064] The scheduling optimization module 20 integrates a plurality of distributed optimization algorithms and is also built-in with a node computing power model.

[0065] In an embodiment of the present application, the distributed optimization algorithm includes one of the following: genetic algorithm, ant colony optimization algorithm, particle swarm optimization algorithm, distributed reinforcement learning and heuristic scheduling algorithm.

[0066] The genetic algorithm (Genetic Algorithm, GA) can search for a global optimal scheduling scheme by simulating natural selection and genetic mechanisms. The ant colony optimization algorithm (Ant Colony Optimization, ACO) can simulate the foraging behavior of ants and find the optimal task allocation path through the cooperation between nodes. The particle swarm optimization algorithm (Particle Swarm Optimization, PSO) can simulate swarm intelligence and optimize resource allocation and scheduling strategies. The distributed reinforcement learning can use reinforcement learning methods to find the optimal scheduling scheme through trial and error and learning in a distributed environment. The heuristic scheduling algorithm can quickly generate an approximately optimal solution according to the task priority, computing demand and resource state.

[0067] In an embodiment of the present application, the node computing power model includes a resource utilization rate model, a resource performance model, an energy consumption model and a computing power balance model.

[0068] The node computing power model is a mathematical model describing the computing nodes and resources, which is used to evaluate the processing capacity and resource allocation state of the nodes.

[0069] The resource utilization model is used to describe the current utilization and availability of the CPU, NPU, memory, disk and other resources of the node. The resource performance model is used to predict the execution performance (such as computing delay and throughput) of different types of tasks on the node according to historical data. The energy consumption model is used to evaluate the energy consumption of different nodes when running tasks to achieve energy-saving scheduling. The computing power balancing model is used to ensure the fairness of resource allocation and avoid long-term overload or idling of some nodes.

[0070] After the scheduling optimization module 20 receives the tasks and task characteristics, the distributed optimization algorithm and the node computing power model are called to determine the scheduling scheme of the tasks according to the task characteristics and the real-time resource state of each node, and the scheduling scheme is sent to the master node 101. The scheduling scheme includes the working node 102 of the task, the CPU core number and the NPU number.

[0071] In an embodiment of the present application, the scheduling scheme can specifically include: task allocation, that is, which node each task is allocated to execute; resource allocation, that is, the specific resource number allocated to the task, such as CPU core number, NPU number, memory and storage; execution time, that is, the scheduled start time and completion time of the task on the target node; data path, that is, the reading path of the task input data and the storage location of the output result; priority adjustment, that is, real-time adjustment of the task priority according to the dynamic changes in the scheduling process.

[0072] The master node 101 is also used to schedule the tasks according to the scheduling scheme.

[0073] The master node 101 of the K8s cluster 10, that is, the control plane, is also used to receive the scheduling scheme sent by the scheduling optimization module 20 through the interface, and schedule the tasks according to the scheduling scheme. Wherein, scheduling the tasks refers to allocating the tasks to the corresponding working nodes 102, and allocating the CPU core number and NPU number required by the tasks in the working nodes 102.

[0074] In this way, the embodiments of the present application realize the dynamic scheduling of CPU and NPU by deploying the dynamic computing power perception module 1021 on each working node 102 of the K8s cluster 10, real-time monitoring the state of CPU and NPU in the working node 102, and combining with the distributed optimization algorithm and the node computing power model. Without changing the hardware, the support of NPU and the dynamic scheduling of tasks in the heterogeneous computing power environment to adapt to the resource demand and computing power demand are realized.

[0075] In an embodiment of the present application, a scheduling scheme of a task is determined according to a distributed optimization algorithm, a task characteristic, a node computing power model, and a real-time resource state of each node, including: obtaining the real-time resource state of each working node 102 based on each dynamic computing power perception module 1021; selecting a plurality of candidate working nodes from a plurality of working nodes 102 according to the computing resource requirement shown by the task characteristic and the real-time resource state of each working node 102; constructing a scheduling objective function and a constraint condition of the task; and determining the scheduling scheme based on the plurality of candidate working nodes, the distributed optimization algorithm, the scheduling objective function, the constraint condition, and the node computing power model.

[0076] In an embodiment of the present application, the scheduling objective function is:

[0077]

[0078] wherein UCPU and UNPU are the utilization rates of CPU and NPU, t is the time delay, a is a weight factor, a is used to control the balance between t and (UCPU, UNPU); UCPU and UNPU are obtained based on the resource utilization rate model in the node computing power model; and t is calculated based on the resource performance model in the node computing power model.

[0079] In an embodiment of the present application, the constraint condition includes at least one of the following: the utilized resource of the working node 102 does not exceed the node limit resource; the task time delay does not exceed the time delay requirement; and the working node 102 assigned by the task is not more than a set distance away from the data location of the task.

[0080] When determining the scheduling scheme, the scheduling optimization module 20 first needs to obtain the real-time resource state of each working node 102 monitored by the dynamic computing power perception module 1021, and then performs node matching according to the real-time resource state and the task characteristic to select a plurality of candidate working nodes. The matching can be matching the computing requirement (CPU, NPU, memory size) of the task with the computing resource requirement of the working node 102.

[0081] After the candidate working nodes are determined, the scheduling objective function and the constraint condition are constructed. The scheduling objective function is used to balance the resource utilization rate and the task delay. The constraint condition is a condition that needs to be met by the algorithm when generating the scheduling scheme, including the node resource limit, the task time delay constraint, and the data location condition. The node resource limit refers to that the utilized resource of the working node 102 does not exceed the node limit resource, the task time delay constraint refers to that the task time delay does not exceed the time delay requirement, and the data location condition refers to that the working node 102 assigned by the task is not more than a set distance away from the data location of the task.

[0082] In the process of algorithm calling, the node computing power model is also needed to be continuously evaluated to ensure that the performance of the final scheduling scheme is optimal. For example, the node computing power model can be used to calculate the execution time delay and throughput of the task on each candidate working node or calculate the energy consumption of the task on each candidate working node, and the working node 102 with low node time delay and energy consumption is preferentially selected to generate the final scheduling scheme.

[0083] In an embodiment of the present application, the system further comprises a task optimization migration module, which is used to monitor the performance of the task and migrate the task from the current node to other nodes when the performance does not meet the set condition. The performance includes multiple performance parameters of the task when it runs on its corresponding node.

[0084] In an embodiment of the present application, the performance parameters include task execution time delay, resource utilization and throughput, and the set condition is that the performance parameters are lower than the set performance parameter threshold.

[0085] In an embodiment of the present application, migrating the task from the current node to other nodes includes: obtaining the running state data of the task; determining the updated working node according to the running state data of the task, the distributed optimization algorithm, the task characteristics, the node computing power model and the real-time resources of other nodes; and transmitting the task and the task running state data to the updated working node.

[0086] In the process of task running, the task migration optimization module monitors its performance. When the performance of the task does not meet the set condition, the task migration mechanism is triggered to transfer the task to a more suitable computing power unit. The performance of the task includes multiple performance parameters, and the set condition is that the performance parameters are lower than the set performance parameter threshold. The multiple performance parameters can include task execution time delay, resource utilization and throughput. The set condition not being met can include that the actual time delay of the task is higher than the expected time delay, the task consumes too much resource, the throughput is lower than the expected value, etc.

[0087] Whether to perform task migration is mainly determined by whether the task migration mechanism is triggered. The task migration mechanism can be triggered when the performance does not meet the set condition, and can also be triggered when the resources of the node are overloaded or idle.

[0088] In the process of performing task migration, since the task has already been executed for a part, the real-time running state data of the task needs to be obtained, and the working node 102 of the task migration, i.e. the updated working node, is determined according to the running state data of the task. The process of determining the updated working node is similar to the process of determining the task scheduling scheme for the first time. The task characteristics in the first determined scheduling scheme are replaced by the running state data of the task, i.e. the updated working node is determined according to the running state data of the task, the distributed optimization algorithm, the task characteristics, the node computing power model and the real-time resources of other nodes.

[0089] When migrating the task, the running state data of the task is also migrated, so that the updated worker node can restart or continue the task according to the running state data, that is, restart the task or continue to execute the task from the breakpoint on the updated worker node.

[0090] In an embodiment of the present application, the running state data of the task can include intermediate calculation results, data of completed parts, etc.

[0091] In order to further understand the scheme of the present application, a specific application example is described below.

[0092] This specific application example of the present application is used to describe the modules of the CPU and NPU computing power distribution scheduling system based on the K8s cluster 10 and the cooperative working process of the modules, specifically, the modules of the specific application example of the present application and the cooperative working process of the modules include:

[0093] 1) Dynamic computing power perception module

[0094] Deployed on the K8s cluster worker node, periodically collect the resource state of CPU and NPU, including utilization, memory occupation, task queue length and network bandwidth, etc. The data is uploaded to the corresponding module through API.

[0095] 2) Task characteristic analysis module

[0096] Running on the control plane of the K8s cluster, analyzing the computing demand, data throughput and priority of the task, combining with the historical task record, evaluating the matching degree of the task and the computing power resource.

[0097] 3) Scheduling optimization module

[0098] Extended in the native K8s scheduler, integrated with multiple distributed optimization algorithms, dynamically adjusting the task allocation strategy. This module combines real-time computing power perception and task characteristics to generate the optimal allocation scheme and submit it to the control plane.

[0099] 4) Task migration optimization module

[0100] By monitoring the task running performance, detecting the adaptation of the task on CPU and NPU. When the task does not match the current computing power unit, the system automatically triggers migration to reduce performance loss.

[0101] In this way, the embodiments of the present application realize efficient scheduling optimization of CPU and NPU on the Kubernetes platform through the above improvements, which significantly improves the resource utilization and task performance in the heterogeneous computing power environment.

[0102] It should be noted that the embodiments of the present application are based on the description of the specific application examples of the CPU and NPU computing power distribution scheduling system based on the K8s cluster, which is similar to the description of the above-mentioned CPU and NPU computing power distribution scheduling system based on the K8s cluster, and has similar beneficial effects as the CPU and NPU computing power distribution scheduling system based on the K8s cluster. Therefore, it is not repeated. For the technical details not described in the specific application examples of the CPU and NPU computing power distribution scheduling system based on the K8s cluster provided by the embodiments of the present application, it can be understood according to the description of Figure 1

[0103] It should be understood that the steps shown above can be reordered, added or deleted. For example, each step described in the present disclosure can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions of the present disclosure can be achieved, and the present disclosure is not limited herein.

[0104] In addition, the terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0105] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.​

Claims

1. A K8s-based CPU and NPU computing power distribution scheduling system, characterized in that, The system comprises a K8s cluster and a scheduling optimization module, the K8s cluster comprises a master node, a plurality of worker nodes and a task characteristic analysis module, each worker node comprises at least one CPU and at least one NPU and is deployed with a dynamic computing power perception module, and each dynamic computing power perception module is connected with the scheduling optimization module; The master node is configured to receive a task submitted by a user; The task characteristic analysis module is configured to obtain a task characteristic of the task and send the task and the task characteristic to the scheduling optimization module; The dynamic computing power perception module is configured to monitor a real-time resource state of a corresponding worker node, wherein the real-time resource state comprises a CPU resource state, an NPU resource state and a memory resource state; The scheduling optimization module is configured to receive the task and the task characteristic sent by the task characteristic analysis module, determine a scheduling scheme of the task according to a distributed optimization algorithm, the task characteristic, a node computing power model and a real-time resource state of each node, and send the scheduling scheme to the master node, wherein the scheduling scheme comprises a worker node, a CPU core number and an NPU number of the task; The master node is further configured to schedule the task according to the scheduling scheme; The method comprises the following steps: obtaining the real-time resource state of each worker node based on each dynamic computing power perception module; selecting a plurality of candidate worker nodes from the plurality of worker nodes according to a computing resource requirement shown by the task characteristic and the real-time resource state of each worker node; constructing a scheduling objective function and a constraint condition of the task; determining the scheduling scheme based on the plurality of candidate worker nodes, the distributed optimization algorithm, the scheduling objective function, the constraint condition and the node computing power model; The scheduling objective function is as follows: wherein UCPU and UNPU are utilization rates of the CPU and the NPU, t is a time delay, and a is a weight factor, the a being used to control the balance between t and (UCPU, UNPU); UCPU and UNPU are predicted based on a resource utilization rate model in the node computing power model; t is calculated based on a resource performance model in the node computing power model.

2. The system of claim 1, wherein, The task characteristic comprises a task type, a computing resource requirement, a time delay requirement, a priority, a dependency relationship and a data location; The computing resource requirement comprises a requirement size of the CPU, the NPU, the memory and the storage.

3. The system of claim 1, wherein, The node computing power model comprises a resource utilization rate model, a resource performance model, an energy consumption model and a computing power balance model.

4. The system of claim 1, wherein, The constraint condition comprises at least one of the following: the utilized resource of the worker node does not exceed a node limit resource; the task time delay does not exceed the time delay requirement; a distance between the task allocation worker node and the data location of the task does not exceed a set distance.

5. The system of claim 1, wherein, The system further comprises a task optimization migration module, the task optimization migration module is configured to monitor a performance of the task, and migrate the task from a current node to another node when the performance does not meet a set condition, wherein the performance comprises a plurality of performance parameters of the task when the task runs on the corresponding node.

6. The system of claim 5, wherein, The performance parameters include task execution delay, resource utilization, and throughput. The setting condition is that the performance parameter is lower than a set performance parameter threshold.

7. The system of claim 5, wherein, The migrating the task from the current node to another node comprises: obtaining running state data of the task; determining an updated worker node according to the running state data of the task, the distributed optimization algorithm, the task characteristics, the node computing power model, and real-time resource states of other nodes; transmitting the task and the running state data to the updated worker node.

8. The system of claim 1, wherein, The distributed optimization algorithm comprises one of a genetic algorithm, an ant colony optimization algorithm, a particle swarm optimization algorithm, distributed reinforcement learning, and a heuristic scheduling algorithm.

Images