GPU (Graphics Processing Unit) and NPU (Network Processing Unit) resource scheduling method for high-performance computing

Abstract

The invention belongs to the technical field of artificial intelligence, and particularly relates to a GPU (Graphics Processing Unit) and NPU (Network Processing Unit) resource scheduling method for high-performance computing, which comprises the following steps of: waiting for a scheduling optimization module to call acquired resource condition data of GPU and NPU hardware accelerators; the task analysis module analyzes a to-be-executed AI calculation task, obtains task feature data and transmits the task feature data to the scheduling optimization module; the scheduling optimization module calculates an optimal task allocation scheme and transmits the optimal task allocation scheme to the dynamic scheduling module; the dynamic scheduling module monitors task execution conditions and resource changes in real time, dynamically adjusts a task allocation scheme according to feedback information, and feeds back the task allocation scheme to the scheduling optimization module; meanwhile, the dynamic scheduling module feeds back the change information of the resource condition data and the task execution condition to the resource monitoring module and the task analysis module respectively; according to the method, dynamic scheduling is carried out on heterogeneous hardware accelerators such as the GPU and the NPU, respective advantages are fully exerted, and the overall calculation performance is improved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically is a GPU and NPU resource scheduling method for high-performance computing. Background Art

[0002] In the field of artificial intelligence technology, in view of the characteristics of AI computing tasks, such as computing complexity, input data scale, real-time requirements, etc., the resource scheduling method of GPU and NPU has become a key factor affecting the computing performance and resource utilization of AI computing tasks. The main methods and main limitations of existing technologies are as follows:

[0003] Mainly static scheduling: Traditional resource scheduling methods use relatively static scheduling strategies, that is, before task allocation, they determine the allocation method of computing tasks on GPU and NPU based on some preset rules or empirical values, which lacks sufficient consideration of real-time hardware resource conditions and dynamic changes of tasks. For example, during the execution of a task, if there is a sudden load change in hardware resources or the real-time requirements of the task itself change, it is difficult for the static scheduling method to make timely and effective adjustments, resulting in resource waste or reduced computing performance.

[0004] Insufficient analysis of task characteristics: Some existing technologies do not fully and deeply analyze the characteristics of tasks when assigning computing tasks. For example, they only consider the computational complexity and ignore other important factors such as the input data size or real-time requirements. This makes it impossible to accurately match hardware resources in task assignment, and it is impossible for the GPU and NPU to process tasks in the best state, thus limiting the improvement of computing performance. This makes task assignment unreasonable, and it is impossible to give full play to the advantages of GPU and NPU, which in turn affects the overall computing efficiency.

[0005] The optimization algorithm is single or not combined with hardware features: Although some resource scheduling solutions use optimization algorithms, they are limited to a certain algorithm and lack the comprehensive application of multiple algorithms to adapt to different scenarios. In addition, when using the algorithm, the specific characteristics of GPU and NPU are not fully combined, resulting in the algorithm being unable to achieve the best optimization effect in actual applications and unable to fully tap the potential of hardware resources.

[0006] Lack of dynamic adaptability: Existing technologies often have difficulty responding quickly and effectively to emergencies during task execution, such as hardware failures, task priority adjustments, etc. In these situations, static scheduling schemes cannot reallocate tasks in a timely manner, resulting in task interruptions or delays, and cannot meet the real-time and stability requirements of high-performance computing.

[0007] Therefore, it is necessary to provide a GPU and NPU resource scheduling method that can be oriented to high-performance computing to overcome the limitations of the above-mentioned existing technologies. Summary of the invention

[0008] The purpose of this invention is to provide a GPU and NPU resource scheduling method for high-performance computing. According to the characteristics of AI computing tasks, such as computational complexity, input data scale, real-time requirements, etc., a multi-objective optimization algorithm is used to dynamically schedule GPU and NPU resources to meet the needs of high-performance AI computing.

[0009] The technical solution adopted by the present invention to achieve the above-mentioned purpose is: a GPU and NPU resource scheduling method for high-performance computing, comprising the following steps:

[0010] 1) Monitor the GPU and NPU hardware accelerators in real time, and store the collected resource status data of the GPU and NPU hardware accelerators for the scheduling optimization module to call;

[0011] 2) The task analysis module analyzes the AI ​​computing tasks to be executed, obtains the task feature data, and transmits the task feature data to the scheduling optimization module;

[0012] 3) The scheduling optimization module uses PSO and GA algorithms to calculate the optimal task allocation plan based on resource status data and task feature data, and passes it to the dynamic scheduling module;

[0013] 4) Setting the parameters of the dynamic scheduling module, the dynamic scheduling module monitors the task execution status and resource changes in real time, dynamically adjusts the task allocation plan according to the feedback information, and feeds back to the scheduling optimization module; at the same time, the dynamic scheduling module feeds back the change information of resource status data and task execution status to the resource monitoring module and the task analysis module respectively;

[0014] 5) Repeat step 4) until the dynamic scheduling module sends a new task allocation plan to the resource acquisition module, and the resource acquisition module notifies the task analysis module of the task allocation change; repeat steps 1) to 4) until the stop condition is met.

[0015] The resource status data includes key indicators of computing power, memory usage, temperature, and power consumption.

[0016] The task characteristic data includes: computational complexity, memory access mode, performance analysis and optimization suggestions, input data scale, data distribution characteristics, and real-time requirements.

[0017] The task analysis module analyzes the AI ​​computing task to be executed and obtains task feature data, specifically:

[0018] 2-1) Use SciTools to perform static analysis on the source code of AI computing tasks to extract computational complexity and memory access patterns;

[0019] 2-2) Run AI computing tasks on small-scale input data through NVIDIA Nsight software, conduct dynamic operation analysis, and obtain performance analysis and optimization suggestions;

[0020] 2-3) Use Pandas and Numpy data analysis libraries to analyze the input data of the task and determine the input data scale and data distribution characteristics;

[0021] 2-4) Use RealTime-UML to perform modeling analysis, including the business requirements of the task, evaluate its real-time requirements for computing delays, and determine real-time constraints.

[0022] The scheduling optimization module uses PSO to calculate the optimal task allocation plan based on resource status data and task feature data, including the following steps:

[0023] 3-1) Initialization:

[0024] N particles are randomly generated, each particle represents a task allocation scheme; the position of each particle indicates the distribution of tasks on the GPU and NPU, and the speed indicates the moving direction and speed of the particle in the search space; at the same time, the historical optimal position of each particle and the historical optimal position of the group are initialized;

[0025] 3-2) Fitness evaluation:

[0026] According to the resource status data and task feature data provided by the resource monitoring module and the task analysis module, the fitness value of each particle is calculated; wherein the fitness function is designed according to the resource utilization efficiency and computing performance;

[0027] 3-3) Speed ​​update:

[0028] Update the velocity of each particle according to the following formula:

[0029] v[i]=w·v[i]+c1·rand()·(pbest[i]-x[i])+c2·rand()·(gbest-x[i])

[0030] Where v[i] represents the velocity of particle i, w is the inertia weight, rand() is a random number generation function, c1 and c2 are learning factors, pbest[i] is the historical optimal position of particle i, gbest is the historical optimal position of the group, and x[i] is the position of particle i;

[0031] 3-4) Location Update:

[0032] According to the updated velocity, update the position of each particle, that is:

[0033] x[i]=x[i]+v[i]

[0034] 3-5) Optimal position update:

[0035] If the current position of a particle is better than its historical optimal position, then update the historical optimal position of the particle; if the historical optimal position of a particle in the group is better than the historical optimal position of the group, then update the historical optimal position of the group;

[0036] 3-6) Repeat steps 3-2) to 3-5) until the maximum number of iterations is reached or the fitness value reaches the set accuracy requirement.

[0037] The scheduling optimization module uses the GA algorithm to calculate the optimal task allocation plan based on resource status data and task feature data, including the following steps:

[0038] 4-1) Initialization:

[0039] Randomly generate N individuals, each of which represents a task allocation scheme;

[0040] 4-2) Fitness evaluation:

[0041] Calculate the fitness value of each individual based on the data provided by the resource monitoring module and the task analysis module;

[0042] 4-3) Select an operation:

[0043] According to the fitness value of the individual, a part of excellent individuals are selected as parent individuals to generate the next generation of individuals; wherein the selection method adopts roulette selection or tournament selection;

[0044] 4-4) Crossover operation:

[0045] Perform a crossover operation on the selected parent individuals to generate new offspring individuals; the crossover operation adopts a single-point crossover or a two-point crossover method;

[0046] 4-5) Mutation operation:

[0047] Perform mutation operation on the newly generated offspring individuals to increase the diversity of individuals; wherein the mutation operation adopts the bit mutation or real number mutation method;

[0048] 4-6) Replacement operation:

[0049] The newly generated offspring individuals replace the individuals with poor fitness in the population to form a new population.

[0050] 4-7) Repeat steps 4-2 to 4-6) until the maximum number of iterations is reached or the fitness value reaches the set accuracy requirement.

[0051] When the scheduling optimization module adopts the PSO and GA algorithms, combined with the characteristics of the hardware accelerator, the following steps are performed:

[0052] In the PSO algorithm, the speed and position update formulas of particles are adjusted according to the parallel computing capability of the GPU and the neural network acceleration capability of the NPU, including: increasing the preference for GPU parallel computing tasks so that particles are more inclined to assign computing tasks to the GPU;

[0053] In the GA algorithm, the crossover and mutation operations are adjusted according to the characteristics of the hardware accelerator, so that the newly generated individuals are more suitable for executing tasks on the hardware accelerator;

[0054] The performance indicators of the hardware accelerator are introduced as part of the fitness function, including: the computing speed of the GPU and the neural network inference speed of the NPU, to ensure that the optimal task allocation solution searched by the algorithm can give full play to the performance of the hardware accelerator.

[0055] The setting of dynamic scheduling module parameters includes: setting monitoring frequency, setting resource change threshold and task execution progress threshold, using resource monitoring module information and setting triggers;

[0056] a. Set the monitoring frequency: This determines the time interval for the system to monitor task execution and resource changes. The higher the frequency, the more timely the information can be obtained.

[0057] b. Set resource change threshold: When resource changes exceed this threshold, trigger the task reallocation;

[0058] c. Task execution progress threshold: used to determine whether a task is close to completion or an exception occurs. For example, a progress percentage threshold is set. When the task progress exceeds a certain value, consider adjusting the allocation of subsequent tasks;

[0059] d. Information about using resource monitoring module:

[0060] According to the information of the real-time resource monitoring module and the progress of the tasks being executed, the execution status of the tasks can be obtained through the task management system or the task monitoring interface;

[0061] e. Set triggers: When resource changes exceed the preset threshold or the task execution progress reaches the set value, the dynamic adjustment process is triggered.

[0062] A GPU and NPU resource scheduling system for high-performance computing:

[0063] The resource monitoring module is used to monitor the GPU and NPU hardware accelerators in real time and store the collected resource status data of the GPU and NPU hardware accelerators for the scheduling optimization module to call;

[0064] The task analysis module is used to analyze the AI ​​computing tasks to be executed, obtain the task feature data, and transmit the task feature data to the scheduling optimization module;

[0065] The scheduling optimization module is used to obtain the optimal task allocation plan based on resource status data and task feature data using PSO and GA algorithms, and pass it to the dynamic scheduling module;

[0066] The dynamic scheduling module is used to set the dynamic scheduling module parameters. The dynamic scheduling module monitors the task execution status and resource changes in real time, dynamically adjusts the task allocation plan according to the feedback information, and feeds back to the scheduling optimization module; at the same time, the dynamic scheduling module feeds back the change information of resource status data and task execution status to the resource monitoring module and task analysis module respectively; and sends the new task allocation plan to the resource acquisition module.

[0067] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, a GPU and NPU resource scheduling method for high-performance computing is implemented.

[0068] The present invention has the following beneficial effects and advantages:

[0069] 1. The present invention optimizes computing performance and resource utilization efficiency and improves the adaptability and robustness of the system through technical means such as resource monitoring, task analysis, scheduling optimization and dynamic scheduling.

[0070] 2. The present invention adopts a method of real-time monitoring of task execution and resource changes, and dynamically adjusts the task allocation scheme according to feedback information, which can better meet the needs of high-performance computing. Compared with fixed resource allocation strategies, dynamic scheduling can improve resource utilization efficiency.

[0071] 3. The present invention adopts multi-objective optimization algorithms such as particle swarm optimization (PSO) and genetic algorithm (GA), combined with the characteristics of hardware accelerators, which can better balance the optimization goals of computing performance and resource utilization efficiency.

[0072] 4. The present invention dynamically schedules heterogeneous hardware accelerators such as GPU and NPU to give full play to their respective advantages and improve the overall computing performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 System framework diagram of the resource scheduling system of the present invention;

[0074] Figure 2 A method flow chart of the resource scheduling method of the present invention. DETAILED DESCRIPTION

[0075] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0076] The present invention proposes an innovative solution to the GPU and NPU resource scheduling problem in high-performance computing scenarios, demonstrating technical advantages in dynamic scheduling, multi-objective optimization, and heterogeneous resource coordination.

[0077] The purpose of this invention is to dynamically optimize the scheduling and allocation of GPU and NPU resources in high-performance computing scenarios to improve computing performance and resource utilization efficiency. It has broad application prospects in the fields of quality control, sales forecasting, and product development in the tobacco industry.

[0078] like Figure 2 As shown, it is a method flow chart of the resource scheduling method of the present invention. The present invention is a GPU and NPU resource scheduling method for high-performance computing, comprising the following steps:

[0079] The resource monitoring module monitors the resource status of the GPU and NPU in real time and passes the data to the scheduling optimization module.

[0080] The task analysis module analyzes the characteristics of the AI ​​computing tasks to be executed and also passes the data to the scheduling optimization module.

[0081] The scheduling optimization module uses PSO and GA algorithms to calculate the optimal task allocation plan based on resource conditions and task characteristics, and pass it to the dynamic scheduling mechanism.

[0082] The dynamic scheduling mechanism monitors task execution and resource changes in real time, and dynamically adjusts task allocation plans based on feedback information.

[0083] The dynamic scheduling module will continuously interact with the scheduling optimization module, resource monitoring module and task analysis module to ensure that high-performance computing needs are met.

[0084] The specific steps include:

[0085] 1) By setting up a resource monitoring module in the system, the resource status data of hardware accelerators such as GPU and NPU are collected in real time, including key indicators such as computing power (FLOPS), memory usage, temperature, and power consumption. This module solves the problem of lack of real-time and accurate understanding of hardware resource status in the existing technology, and provides a data basis for subsequent task allocation.

[0086] The specific steps are as follows: Use the API provided by GPU and NPU manufacturers, such as the nvidia-smi management interface provided by NVIDIA. For NPU manufacturers, such as Huawei HiSilicon and Cambrian, call the corresponding toolkit to obtain the operating status data of the hardware accelerator. Store the collected data in the memory database for subsequent task analysis and scheduling.

[0087] 2) Conduct in-depth analysis of the AI ​​computing tasks to be performed, including characteristics such as computational complexity, input data size, and real-time requirements.

[0088] This step helps to better understand the task requirements and provide a basis for the rational allocation of resources. Use advanced static code analysis, dynamic operation analysis, input data analysis and other technical means to comprehensively and accurately evaluate the various characteristics of the task. Use SciTools to statically analyze the source code of the AI ​​computing task and extract key indicators such as computational complexity and memory access patterns. Use NVIDIA Nsight to run the AI ​​computing task on small-scale input data for dynamic operation analysis. Use Pandas and Numpy data analysis libraries to analyze the input data of the task and determine the data scale, distribution characteristics, etc. to complete the input data analysis. Use RealTime-UML for modeling analysis, including the business requirements of the task, evaluate its requirements for computing latency, and determine real-time constraints. Determine the task's requirements for resources such as CPU, GPU, and memory.

[0089] Corresponding to the above-mentioned task characteristics, actual measures of computational complexity are used, such as the number of operations, degree of parallelism, etc.

[0090] Detailed information about memory access patterns, such as data locality, coherence of memory accesses, etc.

[0091] The impact of input data size on performance, inferring the performance of large-scale datasets through analysis of small-scale datasets.

[0092] 3) Scheduling Optimization:

[0093] Multi-objective optimization algorithms, particle swarm optimization (PSO) and genetic algorithm (GA), combined with the characteristics of hardware accelerators, dynamically adjust the distribution of computing tasks on GPU and NPU. This is one of the core technical means of the present invention, which aims to solve the problems of unreasonable resource allocation and low efficiency in the prior art. Specifically, PSO and GA algorithms search for the optimal task allocation scheme based on the data provided by the resource monitoring module and the task analysis module. These algorithms continuously optimize task allocation by simulating biological evolution and group behavior in nature to improve resource utilization efficiency and computing performance. Combined with the characteristics of hardware accelerators, such as the parallel computing capability of GPU and the neural network acceleration capability of NPU, the performance of the algorithm is further optimized.

[0094] In order to better understand and implement particle swarm optimization and genetic algorithm in the scheduling optimization step, and combine the characteristics of hardware accelerators to improve the efficiency and performance of resource scheduling, the characteristics of hardware accelerators are taken into consideration in each step of PSO and GA algorithms for adjustment and optimization.

[0095] (1) Particle Swarm Optimization (PSO) algorithm

[0096] Algorithm Overview:

[0097] Particle swarm optimization is an optimization algorithm based on swarm intelligence. It simulates the foraging behavior of bird flocks to find the optimal solution. In the resource scheduling problem, each particle represents a task allocation scheme, and the optimal task allocation scheme is searched by continuously adjusting the position and speed of the particles.

[0098] parameter:

[0099] Number of particles (N): determines the coverage of the search space in the algorithm. Generally speaking, the more particles there are, the wider the search space is, but the computational cost is also higher.

[0100] Inertia weight (w): controls the inertia of particles and affects the speed change of particles during the search process. A larger inertia weight is beneficial to global search, while a smaller inertia weight is beneficial to local search.

[0101] Learning factors (c1 and c2): represent the speed at which particles learn from their own historical optimal position and the group's historical optimal position, respectively. A larger learning factor speeds up the convergence of the algorithm, but may cause the algorithm to fall into a local optimum.

[0102] Maximum speed (v_max): limits the speed of particles to prevent them from oscillating excessively during the search process.

[0103] Processing steps:

[0104] 1-1) Initialization:

[0105] N particles are randomly generated, each particle represents a task allocation scheme; the position of each particle indicates the distribution of tasks on the GPU and NPU, and the speed indicates the moving direction and speed of the particle in the search space; at the same time, the historical optimal position of each particle and the historical optimal position of the group are initialized;

[0106] 1-2) Fitness evaluation:

[0107] According to the resource status data and task feature data provided by the resource monitoring module and the task analysis module, the fitness value of each particle is calculated; wherein the fitness function is designed according to the resource utilization efficiency and computing performance;

[0108] 1-3) Speed ​​update:

[0109] Update the velocity of each particle according to the following formula:

[0110] v[i]=w·v[i]+c1·rand()·(pbest[i]-x[i])+c2·rand()·(gbest-x[i])

[0111] Where v[i] represents the velocity of particle i, w is the inertia weight, rand() is a random number generation function, c1 and c2 are learning factors, pbest[i] is the historical optimal position of particle i, gbest is the historical optimal position of the group, and x[i] is the position of particle i;

[0112] 1-4) Location Update:

[0113] According to the updated velocity, update the position of each particle, that is:

[0114] x[i]=x[i]+v[i]

[0115] 1-5) Optimal position update:

[0116] If the current position of a particle is better than its historical optimal position, then update the historical optimal position of the particle; if the historical optimal position of a particle in the group is better than the historical optimal position of the group, then update the historical optimal position of the group;

[0117] 1-6) Repeat steps 1-2) to 1-5) until the maximum number of iterations is reached or the fitness value reaches the set accuracy requirement.

[0118] (2) Genetic Algorithm (GA)

[0119] Algorithm Overview:

[0120] Genetic algorithm is an optimization algorithm based on the principle of biological evolution. It searches for the optimal solution by simulating natural selection, crossover and mutation. In the resource scheduling problem, each individual represents a task allocation scheme, and the optimal task allocation scheme is searched by continuously evolving individuals.

[0121] parameter:

[0122] Population size (N): determines the coverage of the search space in the algorithm. Generally speaking, the larger the population size, the wider the search space coverage, but the higher the computational cost.

[0123] Crossover probability (Pc): controls the probability of crossover operations between individuals. A larger crossover probability increases the algorithm's search ability, but may destroy excellent individuals.

[0124] Mutation probability (Pm): controls the probability of an individual performing a mutation operation. A larger mutation probability increases the diversity of the algorithm, but may cause algorithm instability.

[0125] Maximum number of iterations (T): limits the running time of the algorithm to prevent the algorithm from falling into an infinite loop.

[0126] Processing steps:

[0127] 2-1) Initialization:

[0128] Randomly generate N individuals, each of which represents a task allocation scheme;

[0129] 2-2) Fitness evaluation:

[0130] Calculate the fitness value of each individual based on the data provided by the resource monitoring module and the task analysis module;

[0131] 2-3) Select an operation:

[0132] According to the fitness value of the individual, a part of excellent individuals are selected as parent individuals to generate the next generation of individuals; wherein the selection method adopts roulette selection or tournament selection;

[0133] 2-4) Crossover operation:

[0134] Perform a crossover operation on the selected parent individuals to generate new offspring individuals; the crossover operation adopts a single-point crossover or a two-point crossover method;

[0135] 2-5) Mutation operation:

[0136] Perform mutation operation on the newly generated offspring individuals to increase the diversity of individuals; wherein the mutation operation adopts bit mutation or real number mutation method;

[0137] 2-6) Replacement operation:

[0138] The newly generated offspring individuals replace the individuals with poor fitness in the population to form a new population.

[0139] 2-7) Repeat steps 2-2 to 2-6) until the maximum number of iterations is reached or the fitness value reaches the set accuracy requirement.

[0140] (3) Combined with hardware accelerator features

[0141] Algorithm adjustments:

[0142] In the PSO algorithm, the speed and position update formulas of the particles are adjusted according to the parallel computing capabilities of the GPU and the neural network acceleration capabilities of the NPU. For example, the preference for GPU parallel computing tasks is increased so that particles are more inclined to assign computing tasks to the GPU. In the GA algorithm, the crossover and mutation operations are adjusted according to the characteristics of the hardware accelerator so that the newly generated individuals are more suitable for executing tasks on the hardware accelerator.

[0143] The performance indicators of hardware accelerators are introduced as part of the fitness function, such as the computing speed of GPU, the neural network reasoning speed of NPU, etc. This ensures that the optimal task allocation solution searched by the algorithm can give full play to the performance advantages of hardware accelerators.

[0144] 4) Dynamic Scheduling:

[0145] Monitor task execution and resource changes in real time, and dynamically adjust task allocation plans based on feedback information. This mechanism ensures that the system can adapt to the ever-changing computing environment and task requirements, and improves the stability and adaptability of the system. Use real-time monitoring tools and technologies to obtain information on task execution and resource changes in a timely manner. For example, use performance monitoring tools to track task execution progress and resource usage, or set triggers to trigger task reallocation when resource changes exceed a certain threshold.

[0146] The core of dynamic scheduling is to adjust the task allocation plan in time according to the real-time task execution status and resource changes to achieve efficient resource utilization and high system performance. It mainly includes using the information of the resource monitoring module, setting triggers, and adopting specific adjustment algorithms.

[0147] 5) Dynamic loop

[0148] Analyze the obtained task execution status and resource change information. For example, if the resource utilization of a GPU or NPU is too high, it means that the task load on the device is too heavy, and it is necessary to consider transferring some tasks to other devices. Combined with the task characteristics obtained in the task analysis module, such as computational complexity, input data size, and real-time requirements, evaluate the rationality of the current task allocation plan.

[0149] According to the analysis results, task characteristics and current resource conditions, the most suitable device is selected for allocation. The multi-objective optimization algorithm is used again to find a new optimal task allocation scheme. However, due to the need for real-time adjustment, the algorithm needs to be simplified or an approximate algorithm needs to be adopted to reduce the computing time. The adjusted task allocation scheme is implemented and tasks are reallocated to different GPUs and NPUs. Task execution and resource changes are continuously monitored to form a dynamic feedback loop to continuously optimize task allocation.

[0150] like Figure 1 As shown in FIG. 1 , it is a system framework diagram of the present invention. The present invention provides a GPU and NPU resource scheduling system for high-performance computing, including:

[0151] The resource monitoring module is used to monitor the GPU and NPU hardware accelerators in real time and store the collected resource status data of the GPU and NPU hardware accelerators for the scheduling optimization module to call;

[0152] The task analysis module is used to analyze the AI ​​computing tasks to be executed, obtain the task feature data, and transmit the task feature data to the scheduling optimization module;

[0153] The scheduling optimization module is used to obtain the optimal task allocation plan based on resource status data and task feature data using PSO and GA algorithms, and pass it to the dynamic scheduling module;

[0154] The dynamic scheduling module is used to set the dynamic scheduling module parameters. The dynamic scheduling module monitors the task execution status and resource changes in real time, dynamically adjusts the task allocation plan according to the feedback information, and feeds back to the scheduling optimization module; at the same time, the dynamic scheduling module feeds back the change information of resource status data and task execution status to the resource monitoring module and task analysis module respectively; and sends the new task allocation plan to the resource acquisition module.

[0155] It also sets the parameters of the dynamic scheduling module, including: setting the monitoring frequency, setting the resource change threshold and task execution progress threshold, using the information of the resource monitoring module, and setting the trigger;

[0156] Monitoring frequency: determines the time interval at which the system monitors task execution and resource changes. A higher frequency obtains information more timely, but also increases system overhead.

[0157] Resource change threshold: When the resource change exceeds this threshold, the reallocation task is triggered. The threshold setting needs to be adjusted according to the specific application scenario and system performance requirements.

[0158] Task execution progress threshold: used to determine whether a task is close to completion or an exception occurs. For example, a progress percentage threshold is set. When the task progress exceeds a certain value, consider adjusting the allocation of subsequent tasks.

[0159] Based on real-time monitoring information, including computing power utilization, memory usage, temperature, etc. and tracking the progress of the tasks being executed, the execution status of the tasks is obtained through the task management system or a specific task monitoring interface. Set triggers to trigger the dynamic adjustment process when the resource change exceeds the preset threshold or the task execution progress reaches a specific value.

[0160] In summary, the present invention has the following characteristics:

[0161] a. Application of multi-objective optimization algorithms: Multi-objective optimization algorithms such as particle swarm optimization (PSO) and genetic algorithm (GA) are introduced to dynamically adjust the allocation of GPU and NPU resources. The algorithm can search for the optimal task allocation solution in a complex computing environment, improve resource utilization efficiency and computing performance. Compared with traditional resource allocation methods, it has higher flexibility and adaptability.

[0162] b. Combine the characteristics of hardware accelerators: Take full account of the characteristics of hardware accelerators such as GPU and NPU and integrate them into the resource scheduling process. For example, based on the parallel computing capability of GPU and the neural network acceleration capability of NPU, optimize the task allocation plan, give full play to the advantages of hardware, and improve computing performance.

[0163] c. Real-time monitoring and dynamic adjustment: A real-time monitoring mechanism is established to obtain information on task execution and resource changes in a timely manner. Based on this feedback information, the task allocation plan is dynamically adjusted to ensure that high-performance computing needs are met. This dynamic adjustment capability enables the system to adapt to the ever-changing computing environment and improves the stability and reliability of the system.

[0164] d. Comprehensive task analysis: Comprehensively analyze the AI ​​computing tasks to be executed, including characteristics such as computational complexity, input data size, and real-time requirements. This helps to understand the task requirements more accurately and provide a basis for reasonable resource allocation. Compared with traditional resource scheduling methods, it pays more attention to the characteristics of the task and improves the accuracy and efficiency of resource allocation.

[0165] e. Efficient resource monitoring: Real-time monitoring of the resource status of hardware accelerators such as GPU and NPU, including indicators such as computing power, memory usage, and temperature, provides an accurate data basis for resource scheduling. This enables the system to better understand the usage of hardware resources, adjust task allocation in a timely manner, and improve resource utilization efficiency.

[0166] Those skilled in the art will appreciate that the above are only preferred embodiments of the present invention, and the various embodiments of the present disclosure and / or the features described in the claims may be combined or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. It is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art may still modify the technical solutions described in the aforementioned embodiments, or perform equivalent substitutions on some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

[0167] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A GPU and NPU resource scheduling method for high performance computing, characterized in that: The following steps are involved: 1) Monitor the GPU and NPU hardware accelerators in real time, and store the collected resource status data of the GPU and NPU hardware accelerators for the scheduling optimization module to call; 2) The task analysis module analyzes the AI ​​computing tasks to be executed, obtains the task feature data, and transmits the task feature data to the scheduling optimization module; 3) The scheduling optimization module uses PSO and GA algorithms to calculate the optimal task allocation plan based on resource status data and task feature data, and passes it to the dynamic scheduling module; 4) Setting the parameters of the dynamic scheduling module, the dynamic scheduling module monitors the task execution status and resource changes in real time, dynamically adjusts the task allocation plan according to the feedback information, and feeds back to the scheduling optimization module; at the same time, the dynamic scheduling module feeds back the change information of resource status data and task execution status to the resource monitoring module and the task analysis module respectively; 5) Repeat step 4) until the dynamic scheduling module sends a new task allocation plan to the resource acquisition module, and the resource acquisition module notifies the task analysis module of the task allocation change; repeat steps 1) to 4) until the stop condition is met.

2. A GPU and NPU resource scheduling method for high performance computing according to claim 1, characterized in that: The resource status data includes key indicators of computing power, memory usage, temperature, and power consumption.

3. The method for scheduling GPU and NPU resources for high performance computing according to claim 1, characterized in that: The task characteristic data includes: computational complexity, memory access mode, performance analysis and optimization suggestions, input data scale, data distribution characteristics, and real-time requirements.

4. The method for scheduling GPU and NPU resources for high performance computing according to claim 1, characterized in that: The task analysis module analyzes the AI ​​computing task to be executed and obtains task feature data, specifically: 2-1) Use SciTools to perform static analysis on the source code of AI computing tasks to extract computational complexity and memory access patterns; 2-2) Run AI computing tasks on small-scale input data through NVIDIA Nsight software, conduct dynamic operation analysis, and obtain performance analysis and optimization suggestions; 2-3) Use Pandas and Numpy data analysis libraries to analyze the input data of the task and determine the input data scale and data distribution characteristics; 2-4) Use RealTime-UML to perform modeling analysis, including the business requirements of the task, evaluate its real-time requirements for computing delays, and determine real-time constraints.

5. The method for scheduling GPU and NPU resources for high performance computing according to claim 1, characterized in that: The scheduling optimization module uses PSO to calculate the optimal task allocation plan based on resource status data and task feature data, including the following steps: 3-1) Initialization: N particles are randomly generated, each particle represents a task allocation scheme; the position of each particle indicates the distribution of tasks on the GPU and NPU, and the speed indicates the moving direction and speed of the particle in the search space; at the same time, the historical optimal position of each particle and the historical optimal position of the group are initialized; 3-2) Fitness evaluation: According to the resource status data and task feature data provided by the resource monitoring module and the task analysis module, the fitness value of each particle is calculated; wherein the fitness function is designed according to the resource utilization efficiency and computing performance; 3-3) Speed ​​update: Update the velocity of each particle according to the following formula: v[i]=w·v[i]+c1·rand()·(pbest[i]-x[i])+c2·rand()·(gbest-x[i])where v[i] represents the velocity of particle i, w is the inertia weight, rand() is a random number generation function, c1 and c2 are learning factors, pbest[i] is the historical optimal position of particle i, gbest is the historical optimal position of the group, and x[i] is the position of particle i; 3-4) Location Update: According to the updated velocity, update the position of each particle, that is: x[i]=x[i]+v[i] 3-5) Optimal position update: If the current position of a particle is better than its historical optimal position, then update the historical optimal position of the particle; if the historical optimal position of a particle in the group is better than the historical optimal position of the group, then update the historical optimal position of the group; 3-6) Repeat steps 3-2) to 3-5) until the maximum number of iterations is reached or the fitness value reaches the set accuracy requirement.

6. The method for scheduling GPU and NPU resources for high performance computing according to claim 1, characterized in that: The scheduling optimization module uses the GA algorithm to calculate the optimal task allocation plan based on resource status data and task feature data, including the following steps: 4-1) Initialization: Randomly generate N individuals, each of which represents a task allocation scheme; 4-2) Fitness evaluation: Calculate the fitness value of each individual based on the data provided by the resource monitoring module and the task analysis module; 4-3) Select an operation: According to the fitness value of the individual, a part of excellent individuals are selected as parent individuals to generate the next generation of individuals; wherein the selection method adopts roulette selection or tournament selection; 4-4) Crossover operation: Perform a crossover operation on the selected parent individuals to generate new offspring individuals; the crossover operation adopts a single-point crossover or a two-point crossover method; 4-5) Mutation operation: Perform mutation operation on the newly generated offspring individuals to increase the diversity of individuals; wherein the mutation operation adopts the bit mutation or real number mutation method; 4-6) Replacement operation: The newly generated offspring individuals replace the individuals with poor fitness in the population to form a new population. 4-7) Repeat steps 4-2 to 4-6) until the maximum number of iterations is reached or the fitness value reaches the set accuracy requirement.

7. The method for scheduling GPU and NPU resources for high performance computing according to claim 1, characterized in that: When the scheduling optimization module adopts the PSO and GA algorithms, combined with the characteristics of the hardware accelerator, the following steps are performed: In the PSO algorithm, the speed and position update formulas of particles are adjusted according to the parallel computing capability of the GPU and the neural network acceleration capability of the NPU, including: increasing the preference for GPU parallel computing tasks so that particles are more inclined to assign computing tasks to the GPU; In the GA algorithm, the crossover and mutation operations are adjusted according to the characteristics of the hardware accelerator, so that the newly generated individuals are more suitable for executing tasks on the hardware accelerator; The performance indicators of the hardware accelerator are introduced as part of the fitness function, including: the computing speed of the GPU and the neural network inference speed of the NPU, to ensure that the optimal task allocation solution searched by the algorithm can give full play to the performance of the hardware accelerator.

8. The method for scheduling GPU and NPU resources for high performance computing according to claim 1, characterized in that: The setting of dynamic scheduling module parameters includes: setting monitoring frequency, setting resource change threshold and task execution progress threshold, using resource monitoring module information and setting triggers; a. Set the monitoring frequency: This determines the time interval for the system to monitor task execution and resource changes. The higher the frequency, the more timely the information can be obtained. b. Set resource change threshold: When resource changes exceed this threshold, trigger the task reallocation; c. Task execution progress threshold: used to determine whether a task is close to completion or an exception occurs. For example, a progress percentage threshold is set. When the task progress exceeds a certain value, consider adjusting the allocation of subsequent tasks; d. Information about using resource monitoring module: According to the information of the real-time resource monitoring module and the progress of the tasks being executed, the execution status of the tasks can be obtained through the task management system or the task monitoring interface; e. Set triggers: When resource changes exceed the preset threshold or the task execution progress reaches the set value, the dynamic adjustment process is triggered.

9. A GPU and NPU resource scheduling system for high performance computing, characterized in that: The resource monitoring module is used to monitor the GPU and NPU hardware accelerators in real time and store the collected resource status data of the GPU and NPU hardware accelerators for the scheduling optimization module to call; The task analysis module is used to analyze the AI ​​computing tasks to be executed, obtain the task feature data, and transmit the task feature data to the scheduling optimization module; The scheduling optimization module is used to obtain the optimal task allocation plan based on resource status data and task feature data using PSO and GA algorithms, and pass it to the dynamic scheduling module; The dynamic scheduling module is used to set the dynamic scheduling module parameters. The dynamic scheduling module monitors the task execution status and resource changes in real time, dynamically adjusts the task allocation plan according to the feedback information, and feeds back to the scheduling optimization module; at the same time, the dynamic scheduling module feeds back the change information of resource status data and task execution status to the resource monitoring module and task analysis module respectively; and sends the new task allocation plan to the resource acquisition module.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, a GPU and NPU resource scheduling method for high-performance computing as described in any one of claims 1 to 8 is implemented.

Images