Computing system and method for GPU (Graphics Processing Unit) computing power scheduling

Abstract

The invention provides a computing system and method for GPU computing power scheduling, and the method comprises the following steps: 1, obtaining computing task data, and carrying out the information collection of each task through receiving a to-be-processed computing task queue; according to the invention, through task information collection and deep analysis, and GPU resource omnibearing monitoring and workload prediction, deep understanding of tasks and GPU resources is realized, highly accurate computing power distribution can be carried out, resource waste is avoided, and the resource utilization rate is improved; advanced technologies such as a knowledge graph, mixed integer programming and deep reinforcement learning are fused through the intelligent scheduling model, the fusion mode is innovative in the GPU computing power scheduling field, the limitation of singleness of a traditional scheduling model is overcome, the method can better adapt to a complex and changeable computing environment, the task execution efficiency is optimized, and the task scheduling efficiency is improved. The adaptability and competitiveness of the whole system in a complex environment are remarkably improved, and an excellent solution is provided for the GPU computing power scheduling problem in the field of high-performance computing.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a computing system and method for GPU computing power scheduling. Background Art

[0002] In today's era of rapid technological progress, fields such as scientific computing, artificial intelligence, and graphics processing are booming. In the field of scientific computing, tasks such as the simulation of complex physical phenomena and the analysis and processing of large-scale data require strong computing power to support. From the simulation of the microscopic world of quantum mechanics to the construction of macroscopic models of galaxy evolution in astrophysics, each computing link has extremely high requirements for computing resources. Against this background of development, GPU (graphics processing unit) has become more and more widely and deeply used in these fields with its excellent parallel computing capabilities, high-bandwidth memory access, and hardware architecture designed for graphics and computing-intensive tasks. It has gradually evolved from hardware that was originally used solely for graphics rendering to a key core component that drives the development of these fields, playing an irreplaceable role in meeting the needs of high-performance computing. However, there are still certain problems: 1. In multi-task, multi-user and complex application scenarios, traditional scheduling methods often lack detailed consideration of task and GPU resource characteristics. For example, in deep learning training, neural networks of different architectures have different requirements for computing power and data transmission, and traditional scheduling methods are difficult to adapt; Second, in multi-scenario interactive applications of graphics rendering, the requirements for GPU real-time computing power and resource coordination capabilities are extremely high. The existing scheduling method easily leads to rendering jams and resource waste; To this end, a computing system and method for GPU computing power scheduling are proposed. Summary of the invention

[0003] In view of this, the embodiments of the present invention hope to provide a computing system and method for GPU computing power scheduling to solve or alleviate the technical problems existing in the prior art and at least provide a beneficial option.

[0004] To solve the above technical problems, a technical solution adopted in this application is: a calculation method for GPU computing power scheduling, comprising the following steps: Step 1: Obtain computing task data, collect information for each task by receiving the computing task queue to be processed; Step 2: Based on the acquired task information, extract the task feature analysis, determine the task feature analysis results, and the performance under different GPU devices; Step 3: Based on the monitoring GPU device, determine the GPU resource monitoring results and historical workload data; Step 4: Based on the GPU resource monitoring results and historical workload data, use time series analysis and machine learning methods to predict the future workload of the GPU device, predict the GPU load in different time windows, and generate GPU workload prediction information; Step 5: Based on the task feature analysis results and GPU workload prediction information, and integrating knowledge graph, mixed integer programming algorithm and deep reinforcement learning algorithm, build an intelligent scheduling model; Step 6: Based on the intelligent scheduling model, training and optimization are performed. By learning from simulation data or experimental data, the mixed integer programming algorithm and deep reinforcement learning algorithm automatically adjust parameters to allocate the optimal GPU computing power to each task.

[0005] As a further preferred embodiment of the present technical solution, the method for constructing an intelligent scheduling model comprises the following steps: Step 1: Using the knowledge graph, based on the relationship between tasks and GPU devices, make initial GPU device allocation recommendations for the tasks and generate initial matching results; Step 2: Use the initial matching result of the knowledge graph as the input of the mixed integer programming algorithm, and obtain the computing power allocation plan by solving the mixed integer programming problem; Step 3: During the task execution, the deep reinforcement learning algorithm dynamically adjusts the computing power allocation plan obtained by the mixed integer programming algorithm according to the real-time changes of the environment; Step 4: Build an intelligent scheduling model by cooperating with knowledge graph, mixed integer programming algorithm and deep reinforcement learning algorithm.

[0006] As a further preferred embodiment of the present technical solution, the task feature analysis results include computational complexity analysis, task priority, the degree of parallelization of the task and the performance sensitivity of the task under different GPU architectures.

[0007] As a further preferred embodiment of the present technical solution, the GPU resource monitoring result includes usage parameters of the GPU device, hardware performance indicators of the GPU device and performance change trends of the GPU device.

[0008] Provided as a further preferred embodiment of the present technical solution, the time series analysis method includes an autoregressive moving average model (ARMA), an autoregressive integrated moving average model (ARIMA), and a seasonal autoregressive integrated moving average model (SARIMA); the machine learning method includes a support vector machine regression (SVR), a random forest regression (RFR), and a long short-term memory network (LSTM); and the length of the time window is dynamically adjusted according to the characteristics of the tasks in the system and the distribution of historical data.

[0009] As a further preferred embodiment of the present technical solution, the task data includes a basic type of task, a source of task, an expected completion time, an input and output data format of the task, and software library and framework information on which the task depends.

[0010] As a further preferred embodiment of the present technical solution, when allocating GPU computing power to each task, load balancing between GPU devices and energy consumption and performance balance between GPU devices are maintained.

[0011] As a further preferred embodiment of the present technical solution, the real-time changes in the environment include changes in task execution progress, fluctuations in GPU device performance parameters, and insertion of new tasks.

[0012] To solve the above technical problems, another technical solution adopted by the present application is: a computing system for GPU computing power scheduling, including: a task receiving and information collection module, a task feature analysis module, a GPU resource monitoring module, a GPU workload prediction module, a model building module, a computing power allocation module, a monitoring and exception handling module and a model optimization and update module; The task receiving and information collection module is used to receive the computing task queue to be processed, and collect information for each task to obtain task data including the basic type of task, the source of the task, the expected completion time, the input and output data format of the task, and the software library and framework information on which the task depends; The task feature analysis module is used to perform task feature analysis based on the task data acquired by the task receiving and information collecting module, and generate task feature analysis results; The GPU resource monitoring module is used to monitor the GPU devices in the system in real time and obtain GPU resource monitoring results and historical workload data; The GPU workload prediction module is used to predict the future workload of the GPU based on the GPU resource monitoring results and historical workload data obtained by the GPU resource monitoring module, using time series analysis and machine learning methods to predict the load of the GPU in different time windows and generate GPU workload prediction information; The model building module is used to build an intelligent scheduling model according to the task feature analysis results generated by the task feature analysis module and the GPU workload prediction information generated by the GPU workload prediction module; The computing power allocation module is used to allocate the optimal GPU computing power to each task based on the intelligent scheduling model constructed by the intelligent scheduling model construction module; The monitoring and exception handling module is used to monitor the task execution progress, GPU performance parameters and system environment changes in real time using the monitoring system during the task execution process; The model optimization and update module is used to optimize and update the intelligent scheduling model based on the collected task data, GPU resource monitoring data and historical workload data.

[0013] As a further preferred embodiment of the present technical solution, the monitoring frequency of the GPU resource monitoring module is dynamically adjusted according to the system task load.

[0014] The embodiment of the present invention has the following advantages due to the adoption of the above technical solution: 1. The present invention achieves an in-depth understanding of tasks and GPU resources through comprehensive collection and in-depth analysis of task information, as well as all-round monitoring of GPU resources and workload prediction, so as to enable highly accurate computing power allocation, avoid resource waste, and improve resource utilization; 2. The present invention integrates multiple advanced technologies such as knowledge graphs, mixed integer programming and deep reinforcement learning through an intelligent scheduling model. This fusion method is innovative in the field of GPU computing power scheduling, overcomes the limitations of the singleness of traditional scheduling models, and can better adapt to complex and changeable computing environments, optimize task execution efficiency, and significantly improve the adaptability and competitiveness of the entire system in multi-task, multi-GPU and complex environments, providing an excellent solution to the GPU computing power scheduling problem in the field of high-performance computing.

[0015] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present invention will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A flowchart of a method for calculating GPU computing power scheduling according to the present invention; Figure 2 A schematic diagram of a process for constructing an intelligent scheduling model according to the present invention; Figure 3 A schematic diagram of the functional modules of a computing system for GPU computing power scheduling according to the present invention. DETAILED DESCRIPTION

[0018] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0019] It should be clear that the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0020] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0021] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0022] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0023] Figure 1 is a flowchart of a method for calculating GPU computing power scheduling according to an embodiment of the present invention. It should be noted that if there are substantially the same results, the method of the present application is not based on Figure 1 The process sequence shown is limited. Figure 1-Figure 2 As shown: A calculation method for GPU computing power scheduling includes the following steps: Step 1: Obtain computing task data, collect information for each task by receiving the computing task queue to be processed; Specifically: the system or platform has an interface or mechanism to receive computing tasks from different sources (such as user submissions, pushes from other systems, etc.). These tasks are usually organized into a queue so that they can be processed in sequence or according to priority. The received tasks need to be stored in some form of storage medium, such as a database, memory queue, file system, etc. The choice of storage method should take into account factors such as the nature and quantity of the tasks and the processing speed. The system is able to identify and extract key information from each task. This information usually includes the type of task, input data, output requirements, execution conditions, etc. The identification process may involve parsing task descriptions, reading configuration files, or calling specific APIs. Specialized parsers or processing programs can be written to extract task information. These programs need to be able to handle different types of tasks and adapt to changes in task descriptions. The extracted task information needs to be stored in an appropriate data structure for use in subsequent steps. Databases, memory data structures (such as hash tables, lists, etc.) or files can be used to store this information. At the same time, an effective information management mechanism needs to be established to ensure the accuracy and consistency of the information.

[0024] Step 2: Based on the acquired task information, extract the task feature analysis, determine the task feature analysis results, and the performance under different GPU devices; Among them, computational complexity analysis is a measure of the computational resources (e.g., time and space) required to perform a task; Task priority is the order determined by the urgency, importance or other business logic of the task; The parallelizability of a task refers to whether the task can be divided into multiple subtasks and executed simultaneously on multiple processors or GPUs; The performance sensitivity of a task under different GPU architectures refers to the degree of performance difference when the task is executed on different GPU architectures (such as NVIDIA, AMD, etc.).

[0025] Step 3: Based on the monitoring GPU device, determine the GPU resource monitoring results and historical workload data; Specifically, GPU resource monitoring collects real-time status information of GPU devices, such as utilization, temperature, power consumption, memory usage, etc., through special monitoring tools or APIs; the monitoring frequency should be determined according to system requirements and resource consumption to ensure the accuracy and timeliness of the data; the collected data should be integrated into one or more data structures for subsequent analysis and processing; in addition to basic indicators such as utilization and temperature, other indicators that may affect GPU performance should also be monitored, such as memory bandwidth, cache hit rate, etc.; if there are multiple GPU devices in the system, each device should be monitored to obtain a comprehensive system status; real-time monitoring data should be combined with historical workload data to more fully understand the performance and resource utilization of GPU devices; Historical workload data refers to the task execution status on the GPU device over a period of time in the past, including task type, execution time, resource consumption, etc.; historical workload data of GPU devices is collected through log recording, database storage, etc.; historical data is cleaned to remove outliers and duplicate data to ensure data accuracy and consistency; the cleaned data is archived and stored for long-term preservation and subsequent analysis.

[0026] Step 4: Based on the GPU resource monitoring results and historical workload data, use time series analysis and machine learning methods to predict the future workload of the GPU device, predict the GPU load in different time windows, and generate GPU workload prediction information; Specifically, remove outliers, missing values ​​or duplicate data in historical workload data to ensure data accuracy and consistency; extract useful features from raw data, such as time series trends, seasonality, periodicity, etc., as well as correlations with other system indicators (such as CPU utilization, memory usage); divide the data set into training sets, validation sets and test sets for model training and evaluation; select appropriate time series analysis methods such as ARIMA (autoregressive integrated moving average), exponential smoothing, state space models, etc. according to data characteristics and forecasting requirements; use training set data to train time series models to capture dynamic changes in data; use validation set data to evaluate model performance, such as forecast accuracy, stability, etc.; According to the complexity of the problem and the characteristics of the data, select the appropriate machine learning model, such as support vector machine (SVM), random forest, neural network (including deep learning model), etc.; use the preprocessed features as the input of the machine learning model; use the training set data to train the model, and optimize the model performance through cross-validation, hyperparameter tuning and other methods; In order to improve the accuracy of prediction, the prediction results of time series analysis and machine learning models can be combined, such as weighted average, linear regression combination, etc.; combined prediction can take advantage of the advantages of different models, reduce the limitations of a single model, and improve the robustness of the prediction; Based on the trained model, the GPU workload in different time windows in the future is predicted and the prediction results are output; the prediction results are interpreted and analyzed, such as prediction trends, fluctuation ranges, anomalies, etc.; the prediction results are visualized to more intuitively understand the changes in future GPU workloads.

[0027] Step 5: Based on the task feature analysis results and GPU workload prediction information, and integrating knowledge graph, mixed integer programming algorithm and deep reinforcement learning algorithm, build an intelligent scheduling model; Among them, the design of the mixed integer programming algorithm includes the following steps: First, determine the decision variables; set up is a binary decision variable, representing the task Is it assigned to GPU? .set up To represent GPU Resources (such as computing power, video memory, etc.) allocation ratio continuous variable, where Covers various resource types of GPU; Then, define the objective function; The goal is to minimize the total execution time of all tasks or maximize the overall resource utilization of the system. For example, if the total execution time is the goal, it can be expressed as: ; in, It's a task On GPU The estimated execution time on the GPU can be estimated based on the computational complexity of the task and the performance parameters of the GPU; Next, set constraints; Task allocation constraints: Each task must and can only be allocated to one or more GPUs, that is: ; For tasks that can be executed in parallel on multiple GPUs, this constraint can be adjusted appropriately according to their parallelization degree; Resource limit constraints: For each GPU , resources assigned to all tasks Cannot exceed GPU The resource limit is: ; in, It's a task About Resources The demand, It is GPU resource The upper limit value of Task priority constraint: high priority tasks are allocated sufficient resources first, which can be achieved by setting priority weights , and reflected in the objective function or constraints, such as: ; in, Is a measure of GPU To the task Parameters of priority satisfaction; Other constraints: Consider the dependencies between tasks. Depends on the task , they need to meet the corresponding sequence or collaborative execution conditions when allocating GPUs; at the same time, combined with the real-time performance and prediction performance of the GPU, the allocation plan is restricted, for example, avoiding allocating tasks with high real-time requirements to GPUs that are predicted to have performance fluctuations.

[0028] Among them, the application steps of the deep reinforcement learning algorithm are as follows: The GPU computing power scheduling system is defined as a reinforcement learning environment. The environment state includes the current task queue, the characteristics of each task (such as computational complexity, priority, etc.), and the current state of the GPU (such as resource utilization, performance parameters, predicted workload, etc.); Defining actions includes adjusting the allocation of tasks to GPUs and the allocation of GPU resources, such as migrating a task from one GPU to another, or changing the proportion of computing power allocated to each task by a GPU; Design a reward function based on the system's goals. For example, if the average execution time of tasks is reduced or the GPU resource utilization rate is improved, a positive reward is given if a scheduling decision reduces the total execution time of all tasks compared to the previous scheduling. On the contrary, if there is a delay in task execution or excessive idleness or overload of GPU resources, a negative reward is given. The agent continuously selects actions in the environment and chooses a scheduling action based on the current environmental state. The environment updates its state based on the action and feeds back rewards. The agent learns the optimal scheduling strategy under different environmental states by continuously trying different actions to maximize long-term cumulative rewards. The training process can use experience replay, target network and other technologies to improve learning efficiency and stability.

[0029] Step 6: Based on the intelligent scheduling model, training and optimization are performed. By learning from simulation data or experimental data, the mixed integer programming algorithm and deep reinforcement learning algorithm are automatically adjusted to allocate the optimal GPU computing power to each task. Specifically, the task and its assigned GPU computing power information are passed to the task execution system to start task execution; during task execution, GPU resource usage is continuously monitored to ensure that the task can be executed as expected; if insufficient or wasted resources occur during task execution, dynamic adjustments are made based on real-time monitoring data.

[0030] In one embodiment, specifically, in step 1, the task data includes the basic type of the task, the source of the task, the expected completion time, the input and output data formats of the task, and the software library and framework information on which the task depends; The basic types of tasks include various types of model training for deep learning and different types of rendering technology applications for graphics rendering. Among them, various types of model training for deep learning include training of convolutional neural networks (CNN), training of recurrent neural networks (RNN), training of generative adversarial networks (GAN), etc. Different types of model training may require different computing resources and time. Different types of rendering technology applications for graphics rendering include ray tracing rendering, rasterization rendering, real-time rendering, etc. Each rendering technology has its specific computing requirements and resource consumption patterns. Task sources include user-side or server-side applications; The expected completion time is specified by the user or estimated based on the task's historical execution data; The input and output data formats of the task include data dimensions and data types; The software library and framework information that the task depends on includes the version compatibility information of the software library and the optimization parameter information of the framework, which is used to analyze the impact of different versions and optimization parameters on the task computing complexity and resource requirements.

[0031] In one embodiment, specifically, in step 2, the task feature analysis results include computational complexity analysis, task priority, task parallelization degree, and task performance sensitivity under different GPU architectures; Among them, computational complexity analysis is quantified by analyzing the algorithm structure, data volume, number of model parameters and data dependency using an improved complexity evaluation algorithm. This improved complexity evaluation algorithm comprehensively considers factors such as the algorithm's time complexity, data volume, data dimension and number of model parameters, thereby more accurately reflecting the computational complexity of the task. For example, for deep learning tasks, the impact of factors such as the number of neural network layers, the number of neurons in each layer, and the size of the convolution kernel on the computational complexity will be considered. Task priority is determined by comprehensively considering user settings, the impact of tasks on the overall system performance, business value and urgency. If a task is marked as high-urgency by the user, such as a deep learning task related to real-time financial risk prediction, its priority will be increased accordingly. If a task has a significant impact on the smooth operation of the entire system, such as a core task that provides graphics rendering services to multiple users at the same time, it will also be given a higher priority. The degree of parallelization of a task is determined by analyzing the task calculation logic, including clarifying the parts of the task that can be executed in parallel and the parallel granularity; for tasks with a large number of independent data blocks, such as the processing of different images in image recognition, the degree of parallelization is higher; for tasks with strict sequence dependencies, the degree of parallelization is lower; by analyzing the task calculation graph and data dependencies, the appropriate parallel execution scheme can be determined. For example, in some scientific computing tasks, different rows or columns of matrix operations can be assigned to different computing units for parallel processing; The performance sensitivity of tasks under different GPU architectures refers to the degree to which the efficiency of task execution is affected by the GPU core architecture and cache structure factors. For example, for tasks with a large amount of data reuse, the performance may be better under a GPU architecture with a larger cache; for tasks that are computationally intensive and highly data parallel, they can be executed more efficiently on GPUs with more streaming multiprocessors; by testing and analyzing the performance of different types of tasks under multiple GPU architectures, a performance correlation model between tasks and GPU architectures can be established, thus providing a basis for subsequent scheduling.

[0032] In one embodiment, specifically, in step 3, the GPU resource monitoring result includes the current usage parameters of the GPU, the hardware performance index of the GPU, and the potential performance change trend of the GPU; the current usage parameters of the GPU include the current usage rate of the GPU, the occupancy of the video memory, the core frequency and temperature of the GPU; Among them, GPU current utilization rate: reflects the proportion of resources occupied by the task currently being processed by the GPU; Video memory usage: Displays the used and available space of GPU video memory, which is particularly important for tasks that require a large amount of video memory; GPU core frequency: The operating frequency of the GPU core directly affects computing performance; Temperature: The temperature status of the GPU. Too high a temperature may cause performance degradation or even hardware damage.

[0033] The hardware performance indicators of GPU include the performance status of each computing unit, the read and write speed and utilization rate of different levels of storage structures. The computing units include streaming multiprocessors and texture units. The storage structures include registers, shared memory, local memory and video memory. Among them, registers: high-speed, low-latency storage structures used to store temporary data between threads; Shared memory: allows threads within a thread block to share data, with faster reading and writing speeds; Local memory: Each thread has its own unique storage space, with relatively slow reading and writing speeds; Video memory: large capacity, but slow reading and writing speeds, used to store global data.

[0034] The potential performance change trend of the GPU uses the GPU hardware performance fluctuation model to compare historical data and real-time monitoring data, capture the slight fluctuations in GPU performance and make predictions.

[0035] In one embodiment, specifically, in step 4, the time series analysis method includes an autoregressive moving average model (ARMA), an autoregressive integrated moving average model (ARIMA), and a seasonal autoregressive integrated moving average model (SARIMA), which are used to analyze the regularity of GPU load data in time series, including periodic fluctuations and trend changes; Among them, the autoregressive moving average model (ARMA) is suitable for modeling and forecasting of stationary time series data, and reveals the inherent dynamic structure of the data by capturing the autoregressive and moving average components in the data; The Autoregressive Integrated Moving Average (ARIMA) model is an extension of the ARMA model and is suitable for modeling and forecasting non-stationary time series data. The non-stationary data is converted into stationary data through difference operations, and then the ARMA model is applied. The Seasonal Autoregressive Integrated Moving Average Model (SARIMA) further extends the ARIMA model to handle time series data with seasonal variations by introducing seasonal autoregressive and seasonal moving average components to capture seasonal fluctuations.

[0036] Machine learning methods include support vector machine regression (SVR), random forest regression (RFR), and long short-term memory network (LSTM) to predict future GPU workloads based on historical workload data and current GPU resource monitoring results; Among them, support vector machine regression (SVR) is a regression analysis method based on support vector machine. It approximates the target function by finding an optimal hyperplane in high-dimensional space and is suitable for processing data with nonlinear relationships. Random forest regression (RFR) is an ensemble learning method that improves prediction accuracy and robustness by building multiple decision trees and averaging them; Long short-term memory network (LSTM) is a special recurrent neural network (RNN) structure that captures long-term dependencies in time series data by introducing mechanisms such as memory units and forget gates. It is suitable for processing data with complex time dynamics.

[0037] The length of the time window is dynamically adjusted according to the characteristics of the tasks in the system and the distribution of historical data. The adjustment range and rules are determined according to the system's performance requirements and historical experience. The system can dynamically adjust the length of the time window based on real-time monitoring of GPU load data and prediction results, as well as the needs of task scheduling and resource allocation. For example, when it is predicted that the future load will increase significantly, the time window can be shortened to update the prediction results more frequently, and the resource allocation strategy can be adjusted accordingly.

[0038] In one embodiment, specifically, in step five, the method for constructing an intelligent scheduling model includes the following steps: Step 1: Use the knowledge graph to make initial GPU allocation recommendations for tasks based on the relationship between tasks and GPUs, and generate initial matching results; Specifically, the knowledge graph is a structured form of knowledge representation that can integrate various relevant knowledge information about tasks and GPUs, such as the requirements of different types of tasks for GPU performance indicators, and the scope of tasks that different GPU models are suitable for processing. Through these existing knowledge associations, it can make preliminary analysis and judgments on tasks based on the intrinsic relationship between tasks and GPUs. Based on the rich knowledge contained in the knowledge graph, for each task to be processed, the system will refer to the various attributes of the task (such as task type, data volume, demand for parallel computing, etc.) and the characteristics of the GPU (such as computing power, video memory size, bandwidth, etc.), so as to give a preliminary and relatively reasonable suggestion on which GPU device or GPU resource allocation ratio the task is suitable for. After the above analysis and suggestion process based on the knowledge graph, a preliminary correspondence result between each task and the possible matching GPU resources will be obtained. This result is the initial matching result, which lays the foundation for further optimization allocation.

[0039] Step 2: Use the initial matching results of the knowledge graph as the input of the mixed integer programming algorithm. By solving the mixed integer programming problem, a more optimized computing power allocation solution is obtained while satisfying complex constraints. Specifically, mixed integer programming is a mathematical programming method that deals with problems that contain both continuous variables and integer variables. In this scenario, it may involve comprehensive optimization decisions on multiple variables such as the amount of GPU memory allocated (continuous variable) and the number of GPU cores allocated to the task (integer variable). The optimal solution is found by setting an objective function (such as minimizing task completion time, maximizing resource utilization, etc.) and a series of complex constraints (such as the total resource limit of the GPU, the minimum resource requirements of the task, etc.).

[0040] Using the initial matching results obtained through the knowledge graph in step 1 as the input of the mixed integer programming algorithm means taking the preliminarily determined correspondence between tasks and GPUs and the preliminary idea of ​​related resource allocation as the starting conditions of the algorithm. On this basis, the computing power and optimization mechanism of the mixed integer programming algorithm are used for further in-depth analysis and processing.

[0041] When solving mixed integer programming problems, it is necessary to consider numerous constraints, which reflect various limitations and requirements in the actual computing environment. For example, the GPU device itself has its own hardware performance limitations, such as video memory capacity, core frequency, etc. cannot exceed its specified range; at the same time, the task also has its own demand constraints, such as some tasks require a certain amount of video memory to run normally, or have minimum requirements for the parallel processing capabilities of the GPU core. By performing calculations and optimizations under the premise of satisfying these complex constraints, a more reasonable and optimized computing power allocation plan is finally obtained than the initial matching result, which can better balance task requirements and resource supply.

[0042] Step 3: During the task execution, the deep reinforcement learning algorithm dynamically adjusts the computing power allocation plan obtained by the mixed integer programming algorithm according to the real-time changes of the environment; Specifically, deep reinforcement learning is a method that combines deep learning and reinforcement learning. It allows the agent (here it can be seen as the part of the system responsible for computing power scheduling) to continuously take actions in the environment (such as adjusting the computing power allocation plan) and learn and optimize its own decision-making strategy based on the rewards brought by the actions (such as shortened task completion time, improved resource utilization, etc.). In this process, the agent constantly tries different actions and observes the feedback from the environment, so as to gradually find the optimal behavior pattern.

[0043] During the task execution process, the environment is constantly changing, including multiple aspects, such as changes in task execution progress (some tasks may complete some stages ahead of schedule, or encounter difficulties that cause progress delays), fluctuations in GPU device performance parameters (such as GPU core frequency drops due to temperature increases, changes in video memory read and write speeds, etc.), and the insertion of new tasks (new computing tasks are suddenly added to the task queue), etc. These real-time changes will affect the originally formulated computing power allocation plan, and may make the originally appropriate allocation plan no longer appropriate.

[0044] The deep reinforcement learning algorithm will monitor the changes in these environments in real time, and make timely adjustments to the computing power allocation plan obtained by the mixed integer programming algorithm based on the impact of these changes and the decision-making strategies it has learned. For example, if it is found that a task is executing slower due to a decrease in the GPU core frequency, the deep reinforcement learning algorithm may decide to allocate some computing power from other tasks to support it; or when a new task is inserted, it will re-evaluate the priority and resource requirements of all tasks, and then adjust the entire computing power allocation plan to ensure that the computing power allocation remains relatively reasonable in the ever-changing environment, thereby ensuring that the task can be executed efficiently and smoothly.

[0045] Step 4: By combining knowledge graph, mixed integer programming algorithm and deep reinforcement learning algorithm, an intelligent scheduling model that adapts to complex computing environments is constructed. Specifically, the knowledge graph provides the initial knowledge foundation and preliminary resource allocation ideas for the entire scheduling process; the mixed integer programming algorithm performs in-depth mathematical optimization on this basis to obtain a relatively optimized computing power allocation plan while satisfying actual constraints; the deep reinforcement learning algorithm dynamically adjusts this plan according to real-time changes in the environment during task execution. These three algorithms do not operate in isolation, but rather cooperate and complement each other.

[0046] Through the close collaboration of these three algorithms, a complete intelligent scheduling model is finally constructed. This model can adapt to complex computing environments because it comprehensively considers factors such as the characteristics of the task (through knowledge graphs and task feature analysis), resource limitations (processing complex constraints through mixed integer programming algorithms), and dynamic changes in the environment (through real-time monitoring and adjustment of deep reinforcement learning algorithms). In such a complex computing environment, the intelligent scheduling model can efficiently schedule GPU computing power and reasonably allocate resources to achieve fast and smooth completion of tasks and efficient use of resources.

[0047] In one embodiment, specifically, in step six, when allocating appropriate GPU computing power to each task, high-priority and high-complexity tasks are given priority to obtain sufficient computing power, maintain load balance among GPU devices, and consider synergy and data locality among tasks as well as energy consumption and performance balance of the GPU; Considering the synergy and data locality between tasks, for task groups with data interaction, they are assigned to GPU combinations with efficient communication mechanisms, and the layout of data in the GPU storage structure is optimized according to the data access pattern; Considering the balance between GPU energy consumption and performance, the GPU voltage and frequency are dynamically adjusted according to the GPU power consumption curve and the performance requirements of the task to allocate energy-saving computing power.

[0048] In one embodiment, specifically, the real-time changes in the environment include changes in task execution progress, fluctuations in GPU device performance parameters, and insertion of new tasks.

[0049] Figure 3 is a functional module diagram of a computing system for GPU computing power scheduling in an embodiment of the present application, such as Figure 3 As shown, a computing system for GPU computing power scheduling includes: a task receiving and information collection module, a task feature analysis module, a GPU resource monitoring module, a GPU workload prediction module, a model building module, a computing power allocation module, a monitoring and exception handling module, and a model optimization and update module; The task receiving and information collection module is used to receive the computing task queue to be processed and collect information for each task to obtain task data including the basic type of task, task source, expected completion time, input and output data format of the task, and software library and framework information on which the task depends; The task feature analysis module is used to perform task feature analysis based on the task data obtained by the task receiving and information collection module, and generate task feature analysis results including computational complexity analysis, task priority, task parallelization degree and task performance sensitivity under different GPU architectures; The GPU resource monitoring module is used to conduct all-round real-time monitoring of the GPU devices in the system, obtain GPU resource monitoring results including the current usage parameters of the GPU, the hardware performance indicators of the GPU, and the potential performance change trend of the GPU, and collect historical workload data; The GPU workload prediction module is used to predict the future workload of the GPU based on the GPU resource monitoring results and historical workload data obtained by the GPU resource monitoring module, using time series analysis and machine learning methods to predict the GPU load in different time windows and generate GPU workload prediction information; A model building module, used to build an intelligent scheduling model according to the task feature analysis results generated by the task feature analysis module and the GPU workload prediction information generated by the GPU workload prediction module; The computing power allocation module is used to allocate appropriate GPU computing power to each task based on the intelligent scheduling model built by the intelligent scheduling model building module; The monitoring and exception handling module is used to monitor the progress of task execution, GPU performance parameters, and system environment changes in real time during task execution using the monitoring system. If an abnormal situation occurs, the root cause of the problem is determined through an intelligent diagnosis mechanism based on causal analysis and Bayesian networks, and the computing power allocation is adjusted dynamically in real time based on the diagnosis results. The model optimization and update module is used to continuously optimize and update the intelligent scheduling model based on newly collected task data, GPU resource monitoring data, and historical workload data.

[0050] In one embodiment, specifically, the monitoring frequency of the GPU resource monitoring module is dynamically adjusted according to the system task load.

[0051] In summary, the computing system for GPU computing power scheduling provided by the embodiment of the present invention is an efficient, intelligent and flexible solution, which can significantly improve the utilization efficiency of GPU resources and the efficiency of task execution.

[0052] For other details about the technical solutions for implementing each module in the computing system for GPU computing power scheduling in the above embodiment, please refer to the description of a calculation method for GPU computing power scheduling in the above embodiment, which will not be repeated here.

[0053] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0054] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0055] In the present disclosure, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0056] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0057] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0058] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0059] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0060] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A calculation method for GPU computing power scheduling, characterized in that: The following steps are involved: Step 1: Obtain computing task data, collect information for each task by receiving the computing task queue to be processed; Step 2: Based on the acquired task information, extract the task feature analysis, determine the task feature analysis results, and the performance under different GPU devices; Step 3: Based on the monitoring GPU device, determine the GPU resource monitoring results and historical workload data; Step 4: Based on the GPU resource monitoring results and historical workload data, use time series analysis and machine learning methods to predict the future workload of the GPU device, predict the GPU load in different time windows, and generate GPU workload prediction information; Step 5: Based on the task feature analysis results and GPU workload prediction information, and integrating knowledge graph, mixed integer programming algorithm and deep reinforcement learning algorithm, build an intelligent scheduling model; Step 6: Based on the intelligent scheduling model, training and optimization are performed. By learning from simulation data or experimental data, the mixed integer programming algorithm and deep reinforcement learning algorithm automatically adjust parameters to allocate the optimal GPU computing power to each task.

2. The method for calculating GPU computing power scheduling according to claim 1, characterized in that: The method for constructing an intelligent scheduling model comprises the following steps: Step 1: Using the knowledge graph, based on the relationship between tasks and GPU devices, make initial GPU device allocation recommendations for the tasks and generate initial matching results; Step 2: Use the initial matching result of the knowledge graph as the input of the mixed integer programming algorithm, and obtain the computing power allocation plan by solving the mixed integer programming problem; Step 3: During the task execution, the deep reinforcement learning algorithm dynamically adjusts the computing power allocation plan obtained by the mixed integer programming algorithm according to the real-time changes of the environment; Step 4: Build an intelligent scheduling model by cooperating with knowledge graph, mixed integer programming algorithm and deep reinforcement learning algorithm.

3. The calculation method for GPU computing power scheduling according to claim 1, characterized in that: The task feature analysis results include computational complexity analysis, task priority, task parallelization degree and task performance sensitivity under different GPU architectures.

4. The method for calculating GPU computing power scheduling according to claim 1, characterized in that: The GPU resource monitoring results include usage parameters of the GPU device, hardware performance indicators of the GPU device, and performance change trends of the GPU device.

5. The method for calculating GPU computing power scheduling according to claim 1, characterized in that: The time series analysis methods include autoregressive moving average model (ARMA), autoregressive integrated moving average model (ARIMA), and seasonal autoregressive integrated moving average model (SARIMA); the machine learning methods include support vector machine regression (SVR), random forest regression (RFR), and long short-term memory network (LSTM); the length of the time window is dynamically adjusted according to the characteristics of the tasks in the system and the distribution of historical data.

6. The method for calculating GPU computing power scheduling according to claim 1, characterized in that: The task data includes the basic type of the task, the source of the task, the expected completion time, the input and output data formats of the task, and the software library and framework information that the task depends on.

7. The method for calculating GPU computing power scheduling according to claim 1, characterized in that: When allocating GPU computing power to each task, the load balance between GPU devices and the energy consumption and performance balance of GPU devices are maintained.

8. The method for calculating GPU computing power scheduling according to claim 2, characterized in that: The real-time changes in the environment include changes in task execution progress, fluctuations in GPU device performance parameters, and insertion of new tasks.

9. A computing system for GPU computing power scheduling, applied to a computing method for GPU computing power scheduling according to any one of claims 1 to 9, characterized in that: include: Task reception and information collection module, task feature analysis module, GPU resource monitoring module, GPU workload prediction module, model building module, computing power allocation module, monitoring and exception handling module, and model optimization and update module; The task receiving and information collection module is used to receive the computing task queue to be processed, and collect information for each task to obtain task data including the basic type of task, the source of the task, the expected completion time, the input and output data format of the task, and the software library and framework information on which the task depends; The task feature analysis module is used to perform task feature analysis based on the task data acquired by the task receiving and information collecting module, and generate task feature analysis results; The GPU resource monitoring module is used to monitor the GPU devices in the system in real time and obtain GPU resource monitoring results and historical workload data; The GPU workload prediction module is used to predict the future workload of the GPU based on the GPU resource monitoring results and historical workload data obtained by the GPU resource monitoring module, using time series analysis and machine learning methods to predict the load of the GPU in different time windows and generate GPU workload prediction information; The model building module is used to build an intelligent scheduling model according to the task feature analysis results generated by the task feature analysis module and the GPU workload prediction information generated by the GPU workload prediction module; The computing power allocation module is used to allocate the optimal GPU computing power to each task based on the intelligent scheduling model constructed by the intelligent scheduling model construction module; The monitoring and exception handling module is used to monitor the task execution progress, GPU performance parameters and system environment changes in real time using the monitoring system during the task execution process; The model optimization and update module is used to optimize and update the intelligent scheduling model based on the collected task data, GPU resource monitoring data and historical workload data.

10. The computing system for GPU computing power scheduling according to claim 9, characterized in that: The monitoring frequency of the GPU resource monitoring module is dynamically adjusted according to the system task load.

Images