A GPU heterogeneous resource scheduling method based on mixed execution of multiple simulation jobs

Abstract

This invention relates to the field of engineering simulation technology and discloses a GPU heterogeneous resource scheduling method based on the mixed execution of multiple simulation jobs. This invention considers the computational characteristics of different simulation tasks and introduces computation time prediction models and memory usage prediction models to predict resource consumption during job execution in advance, achieving accurate characterization of resource requirements in complex multiphysics simulation scenarios. When potential conflicts are detected, Monte Carlo tree search technology is used to explore multiple scheduling strategies and select the scheduling scheme that achieves the optimal balance between computation latency, throughput, and energy consumption. This invention comprehensively analyzes historical data and real-time job requirements of the simulation platform, achieving accurate prediction and dynamic allocation of GPU heterogeneous resources, effectively improving the utilization rate and overall performance of GPU heterogeneous resources in a mixed execution environment of multiple simulation jobs, and providing support for efficient resource scheduling in complex simulation scenarios.

Description

Technical Field

[0001] This invention relates to the field of engineering simulation technology and discloses a GPU heterogeneous resource scheduling method based on the mixed execution of multiple simulation jobs. Background Technology

[0002] As the scale and complexity of engineering simulations continue to increase, GPUs have become a key acceleration unit in multiphysics coupled simulations. However, existing GPU heterogeneous resource scheduling solutions have several shortcomings that make it difficult to meet the needs of professional simulations: First, static quotas do not match the simulation load. For example, solutions such as KubernetesDevicePlugin and NVIDIAMPS are based on fixed memory or full card quotas and cannot perceive the iterative FLOPs, mesh size, and dynamic memory growth of CFD / combustion / structure / heat transfer simulations, resulting in generally low resource utilization. Second, there is a lack of simulation-level prediction and pre-scheduling. When different simulation modules share GPUs, memory contention and computational pipeline conflicts are likely to occur. Existing solutions rely only on runtime monitoring or threshold alarms and cannot predict the risk of memory fragmentation or SM / CU saturation before startup. They are often forced to reschedule due to OOM or performance fluctuations in the middle of the process. Summary of the Invention

[0003] The purpose of this invention is to provide a GPU heterogeneous resource scheduling method based on the mixed execution of multiple simulation jobs, which can achieve accurate prediction and dynamic allocation of GPU heterogeneous resources, and improve the utilization rate and overall performance of GPU heterogeneous resources in the mixed execution environment of multiple simulation jobs.

[0004] To achieve the above-mentioned technical effects, the technical solution adopted by the present invention is as follows:

[0005] A GPU heterogeneous resource scheduling method based on the hybrid execution of multiple simulation jobs includes:

[0006] The historical simulation job data of the simulation platform is obtained. The historical simulation job data includes the number of grid cells, floating-point operation volume, computing time, and initial memory allocation of each simulation task in the historical simulation job, as well as the memory usage per second, the theoretical peak computing power of the GPU occupied, and the resource scheduling overhead time constant of the simulation platform when each simulation task is executed. The simulation tasks include CFD simulation tasks, combustion simulation tasks, structural simulation tasks, and heat transfer simulation tasks.

[0007] The system obtains the floating-point computation volume, number of grid cells, required initial memory allocation, and required GPU operating frequency for each simulation task in each current scheduled simulation job. Using the floating-point computation volume of each simulation task as input, it analyzes and predicts the computation time value of each current scheduled simulation job using a simulation job computation time prediction model. Using the number of grid cells and the required initial memory allocation for each simulation task as input, it analyzes and predicts the memory usage per second of each current scheduled simulation job during execution using a simulation job memory usage prediction model. Finally, using the required GPU operating frequency as the search condition, it queries the LookupTable containing GPU frequency-latency-power consumption presets to find the power consumption and latency corresponding to each GPU in the simulation platform at the specified GPU operating frequency.

[0008] Obtain the timestamp corresponding to the preset deadline for submitting simulation jobs to the simulation platform, the current timestamp of the simulation platform, and combine the total floating-point operation volume, power consumption, and predicted computation time of each simulation job to be scheduled to calculate the real-time priority of each simulation job to be scheduled.

[0009] Based on the total floating-point operations and predicted computation time of each scheduled simulation job, as well as the number of grid cells, initial memory allocation, predicted memory usage per second, required GPU operating frequency, and corresponding power consumption and latency of each simulation task within each scheduled simulation job, all scheduled simulation jobs are simulated in a virtualized environment to predict whether there is a risk of resource conflict on the simulation platform. If there is no risk of conflict, a baseline table-based fast allocation strategy is used to determine the scheduling scheme; otherwise, Monte Carlo tree search is used to generate multiple migration / precision / dynamic voltage-frequency adjustment DVFS combinations of proposed scheduling schemes. Among the proposed scheduling schemes, the Pareto optimal solution that satisfies the computational latency, throughput, and energy consumption is selected as the scheduling scheme. The resource conflict risks include memory overflow and / or SM / CU utilization saturation. The scheduling scheme includes the computing power allocation scheme and GPU operating frequency scheme corresponding to each simulation task in the scheduled simulation job.

[0010] Based on the aforementioned scheduling scheme, SM computing units / CU computing units are allocated to each simulation task in the currently scheduled simulation job, and the GPU operating frequency is determined. Then, the simulation is executed according to the real-time priority of the currently scheduled simulation job.

[0011] Furthermore, the method for constructing the simulation job calculation time prediction model includes:

[0012] Using the floating-point computation volume of several CFD simulation tasks, the theoretical peak computing power of the corresponding GPU, and the resource scheduling overhead time constant as inputs, and the computation time of each CFD simulation task as output, a CFD simulation task computation time prediction function based on the first calibration coefficient is fitted and constructed.

[0013] Using the floating-point computation amount of several combustion simulation tasks, the theoretical peak computing power of the corresponding GPU, and the resource scheduling overhead time constant as inputs, and the computation time of each combustion simulation task as output, a combustion simulation task computation time prediction function based on the second calibration coefficient is fitted and constructed.

[0014] Using the floating-point computation volume of several structural simulation tasks, the theoretical peak computing power of the corresponding GPU, and the resource scheduling overhead time constant as inputs, and the computation time of each structural simulation task as output, a structural simulation task computation time prediction function based on a third calibration coefficient is fitted and constructed.

[0015] Using the floating-point computation amount of several heat transfer simulation tasks, the theoretical peak computing power of the corresponding GPU, and the resource scheduling overhead time constant as inputs, and the computation time of each heat transfer simulation task as output, a heat transfer simulation task computation time prediction function based on the fourth calibration coefficient is fitted and constructed.

[0016] The simulation task calculation time prediction model is formed by adding the CFD simulation task calculation time prediction function based on the first calibration coefficient, the combustion simulation task calculation time prediction function based on the second calibration coefficient, the structural simulation task calculation time prediction function based on the third calibration coefficient, and the heat transfer simulation task calculation time prediction function based on the fourth calibration coefficient.

[0017] Furthermore, the CFD simulation task computation time prediction function based on the first calibration coefficient is constructed as follows:

[0018] ;

[0019] The combustion simulation task calculation time prediction function based on the second calibration coefficient is constructed as follows:

[0020] ;

[0021] The computation time prediction function for the structural simulation task based on the third calibration coefficient is constructed as follows:

[0022] ;

[0023] The heat transfer simulation task calculation time prediction function based on the fourth calibration coefficient is constructed as follows:

[0024] ;

[0025] in, Calculate the time for CFD simulation tasks. The first calibration coefficient, The floating-point computational complexity of CFD simulation tasks. This represents the theoretical peak computing power of the GPU used during the execution of a CFD simulation task. The calculation time is for the combustion simulation task. This is the second calibration coefficient. The floating-point computational load for combustion simulation tasks, This represents the theoretical peak computing power of the GPU used during the execution of the combustion simulation task. This refers to the computation time for the structural simulation task. For the third calibration coefficient, This refers to the floating-point computational complexity of the structural simulation task. This represents the theoretical peak computing power of the GPU used during the execution of the structural simulation task. The computation time for the heat transfer simulation task. This is the fourth calibration coefficient. The floating-point computational load for the heat transfer simulation task. This represents the theoretical peak computing power of the GPU used during the execution of the heat transfer simulation task. This is the time constant for resource scheduling overhead of the simulation platform during simulation task execution.

[0026] Furthermore, the simulation job memory usage prediction model is constructed as follows:

[0027] ;

[0028] in, For the simulation operation in the first Predicted memory usage per second. For the same simulation job, the CFD simulation task in the first... Predicted memory usage per second. For the same simulation job, the combustion simulation task in the first... Predicted memory usage per second. For the structural simulation task in the same simulation job, in the first... Predicted memory usage per second. For the heat transfer simulation task in the same simulation job, in the first... Predicted memory usage per second.

[0029] Furthermore, in the same simulation job, the CFD simulation task is in the... The predicted memory usage per second is calculated using the following formula:

[0030] ;

[0031] in: For the first CFD simulation task execution +1 second of predicted video memory usage; For the first CFD simulation task execution Video memory usage per second; This is the initial memory allocation when a CFD simulation task starts. The video memory usage per second for each mesh cell in a CFD simulation task; The number of mesh cells for a CFD simulation task; The first autoregressive coefficient; These are exogenous input coefficients; For the intercept term; White noise error

[0032] Furthermore, the real-time priority calculation formula for each simulation job to be scheduled is as follows:

[0033] ;

[0034] in: The real-time priority of the simulation jobs to be scheduled; , , Preset weighting coefficients; The timestamp corresponding to the preset deadline for submitting the simulation job to the simulation platform; This is the current timestamp of the simulation platform; The predicted computation time for the simulation job to be scheduled; The total floating-point computation of the simulation jobs to be scheduled; The power consumption of the simulation job to be scheduled; In order to be in , Take the maximum value between the two.

[0035] Furthermore, based on the predicted memory usage per second for each scheduled simulation job during execution, if the peak value of the predicted memory usage exceeds a preset available memory threshold, the real-time priority of the corresponding scheduled simulation job will be forcibly increased. Place it at the lowest level.

[0036] Furthermore, the method for determining the LookupTable containing GPU frequency-latency-power consumption presets includes: for each GPU of the simulation platform, running selected representative simulation cases at different preset GPU operating frequencies, measuring the power consumption and latency of each GPU running at the preset GPU operating frequency, recording each preset GPU operating frequency and its corresponding power consumption and latency as triples, and storing them as a lookup table.

[0037] Compared with the prior art, the beneficial effects of this invention are:

[0038] This invention considers the computational characteristics of different simulation tasks (such as CFD, combustion, structural, and heat transfer simulations) and introduces computation time prediction models and memory usage prediction models to predict resource consumption during job execution. Simultaneously, it constructs a comprehensive resource conflict risk assessment system by combining the GPU's frequency-latency-power characteristics and the resource scheduling overhead of the simulation platform. When potential conflicts are detected, Monte Carlo tree search technology is used to explore multiple scheduling strategies, and the scheduling scheme that achieves the optimal balance between computational latency, throughput, and energy consumption is selected. This invention comprehensively analyzes historical data and real-time job requirements of the simulation platform, achieving accurate prediction and dynamic allocation of heterogeneous GPU resources. This effectively improves the utilization rate and overall performance of heterogeneous GPU resources in multi-simulation job mixed execution environments, providing support for efficient resource scheduling in complex simulation scenarios. Attached Figure Description

[0039] Figure 1 This is a flowchart of a GPU heterogeneous resource scheduling method based on the mixed execution of multiple simulation jobs. Detailed Implementation

[0040] The present invention will now be described in further detail with reference to the embodiments and accompanying drawings. However, this should not be construed as limiting the scope of the above-described subject matter of the present invention to the following embodiments; all technologies implemented based on the content of the present invention fall within the scope of the present invention.

[0041] This invention provides a GPU heterogeneous resource scheduling method based on the mixed execution of multiple simulation jobs. It addresses the problem of inefficient GPU resource allocation in multi-task heterogeneous environments, where the heterogeneous environment refers to the mixed deployment of NVIDIA GPUs and AMD GPUs, or the mixed deployment of NVIDIA GPUs, AMD GPUs, and domestically produced GPUs, within a simulation platform. The scheduling method includes the following steps:

[0042] Step 1: Obtain historical simulation job data from the simulation platform using the perf performance analysis software. This historical simulation job data includes the number of mesh cells, floating-point operations, computation time, initial memory allocation, and the memory usage per second, theoretical peak GPU computing power, and resource scheduling overhead time constant of each simulation task. The simulation tasks include CFD simulation tasks, combustion simulation tasks, structural simulation tasks, and heat transfer simulation tasks. In a single simulation job, the sum of the floating-point operations of all simulation tasks equals the total floating-point operations of the simulation job; the sum of the computation time of all simulation tasks equals the computation time of the simulation job; the sum of the initial memory allocation of all simulation tasks equals the initial memory allocation of the simulation job; and the sum of the dynamic memory growth rates of all simulation tasks equals the dynamic memory growth rate of the simulation job.

[0043] Step 2: (1) Using the floating-point computation amount of each simulation task in the historical simulation job as input and the calculation time of the historical simulation job as output, fit and construct a simulation job calculation time prediction model. The specific construction method includes:

[0044] Step 201: The floating-point computational complexity of several CFD simulation tasks The theoretical peak computing power of the corresponding GPU Resource scheduling overhead time constant As input, the computation time for each of the CFD simulation tasks. For the output, a fitting is constructed based on the first calibration coefficient. The function for predicting the computation time of a CFD simulation task is expressed as follows: .

[0045] Step 202: The floating-point computational complexity of several combustion simulation tasks The theoretical peak computing power of the corresponding GPU Resource scheduling overhead time constant As input, the computation time for each of the combustion simulation tasks. For the output, a fitting is constructed based on the second calibration coefficient. The calculation time prediction function for the combustion simulation task is expressed as follows: .

[0046] Step 203: Calculate the floating-point computational complexity of several structural simulation tasks. The theoretical peak computing power of the corresponding GPU Resource scheduling overhead time constant As input, the computation time for each of the structural simulation tasks. For the output, a fitting is constructed based on the third calibration coefficient. The function for predicting the computation time of the structural simulation task is expressed as follows: .

[0047] Step 204: The floating-point computational complexity of several heat transfer simulation tasks The theoretical peak computing power of the corresponding GPU Resource scheduling overhead time constant As input, the computation time for each of the heat transfer simulation tasks. For the output, a fitting is constructed based on the fourth calibration coefficient. The function for predicting the computation time of the heat transfer simulation task is expressed as: .

[0048] Step 205: Add the CFD simulation task calculation time prediction function based on the first calibration coefficient, the combustion simulation task calculation time prediction function based on the second calibration coefficient, the structural simulation task calculation time prediction function based on the third calibration coefficient, and the heat transfer simulation task calculation time prediction function based on the fourth calibration coefficient to form the simulation operation calculation time prediction model, the expression of which is: In the formula, This is the predicted computation time for the simulation task. This refers to the predicted computation time for the CFD simulation task in the simulation operation. This is the predicted computation time for the combustion simulation task in the aforementioned simulation operation. This is the predicted computation time for the structural simulation task in the simulation operation. This is the predicted computation time for the heat transfer simulation task in the simulation operation.

[0049] (2) Based on the initial memory allocation of each simulation task in the historical simulation operation, and the execution of the simulation task in the first... The memory usage and mesh cell count per second are used as inputs, and the result is calculated based on the execution time of each simulation task. The memory usage per second is used as the output. A model for predicting memory usage in simulation jobs is constructed by fitting the data. The specific construction method includes:

[0050] Step 211: Use the perf performance analysis software to capture the observed memory usage and number of mesh cells per second for each simulation task in each historical simulation job. Specifically, for CFD simulation tasks in the simulation job, use the perf performance analysis software to capture the initial memory allocation at the start of the CFD simulation task, the observed memory usage per second during execution, and the number of mesh cells. The same applies to combustion simulation tasks, structural simulation tasks, and heat transfer simulation tasks.

[0051] Step 212: The initial memory allocation at startup for several CFD simulation tasks, and the... The observed memory usage per second and the number of mesh cells are used as inputs for each CFD simulation task. The observed video memory usage per second is output, and a prediction function for video memory usage in the CFD simulation task is constructed by least squares regression fitting. The expression is:

[0052] ;

[0053] in: For the first CFD simulation task execution +1 second of predicted video memory usage; For the first CFD simulation task execution Video memory usage per second; The initial memory allocation when a CFD simulation task starts is equal to the amount of video memory allocated during the CFD simulation task. Memory usage observations per second; The memory usage per second for each mesh cell corresponding to the CFD simulation task was obtained through fitting. The number of mesh cells corresponding to the CFD simulation task; The first autoregressive coefficient corresponding to the CFD simulation task is obtained through fitting. These are the exogenous input coefficients corresponding to the CFD simulation task, obtained through fitting. The intercept term corresponding to the CFD simulation task is obtained through fitting. The white noise error corresponding to the CFD simulation task is obtained through fitting.

[0054] Similarly, for combustion simulation, structural simulation, and heat transfer simulation tasks, memory usage prediction functions for each task are constructed using least squares regression. The only difference between these functions and the memory usage prediction function for CFD simulation tasks is that the combustion simulation task memory usage prediction function uses autoregressive coefficients. Exogenous input coefficients Memory usage per grid cell per second Intercept term White noise error The memory usage prediction function for structural simulation tasks uses autoregressive coefficients. Exogenous input coefficients Memory usage per grid cell per second Intercept term White noise error The memory usage prediction function for the heat transfer simulation task uses autoregressive coefficients. Exogenous input coefficients Memory usage per grid cell per second Intercept term White noise error .

[0055] Step 213: For each simulation task in the historical simulation jobs, in the [number]th [stage]... The memory usage prediction functions for each second are summed to obtain the historical simulation job in the [number]th second. The memory usage prediction model for each second, i.e., the simulation job in the second... The expression for the memory usage prediction model per second is:

[0056] ;

[0057] In the formula: For the simulation operation in the first Predicted memory usage per second. For the same simulation job, the CFD simulation task in the first... Predicted memory usage per second. For the same simulation job, the combustion simulation task in the first... Predicted memory usage per second. For the structural simulation task in the same simulation job, in the first... Predicted memory usage per second. For the heat transfer simulation task in the same simulation job, in the first... Predicted memory usage per second.

[0058] (3) For each GPU of the simulation platform, run selected representative simulation cases at different preset GPU operating frequencies, measure the power consumption and latency of each GPU when running at the preset GPU operating frequency, record each preset GPU operating frequency and its corresponding power consumption and latency as triples, and store them as a lookup table; specifically including the following steps:

[0059] Step 221: For each GPU on the simulation platform, set a list of preset GPU operating frequencies for the frequency sweep experiment. , , ,..., The preset GPU operating frequencies in the frequency list cover the main operating frequencies supported by the GPU.

[0060] Step 222: For each GPU, run the selected representative simulation test cases sequentially according to the preset GPU operating frequencies in the frequency list, and use the perf performance analysis software to collect the power consumption of each GPU at each preset GPU operating frequency. and delay Then record as The triples are stored as a LookupTable. It should be noted that the representative simulation cases can be CFD simulation tasks with medium grid sizes or combustion simulations using typical chemical reaction mechanisms to avoid excessive deviations between no-load data and actual loads.

[0061] (4) Construct a simulation task resource configuration baseline table for rapid resource allocation decisions in the absence of resource conflicts. This includes the following steps:

[0062] Step 231: For each of the CFD simulation, combustion simulation, structural simulation, and heat transfer simulation tasks, select several existing representative examples. These representative examples should cover small, medium, and large mesh sizes. Perform scanning tests on each GPU of the simulation platform. When testing each simulation task, fix the GPU operating frequency as the base frequency, and then gradually increase the allocated number of SM / CUs, for example, from 8 SM computing units to the maximum number of SMs on the GPU. Record the computation time and SM utilization rate of the simulation task under each number of SMs. When the computation time reduction due to further increasing the number of SMs is less than 5%, record the number of SMs at this point and use it as the recommended number of SMs allocated for that simulation task at that mesh size.

[0063] Step 232: For a representative example of each simulation task, under the recommended SM allocation quantity, traverse the list of preset GPU operating frequencies in the LookupTable and record the computation time at each preset GPU operating frequency. and power consumption Calculate the energy efficiency ratio ,in The floating-point computation load for the corresponding simulation task is determined. Finally, the frequency with the highest energy efficiency is selected as the recommended GPU operating frequency for this simulation task.

[0064] Step 233: Set default layout strategies for CFD simulation tasks, combustion simulation tasks, structural simulation tasks, and heat transfer simulation tasks respectively: (1) CFD simulation tasks adopt a continuous SM layout strategy, giving priority to allocating physically adjacent SM units to reduce cross-SM data exchange and thread synchronization overhead; (2) Combustion simulation tasks adopt a memory buffer reservation strategy, reserving 10%~15% of memory margin to cope with the dynamic memory allocation of chemical reaction mechanisms; (3) Structural simulation tasks adopt a high-bandwidth SM priority strategy, giving priority to allocating SM units close to the HBM controller to meet the memory access bandwidth requirements of large-scale sparse matrix solving; (4) Heat transfer simulation tasks: adopt a balanced distribution strategy, evenly distributing SMs in different partitions of the GPU to balance the thermal load.

[0065] Step 234: Store the above test results as a simulation task resource configuration baseline table for later querying.

[0066] Step 3: Obtain the floating-point computation amount of each simulation task in each simulation job to be scheduled from the configuration file of each simulation job to be scheduled on the simulation platform. Using the floating-point computation amount of each simulation task in each simulation job to be scheduled as input, analyze and predict the computation time value of each simulation job to be scheduled. .

[0067] Obtain the number of grid cells and the required initial memory allocation for each simulation task in each currently scheduled simulation job from the configuration file of each job on the simulation platform. And based on the number of grid cells for each simulation task in each currently scheduled simulation job and the required initial memory allocation. Using the aforementioned memory usage prediction model as input, the system analyzes and predicts the memory usage per second for each scheduled simulation job during execution.

[0068] From the configuration file of each simulation job to be scheduled in the simulation platform, obtain the GPU operating frequency required for each simulation task in each simulation job to be scheduled. Using the GPU operating frequency as the search condition, query the LookupTable to find the power consumption and latency of each GPU in the simulation platform at the GPU operating frequency.

[0069] Step 4: Obtain the current timestamp of the simulation platform and the timestamp corresponding to the preset deadline for submitting simulation jobs to the simulation platform. Combine this with the total floating-point computation, power consumption, and predicted computation time for each simulation job to be scheduled, calculate the real-time priority of each simulation job to be scheduled. The calculation formula is as follows:

[0070] ;

[0071] in: The real-time priority of the simulation jobs to be scheduled; , , Preset weighting coefficients are used to reflect the relative importance of "timeliness", "flexibility", and "energy efficiency" respectively. , , The value can be adjusted according to actual needs; The timestamp corresponding to the preset deadline for submitting the simulation job to the simulation platform; This is the current timestamp of the simulation platform; The predicted computation time for the simulation job to be scheduled; The total floating-point computation of the simulation jobs to be scheduled is equal to the sum of the floating-point computation of each simulation task in the simulation jobs to be scheduled. The power consumption of the simulation job to be scheduled; It equals the sum of the power consumption of each simulation task in the simulation job at the corresponding GPU operating frequency. The power consumption value of each simulation task is obtained through the LookupTable. In order to be in , Take the maximum value between the two.

[0072] It should be noted that, based on the predicted memory usage per second for each scheduled simulation job during execution, if the peak value of the predicted memory usage exceeds a preset available memory threshold, the real-time priority of the corresponding scheduled simulation job will be forcibly adjusted. Place it at the lowest level.

[0073] Step 5: Based on the total floating-point computation load of each simulation job to be scheduled Calculate the predicted time value The number of grid cells for each simulation task in each simulation job to be scheduled and the required initial allocation of video memory The system calculates the predicted per-second memory usage for each scheduled simulation job, the GPU operating frequency and corresponding power consumption and latency of each simulation task within each job, and constructs a virtualization environment using gem5-GPU and a virtual memory block model. In this virtualization environment, all scheduled simulation jobs are simulated in a mixed execution environment to predict whether there are resource conflict risks on the simulation platform. If no conflict risk exists, a baseline table-based fast allocation strategy is used to determine the scheduling scheme. Otherwise, Monte Carlo tree search is used to generate multiple proposed scheduling schemes combining migration / precision / dynamic voltage-frequency adjustment (DVFS). The scheme that satisfies the Pareto optimal solution for computational latency, throughput, and energy consumption is selected as the scheduling scheme. The resource conflict risks include memory overflow and / or SM / CU utilization saturation. The GPU computing units include SM computing units and CU computing units. The migration / precision / dynamic voltage frequency adjustment DVFS combination refers to the action space of Monte Carlo tree search, which includes three scheduling strategies: (1) Migration strategy: migrate simulation tasks from resource-constrained GPUs to idle GPUs, or perform cross-architecture migration between NVIDIA GPUs, AMD GPUs, and domestic GPUs; (2) Precision strategy: use mixed precision computing, such as FP16 / FP32 hybrid, for computationally intensive simulation tasks to reduce memory usage and computational load; (3) DVFS strategy: dynamically adjust the GPU operating frequency to find a balance between performance and power consumption. The scheduling scheme includes the computing power allocation scheme, GPU operating frequency scheme, and layout strategy for each simulation task in the simulation job to be scheduled. The computing power allocation scheme refers to the number of SM / CU computing power allocated to each simulation task in the simulation job to be scheduled; the GPU operating frequency scheme refers to the GPU operating frequency corresponding to each simulation task; the layout strategy refers to the layout strategy of each simulation task on the GPU, including continuous SM layout, memory buffer reservation, etc. For example, in the final scheduling scheme, for a certain simulation job to be scheduled, the CFD simulation task is allocated 32 SM computing units, and priority is given to ensuring continuous SM layout to reduce thread synchronization overhead, and the GPU operating frequency is kept at the base frequency (×1.0); the combustion simulation task is allocated 16 SM computing units, and the memory buffer is reserved, and the GPU operating frequency is reduced to 0.9 times the base frequency; the structural simulation task is allocated 20 SM computing units, and high-bandwidth SMs are given priority. During the iterative steady state phase, the frequency can be reduced to 0.85 times the base frequency to save energy, and then temporarily increased again during the convergence critical period.

[0074] It should be noted that if the simulation platform does not have the risk of resource conflicts, the steps for determining the scheduling scheme using the baseline table quick allocation strategy are as follows:

[0075] Step 501: Sort the simulation jobs to be scheduled from high to low according to their real-time priority to form a queue of jobs to be scheduled.

[0076] Step 502: For the current simulation job to be scheduled in the job queue, based on each simulation task (CFD, combustion, structure, heat transfer) in the simulation job, according to its task type and mesh size level, query the corresponding recommended number of SM / CU, recommended GPU operating frequency and recommended layout strategy in the simulation task resource configuration baseline table.

[0077] Step 503: Based on the currently available GPU resources on the simulation platform, allocate SM / CU computing units to each simulation task of the currently scheduled simulation job. The allocation principle is as follows:

[0078] Prioritize assigning multiple simulation tasks of the same simulation job to the same GPU to reduce cross-GPU data transfer overhead;

[0079] If the number of remaining SM / CUs on a single GPU is insufficient, the simulation task will be assigned to another available GPU.

[0080] The allocation follows the layout strategy specified in the baseline table.

[0081] Step 504: Update the GPU resource usage status of the simulation platform, process the next simulation job, until the queue of jobs to be scheduled is empty;

[0082] Step 505: Output the scheduling scheme, which includes the GPU number, SM / CU allocation quantity, GPU operating frequency, and layout strategy for each simulation task in each simulation job.

[0083] Step Six: Based on the computing power allocation scheme, GPU operating frequency scheme, and layout strategy in the scheduling scheme, allocate SM computing units / CU computing units and determine the GPU operating frequency for each simulation task in the current simulation job to be scheduled. Then, the KubernetesDevice-Plugin calls NVIDIAMIG or AMDMxGPU to create SM / CU sub-instances corresponding to each simulation task in the current simulation job to be scheduled, binds them to the corresponding job container, and then executes the simulation according to the real-time priority of the current simulation job to be scheduled.

[0084] Step 7: Status monitoring and feedback, which involves periodically saving the simulation job context to CPU memory and quickly migrating to backup devices in case of GPU failure. In multi-task hybrid simulation scenarios, based on the current simulation job's floating-point operation volume, predicted memory usage, predicted computation time, and future simulation job plans, the peak load of future simulation jobs is predicted, automatically triggering GPU node expansion in the heterogeneous GPU environment.

[0085] This invention considers the computational characteristics of different simulation tasks (such as CFD, combustion, structural, and heat transfer simulations), and introduces computation time prediction models and memory usage prediction models to predict resource consumption during job execution in advance, achieving accurate characterization of resource requirements in complex multiphysics simulation scenarios. Simultaneously, combining the frequency-latency-power characteristics of GPUs and the resource scheduling overhead of the simulation platform, a comprehensive resource conflict risk assessment system is constructed. When potential conflicts are detected, Monte Carlo tree search technology is used to explore multiple scheduling strategies, and the scheduling scheme that achieves the optimal balance between computational latency, throughput, and energy consumption is selected. This invention comprehensively analyzes historical data and real-time job requirements of the simulation platform, achieving accurate prediction and dynamic allocation of heterogeneous GPU resources, effectively improving the utilization rate and overall performance of heterogeneous GPU resources in multi-simulation job mixed execution environments, and providing support for efficient resource scheduling in complex simulation scenarios.

[0086] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A GPU heterogeneous resource scheduling method based on the hybrid execution of multiple simulation jobs, characterized in that, include: The historical simulation job data of the simulation platform is obtained. The historical simulation job data includes the number of grid cells, floating-point operation volume, computing time, and initial memory allocation of each simulation task in the historical simulation job, as well as the memory usage per second, the theoretical peak computing power of the GPU occupied, and the resource scheduling overhead time constant of the simulation platform during the execution of each simulation task. The simulation tasks include CFD simulation tasks, combustion simulation tasks, structural simulation tasks, and heat transfer simulation tasks. Obtain the floating-point operation volume, number of grid cells, required initial memory allocation, and required GPU operating frequency for each simulation task in each currently scheduled simulation job; Using the floating-point computation amount of each simulation task in each simulation job to be scheduled as input, the simulation job computation time prediction model is used to analyze and predict the computation time value of each simulation job to be scheduled. Using the number of grid cells for each simulation task in each current simulation job to be scheduled and the required initial memory allocation as input, the simulation job memory usage prediction model is used to analyze and predict the memory usage per second for each current simulation job to be scheduled during execution; using the required GPU operating frequency as the search condition, the power consumption and latency of each GPU in the simulation platform at the GPU operating frequency are queried in the LookupTable containing GPU frequency-latency-power consumption presets. Obtain the timestamp corresponding to the preset deadline for submitting simulation jobs to the simulation platform, the current timestamp of the simulation platform, and combine the total floating-point operation volume, power consumption, and predicted computation time of each simulation job to be scheduled to calculate the real-time priority of each simulation job to be scheduled. Based on the total floating-point operations and predicted computation time of each scheduled simulation job, as well as the number of grid cells, the initial amount of video memory allocated, the predicted video memory usage per second during execution, the required GPU operating frequency, and the corresponding power consumption and latency of each simulation task in each scheduled simulation job, the mixed execution of all scheduled simulation jobs is simulated in a virtualized environment to predict whether there is a risk of resource conflict on the simulation platform; if there is no risk of conflict, a baseline table fast allocation strategy is used to determine the scheduling scheme. Otherwise, Monte Carlo tree search is used to generate multiple migration / precision / dynamic voltage frequency adjustment DVFS combinations of proposed scheduling schemes. Among the proposed scheduling schemes, the Pareto optimal solution that satisfies the computational latency, throughput and energy consumption is selected as the scheduling scheme. The resource conflict risks include memory overflow and / or SM / CU utilization saturation. The scheduling scheme includes the computing power allocation scheme and GPU operating frequency scheme corresponding to each simulation task in the simulation job to be scheduled. According to the scheduling scheme, SM computing units / CU computing units are allocated to each simulation task in the current simulation job to be scheduled, and the GPU operating frequency is determined. Then, the simulation is executed according to the real-time priority of the current simulation job to be scheduled.

2. The GPU heterogeneous resource scheduling method based on the hybrid execution of multiple simulation jobs according to claim 1, characterized in that, The method for constructing the simulation job calculation time prediction model includes: Using the floating-point computation volume of several CFD simulation tasks, the theoretical peak computing power of the corresponding GPU, and the resource scheduling overhead time constant as inputs, and the computation time of each CFD simulation task as output, a CFD simulation task computation time prediction function based on the first calibration coefficient is fitted and constructed. Using the floating-point computation amount of several combustion simulation tasks, the theoretical peak computing power of the corresponding GPU, and the resource scheduling overhead time constant as inputs, and the computation time of each combustion simulation task as output, a combustion simulation task computation time prediction function based on the second calibration coefficient is fitted and constructed. Using the floating-point computation volume of several structural simulation tasks, the theoretical peak computing power of the corresponding GPU, and the resource scheduling overhead time constant as inputs, and the computation time of each structural simulation task as output, a structural simulation task computation time prediction function based on a third calibration coefficient is fitted and constructed. Using the floating-point computation amount of several heat transfer simulation tasks, the theoretical peak computing power of the corresponding GPU, and the resource scheduling overhead time constant as inputs, and the computation time of each heat transfer simulation task as output, a heat transfer simulation task computation time prediction function based on the fourth calibration coefficient is fitted and constructed. The simulation task calculation time prediction model is formed by adding the CFD simulation task calculation time prediction function based on the first calibration coefficient, the combustion simulation task calculation time prediction function based on the second calibration coefficient, the structural simulation task calculation time prediction function based on the third calibration coefficient, and the heat transfer simulation task calculation time prediction function based on the fourth calibration coefficient.

3. The GPU heterogeneous resource scheduling method based on the hybrid execution of multiple simulation jobs according to claim 2, characterized in that, The CFD simulation task computation time prediction function based on the first calibration coefficient is constructed as follows: ； The combustion simulation task calculation time prediction function based on the second calibration coefficient is constructed as follows: ； The computation time prediction function for the structural simulation task based on the third calibration coefficient is constructed as follows: ； The heat transfer simulation task calculation time prediction function based on the fourth calibration coefficient is constructed as follows: ； in, Calculate the time for CFD simulation tasks. The first calibration coefficient, The floating-point computational complexity of CFD simulation tasks. This represents the theoretical peak computing power of the GPU used during the execution of a CFD simulation task. The calculation time is for the combustion simulation task. This is the second calibration coefficient. The floating-point computational load for combustion simulation tasks, This represents the theoretical peak computing power of the GPU used during the execution of the combustion simulation task. This refers to the computation time for the structural simulation task. This is the third calibration coefficient. This refers to the floating-point computational complexity of the structural simulation task. This represents the theoretical peak computing power of the GPU used during the execution of the structural simulation task. The computation time for the heat transfer simulation task. This is the fourth calibration coefficient. The floating-point computational cost of the heat transfer simulation task. This represents the theoretical peak computing power of the GPU used during the execution of the heat transfer simulation task. This is the time constant for resource scheduling overhead of the simulation platform during simulation task execution.

4. The GPU heterogeneous resource scheduling method based on the hybrid execution of multiple simulation jobs according to claim 3, characterized in that, The simulation job memory usage prediction model is constructed as follows: ； in, For the simulation operation in the first Predicted memory usage per second. For the same simulation job, the CFD simulation task in the first... Predicted memory usage per second. For the same simulation job, the combustion simulation task in the first... Predicted memory usage per second. For the structural simulation task in the same simulation job, in the first... Predicted memory usage per second. For the heat transfer simulation task in the same simulation job, in the first... Predicted memory usage per second.

5. The GPU heterogeneous resource scheduling method based on the hybrid execution of multiple simulation jobs according to claim 4, characterized in that, In the same simulation job, the CFD simulation task is in the first The predicted memory usage per second is calculated using the following formula: ； in: For the first CFD simulation task execution +1 second of predicted video memory usage; For the first CFD simulation task execution Video memory usage per second; This is the initial memory allocation when a CFD simulation task starts. The video memory usage per second for each mesh cell in a CFD simulation task; The number of mesh cells for a CFD simulation task; The first autoregressive coefficient; These are exogenous input coefficients; For the intercept term; This is white noise error.

6. The GPU heterogeneous resource scheduling method based on the hybrid execution of multiple simulation jobs according to claim 5, characterized in that, The current real-time priority calculation formula for each simulation job to be scheduled is as follows: ； in: The real-time priority of the simulation jobs to be scheduled; , , Preset weighting coefficients; The timestamp corresponding to the preset deadline for submitting the simulation job to the simulation platform; This is the current timestamp of the simulation platform; The predicted computation time for the simulation job to be scheduled; The total floating-point computation of the simulation jobs to be scheduled; The power consumption of the simulation job to be scheduled; In order to be in , Take the maximum value between the two.

7. The GPU heterogeneous resource scheduling method based on the hybrid execution of multiple simulation jobs according to claim 6, characterized in that, Based on the predicted memory usage per second for each scheduled simulation job, if the peak value of the predicted memory usage exceeds a preset available memory threshold, the real-time priority of the corresponding scheduled simulation job will be forcibly adjusted. Place it at the lowest level.

8. The GPU heterogeneous resource scheduling method based on the hybrid execution of multiple simulation jobs according to claim 7, characterized in that, The method for determining the LookupTable, which includes GPU frequency-latency-power presets, includes: For each GPU in the simulation platform, selected representative simulation cases are run at different preset GPU operating frequencies. The power consumption and latency of each GPU are measured when running at the preset GPU operating frequency. Each preset GPU operating frequency and its corresponding power consumption and latency are recorded as triples and stored as a lookup table.

Images