GEMM load-oriented GPU modeling method

Abstract

A GEMM load-oriented GPU modeling method is characterized in that through a multi-stage collaborative modeling mechanism, cache behaviors, instruction overhead and calculation intensity are deeply coupled, accurate performance prediction of GPU execution GEMM operators is realized, the method can be widely applied to scheduling optimization of GPU intensive scenes such as AI training and scientific calculation, firstly, a three-stage cache weight distribution mechanism is established, and then, a three-stage cache weight distribution mechanism is established; quantifying the contribution of the L1 / L2 cache hit rate and the DRAM bandwidth degradation factor to the effective bandwidth; secondly, an instruction-level memory access overhead correction mechanism is introduced, and the mixing precision and the real calculation strength of a sparse calculation scene are captured through dynamic parameter adjustment and optimization; then combining the calculation force peak value and the bandwidth upper limit to construct a double-boundary constraint model, and generating a theoretical performance critical value; further predicting a stream multiprocessor utilization rate based on a neural network, and quantifying efficiency loss caused by hardware resource contention through a multi-layer perceptron structure; and finally, the integration module outputs task execution time to realize end-to-end performance prediction.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a GPU modeling method for GEMM workloads. Background Technology

[0002] In the deep learning technology system, the graphics processing unit (GPU) serves as the core computing power carrier for accelerating the training and inference of deep neural networks (DNNs), playing an irreplaceable supporting role in the engineering implementation of large Transformer architecture models (such as Llama and BERT). In the massive computational load of these models, the general matrix multiplication (GEMM) operator accounts for over 75% of the time consumption; for example, it reaches 78.2% in the ResNet-50 inference task and 81.5% in BERT-large training, becoming the performance bottleneck for end-to-end execution time.

[0003] Existing research employs a neural network-based operator modeling approach. This method deconstructs a Directed Neural Network (DNN) into a set of operators, constructs an end-to-end prediction model, and trains this set using neural network algorithms to model the DNN load. This avoids the cost of real hardware execution and significantly shortens the modeling cycle by over 60%. However, this approach completely decouples the physical meaning of GPU hardware, causing prediction errors to amplify to over ±20% during cross-architecture migration. Furthermore, extrapolation fails when encountering novel operators not covered in the training set, with errors exceeding 40%. This severely restricts agile optimization of large model architectures. There is an urgent need to establish a new modeling paradigm that integrates physical hardware principles with data-driven learning to achieve efficient performance prediction of GEMM operators. Summary of the Invention

[0004] This invention provides a GPU modeling method for GEMM operators, which embeds cache hit rate and memory access instruction behavior into the inference model within the modeling path of computational intensity and effective bandwidth. This improves the performance prediction of GPUs for GEMM operators.

[0005] The technical solution of the present invention is as follows:

[0006] A GPU modeling method for GEMM workloads, characterized by the following steps:

[0007] Step 1, Cache-weighted bandwidth calculation model: L1 cache hit rate H of the function to be predicted on the GPU L1 L2 cache hit rate H L2 The DRAM bandwidth degradation factor ηDRAM and the nominal memory bandwidth mem_bw are used to calculate the effective memory bandwidth MemBw through a three-level cache weighted model.

[0008] Step 2, Instruction-Aware Arithmetic Strength Modeling: Based on computational load TileOps, total data volume TileMem, and number of read instructions R inst Write instruction count W inst And the instruction overhead conversion coefficients λ1 and λ2, the effective arithmetic strength I is calculated by introducing the instruction overhead computational strength correction model;

[0009] Step 3, Performance ceiling modeling: Combining MemBw obtained in Step 1 and I obtained in Step 2, calculate the theoretical performance ceiling RooflineBW based on the constraint relationship between peak computing power Flops and bandwidth ceiling;

[0010] Step 4, Dynamic Utilization Modeling of Streaming Multiprocessors: Based on the number of parallel execution units NumWave, calculate the actual hardware utilization util using the resource utilization function;

[0011] Step 5, Task execution time prediction: Based on the RooflineBW from Step 3, the util from Step 4, and the parallel load parameters, output the total task latency Td.

[0012] The cache-weighted bandwidth calculation model is as follows:

[0013] MenBw = mem_bw * (H L1 +(1-H L1 )*H L2 +(1-H L1 (1-H) L2 ))ηDRAM

[0014] Among them, H L1 H L2 The cache hit rates for L1 and L2 are respectively obtained from the feature vectors in the dataset, and ηDRAM is the DRAM bandwidth degradation factor.

[0015] The arithmetic strength modeling of the instruction-awareness satisfies:

[0016]

[0017] Among them, R inst W inst λ1 and λ2 are the number of read / write instructions, respectively, and the instruction overhead conversion coefficients are adjusted by the backpropagation algorithm of machine learning.

[0018] The performance upper limit modeling satisfies:

[0019] RooflineBW=min(Flops,I*MemBw)

[0020] By calculating the peak computing power, as well as the constraints on arithmetic strength and bandwidth limits, and then calculating the minimum of the two, the upper limit of performance is limited.

[0021] The dynamic utilization modeling of the streaming multiprocessor satisfies:

[0022] util=R(x)

[0023] Where x is the feature vector, including but not limited to the matrix operation dimensions (M, N, K) and the number of read instructions R. inst Number of write instructions W inst L1 cache hit rate H L1 L2 cache hit rate H L2 ;

[0024] R(x) is a neural network modeling method that includes an input layer for normalizing input features, bringing the feature vector to the range [-1, 1]; a fully connected layer with 16 hidden layers, each containing 128 neurons, for fine-grained feature extraction, and employing the ReLU activation function to enhance model fit; and an output layer that calculates the difference between the predicted and true values ​​using a loss function, and corrects the modeling values ​​using backpropagation based on this difference, outputting the predicted value during the prediction phase. This method aims to achieve efficient model training and improve the model's prediction accuracy.

[0025] The overall modeling for predicting task execution time satisfies:

[0026]

[0027] Where NumWave is the number of blocks in the GPU computation, and OpsPerWave is the number of operations performed on each block;

[0028] They respectively satisfy:

[0029]

[0030] OpsPerWave=2*M*N*K*Num_Sm*1e-9,

[0031] Where M, N, and K are the matrix dimensions, Num_Sm is the number of streaming multiprocessors of the measured GPU, and B is the batch size, all obtained from the dataset. tile_m and tile_n are the block sizes in the M / N dimensions, respectively, obtained through program definition.

[0032] The training process of the neural network modeling method includes:

[0033] (1) Collect real GPU hardware execution datasets, including feature vectors x and corresponding real latency values;

[0034] (2) Forward propagation calculation of prediction delay Td:

[0035]

[0036] (3) Calculate the error using the loss function Loss:

[0037] Loss = |Predicted value - Actual value| 2 ;

[0038] (4) Update the neural network weights and parameters λ1, λ1, and ηDRAM using the backpropagation algorithm;

[0039] (5) Repeat the iteration until the loss function converges.

[0040] The technical effects of this invention are as follows: This invention provides a GPU modeling method for GEMM workloads. Through a multi-level collaborative modeling mechanism, it deeply couples cache behavior, instruction overhead, and computational intensity to achieve accurate performance prediction of GPU execution of GEMM operators. It can be widely applied to scheduling optimization in GPU-intensive scenarios such as AI training and scientific computing. First, a three-level cache weight allocation mechanism is established to quantify the contribution of L1 / L2 cache hit rate and DRAM bandwidth degradation factor to effective bandwidth. Second, an instruction-level memory access overhead correction mechanism is introduced to capture the true computational intensity of mixed precision and sparse computing scenarios through dynamic parameter tuning. Then, a dual-boundary constraint model is constructed by combining peak computing power and bandwidth limit to generate theoretical performance critical values. Furthermore, based on neural network prediction of streaming multiprocessor utilization, the efficiency loss caused by hardware resource contention is quantified through a multi-layer perceptron structure. Finally, the integrated module outputs the task execution time to achieve end-to-end performance prediction. Attached Figure Description

[0041] Figure 1 This is a schematic flowchart of the GEMM performance modeling method provided in an exemplary embodiment of this application. Specific implementation methods

[0042] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] The following is in conjunction with the attached diagram ( Figure 1 The invention will be described in the following sections and examples.

[0044] This application provides a GPU modeling method for GEMM (Generalized Matrix Multiplication) workloads. This method deeply couples cache behavior, instruction overhead, and computational intensity through a multi-level collaborative modeling mechanism, achieving accurate performance prediction of GPU execution of GEMM operators. First, a three-level cache weight allocation mechanism is established to quantify the contribution of L1 / L2 cache hit rate and DRAM bandwidth degradation factor to effective bandwidth. Second, an instruction-level memory access overhead correction mechanism is introduced to capture the true computational intensity of mixed-precision and sparse computing scenarios through dynamic parameter tuning. Then, a dual-boundary constraint model is constructed by combining peak computing power and bandwidth limits to generate theoretical performance critical values. Furthermore, based on neural network prediction of streaming multiprocessor utilization, the efficiency loss caused by hardware resource contention is quantified through a multi-layer perceptron structure. Finally, the integrated module outputs the task execution time, achieving end-to-end performance prediction. This method breaks through the simplification assumptions of traditional models, significantly improving the prediction accuracy for complex scenarios such as mixed precision and sparse matrices, and can be widely applied to scheduling optimization in GPU-intensive scenarios such as AI training and scientific computing.

[0045] This application provides a GPU modeling method for GEMM operators, which embeds cache hit rate and memory access instruction behavior into the inference model within the modeling path of computational intensity and effective bandwidth. This is used to improve the performance prediction of GPUs for GEMM operators.

[0046] The simulator is mainly composed of cache-weighted bandwidth calculation modeling, instruction-aware arithmetic strength modeling, performance ceiling modeling, streaming multiprocessor dynamic utilization modeling, and task execution time prediction.

[0047] In one feasible approach design, the simulator includes cache bandwidth computation modeling, instruction-aware arithmetic strength modeling, performance ceiling modeling, and streaming multiprocessor dynamic utilization modeling.

[0048] Cache bandwidth calculation modeling uses a three-level cache weight allocation mechanism to accurately quantify the actual contribution of different storage levels to effective memory bandwidth, thereby achieving accurate cache modeling and satisfying the following:

[0049] MemBw=mem_bw*(H L1 +(1-H L1 )*H L2 +(1-H L1 (1-HL2))ηDRAM

[0050] Among them, H L1 H L2 The cache hit rates for L1 and L2 are respectively obtained from the feature vectors in the dataset, and ηDRAM is the DRAM bandwidth degradation factor, which is tuned by the backpropagation algorithm.

[0051] Instruction-aware arithmetic strength modeling accurately represents the true computational strength of operators such as mixed-precision and sparse computation by dynamically quantifying instruction-level memory access overhead and adjusting mechanisms, satisfying the following:

[0052]

[0053] Among them, R inst W ins t represents the number of read / write instructions, obtained from the feature vectors in the dataset. λ1 and λ2 are instruction overhead conversion coefficients, which are corrected using the backpropagation algorithm of machine learning. λ1R inst The latency during quantization read commands, λ2W inst Used to capture buffer races during write instructions. TileOps represents the total number of computational operations performed within each block, used to quantify the number of floating-point operations required for matrix multiplication, and satisfies:

[0054] TileOps = 2 * M * N * K * 1e-9

[0055] M, N, and K are the dimensions of the matrix operation. The calculation result is divided by 1e-9 to convert to GFLOPS, and the calculation unit is unified.

[0056] TileMem represents the total amount of memory access required for each block, including the amount of data read and written to the input and output matrices, and it satisfies the following:

[0057] TileMem = 4*(M*N + N*K + M*K) / 2 20

[0058] Performance ceiling modeling, through dual boundaries of physical constraints and peak computing power, accurately quantifies the theoretical peak performance of GPUs executing GEMM operators, providing critical criteria for dynamic resource scheduling, which satisfies:

[0059] RooflineBW=min(Flops,I*MemBw)

[0060] Flops represents the measured nominal peak computing power of the GPU, I represents the result of the instruction-aware arithmetic strength modeling above, and MemBw represents the result of the cache bandwidth calculation modeling above. By calculating the peak computing power and the arithmetic strength and bandwidth upper limit constraints, the minimum value between the two is calculated to limit the performance upper limit.

[0061] Dynamic utilization modeling of streaming multiprocessors quantifies the efficiency loss caused by GPU hardware resource contention through neural networks, solving the load imbalance problem that traditional linear models cannot capture, and achieving accurate prediction of GPU computing power utilization. It satisfies the following:

[0062] util=R(x)

[0063] Where x is the feature vector, including but not limited to the matrix operation dimensions (M, N, K) and the number of read instructions R. inst Number of write instructions W inst L1 cache hit rate H L1 L2 cache hit rate H L2 .

[0064] R(x) is a neural network modeling method that includes an input layer for normalizing input features, bringing the feature vector to the range [-1, 1]; a fully connected layer with 16 hidden layers, each containing 128 neurons, for fine-grained feature extraction, and employing the ReLU activation function to enhance model fit; and an output layer that calculates the difference between the predicted and true values ​​using a loss function, and corrects the modeling values ​​using backpropagation based on this difference, outputting the predicted value during the prediction phase. This method aims to achieve efficient model training and improve the model's prediction accuracy.

[0065] The overall formula for GPU modeling for GEMM loads is as follows:

[0066]

[0067] Where NumWave is the number of blocks in the GPU computation, and OpsPerWave is the number of operations performed on each block.

[0068] They respectively satisfy:

[0069] NumWave=(B*M / (tile_m)*N / (tile_n)) / (Num_Sm)

[0070] opsperWave=2*M*N*K*Num_Sm*1e-9,

[0071] Where M, N, and K are the matrix dimensions, Num_Sm is the number of streaming multiprocessors of the measured GPU, and B is the batch size, all obtained from the dataset. tile_m and tile_n are the block sizes in the M / N dimensions, respectively, obtained through program definition.

[0072] In actual implementation, the training process of the streaming multiprocessor dynamic utilization modeling method is executed only once, and its training process roughly follows these steps:

[0073] (1) Collect real GPU hardware execution datasets, including feature vectors x and corresponding real latency values;

[0074] (2) Forward propagation calculation of prediction delay Td:

[0075]

[0076] (3) Calculate the error using the loss function Loss:

[0077] Loss = |Predicted value - Actual value| 2 ;

[0078] (4) Update the neural network weights and parameters λ1, λ1, and ηDRAM using the backpropagation algorithm;

[0079] (5) Repeat the iteration until the loss function converges.

[0080] Training ends when the loss function converges. The training weights and parameters are saved, and accurate prediction of unknown GPU and GEMM loads can be achieved by calling the training weights and parameters during the prediction phase.

[0081] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0082] In the technological revolution driven by artificial intelligence, large-scale deep neural networks (DNNs) have become the core engine for fields such as computer vision and natural language processing. Large models, represented by Transformers (such as GPT-4 and Llama 2), are essentially complex computational graphs composed of hundreds of billions of matrix multiplication-addition operations (GEMMs). GEMM operators account for over 75% of the time consumed during training and inference, becoming a key bottleneck for end-to-end performance. Accurate performance modeling of GEMM operators can not only provide quantitative basis for dynamic scheduling of computing power but also guide network structure optimization and hardware resource allocation, promoting the efficient deployment of large models. However, existing modeling methods based on purely data-driven approaches, due to the complete removal of the physical meaning of GPU microarchitecture, face the dual dilemma of cross-platform generalization collapse and the failure of predictions by novel operators.

[0083] To overcome the aforementioned shortcomings, this application provides a GPU modeling method for GEMM operators. This application combines a specific embodiment with... Figure 1 To illustrate the workflow of this invention, the method includes: cache-weighted bandwidth calculation modeling, instruction-aware arithmetic strength modeling, performance ceiling modeling, and streaming multiprocessor dynamic utilization modeling, ultimately achieving execution time prediction for GEMM load.

[0084] The execution of the GEMM operator in the GPU is broken down into multiple blocks, and the execution time of each block is calculated separately to achieve the calculation of the overall load.

[0085] S110, cache-weighted bandwidth calculation model.

[0086] This method is used to accurately quantify the contribution of the L1, L2, and DRAM caches in a GPU to effective memory bandwidth. It obtains the L1 cache hit rate H from the dataset. L1 L2 cache hit rate H L2 The measured GPU nominal memory bandwidth mem_bw is used to calculate the effective memory bandwidth, which satisfies the following:

[0087] MemBw=mem_bw*(H L1 +(1-H L1 )*H L2 +(1-H L1 (1-H) L2 ))ηDRAM

[0088] Where mem_bw is the measured nominal memory bandwidth of the GPU, and ηDRAM is dynamically optimized through the backpropagation algorithm to adapt to different hardware configurations.

[0089] In one feasible embodiment, H L1 =0.8, H L2 =0.7, ηDRAM=0.9, output the corrected effective bandwidth value.

[0090] S120, instruction-aware arithmetic strength modeling.

[0091] This is used for dynamically quantizing instruction-level memory access overhead, supporting real computational intensity modeling for mixed-precision and sparse computation operators, by extracting the number of read instructions R from the dataset. inst Write instruction count W inst The matrix dimensions (M, N, K) are used to calculate the total number of operations (TileOps) for each computation block.

[0092] TileMem=2*M*N*K*1e-9

[0093] The factor 2×M×N×K represents the number of floating-point operations in GEMM, and 10⁻⁹ is used for unit conversion.

[0094] Calculate the total memory access volume for each block, TileMem.

[0095] TileMem = 4*(M*K + N*K + M*K) / 2 20

[0096] Wherein, the coefficient 4 indicates that a single-precision floating-point number occupies 4 bytes, 2 20 Used to convert bytes to MB.

[0097] Calculate the arithmetic strength I, satisfying:

[0098]

[0099] Where λ1 and λ2 are instruction overhead conversion coefficients, corrected using a machine learning backpropagation algorithm. λ1R inst Quantization read instruction latency, λ2w inst Capture write instruction buffer contention.

[0100] S130, performance ceiling modeling.

[0101] By using both physical constraints and peak computing power as boundaries, the performance upper limit of GEMM is defined, providing a critical criterion for resource scheduling. The measured GPU peak computing power (Flops) is obtained from the dataset, and the performance upper limit is calculated based on the results I and MemBw from S110 and S120, as shown in the following formula:

[0102] RooflineBW=min(Flops,I*MemBw)

[0103] For example, when Flops = 10 TFLOPS and I*MemBw = 8 TFLOPS, the upper limit is 8 TFLOPS, which identifies the bandwidth bottleneck area.

[0104] S140, Dynamic Utilization Modeling of Streaming Multiprocessors.

[0105] The efficiency loss caused by GPU hardware resource contention is quantified by neural networks to solve the load imbalance problem. This involves two phases: the training phase and the prediction phase.

[0106] During the training phase, it is necessary to collect the training dataset by running the GEMM workload on a real GPU and collecting feature vectors x, including but not limited to (M, N, K, R). inst W inst H L1 H L2 The calculation delay is trained using the forward propagation algorithm, satisfying the following formula:

[0107]

[0108] Where NumWave = (B*M / (tile_m)*N / (tile_n)) / (Num_Sm) calculates the number of blocks in the GEMM load split. OpsPerWave = 2*M*N*K*Num_Sm*1e-9 calculates the number of operations per block.

[0109] util = R(x) is the output of the neural network, and x is the feature vector.

[0110] The training process is corrected using the mean squared error loss function, which satisfies the following formula:

[0111] Loss = |Predicted value - Actual value| 2

[0112] The backpropagation algorithm is used to update and correct the weights and parameters λ1, λ2, and ηDRAM of the neural network until the loss function converges to a certain value and stops decreasing.

[0113] The neural network R(x) includes an input layer for normalizing the feature vector to the [-1, 1] interval; a fully connected layer containing 16 hidden layers, each with 128 neurons, using the ReLU activation function to enhance nonlinear fitting capabilities; and an output layer with an output utilization of util.

[0114] It should be noted that the training phase described in this application only needs to be executed once and must be executed before the prediction phase.

[0115] During the prediction phase, the training weights and parameters are loaded, new load features are input, and the output util is used for performance prediction.

[0116] S150, Task execution time prediction.

[0117] Integrate the above modules to output the final predicted delay Td.

[0118] During implementation, input matrix dimensions (M, N, K), BatchSize (B), and M / N dimension tile size (tile). n , tile n The number of GPU stream multiprocessors measured (Num_Sm) and the number of GPU stream multiprocessors measured.

[0119] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0120] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0121] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0122] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0123] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0124] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A GPU modeling method for GEMM load, characterized in that, The method comprises the following steps: Step 1, Cache Weighted Bandwidth Calculation Modeling: L1 cache hit rate H of the function to be predicted on the GPU L1 , L2 cache hit rate H L2 , DRAM bandwidth degradation factor ηDRAM and nominal memory bandwidth mem_bw, effective memory bandwidth MemBw is calculated by a three-level cache weighted model; Step 2, instruction-aware arithmetic intensity modeling: based on the computation load TileOps, the total amount of data TileMem, the number of read instructions R inst , the number of write instructions W inst and instruction overhead conversion coefficients λ1, λ2, the effective arithmetic intensity I is calculated by introducing the computation intensity correction model of instruction overhead; Step 3, performance upper limit modeling: combining MemBw obtained in step 1 and I obtained in step 2, the theoretical performance upper limit RooflineBW is calculated according to the constraint relationship between the peak computing power Flops and the upper limit of the bandwidth; Step 4, modeling of dynamic utilization rate of stream multiprocessor: based on the number of parallel execution units NumWave, the actual utilization rate of hardware util is calculated through a resource utilization rate function; Step 5, task execution time prediction: according to RooflineBW in step 3, util in step 4 and parallel load parameters, the total delay Td of the task is output.

2. The method of claim 1, wherein, The cache weighted bandwidth calculation model is: MemBw = mem_bw * (H L1 +(1-H L1 )*H L2 +(1-H L1 )(1-H L2 )) η DRAM where H L1 , H L2 are the cache hit rates of L1, L2, respectively, obtained by the data-centric feature vector, and ηDRAMis the DRAM bandwidth degradation factor.

3. The method of claim 1, wherein, The instruction-aware arithmetic intensity modeling satisfies: Wherein, R inst , W inst Respectively, read / write instruction number, λ1, λ1 Instruction overhead conversion coefficient, the parameters are corrected by the back propagation algorithm of machine learning.

4. The method of claim 1, wherein, The performance upper limit modeling satisfies: RooflineBW = min(Flops, I * MemBw) By calculating the peak computing power, the arithmetic intensity and the upper limit of the bandwidth, the minimum value of the two is calculated to limit the performance upper limit.

5. The method of claim 1, wherein, The stream multiprocessor dynamic utilization rate modeling satisfies: util = R(x) Wherein, x is a feature vector, including but not limited to matrix operation dimension (M, N, K), read instruction number R inst , write instruction number W inst , L1 cache hit rate H L1 , L2 cache hit rate H L2 ; R(x) is a neural network modeling method, which includes an input layer for normalizing input features, a fully connected layer including 16 hidden layers with 128 neurons in each layer for fine feature extraction, and a ReLU activation function for enhancing the fitting degree of the model, and an output layer for calculating the difference between the predicted value and the true value through a loss function, and correcting the values in the modeling through a back propagation algorithm according to the difference, and outputting the predicted value in the prediction stage, for realizing efficient model training and improving the prediction accuracy of the model.

6. The method of claim 1, wherein, The task execution time prediction modeling satisfies: Wherein, NumWave is the number of blocks split in GPU operation, and OpsPerWave is the number of operations calculated by each block; respectively satisfy OpsPerWave = 2 * M * N * K * Num_Sm * le-9, Wherein, M, N and K are matrix dimensions, Num_Sm is the number of stream multiprocessors of the measured GPU, and B is the Batchsize size, which are obtained through a data set. tile_m and tile_n are the block sizes of M / N dimensions, which are obtained through program definition.

7. The method of claim 5, wherein, The training process of the neural network modeling method comprises: (1) collecting a real GPU hardware execution data set, including a feature vector x and a corresponding delay true value; (2) Forward propagation computes prediction delay (3) by a loss function Loss = |predicted value - true value| 2 computational error; (4) updating the neural network weights and parameters λ1, λ1 and ηDRAM through a back propagation algorithm; (5) repeating the iteration until the loss function converges.

Images