A method, apparatus and system for GPU performance prediction

Abstract

This invention discloses a method, apparatus, and system for GPU performance prediction. In this embodiment, the following steps are taken: First, the hardware parameters of the function to be executed and the GPU to be tested are obtained. Then, the function to be executed is decomposed into multiple tasks according to a set decomposition method. These tasks are then allocated to multiple streaming multiprocessors (SMs) of the GPU according to a set scheduling method, and at least one task is assigned to each SM. Finally, the at least one task assigned to each SM is assigned to multiple instruction pipelines within that SM. A multi-level feature set is determined based on the multiple instruction pipelines. The multi-level feature set is then input into a multilayer perceptron (MLP), and the predicted latency is output. Through this method, the complex workflow of the function to be executed can be mapped to a fine-grained instruction pipeline. A multi-level feature set is constructed based on the instruction pipelines, and the MLP is used to capture nonlinear competition between different instruction pipelines, accurately predicting the latency of the GPU under test and determining its performance.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically, to a method, apparatus, and system for predicting GPU performance. Background Technology

[0002] With the rapid development of Large Language Models (LLMs), their capabilities have grown exponentially, fundamentally changing the field of Artificial Intelligence (AI) and bringing about huge computing power demands. In order to meet the performance requirements of LLMs, Graphics Processing Units (GPUs) are constantly being updated and come in a wide variety. Different GPUs have different latency times when used for different LLMs, so it is necessary to test the performance of different GPUs.

[0003] Existing technologies employ periodic simulators, pure analytical models, and data-driven / grey-box methods to test GPU performance. However, these methods suffer from slow computation speed, poor accuracy, inability to model new GPU hardware, or large prediction errors.

[0004] In conclusion, how to efficiently and accurately predict GPU performance is a problem that needs to be solved. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method, apparatus and system for GPU performance prediction, which can map the complex workflow of the function to be executed into a fine instruction pipeline, construct a multi-level feature set based on the instruction pipeline, use MLP to capture nonlinear competition between different instruction pipelines, accurately predict the latency of the GPU under test, and determine the performance of the GPU under test.

[0006] In a first aspect, embodiments of the present invention provide a method for GPU performance prediction, the method comprising: acquiring a function to be executed and hardware parameters of a graphics processing unit (GPU) to be tested; decomposing the function to be executed into multiple tasks according to a set decomposition method; allocating the multiple tasks to multiple streaming multiprocessors (SMs) of the GPU according to a set scheduling method, and determining at least one task allocated to each SM; allocating the at least one task allocated to each SM to multiple instruction pipelines in the SM; determining a multi-level feature set based on the multiple instruction pipelines, wherein the multi-level feature set includes the theoretical cycle count and resource requirement feature vector at the SM level, and the theoretical cycle count and resource requirement feature vector at the GPU level; inputting the multi-level feature set into a multilayer perceptron (MLP), and outputting the predicted latency time.

[0007] Optionally, the step of decomposing the function to be executed into multiple tasks according to the set decomposition method specifically includes: determining the decomposition logic of the function to be executed by reverse engineering the source code or closed-source library of the function to be executed; and decomposing the function to be executed into multiple tasks according to the decomposition logic of the function to be executed.

[0008] Optionally, the step of allocating the multiple tasks to multiple streaming multiprocessors (SMs) of the GPU according to a set scheduling method, and determining at least one task allocated to each SM, specifically includes: allocating the multiple tasks to multiple SMs of the GPU according to a round-robin scheduling or persistent thread block scheduling method, and determining at least one task allocated to each SM.

[0009] Optionally, the step of assigning at least one task to each SM to multiple instruction pipelines in the SM specifically includes: decomposing each task assigned to the SM into multiple instructions; and assigning the multiple instructions to the corresponding multiple instruction pipelines.

[0010] Optionally, the instruction pipeline includes tensors, fused multiply-accumulate, and memory input / output.

[0011] Optionally, determining the multi-level feature set based on the multiple instruction pipelines specifically includes: calculating the primary theoretical cycle number and primary resource requirement feature vector for each task in each instruction pipeline; and determining the multi-level feature set based on the multiple primary theoretical cycle numbers and primary resource requirement feature vectors of the multiple instruction pipelines.

[0012] Optionally, determining the multi-level feature set based on the multiple primary theoretical cycle counts and primary resource requirement feature vectors of the multiple instruction pipelines specifically includes: aggregating the multiple primary theoretical cycle counts and primary resource requirement feature vectors of the multiple instruction pipelines to generate task-level theoretical cycle counts and task-level resource requirement feature vectors; aggregating the multiple task-level theoretical cycle counts and task-level resource requirement feature vectors to generate SM-level theoretical cycle counts and SM-level resource requirement feature vectors; aggregating the multiple SM-level theoretical cycle counts and SM-level resource requirement feature vectors to generate GPU-level theoretical cycle counts and GPU resource requirement feature vectors; and determining the multi-level feature set based on the SM-level theoretical cycle counts and SM-level resource requirement feature vectors and the GPU-level theoretical cycle counts and GPU-level resource requirement feature vectors.

[0013] Optionally, determining the multi-level feature set based on the SM-level theoretical cycle count, the SM-level resource requirement feature vector, the GPU-level theoretical cycle count, and the GPU-level resource requirement feature vector specifically includes: determining the SM-level theoretical cycle count and SM-level resource requirement feature vector corresponding to the SM with the highest load among multiple SM-level theoretical cycle counts and SM-level resource requirement feature vectors; and forming the multi-level feature set by combining the SM-level theoretical cycle count and SM-level resource requirement feature vector corresponding to the SM with the highest load with the GPU-level theoretical cycle count and GPU-level resource requirement feature vector.

[0014] Secondly, embodiments of the present invention provide a GPU performance prediction apparatus, the apparatus comprising: an acquisition unit for acquiring a function to be executed and hardware parameters of a GPU to be tested; a decomposition unit for decomposing the function to be executed into multiple tasks according to a set decomposition method; a scheduling unit for allocating the multiple tasks to multiple streaming multiprocessors (SMs) of the GPU according to a set scheduling method, and determining at least one task allocated to each SM; an allocation unit for allocating at least one task allocated to each SM to multiple instruction pipelines in the SM; a determination unit for determining a multi-level feature set based on the multiple instruction pipelines, wherein the multi-level feature set includes the theoretical cycle count and resource requirement feature vector of the SM level, and the theoretical cycle count and resource requirement feature vector of the GPU level; and a prediction unit for inputting the multi-level feature set into a multilayer perceptron (MLP) and outputting a prediction latency time.

[0015] Optionally, the decomposition unit is specifically used to: determine the decomposition logic of the function to be executed by reverse engineering the source code or closed-source library of the function to be executed; and decompose the function to be executed into multiple tasks according to the decomposition logic of the function to be executed.

[0016] Optionally, the scheduling unit is specifically used to: allocate the multiple tasks to multiple SMs of the GPU in a round-robin scheduling or persistent thread block scheduling manner, and determine at least one task allocated to each SM.

[0017] Optionally, the allocation unit is specifically used to: decompose each task allocated by the SM into multiple instructions; and allocate the multiple instructions to the corresponding multiple instruction pipelines.

[0018] Optionally, the instruction pipeline includes tensors, fused multiply-accumulate, and memory input / output.

[0019] Optionally, the determining unit is specifically used to: calculate the primary theoretical cycle number and primary resource requirement feature vector for each task in each instruction pipeline; and determine the multi-level feature set based on the primary theoretical cycle number and primary resource requirement feature vector of multiple instruction pipelines.

[0020] Optionally, the determining unit is further configured to: aggregate multiple primary theoretical cycle counts and primary resource requirement feature vectors of the multiple instruction pipelines to generate task-level theoretical cycle counts and task-level resource requirement feature vectors; aggregate multiple task-level theoretical cycle counts and task-level resource requirement feature vectors to generate SM-level theoretical cycle counts and SM-level resource requirement feature vectors; aggregate multiple SM-level theoretical cycle counts and SM-level resource requirement feature vectors to generate GPU-level theoretical cycle counts and GPU-level resource requirement feature vectors; and determine the multi-level feature set based on the SM-level theoretical cycle counts and SM-level resource requirement feature vectors and the GPU-level theoretical cycle counts and GPU-level resource requirement feature vectors.

[0021] Optionally, the determining unit is further configured to: determine the SM-level theoretical cycle number and SM-level resource demand feature vector corresponding to the SM with the largest load among the multiple SM-level theoretical cycle numbers and SM-level resource demand feature vectors; and combine the SM-level theoretical cycle number and SM-level resource demand feature vector corresponding to the SM with the largest load with the GPU-level theoretical cycle number and GPU-level resource demand feature vector to form the multi-level feature set.

[0022] Thirdly, embodiments of the present invention provide a system for GPU performance prediction, the system comprising: a kernel decomposer, a scheduling simulator, a feature analyzer, and a performance estimator; wherein, the kernel decomposer is used to acquire the hardware parameters of the function to be executed and the GPU to be tested, and to decompose the function to be executed into multiple tasks according to a set decomposition method; the scheduling simulator is used to allocate the multiple tasks to multiple streaming multiprocessors (SMs) of the GPU according to a set scheduling method, and to determine at least one task allocated to each SM; the feature analyzer is used to allocate at least one task allocated to each SM to multiple instruction pipelines in the SM; and to determine a multi-level feature set based on the multiple instruction pipelines, wherein the multi-level feature set includes the theoretical cycle count and resource requirement feature vector of the SM level, and the theoretical cycle count and resource requirement feature vector of the GPU level; the performance estimator is used to input the multi-level feature set into a multilayer perceptron (MLP) and output the predicted latency time.

[0023] Fourthly, embodiments of the present invention provide an electronic device, including a memory and a processor, the memory being used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described in the first aspect or any one of the possible methods of the first aspect.

[0024] Fifthly, embodiments of the present invention provide a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the method as described in the first aspect or any one of the possible methods described in the first aspect.

[0025] In this embodiment of the invention, the following steps are taken: The function to be executed and the hardware parameters of the GPU to be tested are obtained; the function to be executed is decomposed into multiple tasks according to a set decomposition method; the multiple tasks are allocated to multiple streaming multiprocessors (SMs) of the GPU according to a set scheduling method, and at least one task is assigned to each SM; the at least one task assigned to each SM is assigned to multiple instruction pipelines within the SM; a multi-level feature set is determined based on the multiple instruction pipelines, wherein the multi-level feature set includes the theoretical cycle count and resource requirement feature vector at the SM level, and the theoretical cycle count and resource requirement feature vector at the GPU level; the multi-level feature set is input into a multilayer perceptron (MLP), and the predicted latency is output. Through this method, the complex workflow of the function to be executed is mapped to a fine-grained instruction pipeline, and then a multi-level feature set is constructed based on the fine-grained instruction pipeline. Finally, a lightweight MLP is used to capture the nonlinear competition between different instruction pipelines, thereby accurately predicting the latency of the GPU under test and determining the performance of the GPU under test. Attached Figure Description

[0026] The above and other objects, features and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which: Figure 1 This is a flowchart of a GPU performance prediction method according to an embodiment of the present invention; Figure 2 This is a flowchart of another GPU performance prediction method in an embodiment of the present invention; Figure 3 This is a schematic diagram of a system framework for GPU performance prediction in an embodiment of the present invention; Figure 4 This is a schematic diagram of a GPU performance prediction device according to an embodiment of the present invention; Figure 5 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0027] The present application is described below based on embodiments, but it is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without these details. To avoid obscuring the substance of the present application, well-known methods, processes, flows, elements, and circuits are not described in detail.

[0028] Furthermore, those skilled in the art should understand that the accompanying drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0029] Unless the context explicitly requires it, words such as "including" or "contains" throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, meaning "including but not limited to".

[0030] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0031] In existing technologies, GPU performance is tested using methods such as periodic simulators, pure analytical models, and data-driven / gray-box methods. Specifically, periodic simulators are computationally expensive and slow; they lack portability and are difficult to model new hardware (i.e., new GPUs) without publicly available documentation. Pure analytical models have limited accuracy and heavily rely on micro-benchmarks for specific hardware, limiting their versatility for unseen hardware. Data-driven / gray-box methods suffer from granularity mismatch, inability to model fused kernels, idealized fluctuation modeling, and limited accuracy on unseen hardware. Therefore, how to efficiently and accurately predict the performance of new GPUs is a problem that needs to be solved.

[0032] In this embodiment of the invention, to solve the above problems, a method for GPU performance prediction is proposed, specifically as follows: Figure 1 As shown, the method includes: Step S101: Obtain the hardware parameters of the function to be executed and the graphics processing unit (GPU) to be tested.

[0033] Specifically, the function to be executed is a function called by the upper-layer load or large language model LLM during execution; the function to be executed can also be called a kernel, which is a function unit that needs to be executed on a graphics processing unit (GPU), such as general matrix multiplication (GEMM), attention mechanism, etc.

[0034] In one possible implementation, the input parameters of the function to be executed are obtained at the same time as the function to be executed.

[0035] Step S102: Decompose the function to be executed into multiple tasks according to the set decomposition method.

[0036] Specifically, the decomposition logic of the function to be executed is determined by reverse engineering the source code or closed-source library of the function to be executed; based on the decomposition logic of the function to be executed, the function to be executed is decomposed into multiple tasks.

[0037] In one possible implementation, the closed-source library can be cuBLAS, and the reverse engineering can be Profiling, which is a dynamic analysis method that studies program behavior by collecting information during program runtime, aiming to improve software efficiency and performance.

[0038] In this embodiment of the invention, if the function to be executed is a traditional kernel, the task is a Cooperative Thread Array (CTA), which can also be called a cooperative thread array; if the function to be executed is a persistent kernel, the task is a finer-grained computing unit, such as a tile; the decomposition of the function to be executed into multiple tasks is a decomposition of data volume, that is, decomposing a large amount of data into multiple small amounts of data into a function to be executed.

[0039] Step S103: Assign the multiple tasks to the multiple streaming multiprocessors (SMs) of the GPU according to the set scheduling method, and determine at least one task assigned to each SM.

[0040] Specifically, the multiple tasks are allocated to multiple streaming multiprocessors (SMs) of the GPU using either round-robin scheduling or persistent thread block scheduling (TileSchedule), and at least one task is assigned to each SM; wherein, the SM is the core computing unit of the GPU.

[0041] In one possible implementation, the polling scheduling supports the hardware implementation of the GPU, while the persistent thread block scheduling supports the software implementation of the function to be executed.

[0042] In this embodiment of the invention, by allocating multiple tasks according to a set scheduling method, the number of tasks allocated to each SM can be accurately determined, and a specific task distribution for each SM can be generated. This allows the load of each SM to be determined, and thus the load imbalance among multiple SMs can be identified.

[0043] Step S104: Assign at least one task assigned to each SM to multiple instruction pipelines within the SM.

[0044] Specifically, each task assigned to the SM is broken down into multiple instructions; the multiple instructions are assigned to the corresponding multiple instruction pipelines; wherein, the instruction pipeline includes tensors, fused multiply-add (FMA) and memory input / output (MIO), and the MIO is responsible for managing data movement units, such as L1 cache, shared memory (SMEM) and load-store unit (LSU).

[0045] In one possible implementation, the SM also includes multiple other instruction pipelines, each of which needs to process a portion of the instructions decomposed from the task. Here, we only cite three key pipelines as examples: the tensor pipeline, the fused multiply-accumulate pipeline, and the memory input / output pipeline. The specific instruction pipeline configuration will be determined based on the actual situation.

[0046] In this embodiment of the invention, the tensor pipeline and the fused multiply-accumulate pipeline are computation pipelines, the memory input-output pipeline is a memory / load-store pipeline, and different instruction pipelines are explicitly classified and computed separately.

[0047] Step S105: Determine the multi-level feature set based on the multi-instruction pipeline.

[0048] Specifically, the multi-level feature set includes SM-level theoretical cycle count and SM-level resource requirement feature vector, as well as GPU-level theoretical cycle count and GPU-level resource requirement feature vector, wherein the resource requirement feature vector is a feature vector corresponding to memory size, number of operations, etc.; the theoretical cycle count is the computation time.

[0049] In one possible implementation, determining the multi-level feature set based on the multiple instruction pipelines specifically includes: calculating the primary theoretical cycle number and primary resource requirement feature vector for each task in each instruction pipeline; and determining the multi-level feature set based on the multiple primary theoretical cycle numbers and primary resource requirement feature vectors of the multiple instruction pipelines.

[0050] In this embodiment of the invention, the determination of the multi-level feature set based on the primary theoretical cycle number and primary resource requirement feature vector of multiple instruction pipelines is specifically as follows: Figure 2 As shown, it includes the following: Step 201: Aggregate the primary theoretical cycle counts and primary resource requirement feature vectors of the multiple instruction pipelines to generate task-level theoretical cycle counts and task-level resource requirement feature vectors.

[0051] In one possible implementation, each task corresponds to multiple instruction pipelines. The primary theoretical cycle number and primary resource requirement feature vector of each instruction pipeline are calculated. For each task, the primary theoretical cycle number and primary resource requirement feature vector of the multiple instruction pipelines are aggregated to generate the task-level theoretical cycle number and task-level resource requirement feature vector of each task. The aggregation can be clustering or summation, which is determined according to the actual situation.

[0052] Step 202: Aggregate the multiple task-level theoretical cycle numbers and the task-level resource requirement feature vectors to generate the SM-level theoretical cycle number and the SM-level resource requirement feature vector.

[0053] In one possible implementation, since each SM includes multiple tasks, the theoretical cycle number of each task level and the resource requirement feature vector of each task level included in the SM are aggregated to generate the theoretical cycle number of each SM level and the resource requirement feature vector of each SM level.

[0054] Step 203: Aggregate the multiple SM-level theoretical cycle counts and the SM-level resource requirement feature vectors to generate the GPU-level theoretical cycle counts and the GPU-level resource requirement feature vectors.

[0055] In one possible implementation, since each GPU includes multiple SMs, the theoretical cycle number of the multiple SMs and the resource requirement feature vector of the SMs included in the GPU are aggregated to generate the theoretical cycle number of the SMs and the resource requirement feature vector of the GPU.

[0056] Step 204: Determine the multi-level feature set based on the theoretical number of cycles at the SM level, the resource requirement feature vector at the SM level, the theoretical number of cycles at the GPU level, and the resource requirement feature vector at the GPU level.

[0057] Specifically, the theoretical cycle count and resource requirement feature vector of the SM with the largest load among the multiple SM-level theoretical cycle counts and resource requirement feature vectors are determined; the theoretical cycle count and resource requirement feature vector of the SM with the largest load and the theoretical cycle count and resource requirement feature vector of the GPU level are combined to form the multi-level feature set.

[0058] In this embodiment of the invention, based on the concept of multidimensional Roofline, the task distribution is transformed into a multi-level feature set that captures the requirements of multiple instruction pipelines. The Roofline is a visual performance analysis model used to evaluate the performance ceiling of a program on a multi-core, many-core, or accelerator architecture, locate performance bottlenecks, and guide optimization.

[0059] Step S106: Input the multi-level feature set into the multilayer perceptron (MLP) and output the prediction delay time.

[0060] In one possible implementation, the multi-layer perceptron (MLP) is lightweight and can output prediction latency with a small computational load.

[0061] In one possible implementation, a system for predicting GPU performance is proposed, specifically as follows: Figure 3 As shown, the system includes: a kernel decomposer 301, a scheduling simulator 302, a feature analyzer 303, and a performance estimator 304; The kernel decomposer is used to acquire the hardware parameters of the function to be executed and the GPU to be tested, and to decompose the function to be executed into multiple tasks according to a set decomposition method; the multiple tasks are input to the scheduling simulator, which is used to allocate the multiple tasks to multiple streaming multiprocessors (SMs) of the GPU according to a set scheduling method, and to determine at least one task assigned to each SM; the at least one task is input to the feature analyzer, which is used to allocate at least one task assigned to each SM to multiple instruction pipelines in the SM, and to determine a multi-level feature set according to the multiple instruction pipelines; the multi-level feature set is input to the performance estimator, which is used to input the multi-level feature set to a multilayer perceptron (MLP) and output the predicted latency time.

[0062] Through the above embodiments, a feature analyzer is used to capture heterogeneous competition in the instruction pipeline at the pipeline level, overcoming granularity mismatch. Furthermore, the kernel decomposer, the scheduling simulator, and the feature analyzer are used for analysis, relying solely on hardware parameters. The MLP in the performance estimator can learn the strong regularity of the mapping relationship between theoretical cycle counts and actual cycle counts across different generations of GPUs, thus achieving high-precision prediction for unseen hardware and overcoming poor generalization. Additionally, the scheduling simulator realistically simulates the distribution of tasks on the SM, explicitly modeling load imbalance and overcoming idealized fluctuations. The kernel decomposer, the scheduling simulator, and the feature analyzer are knowledge-driven analysis models, while the performance estimator is a data-driven learning model. The GPU performance prediction system in this embodiment combines the aforementioned knowledge-driven analysis model and the aforementioned data-driven learning model into a unified framework, balancing interpretability, speed, and accuracy. Therefore, it achieves efficient and accurate prediction of GPU performance.

[0063] In this embodiment of the invention, a device for GPU performance prediction is provided, such as... Figure 4 As shown, it specifically includes: an acquisition unit 401, a decomposition unit 402, a scheduling unit 403, an allocation unit 404, a determination unit 405, and a prediction unit 406; The acquisition unit 401 is used to acquire the hardware parameters of the function to be executed and the GPU to be tested; the decomposition unit 402 is used to decompose the function to be executed into multiple tasks according to a set decomposition method; the scheduling unit 403 is used to allocate the multiple tasks to multiple streaming multiprocessors (SMs) of the GPU according to a set scheduling method, and determine at least one task allocated to each SM; the allocation unit 404 is used to allocate at least one task allocated to each SM to multiple instruction pipelines in the SM; the determination unit 405 is used to determine a multi-level feature set according to the multiple instruction pipelines, wherein the multi-level feature set includes the theoretical cycle count and resource requirement feature vector of the SM level, and the theoretical cycle count and resource requirement feature vector of the GPU level; the prediction unit 406 is used to input the multi-level feature set into a multilayer perceptron (MLP) and output the prediction latency time.

[0064] Furthermore, the decomposition unit is specifically used to: determine the decomposition logic of the function to be executed by reverse engineering the source code or closed-source library of the function to be executed; and decompose the function to be executed into multiple tasks according to the decomposition logic of the function to be executed.

[0065] Furthermore, the scheduling unit is specifically used to: allocate the multiple tasks to multiple SMs of the GPU in a round-robin scheduling or persistent thread block scheduling manner, and determine at least one task allocated to each SM.

[0066] Furthermore, the allocation unit is specifically used to: decompose each task allocated by the SM into multiple instructions; and allocate the multiple instructions to the corresponding multiple instruction pipelines.

[0067] Furthermore, the instruction pipeline includes tensors, fused multiply-accumulate, and memory input / output.

[0068] Furthermore, the determining unit is specifically used to: calculate the primary theoretical cycle number and primary resource requirement feature vector for each task in each instruction pipeline; and determine the multi-level feature set based on the primary theoretical cycle number and primary resource requirement feature vector of multiple instruction pipelines.

[0069] Furthermore, the determining unit is specifically configured to: aggregate multiple primary theoretical cycle counts and primary resource requirement feature vectors of the multiple instruction pipelines to generate task-level theoretical cycle counts and task-level resource requirement feature vectors; aggregate multiple task-level theoretical cycle counts and task-level resource requirement feature vectors to generate SM-level theoretical cycle counts and SM-level resource requirement feature vectors; aggregate multiple SM-level theoretical cycle counts and SM-level resource requirement feature vectors to generate GPU-level theoretical cycle counts and GPU-level resource requirement feature vectors; and determine the multi-level feature set based on the SM-level theoretical cycle counts and SM-level resource requirement feature vectors and the GPU-level theoretical cycle counts and GPU-level resource requirement feature vectors.

[0070] Furthermore, the determining unit is specifically used to: determine the SM-level theoretical cycle number and SM-level resource demand feature vector corresponding to the SM with the largest load among the multiple SM-level theoretical cycle numbers and SM-level resource demand feature vectors; and form the multi-level feature set by combining the SM-level theoretical cycle number and SM-level resource demand feature vector corresponding to the SM with the largest load and the GPU-level theoretical cycle number and GPU-level resource demand feature vector.

[0071] Figure 5 This is a schematic diagram of the structure of the electronic device described in an embodiment of the present invention. Figure 5As shown, it includes a general computer hardware architecture, which includes at least a processor 501 and a memory 502. The processor 501 and the memory 502 are connected via a bus 503. The memory 502 is adapted to store instructions or programs executable by the processor 501. The processor 501 can be a standalone microprocessor or a collection of one or more microprocessors. Thus, the processor 501 executes the instructions stored in the memory 502 to perform the method flow of the embodiments of the present invention as described above, thereby realizing data processing and control of other devices. The bus 503 connects the above-mentioned components together, and also connects the above-mentioned components to the display controller 504, the display device, and the input / output (I / O) device 505. The input / output (I / O) device 505 can be a mouse, keyboard, modem, network interface, touch input device, motion-sensing input device, printer, and other devices known in the art. Typically, the input / output device 505 is connected to the system via an input / output (I / O) controller 506.

[0072] The instructions stored in memory 502 are executed by at least one processor 501 to: acquire the hardware parameters of the function to be executed and the graphics processing unit (GPU) to be tested; decompose the function to be executed into multiple tasks according to a set decomposition method; allocate the multiple tasks to multiple streaming multiprocessors (SMs) of the GPU according to a set scheduling method, and determine at least one task allocated to each SM; allocate at least one task allocated to each SM to multiple instruction pipelines in the SM; determine a multi-level feature set according to the multiple instruction pipelines; input the multi-level feature set into a multilayer perceptron (MLP) and output the prediction delay time.

[0073] Specifically, the electronic device includes: one or more processors 501 and a memory 502. Figure 5 Take processor 501 as an example. Processor 501 and memory 502 can be connected via a bus or other means. Figure 5 Taking a bus connection as an example, memory 502, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Processor 501 executes various functional applications and data processing of the device by running the non-volatile software programs, instructions, and modules stored in memory 502, thereby implementing the aforementioned method for determining GPU performance prediction.

[0074] Memory 502 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store an option list, etc. Furthermore, memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 502 may optionally include memory remotely located relative to processor 501, and these remote memories can be connected to external devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0075] One or more modules are stored in memory 502 and, when executed by one or more processors 501, perform the GPU performance prediction method in any of the above method embodiments.

[0076] As those skilled in the art will recognize, various aspects of the embodiments of the present invention can be implemented as a system, method, or computer program product. Therefore, various aspects of the embodiments of the present invention can take the form of a completely hardware implementation, a completely software implementation (including firmware, resident software, microcode, etc.), or an implementation combining software and hardware aspects, which may generally be referred to herein as a "circuit," "module," or "system." Furthermore, various aspects of the embodiments of the present invention can take the form of a computer program product implemented in one or more computer-readable media having computer-readable program code implemented thereon.

[0077] Any combination of one or more computer-readable media can be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, (but not limited to) an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination thereof. More specific examples (not an exhaustive list) of computer-readable storage media will include: an electrical connection having one or more wires, a portable computer floppy disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable optical disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the context of embodiments of the present invention, a computer-readable storage medium can be any tangible medium capable of containing or storing a program used by or in conjunction with an instruction execution system, device, or apparatus.

[0078] Computer-readable signal media may include propagated digital signals having computer-readable program code implemented therein, such as in baseband or as part of a carrier wave. Such propagated signals may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium that is not a computer-readable storage medium and can communicate, propagate, or transmit a program used by or in conjunction with an instruction execution system, device, or apparatus.

[0079] Program code implemented on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, fiber optic cable, RF, or any suitable combination thereof.

[0080] Computer program code for performing operations relating to various aspects of embodiments of the present invention can be written in any combination of one or more programming languages, including: object-oriented programming languages ​​such as Java, Smalltalk, C++, etc.; and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages. The program code can be executed as a standalone software package entirely on the user's computer, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet provided by an Internet service provider).

[0081] The flowchart illustrations and / or block diagrams of the methods, apparatus (systems), and computer program products according to embodiments of the present invention describe various aspects of the embodiments of the present invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions (executed via the processor of the computer or other programmable data processing apparatus) create means for implementing the functions / actions specified in the flowchart and / or block diagram blocks or blocks.

[0082] These computer program instructions may also be stored in a computer-readable medium that can direct a computer, other programmable data processing apparatus or other means to operate in a particular manner, such that the instructions stored in the computer-readable medium produce an article of writing that includes instructions that implement the functions / actions specified in flowchart and / or block diagram blocks or blocks.

[0083] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operable steps to be performed on the computer, other programmable apparatus or other device to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide for implementing the functions / actions specified in flowchart and / or block diagram blocks or blocks.

[0084] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0085] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of such data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding access points are provided for users to choose to authorize or refuse processing. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

Claims

1. A method for predicting GPU performance, characterized in that, The method includes: Obtain the hardware parameters of the function to be executed and the GPU to be tested; The function to be executed is decomposed into multiple tasks according to the set decomposition method; The multiple tasks are allocated to multiple streaming multiprocessors (SMs) of the GPU according to a set scheduling method, and at least one task is assigned to each SM. At least one task assigned to each of the SMs is assigned to multiple instruction pipelines within the SM; A multi-level feature set is determined based on the multiple instruction pipelines, wherein the multi-level feature set includes the SM-level theoretical cycle count and SM-level resource requirement feature vector, as well as the GPU-level theoretical cycle count and GPU-level resource requirement feature vector. The multi-level feature set is input into a multilayer perceptron (MLP), and the prediction delay time is output.

2. The method according to claim 1, characterized in that, The step of decomposing the function to be executed into multiple tasks according to a set decomposition method specifically includes: The decomposition logic of the function to be executed is determined by reverse engineering the source code or closed-source library of the function to be executed. Based on the decomposition logic of the function to be executed, the function to be executed is decomposed into multiple tasks.

3. The method according to claim 1, characterized in that, The step of allocating the multiple tasks to multiple streaming multiprocessors (SMs) of the GPU according to a set scheduling method, and determining at least one task allocated to each SM, specifically includes: The multiple tasks are allocated to multiple SMs of the GPU using either round-robin scheduling or persistent thread block scheduling, and at least one task is assigned to each SM.

4. The method according to claim 1, characterized in that, The step of assigning at least one task to each instruction stream (SM) to multiple instruction pipelines within the SM specifically includes: Each task assigned to the SM is broken down into multiple instructions; The multiple instructions are assigned to the corresponding multiple instruction pipelines.

5. The method according to claim 1, characterized in that, The instruction pipeline includes tensors, fused multiply-accumulate, and memory input / output.

6. The method according to claim 1, characterized in that, The determination of the multi-level feature set based on the multiple instruction pipelines specifically includes: Calculate the primary theoretical cycle number and primary resource requirement feature vector for each task in each instruction pipeline; The multi-level feature set is determined based on the primary theoretical cycle number and primary resource requirement feature vector of multiple instruction pipelines.

7. The method according to claim 6, characterized in that, The determination of the multi-level feature set based on the primary theoretical cycle number and primary resource requirement feature vector of the multiple instruction pipelines specifically includes: The primary theoretical cycle number and primary resource requirement feature vector of the multiple instruction pipelines are aggregated to generate the task-level theoretical cycle number and task-level resource requirement feature vector. Multiple task-level theoretical cycle numbers and task-level resource requirement feature vectors are aggregated to generate SM-level theoretical cycle numbers and SM-level resource requirement feature vectors; The multiple theoretical cycle counts of the SM level and the SM level resource requirement feature vectors are aggregated to generate the theoretical cycle count of the GPU level and the GPU level resource requirement feature vectors; The multi-level feature set is determined based on the theoretical number of cycles at the SM level, the resource requirement feature vector at the SM level, the theoretical number of cycles at the GPU level, and the resource requirement feature vector at the GPU level.

8. The method according to claim 7, characterized in that, The step of determining the multi-level feature set based on the theoretical cycle count of the SM level and the resource requirement feature vector of the SM level, and the theoretical cycle count of the GPU level and the resource requirement feature vector of the GPU level, specifically includes: Determine the SM-level theoretical cycle number and SM-level resource demand feature vector corresponding to the SM with the largest load among the multiple SM-level theoretical cycle numbers and SM-level resource demand feature vectors; The multi-level feature set is composed of the theoretical number of cycles at the SM level and the SM level resource requirement feature vector corresponding to the SM with the highest load, and the theoretical number of cycles at the GPU level and the GPU level resource requirement feature vector.

9. An apparatus for predicting GPU performance, characterized in that, The device includes: The acquisition unit is used to acquire the hardware parameters of the function to be executed and the graphics processing unit (GPU) to be tested. A decomposition unit is used to decompose the function to be executed into multiple tasks according to a set decomposition method; The scheduling unit is used to allocate the multiple tasks to multiple streaming multiprocessors (SMs) of the GPU according to a set scheduling method, and to determine at least one task allocated to each SM. An allocation unit is used to allocate at least one task assigned to each of the SMs to multiple instruction pipelines within the SM; The determining unit is used to determine a multi-level feature set based on the multiple instruction pipelines, wherein the multi-level feature set includes SM-level theoretical cycle count and SM-level resource requirement feature vector, and GPU-level theoretical cycle count and GPU-level resource requirement feature vector. The prediction unit is used to input the multi-level feature set into the multilayer perceptron (MLP) and output the prediction delay time.

10. A system for predicting GPU performance, characterized in that, The system includes: a kernel decomposer, a scheduling simulator, a feature analyzer, and a performance estimator; The kernel decomposer is used to acquire the hardware parameters of the function to be executed and the GPU to be tested, and to decompose the function to be executed into multiple tasks according to a set decomposition method; the scheduling simulator is used to allocate the multiple tasks to multiple streaming multiprocessors (SMs) of the GPU according to a set scheduling method, and to determine at least one task assigned to each SM; the feature analyzer is used to allocate at least one task assigned to each SM to multiple instruction pipelines in the SM; and to determine a multi-level feature set based on the multiple instruction pipelines, wherein the multi-level feature set includes the theoretical cycle count and resource requirement feature vector of the SM level, and the theoretical cycle count and resource requirement feature vector of the GPU level; the performance estimator is used to input the multi-level feature set into a multilayer perceptron (MLP) and output the predicted latency time.

11. An electronic device comprising a memory and a processor, characterized in that, The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described in any one of claims 1-8.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-8.

Images