Chip execution operator performance prediction method, device, electronic device and medium

Abstract

The present invention provides a method, device, electronic device, and medium for predicting the performance of a chip executing an operator, relating to the field of artificial intelligence chip technology, including: obtaining the actual execution time of a reference chip executing a current operator, and the actual utilization rate of each core component in the reference chip; determining the actual workload of each core component in the reference chip based on the actual utilization rate, actual execution time, and hardware configuration specifications of each core component in the reference chip; determining the predicted occupancy time of each core component when the target chip executes the current operator based on the actual workload of each core component in the reference chip and the hardware configuration specifications of each core component in the target chip; and determining the predicted execution time of the target chip executing the current operator. The method and device provided by the present invention accurately predict the execution performance of an operator on an unknown chip without actual execution or simulation.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence chip technology, and in particular to a method, device, electronic device, and medium for predicting the performance of a chip execution operator. Background Art

[0002] With the rapid development of artificial intelligence (AI) and high-performance computing (HPC) applications, new generations of general-purpose graphics processing units (GPGPUs) are constantly evolving, with each generation exhibiting significant differences in computing power, bandwidth, and microarchitecture design. Accurately predicting the performance of AI operators on next-generation or alternative GPGPU configurations without actual execution or simulation has long been a technical challenge in the industry.

[0003] Therefore, how to accurately predict the execution performance of chip execution operators has become a technical problem that needs to be solved urgently in the industry. Summary of the Invention

[0004] The present invention provides a method, device, electronic device and medium for predicting the performance of a chip execution operator, which are used to solve the technical problem of accurately predicting the execution performance of a chip execution operator.

[0005] The present invention provides a method for predicting the performance of a chip execution operator, comprising:

[0006] Obtaining the actual execution time of the current operator on the reference chip and the actual utilization rate of each core component in the reference chip;

[0007] Determining an actual workload of each core component in the reference chip based on an actual utilization rate of each core component in the reference chip, the actual execution duration, and a hardware configuration specification of each core component in the reference chip;

[0008] Determining a predicted occupancy time of each core component when the target chip executes the current operator based on an actual workload of each core component in the reference chip and a hardware configuration specification of each core component in a target chip; the target chip and the reference chip have the same type of core components;

[0009] The predicted execution time of the target chip executing the current operator is determined based on the predicted occupancy time of each core component when the target chip executes the current operator and the predicted utilization rate of the target chip.

[0010] In some embodiments, the core components include a tensor core, a vector core, and a global memory; the global memory is used for access by the tensor core and the vector core.

[0011] In some embodiments, determining the actual workload of each core component in the reference chip based on the actual utilization rate of each core component in the reference chip, the actual execution time, and the hardware configuration specifications of each core component in the reference chip includes:

[0012] Obtaining the actual clock frequency of the reference chip executing the current operator and the peak frequency of the reference chip;

[0013] An actual workload of the tensor core is determined based on the actual utilization of the tensor core, the actual clock frequency, the peak frequency, the actual execution duration, and a hardware configuration specification of the tensor core.

[0014] In some embodiments, determining the actual workload of each core component in the reference chip based on the actual utilization rate of each core component in the reference chip, the actual execution time, and the hardware configuration specifications of each core component in the reference chip includes:

[0015] An actual workload of the global memory is determined based on the actual utilization rate of the global memory, the actual execution duration, and a hardware configuration specification of the global memory.

[0016] In some embodiments, determining the actual workload of each core component in the reference chip based on the actual utilization rate of each core component in the reference chip, the actual execution time, and the hardware configuration specifications of each core component in the reference chip includes:

[0017] Obtaining the number of instruction executions of each operation type of the vector core;

[0018] Based on the number of times instructions of each operation type are executed and the number of operations of each operation type executed by each thread warp in the reference chip, the actual workload of the vector core executing each operation type is determined.

[0019] In some embodiments, determining the predicted occupancy time of each core component when the target chip executes the current operator based on the actual workload of each core component in the reference chip and the hardware configuration specifications of each core component in the target chip includes:

[0020] Based on the actual workload of the tensor core and the hardware configuration specifications of the tensor core in the target chip, a predicted occupancy time of the tensor core when the target chip executes the current operator is determined.

[0021] In some embodiments, determining the predicted occupancy time of each core component when the target chip executes the current operator based on the actual workload of each core component in the reference chip and the hardware configuration specifications of each core component in the target chip includes:

[0022] Based on the actual workload of the global memory and the hardware configuration specifications of the global memory in the target chip, a predicted occupation time of the global memory when the target chip executes the current operator is determined.

[0023] In some embodiments, determining the predicted occupancy time of each core component when the target chip executes the current operator based on the actual workload of each core component in the reference chip and the hardware configuration specifications of each core component in the target chip includes:

[0024] Determining a predicted occupancy time for the vector core in the target chip to execute each operation type based on an actual workload of the vector core in executing each operation type and a hardware configuration specification of the vector core in the target chip in executing each operation type;

[0025] Based on the predicted occupancy time of the vector core in the target chip when executing each operation type, the predicted occupancy time of the vector core when the target chip executes the current operator is determined.

[0026] In some embodiments, the method further comprises:

[0027] Determining a predicted utilization rate of each core component based on resource constraints imposed on the current operator in each core component;

[0028] Based on the predicted utilization rates of the respective core components, the predicted utilization rate of the target chip is determined.

[0029] In some embodiments, the method further comprises:

[0030] When the actual utilization rate of each core component in the reference chip is less than a preset threshold, the actual execution duration is determined as the predicted execution duration of the target chip executing the current operator.

[0031] The present invention provides a performance prediction device for a chip execution operator, comprising:

[0032] An acquisition module is used to obtain the actual execution time of the reference chip executing the current operator and the actual utilization rate of each core component in the reference chip;

[0033] a determination module, configured to determine an actual workload of each core component in the reference chip based on an actual utilization rate of each core component in the reference chip, the actual execution duration, and a hardware configuration specification of each core component in the reference chip;

[0034] a calculation module, configured to determine a predicted occupancy time of each core component when the target chip executes the current operator based on an actual workload of each core component in the reference chip and a hardware configuration specification of each core component in the target chip; the target chip and the reference chip having the same type of core components;

[0035] The prediction module is used to determine the predicted execution time of the target chip executing the current operator based on the predicted occupancy time of each core component when the target chip executes the current operator and the predicted utilization rate of the target chip.

[0036] The present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for predicting the performance of a chip execution operator is implemented.

[0037] The present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the performance prediction method of a chip execution operator is implemented.

[0038] The present invention provides a method, device, electronic device and medium for predicting the performance of chip execution operators, which obtains the actual execution time of the reference chip executing the current operator and the actual utilization rate of each core component in the reference chip; based on the actual utilization rate, actual execution time and hardware configuration specifications of each core component in the reference chip, determines the actual workload of each core component in the reference chip; based on the actual workload of each core component in the reference chip and the hardware configuration specifications of each core component in the target chip, determines the predicted occupancy time of each core component when the target chip executes the current operator; based on the predicted occupancy time of each core component when the target chip executes the current operator The predicted execution time of the target chip to execute the current operator is determined by the occupancy time and the predicted utilization rate of the target chip; since the target chip and the reference chip have the same type of core components, the execution time of the reference chip to execute the current operator can be used to predict the execution time of the target chip to execute the current operator, which realizes the accurate prediction of the execution performance of the operator on the unknown chip without actual execution or simulation; and in the prediction process, the hardware configuration specifications of the core components of the reference chip and the target chip are combined, and the prediction is made in units of the occupancy time of the core components, which improves the accuracy of performance prediction and provides strong support for resource management, hardware design, software optimization and system reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0040] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1 This is one of the flow charts of the performance prediction method of the chip execution operator provided by the present invention.

[0042] Figure 2 Schematic diagram of the architecture of the general-purpose graphics processor provided by the present invention.

[0043] Figure 3 This is the second flow chart of the performance prediction method of the chip execution operator provided by the present invention.

[0044] Figure 4 It is a structural diagram of the performance prediction device of the chip execution operator provided by the present invention.

[0045] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0046] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0047] It should be noted that the terms "first," "second," and the like in the present invention are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps, units, or modules is not necessarily limited to those steps, units, or modules that are explicitly listed, but may include other steps, units, or modules that are not explicitly listed or that are inherent to these processes, methods, products, or apparatuses.

[0048] Due to the continuous evolution of general-purpose GPU architectures, each generation exhibits significant differences in computing power, bandwidth, and microarchitecture design. Accurately predicting the performance of various operators in AI models on next-generation GPUs or GPUs with different hardware configurations, without actual execution or simulation, not only facilitates the selection of appropriate GPUs for AI models but also provides a scientific basis for GPU hardware configuration and optimization.

[0049] Related technologies typically use roofline models and architecture simulators to predict the performance of operators executed by general-purpose graphics processors, or use historical performance data and mathematical models to predict the performance of operators executed by general-purpose graphics processors. These methods ignore the complexity of the general-purpose graphics processor architecture and cannot accurately simulate the architecture of the general-purpose graphics processor. They also rely on historical performance data and cannot adapt to different operators or general-purpose graphics processors with different architectures.

[0050] In order to solve the shortcomings of related technologies, Figure 1 This is one of the flow charts of the performance prediction method of the chip execution operator provided by the present invention, such as Figure 1 As shown, the method includes step 110 , step 120 , step 130 and step 140 .

[0051] Step 110: Obtain the actual execution time of the reference chip executing the current operator and the actual utilization rate of each core component in the reference chip.

[0052] Specifically, the performance prediction method for chip execution operators provided in the embodiments of the present invention is implemented by a performance prediction device. This device can be implemented through software, such as a performance prediction program for chip execution operators running on a computer, or through hardware, such as a computer or server that executes the performance prediction method for chip execution operators.

[0053] The application scenario of the method provided by the embodiment of the present invention is to run a specific operator in a general-purpose graphics processor to predict the performance of the general-purpose graphics processor.

[0054] A reference chip is a general-purpose graphics processor used for comparison and evaluation. A general-purpose graphics processor with known performance, power consumption, architecture, and other characteristics can be used as a reference chip. Core components are the primary components of a chip that play a key role in overall performance and functionality. They collectively determine the chip's capabilities and application scenarios. These components are typically responsible for performing key computing tasks and data processing. For general-purpose graphics processors, core components include tensor cores, vector cores, and global memory.

[0055] Figure 2 Schematic diagram of the architecture of the general graphics processor provided by the present invention, such as Figure 2 As shown, a general purpose graphics processor 200 may include multiple streaming processor clusters 210 (Streaming Processor Clusters, SPCs) and a global memory 220 accessible to all processors. The computing units in each streaming processor cluster 210 can generally be divided into tensor cores 211 and vector cores 212.

[0056] Tensor cores are compute units specifically designed to perform tensor operations (such as matrix multiplication and convolution). Vector cores are compute units used to perform vector operations (such as vector addition and multiplication) and are suitable for processing data-parallel tasks. Global memory is a storage resource on the chip accessible to tensor cores and vector cores for storing data and intermediate results.

[0057] An operator represents an entity that performs a specific operation or function. It can be a basic component that performs mathematical calculations, data conversion, or logical inference. In neural networks, operators refer to the specific implementations of various layers or operations, which are the basic units that make up a neural network. Currently, operators can include activation function calculations in neural networks, normalization operations in attention mechanisms, and various custom operators. The embodiments of the present invention do not specifically limit the specific types of operators.

[0058] Actual execution time refers to the time it takes for the current operator to execute from start to finish on the reference chip. Actual utilization refers to how busy the core components are in executing computing tasks. It can be the ratio of the actual time each core component in the reference chip uses to execute an operator to the total available time, usually expressed as a percentage.

[0059] The process data of the reference chip executing the current operator can be collected to obtain the actual execution time of the reference chip executing the current operator and the actual utilization rate of each core component.

[0060] Step 120: Determine the actual workload of each core component in the reference chip based on the actual utilization rate, actual execution time, and hardware configuration specifications of each core component in the reference chip.

[0061] Specifically, hardware configuration specifications refer to the specific technical parameters and performance indicators of each core component in the chip. These parameters determine the processing power, efficiency and resource limitations of the core components.

[0062] Workload refers to the amount of work performed by each core component during the execution of the current operator on the reference chip. For tensor cores and vector cores, workload refers to the computational load, or the amount of computational work; for global memory, workload refers to the bandwidth load, or the amount of bandwidth used.

[0063] By comprehensively considering the actual utilization, actual execution time, and hardware configuration specifications of each core component in the reference chip, the actual workload borne by these core components when executing the current operator is determined.

[0064] For example, for a tensor core, its actual workload can be expressed as the actual utilization of the core in the process of executing operators multiplied by the actual execution time, and then measured in combination with its hardware configuration specifications (such as clock frequency, etc.).

[0065] Step 130: Based on the actual workload of each core component in the reference chip and the hardware configuration specifications of each core component in the target chip, determine the predicted occupancy time of each core component when the target chip executes the current operator; the target chip and the reference chip have the same type of core components.

[0066] Specifically, a target chip refers to a chip currently being designed, evaluated, or optimized. It is the primary target of research and development efforts. Because the target chip and the reference chip share the same core components, the reference chip can be used as a benchmark for comparing the target chip's performance and characteristics. The embodiments of the present invention do not specifically limit the chip's instruction set architecture or microarchitecture.

[0067] Because the core components are of the same type, we can predict the busy time of each core component when executing the current operator on the target chip based on the actual workload of each core component in the reference chip and the hardware specifications of each core component in the target chip. This results in a predicted busy time. Busy time refers to the amount of time a core component spends performing a specific task. It reflects the length of time the core component is busy while completing the task.

[0068] Step 140 : Determine the predicted execution time of the target chip executing the current operator based on the predicted occupancy time of each core component when the target chip executes the current operator and the predicted utilization rate of the target chip.

[0069] Specifically, the predicted utilization refers to the predicted value of the utilization of each core component in the target chip when executing the current operator.

[0070] The predicted execution time of the target chip executing the current operator, that is, the predicted value of the execution time, can be calculated based on the predicted occupancy time of each core component when the target chip executes the current operator and the predicted utilization rate of the target chip.

[0071] There's a direct correlation between the predicted execution time of the current operator on the target chip and the target chip's performance. Predicted execution time is a key performance metric, reflecting the efficiency of the operator's execution on the target chip. As you can see, the longer the predicted execution time, the lower the target chip's performance.

[0072] The performance prediction method for chip execution operators provided by the embodiment of the present invention obtains the actual execution time of the reference chip executing the current operator and the actual utilization rate of each core component in the reference chip; based on the actual utilization rate, actual execution time and hardware configuration specifications of each core component in the reference chip, the actual workload of each core component in the reference chip is determined; based on the actual workload of each core component in the reference chip and the hardware configuration specifications of each core component in the target chip, the predicted occupancy time of each core component when the target chip executes the current operator is determined; based on the predicted occupancy time of each core component when the target chip executes the current operator , and the predicted utilization rate of the target chip, determine the predicted execution time of the target chip to execute the current operator; since the target chip and the reference chip have the same type of core components, the execution time of the reference chip to execute the current operator can be used to predict the execution time of the target chip to execute the current operator, realizing the accurate prediction of the execution performance of the operator on the unknown chip without actual execution or simulation; and in the prediction process, the hardware configuration specifications of the core components of the reference chip and the target chip are combined, and the prediction is made in units of the occupancy time of the core components, which improves the accuracy of performance prediction and provides strong support for resource management, hardware design, software optimization and system reliability.

[0073] It should be noted that each embodiment of the present invention can be freely combined, the order can be changed, or it can be executed separately, and does not need to rely on or depend on a fixed execution order.

[0074] In some embodiments, determining the actual workload of each core component in the reference chip based on the actual utilization rate, actual execution time, and hardware configuration specifications of each core component in the reference chip includes:

[0075] Obtain the actual clock frequency of the reference chip executing the current operator, as well as the peak frequency of the reference chip;

[0076] Determine the actual workload of the Tensor Core based on the actual utilization, actual clock frequency, peak frequency, actual execution time of the Tensor Core, and the hardware configuration specifications of the Tensor Core.

[0077] Specifically, clock frequency refers to the frequency of a chip's main clock. It represents the number of oscillation cycles per second and is a key indicator of a chip's operating speed. Chip speed is typically measured in terms of operations performed per second or the amount of data processed per second.

[0078] The clock frequency reflects the operating speed of a Tensor Core. The actual clock frequency refers to the actual operating frequency of the reference chip when executing the current operator. This reflects the actual operating speed of the Tensor Core and directly affects its computing performance. The peak frequency refers to the theoretical maximum operating frequency of the reference chip and reflects the maximum operating speed of the Tensor Core under ideal conditions.

[0079] The ratio of the actual clock frequency to the peak frequency reflects the efficiency of the Tensor Core in actual operation. The closer the ratio is to 1, the closer the actual operating frequency of the Tensor Core is to its theoretical peak frequency, and the more fully utilized the performance is.

[0080] Collect data from the reference chip executing the current operator to obtain the actual clock frequency and execution time of the operator, as well as the actual utilization of the Tensor Cores. The reference chip's peak frequency can be obtained by querying its hardware specifications.

[0081] The peak frequency of the reference chip ChipA is Peak_clk; the actual clock frequency when executing the current operator is Real_clk; the actual execution duration is ChipA_duration; the hardware configuration specification of the tensor core is ChipA_Tcore_spec, which represents the theoretical peak computing power of the tensor core of the reference chip; the actual utilization of the tensor core is ChipA_Tcore_util. The actual workload of the tensor core can be calculated as ChipA_Tcore_GFLOP:

[0082] ChipA_Tcore_GFLOP=ChipA_Tcore_util Real_clk / Peak_clk ChipA_duration ChipA_Tcore_spec.

[0083] The performance prediction method for a chip execution operator provided in an embodiment of the present invention can accurately determine the actual workload of a tensor core based on the actual utilization, actual clock frequency, peak frequency, actual execution time, and hardware configuration specifications of the tensor core, thereby improving the accuracy of performance prediction.

[0084] In some embodiments, determining the actual workload of each core component in the reference chip based on the actual utilization rate, actual execution time, and hardware configuration specifications of each core component in the reference chip includes:

[0085] Determine the actual workload of global memory based on the actual global memory utilization, actual execution time, and global memory hardware configuration specifications.

[0086] Specifically, by querying the hardware specification (spec) of the reference chip, the hardware configuration specifications of the global memory can be obtained. The hardware configuration specifications of the global memory may include the capacity, bandwidth, cache line size, etc. In this embodiment of the present invention, the memory bandwidth access volume can be selected as the workload.

[0087] The actual global memory utilization is ChipA_Dram_util; the actual execution duration is ChipA_duration; the hardware configuration specification of the global memory is ChipA_Dram_spec, which represents the global memory bandwidth design parameters of the reference chip. The actual global memory workload ChipA_Dram_GB can be calculated to represent the actual memory access volume of the current operator on the target chip:

[0088] ChipA_Dram_GB=ChipA_Dram_util ChipA_Dram_spec ChipA_duration.

[0089] The performance prediction method of the chip execution operator provided by the embodiment of the present invention can accurately determine the actual workload of the global memory based on the actual utilization rate of the global memory, the actual execution time and the hardware configuration specifications of the global memory, thereby improving the accuracy of performance prediction.

[0090] In some embodiments, determining the actual workload of each core component in the reference chip based on the actual utilization rate, actual execution time, and hardware configuration specifications of each core component in the reference chip includes:

[0091] Get the number of instruction executions of each operation type of the vector core;

[0092] Based on the number of instruction executions of each operation type and the number of operations executed by each thread warp in the reference chip, the actual workload of the vector core executing each operation type is determined.

[0093] Specifically, as a general-purpose computing unit, a vector core generally includes instructions for vector floating-point calculations, vector integer calculations, vector special function calculations, vector logical calculations, scalar calculations, data access, conditional control, and synchronization processing. When calculating the actual workload of a vector core, the various operations that the vector core can perform can be categorized by operation type.

[0094] According to the instruction set of the vector core, it can be determined that the operation types of the instructions executed by the vector core can be divided into the following four categories:

[0095] 1. Floating-point operation type (float), including vector floating-point calculations (addition, subtraction, multiplication, division, square root, etc.);

[0096] 2. Integer operation type (int), including vector integer calculation (bit operations, addition, subtraction, multiplication and division, etc.);

[0097] 3. Special function operation type (SFU), including vector special function calculation (exponential, logarithmic, trigonometric functions, etc.);

[0098] 4. General operation type (general). The instructions in the general operation type do not belong to floating-point operation instructions, integer operation instructions and special function operation instructions, including vector logical operations (comparison, selection, masking, etc.), scalar calculations (various operations related to control flow), memory access operations (global memory, shared memory, texture memory, etc. read and write operations) and thread control and synchronization operations (branch prediction, thread synchronization, etc.).

[0099] The number of instruction executions of each operation type of the vector core can be counted, and then the actual workload of the vector core executing each operation type can be determined based on the number of instruction executions of each operation type and the number of operations executed by each thread warp in the reference chip.

[0100] The number of floating-point operation instructions executed by the vector core is ChipA_Vcore_float_instructions. The number of floating-point operation instructions executed by each warp in the reference chip is ChipA_float_GOP_per_Warp_Inst. The actual workload Vcore_float_GOP of each floating-point operation type executed by the vector core can be calculated:

[0101] Vcore_float_GOP=ChipA_Vcore_float_instructions ChipA_float_GOP_per_Warp_Inst.

[0102] The number of times the vector core executes integer operation type instructions is ChipA_Vcore_int_instructions. The number of operations executed by each thread bundle (warp) in the reference chip for each integer operation type instruction is ChipA_int_GOP_per_Warp_Inst. The actual workload Vcore_int_GOP of each integer operation type executed by the vector core can be calculated:

[0103] Vcore_int_GOP=ChipA_Vcore_int_instructions ChipA_int_GOP_per_Warp_Inst.

[0104] The number of times the vector core executes instructions of the special function operation type is ChipA_Vcore_sfu_instructions. The number of operations of each special function operation type executed by each thread bundle (warp) in the reference chip is ChipA_sfu_GOP_per_Warp_Inst. The actual workload Vcore_sfu_GOP of each special function operation type executed by the vector core can be calculated:

[0105] Vcore_sfu_GOP=ChipA_Vcore_sfu_instructions ChipA_sfu_GOP_per_Warp_Inst.

[0106] The number of general operation type instruction executions of the vector core is ChipA_Vcore_general_instructions. The number of operations executed by each thread bundle (warp) in the reference chip for each general operation type instruction is ChipA_general_GOP_per_Warp_Inst. The actual workload Vcore_general_GOP of each general operation type executed by the vector core can be calculated:

[0107] Vcore_general_GOP=ChipA_Vcore_general_instructions ChipA_general_GOP_per_Warp_Inst.

[0108] The actual workload of the vector core can be obtained by adding the actual workload of each operation type.

[0109] The performance prediction method for chip execution operators provided in an embodiment of the present invention fully considers the complexity of the types of instructions executed by the vector core, and adopts a method for determining the actual workload for different operation types. It can accurately determine the actual workload of the vector core and improve the accuracy of performance prediction.

[0110] In some embodiments, based on the actual workload of each core component in the reference chip and the hardware configuration specifications of each core component in the target chip, determining the predicted occupancy time of each core component when the target chip executes the current operator includes:

[0111] Based on the actual workload of the tensor core and the hardware configuration specifications of the tensor core in the target chip, determine the predicted occupancy time of the tensor core when the target chip executes the current operator.

[0112] Specifically, the actual workload of the tensor core is ChipA_Tcore_GFLOP; the hardware configuration specification of the tensor core in the target chip ChipB is ChipB_Tcore_spec_GFLOPS, which represents the theoretical peak computing power of the tensor core of the target chip; the predicted occupied time Tcore_busy_time of the tensor core when the target chip executes the current operator can be calculated:

[0113] Tcore_busy_time=ChipA_Tcore_GFLOP / ChipB_Tcore_spec_GFLOPS.

[0114] The performance prediction method for a chip executing an operator provided by an embodiment of the present invention can accurately determine the predicted occupancy time of a tensor core when a target chip executes a current operator, thereby improving the accuracy of performance prediction.

[0115] In some embodiments, based on the actual workload of each core component in the reference chip and the hardware configuration specifications of each core component in the target chip, determining the predicted occupancy time of each core component when the target chip executes the current operator includes:

[0116] Based on the actual workload of the global memory and the hardware configuration specifications of the global memory in the target chip, determine the predicted global memory occupation time when the target chip executes the current operator.

[0117] Specifically, the actual workload of the global memory is ChipA_Dram_GB; the hardware configuration specification of the global memory in the target chip is ChipB_Dcore_spec_GB / s, which represents the theoretical peak memory bandwidth of the target chip; the predicted global memory occupancy time Dram_busy_time when the target chip executes the current operator can be calculated:

[0118] Dram_busy_time=ChipA_Dram_GB / ChipB_Dcore_spec_GB / s.

[0119] The performance prediction method for a chip executing an operator provided by an embodiment of the present invention can accurately determine the predicted occupancy time of the global memory when the target chip executes the current operator, thereby improving the accuracy of performance prediction.

[0120] In some embodiments, based on the actual workload of each core component in the reference chip and the hardware configuration specifications of each core component in the target chip, determining the predicted occupancy time of each core component when the target chip executes the current operator includes:

[0121] Determining a predicted occupancy time for each operation type executed by the vector core in the target chip based on an actual workload of each operation type executed by the vector core and hardware configuration specifications of each operation type executed by the vector core in the target chip;

[0122] Based on the predicted occupancy time of the vector core in the target chip when executing each operation type, the predicted occupancy time of the vector core when the target chip executes the current operator is determined.

[0123] Specifically, the actual workload of the vector core in the reference chip ChipA performing floating-point operations is ChipA_Vcore_float_GOP, and the hardware configuration specification of the vector core in the target chip ChipB performing floating-point operations is ChipB_Vcore_float_spec_GFLOPS. The predicted occupancy time Vcore_float_issue_time of the vector core in the target chip performing floating-point operations can be calculated:

[0124] Vcore_float_issue_time=ChipA_Vcore_float_GOP / ChipB_Vcore_float_spec_GFLOPS.

[0125] The actual workload of the vector core in the reference chip ChipA performing integer operations is ChipA_Vcore_int_GOP, and the hardware configuration specification of the vector core in the target chip ChipB performing integer operations is ChipB_Vcore_int_spec_GFLOPS. The predicted occupancy time Vcore_int_issue_time for the vector core in the target chip performing integer operations can be calculated:

[0126] Vcore_int_issue_time=ChipA_Vcore_int_GOP / ChipB_Vcore_int_spec_GFLOPS.

[0127] The actual workload of the vector core in reference chip ChipA performing special function operations is ChipA_Vcore_sfu_GOP, and the hardware configuration specification of the vector core in target chip ChipB performing special function operations is ChipB_Vcore_sfu_spec_GFLOPS. The predicted duration of the vector core in the target chip performing special function operations, Vcore_sfu_issue_time, can be calculated:

[0128] Vcore_sfu_issue_time=ChipA_Vcore_sfu_GOP / ChipB_Vcore_sfu_spec_GFLOPS.

[0129] The actual workload of the vector core in the reference chip ChipA performing general operations is ChipA_Vcore_general_GOP, and the hardware configuration specification of the vector core in the target chip ChipB performing general operations is ChipB_Vcore_general_spec_GFLOPS. The predicted issue time of the vector core in the target chip performing general operations can be calculated as Vcore_general_issue_time:

[0130] Vcore_general_issue_time=ChipA_Vcore_general_GOP / ChipB_Vcore_general_spec_GFLOPS.

[0131] The predicted busy time of the vector core in the target chip when executing each operation type can be summed to obtain the predicted busy time Vcore_busy_time of the vector core when the target chip executes the current operator:

[0132] Vcore_busy_time=Vcore_float_issue_time+Vcore_int_issue_time+Vcore_sfu_issue_time+Vcore_general_issue_time.

[0133] The performance prediction method for a chip executing an operator provided by an embodiment of the present invention can accurately determine the predicted occupancy time of a vector core when a target chip executes a current operator, thereby improving the accuracy of performance prediction.

[0134] In some embodiments, the predicted execution time of the target chip executing the current operator is determined based on the predicted occupancy time of each core component when the target chip executes the current operator and the predicted utilization rate of the target chip.

[0135] Specifically, based on the predicted busy time Tcore_busy_time of the tensor core, the predicted busy time Vcore_busy_time of the vector core, the predicted busy time Dram_busy_time of the global memory, and the predicted utilization Predict_utilization of the target chip, the predicted execution time Predict_duration of the target chip executing the current operator can be calculated:

[0136] Predict_duration=Max(Tcore_busy_time,Vcore_busy_time,Dram_busy_time) / Predict_utilization.

[0137] The above formula indicates that the predicted execution time of the current operator on the target chip is equal to the longest predicted occupancy time of all core components divided by the predicted utilization rate. This reflects the characteristic that the bottleneck resource of the target chip executing the current operator determines the ultimate performance.

[0138] In some embodiments, the method further comprises:

[0139] Determine the predicted utilization of each core component based on the resource constraints currently imposed on the operator in each core component;

[0140] Based on the predicted utilization of each core component, the predicted utilization of the target chip is determined.

[0141] Specifically, in deep learning and high-performance computing, current operators may be subject to resource constraints of each core component, which will affect the predicted utilization of each core component.

[0142] On the one hand, the performance of the current operator may be limited by the computing power or memory bandwidth of the tensor cores. The upper limit of tensor core and global memory utilization is mainly determined by the operator's algorithm and pipeline implementation, and is less affected by chip specifications and microarchitecture. Therefore, the predicted utilization rate Predict_utilization of the target chip can be the maximum of the predicted utilization rate of the tensor core ChipA_Tcore_util and the predicted utilization rate of the global memory ChipA_Dram_util. On the other hand, the performance of the current operator may be limited by the computing power of the vector cores. The predicted utilization rate ChipA_Vcore_util of the vector cores is generally difficult to reach 100%. However, the vector core instruction granularity is small, and its utilization upper limit is less affected by the operator's algorithm and pipeline implementation. Based on experience, 80% is determined as the minimum upper limit of the predicted utilization rate of the vector cores.

[0143] The predicted utilization rate of the target chip Predict_utilization can be expressed as:

[0144] 1. When the computational power of Tensor Cores or access to memory bandwidth is limited:

[0145] Predict_utilization=Max(ChipA_Tcore_util, ChipA_Dram_util);

[0146] 2. Considering the limited computing power of vector cores:

[0147] Predict_utilization=Max(ChipA_Vcore_util, 0.8).

[0148] The performance prediction method for a chip execution operator provided by an embodiment of the present invention can adaptively adjust the predicted utilization rate according to the resource limitation of the current operator in the target chip, thereby improving the accuracy of performance prediction.

[0149] In some embodiments, the method further comprises:

[0150] When the actual utilization rate of each core component in the reference chip is less than a preset threshold, the actual execution time is determined as the predicted execution time of the target chip executing the current operator.

[0151] Specifically, some operators are significantly impacted by chip startup overhead and scheduling delays, but less constrained by core computing power and memory bandwidth. This results in lower actual utilization of each core component when executing these operators on the reference chip. Therefore, when calculating the predicted execution time of these operators, it can be assumed that their execution time on the reference and target chips is equal.

[0152] A preset threshold can be set as needed. The actual utilization of each core component in the reference chip is compared with the preset threshold. If the actual utilization of each core component is less than the preset threshold, it can be considered that the predicted execution duration Predict_duration of the target chip executing the current operator is equal to the actual execution duration ChipA_duration of the reference chip executing the current operator.

[0153] For example, if the preset threshold is 0.15, indicating a set utilization of 15%, if the maximum value of the actual utilization of tensor cores, vector cores, and global memory is less than the preset threshold, Max(ChipA_Tcore_util, ChipA_Vcore_util, ChipA_Dram_util) < 0.15, then Predict_duration = ChipA_duration.

[0154] The performance prediction method for chip execution operators provided by the embodiment of the present invention takes into account the special situation that some operators are greatly affected by chip startup overhead and scheduling delay, thereby improving the accuracy of performance prediction.

[0155] Figure 3 This is the second flow chart of the performance prediction method of the chip execution operator provided by the present invention, such as Figure 3 As shown, the method includes:

[0156] Step 310: Collect performance data of the reference chip.

[0157] You can use the GPU performance analysis tool to collect detailed performance indicators of the current operator executed on the reference chip. The collected indicators include tensor core utilization, memory read and write bandwidth utilization, vector core instruction count, actual execution time of the current operator, and the actual clock frequency of the reference chip.

[0158] For example, when a convolution operator is executed on reference chip ChipA, the following results can be collected: actual execution time is 2.5 milliseconds; actual utilization of tensor cores is 85%; actual utilization of vector cores is 30%; actual utilization of memory bandwidth is 65%; and actual clock frequency is 1.7 GHz (peak clock frequency is 1.8 GHz). GHz stands for gigahertz.

[0159] Step 320: Calculate the actual workload of each core component based on the hardware configuration specifications of the reference chip.

[0160] The hardware configuration specifications of the reference chip are as follows: the theoretical computing power of the tensor core is 312 trillion floating point operations per second (TFLOPS); the core computing power of the vector core is 39 TFLOPS; and the memory bandwidth is 1.5 terabytes per second (TB / s).

[0161] The actual workload of each core component of the reference chip can be calculated. For example, the actual workload of the tensor core ChipA_Tcore_GFLOP is 0.85 (1.7 / 1.8) 2.5 312 = 626.2 GigaFloating Point Operations (GFLOPs). The actual workload of global memory ChipA_Dram_GB is 0.65 1.5 2.5=2.4 Gigabytes (GB).

[0162] Step 330: Predict the performance of the target chip based on the hardware configuration specifications of the target chip.

[0163] The hardware configuration specifications of the target chip ChipB are as follows: the theoretical computing power of the tensor core is 1000TFLOPS; the core computing power of the vector core is 32TFLOPS; the memory bandwidth is 1.6TB / s; and the clock frequency is 1.0GHz.

[0164] The predicted busy time of each core component can be determined based on the actual workload of each core component of the reference chip and the hardware configuration specifications of each core component of the target chip. For example, the predicted busy time of the tensor core is Tcore_busy_time = 626.2 / 1000 = 0.626ms; the predicted busy time of global memory is Dram_busy_time = 2.4 / 1.6 = 1.5ms; and the predicted busy time of the vector core is Vcore_busy_time = 0.92ms.

[0165] The convolution operator is limited by tensor core resources, and its predicted utilization is Predict_utilization = Max(0.85, 0.65) = 0.85. The predicted execution duration of the target chip is Predict_duration = Max(0.626, 1.5, 0.92) / 0.85 = 1.76ms.

[0166] Through actual testing, the actual execution time of the operator on the target chip is 1.8ms, and the prediction error is only 3.3%, which verifies the effectiveness of this method.

[0167] The performance prediction method for chip execution operators provided in an embodiment of the present invention is applicable to different chip products with homogeneous microarchitectures, and supports performance mapping prediction from known chips to unreleased chips; it provides an important reference for chip design, algorithm optimization and system scheduling, and can be used for early performance evaluation of new generation chips, accelerating the product development cycle; it does not require actual operation or construction of complex simulators, significantly reducing prediction costs and time.

[0168] The following describes an apparatus provided by an embodiment of the present invention. The apparatus described below and the method described above can refer to each other.

[0169] Figure 4 FIG. 1 is a schematic diagram of the structure of the performance prediction device for the chip execution operator provided by the present invention. Figure 4 As shown, the device includes:

[0170] An acquisition module 410 is configured to acquire the actual execution time of the current operator executed by the reference chip and the actual utilization rate of each core component in the reference chip;

[0171] a determination module 420 for determining an actual workload of each core component in the reference chip based on an actual utilization rate, an actual execution time, and a hardware configuration specification of each core component in the reference chip;

[0172] A calculation module 430 is configured to determine a predicted occupancy time of each core component when executing a current operator on the target chip based on an actual workload of each core component in the reference chip and a hardware configuration specification of each core component in the target chip; the target chip and the reference chip have the same type of core components;

[0173] The prediction module 440 is configured to determine the predicted execution time of the target chip executing the current operator based on the predicted occupancy time of each core component when the target chip executes the current operator and the predicted utilization rate of the target chip.

[0174] The performance prediction device for chip execution operators provided by the embodiment of the present invention obtains the actual execution time of the reference chip executing the current operator and the actual utilization rate of each core component in the reference chip; based on the actual utilization rate, actual execution time and hardware configuration specifications of each core component in the reference chip, determines the actual workload of each core component in the reference chip; based on the actual workload of each core component in the reference chip and the hardware configuration specifications of each core component in the target chip, determines the predicted occupancy time of each core component when the target chip executes the current operator; based on the predicted occupancy time of each core component when the target chip executes the current operator , and the predicted utilization rate of the target chip, determine the predicted execution time of the target chip to execute the current operator; since the target chip and the reference chip have the same type of core components, the execution time of the reference chip to execute the current operator can be used to predict the execution time of the target chip to execute the current operator, realizing the accurate prediction of the execution performance of the operator on the unknown chip without actual execution or simulation; and in the prediction process, the hardware configuration specifications of the core components of the reference chip and the target chip are combined, and the prediction is made in units of the occupancy time of the core components, which improves the accuracy of performance prediction and provides strong support for resource management, hardware design, software optimization and system reliability.

[0175] Figure 5 Schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 5 As shown, the electronic device may include: a processor (Processor) 510, a communication interface (Communications Interface) 520, a memory (Memory) 530 and a communication bus (Communications Bus) 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 may call the logic commands in the memory 530 to execute the method described in the above embodiments, for example:

[0176] Obtain the actual execution time of the reference chip for executing the current operator, and the actual utilization rate of each core component in the reference chip; determine the actual workload of each core component in the reference chip based on the actual utilization rate, actual execution time and hardware configuration specifications of each core component in the reference chip; determine the predicted occupancy time of each core component when the target chip executes the current operator based on the actual workload of each core component in the reference chip and the hardware configuration specifications of each core component in the target chip; the target chip and the reference chip have the same type of core components; determine the predicted execution time of the target chip for executing the current operator based on the predicted occupancy time of each core component when the target chip executes the current operator and the predicted utilization rate of the target chip.

[0177] Furthermore, the logical commands in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several commands for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0178] The processor in the electronic device provided by the embodiment of the present invention can call the logic instructions in the memory to implement the above method. Its specific implementation method is consistent with the implementation method of the above method and can achieve the same beneficial effects, which will not be repeated here.

[0179] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method provided in the above embodiments is implemented.

[0180] Its specific implementation is consistent with the aforementioned method implementation and can achieve the same beneficial effects, so it will not be repeated here.

[0181] An embodiment of the present invention provides a computer program product, including a computer program. When the computer program is executed by a processor, the method described above is implemented.

[0182] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0183] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0184] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for predicting the performance of a chip execution operator, characterized in that: include: Obtaining the actual execution time of the current operator on the reference chip and the actual utilization rate of each core component in the reference chip; When the core component is a tensor core, obtaining an actual clock frequency of the reference chip executing the current operator and a peak frequency of the reference chip; determining an actual workload of the tensor core based on the actual utilization of the tensor core, the actual clock frequency, the peak frequency, the actual execution duration, and a hardware configuration specification of the tensor core; In a case where the core component is a global memory, determining an actual workload of the global memory based on an actual utilization rate of the global memory, the actual execution duration, and a hardware configuration specification of the global memory; When the core component is a vector core, obtaining the number of instruction executions of each operation type of the core component; determining an actual workload of the core component executing each operation type based on the number of instruction executions of each operation type and the number of operations executed by each thread warp in the reference chip; Determining, based on the actual workload of each core component in the reference chip and the hardware configuration specifications of each core component in the target chip, a predicted occupancy time of each core component when the target chip executes the current operator; The target chip and the reference chip have the same type of core components; Determining a predicted execution time of the target chip executing the current operator based on a predicted occupancy time of each core component when the target chip executes the current operator and a predicted utilization rate of the target chip; The predicted utilization of the target chip is determined based on the following steps: When the performance of the current operator is limited by the computing power of the tensor core or the bandwidth of the global memory, the maximum value of the actual utilization rate of the tensor core and the actual utilization rate of the global memory when the reference chip executes the current operator is determined as the predicted utilization rate of the target chip; When the performance of the current operator is limited by the computing power of the vector core, the maximum value of the actual utilization of the vector core when the reference chip executes the current operator and the minimum value of the upper limit of the predicted utilization of the vector core is determined as the predicted utilization of the target chip; the minimum value of the upper limit of the predicted utilization of the vector core is determined based on experience.

2. The chip execution operator performance prediction method according to claim 1, characterized in that: The global memory is used for access by the tensor core and the vector core.

3. The chip execution operator performance prediction method according to claim 1, characterized in that: The step of determining, based on the actual workload of each core component in the reference chip and the hardware configuration specifications of each core component in the target chip, a predicted occupancy time of each core component when the target chip executes the current operator, includes: Based on the actual workload of the tensor core and the hardware configuration specifications of the tensor core in the target chip, a predicted occupancy time of the tensor core when the target chip executes the current operator is determined.

4. The chip execution operator performance prediction method according to claim 1, characterized in that: The step of determining, based on the actual workload of each core component in the reference chip and the hardware configuration specifications of each core component in the target chip, a predicted occupancy time of each core component when the target chip executes the current operator, includes: Based on the actual workload of the global memory and the hardware configuration specifications of the global memory in the target chip, a predicted occupation time of the global memory when the target chip executes the current operator is determined.

5. The method for predicting the performance of a chip execution operator according to claim 1, wherein: The step of determining, based on the actual workload of each core component in the reference chip and the hardware configuration specifications of each core component in the target chip, a predicted occupancy time of each core component when the target chip executes the current operator, includes: Determining a predicted occupancy time for the vector core in the target chip to execute each operation type based on an actual workload of the vector core in executing each operation type and a hardware configuration specification of the vector core in the target chip in executing each operation type; Based on the predicted occupancy time of the vector core in the target chip when executing each operation type, the predicted occupancy time of the vector core when the target chip executes the current operator is determined.

6. The chip execution operator performance prediction method according to claim 1, characterized in that: The method further comprises: When the actual utilization rate of each core component in the reference chip is less than a preset threshold, the actual execution duration is determined as the predicted execution duration of the target chip executing the current operator.

7. A performance prediction device for a chip execution operator, characterized in that: include: An acquisition module is used to obtain the actual execution time of the reference chip executing the current operator and the actual utilization rate of each core component in the reference chip; a determination module for, when the core component is a tensor core, obtaining an actual clock frequency of the reference chip executing the current operator and a peak frequency of the reference chip; and determining an actual workload of the tensor core based on the actual utilization of the tensor core, the actual clock frequency, the peak frequency, the actual execution time, and the hardware configuration specifications of the tensor core; In a case where the core component is a global memory, determining an actual workload of the global memory based on an actual utilization rate of the global memory, the actual execution duration, and a hardware configuration specification of the global memory; When the core component is a vector core, obtaining the number of instruction executions of each operation type of the core component; determining an actual workload of the core component executing each operation type based on the number of instruction executions of each operation type and the number of operations executed by each thread warp in the reference chip; a calculation module, configured to determine a predicted occupancy time of each core component when the target chip executes the current operator based on an actual workload of each core component in the reference chip and a hardware configuration specification of each core component in the target chip; the target chip and the reference chip having the same type of core components; A prediction module, configured to determine a predicted execution time of the target chip executing the current operator based on a predicted occupancy time of each core component when the target chip executes the current operator and a predicted utilization rate of the target chip; The predicted utilization of the target chip is determined based on the following steps: When the performance of the current operator is limited by the computing power of the tensor core or the bandwidth of the global memory, the maximum value of the actual utilization rate of the tensor core and the actual utilization rate of the global memory when the reference chip executes the current operator is determined as the predicted utilization rate of the target chip; When the performance of the current operator is limited by the computing power of the vector core, the maximum value of the actual utilization of the vector core when the reference chip executes the current operator and the minimum value of the upper limit of the predicted utilization of the vector core is determined as the predicted utilization of the target chip; the minimum value of the upper limit of the predicted utilization of the vector core is determined based on experience.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for predicting the performance of a chip execution operator according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for predicting the performance of a chip execution operator according to any one of claims 1 to 6 is implemented.

Images