Vector core computing overhead quantification method, device, electronic device and storage medium

Abstract

The present invention provides a method, device, electronic device, and storage medium for quantifying vector core computational overhead, relating to the field of artificial intelligence technology, including: determining the number of instruction executions of each operation type based on instruction execution information of the current operator in a target chip; the target chip includes a vector core; the vector core executes the current operator; determining the computational load of each operation type based on the number of instruction executions of each operation type; determining the computational overhead of each operation type based on the computational load of each operation type and the configured computing power of each operation type in the target chip; and determining the computational overhead of the current operator in the target chip based on the computational overhead of each operation type. The method and device provided by the present invention accurately quantify the computational overhead of the vector core when processing different operators, providing a scientific basis for general-purpose graphics processor architecture design and operator optimization.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, electronic device and storage medium for quantifying vector core computing overhead. Background Art

[0002] With the rapid development of artificial intelligence (AI), the scale and complexity of neural network models are growing exponentially, placing increasing demands on computing power. General-Purpose Graphics Processing Units (GPGPUs), with their highly parallel computing architecture and powerful floating-point computing capabilities, have become the core hardware infrastructure supporting AI (Artificial Intelligence) training and inference. Current mainstream GPGPU architectures typically consist of two core computing units: Tensor Cores optimized for matrix operations and Vector Cores for general-purpose vector computations.

[0003] Tensor Cores focus on accelerating specific operations such as matrix multiplication in deep learning, while Vector Cores handle a wider range of parallel computing tasks. In actual application scenarios, although Tensor Cores provide significant peak performance, a large number of complex algorithms still rely on Vector Cores, such as activation function calculations in convolutional neural networks, normalization operations in attention mechanisms, and various custom operators.

[0004] Therefore, how to accurately calculate the computational overhead of vector kernels when processing different operators has become a technical problem that needs to be solved urgently in the industry. Summary of the Invention

[0005] The present invention provides a method, device, electronic device and storage medium for quantifying the computational overhead of a vector core, which are used to solve the technical problem of how to accurately calculate the computational overhead of a vector core when processing different operators.

[0006] The present invention provides a vector kernel computation overhead quantification method, comprising:

[0007] Determining the number of instruction executions of each operation type based on instruction execution information of a current operator in a target chip; the target chip includes a vector core; the vector core executes the current operator;

[0008] Determining the computational load of each operation type based on the number of instruction executions for each operation type;

[0009] Determining a computational overhead for each operation type based on a computational load for each operation type and a configured computational power of each operation type in the target chip;

[0010] Based on the computational overhead of each operation type, the computational overhead of the current operator in the target chip is determined.

[0011] In some embodiments, determining the computational load of each operation type based on the number of instruction executions of each operation type includes:

[0012] The computation load of each operation type is determined based on the number of instruction executions of each operation type and the size of thread warps executing each operation type in the target chip.

[0013] In some embodiments, before determining the computational load of each operation type based on the number of instruction executions of each operation type and the warp size of each operation type executed in the target chip, the method further includes:

[0014] In the case where the current operation type includes multiple operator types, determining an operator type with equivalent computing power among the multiple operator types;

[0015] Determining, based on the operands of each operator type and the configured computing power of each operator type in the target chip, a quantity conversion relationship between instructions of each operator type and instructions of the operator type with equivalent computing power when the computing power is equivalent;

[0016] Determine the number of instruction executions of the computing power equivalent operator type based on the quantity conversion relationship and the number of instruction executions of each operator type;

[0017] The number of instruction executions of the computing power equivalent operator type is determined as the number of instruction executions of the current operation type.

[0018] In some embodiments, determining the computational overhead of each operation type based on the computational load of each operation type and the configured computing power of each operation type in the target chip includes:

[0019] Determine the computation period of each computation type based on the ratio of the computation load of each computation type to the configured computation power of each computation type;

[0020] The computation cycle of each operation type is regarded as the computation overhead of each operation type.

[0021] In some embodiments, determining the computational overhead of the current operator in the target chip based on the computational overhead of each operation type includes:

[0022] In the case where instructions of various operation types use the same instruction issuing unit, the computational overhead of each operation type is accumulated to obtain the computational overhead of the current operator in the target chip.

[0023] In some embodiments, determining the computational overhead of the current operator in the target chip based on the computational overhead of each operation type includes:

[0024] In the case where instructions of different operation types use different instruction issuing units, the maximum value of the computational overhead of each operation type is determined as the computational overhead of the current operator in the target chip.

[0025] In some embodiments, determining the number of instruction executions of each operation type based on the instruction execution information of the current operator in the target chip includes:

[0026] executing the current operator in the target chip, or executing the current operator in an instruction simulator of the target chip, to obtain the instruction execution information;

[0027] The execution times of instructions of each operation type in the instruction execution information are counted to determine the execution times of instructions of each operation type.

[0028] In some embodiments, the operation type includes a floating point operation type, an integer operation type, a special function operation type, and a general operation type;

[0029] The instructions in the general operation type do not belong to floating-point operation instructions, integer operation instructions and special function operation instructions.

[0030] In some embodiments, the method further comprises:

[0031] Adjusting the configured computing power of each operation type in the target chip to obtain the configured updated computing power of each operation type in the target chip;

[0032] Determining a computational overhead update value for each operation type based on a computational load of each operation type and a configured updated computational power of each operation type in the target chip;

[0033] Determining, based on the updated computational overhead values ​​of each operation type, an updated computational overhead value of the current operator in the target chip;

[0034] Based on the computational overhead of the current operator before the configuration computing power is adjusted and the computational overhead update value, a performance improvement value of the target chip executing the current operator is determined.

[0035] The present invention provides a vector core computation overhead quantification device, comprising:

[0036] An instruction statistics module is used to determine the number of instruction executions of each operation type based on instruction execution information of a current operator in a target chip; the target chip includes a vector core; the vector core executes the current operator;

[0037] A load determination module, configured to determine a computational load for each operation type based on the number of instruction executions for each operation type;

[0038] an overhead calculation module, configured to determine the computational overhead of each operation type based on the computational load of each operation type and the configured computing power of each operation type in the target chip;

[0039] The cost summarizing module is used to determine the computation cost of the current operator in the target chip based on the computation cost of each operation type.

[0040] The present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the vector core computation overhead quantification method is implemented.

[0041] The present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, which implements the vector core computation overhead quantization method when the computer program is executed by a processor.

[0042] The vector core computational overhead quantification method, device, electronic device and storage medium provided by the present invention determine the number of instruction executions of each operation type based on the instruction execution information of the current operator in the target chip; the target chip includes a vector core; the vector core executes the current operator; based on the number of instruction executions of each operation type, the computational load of each operation type is determined; based on the computational load of each operation type and the configured computing power of each operation type in the target chip, the computational overhead of each operation type is determined; based on the computational overhead of each operation type, the computational overhead of the current operator in the target chip is determined; since the number of instruction executions, computational load and computational overhead of each operation type are quantitatively analyzed during the process of the vector core of the target chip executing the current operator, the computational overhead of each operation type in the overall computing task can be determined, and finally the computational overhead of the current operator in the target chip is obtained, thereby achieving accurate quantification of the computational overhead of the vector core when processing different operators, which is conducive to achieving a comprehensive and accurate evaluation of the computational overhead of general-purpose graphics processors executing various operators, and provides a scientific basis for general-purpose graphics processor architecture design and operator optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0044] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 Schematic diagram of the architecture of the general-purpose graphics processor provided by the present invention.

[0046] Figure 2 This is one of the flow charts of the vector core computation overhead quantization method provided by the present invention.

[0047] Figure 3 This is the second flow chart of the vector core computing overhead quantification method provided by the present invention.

[0048] Figure 4 It is a structural diagram of the vector core calculation overhead quantization device provided by the present invention.

[0049] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0050] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0051] It should be noted that the terms "first," "second," and the like in the present invention are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps, units, or modules is not necessarily limited to those steps, units, or modules that are explicitly listed, but may include other steps, units, or modules that are not explicitly listed or that are inherent to these processes, methods, products, or apparatuses.

[0052] ‌ Figure 1 Schematic diagram of the architecture of the general graphics processor provided by the present invention, such as Figure 1As shown, a general-purpose graphics processor 100 may include multiple streaming processor clusters 110 (SPCs) and a global memory 120 accessible to all processors. The computing units in each SPC 110 are generally divided into tensor cores 111 and vector cores 112. Tensor cores are used to perform tensor calculations, while vector cores are used to perform general-purpose vector calculations.

[0053] A general-purpose graphics processing unit (GPU) can execute various types of operators. Operators represent entities that perform specific operations or functions. They can be basic components that perform mathematical calculations, data transformations, or logical inferences. In neural networks, operators refer to the specific implementations of various layers or operations, which are the basic units that make up a neural network. Each operator performs a specific mathematical operation or data transformation, and together they implement processes such as forward propagation and backpropagation in the neural network.

[0054] When evaluating the computational overhead of general-purpose graphics processors (GPUs), related technologies focus too much on the number of floating point operations per second (FLOPS). For example, (depth width The matrix multiplication operator simplifies to FLOPS (each multiply-add operation counts as 2 floating-point operations). This method can only evaluate the computational overhead of tensor cores, but cannot effectively quantify the computational overhead of vector cores, ignoring the complexity and diversity of vector cores as general-purpose computing units.

[0055] The main challenge in accurately evaluating the computational overhead of vector cores is their rich instruction set. As general-purpose computing units, vector cores typically include instructions for vector floating-point calculations, vector integer calculations, vector special function calculations, vector logical calculations, scalar calculations, data access, conditional control, and synchronization processing. Different algorithmic tasks utilize vector core resources in different ways, making it difficult to establish a unified performance quantification model.

[0056] In order to solve the above technical problems, Figure 2 This is one of the flow charts of the vector core calculation overhead quantization method provided by the present invention, such as Figure 2 As shown, the method includes step 210 , step 220 , step 230 and step 240 .

[0057] Step 210: Determine the number of instruction executions of each operation type based on instruction execution information of the current operator in the target chip; the target chip includes a vector core; and the vector core executes the current operator.

[0058] Specifically, the vector core computational overhead quantization method provided in the embodiments of the present invention is implemented by a vector core computational overhead quantization device. This device can be implemented in software, such as a vector core computational overhead quantization program running on a computer, or in hardware, such as a processor, chip, computer, or server that executes the vector core computational overhead quantization method.

[0059] The application scenario of the method provided by the embodiment of the present invention is to run a specific operator in a general-purpose graphics processor and to quantify the computational overhead of the vector core of the operator in the general-purpose graphics processor.

[0060] The current operator may be an activation function calculation in a neural network, a normalization operation in an attention mechanism, or various custom operators, etc. The embodiment of the present invention does not specifically limit the specific type of the operator.

[0061] The target chip is a general-purpose graphics processor that requires vector core computational overhead quantization. The target chip includes vector cores that execute the current operator.

[0062] Instruction execution information refers to various data and details related to instruction execution when running the current operator on the target chip. This information can include instruction type, execution count, clock cycle, and execution unit. Based on this information, the number of instruction executions for each operation type can be calculated.

[0063] On the one hand, the current operator can be run on the target chip to obtain instruction execution information; on the other hand, the current operator can be executed in the target chip's instruction simulator to obtain instruction execution information. The execution count of instructions of each operation type in the instruction execution information is counted to determine the execution count of instructions of each operation type.

[0064] The operation type refers to the type of operation that the vector core can perform. According to the vector core's instruction set, the operation type of the instructions executed by the vector core can be determined to be divided into the following four categories:

[0065] 1. Floating-point operation types, including vector floating-point calculations (addition, subtraction, multiplication, division, square root, etc.);

[0066] 2. Integer operation types, including vector integer calculations (bit operations, addition, subtraction, multiplication, division, etc.);

[0067] 3. Special function operation types, including vector special function calculations (exponential, logarithmic, trigonometric functions, etc.);

[0068] 4. General operation type. Instructions in the general operation type do not belong to floating-point operation instructions, integer operation instructions and special function operation instructions, including vector logical operations (comparison, selection, masking, etc.), scalar calculations (various operations related to control flow), memory access operations (global memory, shared memory, texture memory, etc. read and write operations) and thread control and synchronization operations (branch prediction, thread synchronization, etc.).

[0069] Step 220: Determine the computational load of each operation type based on the number of instruction executions of each operation type.

[0070] Specifically, workload refers to the amount of computational work undertaken by the vector core for each operation type when executing the current operator, reflecting the relative importance of the operation type in the overall computing task and the demand for computing resources.

[0071] Instructions of different operation types have different complexities. For example, a floating-point multiply-accumulate operation is generally more complex than a simple integer addition. A floating-point operation instruction also requires more computational work than an integer addition instruction. In other words, instructions of different operation types have different computational workloads.

[0072] Therefore, by analyzing the execution process of the current operator and counting the number of instruction executions for each operation type, the computational load of each operation type can be determined based on the number of instruction executions for each operation type and the computational workload of each operation type.

[0073] Step 230: Determine the computational overhead of each computational type based on the computational load of each computational type and the configured computational power of each computational type in the target chip.

[0074] Specifically, computational overhead refers to the resources consumed when executing the current operator, including time, computing resources, etc. In an embodiment of the present invention, computational overhead can be measured by time, that is, by the computational cycles used by the vector core to process each type of operation. A computational cycle refers to the clock cycle required to complete a computing task or execute an instruction. It reflects the speed and efficiency of the target chip in performing calculations. In chip performance evaluation, computational cycle is an important indicator, which is closely related to factors such as the chip's clock frequency, instruction set architecture, and hardware design. The more computational cycles, the greater the computational overhead.

[0075] Configured computing power refers to the computing capacity designed and provided by the target chip for different computation types. It is related to hardware resources (such as the number of computing units and clock frequency). It represents the target chip's maximum computing potential for a specific computation type. The configured computing power for each computation type on the target chip can be obtained by querying the target chip's hardware specifications.

[0076] The computational overhead of each operation type is determined based on the computational load of each operation type and the configured computing power of each operation type in the target chip. For example, for a certain operation type, the ratio of the computational load to the corresponding configured computing power can be used as the computational overhead of the operation type.

[0077] Step 240: Determine the computational overhead of the current operator in the target chip based on the computational overhead of each operation type.

[0078] Specifically, by comprehensively calculating the computational overhead of each operation type, the computational overhead of the current operator in the target chip can be obtained.

[0079] The vector core computational overhead quantification method provided by the embodiment of the present invention determines the number of instruction executions of each operation type based on the instruction execution information of the current operator in the target chip; the target chip includes a vector core; the vector core executes the current operator; based on the number of instruction executions of each operation type, the computational load of each operation type is determined; based on the computational load of each operation type and the configured computing power of each operation type in the target chip, the computational overhead of each operation type is determined; based on the computational overhead of each operation type, the computational overhead of the current operator in the target chip is determined; since the number of instruction executions, computational load and computational overhead of each operation type are quantitatively analyzed during the process of the vector core of the target chip executing the current operator, the computational overhead of each operation type in the overall computing task can be determined, and finally the computational overhead of the current operator in the target chip is obtained, thereby achieving accurate quantification of the computational overhead of the vector core when processing different operators, which is conducive to achieving a comprehensive and accurate evaluation of the computational overhead of general-purpose graphics processors executing various operators, and provides a scientific basis for general-purpose graphics processor architecture design and operator optimization.

[0080] It should be noted that each embodiment of the present invention can be freely combined, the order can be changed, or it can be executed separately, and does not need to rely on or depend on a fixed execution order.

[0081] In some embodiments, determining the computational load of each operation type based on the number of instruction executions of each operation type includes:

[0082] The computational load of each operation type is determined based on the number of instruction executions of each operation type and the warp size of each operation type executed in the target chip.

[0083] Specifically, in the target chip (general-purpose graphics processor), a warp is the basic execution unit. After a thread block is scheduled to a streaming processor cluster, the threads within the thread block are further divided into multiple warps. The warp size refers to the number of threads contained in the warp. For example, a warp size of 32 means that each warp consists of 32 consecutive threads. All threads within a warp execute in lockstep, executing the same instructions at the same time but operating on different data.

[0084] Therefore, the computational load of each operation type can be determined based on the number of instruction executions of each operation type and the warp size of each operation type in the target chip. Specifically, the computational load of each operation type can be obtained by multiplying the number of instruction executions of each operation type by the warp size.

[0085] The vector core computing overhead quantification method provided by the embodiment of the present invention can accurately calculate the computing load of each operation type by the number of instruction executions of each operation type and the size of the thread warp executing each operation type.

[0086] In some embodiments, before determining the computational load of each operation type based on the number of instruction executions of each operation type and the warp size of each operation type executed in the target chip, the method further includes:

[0087] In the case where the current operation type includes multiple operator types, determining an operator type with equivalent computing power among the multiple operator types;

[0088] Based on the operands of each operator type and the configured computing power of each operator type in the target chip, determine the quantity conversion relationship between the instructions of each operator type and the instructions of the operator type with equivalent computing power when the computing power is equivalent;

[0089] Based on the quantity conversion relationship and the number of instruction executions of each operator type, the number of instruction executions of the operator type with equivalent computing power is determined;

[0090] The number of instruction executions of the computing power equivalent operator type is determined as the number of instruction executions of the current operation type.

[0091] Specifically, after categorizing instructions by operation type, we face the situation where instructions can be further categorized. For example, floating-point operations can include many sub-operators (addition, multiplication, multiply-accumulate, etc.), and the instructions of each sub-operator type operate on different data types (32-bit floating point, 16-bit floating point, etc.), which can be further categorized.

[0092] The classified instructions can be processed using a computing power normalization method to establish an accurate mapping relationship between the instructions and the hardware processing power. The hardware processing power is specifically the configured computing power in the hardware specification.

[0093] The following example uses the floating-point operation type as an example. The current operation type includes multiple subtypes, such as 32-bit floating point (FP32), 16-bit brain floating point (BF16), and 16-bit floating point (FP16).

[0094] If the current operation type includes multiple operator types, determine the equivalent operator type among these types. An equivalent operator type is one that can represent other operator types in terms of computing power and performance, or can be converted to or equivalent to other operator types. For example, the FP32 operation type in the floating-point operation type can be used as an equivalent operator type.

[0095] Based on the operands of each operator type and the configured computing power of each operator type in the target chip, the quantity conversion relationship between the instructions of each operator type and the instructions of the operator type with equivalent computing power when the computing power is equivalent is determined.

[0096] For example, for floating-point operation types such as FP32, BF16, and FP16, instructions of these types can be normalized to the FP32 type. This is because, generally speaking, the ratio of operand types processed by each thread bundle instruction in a general-purpose graphics processor is consistent with the computing power ratio of the corresponding types in the hardware specification (SPEC). For example, in a certain chip, each FP32 / BF16 / FP16 floating-point multiply-accumulate instruction processes 64 / 128 / 128 operands. The hardware specification of this chip lists the FP32 / BF16 / FP16 floating-point performance as 66.9 / 133.8 / 133.8 TeraFLOPS (TFLOPS), with a consistent ratio of 1:2:2. Therefore, one BF16 instruction and one FP16 instruction can each be equivalent to one FP32 instruction. It can also be assumed that, in this chip, the conversion ratio of FP32, BF16, and FP16 instructions, when computing power is equivalent, is 1:1:1.

[0097] Based on the quantity conversion relationship and the number of instruction executions for each operator type, the number of instruction executions for the equivalent operator type is determined. Finally, the number of instruction executions for the equivalent operator type is determined as the number of instruction executions for the current operator type. For example, for a BF16 instruction, an FP16 instruction, and an FP32 instruction, the equivalent operator type for computing power is FP32, and the quantity conversion relationship is BF16:FP16:FP32 = 1:1:1. Therefore, the three instructions of different operator types can be converted into three FP32 instructions.

[0098] Similarly, when evaluating computing power, each instruction that processes integer operations of different data types can be normalized and counted as a 32-bit integer (INT32) accumulation instruction.

[0099] Special function instructions generally include reciprocal, inverse root mean square, sine function, cosine function, exponential, and logarithm. These instructions use the Special Function Unit (SFU) of the general-purpose GPU for calculations and have consistent performance limits and throughput. Therefore, each special function operation type instruction can be normalized into a general-purpose FP32 SFU instruction.

[0100] General operation type instructions can be normalized into a non-computational instruction.

[0101] The vector core computing overhead quantification method provided by an embodiment of the present invention determines a computing power equivalent operator type among the multiple operator types when the current operation type includes multiple operator types, and normalizes the instructions of the multiple operator types according to the computing power equivalent operator type, so as to establish an accurate mapping relationship between the instructions and the hardware processing capability, which is conducive to accurately calculating the computing load of each operation type.

[0102] In some embodiments, determining the computational overhead of each operation type based on the computational load of each operation type and the configured computing power of each operation type in the target chip includes:

[0103] Determine the computation period of each computation type based on the ratio of the computation load of each computation type to the configured computation power of each computation type;

[0104] The computation cycle of each operation type is regarded as the computation overhead of each operation type.

[0105] Specifically, the floating-point operation type is taken as an example for explanation.

[0106] A chip executes an operator with a floating-point operation load of 106.8 Giga Floating Point Operations (GFLOPs). The floating-point operation power of the chip is configured to be 33.8 kFLOP / cycle. The floating-point operation cycle is:

[0107] 106.8 GFLOP / 33.8k FLOP / cycle=3.16 M cycles.

[0108] Among them, M is megabytes; k is kilobytes; FLOP is the number of floating point operations (Floating Point Operations).

[0109] 3.16 M cycles is used as the computational overhead of floating-point operations when executing this operator on this chip.

[0110] The vector core computing overhead quantification method provided by an embodiment of the present invention determines the computing overhead of each operation type based on the computing load of each operation type and the configured computing power of each operation type in the target chip, thereby realizing the quantitative calculation of the computing overhead of each operation type in the overall computing task.

[0111] In some embodiments, determining the computational overhead of the current operator in the target chip based on the computational overhead of each operation type includes:

[0112] When instructions of different operation types use the same instruction issuing unit, the computational overhead of each operation type is accumulated to obtain the computational overhead of the current operator in the target chip.

[0113] Specifically, the instruction issue unit (IRU) is a key component responsible for reading instructions from the instruction cache and sending them to the execution unit.

[0114] If instructions of different operation types use the same instruction issue unit, this means that instructions of different operation types are all scheduled and dispatched through the same instruction issue unit. For example, instructions for floating-point operations, integer operations, and special function operations are all managed by the same instruction issue unit. These instructions of different operation types share the same instruction issue unit and compete for the issuance resources. If one type of instruction occupies the instruction issue unit at a given time, instructions of other types will have to wait.

[0115] Therefore, it is necessary to accumulate the computational overhead of each operation type and determine the total computational overhead as the computational overhead of the current operator in the target chip.

[0116] In some embodiments, determining the computational overhead of the current operator in the target chip based on the computational overhead of each operation type includes:

[0117] In the case where instructions of different operation types use different instruction issuing units, the maximum value of the computational overhead of each operation type is determined as the computational overhead of the current operator in the target chip.

[0118] Specifically, if instructions of different operation types use different instruction issue units, it means that instructions of different operation types each use a dedicated instruction issue unit. For example, floating-point operation instructions are processed by the floating-point instruction issue unit, integer operation instructions are processed by the integer instruction issue unit, and special function operation instructions are processed by the special function instruction issue unit. Different types of instructions are processed independently in their respective instruction issue units, avoiding competition and improving overall issue efficiency.

[0119] Therefore, the maximum value of the computational overhead of each operation type can be determined as the computational overhead of the current operator in the target chip.

[0120] The vector core computational overhead quantification method provided by the embodiment of the present invention determines the computational overhead of the current operator in the target chip based on whether the instruction emission units used by instructions of different operation types are the same, thereby improving the accuracy of determining the computational overhead.

[0121] Figure 3 This is the second flow chart of the vector core calculation overhead quantification method provided by the present invention, such as Figure 3 As shown, the method includes:

[0122] Step 310: Execute the current operator on a chip (general purpose graphics processor) or an instruction simulator to obtain instruction execution information and count the number of instruction executions of each operation type.

[0123] For example, the current operator may be a softmax operator.

[0124] Step 320: Classify the instructions according to the operation type.

[0125] For example, operation types may include floating-point operation types (float op inst), integer operation types (intop inst), special function operation types (sfu op inst), and general operation types (general op inst).

[0126] Step 330: Normalize the instructions of each operation type to determine the computational workload of each operation type.

[0127] For example, the number of integer operation type instructions is Int_instruction_count = 705 M; the number of floating-point operation type instructions is Float_instruction_count = 1669 M; the number of special function operation type instructions is SFU_instruction_count = 201 M; and the number of general operation type instructions is General_instruction_count = 1784M.

[0128] Since each instruction of a certain chip is based on a warp and one multiplication and accumulation operation is 2 floating point operations. Each FP32 floating point multiplication and accumulation instruction can be counted as 2 warp_size floating point operations (float op, flop). Each integer instruction does 2 warp_size integer operations (int op). Each special function operation type instruction can be recorded as warp_size special function operation operations (sfu op); each general operation type instruction is recorded as warp_size general operation (general op).

[0129] The warp size (warp_size) of a chip is 32. Accordingly, the computational load of the softmax operator in each operation type is:

[0130] Int_workload=705 M 2 32 = 45.1 G int op;

[0131] float_workload =1669 M 2 32 = 106.8 G float op;

[0132] sfu_workload = 201 M 32 = 6.4 G sfu op;

[0133] general_workload =1784 M 32 = 57.1 G general op;

[0134] Among them, G stands for gigabit.

[0135] Step 340: Determine the configured computing power according to the hardware specification of a chip, and convert the floating-point operation instructions, integer operation instructions, special function operation instructions, and general operation instructions that can be completed by the chip in each clock cycle.

[0136] For example, from the chip's hardware specifications, we can determine the configured computing power is:

[0137] The floating-point operation power configuration is: 33.8k float op / cycle;

[0138] The computing power configuration for integer operations is: 16.9k int op / cycle;

[0139] The computing power configuration for special function operations is: 2.11k sfu op / cycle;

[0140] The general computing power configuration is: 16.9k general op / cycle;

[0141] Among them, fop is the number of floating-point operations (FLOP); int op is the number of integer operations; sfu op is the number of special function operations; general op is the number of general operations.

[0142] Determine the computational cycles for each operation type and use them as computational overhead.

[0143] For example, based on the above configuration, the computing power can be calculated as follows:

[0144] Int_cycles = 45.1 G int op / 16.9k int op / cycle = 2.67 M cycles for integer operations.

[0145] Floating-point operation type calculation cycle float_cycles = 106.8 G float op / 33.8k fop / cycle = 3.16 M cycles;

[0146] The calculation cycle of the special function operation type sfu_cycles = 6.4 G sfu op / 2.11k sfu op / cycle = 3.03 M cycles;

[0147] The calculation cycle of general operation type general_cycles = 57.1 G general op / 16.9kgeneral op / cycle = 3.38 M cycles.

[0148] Step 350: Determine whether the instruction issuing unit is shared, and determine the computational overhead of the current operator in the chip.

[0149] Instructions of various operation types share one instruction issuing unit, and the computational overhead of each operation type is summed to obtain the total computational overhead.

[0150] Vector_core_cycles = Int_cycles+ float_cycles+ sfu_cycles+ general_cycles = 12.24 M cycles.

[0151] When running the softmax operator on this chip, we detected a frequency of 1.59 GHz and an actual computation time of 8.36 milliseconds. Analysis shows that the actual computation cycle of the softmax operator is 13.29 cycles. This analysis also shows that the softmax operator's vector core utilization on this chip is 0.92, which is close to full utilization of the vector core computing power.

[0152] By calculating the percentage of the classified cycles and the total computing cycles, for example, (general_cycles + Int_cycles) / Vector_core_cycles = 49.4%, we can see that the integer plus general instructions of this operator occupy nearly half of the computing power of the vector core. We can check the operator implementation and compiler optimization to significantly improve the performance of this operator by reducing this instruction overhead.

[0153] In some embodiments, the method further comprises:

[0154] Adjust the configured computing power of each operation type in the target chip to obtain the configured updated computing power of each operation type in the target chip;

[0155] Determine an updated computational overhead value for each operation type based on the computational load of each operation type and the configured updated computational power of each operation type in the target chip;

[0156] Determine the updated computational cost value of the current operator in the target chip based on the updated computational cost value of each operation type;

[0157] Based on the computational overhead of the current operator before the configuration computing power is adjusted and the updated computational overhead value, the performance improvement value of the target chip executing the current operator is determined.

[0158] Specifically, the method provided by the embodiment of the present invention can also be used to improve the performance design of the target chip.

[0159] During the design of the target chip, the configured computing power of each operation type in the target chip can be adjusted to obtain the configured updated computing power of each operation type in the target chip. For example, in the above embodiment, the configured computing power of floating-point operations, integer operations, special function operations, and general operations remains unchanged, while the configured computing power of special function operations is doubled, that is, increased from 2.11k SFU op / cycle to 4.22k SFU op / cycle.

[0160] Based on the computational load of each operation type and the configured updated computing power of each operation type in the target chip, the updated computational overhead value of each operation type is determined. For example, the updated computational overhead value new sfu_cycles = 6.4 Gsfu ops / 4.22k sfu ops / cycle = 1.52 M cycles.

[0161] The updated computational overhead value of the current operator in the target chip is determined based on the updated computational overhead value of each operation type. For example, the updated computational overhead value new Vector_core_cycles = Int_cycles + float_cycles + newsfu_cycles + general_cycles = 10.73 M cycles.

[0162] Based on the computational overhead of the current operator before the configured computing power adjustment and the updated computational overhead value, the performance improvement of the target chip executing the current operator is determined. For example, if the performance improvement is 1 - new Vector_core_cycles / oldVector_core_cycles, it can be seen that there is a maximum performance improvement of 12.3%, and the proportion of special function computation time (sfu_cycles / Vector_core_cycles) will be reduced from 25.6% to 14.2%.

[0163] The vector core computation overhead quantification method provided by the embodiment of the present invention can accurately calculate the performance improvement value of the target chip executing the current operator, providing a scientific basis for the architecture design and operator optimization of general-purpose graphics processors.

[0164] In some embodiments, if the integer operation type instructions in the target chip are changed to be issued in parallel with instructions of other operation types, the total computational overhead of the vector core is:

[0165] Vector_core_cycles=max(Int_cycles,float_cycles+sfu_cycles+general_cycles)= 9.57 M cycles.

[0166] The performance improvement of the microarchitecture that enables parallel issuance of instructions in the chip's integer computing units can be quantified. Using 1 - new Vector_core_cycles / old Vector_core_cycles, we can see that the performance improvement is up to 12.3% to 21.8%.

[0167] The following describes an apparatus provided by an embodiment of the present invention. The apparatus described below and the method described above can refer to each other.

[0168] Figure 4 This is a schematic diagram of the structure of the vector core calculation overhead quantization device provided by the present invention. Figure 4 As shown, the device includes:

[0169] The instruction statistics module 410 is used to determine the number of instruction executions of each operation type based on the instruction execution information of the current operator in the target chip; the target chip includes a vector core; the vector core executes the current operator;

[0170] A load determination module 420 for determining a computational load for each operation type based on the number of instruction executions for each operation type;

[0171] Cost calculation module 430, for determining the computational cost of each operation type based on the computational load of each operation type and the configured computing power of each operation type in the target chip;

[0172] The cost summarizing module 440 is configured to determine the computation cost of the current operator in the target chip based on the computation costs of each operation type.

[0173] The vector core computational overhead quantification device provided by an embodiment of the present invention determines the number of instruction executions of each operation type based on the instruction execution information of the current operator in the target chip; the target chip includes a vector core; the vector core executes the current operator; based on the number of instruction executions of each operation type, the computational load of each operation type is determined; based on the computational load of each operation type and the configured computing power of each operation type in the target chip, the computational overhead of each operation type is determined; based on the computational overhead of each operation type, the computational overhead of the current operator in the target chip is determined; since the number of instruction executions, computational load and computational overhead of each operation type are quantitatively analyzed during the process of the vector core of the target chip executing the current operator, the computational overhead of each operation type in the overall computing task can be determined, and finally the computational overhead of the current operator in the target chip is obtained, thereby realizing accurate quantification of the computational overhead of the vector core when processing different operators, facilitating a comprehensive and accurate evaluation of the computational overhead of general-purpose graphics processors executing various operators, and providing a scientific basis for general-purpose graphics processor architecture design and operator optimization.

[0174] Figure 5 Schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 5 As shown, the electronic device may include: a processor (Processor) 510, a communication interface (Communications Interface) 520, a memory (Memory) 530 and a communication bus (Communications Bus) 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 may call the logic commands in the memory 530 to execute the method described in the above embodiments, for example:

[0175] Based on the instruction execution information of the current operator in the target chip, the number of instruction executions of each operation type is determined; the target chip includes a vector core; the vector core executes the current operator; based on the number of instruction executions of each operation type, the computational load of each operation type is determined; based on the computational load of each operation type and the configured computing power of each operation type in the target chip, the computational overhead of each operation type is determined; based on the computational overhead of each operation type, the computational overhead of the current operator in the target chip is determined.

[0176] Furthermore, the logical commands in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several commands for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0177] The processor in the electronic device provided by the embodiment of the present invention can call the logic instructions in the memory to implement the above method. Its specific implementation method is consistent with the implementation method of the above method and can achieve the same beneficial effects, which will not be repeated here.

[0178] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method provided in the above embodiments is implemented.

[0179] Its specific implementation is consistent with the aforementioned method implementation and can achieve the same beneficial effects, so it will not be repeated here.

[0180] An embodiment of the present invention provides a computer program product, including a computer program. When the computer program is executed by a processor, the method described above is implemented.

[0181] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0182] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0183] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for quantifying vector kernel computational overhead, characterized in that: include: Determining the number of instruction executions of each operation type based on instruction execution information of a current operator in a target chip; the target chip includes a vector core; The vector core executes the current operator; the operation type is the type of operation performed by the vector core; Determining the computational load of each operation type based on the number of instruction executions for each operation type; Determining a computational overhead for each operation type based on a computational load for each operation type and a configured computational power for each operation type in the target chip; Determining the computational overhead of the current operator in the target chip based on the computational overhead of each operation type; The determining of the computational load of each operation type based on the number of instruction executions of each operation type includes: Determining a computational load for each operation type based on the number of instruction executions for each operation type and the size of thread warps executing each operation type in the target chip; Before determining the computational load of each operation type based on the number of instruction executions of each operation type and the warp size of each operation type executed in the target chip, the method further includes: In the case where the current operation type includes multiple operator types, determining an operator type with equivalent computing power among the multiple operator types; Determining, based on the operands of each operator type and the configured computing power of each operator type in the target chip, a quantity conversion relationship between instructions of each operator type and instructions of the operator type with equivalent computing power when the computing power is equivalent; Determine the number of instruction executions of the computing power equivalent operator type based on the quantity conversion relationship and the number of instruction executions of each operator type; The number of instruction executions of the computing power equivalent operator type is determined as the number of instruction executions of the current operation type.

2. The vector core computation overhead quantification method according to claim 1, characterized in that: The determining of the computational overhead of each operation type based on the computational load of each operation type and the configured computing power of each operation type in the target chip includes: Determine the computation period of each computation type based on the ratio of the computation load of each computation type to the configured computation power of each computation type; The computation cycle of each operation type is regarded as the computation overhead of each operation type.

3. The vector core computation overhead quantification method according to claim 1, characterized in that: The determining, based on the computational overhead of each operation type, the computational overhead of the current operator in the target chip includes: In the case where instructions of various operation types use the same instruction issuing unit, the computational overhead of each operation type is accumulated to obtain the computational overhead of the current operator in the target chip.

4. The vector core computation overhead quantification method according to claim 1, characterized in that: The determining, based on the computational overhead of each operation type, the computational overhead of the current operator in the target chip includes: In the case where instructions of different operation types use different instruction issuing units, the maximum value of the computational overhead of each operation type is determined as the computational overhead of the current operator in the target chip.

5. The vector core computation overhead quantification method according to claim 1, characterized in that: The step of determining the number of instruction executions of each operation type based on the instruction execution information of the current operator in the target chip includes: executing the current operator in the target chip, or executing the current operator in an instruction simulator of the target chip, to obtain the instruction execution information; The execution times of instructions of each operation type in the instruction execution information are counted to determine the execution times of instructions of each operation type.

6. The vector core computation overhead quantification method according to claim 1, characterized in that: The operation types include floating point operation types, integer operation types, special function operation types and general operation types.

7. The vector core computation overhead quantification method according to any one of claims 1 to 6, characterized in that: The method further comprises: Adjusting the configured computing power of each operation type in the target chip to obtain the configured updated computing power of each operation type in the target chip; Determining a computational overhead update value for each operation type based on a computational load of each operation type and a configured updated computational power of each operation type in the target chip; Determining, based on the updated computational overhead values ​​of each operation type, an updated computational overhead value of the current operator in the target chip; Based on the computational overhead of the current operator before the configuration computing power is adjusted and the computational overhead update value, a performance improvement value of the target chip executing the current operator is determined.

8. A vector core computation overhead quantization device, characterized in that: include: An instruction statistics module, configured to determine the number of instruction executions of each operation type based on instruction execution information of a current operator in a target chip; the target chip includes a vector core; The vector core executes the current operator; the operation type is the type of operation performed by the vector core; A load determination module, configured to determine a computational load for each operation type based on the number of instruction executions for each operation type; an overhead calculation module, configured to determine the computational overhead of each operation type based on the computational load of each operation type and the configured computing power of each operation type in the target chip; an overhead summary module, configured to determine the computation overhead of the current operator in the target chip based on the computation overhead of each operation type; The determining of the computational load of each operation type based on the number of instruction executions of each operation type includes: Determining a computational load for each operation type based on the number of instruction executions for each operation type and the size of thread warps executing each operation type in the target chip; Before determining the computational load of each operation type based on the number of instruction executions of each operation type and the warp size of each operation type executed in the target chip, the method further includes: In the case where the current operation type includes multiple operator types, determining an operator type with equivalent computing power among the multiple operator types; Determining, based on the operands of each operator type and the configured computing power of each operator type in the target chip, a quantity conversion relationship between instructions of each operator type and instructions of the operator type with equivalent computing power when the computing power is equivalent; Determine the number of instruction executions of the computing power equivalent operator type based on the quantity conversion relationship and the number of instruction executions of each operator type; The number of instruction executions of the computing power equivalent operator type is determined as the number of instruction executions of the current operation type.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the vector core computing overhead quantification method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the vector core computation overhead quantification method according to any one of claims 1 to 7 is implemented.

Images