Information processing method and device, electronic equipment and storage medium

Abstract

The invention discloses an information processing method and device, electronic equipment and a storage medium, and is applied to the field of information processing. The information processing method comprises the steps that a first utilization rate is determined in combination with a data flow between a memory and a shared memory when a processor executes a general matrix multiplication operator, and the first utilization rate is used for indicating performance evaluation of the general matrix multiplication operator at a processor level; determining a second utilization rate in combination with a data stream between a shared memory and a register file in a single computing unit when the processor executes the general matrix multiplication operator, the second utilization rate being used for indicating performance evaluation of the general matrix multiplication operator at the computing unit level; and based on the smaller one of the first utilization rate and the second utilization rate, determining the theoretical performance of the processor when executing the universal matrix multiplication operator. At present, only chip-level performance evaluation is considered in performance evaluation of a GEMM operator, and double-level-dimension theoretical performance evaluation is provided, so that the evaluation result is more accurate.

Description

Technical Field

[0001] The present invention application relates to an information processing method, an information processing device, an electronic device, and a non-transitory computer-readable storage medium. Background Art

[0002] General Matrix Multiply (GEMM) is a basic operation in linear algebra and is widely used in fields such as scientific computing, machine learning, deep learning, image processing, signal processing, and artificial intelligence. In processor architectures such as a graphics processing unit (GPU) or a general-purpose graphics processing unit (GPGPU), performance optimization of the GEMM operator is a key link in computationally intensive tasks. Therefore, performance optimization of the GEMM operator is crucial for improving computing efficiency. Currently, the performance evaluation of the GEMM operator only considers chip-level performance evaluation, and the performance evaluation of the GEMM operator is not accurate enough, making it impossible for users to accurately obtain the execution situation of the GEMM operator, thus affecting subsequent performance optimization and preventing targeted optimization. Summary of the Invention

[0003] The present invention application provides an information processing method for evaluating the theoretical performance of a processor in executing a general matrix multiplication operator. The processor includes a plurality of computing units and a memory, and the plurality of computing units share the memory. Each computing unit includes a shared memory, a register bank, and a tensor core. The tensor core is used to execute tensor-related calculations in the general matrix multiplication operator. When data flows between the memory and the tensor core, it passes through the shared memory and the register bank in sequence. The shared memory is closer to the memory than the register bank. The information processing method includes: combining the data stream between the memory and the shared memory when the processor executes the general matrix multiplication operator to determine a first utilization rate, where the first utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the processor level, and the first utilization rate is reflected by the ratio of the effective working time of the tensor core when executing the general matrix multiplication operator to the total elapsed time of the execution process of the general matrix multiplication operator; combining the data stream between the shared memory and the register bank within a single computing unit when the processor executes the general matrix multiplication operator to determine a second utilization rate, where the second utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the computing unit level, and the second utilization rate is based on the single computing unit as a granularity and is reflected by the ratio of the effective working time of the tensor core when executing the corresponding computing task of the single computing unit to the total elapsed time of the execution process of the computing task; based on the smaller one of the first utilization rate and the second utilization rate, determine the theoretical performance of the processor when executing the general matrix multiplication operator, where in response to the larger the first utilization rate or the second utilization rate, the higher the utilization rate of the computing resources of the tensor core when the processor executes the general matrix multiplication operator.

[0004] For example, in the information processing method provided by the present invention application, the general matrix multiplication operator is used to load a first tensor and a second tensor from the memory to the tensor cores in the plurality of computing units, and the tensor cores in the plurality of computing units execute the tensor multiplication operation of the first tensor and the second tensor to obtain a third tensor and store it in the memory. The third tensor is divided into L slices of the same size. The plurality of computing units execute multiple computing tasks in parallel to obtain multiple slices and store them in the memory. L is a positive integer. The computing task executed by each computing unit includes loading the corresponding tensor part of the first tensor and the second tensor from the memory to the tensor core of the computing unit, and using the tensor core of the computing unit to execute a tensor operation to obtain a slice as the calculation result and store it in the memory.

[0005] For example, in the information processing method provided by the present invention application, the first utilization rate is indicated by the proportion of the first working duration in the first total duration, where the first working duration is the working duration when the tensor cores in the multiple computing units execute the tensor multiplication operation of the first tensor and the second tensor, the first total duration is the total duration for the processor to completely execute the general matrix multiplication operator, the second utilization rate is indicated by the proportion of the second working duration in the second total duration, where the second working duration is the working duration when the tensor core of the single computing unit executes the tensor operation in a computing task, and the second total duration is the total duration for the single computing unit to execute the computing task.

[0006] For example, in the information processing method provided by the present invention application, in combination with the data stream between the memory and the shared memory when the processor executes the general matrix multiplication operator, determining the first utilization rate includes: determining the first equivalent bandwidth in combination with the data stream between the memory and the shared memory when the processor executes the general matrix multiplication operator; determining the first interface duration based on the first equivalent bandwidth and the total data volume for executing the general matrix multiplication operator, where the total data volume is the sum of the data volumes of the first tensor, the second tensor, and the third tensor, and the first interface duration indicates the duration brought by data transmission between the memory and the shared memory when the processor executes the general matrix multiplication operator; determining the first working duration based on the total computation volume for executing the general matrix multiplication operator and the chip computing power of the processor; and determining the first utilization rate based on the first working duration and the first interface duration.

[0007] For example, in the information processing method provided by the present invention application, the processor further includes a secondary cache, and the data also passes through the secondary cache when flowing between the memory and the tensor core. The secondary cache is closer to the memory than the shared memory. Determining the first equivalent bandwidth in combination with the data stream between the memory and the shared memory when the processor executes the general matrix multiplication operator includes: determining the bandwidth and hit rate of the secondary cache; determining the bandwidth of the memory; and determining the first equivalent bandwidth based on the bandwidth and hit rate of the secondary cache, and the bandwidth of the memory.

[0008] For example, in the information processing method provided by the present invention application, when combining the data flow between the shared memory and the register file inside a single computing unit when the processor executes the general matrix multiplication operator, determining the second utilization rate includes: determining the second interface time consumption, where the second interface time consumption indicates the time consumption brought by data transmission between the shared memory inside the single computing unit and the register file of the single computing unit when the single computing unit executes the computing task; determining the second working duration based on the total computing amount of executing the computing task and the computing power of the tensor core in the single computing unit; determining the second utilization rate based on the second working duration and the second interface time consumption.

[0009] For example, in the information processing method provided by the present invention application, the shared memory in each computing unit includes a first memory block and a second memory block. The first memory block is used to cache the corresponding tensor parts of the first tensor and the third tensor required to execute the computing task, and the second memory block is used to cache the corresponding tensor part of the second tensor required to execute the computing task. The first memory block and the second memory block have independent data transmission paths. Determining the second interface time consumption includes: determining the first time consumption based on the total data amount of the corresponding tensor parts in the first tensor and the third tensor and the bandwidth of the first memory block; determining the second time consumption based on the total data amount of the corresponding tensor part in the second tensor and the bandwidth of the second memory block; determining the larger one of the first time consumption and the second time consumption as the second interface time consumption.

[0010] For example, in the information processing method provided by the present invention application, determining the second working duration based on the total computing amount of executing the computing task and the computing power of the tensor core in the single computing unit includes: determining the total computing amount of executing the computing task based on the shape dimensions of the corresponding tensor parts in the first tensor required to execute the computing task and the shape dimensions of the corresponding tensor parts in the second tensor required to execute the computing task; determining the computing power of the tensor core in the single computing unit according to the chip computing power of the processor and the total number of the multiple computing units; determining the second working duration according to the total computing amount of executing the computing task and the computing power of the tensor core in the single computing unit.

[0011] For example, in the information processing method provided by the present invention application, determining the second utilization rate based on the second working duration and the second interface time consumption includes: determining the larger one of the second working duration and the second interface time consumption as the target duration; determining the quotient of the second working duration and the target duration as the second utilization rate.

[0012] For example, in the information processing method provided by the present invention application, based on the smaller one of the first utilization rate and the second utilization rate, determining the theoretical performance of the processor when executing the general matrix multiplication operator includes: determining the smaller one of the first utilization rate and the second utilization rate as the target utilization rate; determining the quotient of the first working duration and the target utilization rate as the theoretical time consumption of the processor when executing the general matrix multiplication operator, where the theoretical time consumption is used to indicate the theoretical performance.

[0013] The present invention application provides an information processing device for evaluating the theoretical performance of a processor executing a general matrix multiplication operator, where the processor includes a plurality of computing units and a memory, the plurality of computing units share the memory, each computing unit includes a shared memory, a register bank, and a tensor core, the tensor core is used to execute tensor-related calculations in the general matrix multiplication operator, data flows through the shared memory and the register bank in sequence when flowing between the memory and the tensor core, the shared memory is closer to the memory than the register bank, and the information processing device includes: a first determination module configured to determine a first utilization rate in combination with the data flow between the memory and the shared memory when the processor executes the general matrix multiplication operator, where the first utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the processor level, and the first utilization rate is reflected by the proportional relationship between the effective working time of the tensor core when executing the general matrix multiplication operator and the total execution time of the general matrix multiplication operator; a second determination module configured to determine a second utilization rate in combination with the data flow between the shared memory and the register bank within a single computing unit when the processor executes the general matrix multiplication operator, where the second utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the computing unit level, the second utilization rate takes the single computing unit as a granularity, and is reflected by the proportional relationship between the effective working time of the tensor core when executing the calculation task corresponding to the single computing unit and the total execution time of the calculation task; a third determination module configured to determine the theoretical performance of the processor when executing the general matrix multiplication operator based on the smaller one of the first utilization rate and the second utilization rate, where in response to the first utilization rate or the second utilization rate being larger, the higher the utilization rate of the computing resources of the tensor core of the processor when executing the general matrix multiplication operator.

[0014] The present invention application provides an electronic device, including: a memory that stores computer-executable instructions non-transiently; a processor configured to run the computer-executable instructions, where the computer-executable instructions, when run by the processor, implement the information processing method according to any one of the embodiments of the present disclosure.

[0015] The present invention application provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, an information processing method according to any embodiment of the present disclosure is implemented.

[0016] The information processing method provided by the present invention application provides a theoretical performance evaluation of a two-level dimension, and the evaluation result is more accurate. Based on a three-level storage structure, focusing on the critical path analysis of two layers of data streams (the data stream from memory to shared memory and the data stream from shared memory to register), comprehensively considering the influence of various factors, a performance evaluation model closer to the actual situation is established, which helps developers better obtain the execution situation of the GEMM operator, so as to facilitate targeted optimization, better utilize the resources of the processor, optimize the program performance, and reduce the development cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the accompanying drawings of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description only relate to some embodiments of the present disclosure and do not limit the present disclosure.

[0018] Figure 1 It is a schematic structural diagram of a general-purpose graphics processing unit (GPGPU); Figure 2 It is a schematic flowchart of the information processing method provided by at least one embodiment of the present disclosure; Figure 3 It is a schematic diagram of a slice provided by an embodiment of the present disclosure; Figure 4 It is a schematic diagram of the data stream of the GEMM operator provided by an embodiment of the present disclosure; Figure 5 It is a schematic block diagram of an information processing device provided by at least one embodiment of the present disclosure; Figure 6 It is a schematic diagram of a non-transitory computer-readable storage medium provided by at least one embodiment of the present disclosure; Figure 7 It is a schematic block diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the following will clearly and completely describe the technical solutions of the embodiments of the present disclosure with reference to the accompanying drawings of the embodiments of the present disclosure. Obviously, the described embodiments are some, but not all, of the embodiments of the present disclosure. All other embodiments obtained by those of ordinary skill in the art based on the described embodiments of the present disclosure without creative efforts shall fall within the scope of protection of the present disclosure.

[0020] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure shall have the ordinary meanings understood by those of ordinary skill in the art to which the present disclosure pertains. The terms "first", "second", and similar terms used in the present disclosure do not denote any order, quantity, or importance, but are only used to distinguish different components. Words such as "including" or "comprising" mean that the elements or items appearing before the word cover the elements or items listed after the word and their equivalents, without excluding other elements or items. Words such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. Terms such as "upper", "lower", "left", "right", etc. are only used to represent relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly. To keep the following description of the embodiments of the present disclosure clear and concise, some detailed descriptions of known functions and known components are omitted in the present disclosure.

[0021] Figure 1 It is a schematic structural diagram of a general-purpose graphics processing unit (GPGPU).

[0022] As Figure 1 shown, the general-purpose graphics processing unit is actually an array of programmable multi-processors. For example, the programmable multi-processor can be a Streaming Processor Cluster (SPC), such as including Figure 1 the streaming processor cluster 1 shown, ..., the streaming processor cluster M, where M is a positive integer greater than 1. In the general-purpose graphics processing unit, 1 streaming processor cluster processes one computing task, or multiple streaming processor clusters process one computing task. Data sharing occurs between multiple streaming processor clusters through a global cache or global memory.

[0023] As Figure 1 shown, taking the streaming processor cluster 1 as an example, 1 streaming processor cluster includes multiple computing units, such as Figure 1The computing units 1, 2, …, N in it, where N is a positive integer. Each computing unit (Compute Unit, abbreviated as CU) is used to perform arithmetic and logical operations, such as accumulation, reduction, conventional addition, subtraction, multiplication, division, etc. A computing unit includes multiple cores (also called computing cores or computing kernels), and each computing core includes an arithmetic logic unit (ALU), a floating-point computing unit, etc. The computing core is used to perform specific computing tasks. In addition, the computing unit also includes a register bank and a shared memory, which are used to hierarchically store the source data and destination data related to the computing tasks. The shared memory in a computing unit is used to share data among the cores of the computing unit. The register bank is closer to the computing core than the shared memory, and the register bank has a larger transmission bandwidth but a smaller capacity than the shared memory.

[0024] As Figure 1 shown, each computing unit also provides a Tensor Core, which is used to perform tensor calculations. For example, it performs matrix multiplication operations of the GEMM operator. A tensor is a very important data structure in deep learning. It is a high-dimensional generalization of scalars, vectors, and matrices. Tensor operations are commonly used operations in the training and inference of current deep learning and other models. The Tensor Core can accelerate matrix multiplication operations. The Tensor Cores in multiple computing units can be uniformly scheduled and controlled.

[0025] In parallel computing, computing tasks are generally executed by multiple threads. These threads are divided into multiple thread blocks before execution in a general-purpose graphics processing unit (or called a parallel computing processor), and then the multiple thread blocks are distributed to each computing unit via a thread block distribution module ( Figure 1 not shown in the figure). All threads in a thread block must be assigned to the same computing unit for execution. At the same time, the thread block will be split into the smallest execution thread bundles (or simply called thread bundles, warps), and each thread bundle contains a fixed number (or less than this fixed number) of threads. For example, 32 threads. Multiple thread blocks can be executed in the same computing unit or in different computing units.

[0026] In each computing unit, a thread bundle scheduling / distribution module ( Figure 1 not shown in the figure) schedules and allocates the thread bundles so that the multiple computing cores in the computing unit can run the thread bundles. According to the number of computing cores in the computing unit, the multiple thread bundles in a thread block can be executed simultaneously or time-sharing. The multiple threads in each thread bundle will execute the same instructions. Memory execution instructions will be issued to the shared memory in the computing unit or further issued to the intermediate-level cache or global cache or global memory for read and write operations, etc.

[0027] AsFigure 1 As shown, the graphics processing unit adopts a three - level memory architecture. The global memory (such as high - bandwidth content) usually has the largest capacity and the lowest bandwidth; the shared memory is usually granular at the computing unit level, with a smaller capacity and a higher bandwidth. For example, in some shared memories, the shared memory is divided into two memory banks, namely the high bank and the low bank, and each memory bank has an independent data transfer path; the register file is closest to the computing core, with the smallest capacity and the highest bandwidth.

[0028] An artificial intelligence model is a mathematical model used to perform specific tasks, usually constructed based on machine learning or deep - learning techniques. These models learn patterns and relationships through training data to achieve tasks such as prediction, classification, and recognition. Common artificial intelligence models include neural networks, decision trees, support vector machines, etc. An operator is the basic computing unit in an artificial intelligence model, used to perform specific mathematical operations. An operator can be a simple mathematical operation (such as addition, multiplication) or a complex matrix operation (such as matrix multiplication, convolution). For example, the GEMM operator used to perform general matrix multiplication.

[0029] The tensor - related calculations to be performed by the GEMM operator include A×B = C, where A represents the first tensor, B represents the second tensor, and C represents the third tensor. The tensor core can be used to calculate A×B to obtain the calculation result C. For example, the first tensor A and the second tensor B are stored in memory. The GEMM operator first needs to load the first tensor A and the second tensor B from memory, and then store the calculated third tensor C back to memory after the calculation.

[0030] The efficient implementation of an operator depends on the parallel computing capabilities of, for example, a graphics processing unit or a general - purpose graphics processing unit. Accurate performance evaluation can help developers better understand the execution of the operator, so as to conduct targeted optimizations. In a hierarchical hardware architecture such as a graphics processing unit or a general - purpose graphics processing unit, the current performance evaluation method for the GEMM operator only considers the data flow from memory to shared memory, ignoring the critical path analysis inside the computing unit, and the obtained performance evaluation results are not accurate enough. Moreover, the current evaluation method has a single evaluation dimension and cannot distinguish different evaluation dimensions at the chip level and the computing unit level, resulting in a deviation between the theoretical performance prediction and the actual hardware behavior.

[0031] At least one embodiment of the present disclosure provides an information processing method, apparatus, electronic device, and storage medium. The information processing method is used to evaluate the theoretical performance of a GEMM operator on a processor. The processor includes multiple computing units and a memory. The multiple computing units share the memory. Each computing unit includes a shared memory, a register bank, and a tensor core. The tensor core is used to perform tensor-related calculations in the general matrix multiplication operator. When data flows between the memory and the tensor core, it passes through the shared memory and the register bank in sequence. The shared memory is closer to the memory than the register bank. The information processing method includes: combining the data flow between the memory and the shared memory when the processor executes the general matrix multiplication operator to determine a first utilization rate, where the first utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the processor level; combining the data flow between the shared memory and the register bank within a single computing unit when the processor executes the general matrix multiplication operator to determine a second utilization rate, where the second utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the computing unit level; determining the theoretical performance of the processor when executing the general matrix multiplication operator based on the smaller one of the first utilization rate and the second utilization rate, where in response to the larger the first utilization rate or the second utilization rate, the higher the utilization rate of the computing resources of the tensor core when the processor executes the general matrix multiplication operator.

[0032] The information processing method provided by at least one embodiment of the present disclosure provides a theoretical performance evaluation with a two-level dimension, and the evaluation result is more accurate. Based on the three-level storage structure, focusing on the critical path analysis of two layers of data flows (the data flow from the memory to the shared memory, and the data flow from the shared memory to the register), comprehensively considering the influence of various factors, a more practical performance evaluation model is established, which helps developers better obtain the execution situation of the GEMM operator, so as to facilitate targeted optimization, better utilize the resources of the processor, optimize the program performance, and reduce the development cost.

[0033] The information processing method provided by the embodiments of the present disclosure can be applied to the information processing apparatus provided by the embodiments of the present disclosure, and the information processing apparatus can be configured on an electronic device. The electronic device can be, for example, a host in a server.

[0034] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings, but the present disclosure is not limited to these specific embodiments.

[0035] Figure 2 It is a schematic flowchart of the information processing method provided by at least one embodiment of the present disclosure. As Figure 2 shown, the information processing method includes steps S10 - S30.

[0036] The information processing method provided by at least one embodiment of the present disclosure is used to evaluate the theoretical performance of a processor in executing a general matrix multiplication operator. For example, this theoretical performance can be reflected by the theoretical time consumption of the processor in executing the general matrix multiplication operator. Of course, in some other embodiments, this theoretical performance can be reflected by other parameters that can reflect the theoretical performance, such as theoretical throughput, theoretical peak performance, theoretical resource utilization, etc.

[0037] The processor includes multiple computing units and a memory. The multiple computing units share the memory. Each computing unit includes a shared memory, a register bank, and a tensor core. The tensor core is used to perform tensor-related calculations in the general matrix multiplication operator. When data flows between the memory and the tensor core, it passes through the shared memory and the register bank in sequence. The shared memory is closer to the memory than the register bank. For example, the structure of the processor can refer to Figure 1 as shown. For example, the processor can be a graphics processing unit or a general graphics processing unit.

[0038] As Figure 1 shown, when data is loaded from the memory to the tensor core of the computing unit, it is first loaded to the shared memory (if the processor provides a secondary cache, it passes through the secondary cache and then is loaded to the shared memory), then from the shared memory to the register bank, and then from the register bank to the tensor core. Similarly, when data is stored from the tensor core to the memory, it is first stored to the register bank, then from the register bank to the shared memory, then from the shared memory to the secondary cache (if any), and then from the secondary cache to the memory.

[0039] For example, the first tensor A and the second tensor B are stored in the memory. The general matrix multiplication operator is used to load the first tensor A and the second tensor B from the memory to the tensor cores in multiple computing units, and the tensor cores in the multiple computing units perform the tensor multiplication operation on the first tensor A and the second tensor B to obtain the third tensor C and store it in the memory. That is, the time consumption of executing the general matrix multiplication operator includes the time consumption caused by data loading and storage, as well as the time consumption of the tensor core performing matrix multiplication operations.

[0040] As Figure 1 described, a graphics processing unit or a general graphics processing unit often includes multiple computing units. When executing the GEMM operator, the third tensor C is usually divided into multiple tiles of the same size. For example, the third tensor C is divided into L tiles of the same size. Multiple computing units execute multiple computing tasks in parallel to obtain multiple tiles and store them in the memory. L is a positive integer. L can be the same as or different from the number of computing units.

[0041] The computing task executed by each computing unit includes loading the corresponding tensor part of the first tensor and the second tensor from the memory to the tensor core of the computing unit, and the tensor core of the computing unit performing a tensor operation to obtain a tile as the calculation result and storing it in the memory.

[0042] For example, the first tensor A and the second tensor B are sliced with the task objective of obtaining each slice as a computing task. Each computing unit executes a computing task to obtain a slice of the third tensor as the computing result. Multiple computing units execute multiple computing tasks in parallel, and the multiple computing tasks are independent of each other.

[0043] Figure 3 Schematic diagram of a slice provided by an embodiment of the present disclosure.

[0044] As Figure 3 shown, the third tensor C is divided into 4 slices, namely C00, C01, C02, and C03. The 4 slices correspond to 4 computing tasks executed in parallel by 4 computing units. Taking the computing task of obtaining slice C00 as an example, the data to be loaded for calculating C00 includes Figure 3 A00, A01, A02, and A03 in the first tensor A in [], and B00, B10, B20, and B30 in the second tensor B. A00, A01, A02, and A03 in the first tensor A are the corresponding tensor parts in the first tensor that need to be loaded into the tensor core, and B00, B10, B20, and B30 in the second tensor B are the corresponding tensor parts in the second tensor that need to be loaded into the tensor core. The logic for other slices such as C01 is similar and will not be elaborated here.

[0045] For example, if the division cannot exactly divide the third tensor C into multiple slices of the same size, then for some slices, they can be aligned to the size of the slice by filling with preset values, etc. When executing the information processing method provided by at least one embodiment of the present disclosure, the total computing amount (Workload), total data amount, etc. are determined according to the aligned size.

[0046] Figure 4 Schematic diagram of the data flow of the GEMM operator provided by an embodiment of the present disclosure.

[0047] The first tensor A and the second tensor B are stored in the global memory.

[0048] As Figure 4 shown, for the GEMM operator, for executing to obtain Figure 3Taking the computing unit 1 for the computing task of the middle slice C00 as an example, when the computing task of C00 is executed by the tensor core in the computing unit 1, A00, A01, A02, and A03 in the first tensor A are first loaded from the global memory to the shared memory of the computing unit 1, then from the shared memory to the register file of the computing unit 1, and then from the register file to the tensor core of the computing unit 1. For B00, B10, B20, and B30 in the second tensor B, they are first loaded from the global memory to the shared memory of the computing unit 1, and then from the shared memory to the tensor core of the computing unit 1. Of course, in some other embodiments, they can also be first loaded from the shared memory to the register file of the computing unit 1, and then from the register file to the tensor core of the computing unit 1.

[0049] After that, the corresponding tensor parts A00, A01, A02, and A03 in the first tensor A and the corresponding tensor parts B00, B10, B20, and B30 in the second tensor B perform tensor operations in the tensor core to obtain the slice C00 as the calculation result. For the slice C00 as the calculation result, it needs to be stored in the memory. The data flow is that the slice C00 is first stored in the register file, then stored in the shared memory of the computing unit 1, and then stored in the memory.

[0050] Of course, it should be noted that the above embodiments are a partitioning method when the entire computing task of the general matrix multiplication operator is allocated to multiple computing units for execution, but the present disclosure is not limited thereto. Those skilled in the art can also adopt other feasible implementation manners to determine the computing tasks executed by each computing unit.

[0051] The following specifically describes the specific process of the information processing method provided by at least one embodiment of the present disclosure.

[0052] As Figure 2 shown, in step S10, in combination with the data flow between the memory and the shared memory when the combined processor executes the general matrix multiplication operator, the first utilization rate is determined.

[0053] The first utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the processor level (for example, the execution efficiency of the general matrix multiplication operator at the processor level or the hardware computing resource utilization rate of the tensor core when the processor / chip executes the general matrix multiplication operator). The first utilization rate is reflected by the proportional relationship between the effective working time of the tensor core when executing the general matrix multiplication operator and the total elapsed time of the execution process of the general matrix multiplication operator. The performance evaluation at the processor level regards multiple computing units as a whole. From the perspective of the overall chip, the input data is the first tensor and the second tensor stored in the memory, the output data is the third tensor that needs to be stored in the memory, and the workload is the calculation of the complete GEMM operator.

[0054] For example, the first utilization rate is indicated by the proportion of the first working duration in the first total duration. The first working duration is the working duration when the tensor cores in multiple computing units execute the tensor multiplication operation of the first tensor and the second tensor, and the first total duration is the total duration for the processor to completely execute the general matrix multiplication operator. As mentioned above, the total duration for completely executing the general matrix multiplication operator takes into account not only the working duration / occupation duration of the tensor cores for executing the general matrix multiplication operator, but also the duration brought by loading data from memory and storing data to memory.

[0055] The first utilization rate is less than or equal to 1. If the first utilization rate is 1, it means that the utilization rate of the tensor cores is the highest. At this time, the duration brought by data loading and storage is completely hidden in the working duration of the tensor cores. The higher the first utilization rate, the higher the utilization rate of the computing resources of the tensor cores when the processor executes the general matrix multiplication operator, the computing resources of the tensor cores are fully utilized, and the tensor cores are mostly performing effective calculations instead of being idle (such as waiting for data transmission, synchronization, or waiting for other computing units to complete work, etc.). The execution speed of the entire GEMM operator is faster, the waiting time is minimized, and the overhead of data transmission and synchronization is smaller.

[0056] For example, in some embodiments, step S10 may include: determining a first equivalent bandwidth in combination with the data stream between the memory and the shared memory when the processor executes the general matrix multiplication operator; determining a first interface duration based on the first equivalent bandwidth and the total data volume for executing the general matrix multiplication operator, where the data volume is the sum of the data volumes of the first tensor, the second tensor, and the third tensor, and the first interface duration indicates the duration brought by data transmission between the memory and the shared memory when the processor executes the general matrix multiplication operator; determining a first working duration based on the total computing volume for executing the general matrix multiplication operator and the chip computing power of the processor; and determining a first utilization rate based on the first working duration and the first interface duration.

[0057] For example, the processor further includes a secondary cache. When data flows between the memory and the tensor cores, it also passes through the secondary cache, and the secondary cache is closer to the memory than the shared memory. In some embodiments, determining a first equivalent bandwidth in combination with the data stream between the memory and the shared memory when the processor executes the general matrix multiplication operator includes: determining the bandwidth and hit rate of the secondary cache; determining the bandwidth of the memory; and determining the first equivalent bandwidth based on the bandwidth and hit rate of the secondary cache and the bandwidth of the memory.

[0058] For example, the first equivalent bandwidth can be obtained using the following formula: BW_equiv = BW_L2 × hit_rate + BW_GLM × (1 - hit_rate) (Formula 1) Among them, BW_equiv represents the first equivalent bandwidth, BW_L2 represents the bandwidth of the second-level cache, hit_rate represents the hit rate of the second-level cache, and BW_GLM represents the memory bandwidth.

[0059] The first equivalent bandwidth takes into account the memory bandwidth and the hit rate of the second-level cache. When data is transferred from memory to the computing unit, it will be cached in the second-level cache. When loading data from memory, if the data exists in the second-level cache, it is called a "hit", and there is no need to read it from memory at this time. If the data does not exist in the second-level cache, it is called a "miss", and it is necessary to read it from memory at this time.

[0060] When loading the first tensor and the second tensor from memory, the hit part in the second-level cache is obtained from the second-level cache, and the data transfer bandwidth is the bandwidth of the second-level cache; the miss part is obtained from memory, and the data transfer bandwidth is the memory bandwidth. When determining the first equivalent bandwidth, the influence of the second-level cache hit rate on the equivalent bandwidth is considered, and the obtained first equivalent bandwidth is more accurate and closer to the actual situation, and thus the accuracy of the obtained first utilization rate is higher.

[0061] For example, in some other embodiments, the first equivalent bandwidth may be determined only considering the memory bandwidth, which will not be elaborated here.

[0062] For example, the following formula can be used to obtain the first interface time consumption: T_IO_chip = Data_total / BW_equiv (Formula 2) Among them, T_IO_chip represents the first interface time consumption, Data_total represents the total data volume for executing the general matrix multiplication operator, which is the sum of the data volumes of the first tensor, the second tensor, and the third tensor, and " / " represents division. As shown in Formula 2, the first interface time consumption reflects the time consumption of the processor in loading the first tensor and the second tensor to the tensor core and storing the third tensor to memory when executing the general matrix multiplication operator.

[0063] The first working duration is the duration spent by the tensor core to complete the tensor multiplication operation A×B between the first tensor A and the second tensor B. For example, the following formula can be used to determine the first working duration: T_Compute_chip = Workload1 / Compute_power1 (Formula 3) Among them, T_Compute_chip represents the first working duration, Workload1 represents the total computing volume for executing the general matrix multiplication operator, with the unit of Tflops, which is the unit of the operand, and Compute_power1 represents the chip computing power of the processor, with the unit of Tflops / s, meaning how many operands are executed per second.

[0064] For example, if the shape and size of the first tensor A are M×K, the shape and size of the second tensor B are K×N, and the shape and size of the third tensor C are M×N, then the total number of calculations can be expressed as M×N×K×2 (Tflops).

[0065] For example, the first utilization rate can be determined with reference to the following formula: Gemm_util_chip=T_Compute_chip / max(T_IO_chip,T_Compute_chip) (Formula 4) Among them, Gemm_util_chip represents the first utilization rate, T_Compute_chip represents the first working duration, max() represents the maximum value function, and T_IO_chip represents the first interface time consumption.

[0066] For example, in step S20, in combination with the data stream between the shared memory and the register file inside a single computing unit when the binding processor executes the general matrix multiplication operator, the second utilization rate is determined.

[0067] The second utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the computing unit level (for example, the execution efficiency of the general matrix multiplication operator at the computing unit level or the hardware computing resource utilization rate of the tensor core when the computing unit executes the general matrix multiplication operator). The second utilization rate is granularity-based on a single computing unit and is reflected by the proportional relationship between the effective working time when the tensor core of a single computing unit executes the computing task corresponding to the single computing unit and the total duration of the execution process of the computing task. Here, the effective working time refers to the effective working time when a single computing unit finishes executing a computing task. The computing task can be implemented by the task division method for the general matrix multiplication operator described in any embodiment of the present disclosure, which will not be elaborated here. The performance evaluation at the computing unit level regards each computing unit as an independent individual. From the perspective of a single computing unit, the input data is the corresponding tensor parts of the first tensor and the second tensor in the shared memory, the output data is the corresponding tensor part (a slice) of the third tensor that needs to be stored in the memory, and the workload is the workload of the computing task for obtaining a slice.

[0068] For example, the second utilization rate is indicated by the proportion of the second working duration in the second total duration, where the second working duration is the working duration of the tensor core of a single computing unit for performing tensor operations in a computing task, and the second total duration is the total duration of a single computing unit for finishing executing a computing task.

[0069] Different from the first utilization rate, the first utilization rate considers the processor as a whole, and focuses on and reflects the hardware utilization rate of completing the matrix multiplication operation of the first tensor and the second tensor. The second utilization rate focuses on a single computing unit, takes a single computing unit as the granularity, and considers the hardware utilization rate of a single computing unit completing a computing task. The total time taken to complete a computing task not only considers the working duration of the tensor core when executing the computing task, but also considers the time taken to load data from the shared memory of the computing unit to the register file and store the data in the register file to the shared memory.

[0070] The second utilization rate is also less than or equal to 1. If the second utilization rate is 1, it means that the utilization rate of the tensor core is the highest. At this time, the time taken for data loading and storage is completely hidden in the working time of the tensor core. The higher the second utilization rate, the higher the computing resource utilization rate of the tensor core in the computing unit when executing the computing task, the computing resources of the tensor core are fully utilized, and the tensor core is mostly performing effective calculations instead of being idle (such as waiting for data transmission, synchronization, or waiting for other computing units to complete work, etc.), the execution speed of the computing task is faster, the waiting time is minimized, and the overhead of data transmission and synchronization is smaller.

[0071] For example, in some embodiments, step S20 may include: determining the second interface time, where the second interface time indicates the time taken for data transmission between the shared memory inside a single computing unit and the register file of the single computing unit when the single computing unit executes a computing task; determining the second working duration based on the total amount of calculations for executing the computing task and the computing power of the tensor core in the single computing unit; and determining the second utilization rate based on the second working duration and the second interface time.

[0072] For example, in some embodiments, determining the second interface time may include: determining the bandwidth of the shared memory; determining the sum of the total data volume of the corresponding tensor part in the first tensor and the total data volume of the corresponding tensor part in the second tensor as the target data volume; and determining the second interface time according to the target data volume and the bandwidth of the shared memory.

[0073] For example, in some other embodiments, the shared memory in each computing unit includes a first memory block and a second memory block. The first memory block is used to cache the corresponding tensor part in the first tensor required for executing the computing task, and the second memory block is used to cache the corresponding tensor part in the second tensor required for executing the computing task. The first memory block and the second memory block have independent data transmission paths.

[0074] For example, refer to Figure 3 and Figure 4, A00, A01, A02, and A03 in the first tensor A are loaded from memory to the first memory block in shared memory, and B00, B10, B20, and B30 in the second tensor B are loaded from memory to the second memory block in shared memory. In addition, C00 in the third tensor is also stored from the register file to the first memory block in shared memory and then stored from the first memory block to memory. The first memory block and the second memory block have independent data transfer paths. As Figure 4 shown, in the data stream corresponding to the first tensor A, the data transfer path from shared memory to the register file is for the first memory block, and in the data stream corresponding to the second tensor B, the data transfer path from shared memory to the tensor core is for the second memory block. Also, the third tensor C also uses the data transfer path of the first memory block.

[0075] For example, determining the time consumption of the second interface may include: determining a first time consumption based on the total data volume of the corresponding tensor parts in the first tensor and the third tensor and the bandwidth of the first memory block; determining a second time consumption based on the total data volume of the corresponding tensor part in the second tensor and the bandwidth of the second memory block; and determining the larger of the first time consumption and the second time consumption as the time consumption of the second interface.

[0076] For example, the following formula can be used to determine the time consumption of the second interface: T_IO_SM = max(Data_low / BW_low, Data_high / BW_high) (Formula 5) where T_IO_SM represents the time consumption of the second interface, Data_low is the total data volume of the corresponding tensor parts in the first tensor and the third tensor, BW_low is the bandwidth of the first memory block, Data_high is the total data volume of the corresponding tensor part in the second tensor, BW_high is the bandwidth of the second memory block, and max() represents the maximum value function.

[0077] For example, for the computing task of obtaining slice C00, Data_low is the total data volume of A00, A01, A02, and A03 in the first tensor A and C00 in the third tensor C. Data_high is the total data volume of B00, B10, B20, and B30 in the second tensor B.

[0078] The time consumption of the second interface takes into account that the two memory blocks in the shared memory have independent data paths, and the two independent data paths transfer data in parallel. Therefore, the one with longer time consumption can be selected as the time consumption of the second interface. Therefore, using the independent bandwidth characteristics of the two memory blocks to determine the time consumption of the second interface fully considers the influencing factors of the data stream between the shared memory and the register file, and the obtained time consumption of the second interface is more accurate and closer to the actual situation. Furthermore, the accuracy of the obtained second utilization rate is higher.

[0079] For example, determining the second working duration based on the total computing volume of an executed computing task and the computing power of the tensor cores in a single computing unit may include: determining the total computing volume of the executed computing task based on the shape and size of the corresponding tensor part in the first tensor required for executing the computing task and the shape and size of the corresponding tensor part in the second tensor required for executing the computing task; determining the computing power of the tensor cores in the single computing unit according to the chip computing power of the processor and the total number of the multiple computing units; and determining the second working duration according to the total computing volume of the executed computing task and the computing power of the tensor cores in the single computing unit.

[0080] The second working duration is the duration spent by the tensor cores during the tensor operation of executing a computing task. For example, when executing the computing task for obtaining slice C00, it is the duration spent on performing the tensor operation between A00, A01, A02, A03 in the first tensor A and B00, B10, B20, B30 in the second tensor B.

[0081] For example, the second working duration can be determined using the following formula: T_Compute_SM = Workload2 / Compute_power2 (Formula 6) Wherein, T_Compute_SM represents the second working duration; Workload2 represents the total computing volume of a single computing unit for executing a computing task. For example, referring to the calculation method of Workload1, the total computing volume Workload2 of a single computing unit for executing a computing task can be determined based on the shape and size of the corresponding tensor part in the first tensor required for executing the computing task and the shape and size of the corresponding tensor part in the second tensor required for executing the computing task; Compute_power2 represents the computing power of the tensor cores in the single computing unit, and Compute_power2 = Compute_power1 / N, where N represents the total number of computing units in the processor, and N is a positive integer greater than 1.

[0082] For example, in some embodiments, determining the second utilization rate based on the second working duration and the second interface time consumption may include: determining the larger one of the second working duration and the second interface time consumption as the target duration; and determining the quotient of the second working duration and the target duration as the second utilization rate.

[0083] For example, the second utilization rate can be determined using the following formula: Gemm_util_SM = T_Compute_SM / max(T_IO_SM, T_Compute_SM) (Formula 7) Among them, Gemm_util_SM represents the second utilization rate, T_Compute_SM represents the second working duration, T_IO_SM represents the second interface time consumption, and max() represents the maximum value function.

[0084] As described above, the second utilization rate is also less than or equal to 1. If the second utilization rate is 1, it means that the utilization rate of the tensor core is the highest. At this time, the time consumption brought by data loading and storage is completely hidden in the working time consumption of the tensor core. The higher the second utilization rate, the higher the utilization rate of the computing resources of the tensor core in the computing unit when executing the computing task. The computing resources of the tensor core are fully utilized, and the tensor core is mostly performing effective calculations instead of being idle (such as waiting for data transmission, synchronization, or waiting for other computing units to complete work, etc.). The execution speed of the computing task is faster, the waiting time is minimized, and the overhead of data transmission and synchronization is smaller.

[0085] After obtaining the first utilization rate and the second utilization rate, as Figure 2 shown, in step S30, based on the smaller one of the first utilization rate and the second utilization rate, determine the theoretical performance of the processor when executing the general matrix multiplication operator.

[0086] In response to the first utilization rate or the second utilization rate being larger, the utilization rate of the computing resources of the tensor core of the processor when executing the general matrix multiplication operator is higher. As described above, the larger / higher the first utilization rate or the second utilization rate, the higher the utilization rate of the computing resources of the tensor core in the computing unit when executing the computing task. The computing resources of the tensor core are fully utilized, and the tensor core is mostly performing effective calculations instead of being idle (such as waiting for data transmission, synchronization, or waiting for other computing units to complete work, etc.). The execution speed of the computing task is faster, the waiting time is minimized, and the overhead of data transmission and synchronization is smaller.

[0087] For example, the theoretical performance can be indicated by the theoretical time consumption.

[0088] For example, step S30 may include: determining the smaller one of the first utilization rate and the second utilization rate as the target utilization rate; determining the quotient of the first working duration and the target utilization rate as the theoretical time consumption of the processor when executing the general matrix multiplication operator, where the theoretical time consumption is used to indicate the theoretical performance.

[0089] For example, the following formula can be used to determine the theoretical time consumption: T = T_Compute_chip / min(Gemm_util_chip, Gemm_util_SM) (Formula 8) Where, T represents the theoretical time consumption, T_Compute_chip represents the first working duration, min() represents the minimum value function, Gemm_util_chip represents the first utilization rate, and Gemm_util_SM represents the second utilization rate.

[0090] In the information processing method provided by at least one embodiment of the present disclosure, a two-level double-layer theoretical performance evaluation model is provided, including chip-level evaluation and computing unit-level evaluation. The chip-level evaluation is from the perspective of the entire processor, where the input and output data are stored in the memory, and the workload refers to completing the matrix operations of the complete first tensor and the second tensor. The processor includes multiple computing units. The computing unit-level evaluation is from the perspective of a single computing unit, where the input and output data are stored in the shared memory, and the workload refers to a single computing unit completing a computing task to obtain a tile of computation.

[0091] Therefore, the double-layer theoretical performance evaluation model provided by at least one embodiment of the present disclosure comprehensively considers the chip-level evaluation and the computing unit-level evaluation, and the evaluation result is more accurate. It is applicable to the programming model based on tile granularity, based on the multi-level storage architecture, focusing on the critical path analysis of the two-layer data stream. It not only considers the data stream from the memory to the shared memory, but also considers the data stream from the shared memory to the register file. In addition, it also comprehensively considers the influence of multiple factors of the bandwidth model, including the influence of the second-level cache hit rate on the equivalent bandwidth, and the influence of the independent bandwidth characteristics of the two memory blocks of the shared memory on the interface time consumption. The performance evaluation obtained thereby is more accurate and closer to the actual situation.

[0092] After obtaining the accurate theoretical time consumption of the processor for executing the GEMM operator, through this accurate theoretical time consumption, it is helpful to reasonably allocate computing resources. For example, according to the theoretical time consumption of the GEMM operator on the GPU, different tasks can be reasonably arranged to execute on different GPUs according to the priority of the tasks and the availability of resources; or through the estimated accurate theoretical time consumption, the GEMM operations with large computational volume and long time consumption can be allocated to GPUs with stronger performance, thereby improving the resource utilization rate of the entire cluster. In addition, the accurate theoretical time consumption can help developers design more reasonable parallel strategies, guide algorithm optimization, assist in model design, accurately predict the task completion time, and the accurate theoretical time consumption can be used as an important basis for the scheduling algorithm, which helps to achieve load balancing and improve the throughput of the overall system.

[0093] Figure 5 It is a schematic block diagram of an information processing device provided by at least one embodiment of the present disclosure.

[0094] This information processing device is used to evaluate the theoretical performance of the processor for executing the general matrix multiplication operator.

[0095] The processor includes multiple computing units and memory. The multiple computing units share the memory. Each computing unit includes a shared memory, a register bank, and a tensor core. The tensor core is used to perform tensor-related calculations in the general matrix multiplication operator. When data flows between the memory and the tensor core, it passes through the shared memory and the register bank in sequence. The shared memory is closer to the memory than the register bank. For more information about the processor, reference can be made to the relevant description of the foregoing information processing method, which will not be elaborated here.

[0096] As Figure 5 shown, the information processing device 100 includes a first determination module 101, a second determination module 102, and a third determination module 103.

[0097] The first determination module is configured to determine a first utilization rate by combining the data flow between the memory and the shared memory when the processor executes the general matrix multiplication operator, where the first utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the processor level.

[0098] The second determination module is configured to determine a second utilization rate by combining the data flow between the shared memory and the register bank within a single computing unit when the processor executes the general matrix multiplication operator, where the second utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the computing unit level.

[0099] The third determination module is configured to determine the theoretical performance of the processor when executing the general matrix multiplication operator based on the smaller one of the first utilization rate and the second utilization rate.

[0100] In response to the larger the first utilization rate or the second utilization rate, the higher the utilization rate of the computing resources of the tensor core when the processor executes the general matrix multiplication operator.

[0101] For example, the first determination module 101, the second determination module 102, and the third determination module 103 include codes and programs stored in a memory. The first determination module 101, the second determination module 102, and the third determination module 103 are implemented as, for example, a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities. The processing unit may be a general-purpose processor and may also be a single-chip microcomputer, a microprocessor, a digital signal processor, a dedicated image processing chip, or a field programmable logic array, etc. The first determination module 101, the second determination module 102, and the third determination module 103 execute the codes and programs to implement some or all of the functions of the first determination module 101, the second determination module 102, and the third determination module 103 as described above. For example, the first determination module 101, the second determination module 102, and the third determination module 103 may be a circuit board or a combination of multiple circuit boards for implementing the functions described above. In the embodiments of the present application, the circuit board or the combination of multiple circuit boards may include: (1) one or more processors; (2) one or more non-transitory memories connected to the processor; and (3) firmware stored in the memory that can be executed by the processor.

[0102] It should be noted that the first determination module 101 can be used to implement Figure 2 the step S10 shown; the second determination module 102 can be used to implement Figure 2 the step S20 shown; the third determination module 103 can be used to implement Figure 2 the step S30 shown. Therefore, for the specific description of the functions that the first determination module 101 can implement, reference can be made to the relevant description of step S10 in the embodiments of the above information processing method. For the specific description of the functions that the second determination module 102 can implement, reference can be made to the relevant description of step S20 in the embodiments of the above information processing method. For the specific description of the functions that the third determination module 103 can implement, reference can be made to the relevant description of step S30 in the embodiments of the above information processing method. Repeated parts will not be elaborated. In addition, the information processing device 100 can achieve technical effects similar to those of the foregoing information processing method, which will not be elaborated here.

[0103] It should be noted that in at least one embodiment of the present disclosure, the information processing device 100 may include more or fewer circuits or units, and the connection relationships between the respective circuits or units are not limited and may be determined according to actual needs. The specific composition manners of the respective circuits or units are not limited and may be composed of analog devices according to circuit principles, may also be composed of digital chips, or may be composed in other applicable manners.

[0104] For example, the information processing device 100 may be implemented in a manner combining hardware, software, or hardware and software. The present disclosure does not make specific limitations thereto.

[0105] In some embodiments, the information processing apparatus provided by at least one embodiment of the present disclosure may be a chip. For example, the chip is a System-on-a-Chip (SoC). The system-on-a-chip includes a processor, which may be a single-core processor or a multi-core processor, a memory, and an I / O interface, etc.

[0106] Figure 6 Schematic diagram of a non-transitory computer-readable storage medium provided by at least one embodiment of the present disclosure. For example, as Figure 6 shown, the storage medium 200 may be a non-transitory computer-readable storage medium, and one or more computer-readable instructions 201 may be non-temporarily stored on the storage medium 200. For example, when the computer-readable instructions 201 are executed by a processor, one or more steps in the information processing method described above may be executed.

[0107] For example, the storage medium 200 may be applied to an electronic device 300. For example, the storage medium 200 may include a storage device 308 in the electronic device 300.

[0108] For example, the storage device may include any combination of one or more computer program products. The computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer-readable instructions may be stored on the computer-readable storage medium, and the processor may run the computer-readable instructions to implement various functions of the processor. Various application programs and various data may also be stored in the storage medium.

[0109] For example, the storage medium may include a memory card of a smart phone, a cache component of a tablet computer, a hard disk of a personal computer, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), flash memory, or any combination of the above storage media, and may also be other applicable storage media.

[0110] Figure 7 Schematic block diagram of an electronic device provided by an embodiment of the present disclosure. As Figure 7 shown, the electronic device 300 is, for example, suitable for implementing the information processing method provided by the embodiments of the present disclosure. It should be noted that Figure 7The components of the electronic device 300 shown are merely exemplary and not restrictive. According to actual application requirements, the electronic device 300 may also have other components.

[0111] As Figure 7 shown, the electronic device 300 may include a processing device 301 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to non-transitory computer-readable instructions stored in the memory to implement various functions.

[0112] For example, when the computer-readable instructions are run by the processing device 301, one or more steps in the information processing method according to any of the above embodiments may be executed. It should be noted that for a detailed description of the processing process of the information processing method, reference may be made to the relevant descriptions in the embodiments of the above information processing method.

[0113] For example, the memory may include any combination of one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) 303 and / or cache memory, etc. For example, computer-readable instructions may be loaded from the storage device 308 into the random access memory (RAM) 303 to run the computer-readable instructions. Non-volatile memory may include, for example, read-only memory (ROM) 302, hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. Various application programs and various data may also be stored in the computer-readable storage media, such as style images, and various data used and / or generated by the application programs, etc.

[0114] For example, the processing device 301, read-only memory (ROM) 302, and random access memory (RAM) 303 are connected to each other through a bus 304. The input / output (I / O) interface 305 is also connected to the bus 304.

[0115] Generally, the following devices may be connected to the input / output (I / O) interface 305: an input device 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, magnetic tape, a hard disk, a flash memory, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other electronic devices wirelessly or wiredly to exchange data. Although Figure 7An electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or have all the shown devices, and the electronic device 300 may alternatively implement or have more or fewer devices. For example, the processing device 301 may control other components in the electronic device 300 to perform desired functions. The processing device 301 may be a central processing unit (CPU), a tensor processing unit (TPU), or a graphics processing unit (GPU), etc., which has data processing capabilities and / or program execution capabilities. The central processing unit (CPU) may be of the X86, ARM, RISC-V architecture, etc. The GPU may be directly integrated into the SOC, directly integrated onto the motherboard, or built into the northbridge chip of the motherboard.

[0116] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0117] The units involved in the embodiments described in the present disclosure may be implemented in software or in hardware. Among them, the name of the unit does not constitute a limitation to the unit itself in some cases.

[0118] The functions described above in this document may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), and so on.

[0119] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

[0120] In addition, although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments may also be implemented combinatorially in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0121] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and acts described above are merely example forms for implementing the claims.

[0122] For the present disclosure, the following points also need to be noted: (1) The drawings of the embodiments of the present disclosure only relate to the structures involved in the embodiments of the present disclosure, and other structures can refer to the general design.

[0123] (2) Without conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0124] The above is only the specific implementation manner of the present disclosure, but the protection scope of the present disclosure is not limited thereto. The protection scope of the present disclosure should be subject to the protection scope of the claimed rights.

Claims

1. An information processing method, characterized in that For evaluating the theoretical performance of a processor in executing a general matrix multiplication operator, wherein the processor includes a plurality of computing units and a memory, the plurality of computing units share the memory, each computing unit includes a shared memory, a register bank, and a tensor core, the tensor core is used to execute tensor-related calculations in the general matrix multiplication operator, and when data flows between the memory and the tensor core, it sequentially passes through the shared memory and the register bank, and the shared memory is closer to the memory than the register bank, The information processing method includes: Combining the data flow between the memory and the shared memory when the processor executes the general matrix multiplication operator, determining a first utilization rate, wherein the first utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the processor level, and the first utilization rate is reflected by the proportional relationship between the effective working time of the tensor core when executing the general matrix multiplication operator and the total elapsed time of the execution process of the general matrix multiplication operator; Combining the data flow between the shared memory and the register bank within a single computing unit when the processor executes the general matrix multiplication operator, determining a second utilization rate, wherein the second utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the computing unit level, the second utilization rate takes the single computing unit as a granularity, and is reflected by the proportional relationship between the effective working time of the tensor core when executing the calculation task corresponding to the single computing unit and the total elapsed time of the execution process of the calculation task; Based on the smaller one of the first utilization rate and the second utilization rate, determining the theoretical performance of the processor when executing the general matrix multiplication operator, wherein in response to the larger the first utilization rate or the second utilization rate, the higher the utilization rate of the calculation resources of the tensor core when the processor executes the general matrix multiplication operator.

2. The information processing method according to claim 1, wherein The general matrix multiplication operator is used to load a first tensor and a second tensor from the memory to the tensor cores in the plurality of computing units, and the tensor cores in the plurality of computing units execute the tensor multiplication operation of the first tensor and the second tensor to obtain a third tensor and store it in the memory, The third tensor is divided into L slices of the same size, the plurality of computing units execute a plurality of computing tasks in parallel to obtain a plurality of slices and store them in the memory, L is a positive integer, The corresponding computing task executed by each computing unit includes loading the corresponding tensor part of the first tensor and the second tensor from the memory to the tensor core of the computing unit, and using the tensor core of the computing unit to execute a tensor operation to obtain a slice as the calculation result and store it in the memory.

3. The information processing method according to claim 2, characterized in that The first utilization rate is indicated by the proportion of the first working duration in the first total elapsed time, the first working duration is the working duration when the tensor cores in the plurality of computing units execute the tensor multiplication operation of the first tensor and the second tensor, and the first total elapsed time is the total elapsed time for the processor to completely execute the general matrix multiplication operator, The second utilization rate is indicated by the ratio of the second working duration to the second total duration, where the second working duration is the working duration of the tensor core in the single computing unit for performing the tensor operation in a computing task, and the second total duration is the total duration for the single computing unit to complete the computing task.

4. The information processing method according to claim 2, characterized in that Determining a first utilization rate in combination with the data stream between the memory and the shared memory when the processor executes the general matrix multiplication operator includes: Determining a first equivalent bandwidth in combination with the data stream between the memory and the shared memory when the processor executes the general matrix multiplication operator; Determining a first interface duration based on the first equivalent bandwidth and the total data volume for executing the general matrix multiplication operator, where the total data volume is the sum of the data volumes of the first tensor, the second tensor, and the third tensor, and the first interface duration indicates the duration caused by data transmission between the memory and the shared memory when the processor executes the general matrix multiplication operator; Determining a first working duration based on the total computing amount for executing the general matrix multiplication operator and the chip computing power of the processor; Determining the first utilization rate based on the first working duration and the first interface duration.

5. The information processing method according to claim 4, wherein The processor further includes a secondary cache, and the data also passes through the secondary cache when flowing between the memory and the tensor core. The secondary cache is closer to the memory than the shared memory. Determining a first equivalent bandwidth in combination with the data stream between the memory and the shared memory when the processor executes the general matrix multiplication operator includes: Determining the bandwidth and hit rate of the secondary cache; Determining the bandwidth of the memory; Determining the first equivalent bandwidth based on the bandwidth and the hit rate of the secondary cache, and the bandwidth of the memory.

6. The information processing method according to claim 2, wherein Determining a second utilization rate in combination with the data stream between the shared memory and the register file inside a single computing unit when the processor executes the general matrix multiplication operator includes: Determining a second interface duration, where the second interface duration indicates the duration caused by data transmission between the shared memory inside the single computing unit and the register file of the single computing unit when the single computing unit executes the computing task; Determining a second working duration based on the total computing amount for executing the computing task and the computing power of the tensor core in the single computing unit; Determining the second utilization rate based on the second working duration and the second interface duration.

7. The information processing method according to claim 6, characterized in that, The shared memory in each computing unit includes a first memory block and a second memory block. The first memory block is used to cache the corresponding tensor parts of the first tensor and the third tensor required for executing the computing task, and the second memory block is used to cache the corresponding tensor part of the second tensor required for executing the computing task. The first memory block and the second memory block have independent data transmission paths. Determining the second interface duration includes: Determining a first duration based on the total data volume of the corresponding tensor parts of the first tensor and the third tensor and the bandwidth of the first memory block; Determine a second elapsed time based on the total data volume of the corresponding tensor part in the second tensor and the bandwidth of the second memory block; Determine the larger one of the first elapsed time and the second elapsed time as the second interface elapsed time.

8. The information processing method according to claim 6, wherein Determine a second working duration based on the total amount of computation for executing the computing task and the computing power of the tensor cores in the single computing unit, including: Determine the total amount of computation for executing the computing task based on the shape and size of the corresponding tensor part in the first tensor required for executing the computing task and the shape and size of the corresponding tensor part in the second tensor required for executing the computing task; Determine the computing power of the tensor cores in the single computing unit according to the chip computing power of the processor and the total number of the multiple computing units; Determine the second working duration according to the total amount of computation for executing the computing task and the computing power of the tensor cores in the single computing unit.

9. The information processing method according to claim 6, wherein Determine the second utilization rate based on the second working duration and the second interface elapsed time, including: Determine the larger one of the second working duration and the second interface elapsed time as the target duration; Determine the quotient of the second working duration and the target duration as the second utilization rate.

10. The information processing method according to claim 1, wherein Determine the theoretical performance of the processor when executing the general matrix multiplication operator based on the smaller one of the first utilization rate and the second utilization rate, including: Determine the smaller one of the first utilization rate and the second utilization rate as the target utilization rate; Determine the quotient of the first working duration and the target utilization rate as the theoretical elapsed time of the processor when executing the general matrix multiplication operator, where the theoretical elapsed time is used to indicate the theoretical performance.

11. An information processing apparatus, characterized in that, For evaluating the theoretical performance of a processor for executing a general matrix multiplication operator, where the processor includes multiple computing units and a memory, the multiple computing units share the memory, each computing unit includes a shared memory, a register bank, and tensor cores, the tensor cores are used to execute tensor-related computations in the general matrix multiplication operator, data flows through the shared memory and the register bank in sequence when flowing between the memory and the tensor cores, and the shared memory is closer to the memory than the register bank, The information processing device includes: A first determination module configured to determine a first utilization rate in combination with the data stream between the memory and the shared memory when the processor executes the general matrix multiplication operator, where the first utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the processor level, and the first utilization rate is reflected by the ratio relationship between the effective working time of the tensor cores when executing the general matrix multiplication operator and the total elapsed time of the execution process of the general matrix multiplication operator; A second determination module, configured to determine a second utilization rate by combining the data stream between the shared memory and the register file inside a single computing unit when the processor executes the general matrix multiplication operator, where the second utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the computing unit level, the second utilization rate is granularity-based on the single computing unit, and is reflected by the proportional relationship between the effective working time when the tensor core executes the computing task corresponding to the single computing unit and the total elapsed time of the execution process of the computing task; A third determination module, configured to determine the theoretical performance of the processor when executing the general matrix multiplication operator based on the smaller one of the first utilization rate and the second utilization rate, where in response to the larger first utilization rate or second utilization rate, the higher the utilization rate of the computing resources of the tensor core when the processor executes the general matrix multiplication operator.

12. An electronic device, characterized in that, Comprising: A memory that non-transiently stores computer-executable instructions; A processor, configured to run the computer-executable instructions, wherein, when the computer-executable instructions are run by the processor, the information processing method according to any one of claims 1-10 is implemented.

13. A non-transitory computer-readable storage medium, characterized in that The non-transient computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the information processing method according to any one of claims 1-10 is implemented.

Images