Information processing method and device, electronic device, and storage medium

Abstract

An information processing method and device, an electronic device, and a storage medium are applied to the field of information processing. The information processing method includes: determining a first utilization rate in combination with the data flow between the memory and the shared memory of the processor when executing a general matrix multiplication operator, wherein the first utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the processor level; determining a second utilization rate in combination with the data flow between the shared memory and the register stack within a single computing unit when the processor executes the general matrix multiplication operator, wherein the second utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the computing unit level; and determining the theoretical performance of the processor when executing the general matrix multiplication operator based on the smaller of the first utilization rate and the second utilization rate. The current performance evaluation of the GEMM operator only considers chip-level performance evaluation. The present application provides a theoretical performance evaluation with a dual-level dimension, and the evaluation result is more accurate.

Description

Technical Field

[0001] The present invention relates to an information processing method, an information processing device, an electronic device, and a non-transitory computer-readable storage medium. Background Art

[0002] General Matrix Multiplication (GEMM) is a fundamental operation in linear algebra, widely used in scientific computing, machine learning, deep learning, image processing, signal processing, artificial intelligence, and other fields. In processor architectures such as graphics processing units (GPUs) and general-purpose graphics processing units (GPGPUs), optimizing the performance of GEMM operators is a key step in compute-intensive tasks and is therefore crucial for improving computational efficiency. Current GEMM operator performance evaluation only considers chip-level performance, which is inaccurate. This makes it difficult for users to accurately assess the execution status of GEMM operators, hindering subsequent performance optimization and preventing targeted optimization. Summary of the Invention

[0003] The present invention application provides an information processing method for evaluating the theoretical performance of a processor executing a general matrix multiplication operator, wherein the processor includes multiple computing units and memory, the multiple computing units share the memory, each computing unit includes shared memory, a register stack and a tensor core, the tensor core is used to execute tensor-related calculations in the general matrix multiplication operator, and data passes through the shared memory and the register stack in sequence when flowing between the memory and the tensor core, and the shared memory is closer to the memory than the register stack. The information processing method includes: combining the data flow between the memory and the shared memory when the processor executes the general matrix multiplication operator, determining a first utilization rate, wherein the first utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the processor level, and the first utilization rate is obtained by the effective working time of the tensor core when executing the general matrix multiplication operator. The second utilization rate is reflected by the proportional relationship between the effective working time of the tensor core when executing the computing task corresponding to the single computing unit and the total execution time of the computing task; the second utilization rate is determined in combination with the data flow between the shared memory and the register stack within a single computing unit when the processor executes the general matrix multiplication operator, wherein the second utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the computing unit level, and the second utilization rate is reflected by the proportional relationship between the effective working time of the tensor core when executing the computing task corresponding to the single computing unit with the single computing unit as the granularity and the total execution time of the computing task; based on the smaller one of the first utilization rate and the second utilization rate, the theoretical performance of the processor when executing the general matrix multiplication operator is determined, wherein, in response to the larger the first utilization rate or the second utilization rate, the higher the computing resource utilization of the tensor core of the processor when executing the general matrix multiplication operator.

[0004] For example, in the information processing method provided in the present application, the general matrix multiplication operator is used to load the first tensor and the second tensor from the memory to the tensor cores in the multiple computing units, and the tensor cores in the multiple computing units perform tensor multiplication operations on the first tensor and the second tensor to obtain a third tensor and store it in the memory. The third tensor is divided into L slices of the same size. The multiple computing units perform multiple computing tasks in parallel to obtain multiple slices and store them in the memory. L is a positive integer. The computing tasks performed by each computing unit include loading the corresponding tensor parts of the first tensor and the second tensor from the memory to the tensor core of the computing unit, and using the tensor core of the computing unit to perform tensor operations to obtain slices as calculation results and store them in the memory.

[0005] For example, in the information processing method provided in the present application, the first utilization rate is indicated by the proportion of the first working time in the first total time, wherein the first working time is the working time of the tensor core in the multiple computing units when performing the tensor multiplication operation of the first tensor and the second tensor, and the first total time is the total time for the processor to completely execute the general matrix multiplication operator, and the second utilization rate is indicated by the proportion of the second working time in the second total time, wherein the second working time is the working time of the tensor core of the single computing unit to perform the tensor operation in a computing task, and the second total time is the total time for the single computing unit to complete the one computing task.

[0006] For example, in the information processing method provided in the present invention application, a first utilization rate is determined in combination with the data flow between the memory and the shared memory when the processor executes the general matrix multiplication operator, including: determining a first equivalent bandwidth in combination with the data flow between the memory and the shared memory when the processor executes the general matrix multiplication operator; determining a first interface time consumption based on the first equivalent bandwidth and the total data volume of executing the general matrix multiplication operator, wherein the total data volume is the sum of the data volume of the first tensor, the second tensor and the third tensor, and the first interface time consumption indicates the time consumption caused by the data transmission between the memory and the shared memory when the processor executes the general matrix multiplication operator; determining a first working time based on the total computing amount of executing the general matrix multiplication operator and the chip computing power of the processor; and determining the first utilization rate based on the first working time and the first interface time consumption.

[0007] For example, in the information processing method provided in the present application, the processor also includes a secondary cache, and the data also passes through the secondary cache when flowing between the memory and the tensor core. The secondary cache is closer to the memory than the shared memory. Combined with the data flow between the memory and the shared memory when the processor executes the general matrix multiplication operator, the first equivalent bandwidth is determined, including: determining the bandwidth and hit rate of the secondary cache; determining the bandwidth of the memory; determining the first equivalent bandwidth based on the bandwidth and hit rate of the secondary cache, and the bandwidth of the memory.

[0008] For example, in the information processing method provided in the present application, the second utilization rate is determined in combination with the data flow between the shared memory and the register stack within a single computing unit when the processor executes the general matrix multiplication operator, including: determining the second interface time, wherein the second interface time indicates the time consumed by the data transmission between the shared memory within the single computing unit and the register stack of the single computing unit when the single computing unit performs the computing task; determining the second working time based on the total computing amount of executing the computing task and the computing power of the tensor core in the single computing unit; and determining the second utilization rate based on the second working time and the second interface time.

[0009] For example, in the information processing method provided in the present application, the shared memory in each computing unit includes a first memory block and a second memory block, the first memory block is used to cache the corresponding tensor parts of the first tensor and the third tensor required to execute the computing task, and the second memory block is used to cache the corresponding tensor parts of the second tensor required to execute the computing task, the first memory block and the second memory block have independent data transmission paths, and determining the second interface time consumption includes: determining the first time consumption based on the total data volume of the corresponding tensor parts of the first tensor and the third tensor and the bandwidth of the first memory block; determining the second time consumption based on the total data volume of the corresponding tensor parts of the second tensor and the bandwidth of the second memory block; and determining the larger of the first time consumption and the second time consumption as the second interface time consumption.

[0010] For example, in the information processing method provided in the present application, the second working time is determined based on the total computing amount of executing the computing task and the computing power of the tensor core in the single computing unit, including: determining the total computing amount of executing the computing task based on the shape size of the corresponding tensor part in the first tensor required to execute the computing task, and the shape size of the corresponding tensor part in the second tensor required to execute the computing task; determining the computing power of the tensor core in the single computing unit according to the chip computing power of the processor and the total number of the multiple computing units; determining the second working time according to the total computing amount of executing the computing task and the computing power of the tensor core in the single computing unit.

[0011] For example, in the information processing method provided in the present invention application, the second utilization rate is determined based on the second working time and the second interface time, including: determining the larger one of the second working time and the second interface time as the target time; determining the quotient of the second working time and the target time as the second utilization rate.

[0012] For example, in the information processing method provided in the present invention application, the theoretical performance of the processor when executing the general matrix multiplication operator is determined based on the smaller of the first utilization rate and the second utilization rate, including: determining the smaller of the first utilization rate and the second utilization rate as the target utilization rate; determining the quotient of the first working time and the target utilization rate as the theoretical time consumption of the processor when executing the general matrix multiplication operator, wherein the theoretical time consumption is used to indicate the theoretical performance.

[0013] The present invention application provides an information processing device for evaluating the theoretical performance of a processor executing a general matrix multiplication operator, wherein the processor includes multiple computing units and memory, the multiple computing units share the memory, each computing unit includes shared memory, a register stack and a tensor core, the tensor core is used to execute tensor-related calculations in the general matrix multiplication operator, and data passes through the shared memory and the register stack in sequence when flowing between the memory and the tensor core, and the shared memory is closer to the memory than the register stack. The information processing device includes: a first determination module, configured to determine a first utilization rate in combination with the data flow between the memory and the shared memory when the processor executes the general matrix multiplication operator, wherein the first utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the processor level, and the first utilization rate is obtained by comparing the effective working time of the tensor core when executing the general matrix multiplication operator with the The second determination module is configured to determine a second utilization rate in combination with the data flow between the shared memory and the register stack within a single computing unit when the processor executes the general matrix multiplication operator, wherein the second utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the computing unit level, and the second utilization rate is embodied by the proportional relationship between the effective working time when the tensor core executes the computing task corresponding to the single computing unit and the total execution time of the computing task with the single computing unit as the granularity; the third determination module is configured to determine the theoretical performance of the processor when executing the general matrix multiplication operator based on the smaller of the first utilization rate and the second utilization rate, wherein, in response to the larger the first utilization rate or the second utilization rate, the higher the computing resource utilization of the tensor core of the processor when executing the general matrix multiplication operator.

[0014] The present invention application provides an electronic device, comprising: a memory, which non-transiently stores computer-executable instructions; and a processor, configured to execute the computer-executable instructions, wherein the computer-executable instructions, when executed by the processor, implement the information processing method according to any embodiment of the present disclosure.

[0015] The present invention application provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the information processing method according to any embodiment of the present disclosure is implemented.

[0016] The information processing method provided by this application provides a two-level theoretical performance evaluation, resulting in more accurate results. Based on a three-tiered storage structure, this method focuses on critical path analysis of two data flows (from memory to shared memory, and from shared memory to registers). By comprehensively considering the impact of multiple factors, a more realistic performance evaluation model is established, helping developers better understand the execution status of GEMM operators. This facilitates targeted optimization, better utilizes processor resources, optimizes program performance, and reduces development costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, rather than limiting the present disclosure.

[0018] Figure 1 A schematic structural diagram of a general-purpose graphics processing unit (GPGPU);

[0019] Figure 2 A schematic flowchart of an information processing method provided in at least one embodiment of the present disclosure;

[0020] Figure 3 A schematic diagram of a sheet provided in accordance with an embodiment of the present disclosure;

[0021] Figure 4 A schematic diagram of the data flow of a GEMM operator provided in one embodiment of the present disclosure;

[0022] Figure 5 A schematic block diagram of an information processing device provided in at least one embodiment of the present disclosure;

[0023] Figure 6 A schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure;

[0024] Figure 7 A schematic block diagram of an electronic device provided in accordance with an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0026] Unless otherwise defined, the technical or scientific terms used in this disclosure should have the usual meanings understood by people with ordinary skills in the field to which this disclosure belongs. The "first", "second" and similar words used in this disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits detailed descriptions of some known functions and known components.

[0027] Figure 1 A schematic structural diagram of a general-purpose graphics processing unit (GPGPU).

[0028] like Figure 1 As shown, the general purpose graphics processor is actually an array of programmable multiprocessors. For example, the programmable multiprocessor can be a streaming processor cluster (SPC), including Figure 1 Streaming processor clusters 1, ..., and M are shown, where M is a positive integer greater than 1. In a general-purpose graphics processor, one streaming processor cluster processes one computing task, or multiple streaming processor clusters process one computing task. Multiple streaming processor clusters share data through a global cache or global memory.

[0029] like Figure 1 As shown, taking stream processor cluster 1 as an example, a stream processor cluster includes multiple computing units, such as Figure 1Compute Unit 1, Compute Unit 2, ..., Compute Unit N, where N is a positive integer. Each Compute Unit (CU) is used to perform arithmetic and logical operations, such as accumulation, reduction, conventional addition, subtraction, multiplication, and division. A CU includes multiple cores (also called computing cores or cores). Each core includes an arithmetic logic unit (ALU), a floating-point unit, and other components. The core is used to perform specific computing tasks. In addition, the CU includes a register file and shared memory for hierarchical storage of source and destination data related to computing tasks. The shared memory in a CU is used to share data between the cores of the CU. The register file is closer to the CU core than the shared memory, and the register file has a higher transmission bandwidth but a smaller capacity than the shared memory.

[0030] like Figure 1 As shown, each compute unit also provides a Tensor Core for performing tensor calculations, such as matrix multiplication using the GEMM operator. Tensors are crucial data structures in deep learning. They are a high-dimensional generalization of scalars, vectors, and matrices. Tensor operations are commonly used in the training and inference of deep learning models, and Tensor Cores can accelerate matrix multiplication. Tensor Cores in multiple compute units can be centrally scheduled and controlled.

[0031] In parallel computing, computing tasks are generally executed by multiple threads. These threads are divided into multiple thread blocks before being executed in a general-purpose graphics processor (or parallel computing processor), and then distributed through the thread block distribution module ( Figure 1 (Not shown) Multiple thread blocks are distributed to various CUs. All threads in a thread block must be assigned to the same CU for execution. Thread blocks are also split into minimum execution warps (or simply warps). Each warp contains a fixed number (or less) of threads, for example, 32 threads. Multiple thread blocks can execute in the same CU or in different CUs.

[0032] In each computational unit, the warp scheduling / dispatching module ( Figure 1 (not shown) schedules and allocates thread warps so that the multiple compute cores of the compute unit can execute the warps. Depending on the number of compute cores in the compute unit, multiple warps in a thread block can execute simultaneously or in a time-sharing manner. Multiple threads in each warp execute the same instruction. Memory execution instructions are emitted to the shared memory within the compute unit or further emitted to the mid-level cache, global cache, or global memory for read and write operations.

[0033] like Figure 1 As shown, GPUs employ a three-level memory architecture. Global memory (e.g., high-bandwidth memory) typically has the largest capacity and the lowest bandwidth. Shared memory, typically at the granularity of a compute unit, has a smaller capacity and higher bandwidth. For example, some shared memory implementations divide shared memory into two banks, called the high bank and the low bank, each with independent data transfer paths. The register stack, located closest to the compute cores, has the smallest capacity and the highest bandwidth.

[0034] Artificial intelligence (AI) models are mathematical models used to perform specific tasks, typically built using machine learning or deep learning techniques. These models learn patterns and relationships from training data to achieve tasks such as prediction, classification, and recognition. Common AI models include neural networks, decision trees, and support vector machines. Operators are the basic computational units in AI models, used to perform specific mathematical operations. Operators can be simple mathematical operations (such as addition and multiplication) or complex matrix operations (such as matrix multiplication and convolution). For example, the GEMM operator is used to perform general matrix multiplication.

[0035] The tensor-related calculations performed by the GEMM operator include A×B=C, where A represents the first tensor, B represents the second tensor, and C represents the third tensor. Tensor cores can be used to calculate A×B to obtain the calculation result C. For example, if the first tensor A and the second tensor B are stored in memory, the GEMM operator must first load the first tensor A and the second tensor B from memory, calculate the third tensor C, and then store it back in memory.

[0036] The efficient implementation of operators relies on the parallel computing capabilities of, for example, graphics processors (GPUs) or general-purpose graphics processors (GPGPUs). Accurate performance evaluation can help developers better understand the execution of operators and implement targeted optimizations. In layered hardware architectures like GPUs or general-purpose graphics processors, current GEMM operator performance evaluation methods only consider the flow of data from memory to shared memory, ignoring critical path analysis within the compute unit. The resulting performance evaluation results are inaccurate. Furthermore, current evaluation methods use a single evaluation dimension and fail to distinguish between chip-level and compute unit-level evaluation dimensions, resulting in a discrepancy between theoretical performance predictions and actual hardware behavior.

[0037] At least one embodiment of the present disclosure provides an information processing method, device, electronic device and storage medium. The information processing method is used to evaluate the theoretical performance of a GEMM operator in a processor, the processor includes multiple computing units and memory, the multiple computing units share memory, each computing unit includes shared memory, register stack and tensor core, the tensor core is used to perform tensor-related calculations in a general matrix multiplication operator, and data flows between the memory and the tensor core through the shared memory and register stack in sequence, and the shared memory is closer to the memory than the register stack. The information processing method includes: combining the data flow between the memory and the shared memory when the processor executes the general matrix multiplication operator, determining a first utilization rate, wherein the first utilization rate is used to indicate the general matrix multiplication operator. The invention relates to a method for performing a performance evaluation of a tensor core at a processor level; determining a second utilization rate in combination with the data flow between the shared memory and the register file within a single computing unit when the processor executes the general matrix multiplication operator, wherein the second utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the computing unit level; and determining the theoretical performance of the processor when executing the general matrix multiplication operator based on the smaller of the first utilization rate and the second utilization rate, wherein, in response to a larger first utilization rate or the second utilization rate, a higher computing resource utilization rate of the tensor core of the processor when executing the general matrix multiplication operator.

[0038] At least one embodiment of the present disclosure provides an information processing method that provides a two-level theoretical performance evaluation, resulting in more accurate results. Based on a three-tiered storage structure, this method focuses on critical path analysis of two data flows (from memory to shared memory, and from shared memory to registers). By comprehensively considering the impact of multiple factors, a more realistic performance evaluation model is established, helping developers better understand the execution status of GEMM operators. This facilitates targeted optimization, better utilizes processor resources, optimizes program performance, and reduces development costs.

[0039] The information processing method provided in the embodiments of the present disclosure may be applied to the information processing apparatus provided in the embodiments of the present disclosure, and the information processing apparatus may be configured on an electronic device, such as a host in a server.

[0040] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings, but the present disclosure is not limited to these specific embodiments.

[0041] Figure 2 This is a schematic flow chart of an information processing method provided by at least one embodiment of the present disclosure. Figure 2 As shown, the information processing method includes steps S10-S30.

[0042] At least one embodiment of the present disclosure provides an information processing method for evaluating the theoretical performance of a processor executing a general matrix multiplication operator. For example, the theoretical performance can be reflected by the theoretical time taken by the processor to execute the general matrix multiplication operator. Of course, in some other embodiments, the theoretical performance can be reflected by other parameters that can reflect the theoretical performance, such as theoretical throughput, theoretical peak performance, theoretical resource utilization, etc.

[0043] The processor includes multiple computing units and memory, and multiple computing units share memory. Each computing unit includes shared memory, register file and tensor core. Tensor core is used to perform tensor-related calculations in general matrix multiplication operators. When data flows between memory and tensor core, it passes through shared memory and register file in turn. Shared memory is closer to memory than register file. For example, the structure of the processor can be referred to Figure 1 As shown, for example, the processor may be a graphics processor or a general-purpose graphics processor.

[0044] like Figure 1 As shown in the figure, when data is loaded from memory to the tensor core of a compute unit, it is first loaded into shared memory (if the processor has a L2 cache, it is first loaded into the L2 cache before being loaded into shared memory), then loaded from shared memory into the register file, and then from the register file into the tensor core. Similarly, when data is stored from a tensor core to memory, it is first stored into the register file, then from the register file to shared memory, then from shared memory to the L2 cache (if available), and finally from the L2 cache to memory.

[0045] For example, a first tensor A and a second tensor B are stored in memory. A general matrix multiplication operator is used to load these first tensors A and B from memory into tensor cores in multiple compute units. These tensor cores then perform the tensor multiplication on these first tensors A and B, generating a third tensor C and storing it in memory. In other words, the time required to execute the general matrix multiplication operator includes the time required to load and store the data, as well as the time required for the tensor cores to perform the matrix multiplication.

[0046] like Figure 1 As mentioned above, a graphics processor or general-purpose graphics processor often includes multiple computational units. When executing a GEMM operator, the third tensor C is typically divided into multiple tiles of equal size. For example, the third tensor C is divided into L tiles of equal size. Multiple computational units perform multiple computational tasks in parallel to obtain multiple tiles and store them in memory, where L is a positive integer. L can be the same as or different from the number of computational units.

[0047] The computing task performed by each computing unit includes loading the corresponding tensor parts of the first tensor and the second tensor from the memory to the tensor core of the computing unit, and the tensor core of the computing unit performs tensor operations to obtain slices as calculation results and store them in the memory.

[0048] For example, the first tensor A and the second tensor B are split by obtaining each slice as the task target of the computing task, each computing unit performs a computing task to obtain a slice of the third tensor as the computing result, and multiple computing units execute multiple computing tasks in parallel, and the multiple computing tasks are independent of each other.

[0049] Figure 3 A schematic diagram of a sheet provided in accordance with an embodiment of the present disclosure.

[0050] like Figure 3 As shown, the third tensor C is divided into 4 slices, namely C00, C01, C02, and C03. The 4 slices correspond to 4 computing tasks executed in parallel by 4 computing units. Taking the computing task of slice C00 as an example, the data to be loaded for computing C00 includes Figure 3 In the example, A00, A01, A02, and A03 in the first tensor A, and B00, B10, B20, and B30 in the second tensor B, are the corresponding tensor parts in the first tensor that need to be loaded into the tensor cores, and B00, B10, B20, and B30 in the second tensor B are the corresponding tensor parts in the second tensor that need to be loaded into the tensor cores. The logic for other slices such as C01 is similar and will not be repeated here.

[0051] For example, if the third tensor C cannot be divided into multiple slices of the same size, some slices can be aligned to the size of the slice by filling with preset values, etc. When executing the information processing method provided by at least one embodiment of the present disclosure, the total computational load, total data volume, etc. are determined according to the aligned size.

[0052] Figure 4 A schematic diagram of the data flow of a GEMM operator provided in one embodiment of the present disclosure.

[0053] The first tensor A and the second tensor B are stored in global memory.

[0054] like Figure 4 As shown, for the GEMM operator, to perform the operation to obtain Figure 3Taking the computing unit 1 of the computing task of the middle slice C00 as an example, when the computing task of C00 is executed by the tensor core in computing unit 1, A00, A01, A02, and A03 in the first tensor A are first loaded from the global memory to the shared memory of computing unit 1, then loaded from the shared memory to the register file of computing unit 1, and then loaded from the register file to the tensor core of computing unit 1. For B00, B10, B20, and B30 in the second tensor B, they are first loaded from the global memory to the shared memory of computing unit 1, and then loaded from the shared memory to the tensor core of computing unit 1. Of course, in other embodiments, they can also be loaded from the shared memory to the register file of computing unit 1 first, and then loaded from the register file to the tensor core of computing unit 1.

[0055] Afterwards, tensor operations are performed in the Tensor Core on the corresponding tensor parts A00, A01, A02, and A03 in the first tensor A and the corresponding tensor parts B00, B10, B20, and B30 in the second tensor B, resulting in slice C00 as the computation result. Slice C00, as the computation result, needs to be stored in memory. The data flow is as follows: slice C00 is first stored in the register file, then in the shared memory of Compute Unit 1, and finally in memory.

[0056] Of course, it should be noted that the above embodiment is a division method for allocating the entire computing task of the general matrix multiplication operator to multiple computing units for execution, but the present disclosure is not limited to this. Those skilled in the art can also adopt other feasible implementation methods to determine the computing tasks performed by each computing unit.

[0057] The specific process of the information processing method provided by at least one embodiment of the present disclosure is described in detail below.

[0058] like Figure 2 As shown, in step S10, the first utilization rate is determined in combination with the data flow between the memory and the shared memory when the processor executes the general matrix multiplication operator.

[0059] The first utilization rate is used to evaluate the performance of a general matrix multiplication operator at the processor level (for example, the processor-level execution efficiency of the general matrix multiplication operator or the hardware computing resource utilization of the tensor core when the processor / chip executes the general matrix multiplication operator). The first utilization rate is reflected by the ratio between the effective working time of the tensor core when executing the general matrix multiplication operator and the total execution time of the general matrix multiplication operator. Processor-level performance evaluation considers multiple computing units as a whole. From the perspective of the entire chip, the input data is the first and second tensors stored in memory, the output data is the third tensor that needs to be stored in memory, and the workload is the calculation of the complete GEMM operator.

[0060] For example, the first utilization rate is indicated by the proportion of the first working time in the first total time. The first working time is the working time when the tensor cores in the multiple computing units perform the tensor multiplication operation of the first tensor and the second tensor, and the first total time is the total time it takes for the processor to completely execute the general matrix multiplication operator. As mentioned above, the total time it takes to completely execute the general matrix multiplication operator not only takes into account the working time / occupancy time of the tensor cores executing the general matrix multiplication operator, but also takes into account the time it takes to load data from the memory and store data to the memory.

[0061] The first utilization is less than or equal to 1. If the first utilization is 1, it indicates the highest utilization of the Tensor Cores. In this case, the time consumed by data loading and storing is completely hidden in the time consumed by the Tensor Cores. The higher the first utilization, the higher the utilization of the Tensor Cores' computing resources when the processor executes the general matrix multiplication operator. The Tensor Cores' computing resources are fully utilized, and the Tensor Cores spend most of their time performing effective calculations rather than being idle (for example, waiting for data transmission, synchronization, or waiting for other computing units to complete work). The entire GEMM operator executes faster, waiting time is minimized, and the overhead of data transmission and synchronization is reduced.

[0062] For example, in some embodiments, step S10 may include: determining a first equivalent bandwidth in combination with the data flow between the memory and the shared memory when the processor executes a general matrix multiplication operator; determining a first interface time consumption based on the first equivalent bandwidth and the total data volume of executing the general matrix multiplication operator, wherein the data volume is the sum of the data volume of the first tensor, the second tensor, and the third tensor, and the first interface time consumption indicates the time consumption caused by the data transmission between the memory and the shared memory when the processor executes the general matrix multiplication operator; determining a first working time based on the total computing amount of executing the general matrix multiplication operator and the chip computing power of the processor; and determining a first utilization rate based on the first working time and the first interface time consumption.

[0063] For example, the processor also includes a L2 cache, and data flows between the memory and the tensor cores through the L2 cache. The L2 cache is closer to the memory than the shared memory. In some embodiments, determining the first equivalent bandwidth based on the data flow between the memory and the shared memory when the processor executes a general matrix multiplication operator includes: determining the bandwidth and hit rate of the L2 cache; determining the bandwidth of the memory; and determining the first equivalent bandwidth based on the bandwidth and hit rate of the L2 cache and the bandwidth of the memory.

[0064] For example, the first equivalent bandwidth can be obtained using the following formula:

[0065] BW_equiv=BW_L2 × hit_rate + BW_GLM × (1-hit_rate) (Formula 1)

[0066] Among them, BW_equiv represents the first equivalent bandwidth, BW_L2 represents the bandwidth of the second-level cache, hit_rate represents the hit rate of the second-level cache, and BW_GLM represents the memory bandwidth.

[0067] The first equivalent bandwidth takes into account memory bandwidth and the hit rate of the L2 cache. When data is transferred from memory to the compute unit, it passes through the L2 cache and is cached there. When loading data from memory, if the data exists in the L2 cache, it's considered a "hit," eliminating the need to read from memory. If the data doesn't exist in the L2 cache, it's considered a "miss," requiring a read from memory.

[0068] When loading the first and second tensors from memory, the hit portion in the L2 cache is retrieved from the L2 cache, and the data transfer bandwidth is the L2 cache bandwidth; the miss portion is retrieved from memory, and the data transfer bandwidth is the memory bandwidth. The first equivalent bandwidth is determined by taking into account the impact of factors such as the L2 cache hit rate on the equivalent bandwidth. This results in a more accurate and realistic first equivalent bandwidth, and thus a more precise first utilization.

[0069] For example, in other embodiments, only the memory bandwidth may be considered to determine the first equivalent bandwidth, which will not be described in detail here.

[0070] For example, the first interface time consumption can be calculated using the following formula:

[0071] T_IO_chip = Data_total / BW_equiv (Formula 2)

[0072] Where T_IO_chip represents the first interface time, Data_total represents the total data size required to execute the general matrix multiplication operator, which is the sum of the data sizes of the first, second, and third tensors. " / " represents division. As shown in Formula 2, the first interface time reflects the time it takes to load the first and second tensors into the Tensor Cores and the time it takes to store the third tensor into memory when executing the general matrix multiplication operator.

[0073] The first working time is the time it takes for the tensor core to complete the tensor multiplication operation A×B between the first tensor A and the second tensor B. For example, the first working time can be determined using the following formula:

[0074] T_Compute_chip=Workload1 / Compute_power1 (Formula 3)

[0075] Among them, T_Compute_chip represents the first working time, Workload1 represents the total computational amount of executing the general matrix multiplication operator, and the unit is Tflops, which is the unit of operands. Compute_power1 represents the chip computing power of the processor, and the unit is Tflops / s, which means the number of operations executed per second.

[0076] For example, if the shape of the first tensor A is M×K, the shape of the second tensor B is K×N, and the shape of the third tensor C is M×N, the total computational effort can be expressed as M×N×K×2 (Tflops).

[0077] For example, the first utilization rate may be determined by referring to the following formula:

[0078] Gemm_util_chip=T_Compute_chip / max(T_IO_chip,T_Compute_chip) (Formula 4)

[0079] Among them, Gemm_util_chip represents the first utilization rate, T_Compute_chip represents the first working time, max() represents the maximum value function, and T_IO_chip represents the first interface time consumption.

[0080] For example, in step S20, the second utilization rate is determined in combination with the data flow between the shared memory and the register file within a single computing unit when the processor executes the general matrix multiplication operator.

[0081] The second utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the computing unit level (for example, the execution efficiency of the general matrix multiplication operator at the computing unit level or the hardware computing resource utilization of the tensor core when the computing unit executes the general matrix multiplication operator). The second utilization rate is based on the granularity of a single computing unit and is reflected by the ratio between the effective working time when the tensor core executes the computing task corresponding to the single computing unit and the total execution time of the computing task. The effective working time here refers to the effective working time when a single computing unit completes a computing task. The computing task can be implemented by the task partitioning method for the general matrix multiplication operator described in any embodiment of the present disclosure, which will not be repeated here. The performance evaluation at the computing unit level treats each computing unit as an independent individual. From the perspective of a single computing unit, the input data is the corresponding tensor portion of the first tensor and the second tensor in the shared memory, the output data is the corresponding tensor portion (a slice) of the third tensor that needs to be stored in the memory, and the workload is the workload of the computing task used to obtain a slice.

[0082] For example, the second utilization rate is indicated by the proportion of the second working time in the second total time, where the second working time is the working time of the tensor core of a single computing unit to perform tensor operations in a computing task, and the second total time is the total time taken for a single computing unit to complete a computing task.

[0083] Unlike the first utilization rate, which considers the processor as a whole and focuses on and reflects the hardware utilization of executing the matrix multiplication operation of the first tensor and the second tensor. The second utilization rate focuses on a single computing unit, using a single computing unit as the granularity, and considers the hardware utilization of a single computing unit to complete a computing task. The total time required to complete a computing task not only considers the working time of the tensor core when executing the computing task, but also takes into account the time required to load data from the computing unit's shared memory to the register stack and store data from the register stack to the shared memory.

[0084] The second utilization rate is also less than or equal to 1. If the second utilization rate is 1, it indicates the highest utilization of the Tensor Cores. In this case, the time consumed by data loading and storing is completely hidden in the tensor core working time. The higher the second utilization rate, the higher the computing resource utilization of the tensor cores in the computing unit when executing computing tasks. The computing resources of the tensor cores are fully utilized, and the tensor cores spend most of their time performing effective calculations rather than being idle (for example, waiting for data transmission, synchronization, or waiting for other computing units to complete work). This speeds up the execution of computing tasks, minimizes waiting time, and reduces the overhead of data transmission and synchronization.

[0085] For example, in some embodiments, step S20 may include: determining a second interface time, wherein the second interface time indicates the time consumed by data transmission between the shared memory within a single computing unit and the register stack of a single computing unit when the single computing unit performs a computing task; determining a second working time based on the total computing amount of the computing task and the computing power of the tensor core in the single computing unit; and determining a second utilization rate based on the second working time and the second interface time.

[0086] For example, in some embodiments, determining the second interface time consumption may include: determining the bandwidth of the shared memory; determining the sum of the total data volume of the corresponding tensor part in the first tensor and the total data volume of the corresponding tensor part in the second tensor as the target data volume; and determining the second interface time consumption based on the target data volume and the bandwidth of the shared memory.

[0087] For example, in other embodiments, the shared memory in each computing unit includes a first memory block and a second memory block, the first memory block is used to cache the corresponding tensor part of the first tensor required to execute the computing task, and the second memory block is used to cache the corresponding tensor part of the second tensor required to execute the computing task, and the first memory block and the second memory block have independent data transmission paths.

[0088] For example, reference Figure 3 and Figure 4 , A00, A01, A02, and A03 in the first tensor A are loaded from the main memory to the first memory block in the shared memory, and B00, B10, B20, and B30 in the second tensor B are loaded from the main memory to the second memory block in the shared memory. In addition, C00 in the third tensor is also stored from the register file to the first memory block in the shared memory, and from the first memory block to the main memory. The first memory block and the second memory block have independent data transfer paths. Figure 4 As shown, in the data flow corresponding to the first tensor A, the data transmission path from shared memory to the register file is the first memory block. In the data flow corresponding to the second tensor B, the data transmission path from shared memory to the tensor core is the second memory block. Furthermore, the third tensor C also uses the data transmission path of the first memory block.

[0089] For example, determining the second interface consumption may include: determining the first consumption based on the total data volume of the corresponding tensor parts in the first tensor and the third tensor and the bandwidth of the first memory block; determining the second consumption based on the total data volume of the corresponding tensor parts in the second tensor and the bandwidth of the second memory block; and determining the larger of the first time consumption and the second time consumption as the second interface consumption.

[0090] For example, the second interface time consumption can be determined using the following formula:

[0091] T_IO_SM=max(Data_low / BW_low, Data_high / BW_high) (Formula 5)

[0092] Among them, T_IO_SM represents the time consumption of the second interface, Data_low is the total data amount of the corresponding tensor parts in the first tensor and the third tensor, BW_low is the bandwidth of the first memory block, Data_high is the total data amount of the corresponding tensor parts in the second tensor, BW_high is the bandwidth of the second memory block, and max() represents the maximum value function.

[0093] For example, for a computation task to obtain slice C00, Data_low is the total data size of A00, A01, A02, and A03 in the first tensor A and C00 in the third tensor C. Data_high is the total data size of B00, B10, B20, and B30 in the second tensor B.

[0094] The second interface duration takes into account the independent data paths of the two memory blocks in the shared memory. Since these two independent data paths transmit data in parallel, the longer data path can be selected as the second interface duration. Therefore, using the independent bandwidth characteristics of the two memory blocks to determine the second interface duration fully considers the impact of data flow between the shared memory and the register file. This results in a more accurate and realistic second interface duration, and thus a more precise second utilization.

[0095] For example, determining the second working time based on the total computing amount of executing the computing task and the computing power of the tensor core in a single computing unit may include: determining the total computing amount of executing the computing task based on the shape size of the corresponding tensor part in the first tensor required to execute the computing task, and the shape size of the corresponding tensor part in the second tensor required to execute the computing task; determining the computing power of the tensor core in the single computing unit according to the chip computing power of the processor and the total number of the multiple computing units; determining the second working time according to the total computing amount of executing the computing task and the computing power of the tensor core in the single computing unit.

[0096] The second working time is the time spent by the tensor core in performing tensor operations in a computing task. For example, when executing a computing task for obtaining slice C00, the time spent calculating the tensor operations between A00, A01, A02, A03 in the first tensor A and B00, B10, B20, B30 in the second tensor B is.

[0097] For example, the second working time can be determined using the following formula:

[0098] T_Compute_SM=Workload2 / Compute_power2 (Formula 6)

[0099] Among them, T_Compute_SM represents the second working time; Workload2 represents the total computing amount of a single computing unit to perform a computing task. For example, you can refer to the calculation method of Workload1, and determine the total computing amount Workload2 of a single computing unit to perform a computing task based on the shape size of the corresponding tensor part in the first tensor required to perform the computing task, and the shape size of the corresponding tensor part in the second tensor required to perform the computing task; Compute_power2 represents the computing power of the tensor core in a single computing unit, Compute_power2=Compute_power1 / N, N represents the total number of computing units in the processor, and N is a positive integer greater than 1.

[0100] For example, in some embodiments, determining the second utilization rate based on the second working duration and the second interface time may include: determining the larger of the second working duration and the second interface time as the target duration; and determining the quotient of the second working duration and the target duration as the second utilization rate.

[0101] For example, the second utilization rate may be determined using the following formula:

[0102] Gemm_util_SM= T_Compute_SM / max(T_IO_SM, T_Compute_SM) (Formula 7)

[0103] Among them, Gemm_util_SM represents the second utilization, T_Compute_SM represents the second working time, T_IO_SM represents the second interface time consumption, and max() represents the maximum value function.

[0104] As mentioned above, the second utilization is also less than or equal to 1. If the second utilization is 1, it indicates that the tensor core utilization is the highest. At this time, the time consumed by data loading and storage is completely hidden in the tensor core's working time. The higher the second utilization, the higher the computing resource utilization of the tensor cores in the computing unit when executing computing tasks. The computing resources of the tensor cores are fully utilized. The tensor cores spend most of their time performing effective calculations rather than being idle (for example, waiting for data transmission, synchronization, or waiting for other computing units to complete work). The computing tasks are executed faster, waiting time is minimized, and the overhead of data transmission and synchronization is reduced.

[0105] After obtaining the first utilization rate and the second utilization rate, as Figure 2 As shown, in step S30, the theoretical performance of the processor when executing the general matrix multiplication operator is determined based on the smaller one of the first utilization rate and the second utilization rate.

[0106] In response to the greater the first utilization rate or the second utilization rate, the processor has a higher computing resource utilization rate for the tensor cores when executing the general matrix multiplication operator. As previously described, a greater / higher first utilization rate or second utilization rate indicates a higher computing resource utilization rate for the tensor cores in the computing unit when executing computing tasks. The computing resources of the tensor cores are fully utilized, and the tensor cores spend most of their time performing active computations rather than being idle (e.g., waiting for data transmission, synchronization, or waiting for other computing units to complete work). This results in faster execution of computing tasks, minimized waiting time, and reduced data transmission and synchronization overhead.

[0107] For example, theoretical performance can be indicated by theoretical elapsed time.

[0108] For example, step S30 may include: determining the smaller one of the first utilization and the second utilization as the target utilization; determining the quotient of the first working time and the target utilization as the theoretical time consumption of the processor when executing a general matrix multiplication operator, wherein the theoretical time consumption is used to indicate theoretical performance.

[0109] For example, the theoretical time can be determined using the following formula:

[0110] T = T_Compute_chip / min(Gemm_util_chip, Gemm_util_SM) (Formula 8)

[0111] Wherein, T represents theoretical time consumption, T_Compute_chip represents the first working time, min() represents the minimum function, Gemm_util_chip represents the first utilization rate, and Gemm_util_SM represents the second utilization rate.

[0112] In the information processing method provided in at least one embodiment of the present disclosure, a two-level dual-layer theoretical performance evaluation model is provided, including chip-level evaluation and computing unit-level evaluation. Chip-level evaluation is from the perspective of the entire processor, where input and output data are stored in memory, and the workload refers to completing the matrix operations of the first tensor and the second tensor. The processor includes multiple computing units, and computing unit-level evaluation is from the perspective of a single computing unit, where input and output data are stored in shared memory, and the workload refers to the calculation of a tile obtained by a single computing unit to complete a computing task.

[0113] Therefore, the two-layer theoretical performance evaluation model provided by at least one embodiment of the present disclosure comprehensively considers chip-level evaluation and computing unit-level evaluation, and the evaluation results are more accurate. It is applicable to programming models based on chip granularity, based on a multi-layer storage architecture, and focuses on the critical path analysis of two-layer data flows, considering not only the data flow from memory to shared memory, but also the data flow from shared memory to register stack. In addition, the influence of multiple factors of the bandwidth model is also comprehensively considered, including the influence of the secondary cache hit rate on the equivalent bandwidth, and the influence of the independent bandwidth characteristics of the two memory blocks of the shared memory on the interface time consumption. The performance evaluation obtained in this way is more accurate and closer to the actual situation.

[0114] After accurately determining the theoretical execution time of a GEMM operator on a processor, this accurate theoretical time can help rationally allocate computing resources. For example, based on the theoretical execution time of the GEMM operator on the GPU, different tasks can be rationally scheduled for execution on different GPUs according to task priority and resource availability. Alternatively, the estimated accurate theoretical time can be used to allocate computationally intensive and time-consuming GEMM operations to higher-performance GPUs, thereby improving resource utilization across the entire cluster. Furthermore, accurate theoretical time can help developers design more reasonable parallel strategies, guide algorithm optimization, assist in model design, and accurately predict task completion times. Accurate theoretical time can serve as an important basis for scheduling algorithms, helping to achieve load balancing and improve overall system throughput.

[0115] Figure 5 A schematic block diagram of an information processing device provided in at least one embodiment of the present disclosure.

[0116] The information processing device is used to evaluate the theoretical performance of a processor executing a general matrix multiplication operator.

[0117] The processor includes multiple compute units and memory. Multiple compute units share memory. Each compute unit includes shared memory, a register file, and a tensor core. The tensor core is used to perform tensor-related computations in general-purpose matrix multiplication operators. Data flows between memory and tensor cores sequentially through the shared memory and register file. The shared memory is closer to the memory than the register file. For more information about the processor, please refer to the aforementioned description of the information processing method and will not be repeated here.

[0118] like Figure 5 As shown, the information processing apparatus 100 includes a first determining module 101 , a second determining module 102 and a third determining module 103 .

[0119] The first determination module is configured to determine a first utilization rate in combination with a data flow between a memory and a shared memory when the processor executes a general matrix multiplication operator, wherein the first utilization rate is used to indicate a performance evaluation of the general matrix multiplication operator at a processor level.

[0120] The second determination module is configured to determine a second utilization rate in combination with the data flow between the shared memory and the register file within a single computing unit when the processor executes the general matrix multiplication operator, wherein the second utilization rate is used to indicate the performance evaluation of the general matrix multiplication operator at the computing unit level.

[0121] The third determination module is configured to determine the theoretical performance of the processor when executing the general matrix multiplication operator based on the smaller one of the first utilization ratio and the second utilization ratio.

[0122] In response to the first utilization rate or the second utilization rate being greater, the computing resource utilization rate of the tensor core is higher when the processor executes the general matrix multiplication operator.

[0123] For example, the first determination module 101, the second determination module 102, and the third determination module 103 include codes and programs stored in a memory. The first determination module 101, the second determination module 102, and the third determination module 103 are implemented as, for example, a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities. The processing unit can be a general-purpose processor, and can also be a single-chip microcomputer, a microprocessor, a digital signal processor, a dedicated image processing chip, or a field programmable logic array, etc. The first determination module 101, the second determination module 102, and the third determination module 103 execute the codes and programs to implement some or all of the functions of the first determination module 101, the second determination module 102, and the third determination module 103 as described above. For example, the first determination module 101, the second determination module 102, and the third determination module 103 can be a circuit board or a combination of multiple circuit boards for implementing the functions described above. In an embodiment of the present application, the circuit board or the combination of multiple circuit boards may include: (1) one or more processors; (2) one or more non-temporary memories connected to the processors; and (3) firmware stored in the memories that can be executed by the processors.

[0124] It should be noted that the first determining module 101 can be used to implement Figure 2 Step S10 shown; the second determination module 102 can be used to implement Figure 2 Step S20 shown; the third determination module 103 can be used to implement Figure 2Therefore, for a detailed description of the functions that the first determination module 101 can implement, reference can be made to the description of step S10 in the embodiment of the aforementioned information processing method. For a detailed description of the functions that the second determination module 102 can implement, reference can be made to the description of step S20 in the embodiment of the aforementioned information processing method. For a detailed description of the functions that the third determination module 103 can implement, reference can be made to the description of step S30 in the embodiment of the aforementioned information processing method. The repeated parts will not be repeated here. In addition, the information processing device 100 can achieve technical effects similar to those of the aforementioned information processing method, which will not be repeated here.

[0125] It should be noted that in at least one embodiment of the present disclosure, the information processing device 100 may include more or fewer circuits or units, and the connection relationship between the various circuits or units is not limited and can be determined according to actual needs. The specific configuration of each circuit or unit is not limited and can be composed of analog devices according to circuit principles, or can be composed of digital chips, or constructed in other applicable ways.

[0126] For example, the information processing device 100 may be implemented using hardware, software, or a combination of hardware and software, and this disclosure does not impose any specific limitations on this.

[0127] In some embodiments, the information processing device provided in at least one embodiment of the present disclosure may be a chip, for example, a system-on-a-chip (SoC). The SoC includes a processor, which may be a single-core processor or a multi-core processor, a memory, an I / O interface, and the like.

[0128] Figure 6 A schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. Figure 6 As shown, the storage medium 200 may be a non-transitory computer-readable storage medium, and one or more computer-readable instructions 201 may be non-transitory stored on the storage medium 200. For example, when the computer-readable instructions 201 are executed by a processor, one or more steps in the information processing method described above may be performed.

[0129] For example, the storage medium 200 may be applied to an electronic device 300 . For example, the storage medium 200 may include a storage device 308 in the electronic device 300 .

[0130] For example, the storage device may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, a flash memory, etc. One or more computer-readable instructions may be stored on the computer-readable storage medium, and the processor may execute the computer-readable instructions to implement various functions of the processor. The storage medium may also store various application programs and various data.

[0131] For example, the storage medium may include a memory card of a smart phone, a cache component of a tablet computer, a hard disk of a personal computer, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other applicable storage media.

[0132] Figure 7 This is a schematic block diagram of an electronic device provided by an embodiment of the present disclosure. Figure 7 As shown, the electronic device 300 is suitable for implementing the information processing method provided by the embodiment of the present disclosure. It should be noted that Figure 7 The components of the electronic device 300 shown are merely exemplary and non-limiting. The electronic device 300 may also have other components according to actual application requirements.

[0133] like Figure 7 As shown, the electronic device 300 may include a processing device 301 (eg, a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to non-transitory computer-readable instructions stored in a memory to implement various functions.

[0134] For example, when the computer readable instructions are executed by the processing device 301, one or more steps of the information processing method according to any of the above embodiments may be executed. It should be noted that for a detailed description of the processing process of the information processing method, reference may be made to the relevant descriptions in the above embodiments of the information processing method.

[0135] For example, the memory may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) 303 and / or cache memory. For example, computer-readable instructions may be loaded from storage device 308 into RAM 303 to execute the computer-readable instructions. Non-volatile memory may include, for example, read-only memory (ROM) 302, a hard disk, an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, a flash memory, and the like. Various applications and various data, such as style images and various data used and / or generated by the applications, may also be stored in the computer-readable storage media.

[0136] For example, the processing device 301, the read-only memory (ROM) 302, and the random access memory (RAM) 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0137] Typically, the following devices may be connected to the input / output (I / O) interface 305: an input device 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, a flash memory, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other electronic devices wirelessly or by wire to exchange data. Although Figure 7 While an electronic device 300 is shown with various devices, it should be understood that implementation or presence of all illustrated devices is not required, and the electronic device 300 may alternatively implement or possess more or fewer devices. For example, the processing device 301 may control other components in the electronic device 300 to perform desired functions. The processing device 301 may be a device with data processing and / or program execution capabilities, such as a central processing unit (CPU), a tensor processing unit (TPU), or a graphics processing unit (GPU). The central processing unit (CPU) may be of an X86, ARM, or RISC-V architecture. The GPU may be directly integrated into the SOC, directly integrated into the motherboard, or built into the motherboard's northbridge chip.

[0138] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0139] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.

[0140] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0141] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the present disclosure is not limited to technical solutions formed by specific combinations of the aforementioned technical features. It also encompasses other technical solutions formed by any combination of the aforementioned technical features or their equivalents, without departing from the scope of the above disclosure. For example, a technical solution formed by replacing the aforementioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0142] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0143] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

[0144] Regarding this disclosure, the following points need to be explained:

[0145] (1) The drawings of the embodiments of the present disclosure only relate to the structures related to the embodiments of the present disclosure. Other structures may refer to conventional designs.

[0146] (2) In the absence of conflict, the embodiments of the present disclosure and the features therein may be combined with each other to form new embodiments.

[0147] The above description is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. The protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. An information processing method, characterized in that: Used to evaluate the theoretical performance of a processor executing a general matrix multiplication operator. The processor includes multiple computing units and memory, the multiple computing units share the memory, each computing unit includes shared memory, a register file and a tensor core, the tensor core is used to perform tensor-related calculations in the general matrix multiplication operator, and data flows between the memory and the tensor core through the shared memory and the register file in sequence. The shared memory is closer to the memory than the register file. The information processing method includes: determining a first utilization rate based on a data flow between the memory and the shared memory when the processor executes the general matrix multiplication operator, wherein the first utilization rate is used to indicate a performance evaluation of the general matrix multiplication operator at a processor level, and the first utilization rate is reflected by a proportional relationship between an effective working time of the tensor core when executing the general matrix multiplication operator and a total execution time of the general matrix multiplication operator; determining a second utilization rate based on a data flow between a shared memory and a register file within a single computing unit when the processor executes the general matrix multiplication operator, wherein the second utilization rate is used to indicate a performance evaluation of the general matrix multiplication operator at the computing unit level, and the second utilization rate is embodied by a proportional relationship between an effective working time when the tensor core executes a computing task corresponding to the single computing unit and a total execution time of the computing task, with the single computing unit as the granularity; Based on the smaller of the first utilization and the second utilization, the theoretical performance of the processor when executing the general matrix multiplication operator is determined, wherein, in response to the larger the first utilization or the second utilization, the higher the computing resource utilization of the tensor core of the processor when executing the general matrix multiplication operator.

2. The information processing method according to claim 1, wherein: The general matrix multiplication operator is used to load a first tensor and a second tensor from the memory to tensor cores in the multiple computing units, and the tensor cores in the multiple computing units perform a tensor multiplication operation on the first tensor and the second tensor to obtain a third tensor and store it in the memory. The third tensor is divided into L slices of the same size, and the multiple computing units perform multiple computing tasks in parallel to obtain multiple slices and store them in the memory, where L is a positive integer. The corresponding computing task performed by each computing unit includes loading the corresponding tensor parts of the first tensor and the second tensor from the memory to the tensor core of the computing unit, and using the tensor core of the computing unit to perform tensor operations, obtaining slices as calculation results and storing them in the memory.

3. The information processing method according to claim 2, wherein: The first utilization rate is indicated by a proportion of a first working time in a first total time, the first working time being a working time when a tensor core in the plurality of computing units performs the tensor multiplication operation of the first tensor and the second tensor, and the first total time being a total time taken by the processor to completely execute the general matrix multiplication operator. The second utilization rate is indicated by the proportion of the second working time in the second total time, wherein the second working time is the working time of the tensor core in the single computing unit to perform the tensor operation in a computing task, and the second total time is the total time taken for the single computing unit to complete the computing task.

4. The information processing method according to claim 2, wherein: Determining a first utilization rate in conjunction with a data flow between the memory and the shared memory when the processor executes the general matrix multiplication operator includes: determining a first equivalent bandwidth in conjunction with a data flow between the memory and the shared memory when the processor executes the general matrix multiplication operator; Determining a first interface time consumption based on the first equivalent bandwidth and a total data volume of executing the general matrix multiplication operator, wherein the total data volume is the sum of data volumes of the first tensor, the second tensor, and the third tensor, and the first interface time consumption indicates a time consumption caused by data transmission between the memory and the shared memory when the processor executes the general matrix multiplication operator; Determining a first working duration based on a total computational effort of executing the general matrix multiplication operator and a chip computing power of the processor; The first utilization rate is determined based on the first working duration and the first interface time consumption.

5. The information processing method according to claim 4, characterized in that The processor further includes a secondary cache, and the data also passes through the secondary cache when flowing between the memory and the tensor core. The secondary cache is closer to the memory than the shared memory. Determining a first equivalent bandwidth in conjunction with a data flow between the memory and the shared memory when the processor executes the general matrix multiplication operator includes: Determining a bandwidth and a hit rate of the secondary cache; determining a bandwidth of the memory; The first equivalent bandwidth is determined based on the bandwidth and the hit rate of the secondary cache and the bandwidth of the memory.

6. The information processing method according to claim 2, wherein: Determining a second utilization rate based on data flow between a shared memory and a register file within a single computing unit when the processor executes the general matrix multiplication operator includes: Determine a second interface time consumption, wherein the second interface time consumption indicates a time consumption caused by data transmission between a shared memory within the single computing unit and a register file of the single computing unit when the single computing unit performs the computing task; Determining a second working duration based on a total computing amount of executing the computing task and a computing power of a tensor core in the single computing unit; The second utilization rate is determined based on the second working duration and the second interface time consumption.

7. The information processing method according to claim 6, characterized in that: The shared memory in each computing unit includes a first memory block and a second memory block, wherein the first memory block is used to cache the first tensor required to perform the computing task and the corresponding tensor part of the third tensor, and the second memory block is used to cache the corresponding tensor part of the second tensor required to perform the computing task, and the first memory block and the second memory block have independent data transmission paths. Determine the time consumption of the second interface, including: determining a first time consumption based on a total data volume of corresponding tensor parts in the first tensor and the third tensor and a bandwidth of the first memory block; Determining a second time consumption based on a total data amount of a corresponding tensor portion in the second tensor and a bandwidth of the second memory block; The larger one of the first time consumption and the second time consumption is determined as the second interface time consumption.

8. The information processing method according to claim 6, wherein: Determining a second working duration based on a total computing amount of executing the computing task and a computing power of a tensor core in the single computing unit includes: Determining a total computational effort for executing the computing task based on a shape and size of a corresponding tensor portion in the first tensor required for executing the computing task, and a shape and size of a corresponding tensor portion in the second tensor required for executing the computing task; Determining the computing power of the tensor core in the single computing unit according to the chip computing power of the processor and the total number of the plurality of computing units; The second working duration is determined according to the total computing amount of executing the computing task and the computing power of the tensor core in the single computing unit.

9. The information processing method according to claim 6, wherein: Determining the second utilization rate based on the second working duration and the second interface time consumption includes: Determine the larger of the second working duration and the second interface time as the target duration; The quotient of the second working duration and the target duration is determined as the second utilization rate.

10. The information processing method according to claim 1, wherein: Determining, based on the smaller of the first utilization ratio and the second utilization ratio, a theoretical performance of the processor when executing the general matrix multiplication operator, comprising: determining the smaller one of the first utilization rate and the second utilization rate as a target utilization rate; A quotient of the first working duration and the target utilization is determined as a theoretical time consumption of the processor when executing the general matrix multiplication operator, wherein the theoretical time consumption is used to indicate the theoretical performance.

11. An information processing device, characterized in that: Used to evaluate the theoretical performance of a processor executing a general matrix multiplication operator. The processor includes multiple computing units and memory, the multiple computing units share the memory, each computing unit includes shared memory, a register file and a tensor core, the tensor core is used to perform tensor-related calculations in the general matrix multiplication operator, and data flows between the memory and the tensor core through the shared memory and the register file in sequence. The shared memory is closer to the memory than the register file. The information processing device includes: a first determining module configured to determine a first utilization rate based on a data flow between the memory and the shared memory when the processor executes the general matrix multiplication operator, wherein the first utilization rate is used to indicate a performance evaluation of the general matrix multiplication operator at a processor level, and the first utilization rate is reflected by a proportional relationship between an effective working time of the tensor core when executing the general matrix multiplication operator and a total execution time of the general matrix multiplication operator; a second determination module configured to determine a second utilization rate based on a data flow between a shared memory and a register file within a single computing unit when the processor executes the general matrix multiplication operator, wherein the second utilization rate is used to indicate a performance evaluation of the general matrix multiplication operator at a computing unit level, and the second utilization rate is embodied by a proportional relationship between an effective working time when the tensor core executes a computing task corresponding to the single computing unit and a total execution time of the computing task, with the single computing unit as the granularity; A third determination module is configured to determine the theoretical performance of the processor when executing the general matrix multiplication operator based on the smaller of the first utilization and the second utilization, wherein, in response to the larger the first utilization or the second utilization, the higher the computing resource utilization of the tensor core of the processor when executing the general matrix multiplication operator.

12. An electronic device, characterized in that: include: a memory that non-transitorily stores computer-executable instructions; a processor configured to execute the computer-executable instructions, The computer executable instructions, when executed by the processor, implement the information processing method according to any one of claims 1 to 10.

13. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium stores computer-executable instructions, When the computer-executable instructions are executed by a processor, the information processing method according to any one of claims 1 to 10 is implemented.

Images