A register resource allocation method, device, storage medium and program product

Abstract

The embodiments of the present application provide a register resource allocation method, device, storage medium and program product, which relate to the field of artificial intelligence technology. The method includes: dividing the total register resources into multiple types of register buffers. Based on the operator combination corresponding to the fusion operator in each computing scenario, the resource amounts of the multiple types of register buffers allocated to the fusion operator are determined, so as to realize targeted register resource allocation for different computing scenarios of the fusion operator, thereby making full use of the high-speed on-chip storage resource of the register and greatly improving the performance of the fusion operator. Secondly, since the present application flexibly adjusts the resource ratios of the multiple types of register buffers for different computing scenarios of the fusion operator, the overall computing efficiency of the fusion operator in the generalized scenario is improved, and it has high performance and generalization versatility.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular to a register resource allocation method, device, storage medium, and program product. Background Art

[0002] Artificial intelligence models usually refer to neural network models that are trained to perform inference and prediction, such as image inference models, speech inference models, etc. The operations of artificial intelligence models can be implemented by operators in the computation graph. Operators refer to various operations performed on tensors at each layer in the artificial intelligence model. For example, the convolution operation performed by the convolution layer of the artificial intelligence model on the input data of the artificial intelligence model is a convolution operator. In order to improve the computing efficiency of artificial intelligence chips, multiple operators are usually fused to obtain a fused operator.

[0003] When allocating register resources to fusion operators, related technologies use a fixed register allocation method to statically allocate register resources to fusion operators, that is, the register resource allocation scheme of the fusion operator is the same in multiple computing scenarios.

[0004] However, the above register resource allocation scheme often needs to sacrifice performance to meet the universal requirements of fusion operators in various computing scenarios, which results in insufficient utilization of register resources and affects the computing efficiency of the operator. Summary of the invention

[0005] The embodiments of the present application provide a register resource allocation method, device, storage medium, and program product for fully utilizing and improving register resources, thereby improving the computing efficiency of operators.

[0006] On the one hand, an embodiment of the present application provides a register resource allocation method, including:

[0007] Divide the total register resources into multiple types of register buffers;

[0008] For multiple computing scenarios of the fusion operator, perform operations separately:

[0009] Based on an operator combination corresponding to the fused operator in a computing scenario, the resource amounts of the multiple types of register buffers allocated to the fused operator are determined.

[0010] On the one hand, an embodiment of the present application provides a register resource allocation device, including:

[0011] A classification module for dividing the total register resources into multiple types of register buffers;

[0012] The allocation module performs operations for multiple computing scenarios of the fusion operator:

[0013] Based on an operator combination corresponding to the fused operator in a computing scenario, the resource amounts of the multiple types of register buffers allocated to the fused operator are determined.

[0014] Optionally, the allocation module is specifically used to:

[0015] Based on the input shape of the fusion operator and the operator combination corresponding to the fusion operator in a computing scenario, the resource amounts of the multiple types of register buffers allocated to the fusion operator are determined.

[0016] Optionally, the operator combination includes: a tensor operator executed by a tensor calculation unit and at least one vector calculation operator executed by a vector calculation unit; the multiple types of register buffers include: an interactive buffer allocated for exchanging calculation results of the tensor operator between the tensor calculation unit and the vector calculation unit, and a temporary buffer allocated for the at least one vector calculation operator;

[0017] The allocation module is specifically used for:

[0018] Allocating a first resource amount corresponding to the interactive buffer from a total resource amount of the total register resources based on an input shape of the fusion operator;

[0019] The second resource amount corresponding to the temporary buffer area is allocated from the remaining resource amount of the total resource amount.

[0020] Optionally, the operator combination includes: a tensor operator executed by a tensor calculation unit and a block accumulation operator executed by a vector calculation unit; the multiple types of register buffers include: an interactive buffer allocated for exchanging block calculation results of the tensor operator between the tensor calculation unit and the vector calculation unit, and an accumulation buffer allocated for the block accumulation operator;

[0021] The allocation module is specifically used for:

[0022] Based on the input shape of the fusion operator and the blocking method of the tensor operator, a third resource amount corresponding to the interaction buffer is allocated from the total resource amount of the total register resources, and a fourth resource amount corresponding to the accumulation buffer is allocated.

[0023] Optionally, the operator combination further includes: at least one vector calculation operator executed by a vector calculation unit; the multiple types of register buffers further include: a temporary buffer allocated for the at least one vector calculation operator;

[0024] The fifth resource amount of the temporary buffer is allocated from the remaining resource amount of the total resource amount.

[0025] Optionally, the allocation module is further used for:

[0026] If, among the at least one vector computing operator, there is a target computing operator with dual inputs and implicit broadcasting characteristics, the multiple types of register buffers further include: a fixed dedicated buffer allocated for the target computing operator;

[0027] A sixth resource amount corresponding to the fixed dedicated buffer area is allocated from the remaining resource amount of the total resource amount.

[0028] Optionally, the classification module is specifically used for:

[0029] For multiple computing scenarios of the fusion operator, perform operations separately:

[0030] Based on the operator combination corresponding to the fused operator in a computing scenario, the total register resources are divided into multiple types of register buffers.

[0031] Optionally, the classification module is specifically used for:

[0032] Based on the input shape of the fused operator and the operator combination corresponding to the fused operator in a computing scenario, the total register resources are divided into multiple types of register buffers.

[0033] On the one hand, an embodiment of the present application provides a computer device, including a memory, an artificial intelligence chip, and a computer program stored in the memory and executable on the artificial intelligence chip, wherein the artificial intelligence chip implements the steps of the above-mentioned register resource allocation method when executing the computer program.

[0034] On the one hand, an embodiment of the present application provides a computer-readable storage medium storing a computer program executable by a computer device. When the computer program runs on the computer device, the computer device executes the steps of the above-mentioned register resource allocation method.

[0035] On the one hand, an embodiment of the present application provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer device, the computer device executes the steps of the above-mentioned register resource allocation method.

[0036] In the embodiment of the present application, the total register resources are divided into multiple types of register buffers. Based on the operator combination corresponding to the fusion operator in each computing scenario, the resource amounts of the multiple types of register buffers allocated to the fusion operator are determined, so that register resources are allocated in a targeted manner for different computing scenarios of the fusion operator, thereby making full use of the high-speed on-chip storage resource of the register and greatly improving the performance of the fusion operator.

[0037] Secondly, since the present application flexibly adjusts the resource ratios of various types of register buffers for different computing scenarios of fusion operators, it achieves an overall improvement in computing efficiency in generalized scenarios of fusion operators, and has high performance and generalization. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0039] Figure 1 A schematic diagram of the structure of an artificial intelligence chip provided in an embodiment of the present application;

[0040] Figure 2 A flowchart of a register resource allocation method provided in an embodiment of the present application;

[0041] Figure 3A A schematic flow chart of a method for obtaining a tensor shape provided in an embodiment of the present application;

[0042] Figure 3B A flowchart of a register resource allocation method provided in an embodiment of the present application;

[0043] Figure 4 A flowchart of a register resource allocation method provided in an embodiment of the present application;

[0044] Figure 5 A flowchart of a register resource allocation method provided in an embodiment of the present application;

[0045] Figure 6 A schematic diagram of the structure of a register resource allocation device provided in an embodiment of the present application;

[0046] Figure 7 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and beneficial effects of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0048] refer to Figure 1 , which is a structural diagram of an artificial intelligence chip applicable to an embodiment of the present application. The artificial intelligence chip 100 at least includes: a video memory 101, an on-chip cache 102, and a processing unit 103. The processing unit 103 includes: a register resource 104, a tensor computing unit 105, and a vector computing unit 106.

[0049] The video memory 101 may be a high bandwidth memory (HBM) or other types of memory. The on-chip cache 102 is a temporary memory, which has a smaller capacity than the video memory 101 but has a faster data exchange speed.

[0050] The processing unit 103 can read data from the video memory 101 or write data to the video memory 101. Similarly, the processing unit 103 can read data from the on-chip cache 102 or write data to the on-chip cache 102.

[0051] The register resource 104 is a register shared by the tensor computing unit 105 and the vector computing unit 106. Compared with the on-chip cache 102, the capacity of the register resource 104 is smaller than that of the on-chip cache 102, but the data exchange speed is faster than that of the on-chip cache 102. The register resource 104 can be used for data exchange between the tensor computing unit 105 and the vector computing unit 106, and can also be used to store relevant data for internal calculations of the tensor computing unit 105 and the vector computing unit 106.

[0052] In addition to the above-mentioned structure, the artificial intelligence chip 100 in the present application may also include other structures, which are not specifically limited in the present application.

[0053] The artificial intelligence chip 100 can be: a graphics processing unit (GPU), a general-purpose graphics processing unit (GPGPU), a domain-specific architecture (DSA), etc.

[0054] In practical applications, in order to improve the computing efficiency of artificial intelligence chips, multiple operators are usually fused to obtain a fused operator. When allocating register resources to the fused operator, the relevant technology uses a fixed register allocation method to statically allocate register resources to the fused operator, that is, the register resource allocation scheme of the fused operator in multiple computing scenarios is the same.

[0055] However, the above register resource allocation scheme often needs to sacrifice performance to meet the universal requirements of fusion operators in various computing scenarios, which results in insufficient utilization of register resources and affects the computing efficiency of the operator.

[0056] In view of this, this application is based on Figure 1 The architecture diagram of the artificial intelligence chip shown in FIG. 1 provides a flow chart of a register resource allocation method, such as Figure 2 As shown, the process of the method is executed by an artificial intelligence chip and includes the following steps:

[0057] Step 201 : Divide the total register resources into multiple types of register buffers.

[0058] Specifically, the register resource types required when the fusion operator is executed in multiple computing scenarios can be counted, and multiple register resource types can be set based on the statistical results. When the fusion operator is compiled, the register resources are directly divided into multiple types of register buffers.

[0059] In some embodiments, the register resource types usually required for the execution process of the fusion operator include: communication buffer, accumulated buffer, tempbuffer, and dedicated buffer. Therefore, the compiler can directly divide the total register resources into the above four types of register buffers.

[0060] In some embodiments, since the fusion operator corresponds to different operator combinations (patterns) in different computing scenarios, the types of register resources required in different computing scenarios may also be different.

[0061] In order to improve the accuracy of register resource allocation, the present application performs operations separately for multiple computing scenarios of fusion operators: based on the operator combination corresponding to the fusion operator in a computing scenario, the total register resources are divided into multiple types of register buffers.

[0062] Specifically, the operator combination includes multiple operators in the fused operator, and the total register resources can be divided into multiple types of register buffers according to the requirements of the multiple operators for the register resource types.

[0063] For example, the fusion operator includes operator 1 and operator 2 in an operator combination in a computing scenario. The execution of operator 1 requires an interactive buffer, and the execution of operator 2 requires a temporary buffer. Therefore, the total register resources are divided into two types of register buffers: an interactive buffer and a temporary buffer.

[0064] In some embodiments, the execution process of the fusion operator is not only associated with the operator combination of the fusion operator, but also associated with the input shape of the fusion operator. In other words, in the same computing scenario, when the input shape is different, the register resource type required by the fusion operator may also be different.

[0065] In order to meet the register resource requirements for achieving the ultimate performance of operators, this application divides the total register resources into multiple types of register buffers based on the input shape of the fused operator and the operator combination corresponding to the fused operator in a computing scenario.

[0066] Specifically, the input shape refers to the length of each order of the input data. When the input data is a tensor, the input shape is the shape of the tensor, that is, the length of each order of the tensor. A tensor is an N-dimensional array. The order of a tensor is the dimension of the array. The length of each order in the tensor refers to the length of the array of the corresponding dimension, where N is greater than or equal to 0.

[0067] For example, Figure 3A The following examples show a 1-dimensional tensor and a 2-dimensional tensor. The shapes of the above two tensors are obtained in the following ways:

[0068] A 1-dimensional tensor corresponds to a 1-dimensional vector, and the length of the 1-dimensional vector is 3, so the shape of the 1-dimensional tensor is (3).

[0069] A 2D tensor corresponds to a 3×3 2D matrix. The length of the 1D and 2D arrays of the 2D matrix is ​​3. Then the shape of the 2D tensor is (3, 3).

[0070] It should be noted that the input data in the present application is not limited to the form of tensors, but can also be other forms of data; accordingly, the input shape is not limited to the shape of a tensor, but can also be other forms, and the present application does not make any specific limitations on this.

[0071] Step 202 , for multiple computing scenarios of the fusion operator, respectively perform operations: based on the operator combination corresponding to the fusion operator in one computing scenario, determine the resource amounts of each of the multiple types of register buffers allocated to the fusion operator.

[0072] Specifically, according to resource requirements of multiple operators in the operator combination for multiple types of register buffers, respective resource amounts of the multiple types of register buffers are allocated from the total resource amount of the total register resources.

[0073] Since the total amount of register resources is limited, and the impact of each operator in the operator combination on the performance of the fusion operator is also different. Therefore, for the operator combination corresponding to each computing scenario, the allocation order and the amount of resources required for the register buffer of each operator can be determined according to the degree of influence of each operator in the operator combination on the performance of the fusion operator. Among them, the greater the influence of the operator on the performance of the fusion operator, the higher the allocation order of the required register buffer, and the amount of resources allocated will also meet the operator requirements as much as possible.

[0074] In some embodiments, the execution process of the fusion operator is not only associated with the operator combination of the fusion operator, but also associated with the input shape of the fusion operator. In other words, in the same computing scenario, when the input shape is different, the amount of resources of various register buffers required by the fusion operator may also be different.

[0075] In order to make the ultimate allocation of register resources and flexibly adjust the resource ratio of various types of register buffers, this application determines the resource amount of each type of register buffer allocated to the fusion operator based on the input shape of the fusion operator and the operator combination corresponding to the fusion operator in a computing scenario.

[0076] In the embodiment of the present application, based on the operator combination corresponding to the fusion operator in each computing scenario, the resource amount of each type of register buffer allocated to the fusion operator is determined, so as to realize targeted allocation of register resources for different computing scenarios of the fusion operator, thereby making full use of the high-speed on-chip storage resource of registers and greatly improving the performance of the fusion operator. Secondly, since the present application flexibly adjusts the resource ratios of each type of register buffer for different computing scenarios of the fusion operator, it realizes the overall improvement of computing efficiency in the generalized scenario of the fusion operator, and has high performance and generalization versatility.

[0077] In some embodiments, the operator combination includes: a tensor operator performed by a tensor computation unit and at least one vector computation operator performed by a vector computation unit.

[0078] Specifically, tensor operators can be matrix operators, convolution operators, etc., where matrix operators are usually represented by MMA (Matrix MultiplyAccumulate), convolution operators are usually represented by CONV (Convolution), and vector calculation operators are usually represented by Elementwise (bit-by-bit calculation).

[0079] As an accelerator in artificial intelligence chips, the tensor computing unit usually performs convolution operations or matrix operations, that is, the tensor computing unit is usually used to execute tensor operators such as matrix operators and convolution operators.

[0080] A vector computing unit usually has multiple single instruction multiple data (SIMD) arithmetic logic units, or multiple single instruction multiple threads (SIMT) arithmetic logic units. These arithmetic logic units are used to perform scalar and vector operations. Therefore, the vector computing unit can be used to execute vector computing operators.

[0081] The multiple types of register buffers include: an interactive buffer allocated for calculation results of interactive tensor operators between a tensor calculation unit and a vector calculation unit, and a temporary buffer allocated for at least one vector calculation operator.

[0082] In the specific implementation, the tensor computing unit and the vector computing unit exchange data through a register file with fast access speed but relatively small capacity. This register resource used for interaction is called an interaction buffer. The tensor computing unit executes tensor operators and writes the calculation results to the interaction buffer.

[0083] The vector computing unit reads the calculation results of the tensor operator from the interactive buffer and executes the vector computing operator based on the calculation results of the tensor operator. For all vector computing operators in the fusion operator, the same temporary buffer is allocated, and the temporary buffer is released immediately after each vector computing operator is completed to make full use of register resources.

[0084] In the computing scenario where the vector computing unit does not participate in the accumulation of the block computing results of the tensor computing unit, based on the input shape of the fusion operator, the first resource amount corresponding to the interactive buffer is allocated from the total resource amount of the total register resources. Then, the second resource amount corresponding to the temporary buffer is allocated from the remaining resource amount of the total resource amount.

[0085] Specifically, since the process of executing tensor operators by the tensor computing unit has a greater impact on the performance of the fusion operator, the first resource amount corresponding to the interaction buffer is preferentially allocated from the total resource amount of the total register resources. Specifically, the interaction buffer can be increased to four ping-pong-ping-pong buffers.

[0086] After the first resource amount corresponding to the interactive buffer area is allocated, the second resource amount corresponding to the temporary buffer area is allocated from the remaining resource amount of the total resource amount.

[0087] For example, see Figure 3B The operator combination of the fusion operator includes: a matrix operator, a vector calculation operator 1 and a vector calculation operator 2, wherein the matrix operator is executed by a tensor calculation unit, and the vector calculation operator 1 and the vector calculation operator 2 are executed by a vector calculation unit.

[0088] From the total register resources, an interaction buffer (corresponding to the first resource amount) is allocated for data interaction between the tensor calculation unit and the vector calculation unit, and a temporary buffer (corresponding to the second resource amount) is allocated for vector calculation operator 1 and vector calculation operator 2.

[0089] The tensor calculation unit executes the matrix operator and writes the matrix calculation results to the interactive buffer. The vector calculation unit reads the matrix calculation results from the interactive buffer and executes vector calculation operator 1 and vector calculation operator 2 based on the matrix calculation results, where the temporary buffer is used to save the intermediate calculation results of vector calculation operator 1 or vector calculation operator 2.

[0090] In an embodiment of the present application, when the operator combination of the fusion operator includes: a tensor operator executed by a tensor calculation unit and at least one vector calculation operator executed by a vector calculation unit, an interactive buffer is allocated for the calculation results of the interactive tensor operators between the tensor calculation unit and the vector calculation unit, and a temporary buffer is allocated for at least one vector calculation operator. Then, the first resource amount corresponding to the interactive buffer is preferentially allocated from the total resource amount of the total register resources, and then the second resource amount corresponding to the temporary buffer is allocated from the remaining resource amount, so that the high-speed on-chip storage resource such as the register is fully utilized, the performance of the fusion operator is greatly improved, and the parallelism between the tensor calculation unit and the vector calculation unit is fully improved.

[0091] In some embodiments, in the tensor operator, the cumulative dimension of the input tensor shape may be large, resulting in excessive repeated loading of the tensor computing unit. In this case, the tensor computing unit can be used to perform block calculations on the input tensor, and then the vector computing unit performs the block accumulation operation to reduce the problem of repeated loading of the tensor computing unit.

[0092] In this computing scenario, the operator combination includes: a tensor operator executed by a tensor computing unit, and a block accumulation operator executed by a vector computing unit. When the tensor operator is a matrix operator, the block accumulation operator is a matrix multiplication block accumulation (Splitk) operator; when the tensor operator is a convolution operator, the block accumulation operator is a convolution block accumulation (ICP) operator.

[0093] In this computing scenario, multiple types of register buffers include: an interactive buffer allocated for the block calculation results of the interactive tensor operators between the tensor computing unit and the vector computing unit, and an accumulation buffer allocated for the block accumulation operator.

[0094] Based on the input shape of the fusion operator and the blocking method of the tensor operator, a third resource amount corresponding to the interaction buffer is allocated from the total resource amount of the total register resources, and a fourth resource amount corresponding to the accumulation buffer is allocated.

[0095] In practical applications, since the process of the tensor computing unit performing block calculation on the input tensor is interrelated with the process of the vector computing unit performing block accumulation and has a greater impact on the performance of the fusion operator, the third resource amount corresponding to the interaction buffer and the fourth resource amount corresponding to the accumulation buffer are preferentially allocated from the total resource amount of the total register resources.

[0096] Compared with the calculation scenario where the vector calculation unit does not execute the block accumulation operator, in the calculation scenario where the vector calculation unit executes the block accumulation operator, the tensor calculation unit performs block calculation on the input tensor. Therefore, the amount of register resources required for data interaction between the tensor calculation unit and the vector calculation unit is reduced. In other words, the third amount of resources corresponding to the interaction buffer in the calculation scenario where the vector calculation unit executes the block accumulation operator is less than the first amount of resources corresponding to the interaction buffer in the calculation scenario where the vector calculation unit does not execute the block accumulation operator. In a specific implementation, for the calculation scenario where the vector calculation unit executes the block accumulation operator, the interaction buffer can be compressed into two ping-pong buffers, and at the same time, as many register resources as possible are used as the accumulation buffer.

[0097] In some embodiments, the operator combination further includes: at least one vector calculation operator executed by a vector calculation unit. Accordingly, the multiple types of register buffers further include: a temporary buffer allocated for at least one vector calculation operator.

[0098] Specifically, after allocating the third resource amount corresponding to the interactive buffer and the fourth resource amount corresponding to the cumulative buffer from the total resource amount, the fifth resource amount of the temporary buffer is allocated from the remaining resource amount of the total resource amount. For all vector calculation operators in the fusion operator, the same temporary buffer is allocated, and the temporary buffer is released immediately after each vector calculation operator completes the calculation.

[0099] For example, see Figure 4 The operator combination of the fusion operator includes: a matrix operator, a matrix multiplication block accumulation operator, a vector calculation operator 1 and a vector calculation operator 2, wherein the matrix operator is executed by a tensor calculation unit, and the matrix multiplication block accumulation operator, the vector calculation operator 1 and the vector calculation operator 2 are executed by a vector calculation unit.

[0100] From the total register resources, an interaction buffer is allocated for data interaction between the tensor calculation unit and the vector calculation unit (corresponding to the third resource amount), an accumulation buffer is allocated for the block accumulation operator (corresponding to the fourth resource amount), and a temporary buffer is allocated for vector calculation operator 1 and vector calculation operator 2 (corresponding to the fifth resource amount).

[0101] The tensor calculation unit executes matrix operators, performs matrix block calculations on the input tensor, and writes the obtained matrix block calculation results to the interaction buffer. The vector calculation unit reads the matrix block calculation results from the interaction buffer, and executes the matrix multiplication block accumulation operator based on the matrix block calculation results, and then writes the obtained block accumulation results to the accumulation buffer.

[0102] The vector calculation unit reads the block accumulation result from the accumulation buffer, and executes vector calculation operator 1 and vector calculation operator 2 based on the block accumulation result, wherein the temporary buffer is used to store the intermediate calculation result of vector calculation operator 1 or vector calculation operator 2.

[0103] In an embodiment of the present application, when the operator combination of the fusion operator includes: a tensor operator executed by a tensor calculation unit, and a block accumulation operator executed by a vector calculation unit, an interactive buffer is allocated for the block calculation results of the tensor operators interacting between the tensor calculation unit and the vector calculation unit, and an accumulation buffer is allocated for the block accumulation operator. Then, the third resource amount corresponding to the interactive buffer and the fourth resource amount corresponding to the accumulation buffer are preferentially allocated from the total resource amount, which not only reduces the load pressure caused by repeated loading of data by the tensor calculation unit, but also improves the utilization rate of the tensor calculation unit and register resources. Secondly, a temporary buffer is allocated for at least one vector calculation operator, and the temporary buffer is released after the operator calculation is completed, making full use of the register resources.

[0104] In some embodiments, if there is a target computing operator with dual inputs and implicit broadcasting features in at least one vector computing operator, the multiple types of register buffers further include: a fixed dedicated buffer allocated for the target computing operator.

[0105] Specifically, the implicit broadcast feature means that when executing a vector calculation operator, one of the dual inputs needs to be repeatedly loaded.

[0106] For example, assume that a vector calculation operator includes a first input and a second input, and the vector calculation operator is used to accumulate the first input and the second input. Set the shape of the first input to [1,10] and the shape of the second input to [1]. It can be seen that the shape of the first input is different from the shape of the second input. Therefore, in the process of executing the vector calculation operator, the second input needs to be loaded repeatedly 10 times, and the data loaded each time is the same. Specifically, each time the second input is loaded from the video memory to the temporary buffer for calculation, and after completing one calculation, the temporary buffer is released. Then the second input is loaded from the video memory to the temporary buffer for calculation again, until 10 calculations are completed.

[0107] That is to say, the related technology allocates a temporary buffer for a target computing operator with dual inputs and implicit broadcast characteristics, which results in the need to repeatedly load data during the execution of the target computing operator, thereby affecting the operator performance.

[0108] In view of this, in order to cooperate with the implicit broadcast within the target computing operator to reduce repeated data loading and optimize memory throughput, the present application allocates a fixed dedicated buffer for a vector computing operator with dual inputs and implicit broadcast features, and then allocates the sixth resource amount corresponding to the fixed dedicated buffer from the remaining resources of the total resource amount.

[0109] Specifically, the fixed dedicated buffer is used to retain data that needs to be reused or used for a long time for a long time, thereby avoiding repeated loading of data that needs to be reused or used for a long time.

[0110] In a calculation scenario where the vector calculation unit does not execute the block accumulation operator, the sixth resource amount corresponding to the fixed dedicated buffer area can be allocated from the remaining resources after the first resource amount corresponding to the interactive buffer area is allocated from the total resource amount; or the sixth resource amount corresponding to the fixed dedicated buffer area can be allocated from the remaining resources after the first resource amount corresponding to the interactive buffer area and the second resource amount corresponding to the temporary buffer area are allocated from the total resource amount; of course, the temporary buffer area and the fixed dedicated buffer area can also be assigned the same allocation priority, that is, after the first resource amount corresponding to the interactive buffer area is allocated from the total resource amount, the second resource amount corresponding to the temporary buffer area and the sixth resource amount corresponding to the fixed dedicated buffer area can be allocated from the remaining resources.

[0111] In the calculation scenario where the vector computing unit executes the block accumulation operator, after the third resource amount corresponding to the interactive buffer and the fourth resource amount corresponding to the accumulation buffer are allocated from the total resource amount, the sixth resource amount corresponding to the fixed dedicated buffer can be allocated from the remaining resources.

[0112] Alternatively, after allocating the third resource amount corresponding to the interactive buffer, the fourth resource amount corresponding to the cumulative buffer, and the fifth resource amount corresponding to the temporary buffer from the total resource amount, the sixth resource amount corresponding to the fixed dedicated buffer area may be allocated from the remaining resource amount.

[0113] Of course, the temporary buffer and the fixed dedicated buffer can also be assigned the same allocation priority, that is, after the third resource amount corresponding to the interactive buffer and the fourth resource amount corresponding to the cumulative buffer are allocated from the total resources, the fifth resource amount of the temporary buffer and the sixth resource amount corresponding to the fixed dedicated buffer are allocated from the remaining resources.

[0114] For example, see Figure 5 The operator combination of the fusion operator includes: a matrix operator, a matrix multiplication block accumulation operator, a vector calculation operator 1 and a vector calculation operator 2, wherein the matrix operator is executed by a tensor calculation unit, and the matrix multiplication block accumulation operator, the vector calculation operator 1 and the vector calculation operator 2 are executed by a vector calculation unit.

[0115] From the total register resources, an interactive buffer is allocated for data interaction between the tensor computing unit and the vector computing unit (corresponding to the third resource amount), an accumulation buffer is allocated for the block accumulation operator (corresponding to the fourth resource amount), and a temporary buffer is allocated for vector computing operator 1 and vector computing operator 2 (corresponding to the fifth resource amount). Since vector computing operator 2 is an operator with dual inputs and implicit broadcast features, a fixed dedicated buffer (corresponding to the sixth resource amount) is allocated for vector computing operator 2.

[0116] The tensor calculation unit executes matrix operators, performs matrix block calculations on the input tensor, and writes the obtained matrix block calculation results to the interaction buffer. The vector calculation unit reads the matrix block calculation results from the interaction buffer, and executes the matrix multiplication block accumulation operator based on the matrix block calculation results, and then writes the obtained block accumulation results to the accumulation buffer.

[0117] The vector calculation unit reads the block accumulation result from the accumulation buffer, and executes the vector calculation operator 1 based on the block accumulation result, wherein the temporary buffer is used to store the intermediate calculation result of the vector calculation operator 1.

[0118] The first input of the vector calculation operator 2 can be a block accumulation result read from the accumulation buffer, and the vector calculation unit loads the second input of the vector calculation operator 2 from the video memory to the fixed dedicated buffer. The vector calculation unit executes the vector calculation operator 2 based on the first input and the second input.

[0119] In an embodiment of the present application, for a vector calculation operator with dual inputs and implicit broadcast features, a fixed dedicated buffer is allocated for long-term retention of data that needs to be reused or used for a long time. This reduces the repeated loading of data during the implicit broadcast process, and at the same time reduces the load pressure of the fusion operator, thereby improving the performance of the fusion operator.

[0120] Based on the same technical concept, the embodiment of the present application provides a structural diagram of a register resource allocation device, such as Figure 6 As shown, the register resource allocation device 600 includes:

[0121] A classification module 601, used to divide the total register resources into multiple types of register buffers;

[0122] The allocation module 602 performs operations respectively for multiple computing scenarios of the fusion operator:

[0123] Based on an operator combination corresponding to the fused operator in a computing scenario, the resource amounts of the multiple types of register buffers allocated to the fused operator are determined.

[0124] Optionally, the allocation module 602 is specifically configured to:

[0125] Based on the input shape of the fusion operator and the operator combination corresponding to the fusion operator in a computing scenario, the resource amounts of the multiple types of register buffers allocated to the fusion operator are determined.

[0126] Optionally, the operator combination includes: a tensor operator executed by a tensor calculation unit and at least one vector calculation operator executed by a vector calculation unit; the multiple types of register buffers include: an interactive buffer allocated for exchanging calculation results of the tensor operator between the tensor calculation unit and the vector calculation unit, and a temporary buffer allocated for the at least one vector calculation operator;

[0127] The allocation module 602 is specifically used for:

[0128] Allocating a first resource amount corresponding to the interactive buffer from a total resource amount of the total register resources based on an input shape of the fusion operator;

[0129] The second resource amount corresponding to the temporary buffer area is allocated from the remaining resource amount of the total resource amount.

[0130] Optionally, the operator combination includes: a tensor operator executed by a tensor calculation unit and a block accumulation operator executed by a vector calculation unit; the multiple types of register buffers include: an interactive buffer allocated for exchanging block calculation results of the tensor operator between the tensor calculation unit and the vector calculation unit, and an accumulation buffer allocated for the block accumulation operator;

[0131] The allocation module 602 is specifically used for:

[0132] Based on the input shape of the fusion operator and the blocking method of the tensor operator, a third resource amount corresponding to the interaction buffer is allocated from the total resource amount of the total register resources, and a fourth resource amount corresponding to the accumulation buffer is allocated.

[0133] Optionally, the operator combination further includes: at least one vector calculation operator executed by a vector calculation unit; the multiple types of register buffers further include: a temporary buffer allocated for the at least one vector calculation operator;

[0134] The fifth resource amount of the temporary buffer is allocated from the remaining resource amount of the total resource amount.

[0135] Optionally, the allocation module 602 is further configured to:

[0136] If, among the at least one vector computing operator, there is a target computing operator with dual inputs and implicit broadcasting characteristics, the multiple types of register buffers further include: a fixed dedicated buffer allocated for the target computing operator;

[0137] A sixth resource amount corresponding to the fixed dedicated buffer area is allocated from the remaining resource amount of the total resource amount.

[0138] Optionally, the classification module 601 is specifically used for:

[0139] For multiple computing scenarios of the fusion operator, perform operations separately:

[0140] Based on the operator combination corresponding to the fused operator in a computing scenario, the total register resources are divided into multiple types of register buffers.

[0141] Optionally, the classification module 601 is specifically used for:

[0142] Based on the input shape of the fused operator and the operator combination corresponding to the fused operator in a computing scenario, the total register resources are divided into multiple types of register buffers.

[0143] In the embodiment of the present application, the total register resources are divided into multiple types of register buffers. Based on the operator combination corresponding to the fusion operator in each computing scenario, the resource amounts of the multiple types of register buffers allocated to the fusion operator are determined, so that register resources are allocated in a targeted manner for different computing scenarios of the fusion operator, thereby making full use of the high-speed on-chip storage resource of the register and greatly improving the performance of the fusion operator.

[0144] Secondly, since the present application flexibly adjusts the resource ratios of various types of register buffers for different computing scenarios of fusion operators, it achieves an overall improvement in computing efficiency in generalized scenarios of fusion operators, and has high performance and generalization.

[0145] Based on the same technical concept, the embodiment of the present application provides a computer device, such as Figure 7 As shown, it includes at least one artificial intelligence chip 100 and a memory 701 connected to the at least one artificial intelligence chip 100. The specific connection medium between the artificial intelligence chip 100 and the memory 701 is not limited in the embodiment of the present application. Figure 7 In the example, the artificial intelligence chip 100 and the memory 701 are connected via a bus. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0146] In an embodiment of the present application, the memory 701 stores instructions that can be executed by at least one artificial intelligence chip 100. At least one artificial intelligence chip 100 can execute the steps of the above-mentioned register resource allocation method by executing the instructions stored in the memory 701.

[0147] Among them, the artificial intelligence chip 100 is the control center of the computer device, and various interfaces and lines can be used to connect various parts of the computer device, and register resource allocation can be realized by running or executing instructions stored in the memory 701 and calling data stored in the memory 701. Optionally, the artificial intelligence chip 100 may include one or more processing units, and the artificial intelligence chip 100 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application program, etc., and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the artificial intelligence chip 100. In some embodiments, the artificial intelligence chip 100 and the memory 701 can be implemented on the same chip, and in some embodiments, they can also be implemented separately on independent chips.

[0148] The artificial intelligence chip 100 can be a general-purpose processor, such as a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware processor, or can be executed by a combination of hardware and software modules in the processor.

[0149] The memory 701 is a non-volatile computer-readable storage medium, which can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 701 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (RandomAccess Memory, RAM), a static random access memory (Static RandomAccess Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 701 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer device, but is not limited thereto. The memory 701 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0150] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium storing a computer program executable by a computer device. When the computer program runs on the computer device, the computer device executes the steps of the above-mentioned register resource allocation method.

[0151] Based on the same inventive concept, an embodiment of the present application provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer device, the computer device executes the steps of the above-mentioned register resource allocation method.

[0152] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0153] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer device or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0154] These computer program instructions may also be stored in a computer readable memory capable of directing a computer device or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0155] These computer program instructions can also be loaded onto a computer device or other programmable data processing device so that a series of operation steps are executed on the computer device or other programmable device to produce a process implemented by the computer device, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0156] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0157] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A register resource allocation method, characterized in that: include: For multiple computing scenarios of the fusion operator, perform operations separately: Based on the input shape of the fused operator and the operator combination corresponding to the fused operator in a computing scenario, the total register resources are divided into multiple types of register buffers; The multiple types of register buffers include: an interactive buffer allocated for the calculation results of the interactive tensor operators between the tensor calculation unit and the vector calculation unit, a temporary buffer allocated for at least one vector calculation operator, an accumulation buffer allocated for the block accumulation operator, and a fixed dedicated buffer allocated for the target calculation operator; the target calculation operator is an operator with dual inputs and implicit broadcast features; Based on the input shape of the fusion operator and the operator combination corresponding to the fusion operator in a computing scenario, determine the resource amount of each of the multiple types of register buffers allocated to the fusion operator, wherein the allocation order and the allocated resource amount of the register buffers required by each operator in the operator combination are determined based on the degree of influence of each operator on the performance of the fusion operator; the fusion operator corresponds to different operator combinations in different computing scenarios.

2. The method according to claim 1, characterized in that The operator combination includes: a tensor operator executed by a tensor calculation unit and at least one vector calculation operator executed by a vector calculation unit; The determining, based on the input shape of the fused operator and the operator combination corresponding to the fused operator in a computing scenario, the resource amounts of the multiple types of register buffers allocated to the fused operator, respectively, includes: Allocating a first resource amount corresponding to the interactive buffer from a total resource amount of the total register resources based on an input shape of the fusion operator; The second resource amount corresponding to the temporary buffer area is allocated from the remaining resource amount of the total resource amount.

3. The method according to claim 1, characterized in that The operator combination includes: a tensor operator executed by a tensor computing unit and a block accumulation operator executed by a vector computing unit; The determining, based on the input shape of the fused operator and the operator combination corresponding to the fused operator in a computing scenario, the resource amounts of the multiple types of register buffers allocated to the fused operator, respectively, includes: Based on the input shape of the fusion operator and the blocking method of the tensor operator, a third resource amount corresponding to the interaction buffer is allocated from the total resource amount of the total register resources, and a fourth resource amount corresponding to the accumulation buffer is allocated.

4. The method according to claim 3, characterized in that The operator combination further includes: at least one vector calculation operator executed by a vector calculation unit; a fifth resource amount of a temporary buffer area allocated to the at least one vector calculation operator is allocated from the remaining resource amount of the total resource amount.

5. The method according to claim 2 or 4, characterized in that Also includes: If the target computing operator exists in the at least one vector computing operator, a sixth resource amount corresponding to the fixed dedicated buffer is allocated from the remaining resource amount of the total resource amount.

6. A computer device comprising a memory, an artificial intelligence chip, and a computer program stored in the memory and running on the artificial intelligence chip, characterized in that: When the artificial intelligence chip executes the computer program, the steps of the method as claimed in any one of claims 1 to 5 are implemented.

7. A computer-readable storage medium, characterized in that: It stores a computer program executed by a computer device. When the computer program is run on the computer device, the computer device executes the steps of the method according to any one of claims 1 to 5.

8. A computer program product, characterized in that The computer program product comprises a computer program stored on a computer-readable storage medium, wherein the computer program comprises program instructions, and when the program instructions are executed by a computer device, the computer device is caused to execute the steps of the method according to any one of claims 1 to 5.

Images