Operator fusion method and device

Abstract

The invention provides an operator fusion method and device, and can be applied to the technical field of computers. The method comprises the steps that a calculation module analyzes operators in a calculation target according to a first operator set to obtain calculation factors; partitioning the calculation factor to obtain a calculation unit; analyzing the calculation target according to the first operator set to obtain a base operator comprising the calculation unit; carrying out fusion calculation on the base operator to obtain a first fusion result; and calculating the first fusion result to obtain a calculation result of the calculation target. Wherein the calculation target comprises P operators and Q operands, the first operator set comprises operation operators except a matrix multiplication operator and a division operator, the calculation factor is an operand at one end of the first operator, and the first operator belongs to the first operator set. Different types of operators can be fused, so that the number of the operators is reduced, the time consumed for calling the operators is shortened, and the performance of the AI model is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an operator fusion method and device. Background Art

[0002] In the fields of artificial intelligence (AI) and machine learning (ML), different functions can be achieved by designing or training different AI models (such as deep learning models, neural network models, machine learning models, etc.). AI models generally include multiple operators, which perform corresponding calculation operations on the data input to the operator and output the corresponding calculation results, thereby realizing the calculation of the AI ​​model on the data.

[0003] In an AI model, the more operators there are, the longer it takes to call an operator, which will seriously affect the end-to-end performance of the AI ​​model. To address this problem, the operators in the AI ​​model can be fused and some operators can be fused into a new operator, thereby reducing the number of operators, reducing the time it takes to call an operator, and improving the end-to-end performance of the AI ​​model.

[0004] However, for hardware devices (such as AI chips) that deploy AI models, it is very difficult to directly fuse different types of operators during the execution of AI models. For example, multiple operators need to perform matrix multiplication calculations before they can be fused into a new operator. Since directly performing complete matrix multiplication calculations on these operators will result in too much calculation, the computing resources on the hardware device cannot support this operation. Therefore, the hardware device will divide these operators into blocks, and then use the block results to perform multiple matrix multiplication calculations to obtain the final calculation results. However, if there is a reduction calculation in some operators, the reduction calculation cannot be completed after the operator is divided into blocks. Therefore, there is currently a lack of a general operator fusion method to fuse different types of operators. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides an operator fusion method and device, which can fuse different types of operators, thereby reducing the number of operators, reducing the time spent in calling operators, and improving the performance of AI models.

[0006] In a first aspect, an operator fusion method is provided, comprising: a calculation module first analyzes the operators in the calculation target according to a first operator set to obtain a calculation factor; then divides the calculation factor into blocks to obtain a calculation unit; then analyzes the calculation target according to the first operator set to obtain a base operator containing the calculation unit; finally, performs fusion calculation on the base operator to obtain a first fusion result, and calculates the first fusion result to obtain a calculation result of the calculation target. Wherein, the calculation target includes P operators and Q operands, P∈N+ , Q∈N + The first operator set includes operation operators other than matrix multiplication operators and division operators, the calculation factor is an operand at one end of the first operator, the first operator belongs to the first operator set, and the fusion calculation includes at least one of matrix multiplication calculation and reduction calculation.

[0007] In the above scheme, the operand at one end of the first operator is used as the object of block division, so that the computing unit obtained after block division can continue to retain the original information of the operand. Therefore, when the base operator composed of the computing units is fused, matrix multiplication calculation can be realized, and reduction calculation can also be realized. Therefore, this technical solution can fuse different types of operators, thereby reducing the number of operators, reducing the time consumption of calling operators, and improving the performance of AI models.

[0008] In some possible implementations, the computing module analyzes the operators in the computing target according to the first operator set to obtain computing factors, including: when the operator is the first operator, the computing module uses the operands at both ends of the operator as computing factors respectively; or, when the operator is a function including the first operator, the computing module uses the operands at both ends of the first operator in the operator as computing factors respectively.

[0009] In some possible implementations, the calculation module divides the calculation factors into blocks to obtain calculation units, including: the calculation module divides the calculation factors into blocks according to the dimensions of the calculation factors to obtain calculation units, wherein the dimension includes at least one of a row dimension, a column dimension, and a depth.

[0010] In the above scheme, the calculation factors can be divided into blocks in a variety of ways, such as dividing the calculation factors according to the row dimension, dividing the calculation factors according to the row dimension and the column dimension, etc. Therefore, according to the computing power of the device that applies the operator fusion method, a suitable computing unit can be provided for it, so that the device can realize the fusion processing of different types of operators.

[0011] In some possible implementations, the above-mentioned computing module blocks the computing factors according to the dimensions of the computing factors to obtain computing units, including: when there is no power operator in the computing factors, the above-mentioned computing module blocks the computing factors according to the above-mentioned dimensions to obtain computing units; or, when there is a power operator in the computing factors, the above-mentioned computing module blocks the operands executing the power operator in the computing factors according to the above-mentioned dimensions to obtain computing units.

[0012] In some possible implementations, the computing module analyzes the computing target according to the first operator set to obtain a base operator including the computing unit, including: the computing module analyzes the computing target according to the first operator set to obtain an analysis factor, and then analyzes the analysis factor with the computing unit as a unit to obtain the base operator.

[0013] In some possible implementations, the computing module analyzes the computing target according to the first operator set to obtain analysis factors, including: when the computing target to be analyzed is an integer, the computing module uses the operands at both ends of the first operator in the computing target to be analyzed as analysis factors; or, when the computing target to be analyzed is a fraction, the computing module uses the denominator of the computing target to be analyzed as the analysis factor, and then uses the operands at both ends of the first operator in the numerator of the computing target to be analyzed as analysis factors.

[0014] In the above scheme, the computing module first parses the computing target, then uses the operands distributed at both ends of the first operator in the parsed computing target as parsing factors, and finally uses the computing units obtained by dividing the computing factors into blocks as analysis units to analyze the parsing factors, so that the computational amount of the base operator obtained after the analysis is also based on the computing unit. Therefore, compared with the computational amount of directly performing a complete calculation on the computing target, the computational amount of the base operator is smaller, so even when computing resources are limited, the fusion of different types of operators can be achieved.

[0015] In some possible implementations, the computing module performs a fusion calculation on the base operators to obtain a first fusion result, including: the computing module first determines a first base operator including a first computing unit from multiple base operators, then determines a second base operator based on the first base operator, and then performs a fusion calculation on the second base operator to obtain a first fusion result, wherein the first computing unit is a computing unit in a first computing factor on a first dimension, the second base operator is a first base operator including a computing unit in a second computing factor, the second computing factor and the first computing factor are different computing factors, and the first fusion result is a calculation result of multiple second base operators on the first dimension.

[0016] In the above scheme, multiple second base operators on the first dimension are determined from multiple base operators, and then these second base operators are fused and calculated, so that the first fusion result is the calculation result on the first dimension. Since the second base operator is a component of the analytical factor on the first dimension, the first fusion result is also the calculation result of the analytical factor on the first dimension. After obtaining the calculation results of multiple analytical factors on the same dimension, multiple first fusion results are obtained, and then the multiple first fusion results are calculated, so the calculation results of the calculation target on the same dimension can be obtained, and then the final calculation result of the calculation target can be obtained. Therefore, this technical solution can realize the fusion of different types of operators, thereby reducing the number of operators, reducing the time consumption of calling operators, and improving the performance of AI models.

[0017] In some possible implementations, the P operators include the matrix multiplication operator, and the Q operands include a reduction result, wherein the reduction result is calculated based on the input matrix of the calculation target, including one or more of a sum, a mean, a maximum value, a minimum value, a variance, a standard deviation, and a cumulative sum.

[0018] In the second aspect, an operator fusion device is provided, including: an analysis module, a block module, a parsing module and a calculation module. The analysis module is used to analyze the operators in the calculation target according to the first operator set to obtain the calculation factors; the block module is used to block the calculation factors to obtain the calculation units; the parsing module is used to parse the calculation target according to the first operator set to obtain the base operator containing the calculation unit; the calculation module is used to perform fusion calculation on the base operator to obtain the first fusion result, and calculate the first fusion result to obtain the calculation result of the calculation target. The calculation target includes P operators and Q operands, P∈N + , Q∈N + The first operator set includes operation operators other than matrix multiplication operators and division operators, the calculation factor is an operand at one end of the first operator, the first operator belongs to the first operator set, and the fusion calculation includes at least one of matrix multiplication calculation and reduction calculation.

[0019] In some possible implementations, the above-mentioned analysis module is specifically used to: when the operator is a first operator, use the operands at both ends of the operator as calculation factors respectively; or, when the operator is a function including the first operator, use the operands at both ends of the first operator in the operator as calculation factors respectively.

[0020] In some possible implementations, the above-mentioned blocking module is specifically used to: block the calculation factors according to the dimensions of the calculation factors to obtain calculation units, wherein the dimension includes at least one of a row dimension, a column dimension, and a depth.

[0021] In some possible implementations, the above-mentioned blocking module is specifically used to: when there is no power operator in the calculation factor, block the calculation factor according to the above-mentioned dimension to obtain the calculation unit; or, when there is a power operator in the calculation factor, block the operands executing the power operator in the calculation factor according to the above-mentioned dimension to obtain the calculation unit.

[0022] In some possible implementations, the above-mentioned parsing module is specifically used to: parse the calculation target according to the first operator set to obtain the parsing factor, and then analyze the parsing factor with the calculation unit as a unit to obtain the base operator.

[0023] In some possible implementations, the above-mentioned analysis module is specifically used for: when the calculation target of the analysis is an integer, the operands at both ends of the first operator in the calculation target of the analysis are used as analysis factors respectively; or, when the calculation target of the analysis is a fraction, the denominator of the calculation target of the analysis is used as the analysis factor, and then the operands at both ends of the first operator in the numerator of the calculation target of the analysis are used as analysis factors respectively.

[0024] In some possible implementations, the above-mentioned computing module is specifically used to: determine a first base operator including a first computing unit from multiple base operators, determine a second base operator based on the first base operator, perform fusion calculation on the second base operator, and obtain a first fusion result, wherein the first computing unit is a computing unit in a first computing factor on a first dimension, the second base operator is a first base operator including a computing unit in a second computing factor, the second computing factor and the first computing factor are different computing factors, and the first fusion result is the calculation result of multiple second base operators on the first dimension.

[0025] In some possible implementations, the P operators include the matrix multiplication operator, and the Q operands include a reduction result, wherein the reduction result is calculated based on the input matrix of the calculation target, including one or more of a sum, a mean, a maximum value, a minimum value, a variance, a standard deviation, and a cumulative sum.

[0026] In a third aspect, a processor is provided, comprising a cache and an operator fusion device as in any one of the second aspect, wherein the cache is used to store data calculated by the operator fusion device.

[0027] In a fourth aspect, a computing device is provided, comprising a processor and a memory, wherein the memory is used to store instructions, and the processor is used to execute the instructions. When the processor executes the instructions, the method of any one of the first aspects is implemented.

[0028] According to a fifth aspect, a computer program product comprising instructions is provided. When the instructions are executed by a computing device, the computing device executes the method as described in any one of the first aspects.

[0029] In a sixth aspect, a computer-readable storage medium is provided, characterized in that it includes computer program instructions. When the computer program instructions are executed by a computing device, the computing device executes a method as described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flowchart of an operator fusion method provided by this application;

[0031] Figure 2 It is a structural schematic diagram of an operator fusion device provided by the present application;

[0032] Figure 3 It is a structural schematic diagram of a computing device provided by this application. DETAILED DESCRIPTION

[0033] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.

[0034] Operators in AI models are operation operators or functions used to perform various mathematical and logical operations. Operators are divided into basic operators and combined operators, where the basic operator is a single operation operator and the combined operator is a function including multiple basic operators and multiple operands. Different types of operators play a key role in building AI models and performing machine learning tasks. Referring to Table 1, which is exemplified in this application, Table 1 provides various types of operators and their functions and effects.

[0035] Table 1

[0036]

[0037]

[0038] It should be understood that the above Table 1 is only an example of operator types and is not a specific limitation here.

[0039] However, in an AI model, the more operators there are, the longer it takes to call an operator, which will seriously affect the end-to-end performance of the AI ​​model. To address this problem, the operators in the AI ​​model can be fused to fuse some operators into a new operator, thereby reducing the number of operators, reducing the time it takes to call operators, and improving the end-to-end performance of the AI ​​model. However, how to fuse different types of operators is an issue that needs to be addressed urgently.

[0040] In order to solve the above problems, the present application provides an operator fusion method that can fuse different types of operators, thereby reducing the number of operators, reducing the time spent in calling operators, and improving the performance of the AI ​​model.

[0041] See also Figure 1 , Figure 1 This is a flowchart of an operator fusion method provided by this application. The operator fusion method provided by this application includes:

[0042] S101: The processor analyzes operators in a calculation target according to a first operator set to obtain calculation factors.

[0043] In some possible implementations, the computation target consists of P operators and Q operands. + , Q∈N + .

[0044] In a specific implementation, the computation target is composed of P operators and Q operands, and the P operators include a matrix multiplication operator (@), and the Q operands include reduction results calculated based on the input matrix of the computation target. The reduction results include summation, mean, maximum, minimum, variance, standard deviation, cumulative summation, etc. For example, the computation target is:

[0045]

[0046] Then the calculation target Y includes the combination operator (layer normalization operator), basic operators @(matrix multiplication operator), +(addition operator), and also include operands X, E(X), Var(X), ε, A, B, where ε is a constant, X, A, B are input matrices, E(X), Var(X) are reduction results calculated based on the input matrix X, specifically, E(X) is the mean of X, and Var(X) is the variance of X.

[0047] In some possible implementations, the processor uses operands at both ends of the first operator as calculation factors, respectively. The first operator belongs to a first operator set. The first operator set includes all operation operators that do not perform matrix multiplication or division operations, such as an addition operator (+), a subtraction operator (-), and a power operator.

[0048] In a specific implementation, when the operator is a basic operator and the basic operator is the first operator, the processor uses the operands at both ends of the basic operator as calculation factors. Continuing with the above basic operator + (addition operator) as an example, the processor uses the operands at both ends of the addition operator as calculation factors to obtain two calculation factors A and B.

[0049] In a specific implementation, when the operator is a combinatorial operator and the combinatorial operator includes a first operator, the processor uses the operands at both ends of the first operator in the combinatorial operator as calculation factors. In particular, when the combinatorial operator is a fraction, the processor uses the denominator of the combinatorial operator as a calculation factor, and then uses the operands at both ends of the first operator in the numerator of the combinatorial operator as calculation factors. (Layer Normalization Operator) As an example, since the combination operator is a fraction, the processor sets the denominator of the combination operator As a calculation factor, the operands at both ends of the subtraction operator in the numerator of the combination operator are respectively used as calculation factors to obtain two calculation factors X and E(X).

[0050] S102: The processor divides the calculation factors into blocks to obtain calculation units.

[0051] In some possible implementations, the processor divides the calculation factor into blocks according to the dimension of the calculation factor to obtain multiple calculation units corresponding to the calculation factor. Specifically, when the calculation factor does not have a power operator, the calculation factor is directly divided into blocks to obtain the calculation unit. When the calculation factor has a power operator, the operands for executing the first power operator are divided into blocks to obtain the calculation unit. For example, when the calculation factor is X, E(X), Var(X), the calculation factor does not have a power operator. When the calculation factor is X, the calculation factor does not have a power operator. 2 , When , the calculation factor has a power operator. And, for Since it is a calculation factor with multiple power operators, the first power operator in the calculation factor is executed The operand E(X) is divided into blocks to obtain computing units.

[0052] In some possible implementations, the determination of the computing unit may include at least the following four methods:

[0053] (1) Use the same dimension to divide different calculation factors of the same operator into calculation units with the same parameters. The three calculation factors X, E(X), As an example, the three calculation factors X, E(X), are divided into N1×N2 computing units. For example, N1=2, N2=2, then the three computing factors X, E(X), They are divided into 2×2 computing units. Specifically, X and E(X) do not have a power operator, so X and E(X) are directly divided into blocks, and the resulting computing units are: and There is a power operator Therefore, the need to implement The operand Var(X)+ε is divided into blocks. Since ε is a constant, Var(X) is divided into blocks, and the resulting calculation units are:

[0054] (2) Use different dimensions to divide different calculation factors of the same operator into calculation units with different parameters. The three calculation factors X, E(X), As an example, the calculation factor X can be divided into N1×N2 calculation units in two dimensions, row and column. For example, N1=2, N2=2, and X does not have a power operator. Therefore, X is directly divided into blocks, and the obtained calculation units are: The calculation factor E(X) is divided into M1×1 calculation units using the row dimension. For example, M1=2, and E(X) does not have a power operator, so E(X) is directly divided into blocks, and the resulting calculation units are: You can also use the calculation factor directly without using dimensions As a computing unit.

[0055] (3) Use the same dimension to divide different calculation factors of different operators into calculation units with the same parameters. Taking the calculation factor X of the basic operator + (addition operator) and the calculation factor A of the basic operator + (addition operator) as an example, the calculation factors X and A can be divided into N1×N2 calculation units in the row and column dimensions. For example, N1=2, N2=2, then the calculation factors X and A are divided into 2×2 calculation units. Specifically, there is no power operator in X and A, so X and A are directly divided into blocks, and the resulting calculation units are:

[0056] (4) Different dimensions are used to divide different calculation factors of different operators into calculation units with different parameters. Taking the calculation factor X of and the calculation factor A of the basic operator + (addition operator) as an example, the calculation factor X can be divided into N1×N2 calculation units in the row and column dimensions. For example, N1=2, N2=2, and there is no power operator in X, so X is directly divided into blocks, and the resulting calculation units are: The calculation factor A is divided into M1×1 calculation units using the row dimension. For example, M1=2, and there is no power operator in A. Therefore, A is directly divided into blocks, and the resulting calculation units are:

[0057] It should be understood that the above-mentioned use of row dimensions and column dimensions to divide the calculation units is only an example. When the calculation factor has a higher dimension, more dimensions (such as depth) can be used to divide the calculation factor into multiple calculation units. This is not a specific limitation here.

[0058] In summary, the calculation factors can be divided into blocks in a variety of ways. Therefore, according to the computing capability of the device to which the operator fusion method is applied, a suitable computing unit can be provided.

[0059] S103: The processor parses the computing target according to the first operator set to obtain a base operator including a computing unit.

[0060] In some possible implementations, the processor may first parse the computing target according to the first operator set to obtain parsing factors, and then analyze the parsing factors using computing units as units to obtain base operators including the computing units.

[0061] First, the processor parses the calculation target according to the first operator set to obtain the analytical factors. Specifically, when the calculation target to be parsed is a whole expression, the processor uses the operands at both ends of the first operator in the calculation target as analytical factors. When the calculation target to be parsed is a fraction, the processor uses the denominator of the calculation target as an analytical factor, and then uses the operands at both ends of the first operator in the numerator of the calculation target as analytical factors. Continue with the calculation target in step S101 above. As an example, the calculation target Y is parsed according to the first operator set to obtain the parsed calculation target:

[0062]

[0063] Since the computation target Y to be parsed is a fraction, the processor will complete the denominator of the computation target Y to be parsed As an analytical factor, the operands at both ends of the addition operator and the subtraction operator in the numerator of the calculated target Y that has been analyzed are respectively used as analytical factors, specifically X@A, E(X)@A, X@B, and E(X)@B.

[0064] Next, the processor analyzes the analytical factors using the computational unit as a unit to obtain the base operator containing the computational unit. As calculation factors X, E(X), An example of the calculation unit of A and B. For the above analytical factor X@A, since the calculation unit of the calculation factor X is The calculation unit for calculating factor A is Therefore, we can get the base operators X1@A1, X2@A2, X3@A1, and X4@A2, which include the calculation unit of X and the calculation unit of A. Similarly, we can get the base operators of the above analytical factors E(X)@A, X@B, and E(X)@B, which are E(X1)@A1, E(X2)@A2, E(X3)@A1, E(X4)@A2, X1@B1, X2@B2, X3@B1, X4@B2, E(X1)@B1, E(X2)@B2, E(X3)@B1, E(X4)@B2, Var(X1), Var(X2), Var(X3), and Var(X4).

[0065] It should be understood that the above-mentioned step S101 "the processor analyzes the operators in the computing target according to the first operator set" and the step S103 "the processor parses the computing target according to the first operator set" can be executed simultaneously; or the step S101 "the processor analyzes the operators in the computing target according to the first operator set" can be executed first, and then the step S103 "the processor parses the computing target according to the first operator set" can be executed; or the step S103 "the processor parses the computing target according to the first operator set" can be executed first, and then the step S101 "the processor analyzes the operators in the computing target according to the first operator set" can be executed.

[0066] S104: The processor performs fusion calculation on the base operators to obtain a first fusion result.

[0067] In some possible implementations, the processor first determines a first base operator including a first computing unit from a plurality of base operators, then determines a second base operator based on the first base operator, and then performs a fusion calculation on the second base operator to obtain a first fusion result. The first computing unit belongs to a first computing unit set. The first computing unit set includes computing units in the first dimension in the first computing factor. The first fusion result is a calculation result of multiple second base operators in the first dimension.

[0068] First, the processor determines the first basic operator containing the first calculation unit. Continue to take the basic operators X1@A1, X2@A2, X3@A1, X4@A2, E(X1)@A1, E(X2)@A2, E(X3)@A1, E(X4)@A2, X1@B1, X2@B2, X3@B1, X4@B2, E(X1)@B1, E(X2)@B2, E(X3)@B1, E(X4)@B2, Var(X1), Var(X2), Var(X3), Var(X4) in step S203 as examples. If the first calculation factor is X, there are two groups of calculation units in the first dimension of the first calculation factor X, namely X1 and X2, X3 and X4. Among them, X1 and X2 are in the first row dimension, and X3 and X4 are in the second row dimension. Therefore, there are two first calculation unit sets corresponding to the first calculation factor X, namely {X1, X2} and {X3, X4}. There are also two groups of calculation units in the same column dimension in the first calculation factor X, namely X1 and X3, and X2 and X4. Among them, X1 and X3 are in the first column dimension, and X2 and X4 are in the second column dimension. Therefore, there are two first calculation unit sets corresponding to the first calculation factor X, namely {X1, X3} and {X2, X4}. Taking the first calculation unit set {X1, X2} as an example, X1 and X2 are both first calculation units. Therefore, the first basic operators containing the first calculation units are X1@A1, X2@A2, X1@B1, and X2@B2.

[0069] Subsequently, the processor determines the second base operator based on the first base operator. In a specific implementation, the processor may use the first base operator of the calculation unit including the second calculation factor as the second base operator. Wherein, the second calculation factor and the first calculation factor are different calculation factors. For example, when the first calculation factor is X, the second calculation factor is A, and the first base operator is X1@A1, X2@A2, X1@B1, and X2@B2, X1@A1 and X2@A2 may be used as the second base operator. In another specific implementation, the processor may also use the first base operator of the calculation unit including the second calculation factor and the calculation unit of the third calculation factor as the second base operator. Wherein, the second calculation factor, the third calculation factor, and the first calculation factor are all different calculation factors. For example, when the first calculation factor is X, the second calculation factor is C, the third calculation factor is D, and the first base operator is X1@A1@C1@D1, X2@A2@C2@E1, X1@B1@C3@E1, X2@B2@C4@E2, X1@A1@C1@D1 can be used as the second base operator. Among them, C1, C2, C3, C4 are calculation units for calculation factor C, D1 is a calculation unit for calculation factor D, and E1 and E2 are calculation units for calculation factor E. It should be understood that the above two specific implementation methods are only used as an example and are not specifically limited here.

[0070] Next, the processor performs a fusion calculation on the second base operator to obtain a first fusion result. The fusion calculation includes at least one of a matrix multiplication calculation and a reduction calculation. The matrix multiplication calculation is used to perform a multiplication operation on two matrices. The reduction calculation is used to perform an accumulation operation or a summary operation on the first calculation unit of the first dimension in the matrix, including a row reduction calculation, a column reduction calculation, and the like. When the fusion calculation includes a reduction calculation, the first fusion result includes overall information of multiple first calculation units on the first dimension, including the sum, mean, maximum value, minimum value, variance, standard deviation, cumulative sum, and the like of the multiple first calculation units.

[0071] Continue with the calculation target in step S203 above As an example:

[0072] (1) About the analytical factor X@A

[0073] If there are two groups of second basis operators of X@A, they are X1@A1 and X2@A2, X3@A1 and X4@A2.

[0074] For the second base operators X1@A1 and X2@A2 of the first group, the processor performs matrix multiplication calculation on the second base operators to obtain the first fusion result R X@A (1), the calculation formula is:

[0075] R X@A (1) = X1@A1+X2@A2

[0076] For the second base operators X3@A1 and X4@A2 of the second group, the processor performs matrix multiplication on the second base operators to obtain the first fusion result R X@A (2), the calculation formula is:

[0077] R X@A (2) = X3@A1+X4@A2

[0078] Therefore, the first fusion result R X@A (1) R X@A (2) respectively represent the calculation results of the analytical factor X@A in different row dimensions, where R X@A (1) represents the calculation result in the first row dimension of X@A, R X@A (2) represents the calculation result in the second row dimension of X@A.

[0079] Similarly, if there are multiple groups of second basis operators, and each group has multiple second basis operators, the processor performs matrix multiplication calculation on multiple second basis operators to obtain the first fusion result R X@A (k), the calculation formula is:

[0080] R X@A (k) = X k,1 @A i,1 +X k,2 @A i,2 +…+X k,m @A i,m

[0081] Among them, R X@A (k) represents the calculation result of the k-th row dimension in the analytical factor X@A, X k,1 , X k,2 , …, X k,m represents the first computational unit for computing the k-th row dimension in factor X, A i,1 , A i,2 , …, A i,m Represents the first computational unit for the i-th column dimension in the computation factor A.

[0082] (2) About the analytical factor E(X)@A

[0083] If there are two groups of second basis operators of E(X)@A, they are E(X1)@A1 and E(X2)@A2, E(X3)@A1 and E(X4)@A2.

[0084] For the second basis operators E(X1)@A1 and E(X2)@A2 of the first group, the processor performs row reduction calculation on the second basis operators to obtain the first fusion result R E(X)@A (1), the calculation formula is:

[0085]

[0086] For the second base operators E(X3)@A1 and E(X4)@A2 of the second group, the processor performs row reduction calculation on the second base operators to obtain the first fusion result R E(X)@A (2), the calculation formula is:

[0087]

[0088] Therefore, the first fusion result R E(X)@A (1) R E(X)@A (2) respectively represent the calculation results of the analytical factor E(X)@A in different row dimensions, where R E(X)@A (1) represents the calculation result of E(X)@A in the first row dimension, R E(X)@A (2) represents the calculation result of E(X)@A in the second row dimension. is the coefficient, E1(X 1,2 ) is the mean of E(X1) and E(X2), specifically Therefore, the first fusion result R E(X)@A (1) Contains the mean of the first computational units E(X1) and E(X2) of the first row dimension of E(X); is the coefficient, E2(X 3,4 ) is the mean of E(X3) and E(X4), specifically Therefore, the first fusion result R E(X)@A (2) contains the means of the first computational units E(X3) and E(X4) of the second row dimension in E(X).

[0089] Similarly, if there are multiple groups of second basis operators, and each group has multiple second basis operators, the processor performs row reduction calculations on multiple second basis operators to obtain the first fusion result R E(X)@A (k), the calculation formula is:

[0090]

[0091] Among them, R E(X)@A(k) represents the calculation result of the analytical factor E(X)@A in the k-th row dimension, R E(X)@A (k-1) represents the calculation result of E(X)@A on the k-1th row dimension. k (X 1，2，..，m ) is E(X k,1 )、E(X k,2 ),...,E(X k,m ), E k (X 1，2，..,,-1 ) is E(X k，1 )、E(X k，1 ),...,E(X k，m-1 ), E(X k，1 )、E(X k，2 ),...,E(X k，m-1 )、E(X k，m ) represents the first calculation unit of the kth row dimension in the calculation factor E(X), A i，m represents the mth first calculation unit of the i-th column dimension in the calculation factor A. Therefore, the first fusion result R E(X)@A (k) contains multiple first calculation units E(X) of the kth row dimension in E(X) k,i ), that is, the first fusion result includes multiple first calculation units E(X k,i )’s overall information.

[0092] (3) About the analytical factor X@B

[0093] If there are two groups of second basis operators of X@A, they are X1@B1 and X2@B2, X3@B1 and X4@B2.

[0094] For the second base operators X1@B1 and X2@B2 of the first group, the processor performs matrix multiplication calculation on the second base operators to obtain the first fusion result R X@B (1), the calculation formula is:

[0095] R X@B (1) = X1@B1+X2@B2

[0096] For the second base operators X3@B1 and X4@B2 of the second group, the processor performs matrix multiplication calculation on the second base operators to obtain the first fusion result R X@B (2), the calculation formula is:

[0097] R X@B (2) = X3@B1+X4@B2

[0098] Therefore, the first fusion result R X@B (1) R X@B(2) respectively represent the calculation results of the analytical factor X@B in different row dimensions, where R X@B (1) represents the calculation result in the first row dimension of X@B, R X@B (2) represents the calculation result in the second row dimension of X@B.

[0099] Similarly, if there are multiple groups of second basis operators, and each group has multiple second basis operators, the processor performs matrix multiplication calculation on multiple second basis operators to obtain the first fusion result R X@B (k), the calculation formula is:

[0100] R X@B (k) = X k，1 @B i,1 +X k,2 @B i,2 +…+X k，m @B i,m

[0101] Among them, R X@B (k) represents the calculation result of the k-th row dimension in the analytical factor X@B, X k,1 , X k,2 , ..., X k，m represents the first calculation unit of the k-th row dimension in the calculation factor X, B i,1 , B i，2 , ..., B i，m Represents the first computational unit for computing the i-th column dimension in factor B.

[0102] (4) About the analytical factor E(X)@B

[0103] If there are two groups of second basis operators of E(X)@B, they are E(X1)@B1 and E(X2)@B2, E(X3)@B1 and E(X4)@B2.

[0104] For the second base operators E(X1)@B1 and E(X2)@B2 of the first group, the processor performs row reduction calculation on the second base operators to obtain the first fusion result R E(X)@B (1), the calculation formula is:

[0105]

[0106] For the second base operators E(X3)@B1 and E(X4)@B2 of the second group, the processor performs row reduction calculation on the second base operators to obtain the first fusion result R E(X)@B (2), the calculation formula is:

[0107]

[0108] Therefore, the first fusion result R E(X)@B (1) R E(X)@B (2) respectively represent the calculation results of the analytical factor E(X)@B in different row dimensions, where R E(X)@B (1) represents the calculation result of E(X)@B in the first row dimension, R E(X)@B (2) represents the calculation result of E(X)@B in the second row dimension. is the coefficient, E1(X 1，2 ) is the mean of E(X1) and E(X2), specifically Therefore, the first fusion result R E(X)@B (1) Contains the mean of the first computational units E(X1) and E(X2) of the first row dimension of E(X); is the coefficient, E2(X 3，4 ) is the mean of E(X3) and E(X4), specifically Therefore, the first fusion result R E(X)@B (2) contains the means of the first computational units E(X3) and E(X4) of the second row dimension in E(X).

[0109] Similarly, if there are multiple groups of second basis operators, and each group has multiple second basis operators, the processor performs row reduction calculations on multiple second basis operators to obtain the first fusion result R E(X)@B (k), the calculation formula is:

[0110]

[0111] Among them, R E(X)@B (k) represents the calculation result of the analytical factor E(X)@B in the k-th row dimension, R E(X)@B (k-1) represents the calculation result of E(X)@B on the k-1th row dimension. k(X1，2，..，m ) is E(X k,1 )、E(X k,2 ),...,E(X k,m ), E k (X 1，2，..,,-1 ) is E(X k，1 )、E(X k，1 ),...,E(X k，m-1 ), E(X k，1 )、E(X k，2 ),...,E(X k，m-1 )、E(X k，m ) represents the first calculation unit of the kth row dimension in the calculation factor E(X), B i,m represents the mth first calculation unit of the i-th column dimension in the calculation factor B. Therefore, the first fusion result RE(X)@B (k) contains multiple first calculation units E(X) of the kth row dimension in E(X) k，i ), that is, the first fusion result includes multiple first calculation units E(X k，i )’s overall information.

[0112] (5) About analytical factors

[0113] like There are two groups of second basis operators, namely Var(X1) and Var(X2), Var(X3) and Var(X4).

[0114] For the second basis operators Var(X1) and Var(X2) of the first group, the processor performs row reduction calculation on the second basis operators to obtain the first fusion result R Var(X) (1), the calculation formula is:

[0115]

[0116] For the second base operators Var(X3) and Var(X4) of the second group, the processor performs row reduction calculation on the second base operators to obtain the first fusion result R Var(X) (2), the calculation formula is:

[0117]

[0118] Therefore, the first fusion result R Var(X) (1) R Var(X) (2) respectively represent the calculation results of the analytical factor Var(X) in different row dimensions, where R Var(X) (1) represents the calculation result of Var(X) in the first row dimension, R Var(X) (2) represents the calculation result of Var(X) in the second row dimension, E1(X 1，2 ) is the mean of E(X1) and E(X2), specifically Therefore, the first fusion result R Var(X) (1) includes the first calculation unit Var(X1) and Var(X2) of the first row dimension in Var(X), the first calculation unit E(X1) and E(X2) of the first row dimension in E(X), and the mean of E(X1) and E(X2); E2(X 3，4 ) is the mean of E(X3) and E(X4), specifically Represents X i Therefore, the first fusion result R Var(X)(2) includes the first calculation units Var(X3) and Var(X4) of the second row dimension in Var(X), the first calculation units E(X3) and E(X4) of the second row dimension in E(X), and the mean of E(X3) and E(X4).

[0119] Similarly, if there are multiple groups of second basis operators, and each group has multiple second basis operators, the processor performs row reduction calculations on multiple second basis operators to obtain the first fusion result R Var(X) (k), the calculation formula is:

[0120]

[0121] Among them, R Var(X) (k) represents the calculation result of the k-th row dimension in the analytical factor Var(X). k,m ) represents the mth first calculation unit of the kth row dimension in the calculation factor Var(X), E(X k，m ) represents the mth first calculation unit of the kth row dimension in the calculation factor E(X), E k (X 1，2，...,m ) is E(X k,1 )、E(X k，2 ),...,E(X k，m ), Represents X k，m Therefore, the first fusion result R Var(X) (k) contains multiple first calculation units Var(X) of the kth row dimension in Var(X) k，m ), multiple first calculation units E(X) of the kth row dimension in E(X) k,m ), and a plurality of first computing units E(X k,m ), that is, the first fusion result contains the overall information of multiple first computing units in the k-th row dimension.

[0122] In summary, the processor is responsible for computing the target The second base operator in is used for fusion calculation, and the first fusion result obtained includes R X@A (k), R E(X)@A (k), R X@B (k), R E(X)@B (k), R Var (X)(k).

[0123] S105: The processor calculates the first fusion result to obtain a calculation result of the calculation target.

[0124] In some possible implementations, the processor calculates the multiple first fusion results in the first dimension to obtain a calculation result of the calculation target. X@A (k), R E(X)@A (k), R X@B (k), R E(X)@B (k), R Var(X) (k) As an example, by calculating the first fusion results on the k-th row dimension, the calculation result Y on the k-th row dimension in the calculation target Y can be obtained. k , the calculation formula is:

[0125]

[0126] Therefore, the calculation result of the calculation target Y can be expressed as: Y = [Y1; Y2; ...; Y k ;…;Y n ].

[0127] It should be understood that the above-mentioned operator fusion method implemented by the processor is only used as an example, and the operator fusion method can also be implemented by the computing module in the processor, and the operator fusion method can also be implemented by a computing device equipped with a processor, which is not specifically limited here. Among them, the computing module in the processor can be implemented by software, or it can be implemented by hardware. The computing device can be a supercomputer, a network computer (such as a server, a workstation, a hub, a switch, a router), an industrial control computer (such as a PC bus industrial computer, a programmable logic control system, a distributed control system, a fieldbus system, a numerical control system), a personal computer (such as a desktop computer, an all-in-one computer, a laptop computer, a handheld computer, a tablet computer), an embedded processor (such as an embedded microcontroller, an embedded microprocessor, an embedded digital signal processor, an embedded system on a chip), etc.

[0128] In summary, by implementing the operator fusion method provided by the present application, the operands at both ends of the first operator in the calculation target are respectively used as calculation factors, so that the calculation unit obtained after the calculation factor is divided into blocks can continue to retain the original information of the operand, and then the calculation unit is used as the analysis unit to analyze the analysis factors in the calculation target that has been resolved, so that the calculation amount of the base operator obtained after the analysis is also based on the calculation unit. As a result, when the base operator is fused, matrix multiplication calculations can be realized, and reduction calculations can also be realized. Therefore, the present technical solution can fuse different types of operators, thereby reducing the number of operators, reducing the time consumption of calling operators, and improving the performance of the AI ​​model. Moreover, compared with the amount of calculation of directly performing a complete calculation on the calculation target, the amount of calculation of the base operator in the present technical solution is smaller, so even in the case of limited computing resources, the fusion of different types of operators can be achieved.

[0129] See also Figure 2 , Figure 2 2 is a schematic diagram of the structure of an operator fusion device provided by the present application. The operator fusion device 200 can be used to implement the above-mentioned operator fusion method. Figure 2 As shown, the operator fusion device 200 includes: an analysis module 201, a block module 202, a parsing module 203 and a calculation module 204.

[0130] The analysis module 201 is used to analyze the operators in the calculation target according to the first operator set to obtain the calculation factors;

[0131] The block module 202 is used to divide the calculation factors into blocks to obtain calculation units;

[0132] The parsing module 203 is used to parse the calculation target according to the first operator set to obtain a basic operator including a calculation unit;

[0133] The calculation module 204 is used to perform fusion calculation on the base operators to obtain a first fusion result;

[0134] The calculation module 204 is also used to calculate the first fusion result to obtain a calculation result of the calculation target;

[0135] The computation target includes P operators and Q operands, P∈N + , Q∈N + The first operator set includes operation operators other than matrix multiplication operators and division operators, the calculation factor is an operand at one end of the first operator, the first operator belongs to the first operator set, and the fusion calculation includes at least one of matrix multiplication calculation and reduction calculation.

[0136] In some possible implementations, the analysis module 201 is specifically used to, when the operator is a first operator, use the operands at both ends of the operator as calculation factors respectively; or, when the operator is a function including the first operator, use the operands at both ends of the first operator in the operator as calculation factors respectively.

[0137] In some possible implementations, the blocking module 202 is specifically configured to block the calculation factors according to the dimensions of the calculation factors to obtain calculation units, where the dimensions include at least one of row dimensions, column dimensions, and depth.

[0138] In some possible implementations, the blocking module 202 is specifically used to block the calculation factors according to dimensions to obtain calculation units when there is no power operator in the calculation factors; or, when there is a power operator in the calculation factors, block the operands that execute the power operator in the calculation factors according to the dimensions to obtain calculation units.

[0139] In some possible implementations, the parsing module 203 is specifically configured to parse the computation target according to the first operator set to obtain parsing factors, and then analyze the parsing factors using the computation unit as a unit to obtain the base operator.

[0140] In some possible implementations, the parsing module 203 is specifically used to use the operands at both ends of the first operator in the parsed calculation target as parsing factors when the parsed calculation target is an integer; or, when the parsed calculation target is a fraction, use the denominator of the parsed calculation target as the parsing factor, and then use the operands at both ends of the first operator in the numerator of the parsed calculation target as parsing factors.

[0141] In some possible implementations, the computing module 204 is specifically used to determine a first base operator including a first computing unit from multiple base operators, determine a second base operator based on the first base operator, perform fusion calculations on the second base operator, and obtain a first fusion result, wherein the first computing unit is a computing unit in a first computing factor on a first dimension, the second base operator is a first base operator including a computing unit in a second computing factor, the second computing factor and the first computing factor are different computing factors, and the first fusion result is a calculation result of multiple second base operators on the first dimension.

[0142] In some possible implementations, the P operators include a matrix multiplication operator, and the Q operands include a reduction result, wherein the reduction result is calculated based on an input matrix of a calculation target, and includes one or more of a sum, a mean, a maximum value, a minimum value, a variance, a standard deviation, and a cumulative sum.

[0143] The above analysis module 201, the block module 202, the parsing module 203 and the calculation module 204 can be implemented by software or by hardware. Exemplarily, the implementation of the analysis module 201 is described below by taking the analysis module 201 as an example. Similarly, the implementation of the block module 202, the parsing module 203 and the calculation module 204 can refer to the implementation of the analysis module 201.

[0144] As an example of a software functional unit, the analysis module 201 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the analysis module 201 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code can be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code can be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with close geographical locations. Among them, usually a region can include multiple AZs.

[0145] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0146] As an example of a hardware functional unit, the analysis module 201 may include at least one computing device, such as a server, etc. Alternatively, the analysis module 201 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0147] The multiple computing devices included in the analysis module 201 can be distributed in the same region or in different regions. The multiple computing devices included in the analysis module 201 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the analysis module 201 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0148] It should be noted that, in other embodiments, the analysis module 201 can be used to execute any step in the operator fusion method, the block module 202 can be used to execute any step in the operator fusion method, the parsing module 203 can be used to execute any step in the operator fusion method, and the computing module 204 can be used to execute any step in the operator fusion method. The steps that the analysis module 201, the block module 202, the parsing module 203 and the computing module 204 are responsible for implementing can be specified as needed. The full functions of the operator fusion device are realized by respectively implementing different steps in the operator fusion method through the analysis module 201, the block module 202, the parsing module 203 and the computing module 204.

[0149] According to the operator fusion method provided in the present application, the present application further provides a processor, which includes a cache and the aforementioned operator fusion device 200. The cache is used to store data calculated by the operator fusion device 200.

[0150] See also Figure 3 , Figure 3 is a schematic diagram of the structure of a computing device provided by this application. Figure 3 As shown, the computing device 300 provided in the present application includes: a bus 301, a processor 302, a memory 303 and a communication interface 304. The processor 302, the memory 303 and the communication interface 304 communicate with each other through the bus 301. The computing device 300 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 300.

[0151] The bus 301 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3The bus 301 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 301 may include a path for transmitting information between various components of the computing device 300 (for example, the memory 303, the processor 302, and the communication interface 304).

[0152] The processor 302 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0153] The memory 303 may include a volatile memory, such as a random access memory (RAM). The memory 303 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0154] The memory 303 stores executable program codes, and the processor 302 executes the executable program codes to respectively implement the functions of the aforementioned analysis module 201, the block module 202, the parsing module 203, and the calculation module 204, thereby implementing the aforementioned Figure 1 That is, the memory 303 stores instructions for executing the operator fusion method.

[0155] The communication interface 304 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 300 and other computing devices or communication networks.

[0156] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be a software or program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on a computing device, the computing device executes the aforementioned operator fusion method.

[0157] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the aforementioned operator fusion method.

[0158] It should be understood that in the embodiments of the present invention, "when", "when" and "if" all mean that the device will make corresponding processing under certain objective circumstances, but do not limit the time, nor do they require that the device must perform a judgment action when implementing it, nor do they mean that there are other limitations.

[0159] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. An operator fusion method, characterized in that: include: The calculation module analyzes the operators in the calculation target according to the first operator set to obtain the calculation factor, wherein the calculation target includes P operators and Q operands, P∈N + , Q∈N + , the first operator set includes operation operators other than matrix multiplication operators and division operators, the calculation factor is an operand at one end of the first operator, and the first operator belongs to the first operator set; The calculation module divides the calculation factors into blocks to obtain calculation units; The computing module parses the computing target according to the first operator set to obtain a base operator including the computing unit; The computing module performs fusion calculation on the base operator to obtain a first fusion result, wherein the fusion calculation includes at least one of matrix multiplication calculation and reduction calculation; The calculation module calculates the first fusion result to obtain a calculation result of the calculation target.

2. The method according to claim 1, characterized in that The calculation module analyzes the operators in the calculation target according to the first operator set to obtain calculation factors, including: In the case where the operator is the first operator, the calculation module uses operands at both ends of the operator as the calculation factors respectively; In the case where the operator is a function including the first operator, the calculation module uses operands at both ends of the first operator in the operator as the calculation factors respectively.

3. The method according to claim 1 or 2, characterized in that: The calculation module divides the calculation factors into blocks to obtain calculation units, including: The calculation module divides the calculation factor into blocks according to the dimension of the calculation factor to obtain the calculation unit, wherein the dimension includes at least one of a row dimension, a column dimension, and a depth.

4. The method according to claim 3, characterized in that The calculation module divides the calculation factor into blocks according to the dimension of the calculation factor to obtain the calculation unit, including: In the case where the calculation factor does not have a power operator, the calculation module divides the calculation factor into blocks according to the dimension to obtain the calculation unit; In the case where the calculation factor has the power operator, the calculation module divides the operands for executing the power operator in the calculation factor into blocks according to the dimension to obtain the calculation unit.

5. The method according to any one of claims 1 to 4, characterized in that: The computing module parses the computing target according to the first operator set to obtain a base operator containing the computing unit, including: The calculation module analyzes the calculation target according to the first operator set to obtain analysis factors, and then analyzes the analysis factors with the calculation unit as a unit to obtain the basic operator.

6. The method according to claim 5, characterized in that The calculation module analyzes the calculation target according to the first operator set to obtain an analysis factor, including: When the calculated target after the analysis is an integer, the calculation module uses the operands at both ends of the first operator in the calculated target after the analysis as the analysis factors respectively; In the case where the calculated target to be analyzed is a fraction, the calculation module uses the denominator of the calculated target to be analyzed as the analytical factor, and then uses the operands at both ends of the first operator in the numerator of the calculated target to be analyzed as the analytical factors respectively.

7. The method according to any one of claims 1 to 6, characterized in that: The calculation module performs fusion calculation on the base operator to obtain a first fusion result, including: The calculation module determines a first base operator including a first calculation unit from the plurality of base operators, wherein the first calculation unit is a calculation unit on a first dimension in the first calculation factor; The calculation module determines a second base operator based on the first base operator, wherein the second base operator is the first base operator including the calculation unit in the second calculation factor, and the second calculation factor is a different calculation factor from the first calculation factor; The calculation module performs the fusion calculation on the second basis operator to obtain the first fusion result, wherein the first fusion result is a calculation result of multiple second basis operators in the first dimension.

8. The method according to any one of claims 1 to 7, characterized in that: The P operators include the matrix multiplication operator, and the Q operands include reduction results, wherein the reduction results are calculated based on the input matrix of the calculation target, including one or more of the sum, mean, maximum value, minimum value, variance, standard deviation, and cumulative sum.

9. An operator fusion device, characterized in that: include: Analysis module, segmentation module, parsing module and calculation module, The analysis module is used to analyze the operators in the calculation target according to the first operator set to obtain the calculation factor, wherein the calculation target includes P operators and Q operands, P∈N + , Q∈N + , the first operator set includes operation operators other than matrix multiplication operators and division operators, the calculation factor is an operand at one end of the first operator, and the first operator belongs to the first operator set; The block division module is used to divide the calculation factors into blocks to obtain calculation units; The parsing module is used to parse the computing target according to the first operator set to obtain a base operator including the computing unit; The computing module is used to perform fusion calculation on the base operators to obtain a first fusion result, wherein the fusion calculation includes at least one of matrix multiplication calculation and reduction calculation; The calculation module is further used to calculate the first fusion result to obtain a calculation result of the calculation target.

10. The device according to claim 9, characterized in that The analysis module is specifically used to, when the operator is the first operator, use the operands at both ends of the operator as the calculation factors respectively; when the operator is a function including the first operator, use the operands at both ends of the first operator in the operator as the calculation factors respectively.

11. The device according to claim 9 or 10, characterized in that The blocking module is specifically used to block the calculation factor according to the dimension of the calculation factor to obtain the calculation unit, wherein the dimension includes at least one of a row dimension, a column dimension, and a depth.

12. The device according to claim 11, characterized in that The blocking module is specifically used to block the calculation factor according to the dimension to obtain the calculation unit when there is no power operator in the calculation factor; In the case where the calculation factor has the power operator, the operands for executing the power operator in the calculation factor are divided into blocks according to the dimension to obtain the calculation unit.

13. The device according to any one of claims 9 to 12, characterized in that: The analysis module is specifically used to analyze the calculation target according to the first operator set to obtain the analysis factor, and then analyze the analysis factor with the calculation unit as a unit to obtain the basic operator.

14. The device according to claim 12, characterized in that The parsing module is specifically configured to use the operands at both ends of the first operator in the parsed calculation target as the parsing factors respectively when the parsed calculation target is a whole number; When the calculated target of the analysis is a fraction, the denominator of the calculated target of the analysis is used as the analysis factor, and the operands at both ends of the first operator in the numerator of the calculated target of the analysis are used as the analysis factors respectively.

15. The device according to any one of claims 9 to 14, characterized in that: The calculation module is specifically used to determine a first basic operator including a first calculation unit from multiple basic operators, determine a second basic operator based on the first basic operator, perform the fusion calculation on the second basic operator, and obtain the first fusion result, wherein the first calculation unit is a calculation unit in a first calculation factor on a first dimension, the second basic operator is a first basic operator including a calculation unit in a second calculation factor, the second calculation factor and the first calculation factor are different calculation factors, and the first fusion result is a calculation result of multiple second basic operators on the first dimension.

16. The device according to any one of claims 9 to 15, characterized in that The P operators include the matrix multiplication operator, and the Q operands include reduction results, wherein the reduction results are calculated based on the input matrix of the calculation target, including one or more of the sum, mean, maximum value, minimum value, variance, standard deviation, and cumulative sum.

17. A processor, characterized in that: It comprises a cache and an operator fusion device as described in any one of claims 9 to 16, wherein the cache is used to store data calculated by the operator fusion device.

18. A computing device, characterized in that The method comprises a processor and a memory, wherein the memory is used to store instructions, and the processor is used to execute the instructions. When the processor executes the instructions, the method according to any one of claims 1 to 8 is implemented.

19. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device, the computing device is caused to perform the method according to any one of claims 1 to 8.

20. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device, the computing device performs the method according to any one of claims 1 to 8.

Images