Code optimization method and optimization equipment

Abstract

The invention provides a code optimization method and optimization equipment. The method is applied to optimization equipment, and comprises the following steps: determining M first general matrix vector multiplication (GEMV) calculations in a code to be optimized based on an identification rule; determining N second GEMV calculations in the M first GEMV calculations based on a merging rule; and inserting general matrix multiplication (GEMM) calculations corresponding to the N second GEMV calculations into insertion positions of the GEMM calculations in the code, and deleting the N second GEMV calculations. According to the scheme, automatic identification and GEMV calculation merging can be realized, and the problem of matrix unit calculation power waste is solved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a code optimization method and an optimization device. Background Art

[0002] As a computing chip, the matrix unit has powerful computing capabilities and is widely used in high-performance computing (HPC), image processing, artificial intelligence (AI) and other fields.

[0003] The code usually contains a large number of general matrix vector multiplication (GEMV) calculations. When the code is running, the GEMV calculation will call the matrix unit for calculation, but the GEMV calculation cannot fully utilize the computing power of the matrix unit, which will waste the computing power of the matrix unit. Summary of the invention

[0004] The present application provides a code optimization method and an optimization device, which can automatically identify GEMV calculations that can be merged in the code and merge them into general matrix multiplication (GEMM) calculations, thereby solving the problem of matrix unit computing power waste.

[0005] In a first aspect, the present application provides a code optimization method, which is applied to an optimization device. The method comprises: the optimization device determines M first general matrix-vector multiplication GEMV calculations in the code to be optimized based on an identification rule, where M is a positive integer; determines N second GEMV calculations in the M first GEMV calculations based on a merging rule, where N is a positive integer; inserts the general matrix multiplication GEMM calculations corresponding to the N second GEMV calculations into the insertion position of the GEMM calculation in the code, and deletes the N second GEMV calculations.

[0006] In the above scheme, the optimization device identifies the input code based on the pre-configured recognition rules, and selects the second GEMV calculation that can be merged based on the pre-configured merging rules, thereby realizing the automatic recognition and merging of the GEMV calculations in the code and solving the problem of waste of matrix unit computing power.

[0007] In a possible implementation of the first aspect, the merging rules include: the matrices of the N second GEMV calculations are the same, the step sizes of the vectors are the same, the control flow characteristics are the same, the input parameters of each second GEMV calculation have no data dependencies on the GEMV calculations before their respective positions, and the output parameters of the second GEMV calculations have no data dependencies on the GEMV calculations after their respective positions.

[0008] In the above scheme, multiple GEMV calculations that can be merged in the code can be determined through the merging rule, so that these multiple GEMV calculations can be merged into GEMM calculations, reducing the number of times the matrix unit is called for GEMV calculations. In addition, the data dependency of the input parameters and the data dependency of the output parameters are considered in the merging rule, which can avoid the calculation order of the GEMM calculation being affected by the merging of GEMV calculations, resulting in errors in the calculation results.

[0009] In a possible implementation of the first aspect, the merging rules include: the second GEMV calculation is located in a loop calculation of the code and the loop calculation does not include other GEMV calculations, the input parameters of the second GEMV calculation have no data dependency on the GEMV calculation before the position of the second GEMV calculation, and the output parameters of the second GEMV calculation have no data dependency on the GEMV calculation after the position of the second GEMV calculation.

[0010] In the above scheme, the GEMV calculations in the code that belong to the loop calculations can be determined through the merging rule, so that the GEMV calculations in the loop calculations can be merged into GEMM calculations, reducing the number of times the loop calculations call matrix units.

[0011] In a possible implementation of the first aspect, when N≠1, the method further includes: determining an insertion position of the GEMM calculation in the code according to a data flow graph of the code.

[0012] Wherein, determining the insertion position of the GEMM calculation in the code according to the data flow graph of the code includes: if it is determined according to the data flow graph that there is data dependency between the output parameters of the N second GEMV calculations and the positions of the N second GEMV calculations for non-GEMV calculations, determining the insertion position to be the position of the first second GEMV calculation among the N second GEMV calculations; if there is data dependency between the input parameters of the N second GEMV calculations and the positions of the N second GEMV calculations according to the data flow graph for non-GEMV calculations, determining the insertion position to be the position of the Nth second GEMV calculation among the N second GEMV calculations; if there is no data dependency between the input parameters and output parameters of the N second GEMV calculations and the positions of the N second GEMV calculations according to the data flow graph for non-GEMV calculations, determining the insertion position to be the position of the first second GEMV calculation.

[0013] In a possible implementation of the first aspect, when N=1 and the second GEMV calculation is located in a loop calculation of the code, the method further includes: if the loop calculation includes a first non-GEMV calculation and / or a second non-GEMV calculation, determining the insertion position of the GEMM calculation in the code according to the data flow diagram of the code, the first non-GEMV calculation is located between the starting row of the loop calculation and the position of the second GEMV calculation, and the second non-GEMV calculation is located between the position of the second GEMV calculation and the end row of the loop calculation; if the loop calculation does not include the first non-GEMV calculation and the second non-GEMV calculation, determining the insertion position as the starting row of the loop calculation.

[0014] Wherein, determining the insertion position of the GEMM calculation in the code according to the data flow graph of the code includes: if it is determined according to the data flow graph that the input parameters of the second GEMV calculation have data dependency on the first non-GEMV calculation, determining the insertion position as the next row of the end row of the loop calculation; if it is determined according to the data flow graph that the output parameters of the second GEMV calculation have data dependency on the second non-GEMV calculation, determining the insertion position as the previous row of the starting row; if it is determined according to the data flow graph that the input parameters of the second GEMV calculation have no data dependency on the first non-GEMV calculation, and the output parameters of the second GEMV calculation have no data dependency on the second non-GEMV calculation, determining the insertion position as the previous row of the starting row.

[0015] In the above scheme, the insertion position of the GEMM calculation will affect the calculation results of the non-GEMV calculation that has data dependency on the GEMV calculation. When determining the insertion position of the GEMM calculation, consider whether the GEMV calculation and the non-GEMV calculation have data dependency, so as to avoid merging the GEMV calculation and causing errors in the calculation results of the non-GEMV calculation that has data dependency on it.

[0016] In a possible implementation of the first aspect, the method further includes: if the specification of the matrix corresponding to the N second GEMV calculations meets the performance conditions corresponding to the matrix unit, determining to insert the GEMM calculation into the insertion position of the GEMM calculation in the code.

[0017] The above solution determines whether to merge based on performance conditions, which can avoid reducing the performance of the code after merging GEMV calculations.

[0018] In a possible implementation of the first aspect, the identification rule includes a GEMV function library, and determining the M first GEMV calculations in the code to be optimized based on the identification rule includes: traversing the code based on the GEMV function library to determine the M first GEMV calculations.

[0019] The above scheme can realize automatic recognition of GEMV calculations according to the GEMV calculations in the codes of each function recognition in the GEMV function library.

[0020] In a second aspect, the present application provides a code optimization device, which includes: an identification module, an analysis module, and a merging module.

[0021] The identification module is used to determine M first general matrix-vector multiplication GEMV calculations in the code to be optimized based on the identification rules, where M is a positive integer.

[0022] The analysis module is used to determine N second GEMV calculations among the M first GEMV calculations based on a merging rule, where N is a positive integer.

[0023] The merging module is used to insert the general matrix multiplication GEMM calculations corresponding to the N second GEMV calculations into the insertion position of the GEMM calculation in the code, and delete the N second GEMV calculations.

[0024] In a possible implementation of the second aspect, the merging rules include: the matrices of the N second GEMV calculations are the same, the step sizes of the vectors are the same, the control flow characteristics are the same, the input parameters of each second GEMV calculation have no data dependencies on the GEMV calculations before their respective positions, and the output parameters of the second GEMV calculations have no data dependencies on the GEMV calculations after their respective positions.

[0025] In a possible implementation of the second aspect, the merging rules include: the second GEMV calculation is located in a loop calculation of the code and the loop calculation does not include other GEMV calculations, the input parameters of the second GEMV calculation have no data dependencies on the GEMV calculation before the position of the second GEMV calculation, and the output parameters of the second GEMV calculation have no data dependencies on the GEMV calculation after the position of the second GEMV calculation.

[0026] In a possible implementation of the second aspect, the merging module is further used to: when N≠1, determine an insertion position of the GEMM calculation in the code according to a data flow graph of the code.

[0027] In a possible implementation of the second aspect, the merging module is specifically used to: if it is determined according to the data flow graph that there is data dependency between the output parameters of the N second GEMV calculations and the positions of the N second GEMV calculations for non-GEMV calculations, determine the insertion position to be the position of the first second GEMV calculation among the N second GEMV calculations; if there is data dependency between the input parameters of the N second GEMV calculations and the positions of the N second GEMV calculations according to the data flow graph for non-GEMV calculations, determine the insertion position to be the position of the Nth second GEMV calculation among the N second GEMV calculations; if there is no data dependency between the input parameters and output parameters of the N second GEMV calculations and the positions of the N second GEMV calculations according to the data flow graph for non-GEMV calculations, determine the insertion position to be the position of the first second GEMV calculation.

[0028] In a possible implementation of the second aspect, the merging module is also used for: when N=1 and the second GEMV calculation is located in a loop calculation of the code, if the loop calculation includes a first non-GEMV calculation and / or a second non-GEMV calculation, determining the insertion position of the GEMM calculation in the code according to the data flow diagram of the code, the first non-GEMV calculation is located between the starting row of the loop calculation and the position of the second GEMV calculation, and the second non-GEMV calculation is located between the position of the second GEMV calculation and the end row of the loop calculation; if the loop calculation does not include the first non-GEMV calculation and the second non-GEMV calculation, determining the insertion position as the starting row of the loop calculation.

[0029] In a possible implementation of the second aspect, the merging module is specifically used to: if it is determined according to the data flow graph that the input parameters of the second GEMV calculation have data dependencies on the first non-GEMV calculation, determine the insertion position to be the next row of the end row of the loop calculation; if it is determined according to the data flow graph that the output parameters of the second GEMV calculation have data dependencies on the second non-GEMV calculation, determine the insertion position to be the previous row of the starting row; if it is determined according to the data flow graph that the input parameters of the second GEMV calculation have no data dependencies on the first non-GEMV calculation, and the output parameters of the second GEMV calculation have no data dependencies on the second non-GEMV calculation, determine the insertion position to be the previous row of the starting row.

[0030] In a possible implementation of the second aspect, the merging module is further used to: if the specifications of the matrices corresponding to the N second GEMV calculations meet the performance conditions corresponding to the matrix units, determine to insert the GEMM calculation into the insertion position of the GEMM calculation in the code.

[0031] In a possible implementation of the second aspect, the identification rule includes a GEMV function library, and the identification module is specifically used to: traverse the code based on the GEMV function library to determine the M first GEMV calculations.

[0032] In a third aspect, the present application provides an optimization device. The optimization device includes a processor and a memory. The processor is used to execute a computer program stored in the memory to implement the code optimization method provided by the first aspect or any possible implementation of the first aspect.

[0033] In a fourth aspect, the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, it enables the computer to execute the code optimization method provided by the first aspect or any possible implementation of the first aspect.

[0034] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the code optimization method provided by the first aspect or any possible implementation of the first aspect.

[0035] Any of the above-mentioned devices, computer storage media or computer program products are used to execute the method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding schemes in the corresponding methods provided above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a logical schematic diagram of an optimization device provided in an embodiment of the present application;

[0037] Figure 2 is a flow chart of a code optimization method provided in an embodiment of the present application;

[0038] Figure 3 is a flow chart of a code optimization method provided in an embodiment of the present application;

[0039] Figure 4A is a schematic diagram of an optimization scenario provided in an embodiment of the present application;

[0040] Figure 4B is a schematic diagram of another optimization scenario provided in an embodiment of the present application;

[0041] Figure 5A It is a schematic diagram of calculation parameters of a GEMV function provided in an embodiment of the present application;

[0042] Figure 5B is a schematic diagram of a code snippet provided in an embodiment of the present application;

[0043] Fig. 6A and Figure 6B It is a schematic diagram of a scenario of merging GEMV calculation provided in an embodiment of the present application;

[0044] Figure 7 is a flow chart of a performance evaluation method provided in an embodiment of the present application;

[0045] Figure 8 is a flow chart of a performance evaluation method provided in an embodiment of the present application;

[0046] Fig. 9 9 is a schematic diagram of the structure of an optimization device 900 provided in an embodiment of the present application:

[0047] Fig.10 It is a structural diagram of an optimization device 1000 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0049] In the description of the embodiments of the present application, words such as "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a concrete way.

[0050] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "multiple" means two or more. For example, multiple systems refers to two or more systems, and multiple screen terminals refers to two or more screen terminals.

[0051] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprises", "has" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0052] Before introducing the embodiments of the present application, the nouns appearing in the embodiments of the present application are introduced below.

[0053] A matrix unit is a computing chip with strong computing power that can be used to implement matrix-related calculations, such as general matrix-vector multiplication GEMV calculations and general matrix multiplication GEMM calculations. The types of matrix units may include but are not limited to tensor cores (TensoreCore), scalable matrix units (SME), advanced matrix extensions (AMX), etc.

[0054] The general matrix-vector multiplication GEMV calculation refers to the outer product operation of the matrix and the vector.

[0055] General matrix multiplication GEMM calculation refers to the multiplication operation of matrices.

[0056] Both GEMV calculation and GEMM calculation can realize calculation functions by calling matrix units. However, compared with GEMM calculation, GEMV calculation cannot fully utilize the computing power of matrix units, which will waste the computing power of matrix units.

[0057] In this field, some codes usually include a large number of GEMV calculations. That is, during the code running process, the matrix unit is called by the GEMV calculation many times, and the computing power is wasted.

[0058] In the related art, GEMV calculations that can be merged in the code are usually manually identified, and the GEMV calculations that can be merged are rewritten as GEMM calculations. In this way, converting GEMV calculations into GEMM calculations can reduce the number of calls to the matrix unit, fully utilize the computing power of the matrix unit, and reduce the waste of computing power of the matrix unit.

[0059] Manual code optimization is labor-intensive and inefficient, and may also omit some GEMV calculations that can be merged. To this end, the present application provides a code optimization method that can solve the above problems.

[0060] Figure 1 1 is a logic diagram of an optimization device provided in an embodiment of the present application. The optimization device is used to execute the code optimization method provided in an embodiment of the present application.

[0061] like Figure 1As shown, the optimization device can identify the M first GEMV calculations in the code to be optimized through the identification rules, and determine the N second GEMV calculations in the M first GEMV calculations through the merging rules, and finally merge the N second GEMV calculations into a GEMM calculation and output it. In this solution, the optimization device can automatically identify and merge GEMV calculations of the code based on the pre-configured identification rules and merging rules. On the one hand, it can solve the problem of waste of computing power in calling matrix units for GEMV calculations in related technologies, and on the other hand, it can solve the problem of large workload and low efficiency caused by manual identification and merging in related technologies.

[0062] Combine the following Figure 1 and Figure 2 A code optimization method provided in an embodiment of the present application is specifically introduced.

[0063] Figure 2 1 is a flow chart of a code optimization method provided by an embodiment of the present application. Figure 2 As shown, the method may include the following S201-S203.

[0064] In S201, the optimization device determines M first general matrix-vector multiplication GEMV calculations in the code to be optimized based on the identification rule, where M is a positive integer.

[0065] In this step, if Figure 3 As shown, the user can input the code to be optimized into the optimization device to start the optimization of the code. The code may include system code or application code, and the embodiment of the present application does not specifically limit the type and writing language of the code.

[0066] Specifically, Figure 4A As shown, the optimization device may be located in the terminal device 401 on the user side, and the user may input the code to be optimized into the optimization device in the terminal device 401. In addition, in other embodiments, such as Figure 4B As shown, the optimization device can also be located in the server 402 of the cloud platform, and the server 402 communicates with the terminal device 401 on the user side. The user can send the code to be optimized to the optimization device in the server 402 through the terminal device 401. It should be noted that the embodiment of the present application does not specifically limit the number of servers in the cloud platform, the type and structure of each server, and the connection method between the servers.

[0067] In this step, after receiving the code input by the user, the optimization device determines the identification rule and the merging rule. After the user inputs the code to the optimization device, the user can configure the customized identification rule and / or merging rule to the optimization device. If the user does not configure the identification rule and / or merging rule, the optimization device can obtain the pre-built-in identification rule and / or merging rule.

[0068] In this step, the code to be optimized may include multiple functions, and the optimization device may process each function in the code in turn according to the identification rule and the merging rule to determine the second GEMV calculation that can be merged in each function in the code.

[0069] When identification starts, the optimization device may initialize l=1, where l represents the lth function in the code. The total number of functions in the code is L. The optimization device may obtain the total number of functions L from the user side, or may traverse the code before identification to determine the total number of functions.

[0070] Take the lth function in the code as an example, Figure 3 As shown, the optimization device can determine all first GEMV calculations in the lth function according to the identification rule, for example, M first GEMV calculations.

[0071] The identification rule may specifically include a GEMV function library. The names and parameters of multiple GEMV functions may be defined in the GEMV function library. The optimization device traverses each line of code in the lth function based on the GEMV function library, and compares the names and parameters of each GEMV function in the GEMV function library with each line of code in the lth function to determine the first GEMV calculation in the lth function.

[0072] Among them, taking the GEMV function "CALLGEMV(trans, m, n, alpha, a, lda, x, incx, beta, y, incy)" as an example, the record format of the GEMV function in the function library can be as follows: Figure 5A As shown. The data type of the name of the GEMV function is a string. The calculation parameters (arguments, Args) of the GEMV function include "trans, m, n, alpha, a, lda, x, incx, beta, y, incy". Among them, the data type of "trans" is a string, the data type of "m, n, ida, incx, incy" is an integer (int), the data type of "alpha, beta" is a single floating point (float), and the data type of "a, x, y" is an object (object).

[0073] If the optimization device identifies the first GEMV calculation from the lth function, the identification is successful, such as Figure 3 As shown, the information of the first GEMV calculation is recorded. The information of the first GEMV calculation may include but is not limited to the location, calculation parameters, control flow characteristics and flags of the first GEMV calculation.

[0074] If the optimization device does not identify the first GEMV calculation from the lth function, the identification fails. Figure 3 As shown, it is determined whether l is less than L. If l is less than L, l=l+1 is set, thereby identifying the first GEMV calculation in the next function based on the identification rule.

[0075] The calculation parameters of the first GEMV calculation may include input parameters and output parameters. When the GEMV calculation belongs to a loop calculation, the calculation parameters may also include an index, an upper bound, and a lower bound of the loop calculation.

[0076] The position of the first GEMV calculation can be represented by the line number of the first GEMV calculation in the code. Figure 5B Taking the first GEMV calculation (underlined) in the code snippet shown as an example, the first GEMV calculation is located at lines 3 and 4, with line 3 being the starting line of the GEMV calculation and line 4 being the ending line of the first GEMV calculation.

[0077] Among them, control flow features can include levels and parent nodes. Figure 5B Taking the GEMV calculation in the code snippet shown as an example, if the level starts from 0, the level of the GEMV calculation is 2; the parent node of the GEMV calculation is "DO".

[0078] The flag bit is used to indicate whether the first GEMV calculation is within a loop calculation.

[0079] In S202, the optimization device determines N second GEMV calculations among M first GEMV calculations based on a merging rule, where N is a positive integer.

[0080] In this step, if Figure 3 As shown, the optimization device can construct a data flow graph according to the position and calculation parameters of the first GEMV calculation in the function. First, based on the input parameters of each GEMV, the definition, assignment and calculation operations of the calculation parameters of other calculations before the position of each GEMV are analyzed, so as to determine the data flow information of the input parameters of each GEMV; and, based on the output parameters of each GEMV, the definition, assignment and calculation operations of the calculation parameters of other calculations after the position of each GEMV are analyzed to determine the data flow information of the output parameters of each GEMV. Among them, other calculations include GEMV calculations and non-GEMV calculations within the current function. Then, the data flow information of the input parameters of each GEMV and the data flow information of the output parameters of each GEMV are integrated to generate a data flow graph.

[0081] In this step, after constructing the data flow graph, the optimization device can determine the second GEMV calculation among the M first GEMV calculations according to the merging rule.

[0082] The merging rules may include: the matrices of the N second GEMV calculations are the same, the step lengths of the vectors are the same, and the control flow features are the same, which is used to determine multiple GEMV calculations outside the loop calculation and multiple GEMV calculations that can be merged within the loop calculation. The same control flow features may include the same level and the same parent node, and the same parent node means that the parent node has the same name and the same number of rows.

[0083] The merging rule may include: the second GEMV calculation is located in a loop calculation of the code and the loop calculation does not include other GEMV calculations, which is used to determine a GEMV calculation located in the loop calculation, for example Figure 6B GEMV calculations shown.

[0084] At the same time, in order to avoid the merged GEMV calculations affecting the calculation order of each GEMV calculation and causing errors in the calculation results, the merge rules may also include: the input parameters of the second GEMV calculation have no data dependency on the GEMV calculation before the second GEMV calculation, and the output parameters of the second GEMV calculation have no data dependency on the GEMV calculation after the second GEMV calculation. Among them, the input parameters of the second GEMV calculation have no data dependency means that the input parameters of the second GEMV calculation are not used by the GEMV calculation before its position, that is, the input parameters of the second GEMV calculation and the calculation parameters of the GEMV calculation before its position are not directly connected and not indirectly connected in the data flow graph. The output parameters of the second GEMV calculation have no data dependency means that the output parameters of the second GEMV calculation are not used by the GEMV calculation after its position, that is, the output parameters of the second GEMV calculation and the calculation parameters of the GEMV calculation after its position are not directly connected and not indirectly connected in the data flow graph.

[0085] In this step, first, Figure 3 As shown, the optimization device can analyze the data dependencies of the M first GEMV calculations according to the data flow graph, thereby determining multiple third GEMV calculations in the M first GEMV calculations. Among them, the input parameters of the third GEMV calculation have no data dependencies with the GEMV calculations before its position, and the output parameters of the third GEMV calculation have no data dependencies with the GEMV calculations after its position.

[0086] Then, if Figure 3 As shown, the optimization device can determine the matrix of the third GEMV calculation and the step size of the vector based on the input parameters of the third GEMV calculation, and by comparing the matrices, vector step sizes and control flow characteristics of multiple third GEMV calculations, N second GEMV calculations with the same matrix, the same vector step size and the same control flow characteristics in the multiple third GEMV calculations are determined.

[0087] In S203, the optimization device inserts the general matrix multiplication GEMM calculations corresponding to the N second GEMV calculations into the insertion position of the GEMM calculation in the code, and deletes the N second GEMV calculations.

[0088] Among them, N second GEMVs are calculated as a merged scenario, and the merged scenario may include the following: Fig. 6A The non-cyclic merge scenario shown and Figure 6B The loop merging scenario shown.

[0089] In this step, the optimization device can determine the insertion position of the GEMM calculation corresponding to the N second GEMV calculations in the code according to the data flow graph, insert the GEMM calculation at the insertion position of the GEMM calculation, and delete the N second GEMV calculations. The optimization device can determine the GEMM calculation corresponding to the N second GEMV calculations according to the GEMM function in the pre-configured GEMM function library.

[0090] Specifically, the optimization device considers the data dependency between GEMV calculation and non-GEMV calculation when determining the insertion position.

[0091] The output parameters of the second GEMV calculation have data dependency on the non-GEMV calculation following the second GEMV calculation, which means that the output parameters of the second GEMV calculation are used by the non-GEMV calculation following the second GEMV calculation, that is, the output parameters of the second GEMV calculation are directly or indirectly connected to the calculation parameters of the non-GEMV calculation following the second GEMV calculation in the data flow graph.

[0092] like Fig. 6A As shown in Figure b, the output parameter y3 of GEMV3 is used by the non-GEMV calculation after it, and the output parameter of GEMV3 has data dependency with the non-GEMV calculation. Figure 6B As shown in Figure a, GEMV7 is the only GEMV calculation in the for loop, and the output parameter yi of GEMV7 is used by the non-GEMV calculation after it. The output parameter of GEMV7 has data dependency with the non-GEMV calculation.

[0093] The input parameters of the second GEMV calculation have data dependencies on the non-GEMV calculation before the second GEMV calculation, and the input parameters of the second GEMV calculation are used by the non-GEMV calculation before the second GEMV calculation, that is, the input parameters of the second GEMV calculation are directly or indirectly connected to the calculation parameters of the non-GEMV calculation before the second GEMV calculation in the data flow graph.

[0094] like Fig. 6AAs shown in Figures c and d in , the input parameter x6 of GEMV6 is used by the non-GEMV calculation before it, and the input parameter of GEMV6 has data dependency with the non-GEMV calculation. Fig. 6A As shown in Figure c in , the input parameter x6 can be generated by the non-GEMV calculation through calculation operations; or, Fig. 6A As shown in Figure d in the figure, the input parameter x6 can be provided to this non-GEMV calculation by other calculations in the code. Figure 6B As shown in Figure b, GEMV8 is the only GEMV calculation in the for loop, the input parameter xi of GEMV8 is used by the non-GEMV calculation before it, and the output parameter of GEMV8 has data dependency with the non-GEMV calculation.

[0095] Specifically, the specific process of the optimization device determining the insertion position can be divided into two cases, and the two cases are specifically introduced below.

[0096] When N≠1, determining the insertion positions of the GEMM calculations corresponding to the N second GEMV calculations according to the data flow graph specifically includes the following three cases.

[0097] Case 1: If it is determined according to the data flow graph that the output parameters of any one of the N second GEMV calculations from the 1st second GEMV calculation to the N-1th second GEMV calculation or the N second GEMV calculations have data dependencies on the non-GEMV calculations between the position of the 1st second GEMV calculation and the position of the Nth second GEMV calculation, the insertion position is determined to be the position of the 1st second GEMV calculation.

[0098] by Fig. 6A Taking Figure a in as an example, GEMV3 is the first GEMV calculation of the merged scene. The output parameter y3 of GEMV3 is used by non-GEMV calculations, and the insertion position of the GEMM calculation is determined to be the position of GEMV3.

[0099] Case 2: If it is determined according to the data flow graph that there is data dependency between the non-GEMV calculations from the 2nd second GEMV calculation to any one of the Nth second GEMV calculations or the input parameters of the N second GEMV calculations and the position of the 1st second GEMV calculation and the position of the Nth second GEMV calculation, the insertion position is determined to be the position of the Nth GEMV calculation.

[0100] by Fig. 6A Taking Figures c and d in as an example, GEMV6 is the last GEMV calculation of the merged scenario, and the input parameter x6 of GEMV6 is used by non-GEMV calculations, which determines that the insertion position of the GEMM calculation is the position of GEMV6.

[0101] Case 3: If it is determined according to the data flow graph that any one of the N second GEMV calculations or the input parameters and output parameters of the N second GEMV calculations and the non-GEMV calculations between the position of the first second GEMV calculation and the position of the Nth second GEMV calculation have no data dependency, the insertion position is determined to be the position of the first second GEMV calculation.

[0102] When N=1, the second GEMV calculation is a loop calculation in the code. If the loop calculation does not include non-GEMV calculations, the insertion position is determined to be the starting line of the loop calculation. Figure 6B Taking Figure c in as an example, the for loop includes only GEMV9, and the insertion position of the GEMM calculation corresponding to the GEMV9 is the starting row of the for loop.

[0103] When N=1, the second GEMV calculation is a loop calculation in the code. If the loop calculation does not include non-GEMV calculations, the insertion position of the GEMV calculation is determined according to the data flow graph, which specifically includes the following three cases.

[0104] Case 1: If the input parameters of the N second GEMV calculations are determined to have data dependencies with the non-GEMV calculations between the start line and the position of the second GEMV calculations according to the data flow graph, the insertion position is determined to be the next line after the end line of the loop calculation. Figure 6B Taking Figure b in as an example, the for loop includes non-GEMV calculations and GEMV8, and the input parameter xi of GEMV8 is used by the non-GEMV calculation. The insertion position of the GEMM calculation corresponding to GEMV8 is the end line of the for loop.

[0105] Case 2: If the output parameters of the N second GEMV calculations are determined to have data dependencies with the positions of the N second GEMV calculations and the non-GEMV calculations between the end row, the insertion position is determined to be the row above the start row. Figure 6B Taking Figure a in as an example, the for loop includes GEMV7 and non-GEMV calculations, the input parameter xi of GEMV7 is used by the non-GEMV calculation, and the insertion position of the GEMM calculation corresponding to GEMV7 is the line above the starting line of the for loop.

[0106] Case 3: If the input parameters and output parameters of the N second GEMV calculations are determined according to the data flow diagram to have no data dependency on the positions and end rows of the N second GEMV calculations, the insertion position is determined to be the row above the start row.

[0107] After the optimization device merges the GEMV calculations in the lth function in the code to be optimized, Figure 3As shown, it can be determined whether l is less than L. If l is less than L, l=l+1 is set, and the next function is identified based on the identification rule. If l is equal to L, the optimized code can be output and fed back to the user. The optimization device can also output the optimized code in binary form.

[0108] The optimization device can also generate an optimization report of the code and provide feedback to the user. The optimization report can include the location of the GEMV calculations to be merged in the code to be optimized, and code optimization suggestions for GEMV calculations that cannot be merged due to data dependencies with GEMV calculations and / or non-GEMV calculations. Code optimization suggestions can include the location of GEMV calculations that cannot be merged and parameters with data dependencies.

[0109] In addition, before the optimization device executes S203, Figure 3 As shown, the performance analysis of the N second GEMV calculations can be performed, and whether to merge the N second GEMV calculations into a GEMM calculation can be determined based on the performance analysis results. If the optimization device determines not to merge the N second GEMV calculations, the code optimization suggestions for the corresponding second GEMV calculations can be fed back to the user. If the optimization device determines to merge the N second GEMV calculations, S203 is executed. The performance analysis process of the optimization device will be described later in conjunction with Figure 7 and Figure 8 I will not go into details here.

[0110] In the above scheme, the optimization device identifies the input code based on the pre-configured identification rules, and selects the second GEMV calculation that can be merged based on the pre-configured merging rules, thereby realizing the automatic identification and merging of the GEMV calculations in the code, and solving the problem of waste of matrix unit computing power. When determining the second GEMV calculation and determining the insertion position of the GEMM calculation, the data dependency relationship between the second GEMV calculation and other GEMV calculations and the data dependency relationship between non-GEMV calculations are considered, which can avoid the calculation data of the code being affected by the merged GEMV calculation, thereby causing errors in the calculation results of the code.

[0111] Figure 7 is a flow chart of a performance evaluation method provided in an embodiment of the present application. The method can be executed by an optimization device. Figure 1 As shown, the optimization device can perform performance analysis on the N second GEMV calculations before merging them into GEMM calculations, and determine whether to merge them according to the performance analysis results. The N second GEMV calculations refer to the corresponding second GEMV calculations in each of the above-mentioned merging scenarios.

[0112] Combine the following Figure 7 and Figure 8The performance evaluation method is introduced in detail.

[0113] like Figure 7 As shown, the method may include the following S701 and S702.

[0114] In S701, the optimization device determines the performance conditions of the matrix unit.

[0115] In this step, if Figure 8 As shown, the user can pre-configure the configuration file of the computing device for the optimization device. The optimization device may include a performance evaluation model, and the performance evaluation model may determine the performance conditions corresponding to the matrix unit based on the configuration file.

[0116] Alternatively, the user may pre-configure a performance evaluation model for the optimization device, wherein the performance evaluation model includes performance conditions corresponding to the matrix units.

[0117] Taking the matrix unit as SME as an example, the performance conditions may include the following conditions 1 and 2, for example.

[0118] Condition 1: The number of rows of the matrix is ​​greater than or equal to the first value X, the number of columns of the matrix is ​​greater than or equal to the second value Y, and the number of N second GEMV calculations is greater than or equal to N.

[0119] Condition 2: The product of X and Y is greater than or equal to Z.

[0120] In S702, the optimization device determines whether the N second GEMV calculations meet the performance conditions and obtains a performance analysis result.

[0121] In this step, the optimization device may calculate the number of rows of the matrix corresponding to the N second GEMV calculations, the number of columns of the matrix, and the number of N second GEMV calculations.

[0122] If the number of rows of the matrix corresponding to the N second GEMV calculations, the number of columns of the matrix, and the number of N second GEMV calculations meet the above conditions 1 or 2, the performance analysis result can be determined to be merged. If the number of rows of the matrix corresponding to the N second GEMV calculations, the number of columns of the matrix, and the number of N second GEMV calculations do not meet the above conditions 1 and 2, the performance analysis result can be determined to be not merged.

[0123] Based on S702, the optimization device may send the performance analysis result to the user and determine whether to merge the N GEMV calculations into a GEMM calculation according to the user's instruction.

[0124] Above Figure 7 In the illustrated embodiment, the optimization device performs performance analysis to avoid degradation of code performance after optimization.

[0125] based on Figure 2 and Figure 7 The method embodiment shown in the present application provides an optimization device. The optimization device is used to perform Figure 2 and Figure 7 The various steps in the method embodiment are shown.

[0126] Fig. 9 is a schematic diagram of the structure of an optimization device 900 provided in an embodiment of the present application. Fig. 9 As shown, the optimization device 900 may include an identification module 901 , an analysis module 902 and a merging module 903 .

[0127] The identification module 901 is used to determine M first general matrix-vector multiplication GEMV calculations in the code to be optimized based on the identification rule, where M is a positive integer.

[0128] The analysis module 902 is used to determine N second GEMV calculations among M first GEMV calculations based on a merging rule, where N is a positive integer.

[0129] The merging module 903 is used to insert the general matrix multiplication GEMM calculations corresponding to the N second GEMV calculations into the insertion position of the GEMM calculation in the code, and delete the N second GEMV calculations. Specifically, the merging module 903 can also be used to perform Figure 7 Steps in the illustrated method embodiment.

[0130] It should be noted that Fig. 9 The optimization device 900 provided in the embodiment shown in the figure only uses the division of the above-mentioned functional modules as an example to illustrate when executing the code optimization method. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Figure 2 The code optimization method embodiment shown and Figure 7 The performance evaluation method embodiments shown belong to the same concept, and their specific implementation processes can be found in the above method embodiments, which will not be repeated here.

[0131] Fig.10 It is a schematic diagram of the hardware structure of an optimization device 1000 provided in an embodiment of the present application.

[0132] The optimization device 1000 may be the terminal device 401 or the server 402. Fig.10, the optimization device 1000 includes a processor 1001, a memory 1002, a communication interface 1003 and a bus 1004, and the processor 1001, the memory 1002 and the communication interface 1003 are connected to each other through the bus 1004. The processor 1001, the memory 1002 and the communication interface 1003 can also be connected in other connection modes except the bus 1004.

[0133] Among them, the memory 1002 can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, optical storage, hard disk, etc.

[0134] The processor 1001 may be a general-purpose processor, which may be a processor that performs specific steps and / or operations by reading and executing the contents stored in a memory (e.g., the memory 1002). For example, the general-purpose processor may be a central processing unit (CPU). The processor 1001 may include at least one circuit to execute Figure 2 The illustrated embodiment provides all or part of the steps of the code optimization method.

[0135] The communication interface 1003 includes an input / output (I / O) interface, a physical interface, and a logical interface for interconnecting the devices inside the optimization device 1000, and an interface for interconnecting the optimization device 1000 with other devices (such as non-GEMV computing devices or user devices). The physical interface can be an Ethernet interface, a fiber optic interface, an ATM interface, etc. Specifically, the communication interface 1003 can communicate with the terminal device 401 to obtain the code to be optimized and the user's configuration.

[0136] The bus 1004 may be any type of communication bus for interconnecting the processor 1001 , the memory 1002 , and the communication interface 1003 , such as a system bus.

[0137] The above devices may be arranged on independent chips, or at least partially or completely on the same chip. Whether to arrange each device independently on different chips or to integrate them on one or more chips often depends on the needs of product design. The embodiments of the present application do not limit the specific implementation form of the above devices.

[0138] Fig.10 The optimization device 1000 shown is merely exemplary. During implementation, the optimization device 1000 may also include other components, which are not listed one by one in this article.

[0139] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0140] It is understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of description and are not used to limit the scope of the embodiments of the present application. It should be understood that in the embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0141] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present application should be included in the scope of protection of the present application.

Claims

1. A code optimization method, characterized in that: Applied to optimizing equipment, the method comprises: Determine M first general matrix-vector multiplication GEMV calculations in the code to be optimized based on the identification rule, where M is a positive integer; Determine N second GEMV calculations among the M first GEMV calculations based on a merging rule, where N is a positive integer; Insert the general matrix multiplication GEMM calculations corresponding to the N second GEMV calculations into the insertion position of the GEMM calculation in the code, and delete the N second GEMV calculations.

2. The method according to claim 1, characterized in that: The merging rules include: the matrices of the N second GEMV calculations are the same, the step sizes of the vectors are the same, the control flow characteristics are the same, and the input parameters of each second GEMV calculation have no data dependency on the GEMV calculations before their respective positions, and the output parameters of the second GEMV calculations have no data dependency on the GEMV calculations after their respective positions.

3. The method according to claim 1, characterized in that The merging rules include: the second GEMV calculation is located in the loop calculation of the code and the loop calculation does not include other GEMV calculations, the input parameters of the second GEMV calculation have no data dependency on the GEMV calculation before the position of the second GEMV calculation, and the output parameters of the second GEMV calculation have no data dependency on the GEMV calculation after the position of the second GEMV calculation.

4. The method according to claim 1 or 2, characterized in that: When the N≠1, the method further includes: An insertion position of the GEMM calculation in the code is determined according to a data flow graph of the code.

5. The method according to claim 4, characterized in that Determining the insertion position of the GEMM calculation in the code according to the data flow graph of the code includes: If it is determined according to the data flow graph that there is a data dependency between the output parameters of the N second GEMV calculations and the positions of the N second GEMV calculations, the insertion position is determined to be the position of the first second GEMV calculation among the N second GEMV calculations; If there is a data dependency between the non-GEMV calculations of the input parameters of the N second GEMV calculations and the positions of the N second GEMV calculations according to the data flow graph, determining the insertion position to be the position of the Nth second GEMV calculation among the N second GEMV calculations; If according to the data flow graph, there is no data dependency between the input parameters and output parameters of the N second GEMV calculations and the positions of the N second GEMV calculations for non-GEMV calculations, the insertion position is determined to be the position of the first second GEMV calculation.

6. The method according to claim 1 or 3, characterized in that: When N=1 and the second GEMV calculation is located within a loop calculation of the code, the method further includes: If the loop calculation includes a first non-GEMV calculation and / or a second non-GEMV calculation, the insertion position of the GEMV calculation in the code is determined according to the data flow graph of the code, the first non-GEMV calculation is located between the start line of the loop calculation and the position of the second GEMV calculation, and the second non-GEMV calculation is located between the position of the second GEMV calculation and the end line of the loop calculation; If the loop calculation does not include the first non-GEMV calculation and the second non-GEMV calculation, the insertion position is determined to be the starting row of the loop calculation.

7. The method according to claim 6, characterized in that Determining the insertion position of the GEMM calculation in the code according to the data flow graph of the code includes: If it is determined according to the data flow graph that the input parameter of the second GEMV calculation has data dependency with the first non-GEMV calculation, the insertion position is determined to be the next row of the end row of the loop calculation; If it is determined according to the data flow graph that the output parameter of the second GEMV calculation has data dependency with the second non-GEMV calculation, the insertion position is determined to be a row above the starting row; If it is determined according to the data flow graph that the input parameters of the second GEMV calculation have no data dependency on the first non-GEMV calculation, and the output parameters of the second GEMV calculation have no data dependency on the second non-GEMV calculation, the insertion position is determined to be the row above the starting row.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: If the specifications of the matrices corresponding to the N second GEMV calculations meet the performance conditions corresponding to the matrix unit, it is determined to insert the GEMM calculation into the insertion position of the GEMM calculation in the code.

9. The method according to any one of claims 1 to 8, characterized in that: The identification rule includes a GEMV function library, and the determination of the M first GEMV calculations in the code to be optimized based on the identification rule includes: The code is traversed based on the GEMV function library to determine the M first GEMV calculations.

10. A code optimization device, characterized in that: The device comprises: An identification module, used for determining M first general matrix-vector multiplication GEMV calculations in the code to be optimized based on the identification rule, where M is a positive integer; An analysis module, configured to determine N second GEMV calculations among the M first GEMV calculations based on a merging rule, where N is a positive integer; A merging module is used to insert the general matrix multiplication GEMM calculations corresponding to the N second GEMV calculations into the insertion position of the GEMM calculation in the code, and delete the N second GEMV calculations.

11. The device according to claim 10, characterized in that The merging rules include: the matrices of the N second GEMV calculations are the same, the step sizes of the vectors are the same, the control flow characteristics are the same, and the input parameters of each second GEMV calculation have no data dependency on the GEMV calculations before their respective positions, and the output parameters of the second GEMV calculations have no data dependency on the GEMV calculations after their respective positions.

12. The device according to claim 10, characterized in that The merging rules include: the second GEMV calculation is located in the loop calculation of the code and the loop calculation does not include other GEMV calculations, the input parameters of the second GEMV calculation have no data dependency on the GEMV calculation before the position of the second GEMV calculation, and the output parameters of the second GEMV calculation have no data dependency on the GEMV calculation after the position of the second GEMV calculation.

13. The device according to claim 10 or 11, characterized in that The merge module is also used to: When N≠1, the insertion position of the GEMM calculation in the code is determined according to the data flow graph of the code.

14. The device according to claim 13, characterized in that The merging module is specifically used for: If it is determined according to the data flow graph that there is a data dependency between the output parameters of the N second GEMV calculations and the positions of the N second GEMV calculations, the insertion position is determined to be the position of the first second GEMV calculation among the N second GEMV calculations; If there is a data dependency between the non-GEMV calculations of the input parameters of the N second GEMV calculations and the positions of the N second GEMV calculations according to the data flow graph, determining the insertion position to be the position of the Nth second GEMV calculation among the N second GEMV calculations; If according to the data flow graph, there is no data dependency between the input parameters and output parameters of the N second GEMV calculations and the positions of the N second GEMV calculations for non-GEMV calculations, the insertion position is determined to be the position of the first second GEMV calculation.

15. The device according to claim 10 or 12, characterized in that The merge module is also used to: When N=1 and the second GEMV calculation is within the loop calculation of the code, If the loop calculation includes a first non-GEMV calculation and / or a second non-GEMV calculation, the insertion position of the GEMV calculation in the code is determined according to the data flow graph of the code, the first non-GEMV calculation is located between the start line of the loop calculation and the position of the second GEMV calculation, and the second non-GEMV calculation is located between the position of the second GEMV calculation and the end line of the loop calculation; If the loop calculation does not include the first non-GEMV calculation and the second non-GEMV calculation, the insertion position is determined to be the starting row of the loop calculation.

16. The device according to claim 15, characterized in that The merging module is specifically used for: If it is determined according to the data flow graph that the input parameter of the second GEMV calculation has data dependency with the first non-GEMV calculation, the insertion position is determined to be the next row of the end row of the loop calculation; If it is determined according to the data flow graph that the output parameter of the second GEMV calculation has data dependency with the second non-GEMV calculation, the insertion position is determined to be a row above the starting row; If it is determined according to the data flow graph that the input parameters of the second GEMV calculation have no data dependency on the first non-GEMV calculation, and the output parameters of the second GEMV calculation have no data dependency on the second non-GEMV calculation, the insertion position is determined to be the row above the starting row.

17. The device according to any one of claims 10 to 16, characterized in that: The merge module is also used to: If the specifications of the matrices corresponding to the N second GEMV calculations meet the performance conditions corresponding to the matrix unit, it is determined to insert the GEMM calculation into the insertion position of the GEMM calculation in the code.

18. The device according to any one of claims 10 to 17, characterized in that: The identification rules include a GEMV function library, and the identification module is specifically used for: The code is traversed based on the GEMV function library to determine the M first GEMV calculations.

19. An optimization device, characterized in that: The optimization device comprises: a processor and a memory, wherein the processor is used to execute a computer program stored in the memory to implement the method according to any one of claims 1 to 9.

20. A computer-readable storage medium, characterized in that: The method comprises instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 9.

Images