A code optimization method and optimization device

Abstract

A code optimization method and an optimization device are provided. The method is applied to the optimization device, and includes: determining M first general matrix vector multiplication (GEMV) calculations in a code to be optimized based on an identification rule; determining N second GEMV calculations in the M first GEMV calculations based on a merging rule; inserting general matrix multiplication (GEMM) calculations corresponding to the N second GEMV calculations into an insertion position of the GEMM calculations in the code, and deleting the N second GEMV calculations. This scheme can realize automatic identification and merging of GEMV calculations, and solve the problem of waste of matrix unit computing power.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular to a code optimization method and an optimization device. BACKGROUND

[0002] As a kind of computing chip, matrix unit has powerful computing capacity, and is widely used in high performance computing (HPC), image processing, artificial intelligence (AI) and other fields.

[0003] A large number of general matrix vector multiplication (GEMV) calculations are usually included in the code. In the code running, GEMV calculation calls matrix unit for operation, but GEMV calculation cannot fully utilize the computing capacity of matrix unit, and the computing power of matrix unit is wasted. SUMMARY

[0004] The present application provides a code optimization method and an optimization device, which can automatically identify and merge GEMV calculations that can be merged in the code into general matrix multiplication (GEMM) calculations, thereby solving the problem of waste of matrix unit computing power.

[0005] In a first aspect, the present application provides a code optimization method applied to an optimization device. The method comprises: determining M first general matrix vector multiplication (GEMV) calculations in a code to be optimized by the optimization device based on a recognition rule, M being a positive integer; determining N second GEMV calculations in the M first GEMV calculations based on a merging rule, N being a positive integer; inserting general matrix multiplication (GEMM) calculations corresponding to the N second GEMV calculations into an insertion position of the GEMM calculations in the code, and deleting the N second GEMV calculations.

[0006] The above scheme, the optimization device identifies the input code based on the pre-configured recognition rule, and filters out the second GEMV calculations that can be merged based on the pre-configured merging rule, realizes the automatic identification and merging of GEMV calculations in the code, and solves the problem of waste of matrix unit computing power.

[0007] In a possible implementation of the first aspect, the merging rule comprises: the matrices of the N second GEMV calculations are the same, the steps of the vectors are the same, the control flow characteristics are the same, the input parameters of each second GEMV calculation have no data dependency with the GEMV calculation before the respective position, and the output parameters of the second GEMV calculation have no data dependency with the GEMV calculation after the respective position.

[0008] The above scheme can determine the GEMV calculations that can be merged in the code through the merging rule, thereby merging the GEMV calculations into GEMM calculations, and reducing the number of times of calling the matrix unit to perform GEMV calculations. Moreover, the data dependency of the input parameters and the data dependency of the output parameters are considered in the merging rule, so that the calculation sequence of the GEMM calculation is not affected by the merging of the GEMV calculations, and the calculation result is not wrong.

[0009] In a possible implementation of the first aspect, the merging rule comprises that the second GEMV calculation is located in a loop calculation in the code and no other GEMV calculation is included in the loop calculation, the input parameter of the second GEMV calculation has no data dependency with a GEMV calculation before the position of the second GEMV calculation, and the output parameter of the second GEMV calculation has no data dependency with a GEMV calculation after the position of the second GEMV calculation.

[0010] The above scheme can determine the GEMV calculations that belong to the loop calculation in the code through the merging rule, thereby merging the GEMV calculations in the loop calculation into GEMM calculations, and reducing the number of times of calling the matrix unit in the loop calculation.

[0011] In a possible implementation of the first aspect, when the N≠1, the method further comprises determining an insertion position of the GEMM calculation in the code according to a data flow graph of the code.

[0012] The determining of the insertion position of the GEMM calculation in the code according to the data flow graph of the code comprises: if it is determined according to the data flow graph that the output parameter of the N second GEMV calculations has data dependency with a non-GEMV calculation between the position of the N second GEMV calculations, determining the insertion position as the position of the first second GEMV calculation in the N second GEMV calculations; if it is determined according to the data flow graph that the input parameter of the N second GEMV calculations has data dependency with a non-GEMV calculation between the position of the N second GEMV calculations, determining the insertion position as the position of the Nth second GEMV calculation in the N second GEMV calculations; and if it is determined according to the data flow graph that the input parameter and the output parameter of the N second GEMV calculations have no data dependency with a non-GEMV calculation between the position of the N second GEMV calculations, determining the insertion position as the position of the first second GEMV calculation.

[0013] In a possible implementation of the first aspect, when the N is 1 and the second GEMV calculation is located in a loop calculation of the code, the method further includes: if the loop calculation includes a first non-GEMV calculation and / or a second non-GEMV calculation, determining an insertion position of the GEMM calculation in the code according to a data flow graph of the code, the first non-GEMV calculation is located between a start line of the loop calculation and a location of the second GEMV calculation, and the second non-GEMV calculation is located between the location of the second GEMV calculation and an end line of the loop calculation; and if the loop calculation does not include the first non-GEMV calculation and the second non-GEMV calculation, determining the insertion position as the start line of the loop calculation.

[0014] In the method, the determining the insertion position of the GEMM calculation in the code according to the data flow graph of the code includes: if it is determined according to the data flow graph that an input parameter of the second GEMV calculation has data dependency with the first non-GEMV calculation, determining the insertion position as a next line of the end line of the loop calculation; if it is determined according to the data flow graph that an output parameter of the second GEMV calculation has data dependency with the second non-GEMV calculation, determining the insertion position as a previous line of the start line; and if it is determined according to the data flow graph that the input parameter of the second GEMV calculation has no data dependency with the first non-GEMV calculation and the output parameter of the second GEMV calculation has no data dependency with the second non-GEMV calculation, determining the insertion position as the previous line of the start line.

[0015] In the method, the determining the insertion position of the GEMM calculation in the code according to the data flow graph of the code includes: if it is determined according to the data flow graph that an input parameter of the second GEMV calculation has data dependency with the first non-GEMV calculation, determining the insertion position as a next line of the end line of the loop calculation; if it is determined according to the data flow graph that an output parameter of the second GEMV calculation has data dependency with the second non-GEMV calculation, determining the insertion position as a previous line of the start line; and if it is determined according to the data flow graph that the input parameter of the second GEMV calculation has no data dependency with the first non-GEMV calculation and the output parameter of the second GEMV calculation has no data dependency with the second non-GEMV calculation, determining the insertion position as the previous line of the start line.

[0016] In a possible implementation of the first aspect, the method further includes: if the specifications of the matrices corresponding to the N second GEMV calculations satisfy a performance condition corresponding to a matrix unit, determining to insert the GEMM calculation at the insertion position of the GEMM calculation in the code.

[0017] In the method, the determining the insertion position of the GEMM calculation in the code according to the data flow graph of the code includes: if it is determined according to the data flow graph that an input parameter of the second GEMV calculation has data dependency with the first non-GEMV calculation, determining the insertion position as a next line of the end line of the loop calculation; if it is determined according to the data flow graph that an output parameter of the second GEMV calculation has data dependency with the second non-GEMV calculation, determining the insertion position as a previous line of the start line; and if it is determined according to the data flow graph that the input parameter of the second GEMV calculation has no data dependency with the first non-GEMV calculation and the output parameter of the second GEMV calculation has no data dependency with the second non-GEMV calculation, determining the insertion position as the previous line of the start line.

[0018] In a possible implementation of the first aspect, the identification rule comprises a GEMV function library, and the determining the M first GEMV calculations in the code to be optimized comprises: traversing the code based on the GEMV function library to determine the M first GEMV calculations.

[0019] According to the above scheme, the GEMV calculation in the code can be identified according to each function in the GEMV function library, and automatic identification of the GEMV calculation can be implemented.

[0020] In the second aspect, the present application provides a code optimization device. The device comprises an identification module, an analysis module and a merging module.

[0021] The identification module is configured to determine M first general matrix vector multiplication (GEMV) calculations in the code to be optimized based on an identification rule, where M is a positive integer.

[0022] The analysis module is configured to determine N second GEMV calculations in the M first GEMV calculations based on a merging rule, where N is a positive integer.

[0023] The merging module is configured to insert general matrix multiplication (GEMM) calculations corresponding to the N second GEMV calculations into an insertion position of the GEMM calculations in the code, and delete the N second GEMV calculations.

[0024] In a possible implementation of the second aspect, the merging rule comprises: the matrices of the N second GEMV calculations are the same, the steps of the vectors are the same, the control flow characteristics are the same, the input parameters of each second GEMV calculation have no data dependency with a GEMV calculation before the respective position, and the output parameters of the second GEMV calculation have no data dependency with a GEMV calculation after the respective position.

[0025] In a possible implementation of the second aspect, the merging rule comprises: the second GEMV calculation is located in a loop calculation in the code and does not include other GEMV calculations in the loop calculation, the input parameters of the second GEMV calculation have no data dependency with a GEMV calculation before the position of the second GEMV calculation, and the output parameters of the second GEMV calculation have no data dependency with a GEMV calculation after the position of the second GEMV calculation.

[0026] In a possible implementation of the second aspect, the merging module is further configured to: when the N≠1, determine the insertion position of the GEMM calculation in the code according to a data flow graph of the code.

[0027] In a possible implementation of the second aspect, the merging module is specifically configured to: if it is determined according to the data flow graph that there is a data dependency between an output parameter of the N second GEMV calculations and a non-GEMV calculation between positions of the N second GEMV calculations, determining the insertion position as a position of a first second GEMV calculation in the N second GEMV calculations; if it is determined according to the data flow graph that there is a data dependency between an input parameter of the N second GEMV calculations and a non-GEMV calculation between positions of the N second GEMV calculations, determining the insertion position as a position of an Nth second GEMV calculation in the N second GEMV calculations; and if it is determined according to the data flow graph that there is no data dependency between the input parameter and the output parameter of the N second GEMV calculations and a non-GEMV calculation between positions of the N second GEMV calculations, determining the insertion position as the position of the first second GEMV calculation.

[0028] In a possible implementation of the second aspect, the merging module is further configured to: when the N = 1 and the second GEMV calculation is located in a loop calculation of the code, if the loop calculation includes a first non-GEMV calculation and / or a second non-GEMV calculation, determining an insertion position of the GEMM calculation in the code according to a data flow graph of the code, the first non-GEMV calculation is located between a starting line of the loop calculation and the position of the second GEMV calculation, and the second non-GEMV calculation is located between the position of the second GEMV calculation and an ending line of the loop calculation; and if the loop calculation does not include the first non-GEMV calculation and the second non-GEMV calculation, determining the insertion position as the starting line of the loop calculation.

[0029] In a possible implementation of the second aspect, the merging module is specifically configured to: if it is determined according to the data flow graph that there is a data dependency between an input parameter of the second GEMV calculation and the first non-GEMV calculation, determining the insertion position as a next line of the ending line of the loop calculation; if it is determined according to the data flow graph that there is a data dependency between an output parameter of the second GEMV calculation and the second non-GEMV calculation, determining the insertion position as a previous line of the starting line; and if it is determined according to the data flow graph that there is no data dependency between the input parameter of the second GEMV calculation and the first non-GEMV calculation and no data dependency between the output parameter of the second GEMV calculation and the second non-GEMV calculation, determining the insertion position as the previous line of the starting line.

[0030] In a possible implementation of the second aspect, the merging module is further configured to: if specifications of matrices corresponding to the N second GEMV calculations satisfy a performance condition corresponding to a matrix unit, determining to insert the GEMM calculation into the insertion position of the GEMM calculation in the code.

[0031] In a possible implementation of the second aspect, the identification rule comprises a GEMV function library, and the identification module is specifically configured to: traverse the code based on the GEMV function library to determine the M first GEMV calculations.

[0032] In a third aspect, the present application provides an optimization device. The optimization device comprises a processor and a memory. The processor is configured to execute a computer program stored in the memory to implement the code optimization method provided by the first aspect or any possible implementation of the first aspect.

[0033] In a fourth aspect, the present application provides a computer readable storage medium, which stores instructions, when the instructions are executed on a computer, cause the computer to execute the code optimization method provided by the first aspect or any possible implementation of the first aspect.

[0034] In a fifth aspect, the present application provides a computer program product comprising instructions, when the instructions are executed on a computer, cause the computer to execute the code optimization method provided by the first aspect or any possible implementation of the first aspect.

[0035] Any of the above provided devices or computer storage media or computer program products are used to execute the above provided method, and thus the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding method provided above, which will not be described here. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 is a logical schematic diagram of an optimization device provided by an embodiment of the present application;

[0037] Figure 2 is a flowchart of a code optimization method provided by an embodiment of the present application;

[0038] Figure 3 is a flowchart of a code optimization method provided by an embodiment of the present application;

[0039] Figure 4A is a schematic diagram of an optimization scenario provided by an embodiment of the present application;

[0040] Figure 4B is a schematic diagram of another optimization scenario provided by an embodiment of the present application;

[0041] Figure 5A is a schematic diagram of a GEMV function provided by an embodiment of the present application;

[0042] Figure 5B is a schematic diagram of a code segment provided by an embodiment of the present application;

[0043] Figure 6A and Figure 6B is a scenario diagram of merging GEMV calculation provided by an embodiment of the present application;

[0044] Figure 7 is a flowchart of a performance evaluation method provided by an embodiment of the present application;

[0045] Figure 8 is a flowchart of a performance evaluation method provided by an embodiment of the present application;

[0046] Figure 9 is a structural diagram of an optimization device 900 provided by an embodiment of the present application;

[0047] Figure 10 is a structural diagram of an optimization device 1000 provided by an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below with reference to the drawings.

[0049] In the description of the embodiments of the present application, the words “exemplary”, “for example”, or “for instance” are used to mean serving as an example, instance, or illustration. Any embodiment or design solution described as “exemplary”, “for example”, or “for instance” in the embodiments of the present application should not be interpreted as being more advantageous or superior than other embodiments or design solutions. In fact, the words “exemplary”, “for example”, or “for instance” are used to present related concepts in a specific manner.

[0050] In the description of the embodiments of the present application, the term “and / or” merely describes an association relationship of associated objects, and can represent three relationships, for example, A and / or B can represent three cases of A existing alone, B existing alone, and A and B existing simultaneously. In addition, unless otherwise specified, the term “multiple” means two or more. For example, multiple systems mean two or more systems, and multiple screen terminals mean two or more screen terminals.

[0051] In addition, the terms “first” and “second” are used only for descriptive purposes, and should not be construed as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined with “first” and “second” can explicitly or implicitly include one or more features. The terms “include”, “contain”, “have” and their variants mean “include but are not limited to”, unless otherwise specifically emphasized.

[0052] Before introducing the embodiments of the present application, the terms appearing in the embodiments of the present application are introduced first.

[0053] The matrix unit refers to a computing chip with strong computing power, which can be used to implement related calculations of matrices, such as general matrix vector multiplication (GEMV) calculation and general matrix multiplication (GEMM) calculation. The types of the matrix unit can include, but are not limited to, tensor computing core, scalable matrix unit (SME), advanced matrix extensions (AMX), and the like.

[0054] The GEMV calculation refers to the outer product operation of a matrix and a vector.

[0055] The GEMM calculation refers to the multiplication operation of a matrix and a matrix.

[0056] Both the GEMV calculation and the GEMM calculation can realize the calculation function by calling the matrix unit. However, compared with the GEMM calculation, the GEMV calculation cannot fully utilize the computing power of the matrix unit, and will waste the computing power of the matrix unit.

[0057] In the field, a large number of GEMV calculations are usually included in some codes. That is, during the running of the code, the matrix unit is called by the GEMV calculation a large number of times, and the waste of computing power is also large.

[0058] In the related art, the GEMV calculations that can be combined in the code are usually identified by artificial, and the GEMV calculations that can be combined are re-written as GEMM calculations. In this way, the conversion of the GEMV calculation into the GEMM calculation can reduce the number of calls of the matrix unit, fully utilize the computing power of the matrix unit, and reduce the waste of the computing power of the matrix unit.

[0059] By artificially optimizing the code, the workload is large and the efficiency is relatively low, and part of the GEMV calculations that can be combined may be missed. Therefore, the embodiments of the present application provide a code optimization method, which can solve the above problems.

[0060] Figure 1 is a logical schematic diagram of an optimization device provided by the embodiments of the present application. The optimization device is used to execute the code optimization method provided by the embodiments of the present application.

[0061] As Figure 1As shown, the optimization apparatus can identify M first GEMV calculations in the code to be optimized through an identification rule, and determine N second GEMV calculations in the M first GEMV calculations through a merging rule, and finally merge the N second GEMV calculations into a GEMM calculation and output. In this scheme, the optimization apparatus can automatically identify and merge GEMV calculations based on the preconfigured identification rule and merging rule, which can solve the problem of waste of computing power in related technologies due to the matrix element of GEMV calculation calling, and can solve the problem of large workload and low efficiency caused by manual identification and merging in related technologies.

[0062] The following will be described in combination with Figure 1 and Figure 2 The code optimization method provided in the embodiments of the present application will be described in detail.

[0063] Figure 2 is a flowchart of a code optimization method provided in the embodiments of the present application. As Figure 2 shown, the method can include S201-S203.

[0064] In S201, the optimization apparatus determines M first general matrix vector multiplication (GEMV) calculations in the code to be optimized based on an identification rule, where M is a positive integer.

[0065] In this step, as Figure 3 shown, the user can input the code to be optimized into the optimization apparatus to start the optimization of the code. The code can include system code or application code, and the type and writing language of the code are not limited in the embodiments of the present application.

[0066] Specifically, as Figure 4A shown, the optimization apparatus can be located in a terminal device 401 of the user side, and the user can input the code to be optimized into the optimization apparatus in the terminal device 401. In addition, in other embodiments, as Figure 4B shown, the optimization apparatus can also be located in a server 402 of a cloud platform, and the server 402 communicates with the terminal device 401 of the user side. The user can send the code to be optimized to the optimization apparatus in the server 402 through the terminal device 401. It should be noted that the number of servers in the cloud platform, the type, structure and connection mode of each server are not limited in the embodiments of the present application.

[0067] In this step, after receiving the code input by the user, the optimization apparatus determines the identification rule and the merging rule. After the user inputs the code into the optimization apparatus, the user can configure a custom identification rule and / or a merging rule for the optimization apparatus. If the user does not configure the identification rule and / or the merging rule, the optimization apparatus can obtain the prebuilt identification rule and / or the merging rule.

[0068] In this step, the code to be optimized can include multiple functions, and the optimization device can process each function in the code in turn according to the identification rule and the merging rule to determine the second GEMV calculation that can be merged in each function in the code.

[0069] When identification starts, the optimization device can initialize l = 1, where l represents the lth function in the code. The total number of functions in the code is L. The optimization device can obtain the total number of functions L from the user side, or determine the total number of functions in the code before identification.

[0070] Taking the lth function in the code as an example, as shown in Figure 3 , the optimization device can determine all first GEMV calculations in the lth function according to the identification rule, for example, M first GEMV calculations.

[0071] The identification rule can specifically include a GEMV function library. The GEMV function library can define the names and parameters of multiple GEMV functions. The optimization device traverses each line of code in the lth function based on the GEMV function library, compares the names and parameters of each GEMV function in the GEMV function library with each line of code in the lth function, and determines the first GEMV calculation in the lth function.

[0072] Taking the GEMV function "CALLGEMV(trans, m, n, alpha, a, lda, x, incx, beta, y, incy)" as an example, the record form of the GEMV function in the function library can be as shown in Figure 5A . The data type of the name of the GEMV function is string. The calculation parameters (arguments, Args) of the GEMV function include "trans, m, n, alpha, a, lda, x, incx, beta, y, incy". Among them, the data type of "trans" is string, the data type of "m, n, ida, incx, incy" is integer (int), the data type of "alpha, beta" is single floating point (float), and the data type of "a, x, y" is object.

[0073] If the optimization device identifies the first GEMV calculation from the lth function, that is, the identification is successful, as shown in Figure 3 , the information of the first GEMV calculation is recorded. The information of the first GEMV calculation can include but is not limited to the position of the first GEMV calculation, the calculation parameters, the control flow characteristics, and the flag bit, etc.

[0074] If the optimization device does not identify the first GEMV computation from the lth function, i.e., the identification fails, it is determined whether l is less than L, as shown in Figure 3 If l is less than L, l is set to l+1, and the first GEMV computation in the next function is identified based on the identification rule.

[0075] The computation parameters of the first GEMV computation can include input parameters and output parameters. When the GEMV computation belongs to a loop computation, the computation parameters can further include an index, an upper bound and a lower bound of the loop computation.

[0076] The position of the first GEMV computation can be represented by the line number of the first GEMV computation in the code. For example, the first GEMV computation (underlined) in the code snippet shown in Figure 5B has a position of line 3 and line 4, where line 3 is the start line of the GEMV computation and line 4 is the end line of the first GEMV computation.

[0077] The control flow feature can include a level and a parent node, etc. For example, the GEMV computation in the code snippet shown in Figure 5B has a level of 2 if the level is represented starting from 0, and the parent node of the GEMV computation is “DO”.

[0078] The flag is used to indicate whether the first GEMV computation is located within a loop computation.

[0079] In S202, the optimization device determines N second GEMV computations from the M first GEMV computations based on a merging rule, where N is a positive integer.

[0080] In this step, the optimization device can construct a data flow graph according to the position and the computation parameters of the first GEMV computation in the function, as shown in Figure 3 First, the data flow information of the input parameters of each GEMV is determined by analyzing the definition, assignment and computation of other computations before the position of each GEMV based on the input parameters of each GEMV, and the data flow information of the output parameters of each GEMV is determined by analyzing the definition, assignment and computation of other computations after the position of each GEMV based on the output parameters of each GEMV. The other computations include GEMV computations and non-GEMV computations in the current function. Then, the data flow graph is generated by integrating the data flow information of the input parameters of each GEMV and the data flow information of the output parameters of each GEMV.

[0081] In this step, the optimization device can determine the second GEMV computation from the M first GEMV computations according to the merging rule after constructing the data flow graph.

[0082] The merging rules can include: N second GEMV computations having the same matrix, the same vector step size, and the same control flow characteristics. These rules are used to identify multiple GEMV computations outside of loop computations and multiple GEMV computations inside loop computations that can be merged. The same control flow characteristics can include the same level and the same parent node. The same parent node means that the parent node has the same name and the same row number.

[0083] The merging rules may include: the second GEMV computation is located within a loop computation of the code, and the loop computation does not include other GEMV computations, used to determine a GEMV computation located within a loop computation, for example... Figure 6B The GEMV calculation is shown.

[0084] Furthermore, to avoid errors in the calculation results due to the merging of GEMV calculations affecting the calculation order of individual GEMV calculations, the merging rules may also include: the input parameters of the second GEMV calculation have no data dependency with the GEMV calculations preceding it, and the output parameters of the second GEMV calculation have no data dependency with the GEMV calculations following it. Specifically, "no data dependency on the input parameters of the second GEMV calculation" means that the input parameters of the second GEMV calculation are not used by the GEMV calculations preceding it; that is, the input parameters of the second GEMV calculation are neither directly connected nor indirectly connected to the calculation parameters of the GEMV calculations preceding it in the data flow graph. Similarly, "no data dependency on the output parameters of the second GEMV calculation" means that the output parameters of the second GEMV calculation are not used by the GEMV calculations following it; that is, the output parameters of the second GEMV calculation are neither directly connected nor indirectly connected to the calculation parameters of the GEMV calculations following it in the data flow graph.

[0085] In this step, firstly, as Figure 3 As shown, the optimization device can analyze the data dependencies of the M first GEMV computations based on the data flow graph, thereby determining multiple third GEMV computations among the M first GEMV computations. Specifically, the input parameters of the third GEMV computation have no data dependency on the GEMV computations preceding its position, and the output parameters of the third GEMV computation have no data dependency on the GEMV computations following its position.

[0086] Then, as Figure 3 As shown, the optimization device can determine the matrix of the third GEMV calculation and the step size of the vector based on the input parameters of the third GEMV calculation. By comparing the matrices, vector step sizes, and control flow characteristics of multiple third GEMV calculations, it can identify N second GEMV calculations that have the same matrix, the same vector step size, and the same control flow characteristics among multiple third GEMV calculations.

[0087] In S203, the optimization apparatus inserts the N second GEMV calculations into the insertion position of the GEMM calculation in the code and deletes the N second GEMV calculations.

[0088] The N second GEMV calculations are a merged scenario, which can include a non-loop merged scenario as shown in FIG. 2B and a loop merged scenario as shown in FIG. 2C. Figure 6A Figure 6B The N second GEMV calculations are a merged scenario, which can include a non-loop merged scenario as shown in FIG. 2B and a loop merged scenario as shown in FIG. 2C.

[0089] In this step, the optimization apparatus can determine the insertion position of the GEMM calculation corresponding to the N second GEMV calculations in the code according to the data flow graph, insert the GEMM calculation at the insertion position of the GEMM calculation, and delete the N second GEMV calculations. The optimization apparatus can determine the GEMM calculation corresponding to the N second GEMV calculations according to the GEMM functions in the pre-configured GEMM function library.

[0090] Specifically, the optimization apparatus considers the data dependency between the GEMV calculation and the non-GEMV calculation when determining the insertion position.

[0091] The output parameter of the second GEMV calculation has data dependency with the non-GEMV calculation after the position of the second GEMV calculation, that is, the output parameter of the second GEMV calculation is used by the non-GEMV calculation after the position of the second GEMV calculation, that is, the output parameter of the second GEMV calculation is directly or indirectly connected with the calculation parameter of the non-GEMV calculation after the position of the second GEMV calculation in the data flow graph.

[0092] As shown in FIG. 2B, the output parameter y3 of GEMV3 is used by the non-GEMV calculation after the position of GEMV3, and the output parameter of GEMV3 has data dependency with the non-GEMV calculation. Figure 6A As shown in FIG. 2A, GEMV7 is the only GEMV calculation in the for loop, the output parameter yi of GEMV7 is used by the non-GEMV calculation after the position of GEMV7, and the output parameter of GEMV7 has data dependency with the non-GEMV calculation. Figure 6B The input parameter of the second GEMV calculation has data dependency with the non-GEMV calculation before the position of the second GEMV calculation, that is, the input parameter of the second GEMV calculation is used by the non-GEMV calculation before the position of the second GEMV calculation, that is, the input parameter of the second GEMV calculation is directly or indirectly connected with the calculation parameter of the non-GEMV calculation before the position of the second GEMV calculation in the data flow graph.

[0093] As shown in FIG. 2B, the input parameter of GEMV3 is used by the non-GEMV calculation before the position of GEMV3, and the input parameter of GEMV3 has data dependency with the non-GEMV calculation.

[0094] Figure 6A ​​As shown in the c and d diagrams of FIG. 6, the input parameter x6 of the GEMV6 is used by a non-GEMV calculation before the position of the GEMV6, and the input parameter of the GEMV6 has data dependency with the non-GEMV calculation. Among them, as shown in the c diagram of FIG. 6, the input parameter x6 can be generated by the non-GEMV calculation through a calculation operation; or, as shown in the d diagram of FIG. 6, the input parameter x6 can be provided to the non-GEMV calculation by other calculations in the code. As shown in the b diagram of FIG. 6, the GEMV8 is the only GEMV calculation in the for loop, and the input parameter xi of the GEMV8 is used by a non-GEMV calculation before the position of the GEMV8, and the output parameter of the GEMV8 has data dependency with the non-GEMV calculation. Figure 6A Figure 6A Figure 6B As shown in the c and d diagrams of FIG. 6, the input parameter x6 of the GEMV6 is used by a non-GEMV calculation before the position of the GEMV6, and the input parameter of the GEMV6 has data dependency with the non-GEMV calculation. Among them, as shown in the c diagram of FIG. 6, the input parameter x6 can be generated by the non-GEMV calculation through a calculation operation; or, as shown in the d diagram of FIG. 6, the input parameter x6 can be provided to the non-GEMV calculation by other calculations in the code. As shown in the b diagram of FIG. 6, the GEMV8 is the only GEMV calculation in the for loop, and the input parameter xi of the GEMV8 is used by a non-GEMV calculation before the position of the GEMV8, and the output parameter of the GEMV8 has data dependency with the non-GEMV calculation.

[0095] Specifically, the specific process of determining the insertion position by the optimization device can be divided into two cases, which will be introduced in detail as follows.

[0096] When N≠1, the determination of the insertion position of the GEMM calculation corresponding to the N second GEMV calculations according to the data flow diagram specifically includes the following three cases.

[0097] Case 1, if according to the data flow diagram, the output parameter of any one or N of the first to the N-1 of the N second GEMV calculations, and the non-GEMV calculation between the position of the first second GEMV calculation and the position of the N second GEMV calculation has data dependency, it is determined that the insertion position is the position of the first second GEMV calculation.

[0098] Taking the a diagram of FIG. 6 as an example, the GEMV3 is the first GEMV calculation in the merging scenario, and the output parameter y3 of the GEMV3 is used by a non-GEMV calculation, and it is determined that the insertion position of the GEMM calculation is the position of the GEMV3. Figure 6A Case 2, if according to the data flow diagram, the input parameter of any one or N of the second to the N of the N second GEMV calculations, and the non-GEMV calculation between the position of the first second GEMV calculation and the position of the N second GEMV calculation has data dependency, it is determined that the insertion position is the position of the N GEMV calculation.

[0099] Taking the c and d diagrams of FIG. 6 as an example, the GEMV6 is the last GEMV calculation in the merging scenario, and the input parameter x6 of the GEMV6 is used by a non-GEMV calculation, and it is determined that the insertion position of the GEMM calculation is the position of the GEMV6.

[0100] Figure 6A

[0101] ​​​​Case 3, if according to the data flow graph, it is determined that there is no data dependency between the input parameters and the output parameters of any one of the N second GEMV calculations or the N second GEMV calculations, and the non-GEMV calculations between the position of the first second GEMV calculation and the position of the Nth second GEMV calculation, the insertion position is determined to be the position of the first second GEMV calculation.

[0102] When N = 1, the second GEMV calculation is a loop calculation in the code, if the loop calculation does not include non-GEMV calculations, the insertion position is determined to be the starting line of the loop calculation. For example, in the c graph in the code in Figure 6B For example, in the for loop in the c graph in the code in, the only GEMV9 is included in the for loop, and the insertion position of the GEMM calculation corresponding to the GEMV9 is the starting line of the for loop.

[0103] When N = 1, the second GEMV calculation is a loop calculation in the code, if the loop calculation does not include non-GEMV calculations, the insertion position of the GEMM calculation is determined according to the data flow graph, which includes the following three cases.

[0104] Case 1, if according to the data flow graph, it is determined that there is data dependency between the input parameters of the N second GEMV calculations and the non-GEMV calculations between the starting line and the position of the second GEMV calculation, the insertion position is determined to be the next line of the ending line of the loop calculation. For example, in the b graph in the code in Figure 6B For example, in the for loop in the b graph in the code in, non-GEMV calculations and GEMV8 are included in the for loop, and the input parameter xi of the GEMV8 is used by the non-GEMV calculations, and the insertion position of the GEMM calculation corresponding to the GEMV8 is the ending line of the for loop.

[0105] Case 2, if according to the data flow graph, it is determined that there is data dependency between the output parameters of the N second GEMV calculations and the non-GEMV calculations between the position of the N second GEMV calculations and the ending line, the insertion position is determined to be the previous line of the starting line. For example, in the a graph in the code in Figure 6B For example, in the for loop in the a graph in the code in, GEMV7 and non-GEMV calculations are included in the for loop, and the input parameter xi of the GEMV7 is used by the non-GEMV calculations, and the insertion position of the GEMM calculation corresponding to the GEMV7 is the previous line of the starting line of the for loop.

[0106] Case 3, if according to the data flow graph, it is determined that there is no data dependency between the input parameters and the output parameters of the N second GEMV calculations, and the non-GEMV calculations between the position of the N second GEMV calculations and the ending line, the insertion position is determined to be the previous line of the starting line.

[0107] After the optimization device merges the GEMV calculations in the 1th function in the code to be optimized, such as Figure 3As shown, it can be determined whether l is less than L. If l is less than L, let l = l + 1, identify the next function based on the identification rule. If l is equal to L, the optimized code can be output and fed back to the user. Wherein, the optimization device can also output the optimized code in binary form.

[0108] The optimization device can also generate an optimization report of the code and feed back to the user. Wherein, the optimization report can include the location of the merged GEMV calculation in the code to be optimized, and the code optimization suggestion of the GEMV calculation that cannot be merged due to data dependency with GEMV calculation and / or non-GEMV calculation. The code optimization suggestion can include the location of the GEMV calculation that cannot be merged and the parameter with data dependency.

[0109] In addition, before performing S203, the optimization device can perform performance analysis on the N second GEMV calculations, and determine whether to merge the N second GEMV calculations into GEMM calculations according to the performance analysis result. Wherein, if the optimization device determines not to merge the N second GEMV calculations, the code optimization suggestion of the corresponding second GEMV calculation can be fed back to the user. If the optimization device determines to merge the N second GEMV calculations, S203 is performed. Figure 3 As shown, the performance of the N second GEMV calculations can be analyzed, and whether to merge the N second GEMV calculations into GEMM calculations is determined according to the performance analysis result. Figure 7 Figure 8 The performance analysis process of the optimization device will be introduced in the following, which will not be repeated here.

[0110] The above scheme, the optimization device identifies the input code based on the pre-configured identification rule, and filters out the second GEMV calculations that can be merged based on the pre-configured merging rule, realizes the automatic identification and merging of the GEMV calculations in the code, and solves the problem of waste of matrix unit computing power. When determining the second GEMV calculation and determining the insertion position of the GEMM calculation, the data dependency relationship between the second GEMV calculation and other GEMV calculations and the data dependency relationship of the non-GEMV calculation are considered, which can avoid affecting the calculation data of the code due to merging GEMV calculations, thereby causing the calculation result of the code to be wrong.

[0111] Figure 7 is a flowchart of a performance evaluation method provided by an embodiment of the present application. The method can be performed by an optimization device. Wherein, as shown in Figure 1 The optimization device can perform performance analysis on the N second GEMV calculations before merging the N second GEMV calculations into GEMM calculations, and determine whether to merge according to the performance analysis result. Wherein, the N second GEMV calculations refer to the corresponding second GEMV calculations in the above-mentioned each merging scenario.

[0112] The following will be described in combination with Figure 7 Figure 8 ​​The performance evaluation method is specifically introduced.

[0113] As shown in the Figure 7 , the method can include S701 and S702 as follows.

[0114] In S701, the optimization device determines the performance condition of the matrix unit.

[0115] In this step, as shown in the Figure 8 , the user can pre-configure the configuration file of the computing device for the optimization device. The performance evaluation model can be included in the optimization device, and the performance evaluation model can determine the performance condition corresponding to the matrix unit based on the configuration file.

[0116] Alternatively, the user can pre-configure the performance evaluation model for the optimization device, and the performance evaluation model includes the performance condition corresponding to the matrix unit.

[0117] Taking the matrix unit as an example, the performance condition may, for example, include the following condition 1 and condition 2.

[0118] Condition 1: the number of rows of the matrix is greater than or equal to a first value X, the number of columns of the matrix is greater than or equal to a second value Y, and the number of N second GEMV calculations is greater than or equal to N.

[0119] Condition 2: the product of X and Y is greater than or equal to Z.

[0120] In S702, the optimization device determines whether the N second GEMV calculations satisfy the performance condition to obtain the performance analysis result.

[0121] In this step, the optimization device can determine the number of rows of the matrix corresponding to the N second GEMV calculations, the number of columns of the matrix, and the number of N second GEMV calculations.

[0122] If the number of rows of the matrix corresponding to the N second GEMV calculations, the number of columns of the matrix, and the number of N second GEMV calculations satisfy the above condition 1 or condition 2, it can be determined that the performance analysis result is merging. If the number of rows of the matrix corresponding to the N second GEMV calculations, the number of columns of the matrix, and the number of N second GEMV calculations do not satisfy the above condition 1 and condition 2, it can be determined that the performance analysis result is not merging.

[0123] Based on S702, the optimization device can send the performance analysis result to the user, and determine whether to merge the N GEMV calculations into GEMM calculations according to the user's indication.

[0124] As shown in the Figure 7 above embodiment, the performance analysis performed by the optimization device can avoid reducing the performance of the code after optimization.

[0125] Based on Figure 2 andFigure 7 According to the method embodiment shown, the embodiment of the present application provides an optimization device. The optimization device is used for executing Figure 2 and Figure 7 each step in the method embodiment shown.

[0126] Figure 9 is a structural schematic diagram of an optimization device 900 provided by the embodiment of the present application. The optimization device 900 is used for executing Figure 9 As shown, the optimization device 900 can include an identification module 901, an analysis module 902 and a merging module 903.

[0127] The identification module 901 is used for determining M first general matrix vector multiplication (GEMV) calculations in the code to be optimized based on an identification rule, where M is a positive integer.

[0128] The analysis module 902 is used for determining N second GEMV calculations in the M first GEMV calculations based on a merging rule, where N is a positive integer.

[0129] The merging module 903 is used for inserting general matrix multiplication (GEMM) calculations corresponding to the N second GEMV calculations into an insertion position of the GEMM calculations in the code, and deleting the N second GEMV calculations. Specifically, the merging module 903 can also be used for performing Figure 7 the steps in the method embodiment shown.

[0130] It should be noted that, Figure 9 The optimization device 900 provided by the embodiment shown performs the code optimization method, and only the above-mentioned division of each functional module is used as an example. In actual applications, the above-mentioned functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the optimization device 900 provided by the above-mentioned embodiment belongs to the same concept as the code optimization method embodiment shown in Figure 2 and the performance evaluation method embodiment shown in Figure 7 , and its specific implementation process can be seen from the above method embodiments, which will not be repeated here.

[0131] Figure 10 is a hardware structural schematic diagram of an optimization device 1000 provided by the embodiment of the present application.

[0132] The optimization device 1000 can be the terminal device 401 or the server 402 described above. Referring to Figure 10The optimization device 1000 includes a processor 1001, a memory 1002, a communication interface 1003, and a bus 1004. The processor 1001, the memory 1002, and the communication interface 1003 are connected to each other through the bus 1004. The processor 1001, the memory 1002, and the communication interface 1003 can also be connected in other ways other than the bus 1004.

[0133] The memory 1002 can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, optical storage, hard disk, etc.

[0134] The processor 1001 can be a general-purpose processor, which can be a processor that reads and executes the contents stored in the memory (such as the memory 1002) to perform specific steps and / or operations. For example, the general-purpose processor can be a central processing unit (CPU). The processor 1001 can include at least one circuit to perform Figure 2 The embodiments shown provide all or part of the steps of the code optimization method.

[0135] The communication interface 1003 includes an input / output (I / O) interface, a physical interface, and a logical interface for realizing the interconnection of devices inside the optimization device 1000, and an interface for realizing the interconnection of the optimization device 1000 with other devices (such as non-GEMV computing devices or user devices). The physical interface can be an Ethernet interface, a fiber interface, an ATM interface, etc. Specifically, the communication interface 1003 can communicate with the terminal device 401 to obtain the code to be optimized and the user's configuration.

[0136] The bus 1004 can be any type of communication bus for realizing the interconnection of the processor 1001, the memory 1002, and the communication interface 1003, such as a system bus.

[0137] The above devices can be respectively arranged on chips independent of each other, or at least partially or entirely arranged on the same chip. Whether to arrange the devices independently on different chips or to integrate the devices on one or more chips often depends on the needs of product design. The embodiments of the present application do not limit the specific implementation forms of the above devices.

[0138] Figure 10 The illustrated optimization device 1000 is merely exemplary, and in actual implementation, the optimization device 1000 can further include other components, which are not listed herein.

[0139] In the above embodiments, the implementation can be achieved entirely or partially by software, hardware, firmware, or any combination thereof. When implemented by software, the implementation can be achieved entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the flow or function described in the embodiments of the present application is entirely or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (such as floppy disk, hard disk, magnetic tape), optical media (such as DVD), or semiconductor media (such as solid state disk (SSD)), etc.

[0140] It can be understood that the various numbers involved in the embodiments of the present application are only for the convenience of differentiation, and do not limit the scope of the embodiments of the present application. It should be understood that in the embodiments of the present application, the size of the serial number of the above processes does not mean the order of execution, and the execution order of the processes should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0141] The above detailed description of the specific implementation is further detailed for the purpose of the application, technical solutions and beneficial effects, and it should be understood that the above description is only a specific implementation of the application and is not used to limit the protection scope of the application. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the application shall be included in the protection scope of the application.

Claims

1. A code optimization method, characterized in that, Applied to optimizing equipment, the method includes: Based on the identification rules, the M first general matrix vectors in the code to be optimized are multiplied by GEMV, where M is a positive integer; Based on the merging rules, N second GEMV calculations are determined from the M first GEMV calculations, where N is a positive integer; Insert the general matrix multiplication GEMM calculations corresponding to the N second GEMV calculations into the insertion position of the GEMM calculation in the code, and delete the N second GEMV calculations; The merging rules include: the matrices of the N second GEMV calculations are identical, the step size of the vectors is identical, the control flow characteristics are identical, and the input parameters of each second GEMV calculation have no data dependency with the GEMV calculations before its respective position, and the output parameters of the second GEMV calculations have no data dependency with the GEMV calculations after its respective position; or, The merging rules include: the second GEMV calculation is located within a loop calculation of the code and the loop calculation does not include other GEMV calculations; the input parameters of the second GEMV calculation have no data dependency on the GEMV calculations before the position of the second GEMV calculation; and the output parameters of the second GEMV calculation have no data dependency on the GEMV calculations after the position of the second GEMV calculation.

2. The method according to claim 1, characterized in that, When N≠1, the method further includes: The insertion position of the GEMM calculation in the code is determined based on the data flow graph of the code.

3. The method according to claim 2, characterized in that, Determining the insertion position of the GEMM calculation in the code based on the data flow graph of the code includes: If, according to the data flow graph, it is determined that there is a data dependency between the output parameters of the N second GEMV calculations and the non-GEMV calculations at the positions of the N second GEMV calculations, the insertion position is determined to be the position of the first second GEMV calculation among the N second GEMV calculations; If there is a data dependency between the input parameters of the N second GEMV calculations and the non-GEMV calculation positions of the N second GEMV calculations according to the data flow graph, the insertion position is determined to be the position of the Nth second GEMV calculation among the N second GEMV calculations; If, according to the data flow graph, the input and output parameters of the N second GEMV calculations have no data dependency on the non-GEMV calculations between the positions of the N second GEMV calculations, the insertion position is determined to be the position of the first second GEMV calculation.

4. The method according to claim 1, characterized in that, When N=1 and the second GEMV calculation is within a loop calculation of the code, the method further includes: If the loop calculation includes a first non-GEMV calculation and / or a second non-GEMV calculation, the insertion position of the GEMV calculation in the code is determined according to the data flow graph of the code. The first non-GEMV calculation is located between the starting line of the loop calculation and the position of the second GEMV calculation, and the second non-GEMV calculation is located between the position of the second GEMV calculation and the ending line of the loop calculation. If the loop calculation does not include the first non-GEMV calculation and the second non-GEMV calculation, the insertion position is determined as the starting row of the loop calculation.

5. The method according to claim 4, characterized in that, Determining the insertion position of the GEMM calculation in the code based on the data flow graph of the code includes: If, based on the data flow graph, it is determined that the input parameters of the second GEMV calculation have a data dependency with the first non-GEMV calculation, the insertion position is determined to be the line following the end line of the loop calculation; If, based on the data flow graph, it is determined that the output parameters of the second GEMV calculation have a data dependency with the second non-GEMV calculation, the insertion position is determined to be the row above the starting row; If, based on the data flow graph, it is determined that the input parameters of the second GEMV calculation have no data dependency on the first non-GEMV calculation, and the output parameters of the second GEMV calculation have no data dependency on the second non-GEMV calculation, then the insertion position is determined to be the row above the starting row.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: If the specifications of the matrices corresponding to the N second GEMV calculations meet the performance conditions corresponding to the matrix units, determine the insertion position of the GEMM calculation in the code.

7. The method according to any one of claims 1-5, characterized in that, The identification rules include the GEMV function library, and the determination of the M first GEMV calculations in the code to be optimized based on the identification rules includes: Based on the GEMV function library, the code is traversed to determine the M first GEMV calculations.

8. The method according to claim 6, characterized in that, The identification rules include the GEMV function library, and the determination of the M first GEMV calculations in the code to be optimized based on the identification rules includes: Based on the GEMV function library, the code is traversed to determine the M first GEMV calculations.

9. A code optimization device, characterized in that, The device includes: The identification module is used to determine the M first general matrix vectors in the code to be optimized based on the identification rules, where M is a positive integer; The analysis module is used to determine N second GEMV calculations among the M first GEMV calculations based on the merging rules, where N is a positive integer; The merging module is used to insert the general matrix multiplication GEMM calculations corresponding to the N second GEMV calculations into the insertion position of the GEMM calculation in the code, and delete the N second GEMV calculations. The merging rules include: the matrices of the N second GEMV calculations are identical, the step size of the vectors is identical, the control flow characteristics are identical, and the input parameters of each second GEMV calculation have no data dependency with the GEMV calculations before its respective position, and the output parameters of the second GEMV calculations have no data dependency with the GEMV calculations after its respective position; or, The merging rules include: the second GEMV calculation is located within a loop calculation of the code and the loop calculation does not include other GEMV calculations; the input parameters of the second GEMV calculation have no data dependency on the GEMV calculations before the position of the second GEMV calculation; and the output parameters of the second GEMV calculation have no data dependency on the GEMV calculations after the position of the second GEMV calculation.

10. The apparatus according to claim 9, characterized in that, The merging module is also used for: When N≠1, the insertion position of the GEMM calculation in the code is determined according to the data flow graph of the code.

11. The apparatus according to claim 10, characterized in that, The merging module is specifically used for: If, according to the data flow graph, it is determined that there is a data dependency between the output parameters of the N second GEMV calculations and the non-GEMV calculations at the positions of the N second GEMV calculations, the insertion position is determined to be the position of the first second GEMV calculation among the N second GEMV calculations; If there is a data dependency between the input parameters of the N second GEMV calculations and the non-GEMV calculation positions of the N second GEMV calculations according to the data flow graph, the insertion position is determined to be the position of the Nth second GEMV calculation among the N second GEMV calculations; If, according to the data flow graph, the input and output parameters of the N second GEMV calculations have no data dependency on the non-GEMV calculations between the positions of the N second GEMV calculations, the insertion position is determined to be the position of the first second GEMV calculation.

12. The apparatus according to claim 9, characterized in that, The merging module is also used for: When N=1 and the second GEMV calculation is within a loop calculation of the code, If the loop calculation includes a first non-GEMV calculation and / or a second non-GEMV calculation, the insertion position of the GEMV calculation in the code is determined according to the data flow graph of the code. The first non-GEMV calculation is located between the starting line of the loop calculation and the position of the second GEMV calculation, and the second non-GEMV calculation is located between the position of the second GEMV calculation and the ending line of the loop calculation. If the loop calculation does not include the first non-GEMV calculation and the second non-GEMV calculation, the insertion position is determined as the starting row of the loop calculation.

13. The apparatus according to claim 12, characterized in that, The merging module is specifically used for: If, based on the data flow graph, it is determined that the input parameters of the second GEMV calculation have a data dependency with the first non-GEMV calculation, the insertion position is determined to be the line following the end line of the loop calculation; If, based on the data flow graph, it is determined that the output parameters of the second GEMV calculation have a data dependency with the second non-GEMV calculation, the insertion position is determined to be the row above the starting row; If, based on the data flow graph, it is determined that the input parameters of the second GEMV calculation have no data dependency on the first non-GEMV calculation, and the output parameters of the second GEMV calculation have no data dependency on the second non-GEMV calculation, then the insertion position is determined to be the row above the starting row.

14. The apparatus according to any one of claims 9-13, characterized in that, The merging module is also used for: If the specifications of the matrices corresponding to the N second GEMV calculations meet the performance conditions corresponding to the matrix units, determine the insertion position of the GEMM calculation in the code.

15. The apparatus according to any one of claims 9-13, characterized in that, The identification rules include the GEMV function library, and the identification module is specifically used for: Based on the GEMV function library, the code is traversed to determine the M first GEMV calculations.

16. The apparatus according to claim 14, characterized in that, The identification rules include the GEMV function library, and the identification module is specifically used for: Based on the GEMV function library, the code is traversed to determine the M first GEMV calculations.

17. An optimization device, characterized in that, The optimization device includes a processor and a memory, the processor being configured to execute a computer program stored in the memory to implement the method of any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 8.

Images