Data operation device and operator performance improvement method thereof

By selecting appropriate template parameters based on prior knowledge and machine learning algorithm training model, the problem of template parameter selection complexity in Misaka programming is solved, and operator performance is improved and generalized.

CN119026709APending Publication Date: 2024-11-26SHANGHAI BIREN TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411155819.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In Misaka programming, selecting the appropriate template parameters to improve algorithm performance and efficiency is a complex problem, especially because the combination of template parameters is very complex and the prior art is difficult to effectively solve.

Method used

By providing multiple problem scales and template parameters, excluding some parameters based on prior knowledge, traversing the remaining parameters to obtain performance data, using machine learning algorithms such as random forest training models, obtaining the best model to select appropriate template parameters, and for different hardware configurations and operator types generate the best template parameters.

Benefits of technology

The operator performance is improved, ensuring that the model has good generalization of input data of different problem scales, and the best template parameters are generated according to the hardware configuration and operator type, effectively improving operator performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119026709A_ABST
    Figure CN119026709A_ABST
Patent Text Reader

Abstract

The invention provides a data operation device and an operator performance improving method thereof. A plurality of problem scales and a plurality of template parameters are provided. And excluding part of template parameters based on priori knowledge for each problem scale. And traversing the residual template parameters to obtain performance data so as to obtain a training set. A plurality of actual models are provided. And eliminating part of template parameters based on the problem scale of each actual model. And traversing the remaining template parameters to obtain performance data so as to obtain a verification set. And inputting the training set and the verification set into a machine learning algorithm to carry out model training so as to obtain an optimal model. In this way, the optimal model has good generalization performance when facing input data of different problem scales, optimal template parameters are generated corresponding to hardware configuration and operator types, and the operator performance is effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of integrated circuits, and in particular to a data computing device and a method for improving operator performance thereof. Background Art

[0002] When using template programming, you can improve the reusability and versatility of the code by using different template parameters. For example, the Tensor core operator can be programmed using template parameters such as tiling, stage, and splitk. Since the combination of template parameters is generally very complex, how to choose appropriate template parameters to improve the performance and efficiency of the algorithm is a very important topic. Summary of the invention

[0003] The present invention provides a data computing device and an operator performance improvement method thereof, which can effectively improve operator performance.

[0004] The operator performance improvement method of the present invention comprises the following steps. A plurality of problem scales and a plurality of template parameters are provided. For each problem scale, some template parameters are excluded based on prior knowledge. The remaining template parameters are traversed to obtain performance data to obtain a training set. A plurality of actual models are provided. Some template parameters of each actual model are excluded based on the problem scale of each actual model. The remaining template parameters are traversed to obtain performance data to obtain a verification set. The training set and the verification set are input into a machine learning algorithm to perform model training to obtain the best model.

[0005] In one embodiment of the present invention, the above-mentioned operator performance improvement method includes selecting an optimal model corresponding to the hardware configuration and the operator, and generating optimal template parameters through the selected optimal model based on the problem scale of the input data.

[0006] In one embodiment of the present invention, the above-mentioned operator performance improvement method includes generating corresponding optimal models for different hardware configurations and different operator types.

[0007] In one embodiment of the present invention, the above-mentioned operator performance improvement method includes excluding some template parameters according to the characteristics of the matrix and the hardware configuration.

[0008] In one embodiment of the present invention, the above-mentioned machine learning algorithm includes a random forest algorithm.

[0009] The present invention also provides a data operation device, including a storage circuit and a processor. The storage circuit stores multiple problem scales, multiple template parameters and multiple actual models. The processor is connected to the storage circuit. The processor is used to exclude part of the template parameters based on prior knowledge for each problem scale, traverse the remaining template parameters to obtain performance data to obtain as a training set, exclude part of the template parameters of each actual model based on the problem scale of each actual model, traverse the remaining template parameters to obtain performance data to obtain as a verification set, and input the training set and the verification set into a machine learning algorithm to perform model training to obtain the best model.

[0010] In one embodiment of the present invention, the processor corresponds to the hardware configuration and the operator to select the corresponding optimal model, and generates optimal template parameters through the selected optimal model based on the problem scale of the input data.

[0011] In one embodiment of the present invention, the processor generates corresponding optimal models for different hardware configurations and different operator types.

[0012] In one embodiment of the present invention, the processor excludes some template parameters according to the characteristics of the matrix and the hardware configuration.

[0013] In one embodiment of the present invention, the above-mentioned machine learning algorithm includes a random forest algorithm.

[0014] Based on the above, the embodiment of the present invention excludes some template parameters based on prior knowledge, traverses the remaining template parameters to obtain performance data to obtain a training set, excludes some template parameters of each actual model based on the problem scale of the actual model, traverses the remaining template parameters to obtain performance data to obtain a verification set, and inputs the training set and the verification set into the machine learning algorithm to perform model training to obtain the best model. In this way, the best model can have good generalization when facing input data of different problem scales, and generates the best template parameters corresponding to the hardware configuration and operator type, effectively improving the operator performance.

[0015] In order to make the above features and advantages of the present invention more clearly understood, embodiments are given below with reference to the accompanying drawings for detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 FIG. 4 is a circuit block diagram of a data processing device according to an embodiment of the present invention.

[0017] Figure 2 is a flowchart of a method for improving operator performance according to an embodiment of the present invention.

[0018] Figure 3 and Figure 4 Schematic diagram of block processing of matrix data.

[0019] Figure 5 is a flowchart of a method for improving operator performance according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the content of the present invention more understandable, the following embodiments are specifically cited as examples that the present invention can be implemented. In addition, wherever possible, components / members with the same reference numerals in the drawings and embodiments represent the same or similar components.

[0021] The term "coupled (or connected)" used throughout the specification (including claims) of this case may refer to any direct or indirect means of connection. For example, if the text describes a first device coupled (or connected) to a second device, it should be interpreted that the first device can be directly connected to the second device, or the first device can be indirectly connected to the second device through other devices or some connection means.

[0022] Figure 1 is a circuit block diagram of a data processing device 100 according to an embodiment of the present invention. Figure 1 The data processing device 100 shown includes a processor 102 and a storage circuit 104, and the processor 102 is coupled to the storage circuit 104. The processor 102 may include a central processing unit (CPU), a graphics processing unit (GPU) or other programmable general-purpose or special-purpose microprocessor, a digital signal processor (DSP), a programmable controller, an application specific integrated circuit (ASIC), a programmable logic device (PLD) or other similar devices or a combination of these devices. The storage circuit 104 is used to store data, which may include, for example, a volatile storage circuit and a non-volatile storage circuit. The volatile storage circuit is used to store data in a volatile manner. For example, the volatile storage circuit may include a random access memory (RAM) or a similar volatile storage medium. The non-volatile storage circuit is used to store data in a non-volatile manner. For example, the non-volatile storage circuit may include a read only memory (ROM), a solid state disk (SSD), a hard disk drive (HDD), or similar non-volatile storage media.

[0023] Figure 2 1 is a flowchart of an operator performance improvement method according to an embodiment of the present invention, wherein the operator may be, for example, a tensor core operator, but is not limited thereto, and may be, for example, a convolution operator or other operators. The processor 102 may provide multiple problem scales and multiple template parameters (step S202), and exclude some template parameters based on prior knowledge for each problem scale (step S204), wherein the template parameters may be stored in the storage circuit 104, and the problem scale may be, for example, a task scale, which may be the size of the data to be processed, such as the dimension or size of the matrix in matrix operations, but is not limited thereto. Further, the processor 102 may execute the code stored in the storage circuit 104 to randomly generate a large number of problem scales, and the method of excluding some template parameters based on prior knowledge may be, for example, excluding some template parameters based on the characteristics of the matrix and the hardware configuration. The characteristics of the matrix may include, for example, the dimension size and structure of the matrix, but is not limited thereto, and the hardware configuration may include, for example, the computing power of the configured CPU and GPU and the size and access speed of the configured memory, but is not limited thereto. For example, a larger matrix dimension, such as greater than 8192 dimensions, can be correspondingly selected to be applicable to a large matrix template parameter (i.e., excluding template parameters applicable to a small matrix), and a smaller matrix dimension, such as less than 1024 dimensions, can be correspondingly selected to be applicable to a small matrix template parameter (i.e., excluding template parameters applicable to a large matrix). In addition, hardware configuration can also be used as a basis for selecting template parameters at the same time. For example, a large matrix may correspondingly require more computing resources and more memory resources to perform operations. In the case of insufficient computing resources or memory resources, a simpler computing method can be selected to reduce the computational complexity. In the case of sufficient computing resources and memory resources, a more complex computing method can be selected to increase accuracy. In some embodiments, the sparsity or special structure of the matrix (such as a triangular matrix or a diagonal matrix, but not limited thereto) can also be used to exclude template parameters. For example, the template parameters can be excluded based on the number and position of non-zero elements in the matrix. For example, a diagonal matrix has non-zero elements only on the diagonal line, and the template parameters of the non-diagonal matrix can be excluded accordingly.

[0024] For example, Figure 3 It is a schematic diagram of the tile processing of matrix data, where Act is the output result of the neuron in the neural network, Wei is the weight in the neural network, and Out is the output data. Figure 3 The block processing can be expressed in the following pseudo code.

[0025] for(GMB_TILE:input_shape)

[0026] #pragma unroll

[0027] for(GIB_TILE:GMB_TILE)

[0028] #pragma unroll

[0029] for(mma_TILE:GIB_TILE){}

[0030] Among them, input_shape is the shape of the input data (matrix dimension), GMB_TILE is the block of the GPU main buffer (GPUMain Buffer), GIB_TILE is the block of the GPU input buffer (GPU Main Buffer), mma_TILE is the matrix multiply (Matrix Multiply-Accumulate) block, **_TILE has three parameters M, N, and K respectively, where M can represent the number of rows of matrix data, N can represent the number of columns of matrix data, and K can represent the shared dimension in matrix multiplication. For M, N, and K, a preset number of values ​​can be set as the value range. For example, 5 values ​​such as 64, 128, 256, 512, and 1024 can be set as the value range, but it is not limited to this. Figure 3 In the embodiment, as an example of excluding template parameters based on prior knowledge, in order to make the values ​​of the parameters M, N, K of the block smaller than the parameters M, N, K of the blocked matrix data, for example, since GIB_TILE is a block of GMB_TILE, the values ​​of the parameters M, N, K corresponding to GIB_TILE must be smaller than the values ​​of the parameters M, N, K corresponding to GMB_TILE. That is to say, when taking the values ​​of the parameters M, N, K of GIB_TILE, some parameter values ​​in the value range will be removed according to the values ​​of the parameters M, N, K of the corresponding GMB_TILE (removing the values ​​of the parameters M, N, K greater than or equal to the corresponding GMB_TILE).

[0031] For example, in Figure 4In the matrix multiplication operation of the embodiment, for example, the matrix data A and B can be divided into blocks, and the product results tb0-tb3 in the matrix data C can be calculated in a parallel computing manner. For example, the calculation processing of obtaining the matrix data tb0 based on the matrix data a0-a2 and b0-b2 and the calculation processing of obtaining the matrix data tb1 based on the matrix data a0-a2 and b3-b5 can be executed by different processors at the same time. For the template parameter split_k related to whether to divide the matrix data into blocks and the template parameter k_size of the block size, the value range of the template parameter split_k can be, for example, 0, 1, and the value range of the template parameter k_size can be, for example, 2, 3, 4, 5, 6. The template parameter split_k can determine whether to divide the matrix data into blocks based on the computing resources, for example, whether to divide the matrix data into blocks based on whether there are idle processors. When there are idle processors, the matrix data can be divided into blocks (split_k takes the value of 1) to perform parallel processing and improve computing efficiency. If there are no idle processors, the matrix data may not be divided into blocks (split_k takes the value of 0). In addition, the template parameter k_size can exclude some template parameters based on the matrix dimension of the matrix data A, for example, the template parameters with dimensions greater than or equal to the matrix data A can be removed. In addition, the above Figure 3 , Figure 4 The dimension of the matrix data in the embodiment, whether to divide the matrix data into blocks, and the template parameter k_size can all be used as examples of the problem scale.

[0032] After step S204, the processor 102 may traverse the remaining template parameters that have not been excluded for each problem scale to obtain performance data, for example, using different combinations of the remaining template parameters to run the operator through a hardware device (such as the processor 102) to obtain performance data, and use the input data and the corresponding performance data as a training set (step S206), where the performance data may include, for example, running time, accuracy, etc., but is not limited to this.

[0033] On the other hand, the operator performance improvement method also includes providing multiple actual models (step S208), and the actual model can be, for example, an open source model, which can be stored in the storage circuit 104. The processor 102 can exclude some template parameters of each actual model based on the problem scale of each actual model (step S210). For example, in matrix operations, similar to step S204, inappropriate template parameters can be excluded according to the dimension of the matrix, and template parameters can also be excluded according to hardware configuration, such as excluding inappropriate template parameters based on computing resources and memory resources. Similar to step S206, after excluding inappropriate template parameters, the processor 102 can traverse the remaining template parameters that are not excluded to obtain performance data, such as using different combinations of the remaining template parameters to run each actual model to obtain performance data, so as to obtain a verification set (step S212), and the verification set can include input data and corresponding performance data.

[0034] After obtaining the training set and the validation set, the processor 102 may input the training set and the validation set into a machine learning algorithm to perform machine learning model training and obtain an optimal model (step S214), wherein the machine learning algorithm may be, for example, a random forest algorithm, but is not limited thereto. By executing steps S202 to S214, a corresponding optimal model may be obtained for each combination of different hardware configurations and different operator types. It is worth noting that in other embodiments, steps S202 to S206 may also be executed after steps S208 to S212, or steps S208 to S212 may be executed after steps S202 to S206, instead of in steps S202 to S206. Figure 2 The embodiments are limited.

[0035] In actual operation, Figure 3 As shown, the processor 102 can select the corresponding best model from the storage circuit 104 corresponding to the hardware configuration and the operator (step S302), and generate the best template parameters through the selected best model based on the problem scale of the input data (step S304), and then use the best template parameters to run the operator (step S306). In this way, the best model constructed by steps S202 to S214 is used, and the corresponding best model is selected according to the hardware configuration and the operator type to generate the best template parameters, which can effectively ensure that the best model has good generalization when it comes to input data of different problem scales, and ensure that the best template parameters generated by the best model can effectively improve the operator performance.

[0036] In summary, the embodiment of the present invention excludes some template parameters based on prior knowledge, traverses the remaining template parameters to obtain performance data to obtain a training set, excludes some template parameters of each actual model based on the problem scale of the actual model, traverses the remaining template parameters to obtain performance data to obtain a verification set, and inputs the training set and the verification set into the machine learning algorithm to perform model training to obtain the best model. In this way, the best model can have good generalization when facing input data of different problem scales, and generates the best template parameters corresponding to the hardware configuration and operator type, effectively improving the operator performance.

[0037] Although the present invention has been disclosed as above by way of embodiments, it is not intended to limit the present invention. Any person skilled in the art may make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the claims.

Claims

1. A method for improving operator performance, characterized in that: include: Provide multiple problem sizes and multiple template parameters; Eliminate some template parameters based on a priori for each problem size; Traverse the remaining template parameters to obtain performance data to obtain a training set; Provide multiple realistic models; Eliminate some template parameters of each actual model based on the problem scale of each actual model; Traverse the remaining template parameters to obtain performance data to obtain a validation set; as well as The training set and the validation set are input into a machine learning algorithm to perform machine learning model training to obtain the best model.

2. The operator performance improvement method according to claim 1, characterized in that: Also includes: The best model corresponding to the hardware configuration and operator selection; as well as The optimal template parameters are generated by selecting the best model based on the problem size of the input data.

3. The operator performance improvement method according to claim 1, characterized in that: Also includes: Generate the corresponding optimal model for different hardware configurations and different operator types.

4. The operator performance improvement method according to claim 1, characterized in that: The step of excluding some template parameters based on a priori for each problem scale includes: Exclude some template parameters based on matrix characteristics and hardware configuration.

5. The method for improving operator performance according to claim 1, characterized in that: The machine learning algorithm includes a random forest algorithm.

6. A data computing device, characterized in that: include: A storage circuit for storing multiple problem sizes, multiple template parameters, and multiple actual models; as well as a processor, connected to the storage circuit, The processor is used to: Eliminate some template parameters based on a priori for each problem size; Traverse the remaining template parameters to obtain performance data to be used as a training set; Eliminate some template parameters of each actual model based on the problem scale of each actual model; Traverse the remaining template parameters to obtain performance data to be used as a validation set; as well as The training set and the validation set are input into a machine learning algorithm to perform machine learning model training to obtain the best model.

7. The data computing device according to claim 1, characterized in that: The processor selects an optimal model corresponding to the hardware configuration and the operator, and generates optimal template parameters through the selected optimal model based on the problem scale of the input data.

8. The data computing device according to claim 1, characterized in that: The processor generates corresponding optimal models for different hardware configurations and different operator types.

9. The data computing device according to claim 1, characterized in that: The processor excludes some template parameters according to the characteristics of the matrix and the hardware configuration.

10. The data computing device according to claim 1, characterized in that: The machine learning algorithm includes a random forest algorithm.