Matrix multiplication parameter acquisition method and device, model training method and device, equipment and medium

Abstract

The application relates to a matrix multiplication parameter acquisition method, a model training method and device, electronic equipment and a computer readable storage medium, and belongs to the computer field. The matrix multiplication parameter acquisition method comprises the following steps: acquiring the matrix dimension to be subjected to matrix multiplication operation, wherein the matrix dimension comprises three key parameters, i.e. the row number and the column number of an input matrix and an output matrix; inputting the matrix dimension into a trained target classification model to output a matrix multiplication five-tuple parameter, wherein the matrix multiplication five-tuple parameter comprises three block parameters and two calculation parameters, the three block parameters correspond to the row number and the column number of a block input matrix and a block output matrix subjected to single block operation by each thread block, and the two calculation parameters correspond to the row number and the column number of single operation by each thread group in the thread block. According to the application, the model is used to predict the matrix multiplication five-tuple parameter, so that the optimal block parameter can be quickly determined, and the calculation performance of the processor for matrix multiplication is improved.

Description

Technical Field

[0001] This application belongs to the field of computer science, and specifically relates to a method for obtaining matrix multiplication parameters, a model training method, an apparatus, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Matrix multiplication is a core computational operation in the training and inference processes of deep learning models (such as convolutional neural networks and Transformers), and it is also a fundamental operator for many key tasks in scientific computing (such as solving linear algebra problems and simulating fluid dynamics). Its computational efficiency directly affects overall performance. Graphics Processing Units (GPUs), with their highly parallel computing architecture, have become the mainstream hardware platform for accelerating matrix operations. In matrix multiplication, M, N, and K (where M, N, and K are all integers greater than or equal to 1) are three key dimensional parameters: matrix A has a dimension of M×K (M rows and K columns), matrix B has a dimension of K×N (K rows and N columns), and their multiplication yields matrix C, which has a dimension of M×N (M rows and N columns).

[0003] In a GPU, a dedicated hardware unit for accelerating such tensor operations is called a Tensor Core. Each Tensor Core can complete a small block of matrix multiplication and addition operations within a single clock cycle (typically 16×16×16, where M, N, and K are all 16). To fully utilize its performance, large-scale matrix multiplications are typically decomposed into multiple smaller blocks to match the efficient processing specifications of the Tensor Cores. Although block decomposition can significantly improve the computational performance of matrix multiplication, determining the optimal block size remains a key challenge in practical implementation. The performance of matrix multiplication is limited by several hardware resource factors, including shared memory capacity, the number of registers, and the size and number of thread blocks. The block size needs to strike a balance between computational density and memory overhead: excessively large blocks may lead to insufficient shared memory or register resources; excessively small blocks cannot fully utilize the GPU's parallel computing capabilities, thus reducing overall efficiency. Summary of the Invention

[0004] Therefore, the purpose of this application is to provide a method for obtaining matrix multiplication parameters, a model training method, an apparatus, an electronic device, and a computer-readable storage medium to quickly determine the optimal block parameters, thereby improving the computational performance of the processor in matrix multiplication.

[0005] The embodiments of this application are implemented as follows:

[0006] In a first aspect, embodiments of this application provide a method for obtaining matrix multiplication parameters, comprising: obtaining the matrix dimension to be multiplied, the matrix dimension including: three key parameters corresponding to the number of rows and columns of the input matrix and the output matrix; inputting the matrix dimension into a trained target classification model, and outputting matrix multiplication quintuple parameters, wherein the matrix multiplication quintuple parameters include: three block parameters and two calculation parameters, the three block parameters corresponding to the number of rows and columns of the block input matrix and the block output matrix for a single block operation in each thread block, and the two calculation parameters corresponding to the number of rows and columns of each thread group in a single operation within the thread block.

[0007] In the above embodiments, by inputting the M, N, and K values ​​(matrix dimensions) of the matrix into the classification model, the model can predict the optimal matrix multiplication quintuple parameters, namely Bm, Bn, Bk, Tm, and Tn. Subsequently, matrix multiplication operations can be performed using these parameters, thereby fully utilizing the capabilities of the GPU hardware and optimizing the performance of matrix multiplication. Compared to the current method of relying on software engineers' experience to repeatedly adjust these parameters, this application can quickly determine the optimal block parameters and reduce manual costs.

[0008] In one possible implementation of the first aspect embodiment, the method further includes: acquiring a training dataset, the training dataset including multiple training data, each training data including: the matrix dimension of the matrix multiplication operation, the optimal matrix multiplication quintuple parameters, the thread block and grid parameters used in the matrix multiplication operation; and using the training dataset to train a preset classification model to obtain a trained classification model.

[0009] In the above embodiments, a classification model is trained by acquiring training data that includes the matrix dimension of the matrix multiplication operation, the optimal matrix multiplication quintuple parameters, and the thread block and grid parameters used in the matrix multiplication operation. This allows the model to learn the correspondence between the matrix dimension and the optimal matrix multiplication quintuple parameters. Consequently, when the M, N, and K values ​​(matrix dimension) of the matrix are input into the classification model, the model can predict the optimal matrix multiplication quintuple parameters. Furthermore, during training, the thread block and grid parameters used in the matrix multiplication operation are also considered, ensuring that the matrix multiplication quintuple parameters predicted by the model reflect the optimal combination under the influence of multiple factors such as shared memory, registers, thread collaboration, and memory access.

[0010] In one possible implementation of the first aspect embodiment, obtaining multiple training data points from the training dataset includes: obtaining multiple sets of data, each set of data including the matrix dimension of the matrix multiplication operation and the corresponding optimal matrix multiplication quintuple parameters; determining the thread block and grid parameters used in the matrix multiplication operation corresponding to each set of data based on the thread layout of each set of data and the matrix multiplication kernel function; and combining each set of data with the thread block and grid parameters used in the corresponding matrix multiplication operation to obtain multiple training data points from the training dataset.

[0011] In the above embodiments, multiple sets of data, including the matrix dimensions of matrix multiplication operations and the corresponding optimal matrix multiplication quintuple parameters, can be obtained manually. Then, by combining the thread layout of the matrix multiplication kernel function, the thread block and grid parameters used in the matrix multiplication operation can be determined. After adding the thread block and grid parameters used in the matrix multiplication operation to this data, the training data required for training can be obtained. Since the thread block and grid parameters used in the matrix multiplication operation are also considered, the matrix multiplication quintuple parameters predicted by the subsequent model reflect the optimal combination under the influence of multiple factors such as shared memory, registers, thread cooperation, and memory access.

[0012] In one possible implementation of the first aspect embodiment, the method further includes: after obtaining a trained classification model, performing matrix multiplication using the matrix multiplication quintuple parameters predicted by the trained classification model according to a specified matrix dimension, and obtaining a first operating index during the operation; verifying the prediction effect of the trained classification model based on the first operating index; and obtaining the target classification model when the verification is successful.

[0013] In the above embodiments, after obtaining the trained classification model, matrix multiplication is performed using the matrix multiplication quintuple parameters predicted by the model, and the first running index during the operation is obtained to verify the prediction effect of the model, so as to improve the accuracy of the model prediction. This application introduces a performance feedback closed-loop calibration mechanism to form a "prediction → execution → feedback" closed loop, thereby realizing the continuous evolution of the model.

[0014] In one possible implementation of the first aspect embodiment, the method further includes: obtaining the target matrix dimension in the training dataset that is closest to the specified matrix dimension; performing matrix multiplication using the optimal matrix multiplication quintuple parameter corresponding to the target matrix dimension, and obtaining a second running index during the operation; correspondingly, verifying the prediction effect of the trained classification model based on the first running index, including: verifying the prediction effect of the trained classification model based on the first running index and the second running index.

[0015] In the above embodiments, by selecting the target matrix dimension that is closest to the specified matrix dimension, and obtaining the second running index when performing matrix multiplication operation with the optimal matrix multiplication quintuple parameters corresponding to the target matrix dimension, the prediction effect of the trained classification model can be verified based on the first running index and the second running index, thereby improving accuracy. For example, the second running index can be used as a benchmark to compare the difference between the first running index and the second running index. If the difference between the two is too large (e.g., greater than 10%), it indicates that the verification has failed.

[0016] In one possible implementation of the first aspect embodiment, the first operating metric and the second operating metric include at least one parameter selected from processor utilization, storage bandwidth utilization, and execution time.

[0017] In the above embodiments, at least one parameter among processor utilization, storage bandwidth utilization, and execution time is obtained to determine whether it is optimal from multiple dimensions, so as to verify the prediction effect of the classification model and so that the parameters predicted by the final model are optimal.

[0018] In one possible implementation of the first aspect embodiment, the three key parameters are M, N, and K, where M, N, and K are all integers greater than or equal to 1; obtaining the target matrix dimension in the training dataset that is closest to the specified matrix dimension includes: obtaining the first difference between the M parameter in the specified matrix dimension and the M parameter in each matrix dimension of the training dataset; obtaining the second difference between the N parameter in the specified matrix dimension and the N parameter in each matrix dimension of the training dataset; obtaining the third difference between the K parameter in the specified matrix dimension and the K parameter in each matrix dimension of the training dataset; for each matrix dimension, obtaining the mean of the first difference, the second difference, and the third difference corresponding to that matrix dimension; wherein, the matrix dimension with the smallest mean is the target matrix dimension.

[0019] In the above embodiment, by obtaining the differences between MNK and the specified matrix dimension MNK in each matrix dimension of the training dataset, and for each matrix dimension, obtaining the mean of the first difference, the second difference, and the third difference corresponding to that matrix dimension, and selecting the matrix dimension with the smallest mean as the target matrix dimension, the target matrix dimension that is closest to the specified matrix dimension can be found quickly.

[0020] In one possible implementation of the first aspect embodiment, obtaining the target matrix dimension in the training dataset that is closest to the specified matrix dimension includes: obtaining the first product of three key parameters in the specified matrix dimension, and obtaining the second product of three key parameters in each matrix dimension in the training dataset; obtaining the target second product among each second product that is closest to the first product, wherein the matrix dimension corresponding to the target second product is the target matrix dimension.

[0021] In the above embodiment, by obtaining the product of three key parameters, that is, calculating the product of M×N×K, and selecting the second product with the smallest difference from the first product, the matrix dimension corresponding to the second product with the smallest difference from the first product is the target matrix dimension. In this way, the target matrix dimension that is closest to the specified matrix dimension can be found quickly.

[0022] In a possible implementation of the first aspect embodiment, the method further includes: when the trained classification model fails the verification, training the trained classification model again with more training data to obtain the target classification model; wherein, when the amount of training data reaches a preset threshold or the time since the previous training reaches a preset time interval, a new round of training begins.

[0023] In the above embodiments, to improve the accuracy of model predictions, a performance feedback closed-loop calibration mechanism is introduced to form a "prediction → execution → feedback → learning" closed loop, enabling continuous model evolution. Furthermore, to avoid the overhead of retraining the model every time the training dataset is updated, the model can be retrained only when a certain amount of new data is added or a preset time interval is reached.

[0024] Secondly, this application also provides a model training method, including: acquiring a training dataset, the training dataset including multiple training data, each training data including: the matrix dimension of the matrix multiplication operation, the optimal matrix multiplication quintuple parameters, the thread block and grid parameters used in the matrix multiplication operation; and using the training dataset to train a preset classification model to obtain a trained classification model.

[0025] In one possible implementation of the second aspect embodiment, the method further includes: performing matrix multiplication using the matrix multiplication quintuple parameters predicted by the classification model based on a specified matrix dimension, and obtaining a first operating index during the operation; and verifying the prediction performance of the trained classification model based on the first operating index.

[0026] In one possible implementation of the second aspect embodiment, the method further includes: obtaining the target matrix dimension in the training dataset that is closest to the specified matrix dimension; performing matrix multiplication using the optimal matrix multiplication quintuple parameter corresponding to the target matrix dimension, and obtaining a second running index during the operation; correspondingly, verifying the prediction effect of the trained classification model based on the first running index, including: verifying the prediction effect of the trained classification model based on the first running index and the second running index.

[0027] Thirdly, embodiments of this application also provide a matrix multiplication parameter acquisition device, including: a first acquisition module and a prediction module; the first acquisition module is used to acquire the matrix dimension to be multiplied, the matrix dimension including: three key parameters corresponding to the number of rows and columns of the input matrix and the output matrix; the prediction module is used to input the matrix dimension into a trained target classification model and output matrix multiplication quintuple parameters, wherein the matrix multiplication quintuple parameters include: three block parameters and two calculation parameters, the three block parameters corresponding to the number of rows and columns of the block input matrix and the block output matrix for each thread block in a single block operation, and the two calculation parameters corresponding to the number of rows and columns of each thread group in a single operation within the thread block.

[0028] Fourthly, this application embodiment also provides a model training device, including: a second acquisition module and a training module; the second acquisition module is used to acquire a training dataset, the training dataset including multiple training data, each training data including: the matrix dimension of the matrix multiplication operation, the optimal matrix multiplication quintuple parameters, the thread block used in the matrix multiplication operation and the grid parameters; the training module is used to train a preset classification model using the training dataset to obtain a trained classification model.

[0029] Fifthly, embodiments of this application also provide an electronic device, characterized in that it includes: a memory and a processor, the processor being connected to the memory; the memory being used to store a program; the processor being used to invoke the program stored in the memory to perform a method provided by any possible implementation of the first aspect embodiment and / or in combination with the first aspect embodiment, or to perform a method provided by any possible implementation of the second aspect embodiment and / or in combination with the second aspect embodiment.

[0030] Sixthly, embodiments of this application also provide a computer-readable storage medium, characterized in that it stores a computer program thereon, which, when executed by a processor, performs the method provided by any possible implementation of the first aspect embodiment and / or in combination with the first aspect embodiment, or performs the method provided by any possible implementation of the second aspect embodiment and / or in combination with the second aspect embodiment.

[0031] Other features and advantages of this application will be set forth in the following description. The objectives and other advantages of this application can be realized and obtained through the structures particularly pointed out in the written description and the accompanying drawings. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are only some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings. The above and other objects, features, and advantages of this application will become clearer through the accompanying drawings.

[0033] Figure 1a This illustration shows a schematic diagram of decomposing a matrix multiplication into multiple smaller blocks according to an embodiment of this application.

[0034] Figure 1b To be Figure 1a A diagram showing how a small block is further broken down into multiple smaller blocks.

[0035] Figure 1c Responsible for a Warp Figure 1b A schematic diagram of the calculation of a matrix of size (Tm, Tn) in the highlighted part.

[0036] Figure 2 A flowchart illustrating a method for obtaining matrix multiplication parameters provided in an embodiment of this application is shown.

[0037] Figure 3 A schematic flowchart of a model training method provided in an embodiment of this application is shown.

[0038] Figure 4 The diagram illustrates the principle of a model training method provided in an embodiment of this application.

[0039] Figure 5 A schematic diagram of a matrix multiplication parameter acquisition device provided in an embodiment of this application is shown.

[0040] Figure 6 A schematic diagram of a model training device provided in an embodiment of this application is shown.

[0041] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0042] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. The following embodiments are provided as examples to more clearly illustrate the technical solutions of this application, and should not be used to limit the scope of protection of this application. Those skilled in the art will understand that, without conflict, the following embodiments and features can be combined with each other.

[0043] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, relational terms such as "first," "second," etc., in the description of this application are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0044] Furthermore, the term "and / or" in this application is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0045] In the description of the embodiments of this application, unless otherwise expressly specified and limited, the technical term "connection" can be a direct connection or an indirect connection through an intermediate medium.

[0046] Because the performance of Tensor Cores in processors such as Graphics Processing Units (GPUs) is highly dependent on data layout, matrix size, and thread organization, in CUDA (Compute Unified Device Architecture) programming, the size selection of Grids and Blocks must be "aligned" or matched with the computational characteristics of Tensor Cores to maximize hardware utilization. This "alignment" is a process of ensuring that the software-side execution structure (the organization from Grid to Block to Thread) matches the inherent working mode of the underlying hardware computing units (Tensor Cores). It involves multiple levels of alignment, including thread organization alignment, data layout alignment, and computation size alignment. Only when these levels of alignment are well-executed can the CUDA kernel be considered highly optimized, maximizing the computing power of the Tensor Cores and achieving ultimate performance.

[0047] Specifically, the alignment includes: Thread organization alignment: The block size must be aligned with the warp size (including 32 or 64 threads) to generate a complete warp to drive the Tensor Core. Data layout alignment: The way data is arranged in memory must be aligned with the warp's access mode to achieve merged access and avoid conflicts, thus ensuring that the data supply speed can keep up with the Tensor Core's computation speed. Computation size alignment: The data size (matrix size) and partitioning method (tile size) should be aligned with the Tensor Core hardware's operating size (e.g., 16×16×16) to maximize the efficiency of each computation cycle.

[0048] A Grid is a collection of all thread blocks. It represents all the parallel threads created during a single kernel startup. A Grid consists of multiple Blocks, which can be organized into a three-dimensional structure, such as Grid.x, Grid.y, and Grid.z. A Block can contain multiple threads, which can also be organized into a three-dimensional structure, such as Block.x, Block.y, and Block.z. GPUs typically schedule threads in groups of 32 or 64 (called a Warp). Therefore, the size of a Block (Block.x × Block.y × Block.z) is best kept to be a multiple of 32 or 64 to avoid wasting resources. The size of a Grid (e.g., Grid.x × Grid.y × Grid.z) determines how many such thread blocks need to be started to process the entire large matrix. For example, if a 1024×1024 matrix is ​​divided into 64×64 sub-blocks, then (1024 / 64) × (1024 / 64) = 16 × 16 = 256 thread blocks are needed. The size of the Grid can be set as follows: Grid.x=16, Grid.y=16, Grid.z=1.

[0049] For matrix multiplication optimization based on Tensor Cores, please refer to... Figure 1a , Figure 1b , Figure 1c As shown, the matrix size to be calculated for each Block and Warp needs to be determined based on the input M, N, and K. This involves the five matrix multiplication quintuple parameters: Bm, Bn, Bk, Tm, and Tn. Selecting the optimal matrix multiplication quintuple parameters is crucial to achieving the best performance for matrix multiplication of a specific size. For example... Figure 1a As shown, a large matrix can be decomposed into multiple smaller blocks for multiplication. For example, matrix A (M×K=512×512) can be horizontally divided into four blocks, each with a size of (Bm×K=128×512). Matrix B (K×N=512×512) can be vertically divided into four blocks, each with a size of (K×Bn=512×128). The corresponding blocks of A and B (the shaded blocks in the diagram) form a Block in CUDA programming. There are 4×4=16 Blocks in total. Each Block is responsible for calculating the portion of matrix C of size (Bm, Bn). Then, the blocks of A and B are moved from GPU memory or global memory to SMEM (Shared Memory, L1 cache or shared memory). A series of read operations are then performed from SMEM to calculate C. This process is repeated until all blocks of C have been calculated.

[0050] If according to Figure 1aIf, after the partitioning method shown, the SMEM still cannot accommodate blocks of size (Bm, K) and (K, Bn), then further partitioning is required, such as... Figure 1b The highlighted blocks (Bm, Bk) in matrix A and the highlighted blocks (Bk, Bn) in matrix B are the results of further segmentation. Assuming Bk = 8, each block takes a portion of matrix A (Bm, Bk) = (128, 8) and a portion of matrix B (Bk, Bn) = (8, 128), multiplying them to obtain a portion of size (Bm, Bn) in matrix C. For matrix A, the next block is found to the right; for matrix B, the next block is found downwards, and then multiplied to obtain the segmented matrix C, which is then added to the previous segmented matrix C. This process is repeated until all blocks A and B have been traversed, resulting in the final segmented matrix C. Figure 1b The shaded area of ​​matrix C.

[0051] A single block performs calculations in several loops. Within each loop, the block loads a highlight block (Bm, Bk) from matrix A and a highlight block (Bk, Bn) from matrix B from video memory onto SMEM (each thread loads a portion of highlight blocks (Bm, Bk) and (Bk, Bn)). During a single loop, the actual calculations are performed by its thread group, i.e., the Warp mentioned above. Figure 1c As shown, each Warp is responsible for calculating a matrix of size (Tm, Tn) within this Block.

[0052] Therefore, in matrix multiplication, the settings of the five block parameters Bm, Bn, Bk, Tm, and Tn play a decisive role in performance. Currently, commonly used block strategies largely rely on the experience of software engineers: for different matrix dimensions M, N, and K, it is often necessary to repeatedly adjust these parameters on specific GPU hardware, and determine the optimal block size through multiple trials, thereby achieving the best performance for matrix multiplication.

[0053] Given the inefficiency and laboriousness of existing block-based strategies, this application provides a model-based method for obtaining matrix multiplication parameters in order to quickly and accurately obtain the aforementioned matrix multiplication quintuple parameters. Specifically, by inputting the M, N, and K values ​​of the matrix into the model, the model can predict the optimal Bm, Bn, Bk, Tm, and Tn, thereby fully utilizing the capabilities of the GPU hardware and optimizing the performance of matrix multiplication. The matrix multiplication parameters in this application include, but are not limited to, matrix multiplication quintuple parameters. In some possible implementations, they may also include the thread block and grid parameters used during matrix multiplication operations.

[0054] The following is combined with Figure 2The method for obtaining matrix multiplication parameters provided in the embodiments of this application will be described. The method for obtaining matrix multiplication parameters includes S1 and S2.

[0055] S1: Obtain the dimension of the matrix to be multiplied.

[0056] The matrix dimension includes three key parameters (M, N, K), corresponding to the number of rows and columns of the input and output matrices, respectively. For example, matrix A has a dimension of M×K (M rows and K columns), matrix B has a dimension of K×N (K rows and N columns), and their product matrix C has a dimension of M×N (M rows and N columns).

[0057] S2: Input the matrix dimension into the trained target classification model and output the matrix multiplied by a quintuple parameter.

[0058] After obtaining the matrix dimensions, inputting the matrix dimensions into the trained target classification model will output the matrix multiplication into quintuple parameters, which are Bm, Bn, Bk, Tm, and Tn as mentioned above.

[0059] The matrix multiplication quintuple parameters include three block parameters (Bm, Bn, Bk) and two computational parameters (Tm, Tn). The three block parameters correspond to the number of rows and columns of the block input and output matrices for each thread block in a single block operation. For example, if block matrix A has dimensions (Bm, Bk), block matrix B has dimensions (Bk, Bn), and their multiplication block matrix C has dimensions (Bm, Bn). The two computational parameters correspond to the number of rows and columns for each thread group within the thread block in a single operation.

[0060] In some possible implementations, the target classification model, in addition to outputting the matrix multiplication quintuple parameters, can also output the thread block and grid parameters used in the matrix multiplication operation, namely Block.x, Block.y, Block.z, Grid.x, Grid.y, and Grid.z mentioned above. Here, Block.x, Block.y, and Block.z are called thread block parameters, and Grid.x, Grid.y, and Grid.z are called grid parameters. When performing matrix multiplication, in addition to using the matrix multiplication quintuple parameters, the size selection of the Grid and Block is also involved.

[0061] In one possible implementation, the Grid and Block parameters predicted by the target classification model can be used directly. In other implementations, the Grid and Block parameters predicted by the target classification model can be omitted, and only the Bm, Bn, Bk, Tm, and Tn predicted by the target classification model can be used. Since the values ​​of Grid and Block are deterministic, such as those determined based on M, N, Bm, Bn, Tm, and Tn, the predicted values ​​are not required in actual use.

[0062] For a given set of M, N, and K, the five parameters Bm, Bn, Bk, Tm, and Tn affect the number and size of blocks in the matrix multiplication kernel function. For example, the thread layout calculation for a certain matrix multiplication kernel function is as follows:

[0063] Block.x = (Bm / Tm) * (Bn / Tn) * WarpSize

[0064] Block.y = 1

[0065] Block.z = 2

[0066] Grid.x = (N + Bn - 1) / Tn

[0067] Grid.y = (M + Bm - 1) / Tm

[0068] Grid.z = 1

[0069] Ultimately, the number and size of Blocks reflect the optimal combination of factors such as shared memory, registers, thread cooperation, and memory access. WarpSize is either 32 or 64, depending on the number of threads supported by the hardware.

[0070] The target classification model mentioned above can be a decision tree model, a random forest, or a gradient boosting tree, etc. The following examples will use the decision tree model as an example. For instance, the decision tree model can be trained in advance and then used to predict the parameters of the matrix multiplication quintuple.

[0071] The target classification model mentioned above can be a pre-trained classification model, or, in one possible implementation, a pre-trained and validated classification model. For example, after training the classification model, its prediction performance can be validated, and the target classification model is obtained when the validation is successful.

[0072] In one possible implementation, the target classification model can be trained by a third party, and the pre-trained target classification model can be used directly. In some implementations, the above method may further include a model training process, that is, the above method may further include the following model training methods.

[0073] The following is combined with Figure 3 The principle of the model training method provided in the embodiments of this application will be explained.

[0074] S10: Obtain the training dataset.

[0075] The training dataset includes multiple training data sets, each of which includes: the matrix dimension for matrix multiplication, the parameters of the optimal matrix multiplication quintuple, the thread block used in the matrix multiplication operation, and the grid parameters.

[0076] In constructing the training dataset, the input is (M, N, K), and the output label is defined as (Bm, Bn, Bk, Tm, Tn, Block.x, Block.y, Block.z, Grid.x, Grid.y, Grid.z). Since the values ​​of Block.y, Block.z, and Grid.z are fixed, the three dimensions of Block and Grid introduced in the output label can be selected according to actual needs. For example, the output label after selection can be (Bm, Bn, Bk, Tm, Tn, Block.x, Grid.x, Grid.y).

[0077] In one possible implementation, the process of obtaining multiple training data points from the training dataset may include: obtaining multiple sets of data, each set of data including the matrix dimension (M, N, K) of the matrix multiplication operation and the corresponding optimal matrix multiplication quintuple parameters (Bm, Bn, Bk, Tm, Tn); determining the thread block and grid parameters (such as Block.x, Grid.x, Grid.y) used for the matrix multiplication operation corresponding to each set of data based on the thread layout of each set of data and the matrix multiplication kernel function; and combining each set of data with the corresponding thread block and grid parameters used for the matrix multiplication operation to obtain multiple training data points from the training dataset.

[0078] When acquiring each set of data, an existing block-based strategy can be used: for different matrix dimensions MNK, the five parameters Bm, Bn, Bk, Tm, and Tn are repeatedly adjusted on the specific GPU hardware until optimal. Then, based on the thread layout of the matrix multiplication kernel function, the thread block and grid parameters used for the matrix multiplication operation corresponding to each set of data are determined, such as Block.x, Grid.x, and Grid.y. Finally, Block.x, Grid.x, and Grid.y are combined with M, N, K, Bm, Bn, Bk, Tm, and Tn to obtain a training dataset.

[0079] Taking decision trees as an example, a decision tree is a supervised learning algorithm. Building this model requires a certain amount of data, such as dozens of data points. Following the input and output data format mentioned above, the optimal combination of Bm, Bn, Bk, Tm, and Tn is manually selected from a certain number of MNK values. Then, the specific values ​​of the three dimensions Grid and Block are calculated (for a specific matrix multiplication kernel function, the calculation of Grid and Block is usually deterministic), thus constructing the required dataset. For example, the three constructed data points are as follows:

[0080]

[0081] S20: Use the training dataset to train the preset classification model to obtain the trained classification model.

[0082] After obtaining the training dataset, you can use it to train the pre-defined classification model to obtain a trained classification model. Common model training methods can be used during training, which will not be described here.

[0083] In addition, to improve the accuracy of model predictions, a performance feedback closed-loop calibration mechanism can be introduced to form a closed loop of "prediction → execution → feedback → learning" to achieve continuous model evolution.

[0084] In one possible implementation, the above method further includes: after obtaining the trained classification model, performing matrix multiplication using the matrix multiplication quintuple parameters predicted by the trained classification model according to the specified matrix dimension, and obtaining a first operating index during the operation; verifying the prediction effect of the trained classification model based on the first operating index; and obtaining the target classification model when the verification is successful.

[0085] For example, after obtaining a trained classification model, inputting a new MNK (e.g., M=1024, N=32, K=5120), the model predicts the specific values ​​of three dimensions: Bm, Bn, Bk, Tm, Tn, and Grid and Block. The values ​​of Grid and Block are predetermined and can be omitted in actual use.

[0086] After predicting the parameters of the matrix multiplication quintuple, a lightweight performance probe is executed. This involves performing one or more matrix multiplication operations based on the specified MNK and the predicted Bm, Bn, Bk, Tm, and Tn. Simultaneously, actual runtime metrics are collected, such as processor (e.g., GPU) utilization, storage (including memory and / or video memory) bandwidth utilization, and execution time. For example, the matrix multiplication kernel function is run on the GPU, and the CUDA Profiler tool is used to collect metrics such as execution time, GPU utilization, and video memory bandwidth utilization. This allows for multi-dimensional evaluation to determine optimal performance and validate the classification model's prediction capabilities. For instance, when validating the model based on collected metrics such as execution time, GPU utilization, and video memory bandwidth utilization, the following table can be used as a reference:

[0087]

[0088] In one possible implementation, the above method further includes: when the trained classification model fails validation, retraining the trained classification model with more training data to obtain the target classification model; wherein, a new round of training begins when the amount of training data reaches a preset threshold (e.g., adding 5 data points, the value of which is configurable) or when the time since the last training reaches a preset time interval (configurable, e.g., one week). For example, when validating the prediction effect of the trained classification model, if the actual performance index deviates from the preset optimal value by more than a threshold (e.g., 10%), the validation is considered to have failed. In this case, it is necessary to automatically search (or manually try gradually) to find better matrix multiplication quintuple parameters (Bm, Bn, Bk, Tm, Tn), add (MNK + optimal matrix multiplication quintuple parameters and Grid, Block parameters) to the training set, and retrain the model periodically (or after accumulating a certain amount of new data).

[0089] If multiple metrics exceed the threshold, the optimal block parameters (Bm, Bn, Bk, Tm, Tn) are found using the traditional method mentioned above, and then added to the training dataset. To avoid the overhead of retraining the model every time the training dataset is updated, the model can be retrained only when a certain amount of new data has been added.

[0090] In one possible implementation, the method further includes: obtaining the target matrix dimension in the training dataset that is closest to the specified matrix dimension; performing matrix multiplication using the optimal matrix multiplication quintuple parameters corresponding to the target matrix dimension, and obtaining a second performance index during the operation. Correspondingly, the process of verifying the prediction performance of the trained classification model based on the first performance index may include: verifying the prediction performance of the trained classification model based on the first performance index and the second performance index.

[0091] In this implementation, a target matrix dimension (e.g., 5120, 16384, 1280) that is closest to the specified matrix dimension (e.g., 6000, 16384, 2560) in the training dataset is also selected. Since the matrix multiplication quintuple parameters in the training dataset are all optimal matrix multiplication quintuple parameters, matrix multiplication is performed using the optimal matrix multiplication quintuple parameters corresponding to the target matrix dimension, and the second running index is obtained during the operation. Then, the prediction effect of the trained classification model can be verified based on the first running index and the second running index.

[0092] For example, a second performance metric can be used as a benchmark to compare the differences between the first and second performance metrics. If the difference is too large (e.g., greater than 10%), the validation fails. In some possible implementations, the first and second performance metrics include at least one parameter among processor utilization, memory bandwidth utilization, and execution time. For example, they may include execution time, GPU utilization, and video memory bandwidth utilization. When validating the model, the differences in execution time between the first and second performance metrics, the differences in GPU utilization between the first and second performance metrics, and the differences in video memory bandwidth utilization between the first and second performance metrics can be compared. If most metrics are not significantly different and only a few metrics are unqualified, the model validation can be considered successful. For example, when validating the model based on collected metrics such as execution time, GPU utilization, and video memory bandwidth utilization, the following table can be referenced:

[0093]

[0094] The process of obtaining the target matrix dimension that is closest to the specified matrix dimension in the training dataset can include: obtaining the first product of the three key parameters in the specified matrix dimension; obtaining the second product of the three key parameters in each matrix dimension in the training dataset; and obtaining the target second product that is closest to the first product among all the second products. The matrix dimension corresponding to the target second product is the target matrix dimension. That is, calculate the product of M×N×K, select the second product with the smallest difference from the first product, and the matrix dimension corresponding to the second product with the smallest difference from the first product is the target matrix dimension.

[0095] In some other possible implementations, the process of obtaining the target matrix dimension that is closest to the specified matrix dimension in the training dataset may include: obtaining the first difference between the M parameter in the specified matrix dimension and the M parameter in each matrix dimension in the training dataset; obtaining the second difference between the N parameter in the specified matrix dimension and the N parameter in each matrix dimension in the training dataset; obtaining the third difference between the K parameter in the specified matrix dimension and the K parameter in each matrix dimension in the training dataset; for each matrix dimension, obtaining the mean of the first difference, the second difference, and the third difference corresponding to that matrix dimension; and selecting the matrix dimension with the smallest mean as the target matrix dimension.

[0096] For example, suppose the training dataset contains three matrix dimensions. Calculate the first mean of the differences between the three key parameters in matrix dimension 1 and the three key parameters in the specified matrix dimension. Calculate the second mean of the differences between the three key parameters in matrix dimension 2 and the three key parameters in the specified matrix dimension. Calculate the third mean of the differences between the three key parameters in matrix dimension 3 and the three key parameters in the specified matrix dimension. Then select the minimum mean from these three means. For example, if the third mean is the smallest, then matrix dimension 3 is the target matrix dimension.

[0097] In one possible implementation, the principle of the model training method provided in this application embodiment can be as follows: Figure 4 As shown, this training method introduces a performance feedback closed-loop calibration mechanism to form a "prediction → execution → feedback → learning" closed loop, enabling the continuous evolution of the block prediction model.

[0098] On a certain domestically produced GPU, after optimizing the matrix multiplication operator using the matrix multiplication parameter acquisition method shown in this application, the inference performance of a large language model based on FP16 is improved by 25%. This method improves the core performance of GPU products, while reducing hardware costs and energy consumption, enhancing ecosystem compatibility, and ultimately enabling products to gain an advantage in the "speed-cost-experience" triangle. This will help them seize the initiative in AI (Artificial Intelligence), gaming, scientific computing, and other fields, and promote the development and implementation of more AI technologies and products.

[0099] The method described in this application is applicable to various scenarios requiring matrix multiplication. For example, it can be involved in the core computational operations of deep learning models (such as convolutional neural networks and Transformers) during training and inference, or it can be a fundamental operator for many key tasks in scientific computing (such as linear algebra solving and fluid dynamics simulation). The data involved can be image data, text data, etc. For example, it can be used for image recognition, object detection, medical image analysis, and autonomous driving visual perception based on convolutional neural networks. Input data: Image data (e.g., a 224×224×3 RGB image tensor). Core computation involved: Convolutional computation can be converted into large matrix multiplication on a GPU using the im2col algorithm. The filter matrix is ​​multiplied by the unfolded image patch matrix, and its size (M, N, K) is determined by the input feature map size, filter size, and number. For scenarios using Transformer models, it can be involved in large language models (such as Chat GPT), machine translation, text summarization, and code generation. Input data: Text data (usually converted into a sequence of word embedding vectors). Core computation: Self-attention mechanism: Query (Q), key (K), and value (V) matrix multiplication and Attention are the core matrix multiplications. Feedforward neural network: The FFN (Feed-Forward Network) layer in each Transformer block contains two linear transformations, i.e., two large matrix multiplications.

[0100] This application also provides a matrix multiplication parameter acquisition device, such as... Figure 5 As shown, the matrix multiplication parameter acquisition device includes a first acquisition module and a prediction module.

[0101] The first acquisition module is used to acquire the matrix dimension of the matrix to be multiplied. The matrix dimension includes three key parameters, corresponding to the number of rows and columns of the input matrix and the output matrix.

[0102] The prediction module is used to input the matrix dimension into the trained target classification model and output matrix multiplication quintuple parameters. The matrix multiplication quintuple parameters include three block parameters and two calculation parameters. The three block parameters correspond to the number of rows and columns of the block input matrix and block output matrix for each thread block in a single block operation. The two calculation parameters correspond to the number of rows and columns of each thread group in a single operation within the thread block.

[0103] In one possible implementation, the matrix multiplication parameter acquisition device further includes a second acquisition module and a training module.

[0104] The second acquisition module is used to acquire the training dataset, which includes multiple training data, each of which includes: the matrix dimension of the matrix multiplication operation, the optimal matrix multiplication quintuple parameters, the thread block and grid parameters used in the matrix multiplication operation;

[0105] The training module is used to train a preset classification model using the training dataset to obtain a trained classification model.

[0106] Optionally, the second acquisition module is used to acquire multiple sets of data, each set of data including the matrix dimension of the matrix multiplication operation and the corresponding optimal matrix multiplication quintuple parameters; determine the thread block and grid parameters used in the matrix multiplication operation corresponding to each set of data according to the thread layout of each set of data and the matrix multiplication kernel function; and combine each set of data and the thread block and grid parameters used in the corresponding matrix multiplication operation to obtain multiple training data in the training dataset.

[0107] In one possible implementation, the matrix multiplication parameter acquisition device further includes a verification module.

[0108] The verification module is used to perform matrix multiplication operations using the matrix multiplication quintuple parameters predicted by the trained classification model according to the specified matrix dimension after obtaining the trained classification model, and to obtain the first running index during the operation process; to verify the prediction effect of the trained classification model according to the first running index; and to obtain the target classification model when the verification is successful.

[0109] Optionally, the verification module is further configured to obtain the target matrix dimension in the training dataset that is closest to the specified matrix dimension; perform matrix multiplication using the optimal matrix multiplication quintuple parameter corresponding to the target matrix dimension; and obtain a second performance index during the operation. Accordingly, when verifying the prediction performance of the trained classification model based on the first performance index, the verification module is specifically configured to verify the prediction performance of the trained classification model based on both the first and second performance indices.

[0110] Optionally, the verification module is specifically used to obtain the first product of three key parameters in the specified matrix dimension, and to obtain the second product of three key parameters in each matrix dimension of the training dataset; and to obtain the second product among the second products that is closest to the first product, wherein the matrix dimension corresponding to the target second product is the target matrix dimension.

[0111] The training module is also used to train the trained classification model again with more training data when the trained classification model fails the validation, so as to obtain the target classification model; wherein, when the amount of training data reaches a preset threshold or the time since the last training reaches a preset time interval, a new round of training begins.

[0112] The matrix multiplication parameter acquisition device provided in this application embodiment has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0113] This application also provides a model training device, such as... Figure 6 As shown. The model training device includes: a second acquisition module and a training module.

[0114] The second acquisition module is used to acquire the training dataset, which includes multiple training data, each of which includes: the matrix dimension of the matrix multiplication operation, the optimal matrix multiplication quintuple parameters, the thread block and grid parameters used in the matrix multiplication operation;

[0115] The training module is used to train a preset classification model using the training dataset to obtain a trained classification model.

[0116] The model training device provided in this application embodiment has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0117] This application also provides an electronic device, such as... Figure 7 As shown, the electronic device includes: a transceiver, a memory, a communication bus, and a processor.

[0118] The transceiver, memory, and processor are electrically connected directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected through one or more communication buses or signal lines. The transceiver is used to send and receive data. The memory stores computer programs, such as the software functional modules shown in the matrix multiplication parameter acquisition device or model training device described above. The computer program includes at least one software functional module that can be stored in the memory or embedded in the operating system (OS) of the electronic device in the form of software or firmware. The processor executes the software functional modules or computer programs stored in the memory. For example, the processor executes the matrix multiplication parameter acquisition method described above, or executes the model training method described above.

[0119] The memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc.

[0120] The processor may be an integrated circuit chip with signal processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), Network Processor (NP), Graphics Processing Unit (GPU), Accelerated Processing Unit (ACCU), Multimedia Application Processor (MAP), microprocessor, etc.; it can also be a Digital Signal Processor (DSP), Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. Alternatively, the processor can be any conventional processor.

[0121] The aforementioned electronic devices include, but are not limited to, mobile phones, tablets, computers, servers, etc.

[0122] This application also provides a non-volatile computer-readable storage medium (hereinafter referred to as the storage medium) storing a computer program. When the computer program is run by a computer such as the electronic device described above, it executes the matrix multiplication parameter acquisition method or the model training method described above.

[0123] This application also provides a computer program product, which includes a computer program. When the computer program is executed by a computer, it performs the matrix multiplication parameter acquisition method described above, or performs the model training method described above.

[0124] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0125] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0126] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0127] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a computer-readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, laptop, server, or electronic device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned computer-readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0128] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for obtaining matrix multiplication parameters applied to an image processor, characterized in that, include: Obtain the matrix dimension of the matrix to be multiplied, which includes three key parameters: the number of rows and columns of the input matrix and the output matrix, respectively. The matrix dimension is input into the trained target classification model, and the output matrix multiplication quintuple parameters are given. The matrix multiplication quintuple parameters include three block parameters and two calculation parameters. The three block parameters correspond to the number of rows and columns of the block input matrix and block output matrix for each thread block in a single block operation. The two calculation parameters correspond to the number of rows and columns of each thread group in a single operation within the thread block. The target classification model is obtained by training a preset classification model using a training dataset. The training dataset includes multiple training data sets, each of which includes: the matrix dimension of the matrix multiplication operation, the optimal matrix multiplication quintuple parameters, the thread block used in the matrix multiplication operation, and the grid parameters.

2. The method according to claim 1, characterized in that, The method further includes: Obtain the training dataset; The pre-defined classification model is trained using the training dataset to obtain a trained classification model.

3. The method according to claim 2, characterized in that, Obtain multiple training data points from the training dataset, including: Obtain multiple sets of data, each set including the matrix dimension of the matrix multiplication operation and the corresponding optimal matrix multiplication quintuple parameters; Based on the thread layout of each set of data and the matrix multiplication kernel function, determine the thread block and grid parameters used for the matrix multiplication operation corresponding to each set of data; Each set of data is combined with the thread block and grid parameters used in the corresponding matrix multiplication operation to obtain multiple training data sets in the training dataset.

4. The method according to claim 2, characterized in that, The method further includes: After obtaining the trained classification model, matrix multiplication is performed using the matrix multiplication quintuple parameters predicted by the trained classification model based on the specified matrix dimension, and the first running index during the operation is obtained. The prediction performance of the trained classification model is verified based on the first operational metric. Upon successful verification, the target classification model is obtained.

5. The method according to claim 4, characterized in that, The method further includes: Obtain the target matrix dimension that is closest to the specified matrix dimension in the training dataset; Matrix multiplication is performed using the optimal matrix multiplication quintuple parameters corresponding to the dimension of the target matrix, and the second running index is obtained during the operation. Accordingly, the prediction performance of the trained classification model is verified based on the first operational metric, including: The prediction performance of the trained classification model is verified based on the first and second operational metrics.

6. The method according to claim 5, characterized in that, The three key parameters are M, N, and K, where M, N, and K are all integers greater than or equal to 1; obtaining the target matrix dimension in the training dataset that is closest to the specified matrix dimension includes: Obtain the first difference between the M-parameters in the specified matrix dimension and the M-parameters in each matrix dimension of the training dataset; Obtain the second difference between the N-parameters in the specified matrix dimension and the N-parameters in each matrix dimension of the training dataset; Obtain the third difference between the K parameter in the specified matrix dimension and the K parameter in each matrix dimension of the training dataset; For each matrix dimension, obtain the mean of the first difference, second difference, and third difference corresponding to that matrix dimension; among them, the matrix dimension with the smallest mean is the target matrix dimension.

7. The method according to claim 5, characterized in that, Obtaining the target matrix dimension that is closest to the specified matrix dimension in the training dataset includes: Obtain the first product of the three key parameters in the specified matrix dimension, and obtain the second product of the three key parameters in each matrix dimension of the training dataset; Obtain the target second product among all the second products that is closest to the first product, wherein the matrix dimension corresponding to the target second product is the target matrix dimension.

8. The method according to claim 4, characterized in that, The method further includes: If the trained classification model fails the validation, it is trained again using more training data to obtain the target classification model. Specifically, a new round of training begins when the amount of training data reaches a preset threshold or when the time since the last training reaches a preset time interval.

9. A model training method applied to an image processor, characterized in that, include: Obtain a training dataset, which includes multiple training data points. Each training data point includes: the matrix dimension of the matrix multiplication operation, the optimal matrix multiplication quintuple parameters, and the thread block and grid parameters used in the matrix multiplication operation. The matrix dimension includes three key parameters, corresponding to the number of rows and columns of the input and output matrices. The optimal matrix multiplication quintuple parameters include three block parameters and two computational parameters. The three block parameters correspond to the number of rows and columns of the block input and block output matrices in a single block operation within each thread block. The two computational parameters correspond to the number of rows and columns in a single operation within each thread group within the thread block. The pre-defined classification model is trained using the training dataset to obtain a trained classification model.

10. The method according to claim 9, characterized in that, The method further includes: The matrix multiplication operation is performed using the classification model based on the matrix multiplication quintuple parameters predicted according to the specified matrix dimension, and the first running index during the operation is obtained. The prediction performance of the trained classification model is verified based on the first operational metric.

11. The method according to claim 10, characterized in that, The method further includes: Obtain the target matrix dimension that is closest to the specified matrix dimension in the training dataset; Matrix multiplication is performed using the optimal matrix multiplication quintuple parameters corresponding to the dimension of the target matrix, and the second running index is obtained during the operation. Accordingly, the prediction performance of the trained classification model is verified based on the first operational metric, including: The prediction performance of the trained classification model is verified based on the first and second operational metrics.

12. A matrix multiplication parameter acquisition device applied to an image processor, characterized in that, include: The first acquisition module is used to acquire the matrix dimension of the matrix to be multiplied. The matrix dimension includes three key parameters, corresponding to the number of rows and columns of the input matrix and the output matrix. The prediction module is used to input the matrix dimension into the trained target classification model and output matrix multiplication 5-tuple parameters. The matrix multiplication 5-tuple parameters include: three block parameters and two calculation parameters. The three block parameters correspond to the number of rows and columns of the block input matrix and block output matrix for each thread block in a single block operation. The two calculation parameters correspond to the number of rows and columns of each thread group in a single operation within the thread block. The target classification model is obtained by training a preset classification model using a training dataset. The training dataset includes multiple training data sets, each of which includes: the matrix dimension of the matrix multiplication operation, the optimal matrix multiplication quintuple parameters, the thread block used in the matrix multiplication operation, and the grid parameters.

13. A model training device for an image processor, characterized in that, include: The second acquisition module is used to acquire the training dataset, which includes multiple training data points. Each training data point includes: the matrix dimension of the matrix multiplication operation, the optimal matrix multiplication quintuple parameters, the thread block and grid parameters used in the matrix multiplication operation; wherein, the matrix dimension includes: three key parameters, corresponding to the number of rows and columns of the input matrix and the output matrix; the optimal matrix multiplication quintuple parameters include: three block parameters and two calculation parameters, the three block parameters corresponding to the number of rows and columns of the block input matrix and the block output matrix in a single block operation of each thread block, and the two calculation parameters corresponding to the number of rows and columns of each thread group in a single operation within the thread block; The training module is used to train a preset classification model using the training dataset to obtain a trained classification model.

14. An electronic device, characterized in that, include: A memory and a processor, wherein the processor is connected to the memory; The memory is used to store programs; The processor is configured to invoke a program stored in the memory to execute the method as described in any one of claims 1-8, or to execute the method as described in any one of claims 9-11.

15. A computer-readable storage medium, characterized in that, It stores a computer program thereon, which, when executed by a processor, performs the method as described in any one of claims 1-8, or performs the method as described in any one of claims 9-11.

Images