Parameter prediction method, apparatus, device, medium, and program product

Abstract

The application relates to a parameter prediction method, device, equipment, medium and program product. The method comprises the following steps: obtaining scale information of a matrix operation task and hardware information of an accelerator running the matrix operation task; predicting each kernel parameter according to a preset parameter prediction model corresponding to each kernel parameter of the accelerator based on the scale information, the hardware information and the multiple kernel parameters, to obtain initial kernel configuration parameters corresponding to the matrix operation task; and determining a kernel configuration parameter prediction result based on the initial kernel configuration parameters. In the method, each parameter prediction model is trained based on a training set and a test set containing optimal kernel parameters of all categories, so that the parameter prediction model can identify kernel parameters of all categories, the multiple preset parameter prediction models are used to predict each kernel parameter, and the comprehensiveness and accuracy of prediction are improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a parameter prediction method, apparatus, device, medium, and program product. Background Technology

[0002] With the rapid development of deep learning and high-performance computing technologies, General Matrix Multiplication (GEMM) has become a core operation in various computationally intensive applications. On heterogeneous computing platforms based on accelerator cards (such as GPUs), kernel configuration parameters (such as thread block size, workgroup size, and vector width) directly affect the computational efficiency and memory access bandwidth of GEMM operations. Therefore, determining the optimal kernel parameter configuration for GEMM is of great significance in order to fully utilize the hardware performance of accelerator cards.

[0003] However, the GEMM kernel parameter prediction method in related technologies suffers from low prediction accuracy. Summary of the Invention

[0004] Therefore, it is necessary to provide a parameter prediction method, apparatus, device, medium, and program product that can improve prediction accuracy in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a parameter prediction method, which includes:

[0006] Obtain information about the scale of the matrix operation task and the hardware information of the accelerator running the matrix operation task;

[0007] Based on the scale information, hardware information, and the preset parameter prediction models corresponding to the multiple kernel parameters of the accelerator, the initial kernel configuration parameters corresponding to the matrix operation task are predicted. Each parameter prediction model is trained on a training set and a test set composed of optimal kernel parameters, and the training set and the test set contain the optimal kernel parameters for all categories.

[0008] Based on the initial kernel configuration parameters, determine the predicted kernel configuration parameters.

[0009] The parameter prediction method provided in this application obtains the scale information of the matrix operation task and the hardware information of the accelerator running the matrix operation task. Then, based on the scale information, hardware information, and preset parameter prediction models corresponding to multiple kernel parameters of the accelerator, it predicts each kernel parameter to obtain the initial kernel configuration parameters corresponding to the matrix operation task. Finally, based on the initial kernel configuration parameters, it determines the kernel configuration parameter prediction result. Each parameter prediction model is trained on a training set and a test set composed of optimal kernel parameters, and both the training set and the test set contain the optimal kernel parameters for all categories. In the above method, by obtaining the scale information of the matrix operation task and the hardware information of the accelerator running the matrix operation task, an input basis matching the current task and hardware environment is provided for subsequent prediction. Simultaneously, since each parameter prediction model is trained on a training set and a test set containing the optimal kernel parameters for all categories, the parameter prediction model can identify kernel parameters of each category, avoiding the problem of being unable to predict certain parameters due to missing categories in the training data. Furthermore, by using preset parameter prediction models corresponding to multiple kernel parameters to predict each kernel parameter, the comprehensiveness and accuracy of the prediction are improved. In addition, since the initial kernel configuration parameters are derived from the above-mentioned matching inputs and are predicted by a model with full-category recognition capabilities, they are inherently highly accurate. Consequently, the predicted results of the final kernel configuration parameters also have high accuracy, thereby further improving the overall prediction accuracy.

[0010] In some embodiments, the kernel configuration parameter prediction result is determined based on the initial kernel configuration parameters, including:

[0011] Obtain preset parameter constraints; preset parameter constraints include various types of constraints.

[0012] The initial kernel configuration parameters are corrected based on preset parameter constraints to obtain the kernel configuration parameter prediction results.

[0013] The method described in this application provides a multi-dimensional rule basis for judging the rationality of kernel configuration parameters by obtaining preset parameter constraints including various types of constraints. By correcting the initial kernel configuration parameters through preset parameter constraints, the final predicted kernel configuration parameters meet various physical feasibility and hardware adaptability requirements, avoiding kernel malfunction or performance degradation due to predicted parameters not meeting constraint relationships, thereby improving the rationality and reliability of the prediction results.

[0014] In some embodiments, the initial kernel configuration parameters are corrected according to preset parameter constraints to obtain kernel configuration parameter prediction results, including:

[0015] Determine whether the initial kernel configuration parameters meet all constraints;

[0016] If the initial kernel configuration parameters do not meet any of the constraints, the initial kernel configuration parameters are corrected according to the type of the unmet constraint to obtain the kernel configuration parameter prediction result.

[0017] The method described in this application, by performing comprehensive constraint verification on the initial kernel configuration parameters, can accurately identify unqualified predicted parameters, thereby improving the pertinence and effectiveness of parameter correction. Furthermore, by specifically correcting the initial kernel configuration parameters according to the type of unmet constraint, the problem of mismatch between predicted parameters and constraints can be effectively solved, ensuring that the final kernel configuration parameters meet all constraint requirements, thus improving the rationality and usability of the prediction results.

[0018] In some embodiments, based on the scale information, hardware information, and preset parameter prediction models corresponding to multiple kernel parameters of the accelerator, prediction is performed on each kernel parameter to obtain the initial kernel configuration parameters corresponding to the matrix operation task, including:

[0019] The scale information and hardware information are respectively input into the preset parameter prediction models corresponding to the multiple kernel parameters of the accelerator for prediction, and the kernel parameters output by each parameter prediction model are obtained.

[0020] The kernel parameters output by each parameter prediction model are summarized to obtain the initial kernel configuration parameters corresponding to the matrix operation task.

[0021] The method described in this application embodiment inputs scale information and hardware information into preset parameter prediction models corresponding to multiple kernel parameters of the accelerator for prediction. This allows each parameter prediction model to focus on learning the mapping relationship between specific kernel parameters and input information, avoiding the problem of mutual interference in multi-parameter joint prediction, improving the accuracy of each kernel parameter prediction, and realizing parallel or independent prediction of different kernel parameters. This avoids the inefficient mode of trying parameter combinations one by one in traditional methods, thereby improving the efficiency of kernel parameter prediction.

[0022] In some embodiments, the method further includes:

[0023] Obtain the sample dataset; the sample dataset includes multiple scale information samples, multiple hardware information samples, and optimal kernel parameters for multiple categories;

[0024] Obtain the initial parameter prediction models corresponding to each of the multiple kernel parameters of the accelerator;

[0025] Based on the category of the optimal kernel parameters, the sample dataset is processed to obtain the training set and the test set;

[0026] Based on the training set and the test set, multiple initial parameter prediction models are trained to obtain preset parameter prediction models corresponding to multiple kernel parameters.

[0027] In some embodiments, obtaining the initial parameter prediction model corresponding to each of the multiple kernel parameters of the accelerator includes:

[0028] The dimensions of the multi-dimensional parameters in the kernel parameters are destructured to obtain parameters with multiple dimensions;

[0029] Obtain the initial parameter prediction models corresponding to parameters in multiple dimensions, as well as the initial parameter prediction models corresponding to other parameters in the kernel parameters besides the multi-dimensional parameters, and the initial parameter prediction models corresponding to the multiple kernel parameters of the accelerator.

[0030] In some embodiments, the sample dataset is processed based on the category of the optimal kernel parameters to obtain a training set and a test set, including:

[0031] The percentage of the number of samples with optimal kernel parameters for each category in the sample dataset;

[0032] The sample dataset is divided into training and testing sets with the proportion of the optimal kernel parameters for each category in the training and testing sets as the objective.

[0033] The method described in this application process the sample dataset based on the category of the optimal kernel parameters, and uses a hierarchical sampling strategy to obtain the training set and the test set. This ensures that both the training set and the test set contain the optimal kernel parameters for all categories, making the category distribution of the training set consistent with that of the original dataset. This effectively alleviates the category identification bias problem caused by the long-tailed distribution of the optimal kernel parameter data, and improves the prediction accuracy of the model.

[0034] In some embodiments, the method further includes:

[0035] Determine whether the sample dataset meets the preset parameter constraints;

[0036] The sample data that does not meet the preset parameter constraints are removed to obtain a new sample dataset;

[0037] Based on the category of the optimal kernel parameters, the sample dataset is processed to obtain the training set and the test set, including:

[0038] Based on the category of the optimal kernel parameters, the new sample dataset is processed to obtain the training set and the test set.

[0039] The method described in this application embodiment, by determining whether the sample dataset meets the preset parameter constraints, can identify invalid samples in the sample dataset that do not meet the requirements of the hardware platform and parallel computing architecture, and remove sample data that does not meet the preset parameter constraints, thereby ensuring that the sample data used for model training meets the inherent constraint relationship between kernel parameters in terms of physical feasibility and logical rationality, and avoiding interference to model training caused by invalid data.

[0040] In some embodiments, the method further includes:

[0041] Determine the data types of the sample data in the training and test sets; data types include numerical and categorical.

[0042] Based on the data type, the sample data in the training set and the test set are preprocessed to obtain the preprocessed training set and the standardized test set.

[0043] Based on the training and test sets, multiple initial parameter prediction models are trained to obtain pre-defined parameter prediction models corresponding to multiple kernel parameters, including:

[0044] Based on the preprocessed training set and the preprocessed test set, multiple initial parameter prediction models are trained to obtain the preset parameter prediction models corresponding to each of the multiple kernel parameters.

[0045] In some embodiments, sample data in the training and test sets are preprocessed according to data type, including:

[0046] If the data type is numeric, then the sample data is numerically standardized.

[0047] If the data type is categorical, then the sample data is processed using one-hot encoding.

[0048] The method described in this application eliminates the dimensional differences between different features by performing numerical standardization on numerical sample data, ensuring that each numerical feature has equal influence during model learning, thereby improving the stability and accuracy of model training. By performing one-hot encoding on categorical sample data, discrete category labels are converted into numerical vector representations suitable for model input, enabling the model to effectively utilize information from categorical features, thus improving the model's ability to model categorical features and its prediction accuracy.

[0049] Secondly, this application also provides a parameter prediction device, which includes:

[0050] The first acquisition module is used to acquire the scale information of the matrix operation task and the hardware information of the accelerator running the matrix operation task;

[0051] The prediction module is used to predict each kernel parameter based on the scale information, the hardware information, and the preset parameter prediction models corresponding to the multiple kernel parameters of the accelerator, so as to obtain the initial kernel configuration parameters corresponding to the matrix operation task; wherein, each parameter prediction model is trained based on a training set and a test set composed of optimal kernel parameters, and the training set and the test set contain the optimal kernel parameters of all categories.

[0052] The first determining module is used to determine the kernel configuration parameter prediction result based on the initial kernel configuration parameters.

[0053] Thirdly, this application also provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the embodiments in the first aspect described above.

[0054] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the embodiments in the first aspect described above.

[0055] Fifthly, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of any of the embodiments in the first aspect described above.

[0056] The aforementioned parameter prediction method, apparatus, device, medium, and program products describe a method that acquires the scale information of a matrix operation task and the hardware information of the accelerator running the task. Then, based on the scale information, hardware information, and pre-defined parameter prediction models corresponding to multiple kernel parameters of the accelerator, it predicts each kernel parameter to obtain the initial kernel configuration parameters corresponding to the matrix operation task. Finally, based on the initial kernel configuration parameters, it determines the predicted kernel configuration parameters. Each parameter prediction model is trained on a training set and a test set containing optimal kernel parameters for all categories. By acquiring the scale information of the matrix operation task and the hardware information of the accelerator running the task, the method provides an input foundation that matches the current task and hardware environment for subsequent predictions. Furthermore, since each parameter prediction model is trained on a training set and a test set containing optimal kernel parameters for all categories, the model can identify kernel parameters of each category, avoiding the problem of being unable to predict certain parameters due to missing categories in the training data. This improves the comprehensiveness and accuracy of the prediction by using pre-defined parameter prediction models corresponding to multiple kernel parameters for each kernel parameter. In addition, since the initial kernel configuration parameters are derived from the above-mentioned matching inputs and are predicted by a model with full-category recognition capabilities, they are inherently highly accurate. Consequently, the predicted results of the final kernel configuration parameters also have high accuracy, thereby further improving the overall prediction accuracy. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 These are internal structural diagrams of the computer device in some embodiments;

[0059] Figure 2 This is one of the flowcharts illustrating the parameter prediction method in some embodiments;

[0060] Figure 3 This is a second flowchart illustrating the parameter prediction method in some embodiments;

[0061] Figure 4 This is the third flowchart illustrating the parameter prediction method in some embodiments;

[0062] Figure 5 This is the fourth flowchart illustrating the parameter prediction method in some embodiments;

[0063] Figure 6 This is the fifth flowchart illustrating the parameter prediction method in some embodiments;

[0064] Figure 7 This is the sixth flowchart illustrating the parameter prediction method in some embodiments;

[0065] Figure 8 This is a schematic diagram illustrating the optimal kernel parameter quantity distribution in some embodiments;

[0066] Figure 9 This is the seventh flowchart illustrating the parameter prediction method in some embodiments;

[0067] Figure 10 This is a schematic diagram of the structure of multiple initial parameter prediction models in some embodiments;

[0068] Figure 11 This is the eighth flowchart illustrating the parameter prediction method in some embodiments;

[0069] Figure 12 This is the ninth flowchart illustrating the parameter prediction method in some embodiments;

[0070] Figure 13 This is the tenth flowchart illustrating the parameter prediction method in some embodiments;

[0071] Figure 14 This is eleventh of a flowchart illustrating the parameter prediction method in some embodiments;

[0072] Figure 15 This is a structural block diagram of the parameter prediction device in some embodiments. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0074] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more, and other quantifiers are similar. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The term "and / or" used in this application refers to one of the solutions, or any combination of multiple solutions. In the embodiments of this application, the term "at least one" means one or more.

[0075] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0076] With the rapid development of deep learning and high-performance computing technologies, General Matrix Multiplication (GEMM) has become a core operation in many computationally intensive applications, playing a crucial role, especially in deep learning and high-performance computing. For example, in deep learning, the training and inference processes of large language models (LLMs) and multimodal models based on the Transformer architecture heavily rely on efficient matrix operations. These models typically contain billions or even hundreds of billions of parameters, resulting in extremely large computational demands. The performance of GEMM operations directly impacts the training speed and inference efficiency of these models. As model size continues to increase and application scenarios become more complex, the performance requirements for GEMM operations are also rising. Appropriate kernel optimization on domestically produced accelerator cards is a vital way to improve GEMM performance. Heterogeneous acceleration platforms, leveraging their high parallel computing capabilities and dedicated hardware architectures, have become important platforms for accelerating GEMM operations. However, to fully utilize the performance of GPUs, fine-grained optimization of GEMM operations is necessary. Kernel optimization is a key step in high-concurrency computing on accelerator cards, aiming to maximize computational efficiency and performance by rationally configuring the hardware resources of the accelerator card. This includes rationally allocating thread blocks, making full use of shared memory, and optimizing memory access patterns. Furthermore, different models of domestically produced accelerator cards have different hardware characteristics, such as the number of cores and shared memory capacity, which significantly impact the performance of GEMM operations. Therefore, on heterogeneous computing platforms based on accelerator cards (such as GPUs), kernel configuration parameters (such as thread block size, workgroup size, and vector width) directly affect the computational efficiency and memory access bandwidth of GEMM operations. Determining the optimal kernel parameter configuration for GEMM is crucial for fully utilizing the hardware performance of accelerator cards. However, existing GEMM kernel parameter prediction methods suffer from low prediction accuracy.

[0077] In view of this, embodiments of this application propose a parameter prediction method, apparatus, device, medium, and program product, which can predict each kernel parameter through a preset parameter prediction model corresponding to each of the multiple kernel parameters, thereby improving the comprehensiveness and accuracy of the prediction.

[0078] It should be noted that the beneficial effects or technical problems solved by the embodiments of this application are not limited to this one, but may also be other implicit or related problems. For details, please refer to the description of the embodiments below.

[0079] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0080] In some embodiments, the parameter prediction method provided in this application can be applied to, for example... Figure 1 The computer device shown can be a server or a terminal, and its internal structure diagram can be as follows: Figure 1 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When executed by the processor, the computer program implements a parameter prediction method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0081] Those skilled in the art will understand that Figure 1 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0082] In some embodiments, such as Figure 2 As shown, a parameter prediction method is provided, which is applied to... Figure 1 Taking a computer device as an example, the explanation includes the following steps:

[0083] S201, obtain the scale information of the matrix operation task and the hardware information of the accelerator running the matrix operation task.

[0084] The matrix operation task includes General Matrix Multiplication (GEMM). Scale information describes the dimensions of the matrices participating in GEMM, including at least one of the following: the number of rows M of the first matrix, the number of columns N of the second matrix, the number of columns K of the first matrix and the number of rows K of the second matrix, and the batch processing dimension B. The accelerator can be a domestically produced accelerator card or a heterogeneous computing device composed of accelerator cards. Hardware information characterizes the accelerator's computing power and hardware characteristics, including at least one of the following: accelerator model, number of computing units, vectorization width per computing unit, and shared memory capacity.

[0085] In this embodiment, when a computer device receives a matrix operation task to be executed, it can obtain the scale information of the matrix operation task through an application programming interface (API). Simultaneously, the computer device can call a hardware information query interface provided by the operating system or driver layer to obtain the hardware information of the accelerator running the matrix operation task. Furthermore, the computer device can unify the format of the collected scale information and hardware information, organizing the unified scale information and hardware information into structured data, which will then serve as input data for subsequent prediction steps. Optionally, the computer device can also obtain the scale information of the matrix operation task and the accelerator's hardware information through a system configuration file.

[0086] S202 predicts each kernel parameter based on the scale information, hardware information, and the preset parameter prediction models corresponding to the multiple kernel parameters of the accelerator, thereby obtaining the initial kernel configuration parameters corresponding to the matrix operation task.

[0087] Each parameter prediction model is trained on a training and test set composed of optimal kernel parameters. Both the training and test sets contain optimal kernel parameters for all categories to ensure the model's ability to identify and generalize across different kernel parameter categories. The pre-defined parameter prediction models corresponding to each kernel parameter are independently constructed and trained for each kernel parameter to be predicted. Each parameter prediction model outputs the predicted value of the corresponding kernel parameter based on the input scale and hardware information. Optimal kernel parameters are the kernel configuration parameters that achieve the highest computational efficiency under specific hardware and matrix sizes. Kernel parameters include at least one of macroblock parameters, workgroup parameters, thread block parameters, vector width parameters, workgroup mapping method parameters, and global split unit parameters. Initial kernel parameters include at least one of initial macroblock parameters, initial workgroup parameters, initial thread block parameters, initial vector width parameters, initial workgroup mapping method parameters, and initial global split unit parameters. Initial kernel configuration parameters are used to initially describe the kernel parameter settings expected to achieve high computational efficiency when performing matrix operation tasks on the accelerator.

[0088] In this embodiment, after obtaining the scale information of the matrix operation task and the hardware information of the accelerator, the computer device can use these two types of information as a basis to predict each kernel parameter through a preset parameter prediction model corresponding to each of the multiple kernel parameters of the accelerator, thereby obtaining the initial kernel configuration parameters corresponding to the matrix operation task. Specifically, the computer device can independently predict the corresponding kernel parameters using each parameter prediction model, and each parameter prediction model outputs the predicted value of the corresponding kernel parameter based on its pre-learned mapping relationship. After obtaining the prediction results of each kernel parameter through the above prediction process, the computer device aggregates these results to form the initial kernel configuration parameters corresponding to the matrix operation task. For example, the above parameter prediction model adopts an encoder-decoder architecture, and the above parameter prediction process is as follows: the computer device performs feature encoding on the input scale information and hardware information to obtain an input feature sequence, and then sends the input feature sequence into the preset encoder-decoder architecture. The encoder uses a multi-layer convolutional neural network to extract local feature patterns and combines an attention mechanism to weight important features. The decoder uses a long short-term memory network to generate a kernel parameter sequence in sequence, and outputs the prediction result of one kernel parameter at each decoding time step to obtain the initial kernel configuration parameters corresponding to the matrix operation task.

[0089] It should be noted that the computer device can pre-train each parameter prediction model based on a training set and a test set composed of optimal kernel parameters. Specifically, to ensure the model's ability to recognize kernel parameters for all categories, the computer device can use a stratified sampling strategy to construct the training and test sets. Sampling is based on the category of the optimal kernel parameter, following the proportion of samples from each category in the original dataset. This ensures that the category distribution in the training and test sets is consistent with the original dataset, and that both sets contain the optimal kernel parameters for all categories. Then, the computer device uses the training set containing samples from all categories to train the prediction models for each parameter. This allows the model to fully engage with the feature patterns of both head and tail categories during the learning process, avoiding recognition bias caused by class imbalance. After model training is complete, the computer device inputs samples from all categories in the test set into the model for validation, evaluating the model's prediction accuracy for each category's kernel parameters. This ensures the model has good generalization performance, and finally, the validated model is used as the parameter prediction model.

[0090] S203 determines the predicted kernel configuration parameters based on the initial kernel configuration parameters.

[0091] The kernel configuration parameter prediction results are the final kernel parameter configurations that, after verification and processing, can be directly used to drive the accelerator to perform matrix operation tasks. The kernel configuration parameter prediction results include at least one of the following: predicted macroblock parameters, predicted workgroup parameters, predicted thread block parameters, predicted vector width parameters, predicted workgroup mapping method parameters, and predicted global splitting unit parameters.

[0092] In this embodiment, after obtaining the initial kernel configuration parameters, the computer device can further process these parameters to determine the final kernel configuration parameter prediction result. Specifically, the computer device can perform validity verification on the initial kernel configuration parameters to determine whether the predicted values ​​of each parameter meet the physical constraints of hardware operation. Optionally, the computer device can coordinately adjust related parameters in the initial kernel configuration parameters according to the preset correlation between each kernel parameter to ensure that the parameters meet runtime dependency conditions. Optionally, the computer device can also compare the initial kernel configuration parameters with preset empirical configurations and select the more reliable configuration as the final prediction result. Optionally, the computer device can perform performance estimation on the initial kernel configuration parameters, and output the results after appropriate correction of the parameters. After completing the above processing, the computer device can obtain the final kernel configuration parameter prediction result, which can be directly sent to the accelerator's kernel boot interface to drive the accelerator to execute matrix operation tasks.

[0093] The parameter prediction method provided in this application obtains the scale information of the matrix operation task and the hardware information of the accelerator running the matrix operation task. Then, based on the scale information, hardware information, and preset parameter prediction models corresponding to multiple kernel parameters of the accelerator, it predicts each kernel parameter to obtain the initial kernel configuration parameters corresponding to the matrix operation task. Finally, based on the initial kernel configuration parameters, it determines the kernel configuration parameter prediction result. Each parameter prediction model is trained on a training set and a test set composed of optimal kernel parameters, and both the training set and the test set contain the optimal kernel parameters for all categories. In the above method, by obtaining the scale information of the matrix operation task and the hardware information of the accelerator running the matrix operation task, an input basis matching the current task and hardware environment is provided for subsequent prediction. Simultaneously, since each parameter prediction model is trained on a training set and a test set containing the optimal kernel parameters for all categories, the parameter prediction model can identify kernel parameters of each category, avoiding the problem of being unable to predict certain parameters due to missing categories in the training data. Furthermore, by using preset parameter prediction models corresponding to multiple kernel parameters to predict each kernel parameter, the comprehensiveness and accuracy of the prediction are improved. In addition, since the initial kernel configuration parameters are derived from the above-mentioned matching inputs and are predicted by a model with full-category recognition capabilities, they are inherently highly accurate. Consequently, the predicted results of the final kernel configuration parameters also have high accuracy, thereby further improving the overall prediction accuracy.

[0094] In some embodiments, a specific implementation method for determining the prediction result of kernel configuration parameters is also provided, such as... Figure 3 As shown, the "determine the kernel configuration parameter prediction result based on the initial kernel configuration parameters" in S203 above includes:

[0095] S301, Obtain preset parameter constraints.

[0096] Among them, the preset parameter constraints include various types of constraints, which are used to judge the rationality of kernel parameters from different dimensions.

[0097] In this embodiment, after obtaining the initial kernel configuration parameters, the computer device acquires preset parameter constraints. Specifically, the computer device can read the preset parameter constraints from a locally stored configuration file. Optionally, the computer device can obtain constraints matching the current accelerator model from a rule base by calling the accelerator's hardware abstraction layer interface. Optionally, the computer device can dynamically generate constraints based on the accelerator's hardware information.

[0098] The preset parameter constraints specifically include at least one of the following four constraints: (1) The thread block parameter TT, the work group parameter WG, and the macroblock parameter MT satisfy a preset mathematical relationship, which is used to constrain the numerical equality or inequality between the three, such as satisfying the constraint relationship MT=TT×WG. Specifically, the size of MT in the X direction is equal to the size of TT in the X direction multiplied by the size of WG in the X direction, and the size of MT in the Y direction is equal to the size of TT in the Y direction multiplied by the size of WG in the Y direction. Among them, WG, TT, and MT constitute the core parameter group of the accelerator's parallel computing. The three determine the matrix block granularity and work group data sharing mode through collaborative configuration, which directly affects the thread scheduling efficiency and memory access mode of the accelerator. (2) The work group parameter WGM and the work group mapping mode parameter WGM satisfy a preset mapping relationship, which is used to constrain the corresponding matching rules between the work group parameter and the work group mapping mode parameter. Among them, WGM and WG are strongly coupled. WGM determines the space allocation density and load balance of WG by defining the mapping rules of the work group WG on the accelerator hardware unit. (3) The value of the vector width parameter VW satisfies the preset value condition, which limits the vector width parameter to a preset fixed value. Among them, VW mainly affects the thread-level computation throughput and determines the vector instruction width. It has a weak correlation with the parallelism parameter group and belongs to an independent optimization dimension. (4) The global split unit parameter GSU, the work group parameter WG, and the thread block parameter TT satisfy the preset correlation relationship. This correlation relationship is used to constrain the coordination range between the global split unit parameter, the work group parameter, and the thread block parameter. Among them, GSU ​​increases the utilization of the accelerator by splitting the computation task into multiple WGs, but it increases the demand for global memory bandwidth. It needs to be reduced through global memory atomic operations. GSU, WG, and TT parameters affect each other and need to be optimized in a coordinated manner to balance global memory bandwidth and computational efficiency.

[0099] S302, the initial kernel configuration parameters are corrected according to the preset parameter constraints to obtain the kernel configuration parameter prediction results.

[0100] In this embodiment, after obtaining preset parameter constraints, the computer device modifies the initial kernel configuration parameters according to these constraints. Specifically, the computer device compares the initial kernel configuration parameters with the preset parameter constraints to determine whether the initial kernel configuration parameters meet each constraint. Parameters that meet the constraints are retained; parameters that do not meet the constraints are modified according to a preset adjustment strategy to bring them within the reasonable range specified by the constraints. Optionally, the computer device may sequentially verify and modify the initial kernel configuration parameters according to the type of constraint. Optionally, the computer device may coordinately adjust each kernel parameter so that the modified parameter combination as a whole meets all constraints. After the computer device completes the modification, it obtains the final kernel configuration parameter prediction result, which can be directly sent to the accelerator's kernel boot interface to drive the accelerator to execute matrix operation tasks.

[0101] The method described in this application provides a multi-dimensional rule basis for judging the rationality of kernel configuration parameters by obtaining preset parameter constraints including various types of constraints. By correcting the initial kernel configuration parameters through preset parameter constraints, the final predicted kernel configuration parameters meet various physical feasibility and hardware adaptability requirements, avoiding kernel malfunction or performance degradation due to predicted parameters not meeting constraint relationships, thereby improving the rationality and reliability of the prediction results.

[0102] In some embodiments, a specific implementation method for modifying the initial kernel configuration parameters is also provided, such as... Figure 4 As shown, the "correcting the initial kernel configuration parameters according to preset parameter constraints to obtain the kernel configuration parameter prediction result" in S302 above includes:

[0103] S401 determines whether the initial kernel configuration parameters meet all constraints.

[0104] In this embodiment, after obtaining preset parameter constraints, the computer device compares the initial kernel configuration parameters with various constraints to determine whether the initial kernel configuration parameters satisfy all constraints. Specifically, the computer device can verify each parameter in the initial kernel configuration parameters sequentially according to the type of constraint. For mathematical equation constraints, the computer device determines whether the relevant parameters satisfy the preset equation relationship; for value range constraints, the computer device determines whether the parameter values ​​fall within the preset value set or interval; for mapping relationship constraints, the computer device determines whether the parameter combination conforms to the preset correspondence rule; for association relationship constraints, the computer device determines whether the cooperative range between relevant parameters meets the preset upper and lower limit requirements. The computer device determines whether the initial kernel configuration parameters satisfy all constraints based on the verification results. If the initial kernel configuration parameters satisfy all constraints, there is no need to modify the initial kernel configuration parameters; the initial kernel configuration parameters are directly used as the kernel configuration parameter prediction result.

[0105] S402, if the initial kernel configuration parameters do not meet any constraint, the initial kernel configuration parameters are corrected according to the type of the unmet constraint to obtain the kernel configuration parameter prediction result.

[0106] Different types of constraints correspond to different correction strategies.

[0107] In this embodiment of the application, if the computer device determines that the initial kernel configuration parameters do not meet any constraint, the initial kernel configuration parameters are corrected according to the type of constraint that is not met.

[0108] In one implementation, the computer device can perform targeted corrections based on the type of constraint not met. Specifically, if the unmet constraint type is a preset mathematical relationship between thread block parameters, workgroup parameters, and macroblock parameters, then mathematical relationship correction is performed. The macroblock size remains unchanged, and the thread block size or workgroup size is adjusted so that the macroblock X-direction size equals the thread block X-direction size multiplied by the workgroup X-direction size, and the macroblock Y-direction size equals the thread block Y-direction size multiplied by the workgroup Y-direction size, prioritizing parameters with lower confidence. If the unmet constraint type is a preset mapping relationship between workgroup parameters and workgroup mapping method parameters, then mapping relationship correction is performed, replacing the workgroup mapping method with the closest value from the set of valid mapping methods corresponding to the current workgroup size. If the unmet constraint type is a preset value condition for the vector width parameter, then value condition correction is performed, replacing the vector width with the fixed value closest to the original value. If the unmet constraint type is a preset association relationship between global split unit parameters, workgroup parameters, and thread block parameters, then association relationship correction is performed, adjusting the global split unit to the maximum allowable value calculated based on the workgroup size and thread block size. Finally, the computer device combines all the revised kernel parameters to form the kernel configuration parameter prediction result.

[0109] In another implementation, the computer device can perform iterative correction based on a priority order. The specific implementation process is as follows: After determining the types of unmet constraints, the computer device sorts the unmet constraints according to a preset priority order. This priority order can be: preset mathematical relationship constraints take precedence over preset mapping relationship constraints, preset mapping relationship constraints take precedence over preset value condition constraints, and preset value condition constraints take precedence over preset association relationship constraints. Then, the computer device selects the highest priority unmet constraint and performs the corresponding correction operation based on the type of this constraint, ensuring that the corrected highest priority constraint meets the condition. Next, after correcting the current constraint, the computer device re-executes step S401 to determine whether the corrected kernel configuration parameters meet all constraints. If unmet constraints still exist, the next highest priority unmet constraint is selected for correction, and the correction and verification process is repeated. Finally, when the computer device obtains a combination of kernel configuration parameters that meets all constraints after correction, the iteration terminates, and this combination is taken as the kernel configuration parameter prediction result.

[0110] The method described in this application, by performing comprehensive constraint verification on the initial kernel configuration parameters, can accurately identify unqualified predicted parameters, thereby improving the pertinence and effectiveness of parameter correction. Furthermore, by specifically correcting the initial kernel configuration parameters according to the type of unmet constraint, the problem of mismatch between predicted parameters and constraints can be effectively solved, ensuring that the final kernel configuration parameters meet all constraint requirements, thus improving the rationality and usability of the prediction results.

[0111] In some embodiments, a specific implementation method is also provided for predicting each kernel parameter to obtain the initial kernel configuration parameters corresponding to the matrix operation task, such as... Figure 5 As shown, the "predicting each kernel parameter based on the preset parameter prediction model corresponding to each of the scale information, hardware information, and multiple kernel parameters of the accelerator to obtain the initial kernel configuration parameters corresponding to the matrix operation task" in S203 above includes:

[0112] S501 inputs scale information and hardware information into the preset parameter prediction models corresponding to the multiple kernel parameters of the accelerator for prediction, and obtains the kernel parameters output by each parameter prediction model.

[0113] In this embodiment, after obtaining the scale information of the matrix operation task and the hardware information of the accelerator running the matrix operation task, the computer device uses this scale information and hardware information as input and feeds them into the preset parameter prediction models corresponding to the multiple kernel parameters of the accelerator for prediction. Specifically, the computer device can organize the scale information and hardware information into a unified feature vector and input it into each parameter prediction model. Alternatively, the computer device can perform differentiated preprocessing on the input information according to the characteristics of each kernel parameter before feeding it into the corresponding parameter prediction model. Then, each parameter prediction model independently predicts its corresponding kernel parameter based on its pre-learned mapping relationship and outputs the corresponding predicted value. Optionally, the computer device can concurrently call multiple parameter prediction models for synchronous prediction to shorten the overall prediction time. Through the above methods, the computer device obtains the predicted values ​​of the kernel parameters output by each parameter prediction model.

[0114] S502 summarizes the kernel parameters output by each parameter prediction model to obtain the initial kernel configuration parameters corresponding to the matrix operation task.

[0115] In this embodiment, after obtaining the kernel parameter prediction values ​​output by each parameter prediction model, the computer device can summarize the prediction values ​​output by each model to obtain the initial kernel configuration parameters corresponding to the matrix operation task. Specifically, the computer device can assemble the prediction values ​​output by each parameter prediction model according to a preset parameter structure to form a complete configuration set containing multiple kernel parameters, which serves as the initial kernel configuration parameters.

[0116] The method described in this application embodiment inputs scale information and hardware information into preset parameter prediction models corresponding to multiple kernel parameters of the accelerator for prediction. This allows each parameter prediction model to focus on learning the mapping relationship between specific kernel parameters and input information, avoiding the problem of mutual interference in multi-parameter joint prediction, improving the accuracy of each kernel parameter prediction, and realizing parallel or independent prediction of different kernel parameters. This avoids the inefficient mode of trying parameter combinations one by one in traditional methods, thereby improving the efficiency of kernel parameter prediction.

[0117] In some embodiments, a method for training a parameter prediction model is also provided, such as Figure 6 As shown, the training method includes:

[0118] S601, Obtain the sample dataset.

[0119] The sample dataset includes multiple scale information samples, multiple hardware information samples, and multiple categories of optimal kernel parameters. Scale information samples include matrix dimension parameters for different matrix operation tasks; hardware information samples include hardware characteristic parameters for different accelerators; and optimal kernel parameters are the kernel configuration parameters that achieve the highest computational efficiency under specific hardware and matrix size. Multiple categories of optimal kernel parameters refer to the various categories formed by classifying different values ​​or configuration schemes of kernel parameters. Specifically, in matrix operation tasks, for a specific matrix size and hardware platform, there exists a set of kernel configuration parameters that achieve the highest computational efficiency; this set of parameters is the "optimal kernel parameters." Since kernel parameters can take many different values ​​or patterns, each different value or configuration combination constitutes a "category." Taking the workgroup parameter WG as an example, this parameter may have multiple different values ​​in a specific configuration, such as WG=32, WG=64, WG=128, WG=256, etc. WG=32 constitutes one category, WG=64 constitutes another category, and so on. For example, the vector width parameter VW typically has a fixed set of values, such as VW=1, VW=2, and VW=4, where VW=1, VW=2, and VW=4 represent three different categories. Similarly, the workgroup mapping mode parameter WGM may have multiple mapping modes, each representing a category.

[0120] In this embodiment, the computer device can collect historical tuning data running on various accelerators at different matrix sizes, and extract scale information samples, hardware information samples, and corresponding optimal kernel parameters from the data to obtain a sample dataset. The scale information samples include at least one of the following: the number of rows in matrix A, the number of columns in matrix B, the number of columns in matrix A and the number of rows in matrix B, and the batch processing dimension; the hardware information samples include at least one of the following: accelerator model, number of computing units, vectorization width of each computing unit, and shared memory capacity; the optimal kernel parameters include at least one of the following: macroblock parameters, workgroup parameters, thread block parameters, vector width parameters, workgroup mapping method parameters, and global split unit parameters.

[0121] S602 obtains the initial parameter prediction model corresponding to each of the multiple kernel parameters of the accelerator.

[0122] Among them, the initial parameter prediction model refers to the parameter prediction model that has not been trained or has not completed training, and is used to predict the corresponding kernel parameters based on the input scale information and hardware information; the initial parameter prediction models corresponding to multiple kernel parameters refer to independent models built for each kernel parameter to be predicted.

[0123] In this embodiment, the computer device can construct a corresponding number of initial parameter prediction models based on the number of kernel parameters to be predicted. Each initial parameter prediction model corresponds to one kernel parameter and is used to learn the mapping relationship between the kernel parameter and the input features. Optionally, the computer device can load a predefined model structure from a locally stored model library as an initial parameter prediction model. Optionally, the computer device can dynamically create initial parameter prediction models with corresponding structures based on the type and characteristics of the kernel parameters.

[0124] For example, the computer device uses the random forest algorithm as the core prediction model, constructing a separate random forest sub-model as the corresponding initial parameter prediction model for each kernel parameter to be predicted. Specifically, the initial parameter prediction model constructed by the computer device includes the following independent random tree models: a model for predicting workgroup parameters (WG), a model for predicting thread block parameters (TT), a model for predicting macroblock parameters (MT), a model for predicting vector width parameters (VW), a model for predicting workgroup mapping mode parameters (WGM), and a model for predicting global split unit parameters (GSU). The inputs to all these models are the same, including matrix size information (M, N, K, B) and accelerator hardware information (accelerator model, number of compute units, single instruction multiple data width, shared memory size). However, the outputs of each sub-model are different; each sub-model outputs the single kernel parameter it is responsible for predicting.

[0125] Random forests, as a powerful ensemble learning method, can significantly improve the accuracy and stability of a model by constructing multiple decision trees and combining their prediction results. This embodiment constructs an independent random forest sub-model for each kernel parameter to be predicted. Each sub-model focuses on learning the mapping relationship between a specific target parameter and input features, thereby achieving accurate predictions of different key parameters. This independent modeling approach helps improve the model's specificity and generalization ability, enabling it to better adapt to the prediction needs of different parameters.

[0126] Optional, such as Figure 7 As shown, the above S602 includes:

[0127] S6021 deconstructs the dimensions of multi-dimensional parameters in the kernel parameters to obtain parameters with multiple dimensions.

[0128] Multi-dimensional parameters refer to kernel parameters composed of multiple dimensions, including at least one of workgroup parameters, thread block parameters, and macroblock parameters. Deconstruction refers to breaking down a multi-dimensional parameter into multiple independent single-dimensional parameters according to its dimensional composition.

[0129] In this embodiment, before obtaining the initial parameter prediction model, the computer device can first deconstruct the dimensions of the multi-dimensional parameters in the kernel parameters. Specifically, the computer device identifies parameters with multi-dimensional structures in the kernel parameters, such as workgroup parameters including the values ​​of the workgroup in the X and Y dimensions, thread block parameters including the values ​​of the thread block in the X and Y dimensions, and macroblock parameters including the values ​​of the macroblock in the X, Y, and Z dimensions. The computer device decomposes these multi-dimensional parameters according to their dimensional composition, treating the parameter corresponding to each dimension as an independent single-dimensional parameter. For example, the computer device deconstructs the workgroup parameters into X-dimensional and Y-dimensional parameters, the thread block parameters into X-dimensional and Y-dimensional parameters, and the macroblock parameters into X-dimensional, Y-dimensional, and Z-dimensional parameters. Through the above deconstruction operation, the computer device obtains parameters with multiple dimensions, which serve as independent prediction targets for subsequent construction of the corresponding initial parameter prediction model. By deconstructing the dimensions of multi-dimensional parameters, the originally complex parameters are broken down into multiple independent single-dimensional parameters, which effectively reduces the learning complexity of the model. This allows the model to more accurately capture the mapping relationship between each dimension parameter and the input features, thereby improving the targeting of subsequent model training and the accuracy of prediction.

[0130] S6022, obtain the initial parameter prediction model corresponding to parameters of multiple dimensions, and the initial parameter prediction model corresponding to other parameters in the kernel parameters besides the multi-dimensional parameters, as the initial parameter prediction model corresponding to each of the multiple kernel parameters of the accelerator.

[0131] Other parameters refer to independent parameters in the kernel parameters that have not been destructured, including at least one of the following: vector width parameter, workgroup mapping method parameter, and global split unit parameter.

[0132] In this embodiment, after deconstructing the multi-dimensional parameters, the computer device obtains initial parameter prediction models corresponding to the parameters of each dimension, and simultaneously obtains initial parameter prediction models corresponding to other parameters in the kernel parameters besides the multi-dimensional parameters. All of these initial parameter prediction models are then used as the initial parameter prediction models corresponding to the various kernel parameters of the accelerator. Specifically, for each single-dimensional parameter obtained after deconstruction, the computer device constructs an initial parameter prediction model to learn the mapping relationship between that single-dimensional parameter and the input features. Simultaneously, for other parameters that have not been deconstructed, such as vector width parameters, working group mapping method parameters, and global splitting unit parameters, the computer device also constructs corresponding initial parameter prediction models. The computer device summarizes all the above initial parameter prediction models to form a set of initial parameter prediction models corresponding one-to-one with each independent prediction target, serving as the basis for subsequent training. By constructing corresponding initial parameter prediction models for the deconstructed multi-dimensional parameters and other independent parameters, refined modeling of each independent prediction target is achieved, enabling each model to focus on learning the mapping rules of a single parameter, thereby improving the model training effect and the accuracy of the final prediction. Specifically, the initial parameter prediction model includes the following independent models: a model for predicting the X-dimensional parameters (WG_x) of the workgroup, a model for predicting the Y-dimensional parameters (WG_y) of the workgroup, a model for predicting the X-dimensional parameters (TT_x) of the thread block, a model for predicting the Y-dimensional parameters (TT_y) of the thread block, a model for predicting the X-dimensional parameters (MT_x) of the macroblock, a model for predicting the Y-dimensional parameters (MT_y) of the macroblock, a model for predicting the Z-dimensional parameters (MT_z) of the macroblock, a model for predicting the vector width parameter (VW), a model for predicting the workgroup mapping mode parameter (WGM), and a model for predicting the global split unit parameter (GSU).

[0133] S603 processes the sample dataset based on the category of the optimal kernel parameters to obtain the training set and the test set.

[0134] In this embodiment, after acquiring the sample dataset, the computer device can process the dataset using a class-balanced sampling method based on the categories of optimal kernel parameters. Specifically, the computer device first identifies the sample quantity distribution of each category with optimal kernel parameters in the sample dataset, determining the main cluster with a large sample size and the sparse cluster with a small sample size. For categories in the main cluster, the computer device randomly selects a portion of samples according to a preset sampling ratio to form the training set, and the remaining samples form the test set. For categories in the sparse cluster, the computer device simultaneously includes all samples under that category in both the training and test sets, or uses cross-validation to ensure that samples of that category are represented in both the training and test sets. Through these methods, the computer device ensures that both the training and test sets cover the optimal kernel parameters of all categories, while mitigating model training bias that may be caused by imbalanced category sample sizes.

[0135] To address the performance differences of kernels in GEMM operations, this embodiment first performs statistical modeling on the optimal kernel data under different GEMM sizes. Analysis shows that the dataset exhibits a significant long-tail distribution characteristic, such as... Figure 8 The diagram showing the distribution of the number of optimal kernel parameters (BestKernel Count Distribution) shows that the horizontal axis is the index of the optimal kernel parameters (BestKernel_Index) and the vertical axis is the number (Count). Specifically, it shows: (1) Uneven stratification of class sample size: The total number of classes in the optimal kernel is about 1500, and the range of sample size for a single class is several orders of magnitude, showing a bipolar distribution of "concentrated head classes and sparse tail classes". Among them, the classes accounting for about 14% (defined as "main class clusters") account for more than 80% of the total data; the remaining classes accounting for about 86% (defined as "sparse class clusters") have a total sample size of less than 20% of the total, and the sample size of most classes is in the range of single digits to hundreds. (2) Extreme skewness of class distribution: There is a significant difference in sample size between the main class clusters and the sparse class clusters, forming a distribution feature of "a few classes dominating and most classes being sparse". This characteristic makes traditional classification models susceptible to the dominance of the main cluster samples during training, resulting in significant class identification bias and making it difficult to effectively capture the feature patterns of sparse clusters.

[0136] To address the impact of imbalanced data on model training, this embodiment employs a stratified sampling strategy based on the best-performing kernel category during the sample dataset partitioning process. The best kernel parameter category (BestKernel) is used as the stratification basis, and the sampling proportions of each kernel category in the original dataset are considered. (in Sampling is performed using the number of samples of category c (where N is the total number of samples). By controlling the proportion of each sample category in different subsets of data through stratification, model training bias caused by data distribution deviations is reduced, providing a structurally balanced sample foundation for subsequent model training. This ensures that the category distribution in the training and test sets is consistent with the original dataset, maintaining the realism and balance of the data distribution, thereby improving the model's generalization performance and classification accuracy in long-tailed distribution scenarios.

[0137] Optional, such as Figure 9 As shown, the above S603 includes:

[0138] S6031, obtain the percentage of the number of samples with optimal kernel parameters for each category in the sample dataset.

[0139] The sample percentage refers to the proportion of the number of optimal kernel parameter samples for each category to the total number of samples in the sample dataset, which is used to characterize the distribution of each category in the original dataset.

[0140] In this embodiment, after acquiring the sample dataset, the computer device can perform statistical analysis on the optimal kernel parameters for each category in the sample dataset to obtain the sample proportion of each category in the sample dataset. Specifically, the computer device traverses the sample dataset, identifies the optimal kernel parameter category corresponding to each sample, counts the cumulative number of samples in each category, and calculates the ratio of the number of samples in each category to the total number of samples in the sample dataset to obtain the sample proportion of each category. Optionally, the computer device can store the statistical results in the form of a mapping table, recording each category and its corresponding sample proportion.

[0141] S6032 divides the sample dataset into training and testing sets with the proportion of the optimal kernel parameters for each category in the training and testing sets as the sample proportion.

[0142] In this embodiment, after obtaining the sample proportions of the optimal kernel parameters for each category, the computer device divides the sample dataset into a training set and a test set, aiming to maintain consistency between the proportions of each category and the sample proportions in the training and test sets. This ensures that the distribution ratios of each category in the training and test sets are consistent with the distribution ratios of the original dataset. Specifically, the computer device allocates a corresponding number of samples from each category to the training and test sets according to a preset division ratio, based on the sample proportions of each category in the original sample dataset. This ensures that the sample proportions of each category in the training and test sets after the division are substantially consistent with the sample proportions in the original sample dataset. Optionally, the computer device can use stratified sampling, sequentially extracting samples from each category according to their sample proportions to form the training and test sets respectively. After the computer device completes the division, it obtains the training and test sets, both of which maintain the same category distribution characteristics as the original dataset.

[0143] For example, suppose a sample dataset contains optimal kernel parameters for three categories: category A, category B, and category C. The dataset contains a total of 1000 samples, with 700 samples of category A, 200 of category B, and 100 of category C. The sample percentages for each category are: category A 70%, category B 20%, and category C 10%. The computer divides the dataset into training and testing sets, assuming a pre-defined ratio of 80% training set and 20% testing set. The computer performs the division based on the percentage of each category in the training and testing sets relative to the original sample percentages (i.e., 70% category A, 20% category B, and 10% category C). Specifically, the training set should contain 800 samples. The computer will draw 560 samples (70%) from category A (800 × 70%), 160 samples (20%) from category B (800 × 20%), and 80 samples (10%) from category C (800 × 10%). The test set should contain 200 samples. The computer will similarly draw 140 samples from category A, 40 from category B, and 20 from category C, according to the same 70%, 20%, and 10% ratios. After the partitioning, in the training set, category A will account for 560 / 800 = 70%, category B for 160 / 800 = 20%, and category C for 80 / 800 = 10%; in the test set, category A will account for 140 / 200 = 70%, category B for 40 / 200 = 20%, and category C for 20 / 200 = 10%. The proportion of each category in both the training and test sets is consistent with the proportion of samples in the original sample dataset.

[0144] S604 trains multiple initial parameter prediction models based on the training set and the test set to obtain preset parameter prediction models corresponding to each of the multiple kernel parameters.

[0145] Training refers to optimizing the parameters of the initial parameter prediction model using samples in the training set, enabling the model to learn the mapping relationship between input features and kernel parameters. The test set is used to validate the trained model and evaluate its predictive performance. The pre-defined parameter prediction model refers to a model that, after training and validation, can be used to predict the kernel parameters of matrix operation tasks.

[0146] In this embodiment, after obtaining the training and test sets, the computer device can train multiple initial parameter prediction models based on the training and test sets. Specifically, the computer device uses scale information samples and hardware information samples from the training set as input, and the corresponding optimal kernel parameters as labels, to iteratively train each initial parameter prediction model, optimizing the model's internal parameters so that the model learns the mapping relationship between input features and kernel parameters. During training, the computer device can periodically use the test set to validate the model, evaluate the model's prediction accuracy, and adjust the training strategy or model hyperparameters based on the validation results. After completing training, the computer device obtains multiple preset parameter prediction models corresponding to each kernel parameter, which can be used to predict kernel parameters for new matrix operation tasks.

[0147] For example, such as Figure 10 As shown in the diagram, the inputs to multiple initial parameter prediction models include GEMM Size (i.e., scale information, including M, N, K, B) and hardware information. Each initial parameter prediction model is an independent random forest sub-model, and each sub-model corresponds to a kernel parameter to be predicted. Each sub-model consists of multiple decision trees (tree1, tree2, etc.). The final predicted value of the parameter is obtained by combining the results of the decision trees through voting. The output of each initial parameter prediction model is the corresponding kernel parameters (MT_x, MT_y, MT_z, TT_x, TT_y, WG_x, WG_y, VW, WGM, GSU). These output parameters are combined with the input GEMM Size to ultimately select the Best Kernel. The Database on the left is split into a Training Set and a Test Set for training and evaluating the random forest sub-models.

[0148] During model training, a grid search algorithm is used to fine-tune the hyperparameters of each random forest sub-model to achieve optimal performance. Specifically, a comprehensive search and evaluation is conducted within a predefined parameter space for key hyperparameters such as the number of decision trees (n_estimators), maximum depth (max_depth), minimum number of sample splits (min_samples_split), and minimum number of leaf nodes (min_samples_leaf). By comparing the performance metrics of the model under different parameter combinations, an optimal set of hyperparameter configurations is determined for each sub-model targeting a specific variable. After determining the optimal hyperparameters, each sub-model is trained on the training set to ensure that the model can learn the complex relationship between input features and target parameters, thereby achieving accurate prediction of the optimal kernel parameters for GEMM computation.

[0149] The method described in this application process the sample dataset based on the category of the optimal kernel parameters, and uses a hierarchical sampling strategy to obtain the training set and the test set. This ensures that both the training set and the test set contain the optimal kernel parameters for all categories, making the category distribution of the training set consistent with that of the original dataset. This effectively alleviates the category identification bias problem caused by the long-tailed distribution of the optimal kernel parameter data, and improves the prediction accuracy of the model.

[0150] In some embodiments, Figure 6 Based on this, a method for processing sample datasets is also provided, such as... Figure 11 As shown, the processing method includes:

[0151] S701, determine whether the sample dataset meets the preset parameter constraints.

[0152] Among them, the preset parameter constraints include various types of constraints, specifically the preset parameter constraints in S301, which include at least one of preset mathematical relations, preset mapping relations, preset value conditions, and preset association relations.

[0153] In this embodiment, after acquiring the sample dataset, the computer device preprocesses the dataset, first determining whether each sample in the dataset satisfies preset parameter constraints. Specifically, the computer device compares the kernel parameters corresponding to each sample in the dataset with the preset parameter constraints one by one to determine whether the kernel parameters in each sample meet the requirements of various constraints. Optionally, the computer device can verify all samples sequentially according to the type of constraint. Optionally, the computer device can verify multiple samples simultaneously in parallel to improve processing efficiency. Based on the verification results, the computer device determines whether each sample in the dataset satisfies the preset parameter constraints and records the samples that do not meet the constraints.

[0154] S702, remove sample data that does not meet the preset parameter constraints to obtain a new sample dataset.

[0155] The new sample dataset refers to the set of samples that, after the elimination process, all samples meet the preset parameter constraints.

[0156] In this embodiment, after identifying samples in the sample dataset that do not meet the preset parameter constraints, the computer device removes these samples from the sample dataset. Specifically, the computer device can remove all samples marked as not meeting the constraints from the sample dataset based on the verification results, and can reconstruct a new sample dataset based on the remaining samples after removal. Optionally, during the removal process, the computer device can record the number and category distribution of the removed samples for subsequent data quality analysis. After the computer device completes the removal operation, it obtains a new sample dataset in which all samples satisfy the preset parameter constraints, ensuring the consistency and rationality of the data subsequently used for model training.

[0157] Correspondingly, when the computer device executes "processing the sample dataset based on the category of the optimal kernel parameters to obtain the training set and test set" in S603, it specifically executes S703: processing the new sample dataset based on the category of the optimal kernel parameters to obtain the training set and test set.

[0158] In this embodiment, after obtaining a new sample dataset, the computer device processes the dataset based on the category of the optimal kernel parameters to obtain a training set and a test set. Specifically, the computer device uses the category of the optimal kernel parameters as the partitioning basis and allocates samples in the new sample dataset to the training set or test set according to a preset partitioning strategy. During the partitioning process, the computer device ensures that the optimal kernel parameter samples of each category are reflected in both the training set and the test set, so that both the training set and the test set cover the optimal kernel parameters of all categories. Optionally, the computer device can adjust the sampling method according to the quantity distribution of samples of each category in the new sample dataset to ensure that the training set and the test set can reflect the category distribution characteristics of the dataset. After processing, the computer device obtains a training set for model training and a test set for model validation.

[0159] The method described in this application embodiment, by determining whether the sample dataset meets the preset parameter constraints, can identify invalid samples in the sample dataset that do not meet the requirements of the hardware platform and parallel computing architecture, and remove sample data that does not meet the preset parameter constraints, thereby ensuring that the sample data used for model training meets the inherent constraint relationship between kernel parameters in terms of physical feasibility and logical rationality, and avoiding interference to model training caused by invalid data.

[0160] In some embodiments, Figure 6 Based on this, a method for preprocessing sample data in the training and test sets is also provided, such as... Figure 12 As shown, the preprocessing method includes:

[0161] S801, determine the data type of the sample data in the training and test sets; data types include numerical and categorical types.

[0162] In this context, data type is used to characterize the attribute category of each feature field in the sample data. Numerical features refer to features that take continuous or discrete values, such as matrix dimension or number of computational units; categorical features refer to features that take values ​​of a finite number of discrete categories, such as GPU model.

[0163] In this embodiment, after obtaining the training set and the test set, the computer device identifies the data type of the sample data in the training set and the test set to determine the data type of each sample data. Specifically, the computer device can parse the value format of each feature field in the training set and the test set, and determine whether the field belongs to the numerical type or the categorical type based on the distribution characteristics of the values. For example, for features with continuous numerical values ​​or ordered discrete numerical values, the computer device identifies them as numerical; for features with only a finite number of fixed labels and no order between the labels, the computer device identifies them as categorical.

[0164] S802, preprocesses the sample data in the training and test sets according to the data type to obtain the preprocessed training set and the standardized test set.

[0165] In this embodiment, after determining the data types of the sample data in the training and test sets, the computer device preprocesses the sample data according to the data types. Specifically, the computer device adopts corresponding preprocessing methods for numerical features and categorical features respectively, converting the original feature values ​​into a form suitable for model input.

[0166] Specifically, such as Figure 13 As shown, the above S802 includes:

[0167] S8021, if the data type is numeric, then the sample data is numerically standardized.

[0168] Numerical standardization refers to converting the data distribution of numerical features into a standard normal distribution with a mean of 0 and a standard deviation of 1, in order to eliminate the inconsistency in numerical ranges between different features due to differences in units.

[0169] In this embodiment, if the computer device determines that the data type of the sample data is numerical, then it performs numerical standardization on the sample data. Specifically, the computer device calculates the mean and standard deviation of the numerical feature in the training set, and then uses the mean and standard deviation to standardize the corresponding features in the training and test sets, so that the transformed feature data satisfies the distribution characteristics of a mean of 0 and a standard deviation of 1. Through numerical standardization, the computer device ensures that numerical features of different dimensions are comparable during model training, avoids model training bias caused by excessive differences in the range of feature values, and ensures that each numerical feature has equal influence during model learning.

[0170] S8022, if the data type is categorical, then perform one-hot encoding on the sample data.

[0171] One-hot encoding refers to an encoding method that converts categorical features into binary vectors. Each category corresponds to an independent binary bit, which is 1 while the rest are 0. This method is used to convert discrete category labels into a numerical representation suitable for model input.

[0172] In this embodiment, if the computer device determines that the data type of the sample data is categorical, it performs one-hot encoding on the sample data. Specifically, the computer device identifies all possible values ​​of the categorical feature, assigns an independent binary bit to each value, and then converts the category label in the original sample into a corresponding binary vector representation. Through one-hot encoding, the computer device transforms discrete category information into a numerical vector, enabling the model to better capture the potential correlation between categorical features and target parameters.

[0173] Correspondingly, when the computer device executes S604, which "trains multiple initial parameter prediction models based on the training set and the test set to obtain preset parameter prediction models corresponding to each of the multiple kernel parameters", it specifically executes S803: "trains multiple initial parameter prediction models based on the preprocessed training set and the preprocessed test set to obtain preset parameter prediction models corresponding to each of the multiple kernel parameters".

[0174] In this embodiment, after obtaining a preprocessed training set and a preprocessed test set, the computer device trains multiple initial parameter prediction models based on these two datasets. During training, the computer device periodically validates the models using the preprocessed test set, evaluates the prediction accuracy of the models, and adjusts the training strategy or model hyperparameters based on the validation results. After completing training, the computer device obtains multiple preset parameter prediction models corresponding to each kernel parameter. These models can be used to predict kernel parameters for new matrix operation tasks.

[0175] The method described in this application eliminates the dimensional differences between different features by performing numerical standardization on numerical sample data, ensuring that each numerical feature has equal influence during model learning, thereby improving the stability and accuracy of model training. By performing one-hot encoding on categorical sample data, discrete category labels are converted into numerical vector representations suitable for model input, enabling the model to effectively utilize information from categorical features, thus improving the model's ability to model categorical features and its prediction accuracy.

[0176] In summary, based on all the above embodiments, a parameter prediction method is also provided, such as... Figure 14 As shown, the method includes:

[0177] S901, Obtain the sample dataset. The sample dataset includes multiple scale information samples, multiple hardware information samples, and optimal kernel parameters for multiple categories.

[0178] S902, determine whether the sample dataset meets the preset parameter constraints, and remove the sample data that does not meet the preset parameter constraints to obtain a new sample dataset. The preset parameter constraints include various types of constraints.

[0179] S903 deconstructs the dimensions of multi-dimensional parameters in the kernel parameters to obtain parameters with multiple dimensions.

[0180] S904 obtains the initial parameter prediction models corresponding to parameters in multiple dimensions, as well as the initial parameter prediction models corresponding to other parameters in the kernel parameters besides the multi-dimensional parameters, and serves as the initial parameter prediction models corresponding to the multiple kernel parameters of the accelerator.

[0181] S905, obtain the percentage of the number of new samples in the sample dataset that represent the optimal kernel parameters for each category.

[0182] S906 divides the new sample dataset into training and testing sets with the goal of determining the proportion of the optimal kernel parameters for each category in the training and testing sets.

[0183] S907, determine the data type of the sample data in the training set and the test set. If the data type is numerical, perform numerical standardization on the sample data. If the data type is categorical, perform one-hot encoding on the sample data to obtain the preprocessed training set and the standardized test set.

[0184] S908 trains multiple initial parameter prediction models based on the preprocessed training set and the preprocessed test set to obtain the preset parameter prediction models corresponding to each of the multiple kernel parameters.

[0185] S909 obtains information about the scale of the matrix operation task and the hardware information of the accelerator running the matrix operation task.

[0186] S910 inputs scale information and hardware information into the preset parameter prediction models corresponding to the multiple kernel parameters of the accelerator for prediction, and obtains the kernel parameters output by each parameter prediction model.

[0187] S911 summarizes the kernel parameters output by the prediction models of each parameter to obtain the initial kernel configuration parameters corresponding to the matrix operation task.

[0188] S12, obtain the preset parameter constraints and determine whether the initial kernel configuration parameters meet all the constraints.

[0189] S913 If the initial kernel configuration parameters do not meet any constraint, the initial kernel configuration parameters are corrected according to the type of the unmet constraint to obtain the kernel configuration parameter prediction result.

[0190] The method described in this application integrates data analysis and artificial intelligence algorithms to achieve intelligent selection of optimal performance kernel parameters for different matrix operation tasks. This not only significantly improves the performance of matrix operation tasks but also reduces the workload of manual tuning, thereby increasing development efficiency. Based on preset parameter constraints, the dataset is screened during the data preprocessing stage to ensure that the data fully meets the preset constraints, providing a high-quality data foundation for subsequent model training. Simultaneously, after the model completes prediction, a post-processing stage is added to comprehensively correct and verify the prediction results based on the preset parameter constraints, thereby ensuring the rationality and reliability of the prediction results.

[0191] This embodiment significantly improves the prediction efficiency and accuracy of the optimal performance kernel under different GEMM scales by integrating data analysis and random forest algorithms. It realizes intelligent optimization of GEMM computing performance on a domestic accelerator card platform, with the following specific effects:

[0192] (1) By jointly modeling GEMM Size and hardware information, the random forest model achieves an average accuracy of about 85% in predicting key kernel parameters, as shown in Table 1 below.

[0193] Table 1

[0194]

[0195] (2) In view of the long-tail distribution characteristics of the optimal kernel data (the main cluster accounts for 14% and its sample size accounts for more than 80%), the hierarchical sampling strategy ensures that the class distribution of the training set and the test set is consistent with the original data, which effectively alleviates the model's identification bias of sparse clusters.

[0196] (3) It replaces the traditional heuristic search "try one by one" mode. Through parallel prediction and grid search optimization of the random forest model, the tuning time is shortened from "hours" to "minutes" to meet the needs of real-time scenarios.

[0197] The methods described in each of the above steps have been described in the foregoing embodiments. For details, please refer to the foregoing descriptions. They will not be repeated here.

[0198] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0199] Based on the same inventive concept, this application also provides a parameter prediction apparatus for implementing the parameter prediction method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more parameter prediction apparatus embodiments provided below can be found in the limitations of the parameter prediction method described above, and will not be repeated here.

[0200] In some embodiments, such as Figure 15 As shown, a parameter prediction device is provided, comprising:

[0201] The first acquisition module 11 is used to acquire the scale information of the matrix operation task and the hardware information of the accelerator running the matrix operation task.

[0202] The prediction module 12 is used to predict each kernel parameter according to the scale information, the hardware information and the preset parameter prediction models corresponding to the multiple kernel parameters of the accelerator, so as to obtain the initial kernel configuration parameters corresponding to the matrix operation task; wherein, each parameter prediction model is trained based on the training set and the test set composed of the optimal kernel parameters, and the training set and the test set contain the optimal kernel parameters of all categories.

[0203] The first determining module 13 is used to determine the kernel configuration parameter prediction result based on the initial kernel configuration parameters.

[0204] In some embodiments, the determining module includes:

[0205] The first acquisition unit is used to acquire preset parameter constraints; the preset parameter constraints include various types of constraints.

[0206] The correction unit is used to correct the initial kernel configuration parameters according to preset parameter constraints to obtain the kernel configuration parameter prediction results.

[0207] In some embodiments, the above-mentioned correction unit includes:

[0208] The determination subunit is used to determine whether the initial kernel configuration parameters meet all constraints.

[0209] The correction subunit is used to correct the initial kernel configuration parameters according to the type of the unmet constraint if the initial kernel configuration parameters do not meet any constraint, and to obtain the kernel configuration parameter prediction result.

[0210] In some embodiments, the prediction module described above includes:

[0211] The prediction unit is used to input scale information and hardware information into the preset parameter prediction models corresponding to the multiple kernel parameters of the accelerator for prediction, and obtain the kernel parameters output by each parameter prediction model.

[0212] The summarization unit is used to summarize the kernel parameters output by each parameter prediction model to obtain the initial kernel configuration parameters corresponding to the matrix operation task.

[0213] In some embodiments, the parameter prediction device further includes:

[0214] The second acquisition module is used to acquire the sample dataset; the sample dataset includes multiple scale information samples, multiple hardware information samples, and optimal kernel parameters for multiple categories.

[0215] The third acquisition module is used to acquire the initial parameter prediction models corresponding to each of the multiple kernel parameters of the accelerator.

[0216] The processing module is used to process the sample dataset based on the category with optimal kernel parameters to obtain the training set and the test set.

[0217] The training module is used to train multiple initial parameter prediction models based on the training set and the test set, so as to obtain the preset parameter prediction models corresponding to each of the multiple kernel parameters.

[0218] In some embodiments, the third acquisition module described above includes:

[0219] The destructuring unit is used to deconstruct the dimensions of multi-dimensional parameters in the kernel parameters to obtain parameters with multiple dimensions.

[0220] The second acquisition unit is used to acquire the initial parameter prediction model corresponding to parameters of multiple dimensions, as well as the initial parameter prediction model corresponding to other parameters in the kernel parameters besides the multi-dimensional parameters, which serve as the initial parameter prediction models corresponding to the multiple kernel parameters of the accelerator.

[0221] In some embodiments, the above-described processing module includes:

[0222] The third acquisition unit is used to obtain the percentage of the number of samples of the optimal kernel parameters for each category in the sample dataset.

[0223] The partitioning unit is used to divide the sample dataset into training and testing sets, with the proportion of the optimal kernel parameters for each category in the training and testing sets as the sample proportion.

[0224] In some embodiments, the parameter prediction device further includes:

[0225] The second determination module is used to determine whether the sample dataset meets the preset parameter constraints.

[0226] The module is used to remove sample data that does not meet the preset parameter constraints and obtain a new sample dataset.

[0227] Correspondingly, the aforementioned processing module is specifically used to process the new sample dataset based on the category of the optimal kernel parameters to obtain the training set and the test set.

[0228] In some embodiments, the parameter prediction device further includes:

[0229] The third determination module is used to determine the data type of the sample data in the training set and the test set; the data types include numerical and categorical types.

[0230] The preprocessing module is used to preprocess the sample data in the training and test sets according to the data type, so as to obtain the preprocessed training set and the standardized test set.

[0231] Correspondingly, the aforementioned training module is specifically used to train multiple initial parameter prediction models based on the preprocessed training set and the preprocessed test set, thereby obtaining preset parameter prediction models corresponding to each of the multiple kernel parameters.

[0232] In some embodiments, the preprocessing module described above includes:

[0233] The first processing unit is used to perform numerical standardization on the sample data if the data type is numeric.

[0234] The second processing unit is used to perform one-hot encoding on the sample data if the data type is categorical.

[0235] Each module in the aforementioned parameter prediction device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0236] In some embodiments, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the parameter prediction method described in any of the above embodiments.

[0237] In some embodiments, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the parameter prediction method described in any of the above embodiments.

[0238] In some embodiments, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the parameter prediction method described in any of the above embodiments.

[0239] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0240] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0241] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A parameter prediction method, characterized in that, The method includes: Obtain the scale information of the matrix operation task and the hardware information of the accelerator running the matrix operation task; Based on the scale information, the hardware information, and the preset parameter prediction models corresponding to the multiple kernel parameters of the accelerator, each kernel parameter is predicted to obtain the initial kernel configuration parameters corresponding to the matrix operation task; wherein, each parameter prediction model is trained on a training set and a test set composed of optimal kernel parameters, and the training set and the test set contain the optimal kernel parameters of all categories. Based on the initial kernel configuration parameters, the predicted kernel configuration parameters are determined.

2. The method according to claim 1, characterized in that, The step of determining the kernel configuration parameter prediction result based on the initial kernel configuration parameters includes: Obtain preset parameter constraints; the preset parameter constraints include various types of constraints. The initial kernel configuration parameters are corrected according to preset parameter constraints to obtain the predicted kernel configuration parameters.

3. The method according to claim 2, characterized in that, The step of correcting the initial kernel configuration parameters according to preset parameter constraints to obtain the kernel configuration parameter prediction result includes: Determine whether the initial kernel configuration parameters meet all constraints; If the initial kernel configuration parameters do not satisfy any of the constraints, the initial kernel configuration parameters are corrected according to the type of the unsatisfied constraint to obtain the kernel configuration parameter prediction result.

4. The method according to any one of claims 1-3, characterized in that, The step of predicting each kernel parameter based on the scale information, the hardware information, and the preset parameter prediction models corresponding to the multiple kernel parameters of the accelerator to obtain the initial kernel configuration parameters corresponding to the matrix operation task includes: The scale information and the hardware information are respectively input into the preset parameter prediction models corresponding to the multiple kernel parameters of the accelerator for prediction, so as to obtain the kernel parameters output by each parameter prediction model. The kernel parameters output by each parameter prediction model are summarized to obtain the initial kernel configuration parameters corresponding to the matrix operation task.

5. The method according to any one of claims 1-3, characterized in that, The method further includes: Obtain a sample dataset; the sample dataset includes multiple scale information samples, multiple hardware information samples, and optimal kernel parameters for multiple categories; Obtain the initial parameter prediction model corresponding to each of the multiple kernel parameters of the accelerator; Based on the category of the optimal kernel parameters, the sample dataset is processed to obtain the training set and the test set; Based on the training set and the test set, multiple initial parameter prediction models are trained to obtain preset parameter prediction models corresponding to each of the multiple kernel parameters.

6. The method according to claim 5, characterized in that, The step of obtaining the initial parameter prediction model corresponding to each of the multiple kernel parameters of the accelerator includes: The dimensions of the multi-dimensional parameters in the kernel parameters are deconstructed to obtain parameters with multiple dimensions; The initial parameter prediction models corresponding to the parameters of the multiple dimensions and the initial parameter prediction models corresponding to the other parameters in the kernel parameters besides the multi-dimensional parameters are obtained as the initial parameter prediction models corresponding to the multiple kernel parameters of the accelerator.

7. The method according to claim 5, characterized in that, The process of processing the sample dataset based on the category of the optimal kernel parameters to obtain the training set and the test set includes: The percentage of the number of samples in the sample dataset that yields the optimal kernel parameters for each category. The sample dataset is divided into the training set and the test set, with the proportion of the optimal kernel parameters for each category in the training set and the test set as the objective.

8. The method according to claim 5, characterized in that, The method further includes: Determine whether the sample dataset meets the preset parameter constraints; Sample data that does not meet the preset parameter constraints are removed to obtain a new sample dataset; The process of processing the sample dataset based on the category of the optimal kernel parameters to obtain the training set and the test set includes: Based on the category of the optimal kernel parameters, the new sample dataset is processed to obtain the training set and the test set.

9. The method according to claim 5, characterized in that, The method further includes: Determine the data types of the sample data in the training set and the test set; the data types include numerical and categorical types; The sample data in the training set and the test set are preprocessed according to the data type to obtain the preprocessed training set and the standardized test set. The step of training multiple initial parameter prediction models based on the training set and the test set to obtain preset parameter prediction models corresponding to each of the multiple kernel parameters includes: Based on the preprocessed training set and the preprocessed test set, multiple initial parameter prediction models are trained to obtain preset parameter prediction models corresponding to each of the multiple kernel parameters.

10. The method according to claim 9, characterized in that, The preprocessing of sample data in the training set and the test set according to the data type includes: If the data type is numerical, then the sample data is subjected to numerical standardization. If the data type is the categorical type, then the sample data is subjected to one-hot encoding.

11. A parameter prediction device, characterized in that, The device includes: The first acquisition module is used to acquire the scale information of the matrix operation task and the hardware information of the accelerator running the matrix operation task; The prediction module is used to predict each kernel parameter based on the scale information, the hardware information, and the preset parameter prediction models corresponding to the multiple kernel parameters of the accelerator, so as to obtain the initial kernel configuration parameters corresponding to the matrix operation task; wherein, each parameter prediction model is trained based on a training set and a test set composed of optimal kernel parameters, and the training set and the test set contain the optimal kernel parameters of all categories. The first determining module is used to determine the kernel configuration parameter prediction result based on the initial kernel configuration parameters.

12. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

Images