Performance parameter tuning sequence determination method and device, equipment and medium

Abstract

The invention relates to the technical field of computers, and discloses a performance parameter tuning sequence determination method and device, equipment and a medium, and the method comprises the steps: carrying out the sampling of the performance parameters of a deep learning model, and obtaining a plurality of sampling point data; generating a configuration file based on the sampling point data; after the operation in the test environment is finished each time, obtaining a test result; generating a first-order sensitive index and a total sensitive index corresponding to each performance parameter based on a performance index generated by the performance parameters; and determining a tuning sequence of the performance parameters based on the first-order sensitive index corresponding to each performance parameter and the total sensitive index. Through sensitivity analysis of a scientific system, key parameters having significant influence on the reasoning performance of the deep learning model are accurately identified and determined. And the tuning sequence of the performance parameters is determined, so that the tuning process is more efficient. Parameter adjustment is carried out according to the tuning sequence, blind trying of different parameter combinations can be avoided, and tuning time and computing resources are saved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, equipment and medium for determining a performance parameter tuning sequence. Background Art

[0002] In recent years, deep learning research results have been widely used in many fields. However, as the scale and complexity of models continue to expand, the requirements for algorithm performance and efficiency are becoming higher and higher. It is crucial to accurately evaluate and optimize the reasoning ability of deep learning models. However, which specific parameters have a significant impact on performance and how to scientifically determine these key parameters is an urgent problem to be solved. Summary of the invention

[0003] In view of this, the present invention provides a method, device, equipment and medium for determining the order of performance parameter tuning to solve the problem in the related art that it is impossible to determine which parameters have a significant impact on model performance tuning.

[0004] In a first aspect, the present invention provides a method for determining a performance parameter tuning order, the method comprising:

[0005] Sampling the performance parameters of the deep learning model to obtain a plurality of sampling point data, each sampling point data being composed of a set of performance parameters of the deep learning model and a parameter value corresponding to each performance parameter;

[0006] Generate a configuration file based on the sampling point data, and the configuration file is used to run the sampling point data in the test environment in sequence;

[0007] After each run in the test environment is completed, the test results are obtained, wherein the test results include performance indicators of the deep learning model;

[0008] Based on the performance indicators generated by the performance parameters, a first-order sensitivity index corresponding to each performance parameter is generated, wherein the first-order sensitivity index is used to evaluate the influence of a single performance parameter on the model reasoning performance;

[0009] And, based on the performance indicators generated by the performance parameters, generate a total sensitivity index corresponding to all performance parameters, wherein the total sensitivity index is used to indicate the comprehensive influence of all parameters on the model reasoning performance;

[0010] Based on the first-order sensitivity index corresponding to each performance parameter and the total sensitivity index, the tuning order of the performance parameters is determined.

[0011] The method for determining the performance parameter tuning order provided by the present invention has the following advantages:

[0012] By acquiring and testing data from multiple sampling points, we can have a more comprehensive understanding of the performance of the model under different parameter configurations, thereby providing a more accurate performance evaluation. Based on the performance indicators generated by the performance parameters, the first-order sensitivity index and the total sensitivity index corresponding to each parameter are generated. For situations where you need to choose between different models or parameter configurations, the first-order sensitivity index and the total sensitivity index can provide decision support and help make a more informed choice. Through scientific and systematic sensitivity analysis, the key parameters that have a significant impact on the inference performance of the deep learning model can be accurately identified and determined. Then determine the tuning order of the performance parameters, that is, give priority to adjusting the parameters that have a greater impact on performance, so that the tuning process is more efficient. Moreover, adjusting the parameters according to the tuning order can avoid blindly trying different parameter combinations, thereby saving tuning time and computing resources. Moreover, by testing and analyzing the data from different sampling points, the optimal parameter configuration in different application scenarios can be determined to improve the generalization ability and versatility of the model.

[0013] In an optional implementation, the method further includes: generating a parameter matrix based on the sampling point data, wherein the performance parameters of the deep learning model in the sampling point data constitute column elements in the parameter matrix, parameter values ​​of the performance parameters constitute column elements in the parameter matrix, and the performance parameters include a batch size, a number of replication streams, a number of inference streams, a first identification information corresponding to whether to run inference on the replication stream, and a second identification information corresponding to whether to use graph optimization;

[0014] Based on the performance indicators, a performance indicator vector is generated, where the performance indicators include one or more of model inference speed, latency, and resource utilization.

[0015] Specifically, in order to ensure the accuracy and rationality of the data, a parameter matrix can be generated based on the sampling point data, and a performance indicator vector can be generated based on the performance indicator. Subsequently, corresponding parameters can be extracted from the parameter matrix, and corresponding performance indicator values ​​can be obtained from the performance indicator vector.

[0016] In an optional implementation, based on the performance indicators generated by the performance parameters, a first-order sensitivity index corresponding to each performance parameter is generated, specifically including:

[0017] Obtaining the index values ​​of multiple groups of performance indexes corresponding to the target performance parameters generated when the parameter values ​​of the target performance parameters are different in the multiple sampling points and the parameter values ​​of other parameters are the same, wherein the target performance parameter is any performance parameter in the sampling point data;

[0018] Extracting the index values ​​of multiple groups of performance indicators corresponding to the first parameter when the target performance parameter is the first parameter, and extracting the index values ​​of multiple groups of performance indicators corresponding to the second parameter when the target performance parameter is the second parameter, wherein the first parameter is the performance parameter to be calculated and the first-order sensitivity index, the second parameter is any parameter in the sampling point data except the first parameter, and the index values ​​of each group of performance indicators include the index values ​​corresponding to the multiple performance indicators respectively;

[0019] Determining a first variance value corresponding to the first parameter based on the index values ​​of the plurality of groups of performance indicators corresponding to the first parameter;

[0020] Determining a second variance value corresponding to the second parameter based on the index values ​​of the plurality of groups of performance indicators corresponding to the second parameter;

[0021] Determine an output variance of the influence of the second parameter on the first parameter based on the first variance value and the second variance value;

[0022] After determining the output variances of the effects of the performance parameters other than the first parameter in the sampling point data on the first parameter, a first-order sensitivity index corresponding to the first parameter is determined according to the first variance value and the output variances of the effects of the performance parameters other than the first parameter in the sampling point data on the first parameter.

[0023] Specifically, obtain the index values ​​of multiple groups of performance indicators corresponding to the target performance parameter when the parameter values ​​of the target performance parameter are different and the parameter values ​​of other parameters are the same in multiple sampling points. Then, extract the index values ​​of multiple groups of performance indicators corresponding to the first parameter when the target performance parameter is the first parameter, and extract the index values ​​of multiple groups of performance indicators corresponding to the second parameter when the target performance parameter is the second parameter. Among them, the index values ​​of each group of performance indicators include index values ​​corresponding to multiple performance indicators. Based on the index values ​​of multiple groups of performance indicators corresponding to the first parameter, determine the first variance value; and based on the index values ​​of multiple groups of performance indicators corresponding to the second parameter, determine the second variance value; finally, give the first variance and the second variance to determine the output variance of the influence of the second parameter on the first parameter. Finally, based on all the influence output variances, the first-order sensitivity index corresponding to the first parameter. Through the above detailed calculation process, the degree of influence of each performance parameter on the first parameter can be determined more accurately, thereby providing a more accurate first-order sensitivity index. This method comprehensively evaluates the interaction between performance parameters.

[0024] In an optional implementation, based on the performance indicators generated by the performance parameters, a total sensitivity index corresponding to all performance parameters is generated, specifically including:

[0025] Obtaining the index values ​​of multiple groups of performance indexes corresponding to the target performance parameters generated when the parameter values ​​of the target performance parameters are different in the multiple sampling points and the parameter values ​​of other parameters are the same, wherein the target performance parameter is any performance parameter in the sampling point data;

[0026] Determine a comprehensive variance value based on the index values ​​of multiple groups of performance indexes corresponding to each performance parameter included in all sampling point data;

[0027] Determine the output variance value based on the index values ​​of multiple performance indexes corresponding to other performance parameters except any performance parameter included in each sampling point data;

[0028] After the output variance values ​​are determined based on the index values ​​of multiple performance indicators corresponding to other performance parameters in the sampling point data excluding each performance parameter in the sampling point data, the total sensitivity index is determined according to the comprehensive variance value and the output variance values ​​of the index values ​​of multiple performance indicators corresponding to other performance parameters in the sampling point data excluding each performance parameter in the sampling point data.

[0029] Specifically, the index values ​​of multiple groups of performance indicators corresponding to the target performance parameters are obtained when the parameter values ​​of the target performance parameters are different in multiple sampling points and the parameter values ​​of other parameters are the same. Then, the comprehensive variance value is determined based on the index values ​​of the multiple groups of performance indicators corresponding to each performance parameter included in all sampling point data. And after excluding any parameters from the performance parameters included in all sampling point data, the output variance values ​​corresponding to the parameter values ​​of other parameters are calculated to reflect the output variance caused by the changes of other parameters after excluding a certain parameter. And the total sensitivity index is calculated based on the results obtained in these two steps. This method can comprehensively consider the impact of all parameters on the performance indicators and provide a more comprehensive evaluation. Moreover, this method also helps to discover the potential relationship and interaction between parameters, providing deeper insights for the optimization of the model. Determining the total sensitivity index can guide the adjustment and optimization of parameters to maximize the performance of the model. Comprehensively considering the impact of all parameters reduces the workload of analyzing each parameter separately and improves the efficiency of decision-making. Understanding the total sensitivity index helps to better explain the behavior and performance of the model and increase the comprehensibility of the model. Discovering the relationship between parameters can provide direction for model improvement, inspire new optimization strategies and methods suitable for dealing with complex situations with multiple parameters and performance indicators, and can cope with diverse needs in practical applications. By optimizing parameters, it helps to improve the stability and reliability of the model and reduce performance fluctuations. It can be applied to various deep learning models and tasks, and has wide applicability and practical value.

[0030] In an optional implementation, based on the performance indicators generated by the performance parameters, a first-order sensitivity index corresponding to each performance parameter is generated, which is specifically expressed by the following formula:

[0031]

[0032] Among them, V(Y) is the variance value, n is the number of sampling point data, Y i is the i-th indicator value corresponding to any performance indicator in multiple groups of performance indicators, The average value calculated from the indicator values ​​of all performance indicators in multiple groups of performance indicators;

[0033] V j =V(Y) 第一参数 - V(Y) 第二参数 (Formula 2)

[0034] Among them, V j V(Y) is the output variance of the effect of the second parameter on the first parameter. 第一参数 To calculate the first variance value using formula 1, V(Y) 第二参数 is the second variance value calculated by formula 1;

[0035]

[0036] Where h is the total number of performance parameters of the deep learning model in the sampling point data, S1 is the sum of the output variances of the effects of the performance parameters other than the first parameter in the sampling point data on the first parameter. i is the first-order sensitivity index corresponding to the first parameter.

[0037] In an optional implementation, based on the performance indicators generated by the performance parameters, a total sensitivity index corresponding to all performance parameters is generated, which is specifically expressed by the following formula:

[0038]

[0039] Among them, V d (Y) is the comprehensive variance value, d is the total number of indicator values ​​corresponding to multiple groups of performance indicators, and Y q is the qth data among the d data, is the average value calculated for the values ​​corresponding to d data;

[0040]

[0041] Among them, V p(Y) is the output variance value determined for the parameter values ​​of other performance parameters except the third parameter included in each sampling point data, k=dd / n, n is the number of sampling points, Y pi is the i-th performance indicator value corresponding to the p-th performance parameter among the n-1 performance parameters excluding the third parameter, is an average value calculated based on n indicator values ​​of the ith performance indicator corresponding to the pth performance parameter, wherein the third parameter is any one of the performance parameters included in the sampling point data;

[0042]

[0043] in, It is the sum of the output variance values ​​determined by the index values ​​of various performance indicators corresponding to other performance parameters in the sampling point data. i The total sensitivity index corresponding to all performance parameters.

[0044] In an optional example, the test result also includes a test log, and the method further includes:

[0045] Determine whether there is abnormal data in the performance indicator according to the test log, wherein the abnormal data includes data whose indicator value of the performance indicator is not within the corresponding indicator value range;

[0046] When it is determined that abnormal data exists, the test is performed again based on the sampling point data corresponding to the abnormal data.

[0047] In a second aspect, the present invention provides a device for determining a performance parameter tuning order, the device comprising:

[0048] A sampling module is used to sample the performance parameters of the deep learning model and obtain multiple sampling point data, each of which is composed of a set of performance parameters of the deep learning model and a parameter value corresponding to each performance parameter;

[0049] A generation module, used to generate a configuration file based on the sampling point data, and the configuration file is used to run the sampling point data in sequence in the test environment;

[0050] A running module is used to obtain test results after each run in the test environment, wherein the test results include performance indicators of the deep learning model;

[0051] A processing module, used for generating a first-order sensitivity index corresponding to each performance parameter based on the performance index generated by the performance parameter, wherein the first-order sensitivity index is used to evaluate the influence of a single performance parameter on the model reasoning performance;

[0052] And, based on the performance indicators generated by the performance parameters, generate a total sensitivity index corresponding to all performance parameters, wherein the total sensitivity index is used to indicate the comprehensive influence of all parameters on the model reasoning performance;

[0053] The determination module is used to determine the tuning order of the performance parameters based on the first-order sensitivity index corresponding to each performance parameter and the total sensitivity index.

[0054] The device for determining the performance parameter tuning order provided by the present invention has the following advantages:

[0055] By acquiring and testing data from multiple sampling points, we can have a more comprehensive understanding of the performance of the model under different parameter configurations, thereby providing a more accurate performance evaluation. Based on the performance indicators generated by the performance parameters, the first-order sensitivity index and the total sensitivity index corresponding to each parameter are generated. For situations where you need to choose between different models or parameter configurations, the first-order sensitivity index and the total sensitivity index can provide decision support and help make a more informed choice. Through scientific and systematic sensitivity analysis, the key parameters that have a significant impact on the inference performance of the deep learning model can be accurately identified and determined. Then determine the tuning order of the performance parameters, that is, give priority to adjusting the parameters that have a greater impact on performance, so that the tuning process is more efficient. Moreover, adjusting the parameters according to the tuning order can avoid blindly trying different parameter combinations, thereby saving tuning time and computing resources. Moreover, by testing and analyzing the data from different sampling points, the optimal parameter configuration in different application scenarios can be determined to improve the generalization ability and versatility of the model.

[0056] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the performance parameter tuning order determination method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0057] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method for determining a performance parameter tuning order according to the first aspect or any corresponding embodiment thereof.

[0058] In a fifth aspect, the present invention provides a computer program product, including computer instructions, which are used to enable a computer to execute the method for determining the performance parameter tuning order of the above-mentioned first aspect or any corresponding embodiment. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0060] Figure 1 It is a flowchart of a method for determining a performance parameter tuning order provided by an embodiment of the present invention;

[0061] Figure 2 It is a flowchart of another method for determining a performance parameter tuning order provided by an embodiment of the present invention;

[0062] Figure 3 It is a flowchart of another method for determining a performance parameter tuning order provided by an embodiment of the present invention;

[0063] Figure 4 It is a schematic diagram of the overall process of the method for determining the performance parameter tuning order provided by an embodiment of the present invention;

[0064] Figure 5 It is a structural block diagram of a device for determining a performance parameter tuning order provided by an embodiment of the present invention;

[0065] Figure 6 It is a schematic diagram of the hardware structure of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0067] In recent years, deep learning research results have been widely used in many fields. However, as the scale of models continues to expand and the complexity increases, the requirements for algorithm performance and efficiency are also getting higher and higher. Accurately evaluating and optimizing the reasoning ability of deep learning models has become an issue that needs attention.

[0068] The MLPerf benchmark has been widely adopted as an important tool for evaluating the inference performance of AI models on different hardware platforms. In practical applications, the inference speed and resource usage of deep learning models are usually closely related to the model's multiple parameter settings.

[0069] In related technologies, these parameters are usually adjusted through experience and trial and error, which is inefficient and difficult to ensure optimal performance on different hardware platforms. Especially when using the MLPerf benchmark to evaluate the inference performance of deep learning models, scientifically and systematically identifying and determining the key parameters that affect their performance is crucial to improving the model's inference speed and resource utilization efficiency. However, which specific parameters have a significant impact on performance and how to scientifically determine these key parameters is an urgent problem to be solved.

[0070] To solve the above problems, an embodiment of the present invention provides an embodiment for determining the order of performance parameter tuning. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system (computer device) including, for example, a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0071] In this embodiment, a method for determining the order of performance parameter tuning is provided, which can be used for the above-mentioned terminal devices, such as mobile phones, tablet computers, etc. In a specific example, this application can be applied to the MLPerf (an alliance of AI leaders from academia, research laboratories, and related industries, aiming to "build fair and useful benchmarks" and provide fair evaluations of the training and reasoning performance of hardware, software, and services under specified conditions) benchmark test. In order to accurately determine and quantify the key parameters that have a significant impact on the model performance during the MLPerf reasoning process, the key parameters are determined by the method of this application, and then the priority adjustment order of the performance parameters during the model tuning process is determined.

[0072] Figure 1 FIG. 1 is a flow chart of a method for determining a performance parameter tuning order provided by an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0073] Step S101, sampling the deep learning model performance parameters to obtain multiple sampling point data.

[0074] Specifically, each sampling point data consists of a set of performance parameters of the deep learning model and the parameter value corresponding to each performance parameter.

[0075] In an optional example, all parameters that may affect the performance during the deep learning model inference process, including but not limited to batch size, the number of copy streams, the number of infer streams, whether to run infer on copy streams, and whether to use graph optimization.

[0076] In this embodiment, a Sobol sequence is used to generate sampling points covering the entire parameter space. The Sobol sequence is a quasi-random sequence that can effectively explore high-dimensional parameter space. The goal of sampling is to generate a set of parameter combinations that can be used in subsequent benchmark tests to evaluate their impact on the inference performance of deep learning models.

[0077] In a specific example, the performance parameters include all the parameters described above.

[0078] First, organize the determined parameter names and value ranges into a formatted input for use by the Sobol sequence generator. Define the following parameters and value ranges:

[0079] Batch size: 8 to 128

[0080] Number of copy streams: 1 to 8

[0081] Number of infer streams: 1 to 8

[0082] Whether to run infer on copy streams: 0 or 1

[0083] Whether to use graph optimization (use graph): 0 or 1

[0084] Then, the Sobol sequence is used to perform parameter sampling and generate multiple sampling points. The specific steps include:

[0085] 1) Initialize the Sobol sequence generator:

[0086] The sampling process is initialized using the Sobol sequence generator. The Sobol sequence is a quasi-random sequence that can efficiently sample in high-dimensional parameter space and ensure that the sampling points are evenly distributed throughout the parameter space.

[0087] 2) Generate sampling points:

[0088] Based on the number of parameters and the predetermined number of sampling points, the sampling points are generated using a Sobol sequence. Each sampling point is a parameter combination. The Sobol sequence generator generates a set of sampling points that are uniformly distributed between 0 and 1.

[0089] 3) Map sampling points to parameter value ranges:

[0090] Map the sampling points generated by the Sobol sequence to the actual parameter value range.

[0091] For example, for a batch size ranging from 8 to 128, Sobol sequence values ​​between 0 and 1 are linearly converted to integer values ​​within the range. Similarly, for Boolean parameters, Sobol sequence values ​​between 0 and 1 are converted to 0 or 1.

[0092] Take the following data as an example:

[0093] 1. Integer parameter (batch size)

[0094] Assume there is a Sobol sequence value sobol_value between 0 and 1.

[0095] First, calculate the conversion formula that maps 0 to 1 to 8 to 128:

[0096] "batch_size"="floor"(8+"sobol_value"×(128-8+1))

[0097] Among them, floor represents the rounding down operation.

[0098] The specific calculation steps are as follows:

[0099] Calculate the size difference within the range: size_range = 128-8+1 = 121

[0100] Calculate the actual batch size: batch_size = floor(8 + sobol_value × 121)

[0101] For example, if sobol_value is 0.5:

[0102] batch_size=floor(8+0.5×121)

[0103] batch_size = floor(8+60.5)

[0104] batch_size = floor(68.5)

[0105] batch_size=68

[0106] This maps the Sobol sequence values ​​between 0 and 1 to the actual batch size value.

[0107] 2. Boolean parameters

[0108] For Boolean parameters, if you need to map the Sobol sequence value between 0 and 1 to 0 or 1, you can simply use the threshold judgment:

[0109] If sobol_value>=0.5, it is mapped to 1; otherwise it is mapped to 0.

[0110] For example, if sobol_value is 0.3, it is mapped to 0; if sobol_value is 0.7, it is mapped to 1.

[0111] This allows the Sobol sequence value to be converted to an integer or Boolean parameter value as needed.

[0112] 4) Record sampling points:

[0113] All generated sampling points are recorded to form a list of parameter combinations, with each sampling point corresponding to a specific parameter setting, which will be used for subsequent benchmarking and sensitivity analysis.

[0114] In a specific example, the parameter setting format is as follows:

[0115]

[0116] Step S102: Generate a configuration file based on the sampling point data, where the configuration file is used to run the sampling point data in sequence in a test environment.

[0117] Specifically, the following steps may be included:

[0118] 1) Prepare sampling point data:

[0119] All sampling points generated by the Sobol sequence are sorted into a list, each sampling point is a set of specific parameter combinations. For example, each sampling point contains the specific parameter values ​​of parameters such as batch size, number of copystreams, number of infer streams, whether to run infer on copystreams, and whether to use graph optimization.

[0120] 2) Configure the MLPerf benchmark environment:

[0121] Make sure the MLPerf benchmark environment is properly installed and configured. This includes installing necessary packages, setting up the hardware environment, etc. Make sure all dependencies and test scripts are ready to run benchmarks with different parameter combinations.

[0122] 3) Write a configuration script:

[0123] Write an automated script to dynamically generate a configuration file for the MLPerf benchmark based on the parameter combination of each sampling point. The script should be able to read the parameter combination of the sampling point and generate the corresponding configuration file or command line parameters to ensure that the benchmark can be run with these parameters.

[0124] 4) Generate configuration files:

[0125] For each sampling point, use the configuration script to generate the corresponding configuration file or command line parameters. Each configuration file contains all the parameter settings for the sampling point.

[0126] 5) Verify the configuration:

[0127] Before running the benchmark, verify that the generated configuration file or command line parameters are correct. Make sure that the parameter combination for each sampling point can be correctly applied to the MLPerf benchmark.

[0128] It should be noted that many of the procedural operations in this step, such as configuring the test environment, writing configuration scripts, generating configuration files, and verifying configurations, are reusable and do not need to be configured each time they are executed. Instead, an adaptive adjustment can be made based on actual needs.

[0129] Step S103, after each operation in the test environment is completed, the test results are obtained, wherein the test results include performance indicators of the deep learning model.

[0130] Specifically, when the aforementioned configuration files and configuration environment are successfully configured, the configuration files are run in the test environment to obtain performance indicators.

[0131] For details, please refer to the following steps:

[0132] 1) Execute benchmark:

[0133] Use the generated configuration file or command line parameters to run the MLPerf benchmark corresponding to each sampling point in sequence. Each time you run the benchmark, make sure to use the parameter combination of the current sampling point. You can use a written automated script to loop through all sampling points and run the benchmark in sequence.

[0134] 2) Record performance indicators:

[0135] After each benchmark run, the corresponding performance metrics were recorded.

[0136] In an optional example, the performance indicator may include one or more of inference speed (such as samples / s), latency, and resource utilization (such as GPU utilization, memory usage, etc.). In the Mlperf benchmark evaluation, the main performance indicator of the inference model is samples / s.

[0137] Step S104: Based on the performance indicators generated by the performance parameters, a first-order sensitivity index corresponding to each performance parameter is generated.

[0138] Specifically, first, obtain the index values ​​of multiple groups of performance indicators corresponding to the target performance parameters generated when the parameter values ​​of the other parameters are the same except for the target performance parameters in multiple sampling points, wherein the target performance parameter is any performance parameter in the sampling point data.

[0139] Then, when the target performance parameter is the first parameter, the index values ​​of multiple groups of performance indicators corresponding to the first parameter are extracted, and when the target performance parameter is the second parameter, the index values ​​of multiple groups of performance indicators corresponding to the second parameter are extracted. The first parameter is the performance parameter to be calculated and the first-order sensitivity index, the second parameter is any parameter in the sampling point data except the first parameter, and the index value of each group of performance indicators includes the index values ​​corresponding to the multiple performance indicators.

[0140] The output variance of the influence of other parameters on the first parameter is determined by the index values ​​of multiple groups of performance indicators corresponding to the first parameter and the index values ​​of multiple groups of performance indicators corresponding to other parameters. Finally, the first-order sensitivity index of the parameter is determined based on the output variance of the influence of other parameters on the parameter. Among them, the first-order sensitivity index is used to evaluate the influence of a single performance parameter on the model reasoning performance.

[0141] Step S105: Generate a total sensitivity index corresponding to all performance parameters based on the performance indicators generated by the performance parameters.

[0142] Among them, the total sensitivity index is used to indicate the comprehensive impact of all parameters on the model reasoning performance.

[0143] Specifically, first obtain the index values ​​of multiple groups of performance indicators corresponding to the target performance parameters generated when the parameter values ​​of the target performance parameters are different in multiple sampling points and the parameter values ​​of other parameters are the same, where the target performance parameter is any performance parameter in the sampling point data. Then calculate the variance based on these data, that is, the comprehensive variance value; then determine the output variance values ​​of other performance parameters except a certain performance parameter in the sampling point data, that is, determine the output variance caused by changes in all other performance parameters except a certain performance parameter. Finally, determine the total sensitivity index based on the comprehensive variance and all output variances. Among them, the total sensitivity index is used to indicate the comprehensive influence of all parameters on the model reasoning performance.

[0144] Step S106: determining the tuning order of the performance parameters based on the first-order sensitivity index corresponding to each performance parameter and the total sensitivity index.

[0145] Specifically, the degree of influence on the performance index is determined according to the first-order sensitivity index corresponding to each performance parameter and the total sensitivity index, and the tuning order of all performance parameters is determined according to the degree of influence on the performance index.

[0146] Generally speaking, it is to determine the key parameters that have the greatest impact on performance. In subsequent parameter optimization and model adjustment, prioritize the identified key parameters. Test the impact of different values ​​on performance and find the optimal settings. Optimize the settings of key parameters through gradual adjustment and testing to ensure optimal performance on different hardware platforms. After each adjustment, record performance indicators, compare the effects of different parameter combinations, and gradually approach the optimal solution. Finally, verify and fine-tune all parameters to ensure the stability and efficiency of the model in various application scenarios. Ensure that key parameter settings maintain excellent performance under different data sets and hardware environments.

[0147] The method for determining the performance parameter tuning order provided in this embodiment can more comprehensively understand the performance of the model under different parameter configurations by acquiring and testing data from multiple sampling points, thereby providing a more accurate performance evaluation. The performance index generated based on the performance parameters generates a first-order sensitivity index and a total sensitivity index corresponding to each parameter. For situations where it is necessary to choose between different models or parameter configurations, the first-order sensitivity index and the total sensitivity index can provide decision support to help make a more informed choice. Through scientific and systematic sensitivity analysis, the key parameters that have a significant impact on the reasoning performance of the deep learning model are accurately identified and determined. Then determine the tuning order of the performance parameters, that is, give priority to adjusting the parameters that have a greater impact on performance, so that the tuning process is more efficient. Moreover, adjusting the parameters in accordance with the tuning order can avoid blindly trying different parameter combinations, thereby saving tuning time and computing resources. Moreover, by testing and analyzing data from different sampling points, the optimal parameter configuration in different application scenarios can be determined to improve the generalization ability and versatility of the model.

[0148] In an optional example, in order to ensure the accuracy and rationality of the data, after recording the corresponding performance indicators after each benchmark test is completed, the method may further include:

[0149] The parameter combination of each sampling point and its corresponding performance index are recorded together to form a complete data set.

[0150] Furthermore, it may also include:

[0151] Save the test results:

[0152] Save the results of all benchmarks to a structured file or database for subsequent analysis. You can use CSV files, JSON files, or databases to store data. Each record should include a parameter combination and the corresponding performance indicators. CSV files are used in this application.

[0153] Check and handle abnormal data:

[0154] When recording and saving test results, check for any abnormal data or test failures. For abnormal data, record the details and consider rerunning the benchmark at that sampling point.

[0155] The abnormal data includes data whose index value of the performance index is not within the corresponding index value range.

[0156] On this basis, the method may further include:

[0157] Organize the above data into a structured format, such as a table or data frame. Each row represents a sampling point, and each column represents a parameter or performance indicator. Ensure the integrity and consistency of the data and exclude any missing or erroneous data.

[0158] Then, the sorted data is divided into two parts: parameter matrix and performance indicator vector.

[0159] That is, a parameter matrix is ​​generated based on the sampling point data, wherein the performance parameters of the deep learning model in the sampling point data constitute the column elements in the parameter matrix, the parameter values ​​of the performance parameters constitute the column elements in the parameter matrix, and the performance parameters include a batch size, the number of replication streams, the number of inference streams, the first identification information corresponding to whether to run inference on the replication stream, and the second identification information corresponding to whether to use graph optimization.

[0160] Based on the performance indicators, a performance indicator vector is generated, wherein the performance indicators include: one or more of model inference speed, latency, and resource utilization.

[0161] In an optional example, based on the performance indicators generated by the performance parameters, generating the first-order sensitivity index corresponding to each performance parameter can be achieved by the following method steps, see Figure 2 As shown, including:

[0162] Step S201 : obtaining the index values ​​of multiple groups of performance indexes corresponding to the target performance parameters generated when the parameter values ​​of the other parameters are the same except for the target performance parameter in the multiple sampling points.

[0163] The target performance parameter is any performance parameter in the sampling point data.

[0164] Specifically, the indicator values ​​of the multiple groups of performance indicators corresponding to each target performance parameter can be obtained after testing in a test environment in the manner in step 102 .

[0165] Step S202: extracting the index values ​​of multiple groups of performance indicators corresponding to the first parameter when the target performance parameter is the first parameter, and extracting the index values ​​of multiple groups of performance indicators corresponding to the second parameter when the target performance parameter is the second parameter.

[0166] The first parameter is a performance parameter to be calculated and a first-order sensitivity index, the second parameter is any parameter in the sampling point data except the first parameter, and the index value of each group of performance indicators includes index values ​​corresponding to multiple performance indicators.

[0167] Step S203: determining a first variance value corresponding to the first parameter based on the index values ​​of the plurality of groups of performance indexes corresponding to the first parameter.

[0168] Step S204: determining a second variance value corresponding to the second parameter based on the index values ​​of the plurality of groups of performance indexes corresponding to the second parameter.

[0169] Specifically, when calculating the first variance value and the second variance value, they can be calculated using a formula similar to the following:

[0170]

[0171] Among them, V(Y) is the variance value, n is the number of sampling point data, Y i is the i-th indicator value corresponding to any performance indicator in multiple groups of performance indicators, It is the average value of the indicator values ​​of all performance indicators in multiple groups of performance indicators.

[0172] Through this formula, we can get the first variance value V(Y) 第一参数 , and the second variance value V(Y) 第二参数 .

[0173] Step S205: Determine the influence of the second parameter on the first parameter based on the first variance value and the second variance value.

[0174] Specifically, the output variance of the influence of the second parameter on the first parameter is the difference between the two. The specific parameters are shown in the following formula:

[0175] V j =V(Y) 第一参数 - V(Y) 第二参数 (Formula 2)

[0176] Among them, V j Outputs the variance of the effect of the second parameter on the first parameter.

[0177] Step S206, after determining the output variances of the effects of the performance parameters other than the first parameter in the sampling point data on the first parameter, determine the first-order sensitivity index corresponding to the first parameter according to the first variance value and the output variances of the effects of the performance parameters other than the first parameter in the sampling point data on the first parameter.

[0178] Specifically, when determining the first-order sensitivity index corresponding to the first parameter, the following formula can be used:

[0179]

[0180] Where h is the total number of performance parameters of the deep learning model in the sampling point data, S1 is the sum of the output variances of the effects of the performance parameters other than the first parameter in the sampling point data on the first parameter. iis the first-order sensitivity index corresponding to the first parameter.

[0181] In a specific example, suppose we have the following parameters and corresponding performance indicator data:

[0182] 1) Assuming the batch_size parameter is 8, 16, 32, and 64, when other parameters include only the infer_streams parameter, where the infer_streams parameter value is a fixed parameter value, the generated performance indicators Y (inference speed samples / s) are 100, 120, 150, and 200 respectively.

[0183] 2) Assuming the parameter infer_streams: 1, 2, 4, 8, when the other parameter batch_size parameter value is fixed, the generated performance indicator Y (inference speed samples / s) is: 90, 110, 130, 170.

[0184] The calculation steps are as follows:

[0185] 1) Calculate the first variance V(Y) corresponding to batch_size 第一参数 , using the following formula:

[0186]

[0187] For batch_size,

[0188]

[0189]

[0190] That is, V(Y) 第一参数 ==1891.67.

[0191] The second variance corresponding to infer_streams is calculated using the above formula:

[0192]

[0193] That is, V(Y) 第二参数 =1166.67.

[0194] V j =V(Y) 第一参数 -V(Y) 第二参数 =1891.67-1166.67=725

[0195] Then, when determining the first-order sensitivity index corresponding to the first parameter:

[0196]

[0197] In an optional example, based on the performance indicators generated by the performance parameters, generating a total sensitivity index corresponding to all performance parameters can be achieved by the following method steps, see Figure 3 As shown, including:

[0198] Step S301, obtaining index values ​​of multiple groups of performance indexes corresponding to the target performance parameters generated when the parameter values ​​of the target performance parameters in multiple sampling points are different but the parameter values ​​of other parameters are the same, wherein the target performance parameter is any performance parameter in the sampling point data.

[0199] Specifically, the steps of the method are as described above and will not be described in detail here.

[0200] Step S302: determining a comprehensive variance value based on the index values ​​of multiple groups of performance indexes corresponding to each performance parameter included in all the sampling point data.

[0201] Specifically, this method can be implemented by the following formula:

[0202]

[0203] Among them, V d (Y) is the comprehensive variance value, d is the total number of indicator values ​​corresponding to multiple groups of performance indicators, and Y q is the qth data among the d data, The average value calculated for the values ​​corresponding to d data.

[0204] Step S303: determining an output variance value based on parameter values ​​of other performance parameters except any performance parameter included in each sampling point data.

[0205] Specifically, this method can be calculated by the following formula:

[0206]

[0207] Among them, V p (Y) is the output variance value determined for the parameter values ​​of other performance parameters except the third parameter included in each sampling point data, k=dd / n, n is the number of sampling points, Y pi is the i-th performance indicator value corresponding to the p-th performance parameter among the n-1 performance parameters excluding the third parameter, is an average value calculated based on n indicator values ​​of the ith performance indicator corresponding to the pth performance parameter, wherein the third parameter is any one of the performance parameters included in the sampling point data;

[0208] Step S304, after output variance values ​​are determined based on the index values ​​of multiple performance indicators corresponding to other performance parameters in the sampling point data obtained by excluding each performance parameter in the sampling point data, a total sensitivity index is determined based on the comprehensive variance value and the output variance values ​​of multiple performance indicators corresponding to other performance parameters in the sampling point data obtained by excluding each performance parameter in the sampling point data.

[0209] Specifically, this method can be implemented by the following formula:

[0210]

[0211] in, It is the sum of the output variance values ​​determined by the index values ​​of various performance indicators corresponding to other performance parameters in the sampling point data. i The total sensitivity index corresponding to all performance parameters.

[0212] Taking the data in the above specific example as an example, the calculation process of the comprehensive variance value is:

[0213]

[0214]

[0215] Assuming that the change of parameter batch_size is excluded, calculate the variance V~batch_size caused by the change of parameter infer_streams. The variance of parameter infer streams has been calculated and is 1166.67.

[0216] For the parameter batch_size, the total sensitivity index is:

[0217]

[0218] It should be noted that in the above examples, because they are all examples based on two performance parameter settings, there will be multiple performance parameters in the actual application process. The first-order sensitivity index corresponding to each performance parameter and the total sensitivity index can also be calculated using the above formula. The first-order sensitivity index can provide valuable information for decision-making, help determine which parameters have the most significant impact on model performance, and thus guide the adjustment and optimization of parameters. Based on the results of the first-order sensitivity index, the parameters of the model can be optimized in a targeted manner to improve the performance of the model, such as increasing the inference speed, reducing latency, or improving resource utilization. Understanding the sensitivity of each parameter helps to enhance the understanding and interpretation of the model and helps to explore the working mechanism of the model in depth. By determining the first-order sensitivity index, the parameters that have a greater impact on the performance of the model can be found more quickly, thereby improving the efficiency of tuning and reducing unnecessary trial and error. Moreover, in-depth research on parameter sensitivity can provide direction for model improvement and innovation, and inspire new optimization ideas and methods.

[0219] Similarly, understanding the total sensitivity index helps to better explain the behavior and performance of the model and increase the comprehensibility of the model. Discovering the relationship between parameters can provide direction for model improvement and inspire new optimization strategies and methods suitable for dealing with complex situations with multiple parameters and performance indicators, and can cope with diverse needs in practical applications. By optimizing parameters, it helps to improve the stability and reliability of the model and reduce performance fluctuations. It can be applied to various deep learning models and tasks, and has wide applicability and practical value.

[0220] Figure 4 The overall process diagram of the method is provided in the figure, including, for example, defining parameters and parameter ranges, generating Sobol sequence sampling points, configuring benchmark tests, running benchmark tests to record performance indicators, organizing data, performing sensitivity analysis, interpreting results, applying results, and ending.

[0221] Among them, interpreting the results and applying the analysis results include the following processes:

[0222] 1) Analysis result example:

[0223] Assume that the results of Sobol sensitivity analysis are as follows:

[0224] Batch size: S1 = 0.4, ST = 0.6

[0225] Number of copy streams: S1 = 0.3, ST = 0.4

[0226] Number of infer streams: S1 = 0.2, ST = 0.5

[0227] Whether to run infer on copy streams: S1=0.1, ST=0.2

[0228] Whether to use graph optimization (use graph): S1 = 0.05, ST = 0.1

[0229] 2) Identify key parameters:

[0230] Based on the S1 and ST values, identify the parameters that have the greatest impact on performance. In the above example, batch size has the highest S1 and ST values, indicating that it has significant direct and interactive effects on performance. The ST value for the number of inferstreams is significantly higher than the S1 value, indicating that its interactions with other parameters have a significant impact on performance.

[0231] 3) Determine key parameters:

[0232] Based on the analysis results, identify the key parameters that have the greatest impact on performance, such as those with higher S1 and ST values.

[0233] In the above example, based on the data, it can be determined that batch size and the number of infer streams are two key parameters that need to be adjusted first.

[0234] 4) Prioritize adjustment of key parameters:

[0235] In the subsequent parameter optimization and model adjustment, priority is given to adjusting the identified key parameters.

[0236] For example, first adjust the batch size, test the impact of different values ​​on performance, and find the optimal setting.

[0237] Afterwards, we adjust the number of infer streams and test different values ​​and their interaction with the batch size to find the optimal combination.

[0238] 5) Gradual optimization:

[0239] Through gradual adjustment and testing, we optimize the settings of key parameters to ensure the best performance on different hardware platforms. After each adjustment, we record the performance indicators, compare the effects of different parameter combinations, and gradually approach the optimal solution.

[0240] 6) Verification and fine-tuning:

[0241] Finally, all parameters are verified and fine-tuned to ensure the stability and efficiency of the model in various application scenarios. It is ensured that key parameter settings can maintain excellent performance in different data sets and hardware environments.

[0242] Attached Figure 4 The other steps have been introduced in detail in the previous article, so I will not repeat them here. The above method ensures scientificity and systematicity, which helps to improve the reasoning performance of deep learning models on different hardware platforms. It enhances the adaptability and versatility of the model and provides systematic solutions and technical support for the efficient deployment and performance optimization of deep learning models.

[0243] In an optional example, after determining the total sensitivity index, the method may further include:

[0244] After adjusting the index values ​​of multiple performance indicators corresponding to a certain parameter for multiple times, the change of the total sensitivity index is determined when the index values ​​of multiple performance indicators corresponding to other parameters remain unchanged. In this way, the influence of the index values ​​of performance indicators corresponding to different performance parameters on the total sensitivity index is determined. And the performance parameters are sorted according to the influence of the index values ​​of performance indicators corresponding to each performance parameter on the total sensitivity index. When determining the tuning order of the performance parameters, the tuning order can also be determined based on the sorting order determined by the influence of each performance parameter on the total sensitivity index. That is, when considering the first-order sensitivity index and the total sensitivity index, the total sensitivity index has a higher priority.

[0245] Alternatively, when the implementation method of the present application is adopted, when the sorting order of two performance parameters is determined in parallel based on the first-order sensitivity index and the total sensitivity index, the sorting order determined according to the influence of each performance parameter on the total sensitivity index can be considered to determine a more prioritized sorting order from the two.

[0246] In this embodiment, a device for determining the order of performance parameter tuning is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0247] This embodiment provides a device for determining a performance parameter tuning sequence. Figure 5 As shown, it includes: a sampling module 501, a generating module 502, an operating module 503, a processing module 504 and a determining module 505.

[0248] A sampling module 501 is used to sample the performance parameters of the deep learning model to obtain a plurality of sampling point data, each of which is composed of a set of performance parameters of the deep learning model and a parameter value corresponding to each performance parameter;

[0249] A generating module 502, for generating a configuration file based on the sampling point data, the configuration file being used to sequentially run the sampling point data in a test environment;

[0250] An operation module 503 is used to obtain a test result after each operation in the test environment is completed, wherein the test result includes a performance index of the deep learning model;

[0251] A processing module 504 is used to generate a first-order sensitivity index corresponding to each performance parameter based on the performance index generated by the performance parameter, wherein the first-order sensitivity index is used to evaluate the influence of a single performance parameter on the model reasoning performance; and, based on the performance index generated by the performance parameter, generate a total sensitivity index corresponding to all performance parameters, wherein the total sensitivity index is used to indicate the comprehensive influence of all parameters on the model reasoning performance;

[0252] The determination module 505 is used to determine the tuning order of the performance parameters based on the first-order sensitivity index corresponding to each performance parameter and the total sensitivity index.

[0253] In an optional example, the processing module 504 is further configured to:

[0254] Generate a parameter matrix based on the sampling point data, wherein the performance parameters of the deep learning model in the sampling point data constitute column elements in the parameter matrix, parameter values ​​of the performance parameters constitute column elements in the parameter matrix, and the performance parameters include a batch size, a number of replication streams, a number of inference streams, a first identification information corresponding to whether to run inference on the replication stream, and a second identification information corresponding to whether to use graph optimization;

[0255] Based on the performance indicators, a performance indicator vector is generated, where the performance indicators include one or more of model inference speed, latency, and resource utilization.

[0256] In an optional example, the processing module 504 is specifically configured to:

[0257] Obtaining the index values ​​of multiple groups of performance indexes corresponding to the target performance parameters generated when the parameter values ​​of the target performance parameters are different in the multiple sampling points and the parameter values ​​of other parameters are the same, wherein the target performance parameter is any performance parameter in the sampling point data;

[0258] Extracting the index values ​​of multiple groups of performance indicators corresponding to the first parameter when the target performance parameter is the first parameter, and extracting the index values ​​of multiple groups of performance indicators corresponding to the second parameter when the target performance parameter is the second parameter, wherein the first parameter is the performance parameter to be calculated and the first-order sensitivity index, the second parameter is any parameter in the sampling point data except the first parameter, and the index values ​​of each group of performance indicators include the index values ​​corresponding to the multiple performance indicators respectively;

[0259] Determining a first variance value corresponding to the first parameter based on the index values ​​of the plurality of groups of performance indicators corresponding to the first parameter;

[0260] Determining a second variance value corresponding to the second parameter based on the index values ​​of the plurality of groups of performance indicators corresponding to the second parameter;

[0261] Determine an output variance of the effect of the second parameter on the first parameter based on the first variance value and the second variance value;

[0262] After determining the output variances of the effects of the performance parameters other than the first parameter in the sampling point data on the first parameter, a first-order sensitivity index corresponding to the first parameter is determined according to the first variance value and the output variances of the effects of the performance parameters other than the first parameter in the sampling point data on the first parameter.

[0263] In an optional example, the processing module 504 is specifically configured to:

[0264] Obtaining the index values ​​of multiple groups of performance indexes corresponding to the target performance parameters generated when the parameter values ​​of the target performance parameters are different in the multiple sampling points and the parameter values ​​of other parameters are the same, wherein the target performance parameter is any performance parameter in the sampling point data;

[0265] Determine a comprehensive variance value based on the index values ​​of multiple groups of performance indexes corresponding to each performance parameter included in all sampling point data;

[0266] Determine the output variance value based on the index values ​​of multiple performance indexes corresponding to other performance parameters except any performance parameter included in each sampling point data;

[0267] After the output variance values ​​are determined based on the index values ​​of multiple performance indicators corresponding to other performance parameters in the sampling point data excluding each performance parameter in the sampling point data, the total sensitivity index is determined according to the comprehensive variance value and the output variance values ​​of the index values ​​of multiple performance indicators corresponding to other performance parameters in the sampling point data excluding each performance parameter in the sampling point data.

[0268] In an optional example, the processing module 504 generates a first-order sensitivity index corresponding to each performance parameter based on the performance index generated by the performance parameter, which is specifically represented by the following formula:

[0269]

[0270] Among them, V(Y) is the variance value, n is the number of sampling point data, Y i is the i-th indicator value corresponding to any performance indicator in multiple groups of performance indicators, The average value calculated from the indicator values ​​of all performance indicators in multiple groups of performance indicators;

[0271] V j =V(Y) 第一参数 - V(Y) 第二参数 (Formula 2)

[0272] Among them, V j V(Y) is the output variance of the effect of the second parameter on the first parameter. 第一参数 To calculate the first variance value using formula 1, V(Y) 第二参数 is the second variance value calculated by formula 1;

[0273]

[0274] Where h is the total number of performance parameters of the deep learning model in the sampling point data, S1 is the sum of the output variances of the effects of the performance parameters other than the first parameter in the sampling point data on the first parameter. i is the first-order sensitivity index corresponding to the first parameter.

[0275] In an optional example, the processing module 504 generates a total sensitivity index corresponding to all performance parameters based on the performance indicators generated by the performance parameters, which is specifically represented by the following formula:

[0276]

[0277] Among them, V d (Y) is the comprehensive variance value, d is the total number of indicator values ​​corresponding to multiple groups of performance indicators, and Y q is the qth data among the d data, is the average value calculated for the values ​​corresponding to d data;

[0278]

[0279] Among them, V p (Y) is the output variance value determined for the parameter values ​​of other performance parameters except the third parameter included in each sampling point data, k=dd / n, n is the number of sampling points, Y pi is the i-th performance indicator value corresponding to the p-th performance parameter among the n-1 performance parameters excluding the third parameter, is an average value calculated based on n indicator values ​​of the ith performance indicator corresponding to the pth performance parameter, wherein the third parameter is any one of the performance parameters included in the sampling point data;

[0280]

[0281] in, It is the sum of the output variance values ​​determined by the index values ​​of various performance indicators corresponding to other performance parameters in the sampling point data. i The total sensitivity index corresponding to all performance parameters.

[0282] In an optional example, the processing module 504 is further used to determine whether there is abnormal data in the performance indicator according to the test log, wherein the abnormal data includes data whose indicator value of the performance indicator is not within the corresponding indicator value range;

[0283] When it is determined that abnormal data exists, the test is performed again based on the sampling point data corresponding to the abnormal data.

[0284] The performance parameter tuning order determination device in this embodiment is presented in the form of a functional module, where the module refers to an application specific integrated circuit (ASIC), a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0285] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0286] A performance parameter tuning order determination device provided by an embodiment of the present invention can more comprehensively understand the performance of a model under different parameter configurations by acquiring and testing data from multiple sampling points, thereby providing a more accurate performance evaluation. The performance index generated based on the performance parameter generates a first-order sensitivity index and a total sensitivity index corresponding to each parameter. For situations where it is necessary to choose between different models or parameter configurations, the first-order sensitivity index and the total sensitivity index can provide decision support to help make a more informed choice. Through scientific and systematic sensitivity analysis, the key parameters that have a significant impact on the reasoning performance of the deep learning model are accurately identified and determined. Then, the tuning order of the performance parameters is determined, that is, the parameters that have a greater impact on the performance are adjusted first, so that the tuning process is more efficient. Moreover, by adjusting the parameters in the tuning order, blindly trying different parameter combinations can be avoided, thereby saving tuning time and computing resources. Moreover, by testing and analyzing the data from different sampling points, the optimal parameter configuration in different application scenarios can be determined to improve the generalization ability and versatility of the model.

[0287] The embodiment of the present invention also provides a computer device having the above Figure 5 The performance parameter tuning order shown determines the device.

[0288] See also Figure 6 , Figure 6is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0289] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include an integrated circuit. The integrated circuit may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0290] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0291] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of a computer device based on the presentation of a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0292] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0293] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0294] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0295] The embodiment of the present invention also provides a computer-readable storage medium. The method provided in the above embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or is implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0296] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0297] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for determining a performance parameter tuning sequence, characterized in that: The method comprises: Sampling the deep learning model performance parameters to obtain a plurality of sampling point data, each of the sampling point data being composed of a set of performance parameters of the deep learning model and a parameter value corresponding to each of the performance parameters; Based on the sampling point data, generating a configuration file, wherein the configuration file is used to sequentially run the sampling point data in a test environment; After each operation in the test environment is completed, a test result is obtained, wherein the test result includes a performance indicator of the deep learning model; Based on the performance indicators generated by the performance parameters, a first-order sensitivity index corresponding to each performance parameter is generated, wherein the first-order sensitivity index is used to evaluate the influence of a single performance parameter on the model reasoning performance; And, based on the performance indicators generated by the performance parameters, generating a total sensitivity index corresponding to all performance parameters, wherein the total sensitivity index is used to indicate the comprehensive influence of all parameters on the model reasoning performance; Based on the first-order sensitivity index corresponding to each performance parameter and the total sensitivity index, a tuning order of the performance parameters is determined.

2. The method according to claim 1, characterized in that The method further comprises: Generate a parameter matrix based on the sampling point data, wherein the performance parameters of the deep learning model in the sampling point data constitute column elements in the parameter matrix, parameter values ​​of the performance parameters constitute column elements in the parameter matrix, and the performance parameters include a batch size, a number of replication streams, a number of inference streams, a first identification information corresponding to whether to run inference on the replication stream, and a second identification information corresponding to whether to use graph optimization; Based on the performance indicators, a performance indicator vector is generated, wherein the performance indicators include one or more of model inference speed, latency, and resource utilization.

3. The method according to claim 1 or 2, characterized in that: The performance indicator generated based on the performance parameter generates a first-order sensitivity index corresponding to each performance parameter, specifically including: Obtaining, when parameter values ​​of the target performance parameter are different among the multiple sampling points and the parameter values ​​of other parameters are the same, multiple groups of performance index values ​​corresponding to the target performance parameter generated, wherein the target performance parameter is any performance parameter in the sampling point data; Extracting, when the target performance parameter is a first parameter, the index values ​​of multiple groups of performance indicators corresponding to the first parameter, and extracting, when the target performance parameter is a second parameter, the index values ​​of multiple groups of performance indicators corresponding to the second parameter, wherein the first parameter is a performance parameter to be calculated and the first-order sensitivity index, the second parameter is any parameter in the sampling point data except the first parameter, and the index values ​​of each group of performance indicators include the index values ​​corresponding to the multiple performance indicators respectively; Determining a first variance value corresponding to the first parameter based on the index values ​​of the plurality of groups of performance indicators corresponding to the first parameter; Determining a second variance value corresponding to the second parameter based on the index values ​​of the plurality of groups of performance indicators corresponding to the second parameter; Determine, based on the first variance value and the second variance value, an output variance of the influence of the second parameter on the first parameter; After determining the output variances of the effects of the performance parameters other than the first parameter in the sampling point data on the first parameter, the first-order sensitivity index corresponding to the first parameter is determined according to the first variance value and the output variances of the effects of the performance parameters other than the first parameter in the sampling point data on the first parameter.

4. The method according to claim 1 or 2, characterized in that: The performance indicators generated based on the performance parameters generate a total sensitivity index corresponding to all performance parameters, specifically including: Obtaining, when parameter values ​​of the target performance parameter are different among the multiple sampling points and the parameter values ​​of other parameters are the same, multiple groups of performance index values ​​corresponding to the target performance parameter generated, wherein the target performance parameter is any performance parameter in the sampling point data; Determine a comprehensive variance value based on the index values ​​of multiple groups of performance indexes corresponding to each of the performance parameters included in all the sampling point data; Determine the output variance value based on the index values ​​of multiple performance indexes corresponding to other performance parameters except any performance parameter included in each of the sampling point data; After output variance values ​​are determined based on the index values ​​of multiple performance indicators corresponding to other performance parameters in the sampling point data excluding each performance parameter in the sampling point data, the total sensitivity index is determined according to the comprehensive variance value and the output variance values ​​of multiple performance indicators corresponding to other performance parameters in the sampling point data excluding each performance parameter in the sampling point data.

5. The method according to claim 3, characterized in that: The performance index generated based on the performance parameters generates a first-order sensitivity index corresponding to each performance parameter, which is specifically expressed by the following formula: Among them, V(Y) is the variance value, n is the number of sampling point data, Y i is the i-th indicator value corresponding to any performance indicator in multiple groups of performance indicators, The average value calculated from the indicator values ​​of all performance indicators in multiple groups of performance indicators; V j = V(Y) 第一参数 - V(Y) 第二参数 (Formula 2) Among them, V j is the output variance of the influence of the second parameter on the first parameter, V(Y) 第一参数 To calculate the first variance value by formula 1, V(Y) 第二参数 is the second variance value calculated by formula 1; Wherein, h is the total number of performance parameters of the deep learning model in the sampling point data, is the sum of the output variances of the effects of the performance parameters other than the first parameter in the sampling point data on the first parameter, S1 i is the first-order sensitivity index corresponding to the first parameter.

6. The method according to claim 4, characterized in that The performance index generated based on the performance parameters generates a total sensitivity index corresponding to all performance parameters, which is specifically expressed by the following formula: Among them, V d (Y) is the comprehensive variance value, d is the total number of index values ​​corresponding to the multiple groups of performance indicators, and Y q is the qth data among the d data, is the average value calculated for the values ​​corresponding to d data; Among them, V p (Y) an output variance value determined for the parameter values ​​of other performance parameters except the third parameter included in each of the sampling point data, k=dd / n, n is the number of sampling points, Y pi is the i-th performance indicator value corresponding to the p-th performance parameter among the n-1 performance parameters except the third parameter, is an average value calculated based on n indicator values ​​of the ith performance indicator corresponding to the pth performance parameter, wherein the third parameter is any one of the performance parameters included in the sampling point data; in, is the sum of the output variance values ​​determined by the index values ​​of multiple performance indicators corresponding to other performance parameters in the sampling point data, ST i The total sensitivity index corresponding to all performance parameters.

7. The method according to any one of claims 1-2 or 5-6, characterized in that: The test result also includes a test log, and the method further includes: Determine, according to the test log, whether there is abnormal data in the performance indicator, wherein the abnormal data includes data whose indicator value of the performance indicator is not within the corresponding indicator value range; When it is determined that the abnormal data exists, the test is performed again according to the sampling point data corresponding to the abnormal data.

8. A device for determining a performance parameter tuning sequence, characterized in that: The device comprises: A sampling module, used to sample the performance parameters of the deep learning model to obtain a plurality of sampling point data, each of which is composed of a set of performance parameters of the deep learning model and a parameter value corresponding to each performance parameter; A generating module, used to generate a configuration file based on the sampling point data, wherein the configuration file is used to sequentially run the sampling point data in a test environment; A running module, used for obtaining a test result after each operation in the test environment is completed, wherein the test result includes a performance indicator of the deep learning model; A processing module, configured to generate a first-order sensitivity index corresponding to each performance parameter based on the performance index generated by the performance parameter, wherein the first-order sensitivity index is used to evaluate the influence of a single performance parameter on the model reasoning performance; and, based on the performance index generated by the performance parameter, generate a total sensitivity index corresponding to all performance parameters, wherein the total sensitivity index is used to indicate the comprehensive influence of all parameters on the model reasoning performance; The determination module is used to determine the tuning order of the performance parameters based on the first-order sensitivity index corresponding to each performance parameter and the total sensitivity index.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for determining the performance parameter tuning order according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for determining the performance parameter tuning order according to any one of claims 1 to 7.

Images