A model portability evaluation method and system based on domestic GPU platform

By building a model portability evaluation method on the domestic GPU platform, using multi-objective modeling and gradient decision-making to improve the tree algorithm to evaluate the model characteristics and operator performance, the problem of model portability complexity and performance degradation on the domestic GPU platform is solved, and the problem of model transplantation complexity and performance degradation on the domestic GPU platform is realized. Efficient and reliable model transplant evaluation and optimization.

CN118796648BActive Publication Date: 2024-11-29北京麟卓信息科技有限公司
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411287402.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-13
Publication Date
2024-11-29
Estimated Expiration
2044-09-13

AI Technical Summary

Technical Problem

Model transplantation on domestic GPU platforms has problems of complexity and performance degradation. The existing technology is difficult to accurately predict the transplantation effect, resulting in developers investing a lot of time and resources for debugging and optimization.

Method used

Using the model portability evaluation method based on the domestic GPU platform, the portability evaluation model is constructed by extracting model features and operator performance, and the multi-objective modeling algorithm and gradient decision-making improvement tree algorithm are used to construct the portability evaluation model to be transplanted in advance. , simplify the transplant process and improve success rate.

Benefits of technology

It improves the decision-making efficiency of model transplantation, reduces the risk of performance decline and resource waste caused by blind transplantation, ensures the performance and accuracy of the model after transplantation, and provides developers with technical support and decision-making basis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118796648B_ABST
    Figure CN118796648B_ABST
Patent Text Reader

Abstract

The present invention discloses a model portability evaluation method and system based on a domestic GPU platform. A sample data set is established by collecting model features of a model that has been transplanted to a domestic GPU platform and a plurality of performance evaluation features after transplantation. A portability evaluation model is constructed based on the improvement of a multi-objective modeling algorithm by using a gradient decision boosting tree algorithm. The established sample data set is used to train the portability evaluation model. When in use, the extracted model features of the model to be transplanted and the target domestic GPU platform are input into the trained portability evaluation model to obtain an evaluation result related to the portability of the model to be transplanted for the target domestic GPU platform. The decision-making efficiency of model transplantation is improved, the risk of performance degradation or resource waste caused by blind transplantation is reduced, a strong guarantee is provided for the performance and accuracy of the model after transplantation, and technical support and decision-making basis are provided for developers.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of model transplantation, and in particular relates to a model transplantability evaluation method and system based on a domestic GPU platform. Background Art

[0002] With the in-depth development of deep learning technology and the growing demand for intelligent processing of massive data, the field of intelligent computing is facing unprecedented complexity challenges, which has greatly promoted the high standards for hardware computing capabilities. At present, international cutting-edge artificial intelligence computing generally relies on the CUDA framework to achieve efficient hardware acceleration. This framework is designed specifically for NVIDIA GPUs and demonstrates strong performance advantages. However, in building an independent software and hardware ecosystem based on domestic processors, operating systems, and GPUs, the limitations of the CUDA framework are highlighted: the closed nature of its core technology, insufficient support for domestic operating systems, and the lack of a software development environment have all hindered its direct application on domestic platforms.

[0003] To address this challenge, the current common solution is to first complete model training on NVIDIA GPUs, and then try to port the trained model to run on domestic GPUs to achieve model deployment in domestic environments. However, this porting process is complex and challenging. It is not only time-consuming and difficult, but also difficult to accurately predict the effect of the porting. Developers need to invest a lot of time and resources to continuously solve errors in the porting, and perform repeated debugging and optimization, which often leads to adverse consequences such as performance degradation and reduced accuracy when the model runs on domestic GPUs.

[0004] In view of this, there is an urgent need to propose an efficient and reliable model transplantation evaluation method to quickly evaluate the portability of the transplanted model to the domestic GPU platform, so as to simplify the transplantation process, lower the technical threshold, improve the success rate of model transplantation to the domestic GPU platform, and effectively ensure the performance and accuracy of the transplanted model. Summary of the invention

[0005] In view of this, the present invention provides a model portability evaluation method and system based on a domestic GPU platform, which realizes the preliminary evaluation of the portability of transplanting the model to a domestic GPU platform.

[0006] The present invention provides a model portability evaluation method based on a domestic GPU platform, which specifically includes the following steps:

[0007] Step 1: Take the model that has been transplanted to the domestic GPU platform as the sample model, extract the model features of the sample model, evaluate the portability of the sample model to the domestic GPU platform to obtain the operator performance, model performance and portability evaluation results, the model features, operator performance, model performance, portability evaluation results and domestic GPU platform models constitute the model evaluation features, and the model evaluation features of all sample models constitute the sample data set; the model features include model name, technical field, input dimension, output dimension, parameter quantity and calculation amount; perform data preprocessing on the sample data set and divide it into a training set and a test set;

[0008] Step 2, construct a portability evaluation model, take the model characteristics of the model to be transplanted and the target domestic GPU platform model as the evaluation model input, first use the multi-objective modeling algorithm to calculate the evaluation model input to obtain the operator performance and portability evaluation results of the model to be transplanted on the target domestic GPU platform, then use the gradient decision boosting tree algorithm to calculate the operator performance, portability evaluation results and evaluation model input to obtain the model performance, and use the operator performance, model performance and portability evaluation results as the evaluation model output;

[0009] Step 3: Use the sample data set constructed in step 1 to complete the training of the portability evaluation model;

[0010] Step 4: In actual use, extract the model features of the model to be ported, input the extracted model features and the target domestic GPU platform model into the trained portability evaluation model to obtain the output operator performance, model performance and portability evaluation results.

[0011] Furthermore, the method of extracting the model features of the sample model in step 1 is:

[0012] The model name, technical field, network structure, input dimension and output dimension of the sample model are read according to the framework used by the sample model, and then the sample model is converted into a model file in the open network neural exchange format according to the network structure, input dimension and output dimension; the model file is parsed to obtain the parameter quantity and calculation amount of the sample model.

[0013] Furthermore, the method of performing data preprocessing on the sample data set in step 1 is:

[0014] Step 2.1, data identification is performed on the model name, technical field, domestic GPU platform model, operator name, and evaluation index, that is, an ID value is assigned;

[0015] Step 2.2: Use the StandardScaler() function to standardize the model evaluation features after data identification processing;

[0016] Step 2.3, classify the model evaluation features by task, that is, divide them into operator performance prediction tasks and portability evaluation result prediction tasks, and identify the model evaluation feature tasks;

[0017] Step 2.4: Use data analysis tools to write the model evaluation feature set after task division into a character-separated value file to form a feature data set.

[0018] Furthermore, the sample data set constructed in step 1 is used in step 3 to complete the training of the portability evaluation model, specifically:

[0019] Step 3.1, taking the model evaluation features of the samples in the training set as the input of the multi-objective modeling algorithm in the portability evaluation model as the first input, the first input includes model features, operator performance, model performance, portability evaluation results and domestic GPU platform model; the multi-objective modeling algorithm processes the first input to obtain the portability evaluation result prediction value and the operator performance prediction value, which are recorded as the first output; then the parameters other than the operator performance, model performance and portability evaluation result in the first input and the first output are taken as the second input, and the gradient decision boosting tree algorithm processes the second input to obtain the model performance prediction value, which is recorded as the second output; finally, the first output and the second output constitute the prediction result of the portability evaluation model output;

[0020] Step 3.2: Use the test set to test the trained portability evaluation model. If the set conditions are met, the training is completed; otherwise, the training is performed again.

[0021] Furthermore, the setting condition is met in step 3.2 as follows: calculating the consistency index C-index value, correcting the C-index value using the Bootstrap method, and if the C-index value is not less than a set threshold, the setting condition is met.

[0022] Furthermore, the portability evaluation model in step 4 is output and displayed in a graphical form.

[0023] Furthermore, the operator performance is represented by a tuple, in which each data is a tuple including an operator name and an operator time consumption; the portability evaluation result is represented by a tuple including an evaluation index and an evaluation score.

[0024] The present invention provides a model portability evaluation system based on a domestic GPU platform, which specifically includes a model feature acquisition module, a feature preprocessing module, a portability evaluation model training module, a model conversion module, a model parsing module and a portability evaluation model module;

[0025] The model feature acquisition module is used to acquire model features of the model, and acquire operator performance, model performance and portability evaluation results of the transplanted model on the domestic GPU platform, and form model evaluation features from the model features and operator performance, model performance and portability evaluation results of the transplanted model, wherein the model features include model name, technical field, input dimension, output dimension, parameter quantity and calculation quantity;

[0026] The feature preprocessing module is used to perform data preprocessing on the model features and model evaluation features, including data identification processing, standardization processing and type conversion, to form a CSV format feature file, and to construct a sample data set from the format feature file of the transplanted model, and to divide the sample data set into a training set and a test set;

[0027] The portability evaluation model training module is used to train the portability evaluation model using a training set, and to test the portability evaluation model on a test set using a C-index evaluation indicator, and the training is completed when the test result meets the set conditions;

[0028] The model conversion module is used to read the model to be transplanted and convert it into an ONNX format model file;

[0029] The model parsing module is used to parse the ONNX format model file to obtain the network structure, input dimension, output dimension and operator name, and calculate the parameter amount and calculation amount of the model to form the model features;

[0030] The portability evaluation model module is used to analyze the model features of the input model to be transplanted to obtain the prediction results of the model to be transplanted, including operator performance, model performance and portability evaluation results.

[0031] Furthermore, the model portability evaluation system also includes an evaluation result display module, and the evaluation result display module is used to display the model portability results to the user in a graphical manner. Beneficial Effects

[0032] The present invention establishes a sample data set by collecting model features of a model that has been transplanted to a domestic GPU platform and a variety of performance evaluation features after transplantation, constructs a portability evaluation model based on the improvement of a multi-objective modeling algorithm (Progressive Layered Extraction, PLE) using a gradient decision boosting tree algorithm (Gradient Boosting DecistionTree, GBDT), trains the portability evaluation model using the established sample data set, and when in use, inputs the extracted model features of the model to be transplanted and the target domestic GPU platform into the trained portability evaluation model to obtain an evaluation result related to the portability of the model to be transplanted for the target domestic GPU platform, thereby improving the decision-making efficiency of model transplantation, reducing the risk of performance degradation or resource waste caused by blind transplantation, providing a strong guarantee for the performance and accuracy of the model after transplantation, and providing technical support and decision-making basis for developers. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of the processing flow of a model portability evaluation method based on a domestic GPU platform provided by the present invention.

[0034] Figure 2 A structural schematic diagram of a portability evaluation model constructed by a model portability evaluation method based on a domestic GPU platform provided by the present invention.

[0035] Figure 3 A structural schematic diagram of a model portability evaluation system based on a domestic GPU platform provided by the present invention. DETAILED DESCRIPTION

[0036] The present invention is described in detail with reference to the following embodiments.

[0037] The present invention provides a model portability evaluation method and system based on a domestic GPU platform. The basic idea is to collect model features of a model that has been transplanted to a domestic GPU platform and a variety of performance evaluation features after transplantation to establish a sample data set, construct a portability evaluation model based on the improvement of a multi-objective modeling algorithm using a gradient decision boosting tree algorithm, train the portability evaluation model using the established sample data set, and when in use, input the extracted model features of the model to be transplanted and the target domestic GPU platform into the trained portability evaluation model to obtain an evaluation result related to the portability of the model to be transplanted for the target domestic GPU platform.

[0038] The present invention provides a model portability evaluation method based on a domestic GPU platform, the process of which is as follows: Figure 1 As shown, the specific steps include:

[0039] Step 1. Obtain a model that has been transplanted to a domestic GPU platform as a sample model, and perform a portability evaluation on the transplantation process of the sample model to obtain the operator performance, model performance and portability evaluation results of the sample model on the current domestic GPU platform. Extract model features such as the model name, technical field, input dimension, output dimension, parameter quantity and computational amount of the sample model. The model evaluation features are composed of model features, operator performance, model performance, evaluation results and domestic GPU platform models. The model evaluation features of all sample models constitute a sample data set.

[0040] Among them, there are domestic GPU platform models, such as Lingjiuwei GP201, Jingjiawei JM9271, etc. Different domestic GPU platform models support different operators, and the performance of operators running on them, that is, operator performance, is also different.

[0041] Both input dimensions and output dimensions are represented by tuples. The input dimensions are related to the technical field. For example, the input dimensions in the field of image processing technology are represented by N, H, W, C, where N is the number of samples, H is the height, W is the width, and C is the number of channels, such as (1, 320, 320, 3).

[0042] The number of parameters is measured in megabytes (M), the amount of computation is measured in gigabytes (G), and is expressed in MACs. MACs (Multiply-Accumulate Operations) refers to the number of multiplication and addition operations of the model.

[0043] The operator performance of the model is represented by a tuple, where each data is a two-tuple (OpsName, OpsTime), where OpsName is the operator name and OpsTime is the operator time. The operator time refers to the time it takes the operator to calculate the set amount of input data, in microseconds, for example: ((Conv2d,762),(BatchNorm2d,1023),(Linear,193),…).

[0044] Model performance refers to the running time of the entire model on a domestic GPU platform, expressed in milliseconds.

[0045] The evaluation result is the portability evaluation result, which is expressed as a bigram (EvalIndicators, EvalScore), where EvalIndicators is the evaluation indicator and EvalScore is the evaluation score. The evaluation indicator is related to the technical field. For example, the field of image processing technology usually uses indicators such as accuracy, precision, recall, and F1 score. The field of natural language processing technology usually uses indicators such as bilingual evaluation unit (BLEU), summary evaluation based on recall unit (ROUGE), exact match rate (Exact Match), and ROC curve.

[0046] Taking the ResNet50 model as an example, the model evaluation characteristics are shown in Table 1:

[0047] Table 1-ResNet50 model evaluation feature table

[0048] Model Name field Domestic GPU models Input Dimensions Output Dimensions Parameter quantity (M) Computational Amount (GMACs) Operator Performance Model performance (ms) Evaluation results ResNet50 ComputerVision GP201 (1,96,96,3) (1,100) 25.56 4.14 ((Conv2d,762),(BatchNorm2d,1023), (Linear,193),…) 12 (Accuracy,0.8997)

[0049] In order to gain a deeper understanding of the information of the model to be transplanted, the present invention converts the model to be transplanted into a model file in the Open Neural Network Exchange (ONNX) format. The ONNX format can retain the weights, structural information, input and output of each layer in the network, and other auxiliary information of the neural network model. The specific model conversion process includes the following steps:

[0050] Step 1.1.1. Obtain the framework used by the model to be transplanted, such as TensorFlow, PyTorch, Keras and other deep learning frameworks;

[0051] Step 1.1.2, use the Python-based ONNXMLTools toolkit to convert the model according to the framework used by the model to be ported. For example, the model of the Keras framework is converted to an ONNX model using the convert_keras() function in ONNXMLTools, and the model of the TensorFlow framework is converted to an ONNX model using the convert_tensorflow() function in ONNXMLTools. If the conversion is not successful, execute step 1.1.3;

[0052] Step 1.1.3, use the API corresponding to the deep learning framework to read the model name, technical field, network structure, input dimension and output dimension of the model to be transplanted;

[0053] Step 1.1.4: According to the read network structure, input dimension, and output dimension, use the API corresponding to the deep learning framework to map and reconstruct the model to be transplanted into an ONNX model. For example, the model of the PyTorch deep learning framework is converted into an ONNX model using the torch.onnx.export() function.

[0054] On this basis, the ONNX model is parsed to obtain information such as network structure, operator name, parameter quantity, and calculation amount. The parsing process includes:

[0055] Step 1.2.1, load the ONNX model file, use the load() function in the onnx library based on Python language to load the ONNX model, and get the model's calculation graph GraphProto; loop through GraphProto to get the node information NodeProto, that is, the operator information, and then traverse the operator to get the operator's properties and values, as shown in the following code segment:

[0056] for node in GraphProto.graph.node: # Traverse the nodes of GraphProto

[0057] print(node.name) # Operator name

[0058] print(node.op_type) # operator type

[0059] print(node.input) # operator input

[0060] print(node.output) # operator output

[0061] for attr in node.attribute: # traverse the attributes of the node

[0062] print(attr) # operator attribute name and value

[0063] Step 1.2.2. Obtain the weight and bias of the operator according to the operator information obtained in step 1.2.1, and find the parameter calculation formula corresponding to the operator type according to the shape of the operator type, input tensor, and output tensor, and calculate the operator parameter.

[0064] For example, when bias=True, the calculation formula of the Conv2d operator parameter is as follows:

[0065] Parameter=out_channels*in_channels*kernel_size*kernel_size+ out_channels

[0066] When bias=False, the calculation formula of the Conv2d operator parameters is as follows:

[0067] Parameter=out_channels*in_channels*kernel_size*kernel_size

[0068] Among them, out_channels and in_channels are the number of output and input channels respectively, and kernel_size is the size of the convolution kernel.

[0069] For example, when bias=True, the parameter calculation formula of the Linear operator is as follows:

[0070] Parameter=out_features*in_features+out_features

[0071] When bias=False, the parameter calculation formula of the Linear operator is as follows:

[0072] Parameter=out_features*in_features

[0073] Among them, out_features and in_features are the output and input feature numbers respectively.

[0074] Finally, the parameters of all operators are accumulated to obtain the parameters of the model to be transplanted.

[0075] Step 1.2.3: Based on the operator information in step 1.2.1, determine the corresponding calculation formula according to the type of operator, obtain the shape of the input and output tensors of the operator, and calculate the calculation amount of the operator.

[0076] For example, the calculation formula for the Conv2d operator is as follows:

[0077] MAC=out_channel*out_height*out_width*(in_channels*kernel_size*kernel_size)

[0078] Among them, out_channels and in_channels are the number of output and input channels respectively, kernel_size is the size of the convolution kernel, out_height is the output height, and out_width is the output width.

[0079] For example, the calculation formula for the linear operator is as follows:

[0080] MAC=out_features*in_features

[0081] Among them, out_features and in_features are the output and input feature numbers respectively.

[0082] Finally, the computational cost of all operators is accumulated to obtain the computational cost MACs of the model to be transplanted.

[0083] Step 1.2.4: Organize the information obtained and calculated in steps 1.2.1, 1.2.2, and 1.2.3 into a model evaluation feature table as shown in Table 1.

[0084] Step 2: Assign ID values ​​to the model evaluation features such as model name, technical field, domestic GPU platform model, operator name and evaluation index in the sample data set constructed in step 1, then standardize the model evaluation features, and finally add the identification ID of the prediction task to the model evaluation features. The prediction task includes the operator performance prediction task and the evaluation result prediction task, thus completing the data preprocessing of the sample data set.

[0085] The data preprocessing process of the present invention for the model evaluation features in the sample data set includes the following steps:

[0086] Step 2.1: Identify the model name, technical field, domestic GPU platform model, operator name, and evaluation index, and assign an ID value.

[0087] Among them, an ID value is assigned to the model name, and a model name list is formed by the model names of all model evaluation features in the sample data set. The model name list is sorted in English letters using the sorted() function based on the Python language, and then an ID value is assigned to each model name. For example, AlexNet is assigned a value of "1", ResNet50 is assigned a value of "2", YoloV5 is assigned a value of "3", and so on to complete the identification of all model names.

[0088] Assign ID values ​​to technical fields, form a technical field list from the technical fields of all model evaluation features in the sample data set, and use the sorted() function based on the Python language to sort the technical field list alphabetically. Then assign an ID value to each technical field. For example, assign the value "1" to the computer vision technology field (CV), the value "2" to the natural language processing technology field (NLP), the value "3" to the recommendation technology field (Recommendation), and so on to complete the identification of all technical fields.

[0089] Assign ID values ​​to domestic GPU platform models, form a list of domestic GPU models of all model evaluation features in the sample data set, and use the sorted() function based on the Python language to sort the list of domestic GPU models in alphabetical order. Then assign an ID value to each domestic GPU model. For example, assign a value of "1" to GP201, a value of "2" to JM9271, and so on to identify all domestic GPU models.

[0090] Assign ID values ​​to operator names. A list is formed from the operator names of all model evaluation features in the sample data set. The sorted() function based on the Python language is also used to sort the operator name list in alphabetical order. Then, an ID value is assigned to each operator name. For example, the value "1" is assigned to BatchNorm2d, the value "2" is assigned to the Conv2d operator, the value "3" is assigned to the Linear operator, and so on to identify all operator names. Examples of operator performance tuples after operator name identification: ((1,1023), (2,762), (3,193), ...).

[0091] Assign ID values ​​to the evaluation indicators. The evaluation indicators of all model evaluation features in the sample data set form a list. The sorted() function based on the Python language is also used to sort the evaluation indicator list in alphabetical order. Then, an ID value is assigned to each evaluation indicator. For example, the value "1" is assigned to Accuracy, the value "2" is assigned to F1 Score, the value "3" is assigned to Precision, the value "4" is assigned to Recall, and so on to identify all evaluation indicators. An example of an evaluation result tuple after the evaluation indicator is identified: (1,0.8997).

[0092] Taking the ResNet50 model as an example, the model evaluation characteristics after data identification processing are shown in Table 2:

[0093] Table 2-ResNet50 data identification model evaluation feature table

[0094] Model Name field Domestic GPU models Input Dimensions Output Dimensions Parameter quantity (M) Computational Amount (GMACs) Operator Performance Model performance (ms) Evaluation results 2 1 1 (1,96,96,3) (1,100) 25.56 4.14 ((1,1023),(2,762),(3,193),…) 12 (1,0.8997)

[0095] Step 2.2: Use the StandardScaler() function based on the machine learning tool Sklearn (Scikit-learn) in Python language to standardize the model evaluation features after data identification.

[0096] Step 2.3: Classify the model evaluation features standardized in step 2.2 by task, and divide them into two tasks: operator performance prediction task and evaluation result prediction task. The identification task, operator performance prediction task, is represented by the value "0", and the evaluation result prediction task is represented by the value "1". The operator performance prediction task refers to the task used to predict the running time of all operators of the transplanted model when it runs on the domestic GPU platform, and the evaluation result prediction task is the task used to predict the evaluation scores of each evaluation index when the transplanted model runs on the domestic GPU platform. The model evaluation features after task division are shown in Table 3:

[0097] Table 3-ResNet50 task division model evaluation characteristics table

[0098] Task ID Model Name field Domestic GPU models Input Dimensions Output Dimensions Parameter quantity (M) Computational Amount (GMACs) Operator Performance Model performance (ms) Evaluation results 0 2 1 1 (1,96,96,3) (1,100) 25.56 4.14 ((1,1023),(2,762),(3,193),…) 12 -- 1 2 1 1 (1,96,96,3) (1,100) 25.56 4.14 -- 12 (1,0.8997)

[0099] Step 2.4: Use Pandas, a data analysis tool in Python, to write the model evaluation feature set after the task division in step 2.3 into a character-separated value file (CSV) to form a feature data set.

[0100] Step 3: Split the feature data set formed in step 2 into two CSV files: training set and test set.

[0101] First, use Python’s open file function, that is, the open() function, to read the feature data set CSV file; then, use the train_test_split() function of the Python-based machine learning tool Sklearn (Scikit-learn) to split the read data, such as randomly sampling 80% of the data to build a training set, and the rest of the data as a test set, to form a training set and a test set.

[0102] Step 4: Build a portability evaluation model based on deep learning. The structure of the portability evaluation model is as follows: Figure 2 As shown, the training set formed in step 3 is used to complete the training of the portability evaluation model.

[0103] In order to realize the portability evaluation of the model to the domestic GPU platform, that is, to predict the model performance, operator performance and evaluation results of the model to be ported on the domestic GPU platform, the present invention improves the PLE algorithm in the deep learning algorithm. The present invention constructs a portability evaluation model based on the improved PLE algorithm, and trains the portability evaluation model using the two tasks divided in the training set formed in step 3.

[0104] The portability evaluation model constructed based on the improved PLE algorithm of the present invention is composed of the PLE algorithm and the GBDT algorithm, specifically: the two outputs of the PLE algorithm, namely the evaluation result prediction value and the operator performance prediction value, are integrated into a combined prediction value as the final output; then the combined prediction value of the input and output of the PLE algorithm is used as the input of the GBDT algorithm, and the GBDT algorithm further classifies the prediction results of the PLE algorithm. The output of the GBDT algorithm is the model performance prediction value, which is finally composed of the evaluation result prediction value, the operator performance prediction value and the model performance prediction value.

[0105] Among them, the training set formed in step 3 is used to complete the training of the portability evaluation model. The specific process is: the model evaluation characteristics of the samples in the training set are used as the input of the PLE algorithm in the portability evaluation model as the first input, and the first input includes model characteristics, operator performance, model performance, evaluation results and domestic GPU platform models; the PLE algorithm processes the first input to obtain the evaluation result prediction value and the operator performance prediction value, which are recorded as the first output; then the parameters of the first input except the operator performance, model performance and evaluation results and the first output are used together as the second input, and the GBDT algorithm processes the second input to obtain the model performance prediction value, which is recorded as the second output; finally, the first output and the second output constitute the prediction result of the portability evaluation model output.

[0106] Then, the test set formed in step 3 is used to test the trained portability evaluation model, that is, to test whether the model fits the test set. The present invention evaluates the portability evaluation model by calculating the consistency index (C-index value) and correcting the C-index value using the Bootstrap method, specifically:

[0107] Step 4.1: Input the test set data into the trained portability evaluation model to obtain prediction results, including the evaluation result prediction value, operator performance prediction value, and model performance prediction value.

[0108] Step 4.2: Calculate the C-index value based on the predicted results and the actual results in the test set. The calculation process is: C-index = abs ( GAUC ) / 2+0.5, where GAUCis the index parameter in the PLE algorithm, and its value range is [0,1]. For example, if GAUC The value of is 0.472, so the C-index value is calculated as 0.472 / 2+0.5=0.736.

[0109] Step 4.3: Use the Bootstrap method to correct the C-index value and finally obtain the C-index score. Specifically, the Bootstrap method estimates the deviation and variation of the C-index value through repeated sampling, and then obtains the corrected C-index value. For example, after 500 Bootstrap samplings, the original C-index value is 0.472, and the corrected C-index value is 0.4348 / 2+0.5=0.719‌.

[0110] Step 4.4, determine whether the current portability evaluation model meets the requirements by setting the threshold of the C-index value. For example, the threshold is set to 0.7, that is, when the C-index value is greater than or equal to 0.7, it is determined that the current portability evaluation model can be used to evaluate the portability of the model to the domestic GPU platform; otherwise, adjust the training parameters or expand the training set and then retrain the portability evaluation model until the C-index value is greater than 0.7.

[0111] Step 5. In actual use, extract the model features of the model to be transplanted, including model name, technical field, input dimension, output dimension, parameter quantity and calculation amount, and input the extracted model features and the target domestic GPU platform into the portability evaluation model trained to obtain the output operator performance, model performance and portability evaluation results.

[0112] The method of extracting the model features of the model to be transplanted is to extract them using the method of step 1.

[0113] Furthermore, the present invention can display the output information of step 5 in a graphical form, so that the user can more intuitively see the prediction information of the model to be transplanted, and can quickly judge the portability of the model based on this information, and further optimize and adjust the model to be transplanted. In addition, a list of operator names that are not supported by different domestic GPU platforms can also be obtained.

[0114] Furthermore, in order to continuously improve the diversity of the sample data set, the present invention adds the model characteristics of the model to be transplanted in step 5 and the obtained operator performance, model performance and portability evaluation results as sample data to the sample data set, and trains the portability evaluation model when the sample data set is updated to the set ratio.

[0115] The present invention provides a model portability evaluation system based on a domestic GPU platform, the structure of which is as follows: Figure 3As shown, it includes a model feature acquisition module, a feature preprocessing module, a portability evaluation model training module, a model conversion module, a model parsing module, a portability evaluation model module and an evaluation result display module.

[0116] Among them, the model feature collection module is used to collect model features of models that have been ported to the domestic GPU platform or models to be ported. At the same time, for the ported models, the operator performance, model performance and portability evaluation results after porting to the domestic GPU platform are also collected. Among them, the model features include model name, technical field, input dimension, output dimension, parameter quantity and calculation amount. The operator performance is represented by a multi-tuple, and each data in the multi-tuple is a two-tuple containing the operator name and the operator time consumption. The evaluation result is represented by a two-tuple containing the evaluation index and the evaluation score.

[0117] The feature preprocessing module is used to perform data preprocessing on the model evaluation features, including data identification processing, standardization processing, type conversion, etc., to form a CSV format feature file. At the same time, a sample data set is constructed from the format feature file of the transplanted model, and the sample data set is randomly sampled and divided into a training set and a test set.

[0118] The portability evaluation model training module builds a portability evaluation model based on the improved PLE algorithm, uses the training set to complete the training of the portability evaluation model, and uses the C-index evaluation indicator to test the portability evaluation model to evaluate the quality of the trained portability evaluation model.

[0119] The model conversion module is used to read the model to be transplanted imported by the user, and convert it into an ONNX format model by using information such as the framework, network structure, input dimension and output dimension, so that the system can deeply understand the information of the model to be transplanted.

[0120] The model parsing module is used to parse the ONNX model output by the model conversion module to obtain the network structure, input dimension, output dimension, operator name, etc., and calculate the model's parameter quantity and calculation amount to form a model feature information list.

[0121] The portability evaluation model module analyzes the model characteristics of the input model to be transplanted to obtain the prediction results of the model to be transplanted, including operator performance, model performance and portability evaluation results.

[0122] The evaluation result display module is used to display the model portability results to users in the form of charts, including evaluation results, operator performance and other prediction information. The evaluation result information includes evaluation indicators and evaluation scores; the operator performance information includes the running time of the model to be transplanted, the calculation time of each operator, the list of unsupported operator names, etc. Users can optimize and adjust the model to be transplanted based on these results.

[0123] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A model portability evaluation method based on a domestic GPU platform, characterized in that: The specific steps include: Step 1: Take the model that has been transplanted to the domestic GPU platform as the sample model, extract the model features of the sample model, evaluate the portability of the sample model to the domestic GPU platform to obtain the operator performance, model performance and portability evaluation results, the model features, operator performance, model performance, portability evaluation results and domestic GPU platform models constitute the model evaluation features, and the model evaluation features of all sample models constitute the sample data set; the model features include model name, technical field, input dimension, output dimension, parameter quantity and calculation amount; perform data preprocessing on the sample data set and divide it into a training set and a test set; Step 2, construct a portability evaluation model, take the model characteristics of the model to be transplanted and the target domestic GPU platform model as the evaluation model input, first use the multi-objective modeling algorithm to calculate the evaluation model input to obtain the operator performance and portability evaluation results of the model to be transplanted on the target domestic GPU platform, then use the gradient decision boosting tree algorithm to calculate the operator performance, portability evaluation results and evaluation model input to obtain the model performance, and use the operator performance, model performance and portability evaluation results as the evaluation model output; Step 3: Use the sample data set constructed in step 1 to complete the training of the portability evaluation model; Step 4: In actual use, extract the model features of the model to be ported, input the extracted model features and the target domestic GPU platform model into the trained portability evaluation model to obtain the output operator performance, model performance and portability evaluation results.

2. The model portability evaluation method according to claim 1, characterized in that: The method of extracting the model features of the sample model in step 1 is: The model name, technical field, network structure, input dimension and output dimension of the sample model are read according to the framework used by the sample model, and then the sample model is converted into a model file in the open network neural exchange format according to the network structure, input dimension and output dimension; the model file is parsed to obtain the parameter quantity and calculation amount of the sample model.

3. The model portability evaluation method according to claim 1, characterized in that: The method of preprocessing the sample data set in step 1 is: Step 2.1, data identification is performed on the model name, technical field, domestic GPU platform model, operator name, and evaluation index, that is, an ID value is assigned; Step 2.2: Use the StandardScaler() function to standardize the model evaluation features after data identification processing; Step 2.3, classify the model evaluation features by task, that is, divide them into operator performance prediction tasks and portability evaluation result prediction tasks, and identify the model evaluation feature tasks; Step 2.4: Use data analysis tools to write the model evaluation feature set after task division into a character-separated value file to form a feature data set.

4. The model portability evaluation method according to claim 1, characterized in that: The step 3 uses the sample data set constructed in step 1 to complete the training of the portability evaluation model, specifically: Step 3.1, taking the model evaluation features of the samples in the training set as the input of the multi-objective modeling algorithm in the portability evaluation model as the first input, the first input includes model features, operator performance, model performance, portability evaluation results and domestic GPU platform model; after the multi-objective modeling algorithm processes the first input, it obtains the portability evaluation result prediction value and the operator performance prediction value, which are recorded as the first output; Then, the parameters in the first input except the operator performance, model performance and portability evaluation result and the first output are taken as the second input, and the gradient decision boosting tree algorithm processes the second input to obtain the model performance prediction value recorded as the second output; finally, the first output and the second output constitute the prediction result of the portability evaluation model output; Step 3.2: Use the test set to test the trained portability evaluation model. If the set conditions are met, the training is completed; otherwise, the training is performed again.

5. The model portability evaluation method according to claim 4, characterized in that: The setting condition in step 3.2 is as follows: calculating the consistency index C-index value, correcting the C-index value using the Bootstrap method, and satisfying the setting condition if the C-index value is not less than the set threshold.

6. The model portability evaluation method according to claim 1, characterized in that: The output of the portability evaluation model in step 4 is displayed in a graphical form.

7. The model portability evaluation method according to claim 1, characterized in that: The operator performance is represented by a tuple, in which each data is a tuple including an operator name and an operator time consumption; the portability evaluation result is represented by a tuple including an evaluation index and an evaluation score.

8. A model portability evaluation system based on a domestic GPU platform, characterized in that: Specifically, it includes a model feature acquisition module, a feature preprocessing module, a portability evaluation model training module, a model conversion module, a model parsing module and a portability evaluation model module; The model feature acquisition module is used to acquire model features of the model, and acquire operator performance, model performance and portability evaluation results of the transplanted model on the domestic GPU platform, and form model evaluation features from the model features and operator performance, model performance and portability evaluation results of the transplanted model, wherein the model features include model name, technical field, input dimension, output dimension, parameter quantity and calculation quantity; The feature preprocessing module is used to perform data preprocessing on the model features and model evaluation features, including data identification processing, standardization processing and type conversion, to form a CSV format feature file, and to construct a sample data set from the format feature file of the transplanted model, and to divide the sample data set into a training set and a test set; The portability evaluation model training module is used to train the portability evaluation model using a training set, and to test the portability evaluation model on a test set using a C-index evaluation indicator, and the training is completed when the test result meets the set conditions; The model conversion module is used to read the model to be transplanted and convert it into an ONNX format model file; The model parsing module is used to parse the ONNX format model file to obtain the network structure, input dimension, output dimension and operator name, and calculate the parameter amount and calculation amount of the model to form the model features; The portability evaluation model module is used to analyze the model features of the input model to be transplanted to obtain the prediction results of the model to be transplanted, including operator performance, model performance and portability evaluation results.

9. The model portability evaluation system according to claim 8, characterized in that: The model portability evaluation system further comprises an evaluation result display module, and the evaluation result display module is used to display the model portability result to the user in a graphical manner.

Citation Information

Patent Citations

  • System and method for evaluating feasibility of lung transplantation

    CN115036027A

  • Protein-ligand affinity evaluation method based on domestic supercomputing platform

    CN117292759A

  • Ovarian cancer CD8 + T cell related prognosis evaluation method and system and application thereof

    CN117766024A

  • Kidney transplantation graft prognosis evaluation method and system based on multi-modal deep learning

    CN117974592A

  • Multi-modal three-dimensional detection method based on sparse proxy attention

    CN118351410A