Deep learning model conversion method and device, equipment and storage medium

Abstract

The invention provides a deep learning model conversion method and device, equipment and a storage medium, and belongs to the technical field of deep learning. The method comprises the steps of obtaining model data and metadata of an original model; constructing a model structure of the target model according to a model structure in the model data of the original model; traversing operator information of any original operator in the computational graph of the original model according to the computational graph of the original model, and generating a target operator corresponding to the original operator through an operator translation model according to the metadata of the original model and the operator information of the original operator, the calculation graph of the original model comprises an original operator corresponding to any function of the original model, and the operator translation model is used for converting operators of different front-end frames; and obtaining a calculation graph of the target model according to all the target operators obtained through conversion, and generating a converted target model corresponding to the original model. The invention aims to improve the conversion efficiency of the deep learning model.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of deep learning, and specifically, to a deep learning model conversion method, apparatus, device and storage medium. Background Art

[0002] With the rapid development of deep learning, more and more machine learning models have emerged. However, due to differences in different R&D teams and technical routes, many model representation methods in different formats have also been generated, such as machine learning models developed using different front-end frameworks, including Caffe, Pytorch, Onnx, and Tensorflow. In actual engineering applications, the model generated based on the front-end framework is not the final model with the best performance. If the model's reasoning needs to perform at its best, the model also needs to be run on a specified deep learning chip, that is, the model deployment process. Deep learning chips such as GPU, TPU, and NPU.

[0003] During the deployment of deep learning models, different backend frameworks support different input model formats, and there is no guarantee that every backend framework will support models generated by all types of front-end frameworks. However, the structures and parameters represented by different machine learning models are basically the same, making it possible to convert models developed under different front-end frameworks.

[0004] The current model conversion process mainly includes: model loading, calculation graph conversion, node mapping, parameter weight conversion, saving the new model, and accuracy verification. However, in the node mapping process, due to the large number of operators in different front-end frameworks, such as Onnx 1.18.0, there are 193 operators. If the operator mapping is established manually, it is too complicated. When mapping operators, not only the functional meaning of the operator but also the attribute meaning of the operator need to be considered. The professional skills of the personnel are relatively high, resulting in inefficient node mapping process, which makes the model conversion process inefficient. Summary of the invention

[0005] The embodiments of the present application provide a deep learning model conversion method, apparatus, device and storage medium, aiming to improve the efficiency of deep learning model conversion.

[0006] In a first aspect, an embodiment of the present application provides a deep learning model conversion method, the method comprising: Obtain model data and metadata of the original model; Constructing a model structure of a target model according to the model structure in the model data of the original model; According to the calculation graph of the original model, traverse the operator information of any original operator in the calculation graph of the original model, and generate a target operator corresponding to the original operator through an operator translation model according to the metadata of the original model and the operator information of the original operator, wherein the calculation graph of the original model includes the original operator corresponding to any function of the original model, and the operator translation model is used to convert operators of different front-end frameworks; According to all the target operators obtained by the conversion, a calculation graph of the target model is obtained, and a converted target model corresponding to the original model is generated.

[0007] Optionally, traversing the operator information of any original operator in the calculation graph of the original model, and generating a target operator corresponding to the original operator through an operator translation model according to the metadata of the original model and the operator information of the original operator, including: For any original operator, perform word segmentation through a word segmenter based on the operator information of the original operator to obtain a first word segmentation sequence, wherein the operator information includes the name, input and output, and operator attribute information of the operator; Inputting the first word segmentation sequence, the metadata of the original model, and a natural language processing command into a pre-trained operator translation model to obtain a second word segmentation sequence output by the operator translation model, wherein the natural language processing command is used to instruct the operator translation model to convert the operator of the original model into an operator corresponding to the target model; The second word segmentation sequence is restored through a word segmenter to obtain a target operator corresponding to the original operator.

[0008] Optionally, the method further comprises: Constructing a first corpus set, wherein the first corpus set includes operator information of any operator; Constructing a second corpus set, wherein the second corpus set includes model structure texts of models of different front-end frameworks; Performing preliminary training on a large language model based on the first corpus set; The large language model is optimized and trained based on the second corpus set to obtain a trained operator translation model.

[0009] Optionally, before obtaining a computation graph of the target model according to all target operators obtained by conversion, the method further includes: In response to the operator configuration operation, obtain the conversion operator corresponding to any original operator; The conversion operator corresponding to the original operator is used as the target operator of the original operator.

[0010] Optionally, after obtaining the original model, the method further includes: The original model is standardized, and any function of the original model is replaced by a corresponding operator.

[0011] Optionally, after constructing the model structure of the target model according to the model structure of the original model, the method further includes: Obtaining initial parameters of the original model; The initial parameters of the target model are configured according to the initial parameters of the original model.

[0012] Optionally, after generating the converted target model corresponding to the original model, the method further includes: Inputting the validation data set into the original model to obtain the original output; Inputting the validation data set into the converted target model to obtain a target output; A cosine distance between the original output and the target output is calculated as a verification result of the target model.

[0013] In a second aspect, an embodiment of the present application provides a deep learning model conversion device, the device comprising: An acquisition module, used to acquire model data and metadata of the original model; A model structure building module, used to build a model structure of a target model according to the model structure in the model data of the original model; An operator conversion module is used to traverse the operator information of any original operator in the calculation graph of the original model according to the calculation graph of the original model, and generate a target operator corresponding to the original operator through an operator translation model according to the metadata of the original model and the operator information of the original operator. The calculation graph of the original model includes the original operator corresponding to any function of the original model, and the operator translation model is used to convert operators of different front-end frameworks; The target model generation module is used to obtain the calculation graph of the target model according to all target operators obtained by conversion, and generate the converted target model corresponding to the original model.

[0014] In a third aspect, an embodiment of the present application provides a computer device, comprising: at least one processor, and a memory, wherein the memory stores a computer program that can be run on the processor, wherein when the processor executes the computer program, the deep learning model conversion method described in the first aspect of the embodiment is performed.

[0015] In a fourth aspect, an embodiment of the present application provides a non-volatile readable storage medium, which stores a computer program, wherein when the computer program is executed by a processor, the deep learning model conversion method described in the first aspect of the embodiment is performed.

[0016] Beneficial effects: Acquire the model data and metadata of the original model; construct the model structure of the target model according to the model structure in the model data of the original model; traverse the operator information of any original operator in the calculation graph of the original model according to the calculation graph of the original model, and generate the target operator corresponding to the original operator through the operator translation model according to the metadata of the original model and the operator information of the original operator, wherein the calculation graph of the original model includes the original operator corresponding to any function of the original model, and the operator translation model is used to convert operators of different front-end frameworks; obtain the calculation graph of the target model according to all target operators obtained by conversion, and generate the converted target model corresponding to the original model.

[0017] This method can build the model structure of the target model according to the model structure of the original model, and then automatically convert any original operator in the original model into the target operator of the target model. Compared with manually establishing a complex operator mapping process, the efficiency of the model conversion process is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 is a flowchart of the steps of the deep learning model conversion method proposed in one embodiment of the present application; Figure 2 is a flowchart of the steps of training an operator translation model proposed in an embodiment of the present application; Figure 3 is a flowchart of the steps of the operator translation process provided by an embodiment of the present application; Figure 4 is a schematic diagram of a deep learning model conversion method proposed in an embodiment of the present application; Figure 5 It is a functional module diagram of a deep learning model conversion device proposed in one embodiment of the present application; Figure 6 is a schematic diagram of a computer device provided by an embodiment of the present application; Figure 7 It is a schematic diagram of a non-volatile readable storage medium proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] Onnx: Open Neural Network Exchange, open neural network exchange, is an open model format standard that aims to achieve model interoperability between different deep learning frameworks; Onnx defines a set of standard deep learning network operation methods, and other frameworks can achieve the unification of model formats under different frameworks by exporting to Onnx.

[0022] ‌Caffe: Convolutional Architecture for Fast Feature Embedding, is a deep learning framework that combines expressiveness, speed, and modularity, developed by the Berkeley Artificial Intelligence Research Group and the Berkeley Vision and Learning Center‌. The kernel of Caffe is written in C++, but it provides Python and Matlab interfaces, supporting multiple types of deep learning architectures, including convolutional neural networks (CNN), recurrent convolutional neural networks (RCNN), long short-term memory networks (LSTM), and fully connected neural networks; PyTorch, an open source machine learning library, is mainly used for model training. It is known for its excellent flexibility and support for dynamic graphs. It provides a Python-based front-end interface that makes the construction and training of deep learning models more convenient. ‌TensorFlow‌ is an open source machine learning framework that is mainly used for numerical computing and supports a variety of machine learning and deep learning tasks, including natural language processing (NLP), computer vision, audio and video analysis, etc. ‌; GPU: Graphics Processing Unit, graphics processor; TPU: Tensor Processing Unit, tensor processor; NPU: Neural-network Processing Unit, neural network processor; MLU: Hardware device for accelerating deep learning; IR: Intermediate Representation, an intermediate format used to describe the model compilation, interpretation or conversion process, and is used for information transfer and optimization between models at different stages.

[0023] With the rapid development of deep learning, more and more machine learning models have emerged. However, due to differences in different R&D teams and technical routes, many model representation methods in different formats have also emerged, such as machine learning models developed using different front-end frameworks, including Caffe, Pytorch, Onnx, and Tensorflow.

[0024] In actual engineering applications, the model generated based on the front-end framework is not the final model with the best performance. If the model's reasoning needs to perform at its best, the model also needs to be run on a specified deep learning chip, that is, the model's deployment process, deep learning chips such as GPU, TPU, and NPU.

[0025] During the deployment of deep learning models, the input model formats supported by different backend frameworks vary greatly, and it cannot be guaranteed that every backend framework provides support for models generated by all types of front-end frameworks. However, the structures and parameters represented by different machine learning models are basically the same, which makes it possible to convert models developed under different front-end frameworks. The mutual conversion between different models under the front-end framework can provide better support for the large-scale application of models, and models that were not supported before have become possible at this moment.

[0026] In the current implementation technology, according to the different sources of the models that need to be converted, they are mainly divided into the following categories: (1) The network architecture of the models is the same, but the weights are different. In this case, the weights of one model can be directly copied to another model; (2) The front-end framework of the model is the same, but the model structure is different: In this case, some layers of one model can be used as feature extractors, and other layers can be used as layers of another model. This is generally achieved using transfer learning methods. (3) The front-end frameworks of the models are different and the model structures are also different. Technologies such as knowledge distillation are usually used to transfer the knowledge of a large model to another smaller model. This is generally used to reduce the size and computational complexity of the model while maintaining the performance of the model. (4) The front-end framework of the model is different but the model structure is the same: In this case, you can use frameworks such as Onnx as middleware, or directly translate the model to the model operator by operator, or use some model conversion tools to achieve it. Usually, the process of a model conversion mainly includes: model loading, calculation graph conversion, node mapping, parameter weight conversion, saving the new model, and accuracy verification.

[0027] However, in the node mapping process, due to the large number of operators in different front-end frameworks, for example, there are 193 operators in Onnx 1.18.0, it is too complicated to manually establish operator mapping. When mapping operators, it is necessary to consider not only the functional meaning of the operator, but also the attribute meaning of the operator, which requires high professional skills of personnel, resulting in inefficient node mapping process, and thus inefficient model conversion process.

[0028] Therefore, an embodiment of the present application provides a deep learning model conversion method to improve the efficiency of deep learning model conversion.

[0029] Reference Figure 1 , shows a flowchart of a deep learning model conversion method in an embodiment of the present application, and the method may specifically include the following steps: S101: Obtain model data and metadata of the original model.

[0030] In this method, the front-end frameworks of the original model and the target model are different, and the back-ends of the deployed applications are also different, but the model structures of the original model and the target model are the same.

[0031] For example, the original model A developed based on the front-end framework 1 is to be applied to the back-end framework 1, but the back-end framework only supports the models developed by the front-end framework 2. At this time, the original model A can be converted into a target model, and then the model service or model function provided by the original model A can be converted into a target model and deployed in the back-end framework 1.

[0032] When obtaining the original model, it includes obtaining the model data and metadata of the original model. The model data includes data such as model structure, model weights, and initial parameters. The metadata includes the IR version, opset version, input and output names, input and output data types, and input and output shapes of the model.

[0033] IR refers to an intermediate format used to describe the model compilation, interpretation or conversion process. Different front-end frameworks such as Pytorch and Onnx have different IR versions. The description of operator information of any operator in different IR versions is different. Only a certain IR version can clarify the operator information of any operator. The opset version in Onnx refers to the operation set version, which is the version that defines the set of operations used in the model; different opset versions contain a set of specific operations that define the specific calculations that can be performed in the model.

[0034] Input and output shapes refer to the data dimensions of any input data and any output data.

[0035] After obtaining the original model, the original model may be standardized by replacing any function of the original model with a corresponding operator to obtain a calculation graph of the original model.

[0036] In deep learning, deep learning algorithms are composed of computing units, which are called operators (OP). In any deep learning model, the operator corresponds to the computing logic in the layer, such as the convolutional layer is an operator; the weight summation process in the fully connected layer is also an operator.

[0037] After the original model is standardized, any function is replaced by an operator, and the calculation graph of the obtained original model includes all the original operators corresponding to the original model.

[0038] S102: Constructing a model structure of a target model according to the model structure in the model data of the original model.

[0039] Since the model conversion scenario applicable to this method is the conversion between two models with different front-end frameworks and back-end frameworks but the same model structure, the model structure of the target model can be constructed according to the model structure of the original model.

[0040] Model structure refers to the design or framework of a model, which defines the core components of any model and how they are connected. The model structure includes the model's layers, nodes, weights, connections, and other elements, as well as their layout and organization.

[0041] S103: According to the calculation graph of the original model, traverse the operator information of any original operator in the calculation graph of the original model, and generate a target operator corresponding to the original operator through an operator translation model according to the metadata of the original model and the operator information of the original operator. The calculation graph of the original model includes the original operator corresponding to any function of the original model, and the operator translation model is used to convert operators of different front-end frameworks.

[0042] The computational graph of the original model includes the original operators corresponding to all the functions of the original model. Since the front-end frameworks of the original model and the target model are different, the descriptions of the same operator are also different. Therefore, translation conversion of the original operator and the target operator is required. In this embodiment, the operator translation conversion process is performed based on the pre-trained operator translation model.

[0043] Specifically, the operator translation model is a large language model. By constructing a corpus collection, the large language model is pre-trained to allow the large language model to learn the operator information of different operators and the relationship between different front-end frameworks. The trained large language model can realize the conversion between operators of different front-end frameworks.

[0044] Reference Figure 2 , shows a flowchart of the steps of training the operator translation model provided in an embodiment of the present application. In a feasible implementation, the process of training the operator translation model includes the following steps: A1: Construct a first corpus set, wherein the first corpus set includes operator information of any operator.

[0045] The first corpus set includes operator information of any operator, and the operator information includes the name of the operator, input and output, and operator attribute information. For example, the name of operator 1 is Conv1, the shape of the input and output is the data dimension, and the operator attribute is Convolution, which means a convolution calculation is performed.

[0046] A2: Construct a second corpus set, wherein the second corpus set includes model structure texts of models of different front-end frameworks.

[0047] The second corpus set includes model structure texts of models of different front-end frameworks, such as a large number of model structure source codes of Pytorch models, Graph structures of Tensorflow, and model structure texts of Onnx model structures after being converted into text.

[0048] After constructing the first corpus set and the second corpus set, the first corpus set and the second corpus set are segmented, that is, the Token conversion process of the corpus is performed, the operator information of any operator is divided into words, and then converted into a context vector through the encoder, and then the context vector is used by the decoder to generate a Token sequence. This Token sequence is a natural language that can be recognized by the large language model.

[0049] A3: Perform preliminary training on the large language model based on the first corpus set.

[0050] First, the first corpus set that has undergone the Token conversion process is input into the large language model, so that the large language model can learn the operator information of different operators.

[0051] During the training process of the large language model, natural language can be used to provide training instructions to the large language model, such as the training instruction: "Please learn the information of the following corpus."

[0052] A4: Optimize and train the large language model based on the second corpus set to obtain a trained operator translation model.

[0053] Then, the large language model trained on the first corpus is fine-tuned again based on the second corpus that has undergone the Token conversion process, so that the large language model can learn the relationship between operators of different front-end frameworks.

[0054] Since the large language model is already relatively mature, in the actual implementation process, the large language model can be trained based on a large amount of operator corpus and the model structure of different front-end frameworks.

[0055] Reference Figure 3 , shows a flowchart of the steps of the operator translation process provided by an embodiment of the present application. Further, the operator information of any original operator in the calculation graph of the original model is traversed, and according to the metadata of the original model and the operator information of the original operator, the target operator corresponding to the original operator is generated through the operator translation model, including the following steps: B1: For any original operator, a word segmentation is performed through a word segmenter based on the operator information of the original operator to obtain a first word segmentation sequence, wherein the operator information includes the name, input and output, and operator attribute information of the operator.

[0056] First, the operator information of the original operator is segmented by a word segmenter, that is, the Token conversion process of the original operator is performed by the word segmenter to obtain the first word segmentation sequence of the original operator that can be recognized by the trained operator translation model.

[0057] B2: Input the first word segmentation sequence, the metadata of the original model, and the natural language processing command into a pre-trained operator translation model to obtain a second word segmentation sequence output by the operator translation model.

[0058] The metadata of the original model and the first word segmentation sequence of the original operator can allow the operator translation model to determine the accurate description of the original operator under the front-end framework of the original model. The operator translation model has learned the operator conversion relationship under different front-end frameworks through a large number of first corpus sets and second corpus sets. Therefore, the operator translation model can convert the first word segmentation sequence of the original segmentation into the second word segmentation sequence of the target operator corresponding to the target model.

[0059] The natural language processing command is used to instruct the operator translation model to convert the operator of the original model into the operator corresponding to the target model. For example, the natural language processing command may be "Please convert the IR version of operator 1 of front-end framework 1 into the operator under front-end framework 2".

[0060] B3: Restore the second word segmentation sequence through a word segmenter to obtain a target operator corresponding to the original operator.

[0061] Then, the second word segmentation sequence generated by the operator translation model is restored to obtain the target operator corresponding to any original operator.

[0062] In the process of the operator translation model converting the operators of the original models and target models of different front-end frameworks, when any operator exceeds the corpus training set of the operator translation model, such as the original operator is a custom operator and the operator translation model has not learned the operator information of the custom operator and the conversion relationship in different front-end frameworks, the operator translation model can be instructed through natural language to return a placeholder as a replacement, so that subsequent manual supplementation of the target operator of the original operator that failed to be translated by the operator translation model can be performed.

[0063] This method obtains an operator translation model by training a large language model. Based on the powerful natural language processing capability of the large language model, any original operator of the original model can be quickly translated into a target operator of the target model, thereby further accelerating the efficiency of model conversion.

[0064] In a feasible implementation, if the operator translation model fails to convert any original operator, such as when the operator translation model returns a placeholder, the user may be allowed to manually translate the original operator and then configure the target operator corresponding to the original operator. Alternatively, when an erroneous operator translation occurs in the operator translation model, the user may also correct and replace the target operator corresponding to the original operator. Specifically: In response to the operator configuration operation, obtain the conversion operator corresponding to any original operator; The conversion operator corresponding to the original operator is used as the target operator of the original operator.

[0065] In the actual implementation process, it can be set that the priority of the target operator corresponding to any artificially configured original operator is higher than the target operator translated by the operator translation model.

[0066] S104: Obtain a calculation graph of the target model according to all target operators obtained by conversion, and generate a converted target model corresponding to the original model.

[0067] By traversing any original operator in the calculation graph of the original model, generating the target operators corresponding to all the original operators, and manually correcting the erroneous target operators and supplementing the target operators, the calculation graph of the target model is obtained, and then the target model after the original model conversion can be obtained.

[0068] In the actual implementation process, since the original model includes some pre-configured constants of initial parameters in addition to the input parameters, the initial parameters of the original model can be obtained after the model structure of the target model is constructed according to the model structure of the original model; the initial parameters of the target model can be configured according to the initial parameters of the original model, so that the constants of the pre-configured initial parameters of the original model can be configured in the target model in advance.

[0069] In a feasible implementation manner, after generating the converted target model corresponding to the original model, the converted target model may be verified.

[0070] Specifically, the verification data set is input into the original model to obtain the original output; and the verification data set is input into the converted target model to obtain the target output.

[0071] Then, the cosine distance between the original output and the target output is calculated as a verification result of the target model.

[0072] A verification threshold may also be set. If the cosine distance between the original output and the target output is less than the verification threshold, it indicates that the conversion effect is good.

[0073] Reference Figure 4 , showing a schematic diagram of the deep learning model conversion method provided in an embodiment of the present application, the method includes a parsing process of the original model and a generation process of the target model. After the model data and metadata of the original model are obtained, the structure of the original model is parsed and the model structure of the target model is constructed.

[0074] Then, the constants of the initial parameters of the original model are parsed and obtained, corresponding to the constants of the initial parameters of the target model.

[0075] Next, parse the computational graph of the original model and construct the computational graph skeleton of the target model.

[0076] Then, traverse any original operator in the computational graph of the original model, and through the operator translation model, obtain all target operators of the target model, and also obtain any target operator that has been manually modified; After parsing the original model, the converted target model is obtained.

[0077] Finally, a validation dataset can be used to perform validation tests on the original model and the target model to determine the conversion effect.

[0078] In a feasible implementation, taking a video classification model as the original model as an example, the model is an h5 format model generated using the Tensorflow front-end framework. In actual applications, the video classification model needs to be deployed on the MLU220 computing card; however, the MLU220 computing card does not support the h5 format model, but only supports the Pytorch model. Therefore, at this time, the original model: the Tensorflow model can be converted into the target model: the Pytorch model, so that the video classification model can be successfully deployed on the MLU220 computing card.

[0079] The specific steps to convert the original model: Tensorflow model to the target model: Pytorch model include: 1) Obtain the Tensorflow model, standardize the Tensorflow model, replace all functions in the Tensorflow model with basic operators, and make the calculation graph of the Tensorflow model contain only basic operators; 2) Get the metadata of the TensorFlow model, which includes basic information such as the model version, input and output; 3) Build the model structure of the Pytorch model, which is a python file and import the necessary pytorch packages; 4) Parse the computational graph of the Tensorflow model and build the skeleton of the computational graph in the Pytorch model; 5) Traverse any operator in the computational graph of the TensorFlow model, each operator includes information such as the operator name, input and output, and operator attributes; 6) Use the tokenizer to tokenize any operator of the Tensorflow model and convert it into the first token sequence; 7) Input the first word segmentation sequence of the operator of the generated Tensorflow model into the translation model to obtain the second word segmentation sequence of the translated target operator; 8) Use the word segmenter to restore the second word segmentation sequence of the target operator to the operator description corresponding to the Pytorch model; 9) After traversing all operators, obtain the target of manual translation for operators that failed to be translated, and complete the construction of the computational graph of the Pytorch model; 10) Conversion of model weights between Tensorflow model and Pytorch model; 11) Use the validation data set to validate the Tensorflow model and the Pytorch model respectively, and obtain the original output of the Tensorflow model and the target output of the Pytorch model; 12) Generate the cosine distance between the original output and the target output and compare it with the preset validation threshold to determine the accuracy of the model before and after the conversion.

[0080] This method provides a new way to convert any two deep learning models into each other. The difficulty of traditional model conversion lies mainly in the translation of operators. On the one hand, engineers need to have an accurate understanding of the definition of each operator; on the other hand, due to the extremely large number of operators, engineers need to spend a lot of energy to establish mapping tables and design operator translation architectures.

[0081] This method introduces an operator translation model with an Encoder-Decoder architecture and builds a word segmenter to convert the description of any operator into a Token. After passing through the operator translation model, it can be converted into the operator description of a model of another front-end framework, thereby completing the automated translation process of the model operator. This can greatly reduce the workload of engineers in the process of developing model conversion, solve the problem of converting models under different frameworks to each other, and improve the efficiency of the model conversion process.

[0082] The application scenarios of the deep learning model conversion method provided by this implementation include but are not limited to: (1) Model deployment process: During model training or deployment, models of different deep learning front-end frameworks need to be converted to each other. However, with the current solutions, there are no ready-made conversion tools; or the degree of automation is low, requiring manual programming of operator mapping; or manual conversion of operators requires a high level of technology, otherwise the converted model cannot meet the functional requirements or accuracy requirements. In this case, this method can be used for conversion.

[0083] (2) Model standardization: When integrating a system, it is sometimes necessary to convert models from different sources into a standard format. In this case, this method can also be used to build the model.

[0084] (3) Optimize model performance. The performance of a model is not only related to the number of model parameters and the amount of computation, but also has a lot to do with the model's data layout, node parameters, and the representation of the computational graph. These are more obvious in models represented by different front-end frameworks.

[0085] (4) Secure computing: Converting from one model framework to another more secure model framework, such as a model that can be serialized into binary, is usually a necessary requirement in engineering development.

[0086] Reference Figure 5 , shows a functional module diagram of a deep learning model conversion device provided in an embodiment of the present application, the device comprising: An acquisition module 100 is used to acquire model data and metadata of an original model; A model structure building module 200 is used to build a model structure of a target model according to the model structure in the model data of the original model; An operator conversion module 300 is used to traverse the operator information of any original operator in the calculation graph of the original model according to the calculation graph of the original model, and generate a target operator corresponding to the original operator through an operator translation model according to the metadata of the original model and the operator information of the original operator. The calculation graph of the original model includes the original operator corresponding to any function of the original model, and the operator translation model is used to convert operators of different front-end frameworks; The target model generation module 400 is used to obtain the calculation graph of the target model according to all target operators obtained by conversion, and generate the converted target model corresponding to the original model.

[0087] Optionally, the operator conversion module includes: A word segmentation unit, configured to perform word segmentation on any original operator through a word segmenter based on operator information of the original operator to obtain a first word segmentation sequence, wherein the operator information includes the name, input and output, and operator attribute information of the operator; a conversion unit, configured to input the first word segmentation sequence, the metadata of the original model, and a natural language processing command into a pre-trained operator translation model, and obtain a second word segmentation sequence output by the operator translation model, wherein the natural language processing command is used to instruct the operator translation model to convert the operator of the original model into an operator corresponding to the target model; The restoration unit is used to restore the second word segmentation sequence through a word segmenter to obtain a target operator corresponding to the original operator.

[0088] Optionally, the device further comprises: A first construction module, used to construct a first corpus set, wherein the first corpus set includes operator information of any operator; A second construction module is used to construct a second corpus set, wherein the second corpus set includes model structure texts of models of different front-end frameworks; A first training module, used for performing preliminary training on a large language model based on the first corpus set; The second training module is used to optimize and train the large language model based on the second corpus set to obtain a trained operator translation model.

[0089] Optionally, the device further includes an operator configuration module, configured to: An operator configuration module, used to obtain a conversion operator corresponding to any original operator in response to an operator configuration operation; The conversion operator corresponding to the original operator is used as the target operator of the original operator.

[0090] Optionally, the device further comprises a standardization processing module, configured to: The original model is standardized, and any function of the original model is replaced by a corresponding operator.

[0091] Optionally, the device further includes an initial parameter configuration module, configured to: Obtaining initial parameters of the original model; The initial parameters of the target model are configured according to the initial parameters of the original model.

[0092] Optionally, the device further comprises a verification module, configured to: Inputting the validation data set into the original model to obtain the original output; Inputting the validation data set into the converted target model to obtain a target output; A cosine distance between the original output and the target output is calculated as a verification result of the target model.

[0093] Reference Figure 6 , shows a schematic diagram of a computer device provided in an embodiment of the present application, comprising: at least one processor 601, and a memory 602, wherein the memory 602 stores a computer program that can be run on the processor 601, wherein the processor 601 executes the deep learning model conversion method described in the embodiment when executing the computer program.

[0094] Reference Figure 7 , shows a schematic diagram of a non-volatile readable storage medium provided in an embodiment of the present application, wherein the non-volatile readable storage medium 700 stores a computer program 701, wherein the computer program 701, when executed by a processor, performs the deep learning model conversion method described in the embodiment.

[0095] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0096] It should be understood by those skilled in the art that the embodiments of the present application can be provided as methods, devices or computer program products. Therefore, the embodiments of the present application can be in the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the embodiments of the present application can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0097] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0098] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0099] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0100] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0101] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0102] Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A deep learning model conversion method, characterized in that: The method comprises: Obtain model data and metadata of the original model; Constructing a model structure of a target model according to the model structure in the model data of the original model; According to the calculation graph of the original model, traverse the operator information of any original operator in the calculation graph of the original model, and generate a target operator corresponding to the original operator through an operator translation model according to the metadata of the original model and the operator information of the original operator, wherein the calculation graph of the original model includes the original operator corresponding to any function of the original model, and the operator translation model is used to convert operators of different front-end frameworks; According to all the target operators obtained by the conversion, a calculation graph of the target model is obtained, and a converted target model corresponding to the original model is generated.

2. The method according to claim 1, characterized in that: Traversing the operator information of any original operator in the calculation graph of the original model, generating a target operator corresponding to the original operator through an operator translation model according to the metadata of the original model and the operator information of the original operator, including: For any original operator, perform word segmentation through a word segmenter based on the operator information of the original operator to obtain a first word segmentation sequence, wherein the operator information includes the name, input and output, and operator attribute information of the operator; Inputting the first word segmentation sequence, the metadata of the original model, and a natural language processing command into a pre-trained operator translation model to obtain a second word segmentation sequence output by the operator translation model, wherein the natural language processing command is used to instruct the operator translation model to convert the operator of the original model into an operator corresponding to the target model; The second word segmentation sequence is restored through a word segmenter to obtain a target operator corresponding to the original operator.

3. The method according to claim 2, characterized in that The method further comprises: Constructing a first corpus set, wherein the first corpus set includes operator information of any operator; Constructing a second corpus set, wherein the second corpus set includes model structure texts of models of different front-end frameworks; Performing preliminary training on a large language model based on the first corpus set; The large language model is optimized and trained based on the second corpus set to obtain a trained operator translation model.

4. The method according to claim 1, characterized in that Before obtaining the computation graph of the target model according to all target operators obtained by conversion, the method further includes: In response to the operator configuration operation, obtain the conversion operator corresponding to any original operator; The conversion operator corresponding to the original operator is used as the target operator of the original operator.

5. The method according to claim 1, characterized in that After obtaining the original model, the method further includes: The original model is standardized, and any function of the original model is replaced by a corresponding operator.

6. The method according to claim 1, characterized in that After constructing the model structure of the target model according to the model structure of the original model, the method further includes: Obtaining initial parameters of the original model; The initial parameters of the target model are configured according to the initial parameters of the original model.

7. The method according to claim 1, characterized in that After generating the converted target model corresponding to the original model, the method further includes: Inputting the validation data set into the original model to obtain the original output; Inputting the validation data set into the converted target model to obtain a target output; A cosine distance between the original output and the target output is calculated as a verification result of the target model.

8. A deep learning model conversion device, characterized in that: The device comprises: An acquisition module, used to acquire model data and metadata of the original model; A model structure building module, used to build a model structure of a target model according to the model structure in the model data of the original model; An operator conversion module is used to traverse the operator information of any original operator in the calculation graph of the original model according to the calculation graph of the original model, and generate a target operator corresponding to the original operator through an operator translation model according to the metadata of the original model and the operator information of the original operator. The calculation graph of the original model includes the original operator corresponding to any function of the original model, and the operator translation model is used to convert operators of different front-end frameworks; The target model generation module is used to obtain the calculation graph of the target model according to all target operators obtained by conversion, and generate the converted target model corresponding to the original model.

9. A computer device, characterized in that: include: At least one processor, and a memory, wherein the memory stores a computer program that can be run on the processor, wherein the processor executes the deep learning model conversion method described in any one of claims 1-7 when executing the computer program.

10. A non-volatile readable storage medium, characterized in that: The non-volatile readable storage medium stores a computer program, wherein the computer program, when executed by a processor, performs the deep learning model conversion method described in any one of claims 1-7.

Images