Model conversion method, device and electronic equipment thereof

Abstract

An embodiment of the present invention provides a model conversion method, comprising: obtaining a computational graph of a model to be converted, wherein the computational graph comprises an initializer, multiple nodes, and connecting edges between nodes, wherein the initializer comprises model parameters of the model to be converted, each node represents an operation in the model to be converted, and the connecting edges represent data flows between the operation operations; based on the computational graph, converting an instruction set, a multidimensional array access mechanism, a control structure, and a variable storage mechanism for a target hardware space; based on the instruction set, the multidimensional array access mechanism, the control structure, and the variable storage mechanism, generating a target model file of the model to be converted in the target hardware space, wherein the target model file is used to deploy a target model of the model to be converted in the target hardware space. The above method can realize the conversion of different models on different hardware platforms, thereby improving the portability of the model and the interoperability between frameworks.

Description

Technical Field

[0001] The present invention relates to the field of deep learning models, and in particular to a model conversion method, device, electronic device and storage medium thereof. Background Art

[0002] With the development of deep learning technology, neural network models have been widely used in multiple fields. In order to enable neural network models to be used on hardware devices corresponding to different fields, it is often necessary to migrate the trained models from one hardware environment to another.

[0003] Since existing model conversion methods cannot adapt to the compatibility differences between different hardware platforms, complex adjustments and optimizations may be required when transferring models to adapt to the characteristics of the target hardware. Therefore, existing model conversion methods have the problem of conversion between different models, which will lead to a decline in model performance and accuracy, increase the difficulty of deployment and resource consumption. Summary of the Invention

[0004] An embodiment of the present invention provides a model conversion method to solve the problem that the existing model conversion method has conversion between different models, which leads to a decrease in model performance and accuracy, and increases the difficulty of deployment and resource consumption.

[0005] In a first aspect, an embodiment of the present invention provides a model conversion method, the method comprising the following steps:

[0006] Obtain a computation graph of the model to be converted, wherein the computation graph includes an initializer, multiple nodes, and connecting edges between the nodes, the initializer includes model parameters of the model to be converted, each node represents an operation in the model to be converted, and the connecting edges represent data flows between the operation operations;

[0007] Based on the computation graph, convert the instruction set, multi-dimensional array access mechanism, control structure and variable storage mechanism for the target hardware space;

[0008] Based on the instruction set, the multidimensional array access mechanism, the control structure and the variable storage mechanism, a target model file of the model to be converted in the target hardware space is generated, and the target model file is used to deploy the target model of the model to be converted in the target hardware space.

[0009] Optionally, the step of obtaining a computational graph of the model to be converted includes:

[0010] Obtaining the computing operations of the model to be converted in the original space and the data flow between the computing operations;

[0011] Constructing a directed acyclic graph with the computing operations as nodes and the data flows between the computing operations as connecting edges;

[0012] Obtaining model parameters of the model to be converted, and storing the model parameters in an initializer;

[0013] Based on the directed acyclic graph and the initializer, a computational graph of the model to be converted is obtained.

[0014] Optionally, in the computation graph, each node corresponds to an operator, the operator corresponds to the type of the operation, and each connection edge corresponds to a tensor transfer of a data flow. The step of converting the instruction set, multi-dimensional array access mechanism, control structure, and variable storage mechanism for the target hardware space based on the computation graph includes:

[0015] Based on a preset mapping relationship between operators and instructions, converting an instruction set for the target hardware space on the basis of each of the nodes;

[0016] Based on the mapping relationship between the tensor transfer of the data flow and the multidimensional array, converting the multidimensional array access mechanism for the target hardware space on the basis of each of the connection edges;

[0017] Determining a control structure for the target hardware space based on a loop or branch structure of each of the nodes;

[0018] Based on the initializer, a variable storage mechanism of the target hardware space is determined.

[0019] Optionally, the step of converting a multidimensional array access mechanism for the target hardware space based on each of the connection edges based on the mapping relationship between the tensor transfer of the data flow and the multidimensional array includes:

[0020] For each connection edge, determining the tensor size and tensor order of the tensor transfer process;

[0021] Determining a corresponding memory space size based on the tensor size, and determining an access order of the memory space based on the tensor order;

[0022] Based on the size of the memory space and the access order of the memory space, a multidimensional array access mechanism of the target hardware space is generated, where the multidimensional array access mechanism of the target hardware space includes pointer operation and array access.

[0023] Optionally, the step of generating a target model file of the model to be converted in the target hardware space based on the instruction set, the multidimensional array access mechanism, the control structure, and the variable storage mechanism includes:

[0024] Generate a code file corresponding to the target hardware space based on the instruction set, the multidimensional array access mechanism, the control structure, and the variable storage mechanism;

[0025] Based on the code file corresponding to the target hardware space, a target model file of the model to be converted in the target hardware space is determined.

[0026] Optionally, the step of determining a target model file of the model to be converted in the target hardware space based on the code file corresponding to the target hardware space includes:

[0027] The code file is optimized based on the optimization tool chain corresponding to the target hardware space to obtain a target model file of the model to be converted in the target hardware space.

[0028] In a second aspect, an embodiment of the present invention further provides a model conversion device, the model conversion device comprising:

[0029] an acquisition module, configured to acquire a computation graph of the model to be converted, wherein the computation graph includes an initializer, a plurality of nodes, and connecting edges between the nodes, the initializer includes model parameters of the model to be converted, each node represents an operation in the model to be converted, and the connecting edges represent data flows between the operation operations;

[0030] A conversion module, configured to convert, based on the computation graph, an instruction set, a multi-dimensional array access mechanism, a control structure, and a variable storage mechanism for a target hardware space;

[0031] A generation module is used to generate a target model file of the model to be converted in the target hardware space based on the instruction set, the multidimensional array access mechanism, the control structure and the variable storage mechanism, and the target model file is used to deploy the target model of the model to be converted in the target hardware space.

[0032] In a third aspect, an embodiment of the present invention provides an electronic device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the model conversion method provided in an embodiment of the present invention when executing the computer program.

[0033] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the model conversion method provided in the embodiment of the invention are implemented.

[0034] In an embodiment of the present invention, a computational graph of a model to be converted is obtained, wherein the computational graph includes an initializer, multiple nodes, and connecting edges between nodes, wherein the initializer includes model parameters of the model to be converted, each node represents an operation in the model to be converted, and the connecting edges represent data flows between the operations; based on the computational graph, an instruction set, a multidimensional array access mechanism, a control structure, and a variable storage mechanism for a target hardware space are converted; based on the instruction set, the multidimensional array access mechanism, the control structure, and the variable storage mechanism, a target model file of the model to be converted in the target hardware space is generated, wherein the target model file is used to deploy a target model of the model to be converted in the target hardware space. The above method can realize the conversion of different models on different hardware platforms, thereby improving the portability of the model and the interoperability between frameworks. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0036] Figure 1 is a flow chart of a model conversion method provided by an embodiment of the present invention;

[0037] Figure 2 yes Figure 1 Specific flow chart of step S102;

[0038] Figure 3 yes Figure 1 Another specific flow chart of step S102;

[0039] Figure 4 yes Figure 3 Specific flow chart of step S1026;

[0040] Figure 5 yes Figure 3 Specific flow chart of step S1028;

[0041] Figure 6 yes Figure 1 Specific flow chart of step S103;

[0042] Figure 7 yes Figure 6 Specific flow chart of step S105;

[0043] Figure 8 is a structural diagram of a model conversion device provided in an embodiment of the present invention;

[0044] Figure 9 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0046] like Figure 1 As shown, Figure 1 1 is a flow chart of a model conversion method provided by an embodiment of the present invention, the model conversion method comprising the steps of:

[0047] S101. Obtain a computational graph of the model to be converted.

[0048] In an embodiment of the present invention, the above-mentioned model conversion method can be applied to a model conversion device, which has functions such as model data processing, model data transmission and reception, and model data storage, and can be constructed based on a server or a server cluster. The above-mentioned server or server cluster can be an electronic device with model data processing capabilities.

[0049] The computation graph may include, but is not limited to, initializers, multiple nodes, and edges connecting nodes. Specifically, the computation graph can be used to demonstrate the functions of various components in different models to be converted and their roles in the models. This embodiment uses the ONNX model as an example. The computation graph components in the ONNX model may include, but are not limited to, the initializer, multiple nodes, and edges connecting nodes.

[0050] The initializer may include but is not limited to model parameters of the model to be converted, such as weight parameters and bias parameters, which are learned during model training and used to perform predictions during model inference.

[0051] Each of the above nodes can be used to represent an operation in the above model to be converted. For example, each node represents an operation, such as convolution (Conv), linear transformation (such as matrix multiplication), activation function (such as ReLU), etc. These operations are usually performed on high-dimensional data, especially when converting between different models.

[0052] The above-mentioned connection edges can be used to represent the data flow between operations. Specifically, the above-mentioned connection edges can represent data flow, that is, how information flows from one node to another. In the ONNX computation graph, the connection edges usually transmit tensors, which are multidimensional arrays that can contain, for example, image data, weight matrices, etc.

[0053] The above-mentioned model to be converted may include but is not limited to any model capable of machine learning, that is, an original model that can be converted by the above-mentioned model conversion method and generate a corresponding target model file. This embodiment can be illustrated by, for example, an ONNX model. Different models to be converted have different characteristics, such as the composition, operation and control flow of the computational graph. In this embodiment, different target model conversion strategies can be adopted according to the computational graph content of different models to be converted.

[0054] S102. Based on the computation graph, convert the instruction set, multi-dimensional array access mechanism, control structure, and variable storage mechanism for the target hardware space.

[0055] In this embodiment, OPEN64 IR is used as the target hardware space for description. The target hardware space may be the destination of the conversion of the model to be converted or the target address of the model configuration parameters.

[0056] The instruction set described above can include mappings between operators and instructions. Specifically, each operation (node) in the computational graph can be mapped to an instruction supported by the target hardware. For example, the convolution operation in the ONNX model is mapped to a series of underlying scalar and vector operations in the OPEN64 IR.

[0057] The above-mentioned multi-dimensional array access mechanism can be the relationship between tensor and array mapping. Specifically, the data flow (tensor transfer) can be converted into a multi-dimensional array access mechanism suitable for the target hardware, so as to determine the storage layout of the array (such as row priority or column priority), memory address allocation and optimized array access mode (such as cache-friendly access order). Through the multi-dimensional array access mechanism, memory optimization can be performed for large data structures such as tensors to ensure maximum data access efficiency, reduce latency and improve bandwidth utilization.

[0058] The above control structures can be used for control flow analysis, for example, converting the control flow (such as conditional branches and loop structures) in the model into the control structure of the target hardware space.

[0059] The above-mentioned variable storage mechanism can refer to a mechanism for variable conversion and memory allocation. Specifically, the parameters stored in the initializer of the computational graph, including global variables of the model to be converted, such as weight parameters and local variables, can be converted and allocated to the memory space of the target hardware.

[0060] S103 , generating a target model file of the model to be converted in the target hardware space based on the instruction set, multi-dimensional array access mechanism, control structure and variable storage mechanism.

[0061] The above target model file can be used to deploy the target model of the model to be converted in the target hardware space.

[0062] In a possible embodiment, the computational graph of the model to be converted in ONNX format is obtained through software, and the functions of the components in the computational graph and their roles in the model are analyzed. The components in the computational graph are converted to the target hardware space to obtain the instruction set, multi-dimensional array access mechanism, control structure and variable storage mechanism corresponding to the target hardware space. Finally, based on the converted instruction set, multi-dimensional array access mechanism, control structure and variable storage mechanism, a target model file is generated that is directly deployed on the target hardware space and executed.

[0063] In an embodiment of the present invention, a computational graph of a model to be converted is obtained, the computational graph including an initializer, multiple nodes, and connecting edges between nodes, the initializer including model parameters of the model to be converted, each node representing an operation in the model to be converted, and the connecting edges representing the data flow between the operations; based on the computational graph, an instruction set, a multi-dimensional array access mechanism, a control structure, and a variable storage mechanism for a target hardware space are converted; based on the instruction set, the multi-dimensional array access mechanism, the control structure, and the variable storage mechanism, a target model file of the model to be converted in the target hardware space is generated, the target model file is used to deploy the target model of the model to be converted in the target hardware space, and the above method can realize the conversion of different models on different hardware platforms, thereby improving the portability of the model and the interoperability between frameworks.

[0064] Optional, such as Figure 2 As shown, in step S102, the step of obtaining the computational graph of the model to be converted further includes steps S1021-S1024, wherein:

[0065] S1021. Obtain the computing operations of the model to be converted in the original space and the data flow between the computing operations.

[0066] In an embodiment of the present invention, the file of the model to be converted can be read through a parsing tool or library, such as ONNX Runtime or other libraries adapted to the ONNX model, to obtain the structure, operation and properties corresponding to the model to be converted.

[0067] The above operations can be understood as operations on how data flows from a certain node to another node, such as convolution operations.

[0068] The above-mentioned data flow can be a connection edge between nodes, which is used to determine how data flows from one node to another. In this embodiment, it can be explained based on the ONNX model. The connection is usually represented as a flow path of a tensor, and the process of how data interacts between operations can also be shown.

[0069] S1022. Construct a directed acyclic graph using operation operations as nodes and data flows between operation operations as connecting edges;

[0070] S1023. Obtain model parameters of the model to be converted, and store the model parameters in an initializer;

[0071] S1024. Based on the directed acyclic graph and the initializer, a computational graph of the model to be converted is obtained.

[0072] The above directed acyclic graph can be used to show the various operations and the data flow between them.

[0073] In a possible embodiment, the data flow between the calculation operations is parsed to determine the flow direction and specific calculation operations between the data flows. According to the flow direction and the specific calculation operations, a directed acyclic graph is constructed. Finally, the model parameters of the model to be converted stored in the initializer are traversed and optimized according to the directed acyclic graph to obtain the calculation graph of the model to be converted. Specifically, the traversal and optimization can be performed through forward propagation, backward propagation or parallel optimization to improve the precision and construction speed of the calculation graph.

[0074] Optional, such as Figure 3 As shown, in step S102, based on the computation graph, the step of converting the instruction set, multi-dimensional array access mechanism, control structure and variable storage mechanism for the target hardware space also includes steps S1025-S1028, wherein:

[0075] S1025, based on the preset mapping relationship between operators and instructions, converting an instruction set for the target hardware space on the basis of each node;

[0076] In an embodiment of the present invention, the above-mentioned preset mapping relationship may be an intermediate relationship of mapping operators in the model to be converted to basic instructions corresponding to the target hardware space.

[0077] Each of the above nodes can correspond to an operator, and the operator corresponds to the type of operation, and each connection edge corresponds to a tensor transfer of a data flow.

[0078] In a possible embodiment, by consulting a mapping table, the corresponding target hardware instructions between each node in the model to be converted and the target hardware space are determined. For example, operations such as convolution operations in the OXXN model are mapped to instructions of addition and subtraction combinations corresponding to OPEN64IR, and the converted instructions are optimized and integrated to obtain an instruction set for the target hardware space.

[0079] S1026. Based on the mapping relationship between the tensor transfer of the data flow and the multidimensional array, a multidimensional array access mechanism for the target hardware space is converted on the basis of each connection edge.

[0080] The mapping relationship between the tensor transfer of the above data flow and the multidimensional array can be a preset conversion relationship that defines how to effectively convert high-dimensional tensor data into a multidimensional array and enable the corresponding memory layout, access mode, etc. to adapt to the target hardware space.

[0081] In a possible embodiment, through the mapping relationship between the tensor transfer of the data flow and the multidimensional array, the tensor data storage mode is converted into the storage mode between the multidimensional arrays according to the calculation operation content displayed by each connection edge.

[0082] S1027: Based on the loop or branch structure of each node, determine a control structure for the target hardware space.

[0083] In one possible embodiment, by analyzing the loop or branch structure of each node, the optimal execution steps when executing the calculation operation of the current node are determined, and based on the optimal execution steps, a control structure for the target hardware space is generated. It can be understood that the control structure can also take into account the storage mechanism optimized for the target hardware space.

[0084] S1028. Based on the initializer, determine the variable storage mechanism of the target hardware space.

[0085] In one possible embodiment, an initializer stores parameters such as the control structure and multidimensional array access mechanism described above to obtain configuration parameters for the target hardware space. Based on these configuration parameters and the storage optimization content for the target hardware space, a variable storage mechanism corresponding to the target hardware space is determined. It is understood that this variable storage mechanism ensures data alignment and cache optimization during the model conversion process, ensuring efficient memory access.

[0086] Optional, such as Figure 4As shown, in step S1026, based on the mapping relationship between the tensor transfer of the data flow and the multidimensional array, on the basis of each connection edge, the step of converting the multidimensional array access mechanism for the target hardware space also includes steps S10261-S10263, wherein:

[0087] S10261. For each connection edge, determine the tensor size and tensor order of the tensor transfer process.

[0088] In an embodiment of the present invention, the above-mentioned tensor size can be calculated by calculating the size of each data element during the transmission process, and then calculating the size of the entire tensor based on the dimension and dimension size of each corresponding tensor, such as width, height, number of channels, etc.

[0089] The tensor order described above can be the order in which operations are performed during the transfer process. The target hardware space requirement can be selected based on either access speed or maximum storage capacity. Generally speaking, the tensor order can be determined directly by analyzing the flow of operations in the computation graph.

[0090] S10262. Based on the size of the tensor, determine the corresponding memory space size, and based on the order of the tensors, determine the access order of the memory space.

[0091] In one possible embodiment, the corresponding memory space size is determined according to the tensor size. For example, the convolution layer to be converted has 32 3x3 filters, the input tensor is 28x28x3 (height x width x channel), the output tensor size is 26x26x32, and a single floating-point number occupies 4 bytes. The corresponding required memory space size can be calculated. The memory space size can be adjusted according to the specific type and memory requirements of the target hardware space, and the corresponding memory space size is determined for the target hardware space with a non-saturated target.

[0092] Specifically, the order of tensors can be determined according to row priority or column priority. Different tensor orders can be selected for different target hardware space requirements.

[0093] The access order can also be adjusted based on the access requirements of the target hardware space, such as reducing memory access latency and improving overall computing speed. For example, the order of accessing tensors can be adjusted, such as accessing tensors with smaller sizes first, or converting less complex tensor contents.

[0094] S10263. Generate a multi-dimensional array access mechanism for the target hardware space based on the memory space size and the access order of the memory space.

[0095] The multi-dimensional array access mechanism of the above-mentioned target hardware space may include but is not limited to pointer operations and array access, that is, the basic elements of processing data and memory management, and the purpose of corresponding data and memory management of the target hardware space can be achieved through pointer operations and array access.

[0096] In one possible embodiment, based on the tensor data of the ONNX model, its memory space size and access order are analyzed, and it is converted into a multi-dimensional array access mechanism adapted to OPEN64 IR. In view of the hardware characteristics of OPEN64 IR, a row-first memory layout and cache optimization strategy are adopted to ensure that tensors are accessed in an efficient order during the calculation process.

[0097] The above method generates a multi-dimensional array access mechanism with OPEN64 IR as the target hardware space, which reduces memory read latency, improves the execution efficiency of matrix operations, and significantly improves the overall performance of the ONNX model in the OPEN64 IR environment.

[0098] Optional, such as Figure 5 As shown, in step S1028, the step of determining the variable storage mechanism of the target hardware space based on the initializer further includes steps S10281-S10282, wherein:

[0099] S10281. Convert the model parameters in the initializer into global or local variables in the target hardware space;

[0100] S10282. Allocate memory space and a read path for global or local variables to obtain a variable storage mechanism of the target hardware space.

[0101] In a possible embodiment, the ONNX model is used as the model to be converted, and OPEN64 IR is used as the target hardware space for description. The corresponding weight parameters, bias parameters, etc. are determined through the ONNX model parameters in the initializer. In order to adapt these parameters to the OPEN64 IR hardware platform, the global variables or local variables are determined by analyzing the computing requirements and parameter usage frequency corresponding to the model parameters in the ONNX model initializer. Since global variables are frequently used and need to be accessed multiple times, more memory space and the shortest reading path can be allocated to global variables, and thereby a variable storage mechanism of the target hardware space is generated, that is, a variable storage mechanism optimized for access to global variables.

[0102] Optional, such as Figure 6 As shown, in step S103, based on the instruction set, multi-dimensional array access mechanism, control structure and variable storage mechanism, the step of generating a target model file of the model to be converted in the target hardware space also includes steps S104-S105, wherein:

[0103] S104 , generating a code file corresponding to the target hardware space based on the instruction set, multi-dimensional array access mechanism, control structure, and variable storage mechanism.

[0104] In an embodiment of the present invention, a high-level model description is converted into code that can be directly executed, that is, the instruction set, multi-dimensional array access mechanism, control structure and variable storage mechanism of the converted model are converted into code description to generate a code file corresponding to the target hardware space.

[0105] Specifically, high-level operations in the instruction set, such as convolution, activation function, pooling, etc., are mapped to instructions that can be understood and executed by the target hardware space according to preset mapping relationships. For example, operations such as convolution are decomposed into a series of multiplication and addition and subtraction instructions.

[0106] In a possible embodiment, by parsing the instruction set, multi-dimensional array access mechanism, control structure and variable storage mechanism, the corresponding functions can be described through code, and the corresponding code file is generated according to the description content and the code type that can be executed by the target hardware space.

[0107] S105 : Determine the target model file of the model to be converted in the target hardware space based on the code file corresponding to the target hardware space.

[0108] In a possible embodiment, the code file corresponding to the target hardware space is compiled, and the compiled code file is executed. The code file is optimized and compiled again according to the storage space when the target hardware space is executed, thereby obtaining the target model file of the model to be converted in the target hardware space.

[0109] Through the above method, the execution efficiency of the conversion can be improved, and the code file compiled for the first time can be optimized in real time according to the storage space of the target hardware space when the code file is executed, so that after the optimized code file is compiled and executed, the memory access in the target hardware space has reduced delays and improved the overall computing throughput.

[0110] Optional, such as Figure 7 As shown, in step S105, based on the code file corresponding to the target hardware space, the step of determining the target model file of the model to be converted in the target hardware space also includes step S1051:

[0111] S1051. Optimize the code file based on the optimization tool chain corresponding to the target hardware space to obtain a target model file of the model to be converted in the target hardware space.

[0112] In an embodiment of the present invention, the above-mentioned optimization tool chain may be a general term for a collection of software tools and technologies, such as a collection of technologies corresponding to a certain tool.

[0113] In a possible embodiment, the above-mentioned optimization processing can be to reduce the overhead of function calls by optimizing the inline expansion function of the tool chain compiler, and at the same time apply branch prediction and conditional execution technology to reduce branch penalties. In terms of access frequency, the data locality optimization of OPEN64 IR can be used to arrange frequently accessed data in a fast access cache to reduce the need for access to the main memory, or the memory access mode of the tensor operation can be fine-tuned according to the size of the current memory storage space.

[0114] like Figure 8 As shown, the embodiment of the present invention further provides a model conversion device 800, which includes:

[0115] An acquisition module 801 is configured to acquire a computation graph of a model to be converted, wherein the computation graph includes an initializer, multiple nodes, and edges connecting the nodes. The initializer includes model parameters of the model to be converted, each node represents an operation in the model to be converted, and the edges represent data flows between the operations.

[0116] A conversion module 802 is configured to convert, based on the computation graph, an instruction set, a multi-dimensional array access mechanism, a control structure, and a variable storage mechanism for a target hardware space;

[0117] The first generation module 803 is used to generate a target model file of the model to be converted in the target hardware space based on the instruction set, the multi-dimensional array access mechanism, the control structure and the variable storage mechanism, and the target model file is used to deploy the target model of the model to be converted in the target hardware space.

[0118] Optionally, the acquisition module 801 includes:

[0119] A first acquisition submodule is used to acquire the computing operations of the model to be converted in the original space and the data flow between the computing operations;

[0120] A construction submodule, configured to construct a directed acyclic graph using the computing operations as nodes and the data flows between the computing operations as connecting edges;

[0121] A second acquisition submodule is used to acquire model parameters of the model to be converted and store the model parameters in an initializer;

[0122] The third acquisition submodule is used to obtain a computational graph of the model to be converted based on the directed acyclic graph and the initializer.

[0123] Optionally, the generating module 803 includes:

[0124] A first conversion submodule is configured to convert an instruction set for the target hardware space based on each of the nodes based on a preset mapping relationship between operators and instructions;

[0125] A second conversion submodule is configured to convert, based on the mapping relationship between the tensor transfer of the data flow and the multidimensional array, a multidimensional array access mechanism for the target hardware space on the basis of each of the connection edges;

[0126] A first determining submodule, configured to determine a control structure for the target hardware space based on a loop or branch structure of each of the nodes;

[0127] The second determining submodule is configured to determine a variable storage mechanism of the target hardware space based on the initializer.

[0128] Optionally, the second conversion submodule includes:

[0129] A first determining unit is configured to determine, for each connection edge, a tensor size and a tensor order in the tensor transfer process;

[0130] a second determining unit, configured to determine a corresponding memory space size based on the tensor size, and determine an access order of the memory space based on the tensor order;

[0131] A generating unit is used to generate a multi-dimensional array access mechanism of the target hardware space based on the size of the memory space and the access order of the memory space, wherein the multi-dimensional array access mechanism of the target hardware space includes pointer operation and array access.

[0132] Optionally, the second determining submodule includes:

[0133] A first conversion unit, configured to convert the model parameters in the initializer into global or local variables in the target hardware space;

[0134] The first acquisition unit is used to allocate memory space and a reading path for the global or local variable to obtain a variable storage mechanism of the target hardware space.

[0135] Optionally, the above device further includes:

[0136] A second generating module is configured to generate a code file corresponding to the target hardware space based on the instruction set, the multi-dimensional array access mechanism, the control structure, and the variable storage mechanism;

[0137] The first determining module is configured to determine a target model file of the model to be converted in the target hardware space based on a code file corresponding to the target hardware space.

[0138] Optionally, the first determining module includes:

[0139] The optimization processing module is used to optimize the code file based on the optimization tool chain corresponding to the target hardware space to obtain the target model file of the model to be converted in the target hardware space.

[0140] like Figure 9 As shown, an embodiment of the present invention further provides an electronic device 900, including a processor, and the processor can execute any one of the above model conversion methods.

[0141] Specifically, the system includes a processor 901, a memory 902, and a computer program for executing the model conversion method, which is stored in the memory 902 and can be run on the processor 901, wherein:

[0142] The processor 901 runs the computer program of the model conversion method stored in the memory 902 and performs the following steps:

[0143] Obtain a computation graph of the model to be converted, wherein the computation graph includes an initializer, multiple nodes, and connecting edges between the nodes, the initializer includes model parameters of the model to be converted, each node represents an operation in the model to be converted, and the connecting edges represent data flows between the operation operations;

[0144] Based on the computation graph, convert the instruction set, multi-dimensional array access mechanism, control structure and variable storage mechanism for the target hardware space;

[0145] Based on the instruction set, the multidimensional array access mechanism, the control structure and the variable storage mechanism, a target model file of the model to be converted in the target hardware space is generated, and the target model file is used to deploy the target model of the model to be converted in the target hardware space.

[0146] Optionally, the processor 901 executes the step of obtaining the computation graph of the model to be converted, including:

[0147] Obtaining the computing operations of the model to be converted in the original space and the data flow between the computing operations;

[0148] Constructing a directed acyclic graph with the computing operations as nodes and the data flows between the computing operations as connecting edges;

[0149] Obtaining model parameters of the model to be converted, and storing the model parameters in an initializer;

[0150] Based on the directed acyclic graph and the initializer, a computational graph of the model to be converted is obtained.

[0151] Optionally, the processor 901 executes the computation graph, where each node corresponds to an operator, the operator corresponds to the type of the operation, and each connection edge corresponds to a tensor transfer of a data flow. The step of converting an instruction set, a multidimensional array access mechanism, a control structure, and a variable storage mechanism for a target hardware space based on the computation graph includes:

[0152] Based on a preset mapping relationship between operators and instructions, converting an instruction set for the target hardware space on the basis of each of the nodes;

[0153] Based on the mapping relationship between the tensor transfer of the data flow and the multidimensional array, converting the multidimensional array access mechanism for the target hardware space on the basis of each of the connection edges;

[0154] Determining a control structure for the target hardware space based on a loop or branch structure of each of the nodes;

[0155] Based on the initializer, a variable storage mechanism of the target hardware space is determined.

[0156] Optionally, the processor 901 executes the mapping relationship between the tensor transfer based on the data stream and the multidimensional array, and converts the multidimensional array access mechanism for the target hardware space based on each of the connection edges, including:

[0157] For each connection edge, determining the tensor size and tensor order of the tensor transfer process;

[0158] Determining a corresponding memory space size based on the tensor size, and determining an access order of the memory space based on the tensor order;

[0159] Based on the size of the memory space and the access order of the memory space, a multidimensional array access mechanism of the target hardware space is generated, where the multidimensional array access mechanism of the target hardware space includes pointer operation and array access.

[0160] Optionally, the processor 901 executes the step of determining the variable storage mechanism of the target hardware space based on the initializer, including:

[0161] Convert the model parameters in the initializer into global or local variables in the target hardware space;

[0162] Allocate memory space and a read path for the global or local variables to obtain a variable storage mechanism of the target hardware space.

[0163] Optionally, the processor 901 further executes the step of generating a target model file of the model to be converted in the target hardware space based on the instruction set, the multidimensional array access mechanism, the control structure, and the variable storage mechanism, including:

[0164] Generate a code file corresponding to the target hardware space based on the instruction set, the multidimensional array access mechanism, the control structure, and the variable storage mechanism;

[0165] Based on the code file corresponding to the target hardware space, a target model file of the model to be converted in the target hardware space is determined.

[0166] Optionally, the processor 901 further executes the step of determining the target model file of the model to be converted in the target hardware space based on the code file corresponding to the target hardware space, including:

[0167] The code file is optimized based on the optimization tool chain corresponding to the target hardware space to obtain a target model file of the model to be converted in the target hardware space.

[0168] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the computer program implements the various processes of the model conversion method or the application-side model conversion method provided in the embodiment of the present invention, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0169] Those skilled in the art will appreciate that the implementation of all or part of the processes in the above-described method embodiments can be accomplished by a computer program that instructs related hardware and can be stored in a computer-readable storage medium. When executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0170] The above disclosure is merely a preferred embodiment of the present invention and certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A model conversion method, characterized in that: include: Obtain a computation graph of the model to be converted, the computation graph comprising an initializer, a plurality of nodes, and connecting edges between the nodes, the initializer comprising model parameters of the model to be converted, each node representing an operation in the model to be converted, and the connecting edges representing data flows between the operation operations; Based on the computation graph, convert the instruction set, multi-dimensional array access mechanism, control structure and variable storage mechanism for the target hardware space; Generate a target model file of the model to be converted in the target hardware space based on the instruction set, the multidimensional array access mechanism, the control structure, and the variable storage mechanism, wherein the target model file is used to deploy the target model of the model to be converted in the target hardware space; The step of converting the instruction set, multi-dimensional array access mechanism, control structure, and variable storage mechanism for the target hardware space based on the computation graph includes: Based on a preset mapping relationship between operators and instructions, converting an instruction set for the target hardware space on the basis of each of the nodes; Based on the mapping relationship between the tensor transfer of the data flow and the multidimensional array, converting the multidimensional array access mechanism for the target hardware space on the basis of each of the connection edges; Determining a control structure for the target hardware space based on a loop or branch structure of each of the nodes; Determining a variable storage mechanism of the target hardware space based on the initializer; The step of converting a multidimensional array access mechanism for the target hardware space based on each of the connection edges based on the mapping relationship between the tensor transfer of the data flow and the multidimensional array includes: For each connection edge, determining the tensor size and tensor order of the tensor transfer process; Determining a corresponding memory space size based on the tensor size, and determining an access order of the memory space based on the tensor order; Based on the size of the memory space and the access order of the memory space, generating a multidimensional array access mechanism for the target hardware space, wherein the multidimensional array access mechanism for the target hardware space includes pointer arithmetic and array access; The step of determining the variable storage mechanism of the target hardware space based on the initializer includes: Convert the model parameters in the initializer into global or local variables in the target hardware space; Allocate memory space and a read path for the global or local variables to obtain a variable storage mechanism of the target hardware space.

2. The model conversion method according to claim 1, wherein: The step of obtaining the computational graph of the model to be converted includes: Obtaining the computing operations of the model to be converted in the original space and the data flow between the computing operations; Constructing a directed acyclic graph with the computing operations as nodes and the data flows between the computing operations as connecting edges; Obtaining model parameters of the model to be converted, and storing the model parameters in an initializer; Based on the directed acyclic graph and the initializer, a computational graph of the model to be converted is obtained.

3. The model conversion method according to any one of claims 1 to 2, characterized in that: The step of generating a target model file of the model to be converted in the target hardware space based on the instruction set, the multidimensional array access mechanism, the control structure, and the variable storage mechanism includes: Generate a code file corresponding to the target hardware space based on the instruction set, the multidimensional array access mechanism, the control structure, and the variable storage mechanism; Based on the code file corresponding to the target hardware space, a target model file of the model to be converted in the target hardware space is determined.

4. The model conversion method according to claim 3, wherein: The step of determining the target model file of the model to be converted in the target hardware space based on the code file corresponding to the target hardware space includes: The code file is optimized based on the optimization tool chain corresponding to the target hardware space to obtain a target model file of the model to be converted in the target hardware space.

5. A model conversion device, characterized in that: The model conversion device comprises: an acquisition module, configured to acquire a computation graph of the model to be converted, wherein the computation graph includes an initializer, a plurality of nodes, and connecting edges between the nodes, the initializer includes model parameters of the model to be converted, each node represents an operation in the model to be converted, and the connecting edges represent data flows between the operation operations; A conversion module, configured to convert, based on the computation graph, an instruction set, a multi-dimensional array access mechanism, a control structure, and a variable storage mechanism for a target hardware space; A generation module, configured to generate a target model file of the model to be converted in the target hardware space based on the instruction set, the multidimensional array access mechanism, the control structure, and the variable storage mechanism, wherein the target model file is used to deploy the target model of the model to be converted in the target hardware space; The conversion module is further configured to convert an instruction set for the target hardware space based on each of the nodes based on a preset mapping relationship between operators and instructions; convert a multidimensional array access mechanism for the target hardware space based on each of the connection edges based on a mapping relationship between tensor transfer and multidimensional arrays in the data flow; determine a control structure for the target hardware space based on a loop or branch structure of each of the nodes; determine a variable storage mechanism for the target hardware space based on the initializer; for each connection edge, determine a tensor size and a tensor order for the tensor transfer process; determine a corresponding memory space size based on the tensor size, and determine an access order for the memory space based on the tensor order; generate a multidimensional array access mechanism for the target hardware space based on the memory space size and the memory space access order, wherein the multidimensional array access mechanism for the target hardware space includes pointer operations and array access; convert the model parameters in the initializer into global or local variables for the target hardware space; allocate memory space and a read path for the global or local variables to obtain a variable storage mechanism for the target hardware space.

6. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the model conversion method according to any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the model conversion method according to any one of claims 1 to 4.

Images