Model conversion method and device and electronic equipment thereof

Abstract

The embodiment of the invention provides a model conversion method which comprises the steps that a computational graph of a to-be-converted model is obtained, the computational graph comprises an initializer, a plurality of nodes and connecting edges between the nodes, the initializer comprises model parameters of the to-be-converted model, each node represents one operation in the to-be-converted model, and each node represents one operation in the to-be-converted model; the connection edge represents a data stream between operation; based on the computational graph, converting an instruction set, a multi-dimensional array access mechanism, a control structure and a variable storage mechanism for the target hardware space; based on an instruction set, the multi-dimensional array access mechanism, the control structure and the variable storage mechanism, a target model file of a to-be-converted model in the target hardware space is generated, and the target model file is used for deploying a target model of the to-be-converted model in the target hardware space. By means of the method, conversion of different models on different hardware platforms can be achieved, and the portability of the models and interoperability between frames are improved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning models, and in particular to a model conversion method, device, electronic device and storage medium thereof. Background Art

[0002] With the development of deep learning technology, neural network models have been widely used in multiple fields. In order to enable neural network models to be used on hardware devices corresponding to different fields, it is often necessary to migrate the trained models from one hardware environment to another.

[0003] Since existing model conversion methods cannot adapt to the compatibility differences between different hardware platforms, complex adjustments and optimizations may be required when transferring models to adapt to the characteristics of the target hardware. Therefore, existing model conversion methods have the problem of conversion between different models, which will lead to a decrease in model performance and accuracy, and increase the difficulty of deployment and resource consumption. Summary of the invention

[0004] The embodiment of the present invention provides a model conversion method to solve the problem that the existing model conversion method has conversion between different models, which leads to a decrease in model performance and accuracy, and increases the difficulty of deployment and resource consumption.

[0005] In a first aspect, an embodiment of the present invention provides a model conversion method, the method comprising the following steps: Obtain a computation graph of the model to be converted, wherein the computation graph includes an initializer, multiple nodes, and connecting edges between the nodes, the initializer includes model parameters of the model to be converted, each node represents an operation in the model to be converted, and the connecting edges represent data flows between the operation operations; Based on the computation graph, convert the instruction set, multi-dimensional array access mechanism, control structure and variable storage mechanism for the target hardware space; Based on the instruction set, the multidimensional array access mechanism, the control structure and the variable storage mechanism, a target model file of the model to be converted in the target hardware space is generated, and the target model file is used to deploy the target model of the model to be converted in the target hardware space.

[0006] Optionally, the step of obtaining a computational graph of the model to be converted includes: Obtaining the computing operations of the model to be converted in the original space and the data flow between the computing operations; Constructing a directed acyclic graph with the computing operations as nodes and the data flows between the computing operations as connecting edges; Obtaining model parameters of the model to be converted, and storing the model parameters in an initializer; Based on the directed acyclic graph and the initializer, a computational graph of the model to be converted is obtained.

[0007] Optionally, in the computation graph, each of the nodes corresponds to an operator, the operator corresponds to the type of the operation, each connection edge corresponds to a tensor transfer of a data stream, and the step of converting an instruction set, a multidimensional array access mechanism, a control structure, and a variable storage mechanism for a target hardware space based on the computation graph includes: Based on a preset mapping relationship between operators and instructions, converting an instruction set for the target hardware space on the basis of each of the nodes; Based on the mapping relationship between the tensor transfer of the data flow and the multidimensional array, on the basis of each of the connection edges, converting a multidimensional array access mechanism for the target hardware space; Determining a control structure for the target hardware space based on a loop or branch structure of each of the nodes; Based on the initializer, a variable storage mechanism of the target hardware space is determined.

[0008] Optionally, the step of converting a multidimensional array access mechanism for the target hardware space based on each of the connection edges and the mapping relationship between the tensor transfer of the data flow and the multidimensional array includes: For each connection edge, determine the tensor size and tensor order of the tensor transfer process; Based on the tensor size, determining a corresponding memory space size, and based on the tensor order, determining an access order of the memory space; Based on the size of the memory space and the access order of the memory space, a multidimensional array access mechanism of the target hardware space is generated, and the multidimensional array access mechanism of the target hardware space includes pointer operation and array access.

[0009] Optionally, the step of generating a target model file of the model to be converted in the target hardware space based on the instruction set, the multidimensional array access mechanism, the control structure and the variable storage mechanism includes: Generate a code file corresponding to the target hardware space based on the instruction set, the multidimensional array access mechanism, the control structure and the variable storage mechanism; Based on the code file corresponding to the target hardware space, a target model file of the model to be converted in the target hardware space is determined.

[0010] Optionally, the step of determining the target model file of the model to be converted in the target hardware space based on the code file corresponding to the target hardware space includes: The code file is optimized based on the optimization tool chain corresponding to the target hardware space to obtain a target model file of the model to be converted in the target hardware space.

[0011] In a second aspect, an embodiment of the present invention further provides a model conversion device, the model conversion device comprising: An acquisition module, used to acquire a computational graph of a model to be converted, wherein the computational graph includes an initializer, a plurality of nodes, and connecting edges between nodes, wherein the initializer includes model parameters of the model to be converted, each node represents an operation in the model to be converted, and the connecting edges represent data flows between the operation operations; A conversion module, used to convert an instruction set, a multi-dimensional array access mechanism, a control structure, and a variable storage mechanism for a target hardware space based on the computation graph; A generation module is used to generate a target model file of the model to be converted in the target hardware space based on the instruction set, the multidimensional array access mechanism, the control structure and the variable storage mechanism, and the target model file is used to deploy the target model of the model to be converted in the target hardware space.

[0012] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the model conversion method provided in the embodiment of the present invention when executing the computer program.

[0013] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the model conversion method provided in the embodiment of the invention are implemented.

[0014] In an embodiment of the present invention, a computation graph of a model to be converted is obtained, wherein the computation graph includes an initializer, multiple nodes, and connecting edges between nodes, wherein the initializer includes model parameters of the model to be converted, each node represents an operation in the model to be converted, and the connecting edges represent data flows between the operation operations; based on the computation graph, an instruction set, a multidimensional array access mechanism, a control structure, and a variable storage mechanism for a target hardware space are converted; based on the instruction set, the multidimensional array access mechanism, the control structure, and the variable storage mechanism, a target model file of the model to be converted in the target hardware space is generated, wherein the target model file is used to deploy a target model of the model to be converted in the target hardware space. The above method can realize conversion of different models on different hardware platforms, thereby improving the portability of the model and interoperability between frameworks. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 is a flow chart of a model conversion method provided by an embodiment of the present invention; Figure 2 yes Figure 1 Specific flow chart of step S102; Figure 3 yes Figure 1 Another specific flow chart of step S102; Figure 4 yes Figure 3 Specific flow chart of step S1026; Figure 5 yes Figure 3 Specific flow chart of step S1028; Figure 6 yes Figure 1 Specific flow chart of step S103; Figure 7 yes Figure 6 Specific flow chart of step S105; Figure 8 is a structural schematic diagram of a model conversion device provided in an embodiment of the present invention; Fig. 9 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] like Figure 1 As shown, Figure 1 : is a flow chart of a model conversion method provided by an embodiment of the present invention, the model conversion method comprising the steps of: S101. Obtain a computational graph of a model to be converted.

[0019] In an embodiment of the present invention, the above-mentioned model conversion method can be applied to a model conversion device. The above-mentioned model conversion device has functions such as model data processing, model data transmission and reception, and model data storage, and can be constructed based on a server or a server cluster. The above-mentioned server or server cluster can be an electronic device with model data processing capabilities.

[0020] The above computational graph may include but is not limited to initializers, multiple nodes, and connecting edges between nodes. Specifically, the computational graph can be used to show the functions of each component content in different models to be converted and their roles in the models. This embodiment takes the ONNX model as an example, and the computational graph components in the ONNX model may include but are not limited to the above initializers, multiple nodes, and connecting edges between nodes.

[0021] The initializer may include but is not limited to model parameters of the model to be converted, such as weight parameters and bias parameters, which are learned during model training and used to perform predictions during model inference.

[0022] Each of the above nodes can be used to represent an operation in the above model to be converted. For example, each node represents an operation, such as convolution (Conv), linear transformation (such as matrix multiplication), activation function (such as ReLU), etc. These operations are usually performed on high-dimensional data, especially when converting between different models.

[0023] The above-mentioned connection edges can be used to represent the data flow between calculation operations. Specifically, the above-mentioned connection edges can represent the data flow, that is, how information flows from one node to another. In the ONNX calculation graph, the connection edges usually transmit tensors, which are multidimensional arrays that can contain, for example, image data, weight matrices, etc.

[0024] The above-mentioned model to be converted may include but is not limited to any model that can perform machine learning, that is, an original model that can be converted by the above-mentioned model conversion method and generate a corresponding target model file. This embodiment can be illustrated by, for example, an ONNX model. Different models to be converted have different characteristics, such as the composition, operation and control flow of the computational graph. In this embodiment, different target model conversion strategies can be adopted according to the computational graph content of different models to be converted.

[0025] S102. Based on the computation graph, convert the instruction set, multi-dimensional array access mechanism, control structure and variable storage mechanism for the target hardware space.

[0026] In this embodiment, OPEN64 IR is used as the target hardware space for illustration. The target hardware space may be the destination of the conversion of the model to be converted or the target address of the model configuration parameters.

[0027] The above instruction set may include a mapping relationship between operators and instructions. Specifically, each operation (node) in the computational graph may be mapped to an instruction supported by the target hardware. For example, the convolution operation in the ONNX model is mapped to a series of underlying scalar operations and vector operations in the OPEN64 IR.

[0028] The above-mentioned multi-dimensional array access mechanism can be the relationship between tensor and array mapping. Specifically, the data flow (tensor transfer) can be converted into a multi-dimensional array access mechanism suitable for the target hardware, so that the storage layout of the array (such as row priority or column priority), memory address allocation and optimized array access mode (such as cache-friendly access order) can be determined. Through the multi-dimensional array access mechanism, memory optimization can be performed for large data structures such as tensors to ensure maximum data access efficiency, reduce latency and improve bandwidth utilization.

[0029] The above control structures can be used for control flow analysis, for example, converting the control flow (such as conditional branches and loop structures) in the model into control structures in the target hardware space.

[0030] The above-mentioned variable storage mechanism may refer to a mechanism for variable conversion and memory allocation. Specifically, the parameters stored in the initializer of the computational graph, including global variables of the model to be converted, such as weight parameters, local variables, etc., may be converted and allocated to the memory space of the target hardware.

[0031] S103, generating a target model file of the model to be converted in the target hardware space based on the instruction set, multi-dimensional array access mechanism, control structure and variable storage mechanism.

[0032] The above target model file can be used to deploy the target model of the model to be converted in the target hardware space.

[0033] In a possible embodiment, a computational graph of a model to be converted in ONNX format is obtained through software, and the functions of the components in the computational graph and their roles in the model are analyzed. The components in the computational graph are converted for a target hardware space to obtain an instruction set, a multi-dimensional array access mechanism, a control structure, and a variable storage mechanism corresponding to the target hardware space. Finally, based on the converted instruction set, multi-dimensional array access mechanism, control structure, and variable storage mechanism, a target model file is generated that is directly deployed on the target hardware space and executed.

[0034] In an embodiment of the present invention, a computation graph of a model to be converted is obtained, the computation graph includes an initializer, multiple nodes, and connecting edges between nodes, the initializer includes model parameters of the model to be converted, each node represents an operation in the model to be converted, and the connecting edges represent data flows between the operations; based on the computation graph, an instruction set, a multidimensional array access mechanism, a control structure, and a variable storage mechanism for a target hardware space are converted; based on the instruction set, the multidimensional array access mechanism, the control structure, and the variable storage mechanism, a target model file of the model to be converted in the target hardware space is generated, and the target model file is used to deploy the target model of the model to be converted in the target hardware space. The above method can realize the conversion of different models on different hardware platforms, thereby improving the portability of the model and the interoperability between frameworks.

[0035] Optional, such as Figure 2 As shown, in step S102, the step of obtaining the computational graph of the model to be converted also includes steps S1021-S1024, wherein: S1021. Obtain the computing operations of the model to be converted in the original space and the data flow between the computing operations.

[0036] In an embodiment of the present invention, a parsing tool or library, such as ONNX Runtime or other libraries adapted to the ONNX model, can be used to read the file of the model to be converted to obtain the structure, operation, and properties corresponding to the model to be converted.

[0037] The above-mentioned operation can be understood as an operation of how data flows from a certain node to another node, such as convolution operation and other operations.

[0038] The above data flow can be a connection edge between nodes, which is used to determine how data flows from one node to another. In this embodiment, it can be explained based on the ONNX model. The connection is usually represented as a flow path of a tensor, and the process of how data interacts between an operation and another operation can also be shown.

[0039] S1022, constructing a directed acyclic graph with operation operations as nodes and data flows between operation operations as connecting edges; S1023, obtaining model parameters of the model to be converted, and storing the model parameters in an initializer; S1024. Based on the directed acyclic graph and the initializer, a computational graph of the model to be converted is obtained.

[0040] The above directed acyclic graph can be used to show the various operations and the data flow between them.

[0041] In a possible embodiment, the data flow between the calculation operations is analyzed to determine the flow direction and the specific calculation operations between the data flows. According to the flow direction and the specific calculation operations, a directed acyclic graph is constructed. Finally, the model parameters of the model to be converted stored in the initializer are traversed and optimized according to the directed acyclic graph to obtain the calculation graph of the model to be converted. Specifically, the traversal and optimization can be performed through forward propagation, backward propagation or parallel optimization to improve the precision and construction speed of the calculation graph.

[0042] Optional, such as Figure 3 As shown, in step S102, based on the computation graph, the step of converting the instruction set, multi-dimensional array access mechanism, control structure and variable storage mechanism for the target hardware space also includes steps S1025-S1028, wherein: S1025, based on the preset mapping relationship between the operator and the instruction, converting the instruction set for the target hardware space on the basis of each node; In an embodiment of the present invention, the above-mentioned preset mapping relationship may be an intermediate relationship of mapping operators in the model to be converted to basic instructions corresponding to the target hardware space.

[0043] Each of the above nodes can correspond to an operator, and the operator corresponds to the type of operation, and each connection edge corresponds to a tensor transfer of a data flow.

[0044] In a possible embodiment, by consulting a mapping table, the corresponding target hardware instructions between each node in the model to be converted and the target hardware space are determined. For example, operations such as convolution operations in the OXXN model are mapped to instructions of addition and subtraction combinations corresponding to OPEN64IR, and the converted instructions are optimized and integrated to obtain an instruction set for the target hardware space.

[0045] S1026. Based on the mapping relationship between the tensor transfer of the data flow and the multidimensional array, a multidimensional array access mechanism for the target hardware space is converted on the basis of each connection edge.

[0046] The mapping relationship between the tensor transfer of the above data flow and the multidimensional array can be a preset conversion relationship that defines how to effectively convert high-dimensional tensor data into a multidimensional array and enable the corresponding memory layout, access mode, etc. to adapt to the target hardware space.

[0047] In a possible embodiment, through the mapping relationship between the tensor transfer of the data flow and the multi-dimensional array, the tensor data storage mode is converted into a storage mode between the multi-dimensional arrays according to the calculation operation content displayed by each connection edge.

[0048] S1027. Based on the loop or branch structure of each node, determine the control structure for the target hardware space.

[0049] In one possible embodiment, by analyzing the loop or branch structure of each node, the optimal execution steps when executing the calculation operation of the current node are determined, and based on the optimal execution steps, a control structure for the target hardware space is generated. It can be understood that the control structure can also take into account the storage mechanism optimized for the target hardware space.

[0050] S1028. Based on the initializer, determine the variable storage mechanism of the target hardware space.

[0051] In a possible embodiment, the control structure, multi-dimensional array access mechanism and other parameters are stored by an initializer to obtain configuration parameters for the target hardware space, and the variable storage mechanism corresponding to the target hardware space is determined according to the configuration parameters and the storage optimization content for the target hardware space. It can be understood that the variable storage mechanism can ensure data alignment and cache optimization during the model conversion process, and ensure high efficiency during the memory access process.

[0052] Optional, such as Figure 4 As shown, in step S1026, based on the mapping relationship between the tensor transfer of the data flow and the multidimensional array, on the basis of each connection edge, the step of converting the multidimensional array access mechanism for the target hardware space also includes steps S10261-S10263, wherein: S10261. For each connection edge, determine the tensor size and tensor order of the tensor transfer process.

[0053] In an embodiment of the present invention, the above-mentioned tensor size can be calculated by calculating the size of each data element in the transmission process, and then calculating the size of the entire tensor according to the dimension and dimension size of each corresponding tensor, such as width, height, number of channels, etc.

[0054] The tensor order may be the order of various operations during the transfer process. The target hardware space requirement may be to select different tensor orders based on access storage speed priority or maximum storage priority. Generally speaking, the tensor order can be directly determined by analyzing the flow of operations in the computational graph.

[0055] S10262. Based on the size of the tensor, determine the corresponding memory space size, and based on the order of the tensors, determine the access order of the memory space.

[0056] In a possible embodiment, the corresponding memory space size is determined according to the tensor size. For example, the model to be converted has a convolution layer with 32 3x3 filters, the input tensor is 28x28x3 (height x width x channel), the output tensor size is 26x26x32, and a single floating point number occupies 4 bytes. Then, the required memory space size can be calculated. The memory space size can be adjusted according to the specific type and memory requirements of the target hardware space, and the corresponding memory space size is determined for the target hardware space with a non-saturated target.

[0057] Specifically, the tensor order may be determined according to row priority or column priority, and different tensor orders may be selected for different target hardware space requirements.

[0058] The access order can also be adjusted according to the access requirements of the target hardware space, such as reducing memory access latency and improving the overall computing speed, so as to adjust the order of accessed tensors. For example, tensors with smaller sizes can be accessed first, or simple tensor contents can be converted.

[0059] S10263. Generate a multidimensional array access mechanism for the target hardware space based on the size of the memory space and the access order of the memory space.

[0060] The multi-dimensional array access mechanism of the above-mentioned target hardware space may include but is not limited to pointer operations and array access, that is, the basic elements of processing data and memory management, and the purpose of corresponding data and memory management of the target hardware space can be achieved through pointer operations and array access.

[0061] In a possible embodiment, based on the tensor data of the ONNX model, its memory space size and access order are analyzed, and it is converted into a multi-dimensional array access mechanism adapted to OPEN64 IR. In view of the hardware characteristics of OPEN64 IR, a row-first memory layout and cache optimization strategy are adopted to ensure that the tensors are accessed in an efficient order during the calculation process.

[0062] The above method generates a multi-dimensional array access mechanism with OPEN64 IR as the target hardware space, which reduces memory read latency, improves the execution efficiency of matrix operations, and significantly improves the overall performance of the ONNX model in the OPEN64 IR environment.

[0063] Optional, such as Figure 5 As shown, in step S1028, based on the initializer, the step of determining the variable storage mechanism of the target hardware space also includes steps S10281-S10282, wherein: S10281. Based on the model parameters in the initializer, convert them into global or local variables in the target hardware space; S10282. Allocate memory space and a read path for global or local variables to obtain a variable storage mechanism of the target hardware space.

[0064] In a possible embodiment, the ONNX model is used as the model to be converted, and OPEN64 IR is used as the target hardware space for explanation. The corresponding weight parameters, bias parameters, etc. are determined through the ONNX model parameters in the initializer. In order to adapt these parameters to the OPEN64 IR hardware platform, the global variables or local variables are determined by analyzing the computing requirements and parameter usage frequency corresponding to the model parameters in the ONNX model initializer. Since global variables are frequently used and need to be accessed multiple times, more memory space and the shortest reading path can be allocated to global variables, and thereby a variable storage mechanism of the target hardware space is generated, that is, a variable storage mechanism optimized for access to global variables.

[0065] Optional, such as Figure 6 As shown, in step S103, based on the instruction set, multi-dimensional array access mechanism, control structure and variable storage mechanism, the step of generating a target model file of the model to be converted in the target hardware space also includes steps S104-S105, wherein: S104, generating a code file corresponding to the target hardware space based on the instruction set, multi-dimensional array access mechanism, control structure and variable storage mechanism.

[0066] In an embodiment of the present invention, a high-level model description is converted into directly executable code, that is, the instruction set, multi-dimensional array access mechanism, control structure and variable storage mechanism of the model to be converted are converted into code description to generate a code file corresponding to the target hardware space.

[0067] Specifically, high-level operations in the instruction set, such as convolution, activation function, pooling, etc., are mapped to instructions that can be understood and executed by the target hardware space according to a preset mapping relationship. For example, operations such as convolution are decomposed into a series of multiplication and addition and subtraction instructions.

[0068] In a possible embodiment, by parsing the instruction set, multi-dimensional array access mechanism, control structure and variable storage mechanism, the corresponding functions can be described by code, and the corresponding code file is generated according to the description content and the code type that can be executed by the target hardware space.

[0069] S105. Determine the target model file of the model to be converted in the target hardware space based on the code file corresponding to the target hardware space.

[0070] In a possible embodiment, the code file corresponding to the target hardware space is compiled, and the compiled code file is executed. The code file is optimized and compiled again according to the storage space when the target hardware space is executed, so as to obtain the target model file of the model to be converted in the target hardware space.

[0071] Through the above method, the execution efficiency of the conversion can be improved, and the code file compiled for the first time can be optimized in real time according to the storage space of the target hardware space when the code file is executed, so that after the optimized code file is compiled and executed, the memory access in the target hardware space has reduced delays and improved the overall computing throughput.

[0072] Optional, such as Figure 7 As shown, in step S105, based on the code file corresponding to the target hardware space, the step of determining the target model file of the model to be converted in the target hardware space also includes step S1051: S1051. Optimize the code file based on the optimization tool chain corresponding to the target hardware space to obtain the target model file of the model to be converted in the target hardware space.

[0073] In the embodiment of the present invention, the above-mentioned optimization tool chain may be a general term for a collection of a series of software tools and technologies, such as a collection of a certain technology corresponding to a certain tool.

[0074] In a possible embodiment, the above-mentioned optimization processing can be to reduce the overhead of function calls by optimizing the inline expansion function of the tool chain compiler, and at the same time apply branch prediction and conditional execution techniques to reduce branch penalties. In terms of access frequency, the data locality optimization of OPEN64 IR can be used to arrange frequently accessed data in a fast-access cache to reduce the need to access the main memory, or the memory access mode of tensor operations can be fine-tuned according to the size of the current memory storage space.

[0075] like Figure 8 As shown, the embodiment of the present invention further provides a model conversion device 800, which includes: An acquisition module 801 is used to acquire a computation graph of a model to be converted, wherein the computation graph includes an initializer, multiple nodes, and connecting edges between nodes, wherein the initializer includes model parameters of the model to be converted, each node represents an operation in the model to be converted, and the connecting edges represent data flows between the operation operations; A conversion module 802 is used to convert an instruction set, a multi-dimensional array access mechanism, a control structure, and a variable storage mechanism for a target hardware space based on the computation graph; The first generation module 803 is used to generate a target model file of the model to be converted in the target hardware space based on the instruction set, the multi-dimensional array access mechanism, the control structure and the variable storage mechanism, and the target model file is used to deploy the target model of the model to be converted in the target hardware space.

[0076] Optionally, the acquisition module 801 includes: A first acquisition submodule is used to acquire the computing operations of the model to be converted in the original space and the data flow between the computing operations; A construction submodule, used to construct a directed acyclic graph using the operation as nodes and the data flows between the operation as connecting edges; A second acquisition submodule, used for acquiring model parameters of the model to be converted, and storing the model parameters in an initializer; The third acquisition submodule is used to obtain a computational graph of the model to be converted based on the directed acyclic graph and the initializer.

[0077] Optionally, the generating module 803 includes: A first conversion submodule, configured to convert an instruction set for the target hardware space based on each of the nodes based on a preset mapping relationship between operators and instructions; A second conversion submodule is used to convert a multidimensional array access mechanism for the target hardware space based on each of the connection edges based on the mapping relationship between the tensor transfer of the data flow and the multidimensional array; A first determination submodule, configured to determine a control structure for the target hardware space based on a loop or branch structure of each of the nodes; The second determination submodule is used to determine the variable storage mechanism of the target hardware space based on the initializer.

[0078] Optionally, the second conversion submodule includes: A first determining unit, configured to determine, for each connection edge, a tensor size and a tensor order of the tensor transfer process; A second determining unit, configured to determine a corresponding memory space size based on the tensor size, and to determine an access order of the memory space based on the tensor order; A generating unit is used to generate a multi-dimensional array access mechanism of the target hardware space based on the size of the memory space and the access order of the memory space, wherein the multi-dimensional array access mechanism of the target hardware space includes pointer operation and array access.

[0079] Optionally, the second determining submodule includes: A first conversion unit, configured to convert the model parameters in the initializer into global or local variables in the target hardware space; The first acquisition unit is used to allocate memory space and a reading path for the global or local variable to obtain a variable storage mechanism of the target hardware space.

[0080] Optionally, the above device further comprises: A second generating module, configured to generate a code file corresponding to the target hardware space based on the instruction set, the multidimensional array access mechanism, the control structure and the variable storage mechanism; The first determination module is used to determine the target model file of the model to be converted in the target hardware space based on the code file corresponding to the target hardware space.

[0081] Optionally, the first determining module includes: The optimization processing module is used to optimize the code file based on the optimization tool chain corresponding to the target hardware space to obtain the target model file of the model to be converted in the target hardware space.

[0082] like Fig. 9 As shown, an embodiment of the present invention further provides an electronic device 900, including a processor, and the processor can execute any one of the above-mentioned model conversion methods.

[0083] Specifically, it includes a processor 901 and a memory 902, and a computer program for executing the model conversion method which is stored in the memory 902 and can be run on the processor 901, wherein: The processor 901 runs the computer program of the model conversion method stored in the memory 902 to perform the following steps: Obtain a computation graph of the model to be converted, wherein the computation graph includes an initializer, multiple nodes, and connecting edges between the nodes, the initializer includes model parameters of the model to be converted, each node represents an operation in the model to be converted, and the connecting edges represent data flows between the operation operations; Based on the computation graph, convert the instruction set, multi-dimensional array access mechanism, control structure and variable storage mechanism for the target hardware space; Based on the instruction set, the multidimensional array access mechanism, the control structure and the variable storage mechanism, a target model file of the model to be converted in the target hardware space is generated, and the target model file is used to deploy the target model of the model to be converted in the target hardware space.

[0084] Optionally, the processor 901 executes the step of obtaining the calculation graph of the model to be converted, including: Obtaining the computing operations of the model to be converted in the original space and the data flow between the computing operations; Constructing a directed acyclic graph with the computing operations as nodes and the data flows between the computing operations as connecting edges; Obtaining model parameters of the model to be converted, and storing the model parameters in an initializer; Based on the directed acyclic graph and the initializer, a computational graph of the model to be converted is obtained.

[0085] Optionally, the processor 901 executes in the computation graph, each of the nodes corresponds to an operator, the operator corresponds to the type of the operation, each connection edge corresponds to a tensor transfer of a data flow, and the step of converting an instruction set, a multidimensional array access mechanism, a control structure, and a variable storage mechanism for a target hardware space based on the computation graph includes: Based on a preset mapping relationship between operators and instructions, converting an instruction set for the target hardware space on the basis of each of the nodes; Based on the mapping relationship between the tensor transfer of the data flow and the multidimensional array, on the basis of each of the connection edges, converting a multidimensional array access mechanism for the target hardware space; Determining a control structure for the target hardware space based on a loop or branch structure of each of the nodes; Based on the initializer, a variable storage mechanism of the target hardware space is determined.

[0086] Optionally, the processor 901 executes the mapping relationship between the tensor transfer based on the data flow and the multidimensional array, and the step of converting the multidimensional array access mechanism for the target hardware space based on each of the connection edges includes: For each connection edge, determine the tensor size and tensor order of the tensor transfer process; Based on the tensor size, determining a corresponding memory space size, and based on the tensor order, determining an access order of the memory space; Based on the size of the memory space and the access order of the memory space, a multidimensional array access mechanism of the target hardware space is generated, and the multidimensional array access mechanism of the target hardware space includes pointer operation and array access.

[0087] Optionally, the processor 901 executes the step of determining the variable storage mechanism of the target hardware space based on the initializer, including: Based on the model parameters in the initializer, converting into global or local variables in the target hardware space; Allocate memory space and a read path for the global or local variable to obtain a variable storage mechanism of the target hardware space.

[0088] Optionally, the processor 901 further executes the step of generating a target model file of the model to be converted in the target hardware space based on the instruction set, the multidimensional array access mechanism, the control structure, and the variable storage mechanism, including: Generate a code file corresponding to the target hardware space based on the instruction set, the multidimensional array access mechanism, the control structure and the variable storage mechanism; Based on the code file corresponding to the target hardware space, a target model file of the model to be converted in the target hardware space is determined.

[0089] Optionally, the processor 901 further executes the step of determining the target model file of the model to be converted in the target hardware space based on the code file corresponding to the target hardware space, including: The code file is optimized based on the optimization tool chain corresponding to the target hardware space to obtain a target model file of the model to be converted in the target hardware space.

[0090] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the model conversion method or the application-side model conversion method provided in the embodiment of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0091] Those skilled in the art can understand that the implementation of all or part of the processes in the above-mentioned embodiments is a program that can be completed by instructing related hardware through a computer program, and can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0092] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A model conversion method, characterized in that: include: Obtain a computation graph of the model to be converted, wherein the computation graph includes an initializer, multiple nodes, and connecting edges between the nodes, the initializer includes model parameters of the model to be converted, each node represents an operation in the model to be converted, and the connecting edges represent data flows between the operation operations; Based on the computation graph, convert the instruction set, multi-dimensional array access mechanism, control structure and variable storage mechanism for the target hardware space; Based on the instruction set, the multidimensional array access mechanism, the control structure and the variable storage mechanism, a target model file of the model to be converted in the target hardware space is generated, and the target model file is used to deploy the target model of the model to be converted in the target hardware space.

2. The model conversion method according to claim 1, characterized in that: The step of obtaining the computational graph of the model to be converted comprises: Obtaining the computing operations of the model to be converted in the original space and the data flow between the computing operations; Constructing a directed acyclic graph with the computing operations as nodes and the data flows between the computing operations as connecting edges; Obtaining model parameters of the model to be converted, and storing the model parameters in an initializer; Based on the directed acyclic graph and the initializer, a computational graph of the model to be converted is obtained.

3. The model conversion method according to claim 2, characterized in that: In the computation graph, each node corresponds to an operator, the operator corresponds to the type of the operation, each connection edge corresponds to a tensor transfer of a data stream, and the step of converting an instruction set, a multidimensional array access mechanism, a control structure, and a variable storage mechanism for a target hardware space based on the computation graph includes: Based on a preset mapping relationship between operators and instructions, converting an instruction set for the target hardware space on the basis of each of the nodes; Based on the mapping relationship between the tensor transfer of the data flow and the multidimensional array, on the basis of each of the connection edges, converting a multidimensional array access mechanism for the target hardware space; Determining a control structure for the target hardware space based on a loop or branch structure of each of the nodes; Based on the initializer, a variable storage mechanism of the target hardware space is determined.

4. The model conversion method according to claim 3, characterized in that: The step of converting a multidimensional array access mechanism for the target hardware space based on the mapping relationship between the tensor transfer of the data flow and the multidimensional array on the basis of each of the connection edges comprises: For each connection edge, determine the tensor size and tensor order of the tensor transfer process; Based on the tensor size, determining a corresponding memory space size, and based on the tensor order, determining an access order of the memory space; Based on the size of the memory space and the access order of the memory space, a multidimensional array access mechanism of the target hardware space is generated, and the multidimensional array access mechanism of the target hardware space includes pointer operation and array access.

5. The model conversion method according to claim 3, characterized in that: The step of determining the variable storage mechanism of the target hardware space based on the initializer comprises: Based on the model parameters in the initializer, converting into global or local variables in the target hardware space; Allocate memory space and a read path for the global or local variable to obtain a variable storage mechanism of the target hardware space.

6. The model conversion method according to any one of claims 1 to 5, characterized in that: The step of generating a target model file of the model to be converted in the target hardware space based on the instruction set, the multidimensional array access mechanism, the control structure and the variable storage mechanism comprises: Generate a code file corresponding to the target hardware space based on the instruction set, the multidimensional array access mechanism, the control structure and the variable storage mechanism; Based on the code file corresponding to the target hardware space, a target model file of the model to be converted in the target hardware space is determined.

7. The model conversion method according to claim 6, characterized in that: The step of determining the target model file of the model to be converted in the target hardware space based on the code file corresponding to the target hardware space comprises: The code file is optimized based on the optimization tool chain corresponding to the target hardware space to obtain a target model file of the model to be converted in the target hardware space.

8. A model conversion device, characterized in that: The model conversion device comprises: An acquisition module, used to acquire a computational graph of a model to be converted, wherein the computational graph includes an initializer, a plurality of nodes, and connecting edges between nodes, wherein the initializer includes model parameters of the model to be converted, each node represents an operation in the model to be converted, and the connecting edges represent data flows between the operation operations; A conversion module, used to convert an instruction set, a multi-dimensional array access mechanism, a control structure, and a variable storage mechanism for a target hardware space based on the computation graph; A generation module is used to generate a target model file of the model to be converted in the target hardware space based on the instruction set, the multidimensional array access mechanism, the control structure and the variable storage mechanism, and the target model file is used to deploy the target model of the model to be converted in the target hardware space.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the model conversion method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the model conversion method according to any one of claims 1 to 7 are implemented.

Images