A model design method, device, apparatus and storage medium
By inserting transformation operators into NVDLA and utilizing the central processing unit to transform data patterns, the problem that NVDLA cannot handle convolution operators where both inputs are feature data is solved, thus improving device utilization and model processing speed.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHENGZHOU YUNHAI INFORMATION TECH CO LTD
- Filing Date
- 2023-03-24
- Publication Date
- 2026-07-07
AI Technical Summary
The current design of NVDLA cannot natively support convolution operators where both inputs are feature data, making it impossible to execute some newer deep learning models.
First and second transformation operators are inserted before operators that meet preset conditions and assigned to the central processing unit to transform feature data into weight data in real time during the inference phase. Data mode transformation is achieved through the TVM compiler and CPU core.
It improves the utilization of NVDLA devices and the processing speed of models, supports previously unexecutable operators such as batch matrix multiplication, and enhances the execution efficiency of models.
Smart Images
Figure CN116301900B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, and in particular to a model design method, apparatus, device, and storage medium. Background Technology
[0002] NVDLA (The NVIDIA Deep Learning Accelerator) is an open-source, free deep learning inference accelerator architecture launched by NVIDIA. NVDLA can be divided into hardware and software components. The hardware component mainly includes a series of intellectual property cores (IP cores), while the software component mainly includes the kernel-mode driver and user-mode driver. Although there are many deep learning inference accelerators on the market, open-source and free deep learning inference accelerators are very rare, making NVDLA a significant project. NVDLA supports operators such as convolution, activation, pooling, and normalization, enabling the inference of some deep learning models. Convolution is a common operator in deep learning models, and NVDLA can compute the convolution between a feature data point and a weight data point. NVDLA distinguishes between feature data and weight data; feature data is usually an intermediate computation result during the inference process, while weight data is usually obtained through offline pre-training. For NVDLA hardware, feature data and weight data need to be arranged in memory in specific ways, and these arrangements differ. In the NVDLA software, weight data is preprocessed in the NVDLA compiler, and remains unchanged during each inference process. For some classic deep learning models, the weight data of convolution operators does indeed remain unchanged during inference, and NVDLA supports such convolution operators. However, for some recent deep learning models, the weight data of some convolution operators are intermediate computation results of the model; in other words, both inputs to these convolution operators are feature data. The current design of NVDLA does not natively support convolution operators where both inputs are feature data, therefore the original NVDLA does not support some newer deep learning models. Summary of the Invention
[0003] In view of this, the purpose of this invention is to provide a model design method, apparatus, device, and storage medium capable of data mode conversion, improving the device utilization of the target compiler and the model processing speed. The specific solution is as follows:
[0004] Firstly, this application discloses a model design method, including:
[0005] Obtain the original model, and extract operators that meet preset conditions from the original model to obtain the target operator;
[0006] Insert a first transformation operator and a second transformation operator before the target operator;
[0007] The target operator is assigned to the target compiler, and the first transformation operator and the second transformation operator are assigned to the central processing unit to obtain the compiled model.
[0008] Optionally, after obtaining the original model, the process further includes:
[0009] The original model is converted into a TVM intermediate representation using a TVM front-end compiler;
[0010] Accordingly, obtaining the target operator from the original model that satisfies the preset conditions includes:
[0011] Obtain operators that meet preset conditions from the intermediate representation of the TVM to obtain the target operator.
[0012] Optionally, obtaining the target operator from the intermediate representation of the TVM that satisfies preset conditions includes:
[0013] The target operator is obtained by acquiring the operators supported by the NVDLA device, whose current input is feature data and whose specified input is weight data, from all the operators represented in the TVM intermediate representation.
[0014] Optionally, after assigning the target operator to the target compiler and assigning the first transformation operator and the second transformation operator to the central processing unit to obtain the compiled model, the method further includes:
[0015] The compiled model is deployed to the target device so that the target device can obtain the target input information of the target operator based on the first conversion operator and the second conversion operator, and input the target input information into the target operator to perform preset operator processing operations.
[0016] Optionally, obtaining the target input information of the target operator based on the first transformation operator and the second transformation operator includes:
[0017] Obtain the current input information of the target operator;
[0018] The central processing unit is invoked to convert the current input information into the target input information using the first conversion operator and the second conversion operator.
[0019] Optionally, the step of converting the current input information into the target input information through the first conversion operator and the second conversion operator includes:
[0020] Obtain the first element value corresponding to the current input information;
[0021] The current input information is converted into an intermediate representation of the target data using the first conversion operator and the first element value;
[0022] The second element value is determined based on the intermediate representation of the target data and the second transformation operator, so as to determine the target input information based on the second element value.
[0023] Optionally, after obtaining the original model, the process further includes:
[0024] The operators that do not meet the preset conditions are obtained from the original model to obtain the original operators, and the original operators are directly assigned to the target compiler.
[0025] Secondly, this application discloses a model design apparatus, comprising:
[0026] The operator determination module is used to obtain the original model and obtain operators that meet preset conditions from the original model to obtain the target operator;
[0027] A transformation operator insertion module is used to insert a first transformation operator and a second transformation operator before the target operator;
[0028] The operator allocation module is used to allocate the target operator to the target compiler and allocate the first transformation operator and the second transformation operator to the central processing unit to obtain the compiled model.
[0029] Thirdly, this application discloses an electronic device, including:
[0030] Memory, used to store computer programs;
[0031] A processor for executing the computer program to implement the steps of the model design method disclosed above.
[0032] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the model design method disclosed above.
[0033] As can be seen, this application provides a model design method, including: obtaining an original model and obtaining operators that meet preset conditions from the original model to obtain a target operator; inserting a first transformation operator and a second transformation operator before the target operator; assigning the target operator to a target compiler and assigning the first transformation operator and the second transformation operator to a central processing unit (CPU) to obtain a compiled model. Therefore, this application, by adding transformation operators before operators that meet preset conditions and assigning the transformation operators to the CPU to obtain a compiled model, enables the CPU to convert unusable data received by the target operator into usable data when the target operator receives data after the compiled model is deployed on the target device. This data conversion is achieved, improving the device utilization of the target compiler and the model processing speed. Attached Figure Description
[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0035] Figure 1 This is a flowchart of a model design method disclosed in this application;
[0036] Figure 2 This is a schematic diagram of a Tengine method disclosed in this application;
[0037] Figure 3 This is a schematic diagram of an ONNC method disclosed in this application;
[0038] Figure 4 This is a schematic diagram of a system device module disclosed in this application;
[0039] Figure 5 This is a flowchart of a specific model design method disclosed in this application;
[0040] Figure 6 This is a schematic diagram of a model design method disclosed in this application;
[0041] Figure 7 The schematic diagram of the device structure provided for this application;
[0042] Figure 8 This application provides a structural diagram of an electronic device. Detailed Implementation
[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0044] Currently, in the NVDLA software, weight data is preprocessed within the NVDLA compiler, and remains unchanged during each inference process. For some classic deep learning models, the weight data of convolution operators does indeed remain constant during inference, and NVDLA supports such convolution operators. However, for some recent deep learning models, the weight data of some convolution operators are intermediate computation results of the model; in other words, both inputs to these convolution operators are feature data. The current design of NVDLA does not natively support convolution operators where both inputs are feature data, therefore the original NVDLA does not support some newer deep learning models. To address this, this application provides a model design method that enables data mode conversion, improves the device utilization of the target compiler, and increases the model processing speed.
[0045] This invention discloses a model design method, see [link to relevant documentation]. Figure 1 As shown, the method includes:
[0046] Step S11: Obtain the original model and extract operators that meet preset conditions from the original model to obtain the target operator.
[0047] In this embodiment, an original model is obtained, and operators that meet preset conditions are extracted from the original model to obtain target operators. Specifically, operators supported by the NVDLA device, whose current input is feature data and whose specified input is weight data are extracted from all operators in the original model to obtain the target operators. Furthermore, after obtaining the original model, operators that do not meet the preset conditions are extracted from the original model to obtain original operators, and these original operators are directly assigned to the target compiler.
[0048] In existing technology one, Tengine is an open-source deep learning inference engine for embedded devices. Tengine supports NVDLA as a backend for inference. For a deep learning inference computation graph, Tengine allocates operators supported by NVDLA to NVDLA for execution on a subgraph basis, while operators not supported by NVDLA are allocated to other backends (such as CPUs (Central Processing Units)) for execution on a subgraph basis. Figure 2 As shown, for the subgraph assigned to NVDLA for execution, Tengine interfaces with the NVDLA compiler, converting the subgraph into a CanonicalAST object within the NVDLA compiler, thus delegating the subsequent compilation process to the software portion of NVDLA. Because Tengine directly connects to the NVDLA compiler, for operators not supported by NVDLA itself, Tengine can only resort to CPU computation. Specifically, for convolution operators where both inputs are feature data, Tengine cannot support their operation on NVDLA. Furthermore, for the NVDLA backend, Tengine cannot convert feature data into weight data during the inference phase.
[0049] In the second existing technology, ONNC is an open-source compiler primarily targeting deep learning accelerators. ONNC (Open Neural Network Compiler) supports using NVDLA hardware as a backend. The ONNC project implements a compiler specifically for NVDLA, eliminating the dependency on the original NVDLA compiler. ONNC supports compiling ONNX (Open Neural Network Exchange) models into NVDLA loadable files, such as... Figure 3 As shown, the ONNX model is converted into an ONNC intermediate representation at the ONNC compiler frontend, and then the NVDLA task is executed through the ONNC compiler. The weight rearrangement of ONNC is performed in the compiler. It does not support the real-time conversion of feature data into weight data during inference. Therefore, ONNC does not support convolutions where both inputs are feature data to be executed on the NVDLA device.
[0050] Understandably, to address the shortcomings of existing technologies, this solution implements a data mode conversion method during the inference phase. Leveraging the Apache TVM compiler and CPU core, feature data is converted into weight data in real-time during inference. This conversion method allows some operators in NVDLA that require weight data as input to also input feature data, improving the usability of NVDLA computation. Therefore, target operators that meet preset conditions are first identified from all operators to set the necessary conversion operators for the data conversion operation.
[0051] Step S12: Insert the first transformation operator and the second transformation operator before the target operator.
[0052] In this embodiment, after obtaining the target operator by acquiring operators that meet preset conditions from the original model, a first transformation operator and a second transformation operator are inserted before the target operator. It can be understood that two new operators are registered in the TVM compiler; for example, the first transformation operator is denoted as op1 and the second transformation operator as op2. The function of op1 is to transform the feature data into an intermediate data representation, and the function of op2 is to transform the intermediate data representation into weight data.
[0053] Step S13: Assign the target operator to the target compiler, and assign the first transformation operator and the second transformation operator to the central processing unit to obtain the compiled model.
[0054] In this embodiment, after inserting a first transformation operator and a second transformation operator before the target operator, the target operator is assigned to the target compiler, and the first transformation operator and the second transformation operator are assigned to the central processing unit to obtain the compiled model. It can be understood that, using the TVM compilation model, the op operator (i.e., the target operator) is assigned to the NVDLA compiler for compilation, and op1 and op2 are assigned to the CPU. After the model compilation is complete, the compiled model is deployed to the target device. The input is transformed by calling the implementations of op1 and op2 in the kernel-mode driver, and then the transformed weight data is input to the op operator, causing the op operator to execute on the NVDLA.
[0055] To enable NVDLA to transform feature data into weight data during inference, such as Figure 4 As shown, the system used in this application includes not only an NVDLA accelerator but also a CPU core. The NVDLA accelerator and the CPU core can access the same memory. It also includes an external computer responsible for compiling the model and reading and writing data in memory. The external computer has a TVM compiler and a user-mode driver from the NVDLA software installed.
[0056] Furthermore, based on the data mode conversion method proposed in this solution for the NVDLA inference phase, NVDLA can execute some previously unsupported operators, such as batch matrix multiplication. If such operators exist in the model, improving the data mode conversion method can increase the utilization of the NVDLA device and accelerate inference speed. This solution can also be used in other deep learning accelerators designed based on NVDLA.
[0057] As can be seen, this application provides a model design method, including: obtaining an original model and obtaining operators that meet preset conditions from the original model to obtain a target operator; inserting a first transformation operator and a second transformation operator before the target operator; assigning the target operator to a target compiler and assigning the first transformation operator and the second transformation operator to a central processing unit (CPU) to obtain a compiled model. Therefore, this application, by adding transformation operators before operators that meet preset conditions and assigning the transformation operators to the CPU to obtain a compiled model, enables the CPU to convert unusable data received by the target operator into usable data when the target operator receives data after the compiled model is deployed on the target device. This data conversion is achieved, improving the device utilization of the target compiler and the model processing speed.
[0058] See Figure 5 As shown, this embodiment of the invention discloses a model design method. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution.
[0059] Step S21: Obtain the original model and convert it into a TVM intermediate representation using the TVM front-end compiler.
[0060] In this embodiment, the original model is obtained, and then converted into a TVM intermediate representation using a TVM front-end compiler. It is understood that, as... Figure 6 As shown, the original model is input into the TVM front-end compiler, which converts the original model into a TVM intermediate representation (op). Then, a pass operation is performed to determine whether the operators in the original model meet the preset conditions.
[0061] Step S22: Obtain operators that meet preset conditions from the TVM intermediate representation to obtain the target operator.
[0062] Step S23: Insert the first transformation operator and the second transformation operator before the target operator.
[0063] Step S24: Assign the target operator to the target compiler, and assign the first transformation operator and the second transformation operator to the central processing unit to obtain the compiled model.
[0064] Step S25: Deploy the compiled model to the target device so that the target device can obtain the target input information of the target operator based on the first transformation operator and the second transformation operator.
[0065] In this embodiment, the target operator is assigned to the target compiler, and the first transformation operator and the second transformation operator are assigned to the central processing unit (CPU) to obtain the compiled model. The compiled model is then deployed to the target device, so that the target device can obtain the target input information of the target operator based on the first transformation operator and the second transformation operator. Specifically, the current input information of the target operator is obtained; the CPU is invoked to obtain a first element value corresponding to the current input information; the current input information is converted into a target data intermediate representation using the first transformation operator and the first element value; a second element value is determined based on the second transformation operator and the target data intermediate representation, so that the target input information is determined based on the second element value.
[0066] Understandably, given a new original model, TVM is used to convert the original model into an intermediate TVM representation. Then, a pass is executed to determine the operator that meets the following three conditions: (1) NVDLA device support; (2) a certain current input is feature data; (3) but the operator requires the input to be weight data. The operator that meets the above three conditions is determined as the target operator (denoted as op). Two operators, op1 and op2, are inserted before the input that the target operator needs to process so that the feature data can be converted into weight data based on op1 and op2. Feature data and weight data are two different ways of arranging data: Feature data can be understood as a five-dimensional tensor with dimensions N, C / 32, H, W, 32, and the element value of the index (n,c1,h,w,c2) in this data is denoted as b[n*C*H*W+c1*H*W*32+h*W*32+w*32+c2]. On the other hand, weight data can be understood as a six-dimensional tensor with dimensions N / 32, C / 64, H, W, 32, 64, and the element value of the index (k1,c1,h,w,k2,c2) in this data is denoted as b[k1*C*H*W*32+c1*H*W*32*64+h*W*32*64+w*32*64+k2*64+c2]. To transform feature data into weight data, an intermediate data representation is introduced. This intermediate data representation can be understood as a 4-dimensional tensor with dimensions N, C, H, and W. The kernel-mode driver of the system provides the specific implementations of operators op1 and op2. Specifically, for op1, the implementation formula is as follows:
[0067] output[n*C*H*W+c*H*W+h*W+w]=input[n*C*H*W+floor(c / 32)*H*W+h*W*32+w*32+(c-floor(c / 32)*32)];
[0068] For op2, the specific formula for implementation is:
[0069] output[floor(n / 32)*C*H*W*32+floor(c / 64)*H*W*32*64+h*W*32*64+w*32*64+(n-floor(n / 32)*32)*64+(c-floor(c / 64)*64)]=input[n*C*H*W+c*H*W+h*W+w].
[0070] Step S26: Input the target input information into the target operator and perform preset operator processing operations.
[0071] In this embodiment, the compiled model is deployed to the target device so that the target device, after obtaining the target input information of the target operator based on the first and second conversion operators, inputs the target input information into the target operator to perform preset operator processing operations. It is understood that, for example, the Inspur F37X is an FPGA, and the above-mentioned data mode conversion method executed during the NVDLA inference stage is implemented on the Inspur F37X. Two cores are deployed on the F37X: one is a CPU core based on the RISC-V (Reduced Instruction Set Computer Five) instruction set, and the other is an NVDLA accelerator. There is no operating system on the CPU; the development of operators op1 and op2 is completed in the kernel-mode driver. On an external computer, TVM is installed in a Docker environment, and operators op1 and op2 are registered on TVM. The TVM is then integrated with the NVDLA compiler, which modifies the settings regarding feature data and weight data. For operators whose input should be in the form of weight data, the input is allowed to be in the form of feature data, thus completing the data mode conversion executed during the NVDLA inference stage. Therefore, NVDLA can execute two dense operators whose inputs are both feature data. By concatenating these dense operators, batch matrix multiplication is implemented on NVDLA. In this way, the batch matrix operators in the BERT-base model are executed on NVDLA, thus accelerating the execution speed of the BERT-base model.
[0072] For details regarding steps S22 to S24, please refer to the corresponding content disclosed in the foregoing embodiments, which will not be repeated here.
[0073] As can be seen, this embodiment of the application obtains an original model, converts the original model into a TVM intermediate representation using a TVM front-end compiler, obtains operators that meet preset conditions from the TVM intermediate representation to obtain a target operator, inserts a first transformation operator and a second transformation operator before the target operator, assigns the target operator to a target compiler, and assigns the first transformation operator and the second transformation operator to a central processing unit to obtain a compiled model, deploys the compiled model to a target device so that the target device can obtain the target input information of the target operator based on the first transformation operator and the second transformation operator, and inputs the target input information into the target operator to perform preset operator processing operations, thereby realizing the conversion of data mode and improving the device utilization of the target compiler and the processing speed of the model.
[0074] See Figure 7 As shown in the figure, this application also discloses a model design apparatus, including:
[0075] Operator determination module 11 is used to obtain the original model and obtain operators that meet preset conditions from the original model to obtain the target operator;
[0076] The transformation operator insertion module 12 is used to insert a first transformation operator and a second transformation operator before the target operator;
[0077] Operator allocation module 13 is used to allocate the target operator to the target compiler and allocate the first transformation operator and the second transformation operator to the central processing unit to obtain the compiled model.
[0078] As can be seen, this application includes: obtaining an original model, and obtaining operators that meet preset conditions from the original model to obtain a target operator; inserting a first transformation operator and a second transformation operator before the target operator; assigning the target operator to a target compiler, and assigning the first transformation operator and the second transformation operator to a central processing unit (CPU) to obtain a compiled model. Therefore, this application, by adding transformation operators before operators that meet preset conditions and assigning the transformation operators to the CPU to obtain a compiled model, enables the CPU to convert unusable data received by the target operator into usable data when the target operator receives data after the compiled model is deployed on the target device, and then send the unusable data to the target compiler for processing. This achieves data mode conversion, improves the device utilization of the target compiler, and increases the model processing speed.
[0079] In some specific embodiments, the operator determination module 11 specifically includes:
[0080] The model conversion unit is used to convert the original model into a TVM intermediate representation through the TVM front-end compiler;
[0081] The target operator determination unit is used to obtain, from all the operators represented in the TVM intermediate representation, the operators supported by the NVDLA device, whose current input is feature data and whose specified input is weight data, so as to obtain the target operator;
[0082] The original operator determination unit is used to obtain the operators that do not meet the preset conditions from the original model to obtain the original operators, and directly assign the original operators to the target compiler.
[0083] In some specific embodiments, the transformation operator insertion module 12 specifically includes:
[0084] A transformation operator insertion unit is used to insert a first transformation operator and a second transformation operator before the target operator.
[0085] In some specific embodiments, the operator allocation module 13 specifically includes:
[0086] The target operator allocation unit is used to allocate the target operator to the target compiler;
[0087] A transformation operator allocation unit is used to allocate the first transformation operator and the second transformation operator to the central processing unit to obtain the compiled model;
[0088] The compiled model deployment unit is used to deploy the compiled model to the target device;
[0089] The current input information acquisition unit is used to acquire the current input information of the target operator;
[0090] A central processing unit (CPU) invocation unit is used to invoke the CPU.
[0091] The first element value acquisition unit is used to acquire the first element value corresponding to the current input information;
[0092] The target data intermediate representation acquisition unit is used to convert the current input information into a target data intermediate representation using the first conversion operator and the first element value;
[0093] The target input information determination unit is used to determine a second element value based on the intermediate representation between the second conversion operator and the target data, so as to determine the target input information based on the second element value;
[0094] The target input information input unit is used to input the target input information into the target operator to perform preset operator processing operations.
[0095] Furthermore, embodiments of this application also provide an electronic device. Figure 8 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.
[0096] Figure 8 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the model design method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.
[0097] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.
[0098] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.
[0099] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including computer programs capable of performing the model design method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.
[0100] Furthermore, this application also discloses a storage medium storing a computer program, which, when loaded and executed by a processor, implements the model design method steps disclosed in any of the foregoing embodiments.
[0101] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.
[0102] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0103] The above provides a detailed description of the model design method, apparatus, device, and storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A model design method, characterized in that, include: Obtain the original model, and extract operators that meet preset conditions from the original model to obtain the target operator; A first transformation operator and a second transformation operator are inserted before the target operator; the first transformation operator is an operator that transforms feature data into an intermediate data representation; the second transformation operator is an operator that transforms the intermediate data representation into weight data. The target operator is assigned to the target compiler, and the first transformation operator and the second transformation operator are assigned to the central processing unit to obtain the compiled model; the target compiler is the NVDLA compiler. After obtaining the original model, the process also includes: converting the original model into a TVM intermediate representation using a TVM front-end compiler; Obtaining operators that satisfy preset conditions from the original model to obtain the target operator includes: obtaining operators that satisfy preset conditions from the TVM intermediate representation to obtain the target operator; Obtaining operators that meet preset conditions from the TVM intermediate representation to obtain target operators includes: obtaining operators supported by the NVDLA device from all operators in the TVM intermediate representation, where the current input of the operator is feature data and the specified input of the operator is weight data, to obtain target operators.
2. The model design method according to claim 1, characterized in that, After assigning the target operator to the target compiler and assigning the first transformation operator and the second transformation operator to the central processing unit to obtain the compiled model, the method further includes: The compiled model is deployed to the target device so that the target device can obtain the target input information of the target operator based on the first conversion operator and the second conversion operator, and input the target input information into the target operator to perform preset operator processing operations.
3. The model design method according to claim 2, characterized in that, The step of obtaining the target input information of the target operator based on the first transformation operator and the second transformation operator includes: Obtain the current input information of the target operator; The central processing unit is invoked to convert the current input information into the target input information using the first conversion operator and the second conversion operator.
4. The model design method according to claim 3, characterized in that, The step of converting the current input information into the target input information using the first conversion operator and the second conversion operator includes: Obtain the first element value corresponding to the current input information; The current input information is converted into an intermediate representation of the target data using the first conversion operator and the first element value; The second element value is determined based on the intermediate representation of the target data and the second transformation operator, so as to determine the target input information based on the second element value.
5. The model design method according to any one of claims 1 to 4, characterized in that, After obtaining the original model, the process also includes: The operators that do not meet the preset conditions are obtained from the original model to obtain the original operators, and the original operators are directly assigned to the target compiler.
6. A model design device, characterized in that, include: The operator determination module is used to obtain the original model and obtain operators that meet preset conditions from the original model to obtain the target operator; A transformation operator insertion module is used to insert a first transformation operator and a second transformation operator before the target operator; the first transformation operator is an operator that transforms feature data into an intermediate data representation; the second transformation operator is an operator that transforms the intermediate data representation into weight data; An operator allocation module is used to allocate the target operator to the target compiler and to allocate the first transformation operator and the second transformation operator to the central processing unit to obtain the compiled model; the target compiler is an NVDLA compiler; After obtaining the original model, the process also includes: converting the original model into a TVM intermediate representation using a TVM front-end compiler; Obtaining operators that satisfy preset conditions from the original model to obtain the target operator includes: obtaining operators that satisfy preset conditions from the TVM intermediate representation to obtain the target operator; Obtaining operators that meet preset conditions from the TVM intermediate representation to obtain target operators includes: obtaining operators supported by the NVDLA device from all operators in the TVM intermediate representation, where the current input of the operator is feature data and the specified input of the operator is weight data, to obtain target operators.
7. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the model design method as described in any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the model design method as described in any one of claims 1 to 5.