Intelligent model transplanting method based on domestic intelligent chip

Abstract

The invention discloses an intelligent model transplantation method based on a domestic intelligent chip, and the method comprises the steps: carrying out the model rewriting through a transplantation tool provided by the domestic intelligent chip, and obtaining a model file of a corresponding version of the transplantation tool; if the internal logic of the original model is changed, operators are added to the model file of the version corresponding to the transplantation tool, the model file added with the operators is obtained, and the next step is executed; if not, directly entering the next step; performing model quantification on the model file to obtain a model file after model quantification; generating an offline model based on the model file after model quantification; and the original intelligent model and the offline model respectively load the same example, the results of the two models are compared, if the comparison difference of the results meets the requirement, transplantation is ended, and if the comparison difference of the results does not meet the requirement, a transplantation problem is searched for and resumes. According to the invention, the compatibility problem of the intelligent model is solved, and the operation efficiency, response speed and precision of the artificial intelligence model are not affected.

Description

Technical Field

[0001] The invention relates to an intelligent model transplantation method based on a domestic intelligent chip, and belongs to the fields of artificial intelligence and computer science. Background Art

[0002] The migration of intelligent models mainly focuses on how to migrate already trained artificial intelligence models (such as deep learning models, reinforcement learning models, etc.) from one platform or environment to another, while maintaining the performance of the artificial intelligence model. Usually, the platform or environment used for training artificial intelligence models is a processor with strong computing performance. The training processor has high power consumption, high price, large size and weight, and is not suitable for terminal use. Therefore, after the artificial intelligence model is trained, it is generally transplanted to a processor with lower power consumption and smaller size and weight. The existing technology basically performs lightweight processing such as pruning the artificial intelligence algorithm model and then transplants it to the terminal processor. The transplantation process is complicated and takes a long time.

[0003] So far, there have been no reports at home or abroad on methods for transplanting artificial intelligence models to domestically produced smart chips.

[0004] The paper "Soft PLC Controller Design for Easy Transplantation of Intelligent Algorithms" designs an embedded soft PLC controller, mainly for the intelligent control algorithm modeled and simulated by Simulink, which uses tools to generate structured text language that complies with the IEC61131-3 standard, and then automatically loads it into the PLC development software, so as to conveniently transplant the intelligent algorithm into the designed soft PLC controller for testing or engineering application. The intelligent control algorithm involved in this paper does not belong to the artificial intelligence algorithm that uses neural networks, deep learning or reinforcement learning training, but is just a dedicated and customized controller.

[0005] The paper "Research on FPGA Implementation Technology of Hybrid Frog Leaping Algorithm" introduces a solution to transplant a heuristic population evolution algorithm, the Hybrid Frog Leaping Algorithm (SFLA), onto FPGA. The Hybrid Frog Leaping Algorithm used in the article is not an artificial intelligence algorithm, and FPGA is used in fields such as digital signal processing with its powerful parallel processing capabilities, but it is not a dedicated artificial intelligence processor.

[0006] The paper "Research and Implementation of Intelligent Management System for Garbage Recycling Based on NB-IOT" designs a garbage disposal management solution based on embedded artificial intelligence for the garbage disposal process. The solution transplants the artificial intelligence model into the smart terminal. The article introduces how to first train the designed model on a computer and then transplant the trained model into the smart terminal. The smart terminal used in the article is a terminal with 64KbyteSRAM memory and an industrial-grade MKL36Z64VLH4 chip with Cortex-M0+ processor, which is produced by NXP Semiconductors of the Netherlands. The garbage classification model used in the article is based on LeNet-5, a lightweight convolutional neural network, and the specific process and steps of transplantation are not introduced.

[0007] The paper "Research on YOLOv5 Lightweight Algorithm for Pine Tree Count Identification" lightweights the network based on the YOLOv5 network to adapt to the subsequent use of edge computing terminals. It compares the effects before and after the algorithm lightweighting, and does not involve the transplantation of the algorithm model.

[0008] The paper "Research and Implementation of Multimodal Signal Acquisition and Recognition System for Lower Limbs" transplants the trained CNN network to the main control through Cube.AI, compresses the algorithm usage resources after transplantation, and uses STMicroelectronics' STM32Cube tool for transplantation, running on the STM32F746ZGT6 (produced by STMicroelectronics) microcontroller. Summary of the invention

[0009] The technical problem solved by the present invention is: to overcome the shortcomings of the prior art, to provide an intelligent model transplantation method based on domestic intelligent chips, to solve the compatibility problem of intelligent models, to ensure that the original model can operate normally in the environment of domestic intelligent chips, while not affecting the operating efficiency, response speed and accuracy of the artificial intelligence model.

[0010] The technical solution of the present invention is: a method for transplanting an intelligent model based on a domestic intelligent chip, comprising:

[0011] Step 1. To solve the problem that the framework version used by the original intelligent model to be transplanted is inconsistent with the framework version supported by the transplantation tool provided by the domestic intelligent chip, the model is rewritten to obtain the model file of the corresponding version of the transplantation tool;

[0012] Step 2: Determine whether the logic inside the original model has been changed when the model is rewritten. If so, add operators to the model file of the corresponding version of the migration tool to obtain the model file with the added operators and proceed to the next step. If not, proceed directly to the next step.

[0013] Step 3. Perform model quantization on the model file of the corresponding version of the migration tool or the model file with the operator added to obtain the quantized model file;

[0014] Step 4. Generate an offline model based on the model file after model quantization;

[0015] Step 5: Load the same example into the original intelligent model and the offline model respectively, and compare the results of the two models. If the difference in the results meets the requirements, the migration is terminated. If it does not meet the requirements, find the migration problems and return to Step 1 to start again.

[0016] Preferably, the process of rewriting the model is:

[0017] Determine the difference range between the framework version used by the transplanted original intelligent model and the framework version supported by the transplantation tool. If the difference between the framework version used by the transplanted original intelligent model and the framework version supported by the transplantation tool is within an acceptable difference range, decompress the original intelligent model, save the original intelligent model again using the parameters of the corresponding version of the transplantation tool, replace the corresponding operators of the transplanted original intelligent model with the operators used by the transplantation tool, reload the model, and form a model file of the corresponding version of the transplantation tool.

[0018] If the difference between the framework version used by the original intelligent model being transplanted and the framework version supported by the transplantation tool is not within an acceptable range, the transplanted algorithm needs to be rewritten and retrained using the framework version corresponding to the transplantation tool; after training, the model is reloaded to form a model file corresponding to the version of the transplantation tool.

[0019] Preferably, an operator is added to the model file of the corresponding version of the migration tool, specifically:

[0020] In the model file of the corresponding version of the transplantation tool, the following six steps are added: declaring the operator, adding the CPU implementation of the operator, adding the operator encapsulation, adding the declaration and implementation of the operator function, adding the activation function, and recompiling the model using the transplantation tool.

[0021] Preferably, declaring an operator: refers to declaring an operator in the transplant tool chain code provided by the domestic smart chip;

[0022] Adding CPU implementation of operators: refers to adding CPU implementation of operators in the transplantation tool chain code provided by domestic smart chips;

[0023] Add operator package: Generate the package header file based on the transplant tool chain template provided by the domestic smart chip. Each operator corresponds to an operator package.

[0024] Add operator kernel: call the machine learning programming library interface in the transplantation tool chain provided by domestic smart chips according to the programming logic;

[0025] Add activation function: If the framework version supported by the transplantation tool provided by the domestic smart chip does not have the activation function used by the transplanted smart model, manually add the activation function;

[0026] Recompile the model using the transplantation tool: Recompile the intelligent model using the transplantation tool chain provided by the domestic intelligent chip, and enter the coding environment to confirm whether the integration is successful.

[0027] Preferably, the method for performing model quantization is specifically as follows:

[0028] The numerical range of each parameter input in the statistical model file is obtained, and the maximum value of its absolute value is obtained, which is recorded as absmax; the scaling scale of the input floating point number mapped to the integer is calculated as scale1 = the maximum value of the integer data type / absmax, thereby calculating the scaling scale of the input data;

[0029] The numerical range of each weight in the statistical model file is obtained, and the maximum absolute value thereof is obtained, which is recorded as weightmax; the scaling scale of the weight mapped to the integer is calculated as scale2 = the maximum value of the integer data type / weightmax, thereby calculating the scaling scale of the weight;

[0030] Combined with the calculated scaling scale, the integer input data and integer weights are calculated according to the quantization formula, and an integer convolution operation is performed on them to output an integer convolution result. The integer convolution result is then processed according to the dequantization formula to obtain a floating-point convolution output, completing the model quantization and obtaining the model file after the model quantization.

[0031] Preferably, when generating an offline model, a static graph of the intelligent model is constructed using a transplantation tool, the static graph is optimized, and the device type based on the input data is optimized to generate an intelligent model that can run in a terminal based on a domestically produced intelligent chip, i.e., an offline model.

[0032] Preferably, when optimizing a static graph, an optimized static graph is obtained by removing redundant operators, merging small operators, and reusing data blocks.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] (1) The present invention provides a process and transplantation steps for migrating an artificial intelligence model trained using a high-performance GPU to an NPU processor based on the domestically produced Siyuan 270 and Siyuan 220 chips, designs a general transplantation method for intelligent algorithm models that can be applied to the domestically produced Siyuan 270 and Siyuan 220 chips, and realizes the rapid, efficient, and accurate transplantation of convolutional neural network-type intelligent algorithm models, thereby improving the efficiency of intelligent algorithm models from training to terminal application;

[0035] (2) The intelligent model transplantation method designed by the present invention solves the model compatibility problem and ensures that the artificial intelligence model trained with imported high-performance GPU can run normally in the environment of domestic intelligent chips;

[0036] (3) The intelligent algorithm model transplantation method designed in the present invention effectively solves the optimization and accuracy maintenance problems of the intelligent algorithm model, so that the intelligent algorithm model trained with a high-performance GPU can run on a domestic smart chip with reduced computing power without affecting the operating efficiency and response speed of the artificial intelligence model, and can also maintain the accuracy of the model operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flow chart of the overall method of the present invention;

[0038] Figure 2 A flow chart of the rewriting model process of the present invention;

[0039] Figure 3 A flowchart of the process of adding an operator of the present invention;

[0040] Figure 4 A flow chart of the model quantification process of the present invention;

[0041] Figure 5 A flow chart of a process for generating an offline model of the present invention;

[0042] Figure 6 It is a flow chart of comparing transplantation effects of the present invention. DETAILED DESCRIPTION

[0043] The present invention has been improved from the following aspects:

[0044] (1) Solve the compatibility issues of intelligent models and ensure that the original model can run normally in the environment of domestic intelligent chips.

[0045] (2) Solve the problem of intelligent model performance optimization, and ensure that when the hardware computing power and performance are reduced compared to the high-performance GPU used for training, the model performance is optimized without affecting the operating efficiency and response speed of the artificial intelligence model.

[0046] (3) Solve the problem of maintaining accuracy and ensure that when the hardware computing power and performance are reduced compared to the GPU used for training, measures are taken to minimize the accuracy loss of the AI ​​model or to ensure that its accuracy is not affected.

[0047] The intelligent model described in this article refers to the intelligent algorithms and intelligent models trained through artificial intelligence technology.

[0048] In order to realize the transplantation of artificial intelligence models to domestic smart chips, the following technical solutions were adopted:

[0049] (1) By adjusting the model's input and output interfaces, data formats, etc., it is ensured that the artificial intelligence model trained and generated on a high-performance GPU can run normally on the processor of a domestic smart chip, forming an intelligent model transplantation method based on domestic smart chips that can adapt to more than two types of domestic chips.

[0050] (2) By summarizing the transplantation process of artificial intelligence models with different characteristics, an intelligent algorithm model transplantation process is proposed, which effectively solves the problem of quickly upgrading the algorithm model from the simulation state to the algorithm model of embedded operation.

[0051] (3) By transplanting intelligent algorithm models between training GPUs and domestic smart chip terminal devices, it is possible to balance computing resources and response time, improve the efficiency and performance of the overall system, apply advanced artificial intelligence algorithms and models to various practical scenarios, and promote the intelligent upgrading and development of various fields.

[0052] The migration process of the intelligent model is as follows:

[0053] (1) Rewrite the model. (2) Add operators. (3) Quantize the model. (4) Generate an offline model. (5) Compare the transplantation results.

[0054] The technical solution of the present invention is: an intelligent model transplantation method based on domestic intelligent chips, including: rewriting the model, adding operators, model quantification, generating offline models, and comparing transplantation effects.

[0055] Specific:

[0056] Step 1. In order to solve the problem that the framework version used by the original intelligent model to be transplanted is inconsistent with the framework version supported by the transplantation tool provided by the domestic intelligent chip, the model is rewritten to obtain the model file of the corresponding version of the transplantation tool; the input of this step is the original intelligent model to be transplanted, and the output of this step is the intelligent model under the framework version supported by the transplantation tool.

[0057] Step 2: Determine whether the logic inside the original model has been changed when the model is rewritten. If so, add operators to the model file of the corresponding version of the migration tool, and proceed to the next step after obtaining the model file with the added operators. If not, proceed directly to the next step. The input of this step is the intelligent model under the framework version supported by the migration tool, and the output of this step is the intelligent model after the operators are added.

[0058] Step 3: quantize the model file with the added operator to obtain the model file after model quantization. The input of this step is the intelligent model after the operator is added, and the output of this step is the intelligent model after model quantization.

[0059] Step 4: Generate an offline model based on the model file after model quantization; the input of this step is the intelligent model after model quantization, and the output of this step is the generated offline model.

[0060] Step 5: Input the same example to the original intelligent model and the offline model respectively, and compare the results of the two models. If the difference in the results meets the requirements, the migration is terminated. If it does not meet the requirements, find the migration problem and return to Step 1 to start again. The input of this step is the original intelligent model and the offline model generated in the previous step, and the output of this step is the comparison result of the two input models.

[0061] 1. Rewrite the model. To address the inconsistency between the framework version used by the transplanted intelligent model and the framework version supported by the transplantation tool provided by the domestic intelligent chip, rewrite the model. The purpose of model rewriting is to enable the intelligent model to run on the framework supported by the transplantation tool.

[0062] The above-mentioned rewriting model features include: judging whether the framework version used by the transplanted original intelligent model is much higher than the framework version supported by the transplantation tool. If the difference between the framework version used by the transplanted intelligent model and the framework version supported by the transplantation tool is within an acceptable difference range, then the original intelligent model is decompressed, and the original intelligent model is saved again using the parameters of the corresponding version of the transplantation tool, and the corresponding operators of the transplanted original intelligent model are replaced with the operators used by the transplantation tool, and the model is reloaded to form a model file of the corresponding version of the transplantation tool. If the framework version used by the transplanted intelligent model is much higher than the framework version supported by the transplantation tool, the transplanted algorithm needs to be rewritten using the framework version corresponding to the transplantation tool and retrained; after training, the model is reloaded to form a model file of the corresponding version of the transplantation tool.

[0063] Currently, the migration tool supports up to 1.15.4 TensorFlow framework version and 1.3 PyTorch1.3 framework version. If the original intelligent model being migrated uses a higher version, such as TensorFlow2.0 or PyTorch1.7, it needs to be rewritten. There are two ways to rewrite: Method A: Replace some unsupported operators in the model with operators supported by the domestic intelligent chip migration tool chain; Method B: Change the framework version, for example, rewrite the TensorFlow2.0 intelligent model into an intelligent model under the PyTorch1.3 version framework.

[0064] 2. Add operators. To solve the problem that the "rewrite model" step changes the internal logic of the original model, you need to add operators to the model file of the corresponding version of the migration tool. That is, if you use method A to rewrite the model in STEP 1, you need to "add operators" to the replaced operators.

[0065] The above-mentioned operator addition features include: writing operator declarations in the model file of the corresponding version of the migration tool, adding operator CPU implementation, adding operator encapsulation, adding operator kernels, adding activation functions, and recompiling the model using the migration tool. Through these six steps, the whole process of adding operators is realized.

[0066] Declaring an operator: refers to declaring an operator in the transplantation tool chain code provided by the domestic smart chip.

[0067] Adding CPU implementation of operators: refers to adding CPU implementation of operators in the transplantation tool chain code provided by domestic smart chips.

[0068] Add operator encapsulation: Each operator corresponds to an operator wrapper. According to the encapsulation header file generated by the transplantation tool chain template provided by the domestic smart chip, add the implementation of the operator wrapper.

[0069] Adding operator kernels: The specific implementation of operators is mainly completed by calling the CNML (Cambrian Machine Learning Programming) library in the transplantation tool chain provided by domestic smart chips. Adding operator kernels is completed by calling the CNML library interface according to the programming logic.

[0070] Add activation function: If the framework version supported by the migration tool provided by the domestic smart chip does not have the activation function used by the migrated smart model, you need to add the activation function manually.

[0071] Recompile the model using the transplantation tool: Recompile the intelligent model using the transplantation tool chain provided by the domestic intelligent chip, and enter the Python environment to confirm whether the integration is successful.

[0072] 3. Model quantization: Converting a floating-point model (e.g. float32) to an integer model (e.g. int8) can ensure that the calculation accuracy is within the target error range, significantly reduce the storage space and bandwidth occupied by the model, and accelerate reasoning.

[0073] The quantitative features of the above model include:

[0074] The numerical range of each parameter input in the statistical model file is obtained, and the maximum value of its absolute value is obtained, which is recorded as absmax; the scaling scale of the input floating point number mapped to the integer is calculated as scale1 = the maximum value of the integer data type / absmax, for example, the maximum value of the integer int8 data type is 127, thereby calculating the scaling scale of the input data;

[0075] The numerical range of each weight in the statistical model file is obtained, and the maximum absolute value thereof is obtained, which is recorded as weightmax; the scaling scale of the weight mapped to the integer is calculated as scale2 = the maximum value of the integer data type / weightmax, thereby calculating the scaling scale of the weight (parameters of neuron connections in the intelligent model);

[0076] Taking the convolution operator as an example of the process of running the quantization model, the floating-point input of the convolution operator is first quantized, and the scale value in the quantization model is used to calculate the integer input according to the quantization formula. Similarly, the weight of the convolution operator is quantized to obtain the integer weight. Then, an integer convolution operation is performed to output the integer convolution result. According to the inverse quantization formula, the floating-point convolution output is finally obtained.

[0077] 4. Generate an offline model, that is, generate an intelligent model that can run in a terminal based on domestic intelligent chips.

[0078] The above-mentioned generation of offline model features includes four steps: based on the model file after model quantization, using the migration tool to build a static graph of the intelligent model, optimizing the static graph, performing device-based optimization based on the device type of the input data, and generating an offline model. Through these four steps, the whole process of generating an offline model is realized. Among them:

[0079] Build a static graph of the model: Use the transplantation tool chain provided by the domestic smart chip to call the jit.trace() interface to generate a static graph. The layer-by-layer mode (layer-by-layer refers to multiple network layers of the neural network model) will be run on the entire network, and a static graph will be built at the same time.

[0080] The specific optimization of the static graph is to remove redundant operators, merge small operators, reuse data blocks, etc., to obtain an optimized static graph.

[0081] The device-based optimization based on the device type of the input data is specifically: generating an instruction for the current device according to the device type of the input data.

[0082] 5. Compare the transplantation effects. In order to ensure the accuracy of model transplantation, it is necessary to compare the transplantation effects.

[0083] The above-mentioned comparative transplantation effect characteristics include: the original intelligent model and the offline model load the same data (calculation example) respectively, and the GPU-trained model (original intelligent model) and the generated offline model are used to run the results at the same time, and the running results of the two models are compared to determine whether the running results are within the accuracy error range.

[0084] The implementation process of the present invention is described in detail below in conjunction with the accompanying drawings.

[0085] Figure 1 The flowchart of the method of the present invention is a method for transplanting an intelligent model based on a domestic intelligent chip, including five main steps: rewriting the model, adding operators, quantizing the model, generating an offline model, and comparing the transplantation effect. The first two steps are the preparation stage of the intelligent model transplantation, the middle two steps are the operation stage of the intelligent model transplantation, and the last step is the evaluation stage to ensure the correct transplantation. Through these five steps, a complete closed loop of transplanting the intelligent algorithm model from the processor used for training to the processor used for inference operation is realized.

[0086] Figure 2 The flowchart of the model rewriting process of the present invention. First, it is necessary to determine whether the framework version used by the transplanted intelligent model is much higher than the framework version supported by the transplantation tool. If it is much higher than the framework version supported by the transplantation tool, the transplanted algorithm needs to be rewritten and retrained using the framework version corresponding to the transplantation tool; otherwise, the model is first decompressed and the model is saved again using the parameters of the corresponding version of the transplantation tool; the operators used by the transplanted intelligent model are compared to the operators used by the transplantation tool to see if they are consistent. If they are consistent, the model is reloaded; otherwise, the corresponding operators of the transplanted intelligent model are replaced with the operators used by the transplantation tool and the model is reloaded, and finally a model file of the corresponding version of the transplantation tool is formed.

[0087] Figure 3 This is a flow chart of the process of adding operators of the present invention. Adding operators is not a necessary process for intelligent model transplantation, but is performed on the basis of judging whether the previous step "rewriting the model" has changed the internal logic of the original model. If the process of rewriting the model does not change the internal logic of the original model, then continue with subsequent steps such as model quantization. The process of adding operators includes six steps: declaring operators, adding CPU implementation of operators, adding operator encapsulation, adding declaration and implementation of operator functions, adding activation functions, and recompiling the model using transplantation tools. Through these six steps, the whole process of adding operators is realized.

[0088] Figure 4 This is a flow chart of the model quantization process of the present invention. Model quantization is a key link in intelligent model transplantation, and is the process of mapping from floating-point values ​​to integer values. Take the mapping of floating-point numbers to int8 as an example: first, count the numerical range of the input to obtain the maximum absolute value, recorded as absmax; then calculate the scaling scale of the input floating-point mapping, scale = 127 / absmax; similarly, calculate the scaling scale of the weight; save the scaling scale of the input and weight; then, quantize the floating-point input of the operator and quantize the weight of the operator; then, perform an integer convolution operation to obtain an integer convolution result; finally, obtain the floating-point convolution output according to the inverse quantization formula.

[0089] Figure 5 The flowchart of the process of generating an offline model of the present invention includes four steps: using a migration tool to construct a static graph of an intelligent model, optimizing the static graph, optimizing the device type based on input data, and generating an offline model. Through these four steps, the whole process of generating an offline model is realized.

[0090] Figure 6 This is a flowchart of the present invention for comparing transplantation effects. After the offline model is generated, the transplantation of the intelligent model has been completed, but in order to ensure the accuracy of the model transplantation, it is necessary to compare the transplantation effects. The specific process is: load the same data, run the model trained using GPU and the generated offline model at the same time, and compare the running results of the two models. If the accuracy difference of the running results is within 1% or the pixel difference of the target box is within 10 pixels, the model transplantation is successful, otherwise it is necessary to find the problem.

[0091] The contents not described in detail in the specification of the present invention belong to the prior art known to the professional and technical personnel in this field.

Claims

1. A method for transplanting an intelligent model based on a domestic intelligent chip, characterized in that include: Step 1. To solve the problem that the framework version used by the original intelligent model to be transplanted is inconsistent with the framework version supported by the transplantation tool provided by the domestic intelligent chip, the model is rewritten to obtain the model file of the corresponding version of the transplantation tool; Step 2: Determine whether the logic inside the original model has been changed when the model is rewritten. If so, add operators to the model file of the corresponding version of the migration tool to obtain the model file with the added operators and proceed to the next step. If there is no change, go directly to the next step; Step 3. Perform model quantization on the model file of the corresponding version of the migration tool or the model file with the operator added to obtain the quantized model file; Step 4. Generate an offline model based on the model file after model quantization; Step 5: Load the same example into the original intelligent model and the offline model respectively, and compare the results of the two models. If the difference in the results meets the requirements, the migration is terminated. If it does not meet the requirements, find the migration problems and return to Step 1 to start again.

2. According to claim 1, a smart model transplantation method based on domestic smart chips is characterized in that: The process of rewriting the model is: Determine the difference range between the framework version used by the transplanted original intelligent model and the framework version supported by the transplantation tool. If the difference between the framework version used by the transplanted original intelligent model and the framework version supported by the transplantation tool is within an acceptable difference range, decompress the original intelligent model, save the original intelligent model again using the parameters of the corresponding version of the transplantation tool, replace the corresponding operators of the transplanted original intelligent model with the operators used by the transplantation tool, reload the model, and form a model file of the corresponding version of the transplantation tool. If the difference between the framework version used by the original intelligent model being transplanted and the framework version supported by the transplantation tool is not within the acceptable range, the transplanted algorithm needs to be rewritten and retrained using the framework version corresponding to the transplantation tool; Reload the model after training to generate the model file corresponding to the version of the migration tool.

3. According to claim 1, a smart model transplantation method based on domestic smart chips is characterized in that: Add operators to the model file of the corresponding version of the migration tool, specifically: In the model file of the corresponding version of the transplantation tool, the following six steps are added: declaring the operator, adding the CPU implementation of the operator, adding the operator encapsulation, adding the declaration and implementation of the operator function, adding the activation function, and recompiling the model using the transplantation tool.

4. According to claim 3, a method for transplanting an intelligent model based on a domestic intelligent chip is characterized in that: Declare operator: refers to declaring the operator in the transplant tool chain code provided by the domestic smart chip; Adding CPU implementation of operators: refers to adding CPU implementation of operators in the transplantation tool chain code provided by domestic smart chips; Add operator package: Generate the package header file based on the transplant tool chain template provided by the domestic smart chip. Each operator corresponds to an operator package. Add operator kernel: call the machine learning programming library interface in the transplantation tool chain provided by domestic smart chips according to the programming logic; Add activation function: If the framework version supported by the transplantation tool provided by the domestic smart chip does not have the activation function used by the transplanted smart model, manually add the activation function; Recompile the model using the transplantation tool: Recompile the intelligent model using the transplantation tool chain provided by the domestic intelligent chip, and enter the coding environment to confirm whether the integration is successful.

5. According to claim 1, a smart model transplantation method based on domestic smart chips is characterized in that: The specific method for model quantization is: The numerical range of each parameter input in the statistical model file is obtained, and the maximum value of its absolute value is obtained, which is recorded as absmax; the scaling scale of the input floating point number mapped to the integer is calculated as scale1 = the maximum value of the integer data type / absmax, thereby calculating the scaling scale of the input data; The numerical range of each weight in the statistical model file is obtained, and the maximum absolute value is obtained, which is recorded as weightmax; The weight is mapped to the integer scaling scale scale2 = the maximum value of the integer data type / weightmax, thereby calculating the scaling scale of the weight; Combined with the calculated scaling scale, the integer input data and integer weights are calculated according to the quantization formula, and an integer convolution operation is performed on them to output an integer convolution result. The integer convolution result is then processed according to the dequantization formula to obtain a floating-point convolution output, completing the model quantization and obtaining the model file after the model quantization.

6. According to claim 1, a method for transplanting an intelligent model based on a domestic intelligent chip is characterized in that: When generating an offline model, use the migration tool to build a static graph of the intelligent model, optimize the static graph, and optimize the device type based on the input data to generate an intelligent model that can run in a terminal based on domestic smart chips, that is, an offline model.

7. The method for transplanting an intelligent model based on a domestic intelligent chip according to claim 6, characterized in that: When optimizing a static graph, an optimized static graph is obtained by removing redundant operators, merging small operators, and reusing data blocks.

Images