A method for porting intelligent models based on domestically produced intelligent chips

Abstract

This invention discloses a method for porting intelligent models based on domestically produced intelligent chips. The method involves using a porting tool provided by the domestically produced intelligent chip to rewrite the model, obtaining a model file corresponding to the version of the porting tool. If the internal logic of the original model has been changed, operators are added to the model file corresponding to the version of the porting tool, resulting in a model file with added operators, and proceeding to the next step. If no changes have been made, the method proceeds directly to the next step. The model file is then quantized to obtain a quantized model file. An offline model is generated based on the quantized model file. The original intelligent model and the offline model are loaded with the same computational example, and the results of the two models are compared. If the difference in the results meets the requirements, the porting process ends; otherwise, the porting problem is identified and the process restarts. This invention solves the compatibility problem of intelligent models without affecting the operating efficiency, response speed, and accuracy of the artificial intelligence model.

Description

Technical Field

[0001] This invention relates to a method for porting intelligent models based on domestically produced intelligent chips, belonging to the fields of artificial intelligence and computer science. Background Technology

[0002] Porting intelligent models primarily focuses on how to migrate trained artificial intelligence models (such as deep learning models, reinforcement learning models, etc.) from one platform or environment to another while maintaining their performance. Typically, AI models are trained on platforms or environments with high computing power. However, these training processors are power-hungry, expensive, and bulky, making them unsuitable for edge applications. Therefore, after training, AI models are generally ported to processors with lower power consumption and smaller size. Current technologies generally involve lightweighting the AI ​​algorithm model through processes like pruning before porting it to the edge processor; this process is complex and time-consuming.

[0003] There are currently no reports, either domestically or internationally, of methods for porting artificial intelligence models to domestically produced smart chips.

[0004] The paper, "Design of a Soft PLC Controller for Easy Portability of Intelligent Algorithms," designs an embedded soft PLC controller. It primarily targets intelligent control algorithms modeled and simulated using Simulink. The controller generates a structured text language conforming to the IEC 61131-3 standard using a tool, and then automatically loads it into the PLC development software. This facilitates the convenient porting of the intelligent algorithm to the designed soft PLC controller for testing or engineering applications. The intelligent control algorithm discussed in this paper does not belong to artificial intelligence algorithms trained using neural networks, deep learning, or reinforcement learning; rather, it represents a specialized, customized controller.

[0005] The paper "Research on FPGA Implementation Technology of Hybrid Frog Leaping Algorithm" introduces a scheme for porting a heuristic population evolution algorithm—the Hybrid Frog Leaping Algorithm (SFLA)—to FPGA. The SFLA algorithm used in this paper is not an artificial intelligence algorithm, and FPGA, with its powerful parallel processing capabilities, is used in fields such as digital signal processing, not as a dedicated artificial intelligence processor.

[0006] The paper, "Research and Implementation of an Intelligent Waste Management System Based on NB-IoT," designs a waste disposal management scheme based on embedded artificial intelligence for the waste disposal process. This scheme ports an AI model to a smart terminal. The paper describes how the designed model is first trained on a computer, and then ported to the smart terminal. The smart terminal used in the paper has 64Kbytes of SRAM and uses an industrial-grade Cortex-M0+ MKL36Z64VLH4 chip manufactured by NXP Semiconductors in the Netherlands. The waste sorting model used in the paper is based on LeNet-5, a lightweight convolutional neural network; however, the specific porting process and steps are not described.

[0007] The paper "Research on Lightweight YOLOv5 Algorithm for Pine Tree Count Recognition" adapts the YOLOv5 network to subsequent edge computing terminals by lightweighting the network. It compares the effects of the algorithm before and after lightweighting, but does not involve the porting of the algorithm model.

[0008] The paper "Research and Implementation of Lower Limb Multimodal Signal Acquisition and Recognition System" transplants the trained CNN network into the main controller through Cube.AI. After transplantation, the compression algorithm uses resources and is transplanted using STMicroelectronics' STM32Cube tool, running on the STM32F746ZGT6 (manufactured by STMicroelectronics) microcontroller. Summary of the Invention

[0009] The technical problem solved by this invention is to overcome the shortcomings of the prior art and provide a method for porting intelligent models based on domestically produced intelligent chips, thereby solving the compatibility problem of intelligent models, ensuring that the original model can run normally in the environment of domestically produced intelligent chips, and at the same time not affecting the running efficiency, response speed and accuracy of the artificial intelligence model.

[0010] The technical solution of this invention is: a method for porting intelligent models based on domestically produced intelligent chips, comprising:

[0011] Step 1: To address the inconsistency between the framework version used by the original intelligent model being ported and the framework version supported by the porting tool provided by the domestic intelligent chip, the model is rewritten to obtain the model file corresponding to the version of the porting tool.

[0012] Step 2: Determine whether the internal logic of the original model has been changed during the model rewriting. If it has, add operators to the model file of the corresponding version of the porting tool to obtain the model file with added operators and proceed to the next step; if it has not been changed, proceed directly to the next step.

[0013] Step 3: Perform model quantization on the model file corresponding to the version of the porting tool or the model file with added operators to obtain the quantized model file;

[0014] Step 4: Generate an offline model based on the quantized model file;

[0015] Step 5: Load the same computational example into the original intelligent model and the offline model respectively, and compare the results of the two models. If the difference in the results meets the requirements, the migration ends. If the requirements are not met, find the migration problem and return to Step 1 to start again.

[0016] Preferably, the process of rewriting the model is as follows:

[0017] Determine the range of differences between the framework version used by the original intelligent model being ported and the framework version supported by the porting tool. If the difference between the framework version used by the original intelligent model being ported and the framework version supported by the porting tool is within an acceptable range, then decompress the original intelligent model, save the original intelligent model again using the parameters of the corresponding version of the porting tool, replace the corresponding operators of the original intelligent model being ported with the operators used by the porting tool, reload the model, and form a model file corresponding to the version of the porting tool.

[0018] If the difference between the framework version used by the original intelligent model being ported and the framework version supported by the porting tool is not within an acceptable range, the algorithm being ported needs to be rewritten using the framework version corresponding to the porting tool and retrained; after training, the model should be reloaded to form a model file corresponding to the version of the porting tool.

[0019] Preferably, operators are added to the model files corresponding to the version of the porting tool, specifically as follows:

[0020] The following six steps are added to the model file corresponding to the porting tool version: declaring operators, adding CPU implementations of operators, adding operator wrappers, adding declarations and implementations of operator functions, adding activation functions, and recompiling the model using the porting tool.

[0021] Preferably, declaring operators refers to declaring operators in the porting toolchain code provided by the domestic smart chip;

[0022] CPU implementation with added operators: refers to the CPU implementation with added operators in the porting toolchain code provided by domestic smart chips;

[0023] Add operator encapsulation: Based on the encapsulation header file generated by the porting toolchain template provided by the domestic smart chip, each operator corresponds to one operator encapsulation;

[0024] Add operator kernels: Call the machine learning programming library interface in the porting toolchain provided by the domestic smart chip according to the programming logic;

[0025] Add activation function: If the framework version supported by the porting tool provided by the domestic smart chip does not contain the activation function used by the ported smart model, then add the activation function manually;

[0026] Recompile the model using the porting tool: Recompile the intelligent model using the porting toolchain provided by the domestic intelligent chip, and enter the coding environment to confirm whether the integration was successful.

[0027] The preferred method for model quantization is as follows:

[0028] The numerical range of each parameter input in the statistical model file is obtained, and the maximum absolute value is denoted as absmax; the scaling factor scale1, which maps the input floating-point number to the integer type, is calculated as scale1 = maximum value of the integer data type / absmax, thus calculating the scaling factor of the input data;

[0029] The numerical range of each weight in the statistical model file is determined, and the maximum absolute value is obtained, denoted as weightmax. The scaling factor for mapping the weight to an integer is calculated as scale2 = maximum value of the integer data type / weightmax, thus calculating the scaling factor of the weight.

[0030] Based on the calculated scaling scale, integer input data and integer weights are calculated according to the quantization formula. Integer convolution operation is then performed on the integer input data to output the integer convolution result. The integer convolution result is then processed according to the dequantization formula to obtain the floating-point convolution output, thus completing the model quantization and obtaining the quantized model file.

[0031] Preferably, when generating the offline model, a static graph of the intelligent model is constructed using a porting tool, the static graph is optimized, and optimization is performed based on the device type of the input data to generate an intelligent model that can run on a terminal based on a domestically produced intelligent chip, i.e., an offline model.

[0032] Preferably, when optimizing a static graph, an optimized static graph is obtained by removing redundant operators, merging small operators, and reusing data blocks.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] (1) This invention provides a process and porting steps for migrating artificial intelligence models trained using high-performance GPUs to NPU processors based on domestic Siyuan 270 and Siyuan 220 chips. It designs a general porting method for intelligent algorithm models that can be applied to domestic Siyuan 270 and Siyuan 220 chips, realizes the rapid, efficient and accurate porting of convolutional neural network intelligent algorithm models, and improves the efficiency of intelligent algorithm models from training to terminal application.

[0035] (2) The intelligent model porting method designed in this invention solves the model compatibility problem and ensures that the artificial intelligence model trained with imported high-performance GPUs can run normally in the environment of domestic intelligent chips;

[0036] (3) The intelligent algorithm model porting method designed in this invention effectively solves the problems of intelligent algorithm model optimization and accuracy maintenance, so that the intelligent algorithm model trained by high-performance GPU can still run on domestic intelligent chips with reduced computing power without affecting the running efficiency and response speed of the artificial intelligence model, and can maintain the accuracy of the model operation. Attached Figure Description

[0037] Figure 1 This is a flowchart illustrating the overall method of the present invention;

[0038] Figure 2 This is a flowchart of the rewriting process of the model of the present invention;

[0039] Figure 3 This is a flowchart of the operator addition process of the present invention;

[0040] Figure 4 This is a flowchart of the model quantization process of the present invention;

[0041] Figure 5 This is a flowchart of the offline model generation process of the present invention;

[0042] Figure 6 This is a flowchart illustrating the comparative transplantation effects of the present invention. Detailed Implementation

[0043] The present invention has been improved in the following aspects:

[0044] (1) Solve the compatibility problem of intelligent models and ensure that the original model can run normally in the environment of domestic intelligent chips.

[0045] (2) Solve the problem of intelligent model performance optimization, and ensure that the performance of the model is optimized without affecting the running efficiency and response speed of the artificial intelligence model when the hardware computing power and performance are lower than those of the high-performance GPU used for training.

[0046] (3) Solve the problem of accuracy preservation, and ensure that when the hardware computing power and performance are lower than those of the training GPU, measures are taken to minimize the loss of accuracy of the artificial intelligence model or to keep its accuracy unaffected.

[0047] The intelligent models mentioned in this article refer to intelligent algorithms and intelligent models trained through artificial intelligence technology.

[0048] To enable the artificial intelligence model to run on domestically produced smart chips, the following technical solution was adopted:

[0049] (1) By adjusting the input and output interfaces and data formats of the model, we can ensure that the artificial intelligence model trained on the high-performance GPU can run normally on the processor of the domestic smart chip, thus forming a method for porting smart models based on domestic smart chips that can adapt to more than two types of domestic chips.

[0050] (2) By summarizing the porting process of artificial intelligence models with different characteristics, a smart algorithm model porting process is proposed, which effectively solves the problem of quickly upgrading the algorithm model from the simulation state to the embedded running algorithm model.

[0051] (3) By transplanting intelligent algorithm models between training GPUs and domestically produced intelligent chip terminal devices, computing resources and response time can be balanced, the efficiency and performance of the overall system can be improved, advanced artificial intelligence algorithms and models can be applied to various practical scenarios, and the intelligent upgrading and development of various fields can be promoted.

[0052] The process of porting the intelligent model is as follows:

[0053] (1) Rewrite the model. (2) Add operators. (3) Quantize the model. (4) Generate an offline model. (5) Compare the porting results.

[0054] The technical solution of this invention is: a method for porting intelligent models based on domestically produced intelligent chips, comprising five parts: rewriting the model, adding operators, model quantization, generating an offline model, and comparing the porting effect.

[0055] Specifically:

[0056] Step 1: To address the inconsistency between the framework version used by the original intelligent model being ported and the framework version supported by the porting tool provided by the domestic intelligent chip, the model is rewritten to obtain the model file corresponding to the version of the porting tool. The input of this step is the original intelligent model being ported, and the output of this step is the intelligent model under the framework version supported by the porting tool.

[0057] Step 2: Determine whether the internal logic of the original model has been changed during the model rewriting. If it has, add operators to the model file of the corresponding version of the porting tool, and proceed to the next step after obtaining the model file with added operators. If it has not been changed, proceed directly to the next step. The input of this step is the intelligent model under the framework version supported by the porting tool, and the output of this step is the intelligent model after adding operators.

[0058] Step 3: Perform model quantization on the model file with added operators to obtain the quantized model file. The input of this step is the intelligent model with added operators, and the output of this step is the quantized intelligent model.

[0059] Step 4: Generate an offline model based on the quantized model file; the input of this step is the quantized intelligent model, and the output of this step is the generated offline model.

[0060] Step 5: Input the same computational example into both the original intelligent model and the offline model, and compare the results of the two models. If the difference in the results meets the requirements, the migration ends; otherwise, find the migration problem and return to Step 1 to start again. The inputs for this step are the original intelligent model and the offline model generated in the previous step, and the output of this step is the comparison result of the two input models.

[0061] 1. Model rewriting: To address the inconsistency between the framework version used by the ported intelligent model and the framework version supported by the porting tool provided by the domestic intelligent chip, the model is rewritten. The purpose of model rewriting is to enable the intelligent model to run on the framework supported by the porting tool.

[0062] The aforementioned rewritten model features include: determining whether the framework version used by the original intelligent model being ported is significantly higher than the framework version supported by the porting tool. If the difference between the framework version used by the ported intelligent model and the framework version supported by the porting tool is within an acceptable range, the original intelligent model is decompressed, saved again using the parameters corresponding to the version of the porting tool, the corresponding operators of the ported original intelligent model are replaced with the operators used by the porting tool, and the model is reloaded to form a model file corresponding to the version of the porting tool. If the framework version used by the ported intelligent model is significantly higher than the framework version supported by the porting tool, the ported algorithm needs to be rewritten using the framework version corresponding to the porting tool and retrained; after training, the model is reloaded to form a model file corresponding to the version of the porting tool.

[0063] Currently, the highest TensorFlow framework version supported by the porting tool is 1.15.4, and the highest PyTorch 1.3 framework version is 1.3. If the original intelligent model being ported uses a framework version higher than this, such as TensorFlow 2.0 or PyTorch 1.7, it needs to be rewritten. There are two ways to rewrite it: Method A: Replace the unsupported operators in the model with operators supported by the domestic intelligent chip porting toolchain; Method B: Change the framework version, for example, rewrite the intelligent model under the TensorFlow 2.0 framework to an intelligent model under the PyTorch 1.3 framework.

[0064] 2. Adding operators: To address the issue of the "rewriting the model" step altering the internal logic of the original model, operators need to be added to the model file corresponding to the version of the porting tool. Specifically, if method A is used to rewrite the model in STEP1, the replaced operators need to be processed by "adding operators".

[0065] The aforementioned process of adding operator features includes six steps: adding operator declarations to the model file corresponding to the porting tool version, adding the CPU implementation of the operator, adding operator encapsulation, adding operator kernels, adding activation functions, and recompiling the model using the porting tool. These six steps complete the entire process of adding operators.

[0066] Declaring operators: refers to declaring operators in the porting toolchain code provided by domestic smart chips.

[0067] CPU implementation with added operators: refers to the CPU implementation with added operators in the porting toolchain code provided by domestic smart chips.

[0068] Add operator wrappers: Each operator corresponds to an operator wrapper. Based on the wrapper header file generated by the porting toolchain template provided by the domestic smart chip, add the implementation of the operator wrapper.

[0069] Adding operator kernels: The specific implementation of operators is mainly accomplished by calling the CNML (Cambricon Machine Learning Programming) library in the porting toolchain provided by domestic intelligent chips. Adding operator kernels is simply a matter of calling the CNML library interface according to the programming logic.

[0070] Add activation functions: If the framework version supported by the porting tool provided by the domestic smart chip does not contain the activation functions used by the ported smart model, then the activation functions need to be added manually.

[0071] Recompile the model using the porting tool: Recompile the intelligent model using the porting toolchain provided by the domestic intelligent chip, and enter the Python environment to confirm whether the integration was successful.

[0072] 3. Model quantization: Converting floating-point (e.g., float32) models into integer (e.g., int8) models can significantly reduce the storage space and bandwidth occupied by the model while ensuring that the calculation accuracy is within the target error range, thus accelerating inference.

[0073] The quantitative features of the above model include:

[0074] The numerical range of each parameter input in the statistical model file is obtained, and the maximum absolute value is denoted as absmax; the scaling factor scale1 for mapping the input floating-point number to the integer type is calculated as scale1 = maximum value of the integer data type / absmax. For example, the maximum value of the integer int8 data type is 127, thus calculating the scaling factor of the input data.

[0075] The numerical range of each weight in the statistical model file is determined, and the maximum absolute value is obtained, denoted as weightmax. The scaling scale of the weights mapped to integers is calculated as scale2 = maximum value of integer data type / weightmax, thus calculating the scaling scale of the weights (parameters of neuron connections in the intelligent model).

[0076] Taking the convolution operator as an example, the process of running the quantization model first quantizes the floating-point input of the convolution operator. Using the scale value in the quantization model, an integer input is calculated according to the quantization formula. Similarly, the weights of the convolution operator are quantized to obtain integer weights. Then, an integer convolution operation is performed to output the integer convolution result. According to the dequantization formula, the floating-point convolution output is finally obtained.

[0077] 4. Generate offline models, that is, generate intelligent models that can run on terminals based on domestically produced intelligent chips.

[0078] The aforementioned offline model generation features include four steps: based on the quantized model file, constructing a static graph of the intelligent model using a porting tool, optimizing the static graph, performing device-based optimization based on the device type of the input data, and generating the offline model. These four steps complete the entire process of generating an offline model. Specifically:

[0079] Building a static graph of the model: Using the porting toolchain provided by domestic smart chips, the jit.trace() interface is called to generate a static graph. This will run the layer-by-layer mode (layer-by-layer refers to multiple network layers of the neural network model) on the entire network and build a static graph at the same time.

[0080] The optimization of a static graph involves removing redundant operators, merging small operators, and reusing data blocks to obtain an optimized static graph.

[0081] Device-based optimization based on the device type of the input data specifically involves generating instructions for the current device based on the device type of the input data.

[0082] 5. Compare the transplantation results. In order to ensure the accuracy of model transplantation, it is necessary to compare the transplantation results.

[0083] The above-mentioned characteristics of the comparative porting effect include: loading the same data (example) into the original intelligent model and the offline model respectively, running the model trained by GPU (original intelligent model) and the generated offline model at the same time to obtain the results, comparing the results of the two models, and determining whether the results are within the accuracy error range.

[0084] The implementation process of the present invention will now be described in detail with reference to the accompanying drawings.

[0085] Figure 1 This is a flowchart of the method of the present invention. A method for porting an intelligent model based on a domestically produced intelligent chip includes five main steps: rewriting the model, adding operators, model quantization, generating an offline model, and comparing the porting effect. The first two steps are the preparation stage for intelligent model porting, the middle two steps are the operation stage for intelligent model porting, and the last step is the evaluation stage to ensure the correctness of the porting. Through these five steps, a complete closed loop is achieved for porting the intelligent algorithm model from the processor used for training to the processor used for inference execution.

[0086] Figure 2 The flowchart below illustrates the rewriting process of the model in this invention. First, it's necessary to determine if the framework version used by the transplanted intelligent model is significantly higher than the framework version supported by the transplanting tool. If it is, the transplanted algorithm needs to be rewritten using the framework version corresponding to the transplanting tool and retrained. Otherwise, the model is first decompressed, and then saved again using the parameters corresponding to the version of the transplanting tool. Next, the operators used by the transplanted intelligent model and those used by the transplanting tool are compared. If they are consistent, the model is reloaded; otherwise, the corresponding operators of the transplanted intelligent model are replaced with those used by the transplanting tool, and the model is reloaded. Finally, a model file corresponding to the version of the transplanting tool is generated.

[0087] Figure 3 This is a flowchart illustrating the operator addition process of this invention. Adding operators is not a mandatory step in intelligent model porting; rather, it is performed after determining that the previous step, "model rewriting," altered the internal logic of the original model. If the model rewriting process did not change the internal logic of the original model, subsequent steps such as model quantization continue. The operator addition process includes six steps: operator declaration, CPU implementation of the added operator, operator encapsulation, declaration and implementation of the added operator function, addition of the activation function, and recompiling the model using porting tools. These six steps complete the entire operator addition process.

[0088] Figure 4 This is a flowchart of the model quantization process of the present invention. Model quantization is a key step in the porting of intelligent models, which is the process of mapping floating-point values ​​to integer values. Taking the mapping of floating-point numbers to int8 as an example: First, the range of input values ​​is statistically analyzed to obtain the maximum absolute value, denoted as absmax; then, the scaling scale for the input floating-point mapping is calculated as scale = 127 / absmax; similarly, the scaling scale for the weights is calculated; the scaling scales for the input and weights are saved; then, the floating-point input of the operator is quantized, and the weights of the operator are quantized; next, an integer convolution operation is performed to obtain the integer convolution result; finally, the floating-point convolution output is obtained according to the dequantization formula.

[0089] Figure 5 This is a flowchart of the offline model generation process of the present invention. The offline model generation process includes four steps: constructing a static graph of the intelligent model using a porting tool, optimizing the static graph, optimizing based on the device type of the input data, and generating the offline model. These four steps complete the entire process of generating the offline model.

[0090] Figure 6 This is a flowchart comparing the transplantation effects of the present invention. After generating the offline model, the transplantation of the intelligent model has been completed. However, to ensure the accuracy of the model transplantation, a comparison of the transplantation effects is necessary. The specific process is as follows: Load the same data, run the model trained using the GPU and the generated offline model simultaneously, and compare the results of the two models. If the difference in accuracy of the running results is within 1% or the difference in the pixel size of the target box is within 10 pixels, then the model transplantation is successful; otherwise, the problem needs to be investigated.

[0091] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

Claims

1. A method for porting intelligent models based on domestically produced intelligent chips, characterized in that... include: Step 1: To address the inconsistency between the framework version used by the original intelligent model being ported and the framework version supported by the porting tool provided by the domestic intelligent chip, the model is rewritten to obtain the model file corresponding to the version of the porting tool. Step 2: Determine whether the internal logic of the original model has been changed during the model rewriting. If it has been changed, add operators to the model file corresponding to the version of the porting tool to obtain the model file with added operators and proceed to the next step. If there are no changes, proceed directly to the next step; Step 3: Perform model quantization on the model file corresponding to the version of the porting tool or the model file with added operators to obtain the quantized model file; Step 4: Generate an offline model based on the quantized model file; Step 5: Load the same computational example into the original intelligent model and the offline model respectively, and compare the results of the two models. If the difference in the results meets the requirements, the migration ends. If the requirements are not met, find the migration problem and return to Step 1 to start again. Add operators to the model files corresponding to the version of the porting tool, specifically: The following six steps are added to the model file corresponding to the porting tool version: declaring operators, adding CPU implementations of operators, adding operator wrappers, adding declarations and implementations of operator functions, adding activation functions, and recompiling the model using the porting tool. Declaring operators: refers to declaring operators in the porting toolchain code provided by domestic smart chips; CPU implementation with added operators: refers to the CPU implementation with added operators in the porting toolchain code provided by domestic smart chips; Add operator encapsulation: Based on the encapsulation header file generated by the porting toolchain template provided by the domestic smart chip, each operator corresponds to one operator encapsulation; Add operator kernels: Call the machine learning programming library interface in the porting toolchain provided by the domestic smart chip according to the programming logic; Add activation function: If the framework version supported by the porting tool provided by the domestic smart chip does not contain the activation function used by the ported smart model, then add the activation function manually. Recompile the model using the porting tool: Recompile the intelligent model using the porting toolchain provided by the domestic intelligent chip, and enter the coding environment to confirm whether the integration was successful.

2. The method for porting an intelligent model based on a domestically produced intelligent chip according to claim 1, characterized in that: The process of rewriting the model is as follows: Determine the range of differences between the framework version used by the original intelligent model being ported and the framework version supported by the porting tool. If the difference between the framework version used by the original intelligent model being ported and the framework version supported by the porting tool is within an acceptable range, then decompress the original intelligent model, save the original intelligent model again using the parameters of the corresponding version of the porting tool, replace the corresponding operators of the original intelligent model being ported with the operators used by the porting tool, reload the model, and form a model file corresponding to the version of the porting tool. If the difference between the framework version used by the original intelligent model being ported and the framework version supported by the porting tool is not within an acceptable range, then the algorithm being ported needs to be rewritten and retrained using the framework version corresponding to the porting tool. After training, the model is reloaded to generate a model file corresponding to the version of the porting tool.

3. The method for porting an intelligent model based on a domestically produced intelligent chip according to claim 1, characterized in that: The specific method for model quantization is as follows: The numerical range of each parameter input in the statistical model file is calculated, and the maximum absolute value is obtained, denoted as absmax; the scaling factor scale1, which maps the input floating-point number to the integer type, is calculated as scale1 = maximum value of the integer data type / absmax, thus calculating the scaling factor of the input data; The numerical range of each weight in the statistical model file is determined, and the maximum absolute value of each weight is denoted as weightmax. The scaling factor for mapping the weights to integers is calculated as scale2 = the maximum value of the integer data type / weightmax, thus calculating the scaling factor for the weights. Based on the calculated scaling scale, integer input data and integer weights are calculated according to the quantization formula. Integer convolution operation is then performed on the integer input data to output the integer convolution result. The integer convolution result is then processed according to the dequantization formula to obtain the floating-point convolution output, thus completing the model quantization and obtaining the quantized model file.

4. The method for porting an intelligent model based on a domestically produced intelligent chip according to claim 1, characterized in that: When generating the offline model, a static graph of the intelligent model is constructed using a porting tool. The static graph is then optimized based on the device type of the input data to generate an intelligent model that can run on a terminal based on a domestically produced intelligent chip, i.e., an offline model.

5. The method for porting an intelligent model based on a domestically produced intelligent chip according to claim 4, characterized in that: When optimizing a static graph, an optimized static graph is obtained by removing redundant operators, merging small operators, and reusing data blocks.

Images