Automatic AI algorithm model deployment method for heterogeneous hardware platform

Abstract

The invention provides a heterogeneous hardware platform-oriented automatic AI algorithm model deployment method. The method comprises the following steps of: obtaining a to-be-deployed AI algorithm model and a heterogeneous hardware platform; creating an AI algorithm model deployment task, and setting model adaptation task parameters; constructing a model lightweight algorithm library and a rule constraint engine, and establishing an exception handling mechanism; performing model lightweight and unified representation based on a rule constraint engine and an exception handling mechanism to form a recommended AI algorithm model; matching the recommended AI algorithm model with a heterogeneous hardware platform; and performing performance evaluation based on a heterogeneous hardware platform test environment. According to the automatic AI algorithm model deployment method oriented to the heterogeneous hardware platform, an AI algorithm automatic deployment process is designed and a rule constraint engine and an exception handling mechanism are constructed based on quantization, pruning, distillation and other model optimization algorithms, automatic model efficient deployment of the model is realized, and meanwhile, the performance of the deployed model is prevented from being greatly reduced.

Description

Technical Field

[0001] This application relates to the field of AI algorithm models and deployment, and in particular to an automated AI algorithm model deployment method for heterogeneous hardware platforms. Background Technology

[0002] AI algorithm model deployment is a crucial step in applying trained machine learning models to real-world production environments, involving technologies such as model lightweighting, hardware adaptation, performance acceleration, and system integration. In recent years, research focuses and development paths in the field of AI algorithm models have differed domestically and internationally, but both aim to improve model inference efficiency, reduce computational costs, and enhance applicability. Domestic research on AI algorithm model deployment technology mainly focuses on model lightweighting, while the industry focuses on tackling key technical challenges related to the industrialization of deep learning frameworks. Internationally, the emphasis is on developing standardized cross-platform and cross-framework tools, with the ONNX ecosystem achieving cross-framework compatibility of model formats. NVIDIA's TensorRT significantly improves GPU inference performance through layer fusion and dynamic tensor optimization, while Google's TensorFlow Serving provides a highly available solution for enterprise-level model deployment. Domestic AI algorithm model technology excels in industry application and domestic substitution, while international advancements lie in basic tools and cutting-edge research, laying the technological foundation for the development and engineering implementation of algorithm deployment technologies.

[0003] In the existing technology, since the deployment of AI algorithm models by various manufacturers mainly focuses on the research of deployment toolchains and deployment frameworks for chips, it is difficult to establish a unified deployment process and framework. The traditional algorithm deployment method has low deployment efficiency and relies on manual labor for more than 60%. Therefore, the need to achieve automated AI algorithm model deployment is very urgent. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this application aims to provide an automated AI algorithm model deployment method for heterogeneous hardware platforms, enabling automated deployment of AI algorithm models on different hardware platforms, reducing the threshold for AI algorithm model deployment, and improving the efficiency of algorithm deployment.

[0005] To achieve the above objectives, this application provides an automated AI algorithm model deployment method for heterogeneous hardware platforms, including: Acquire the AI ​​algorithm model to be deployed and the heterogeneous hardware platform; Create an AI algorithm model deployment task and set the model to adapt to the task parameters; Build a lightweight model algorithm library, a rule constraint engine, and establish an exception handling mechanism; The model is lightweighted and uniformly represented based on a rule constraint engine and an exception handling mechanism, forming a recommendation AI algorithm model; Adapt the recommended AI algorithm model to heterogeneous hardware platforms; Performance evaluation was conducted using a heterogeneous hardware platform testing environment.

[0006] Furthermore, the model adaptation task parameters also include: the expected speedup ratio, the tolerance for algorithm accuracy degradation, and the optimization task duration.

[0007] Furthermore, the steps of constructing a lightweight model algorithm library, a rule constraint engine, and establishing an exception handling mechanism also include: A lightweight model algorithm library is built for model pruning, model distillation, and model quantization algorithms; Design a constraint rule base, including deployment time constraints and performance degradation constraints, and build a rule constraint engine; An exception handling mechanism is established based on a unified model representation framework.

[0008] Furthermore, the step of lightweighting and unifying the model based on the rule constraint engine and anomaly handling mechanism to form a recommendation AI algorithm model also includes: Input the AI ​​algorithm model to be deployed under rule constraints and exception handling mechanisms; Based on the model lightweight algorithm library, model pruning, model distillation, and model quantization are performed sequentially on the AI ​​algorithm model to be deployed, and a recommended AI algorithm model in ONNX format is output.

[0009] Furthermore, the step of lightweighting and unifying the model based on the rule constraint engine and anomaly handling mechanism to form a recommendation AI algorithm model also includes: According to the aforementioned exception handling mechanism, the operators in the adaptation of the AI ​​algorithm model to be deployed are checked for compatibility, abnormal operators are dynamically optimized and adjusted and replaced, and the model adaptation process is automatically terminated in cases where effective handling is not possible.

[0010] Furthermore, the step of lightweighting and unifying the model based on the rule constraint engine and anomaly handling mechanism to form a recommendation AI algorithm model also includes: The model pruning algorithm is executed on the AI ​​algorithm model to be deployed. The input is a pth format model, the output is a pth format model after pruning, and intermediate performance evaluation of the model is performed. The model distillation algorithm is executed on the pruned pth format model. The pruned pth format model is input, the optimized pth format model is output, and the model performance is evaluated in intermediate tests. The AI ​​algorithm model is subjected to half-precision quantization. The optimized pth format model is input and the pth format model of FP16 is output. The model quantization part is constrained by deployment time and performance loss. A unified model representation is performed. The input is an FP16 pth format model. The exception handling mechanism is automatically executed to perform exception handling and output an ONNX format recommended AI algorithm model, preparing the model for final adaptation and deployment.

[0011] Furthermore, the step of adapting the recommended AI algorithm model to heterogeneous hardware platforms also includes: The heterogeneous hardware platform executes a model adaptation command to convert the ONNX format recommendation AI algorithm model into an OM format recommendation AI algorithm model to adapt to the heterogeneous hardware platform.

[0012] Furthermore, the step of performance evaluation based on a heterogeneous hardware platform testing environment also includes: Computational model inference speedup: , Among them, S 加速比 SP represents the inference speedup ratio of the model, SP' represents the inference efficiency before optimization, and SP' represents the inference efficiency after optimization. Calculate the decrease in model accuracy: , Among them, L 性能损失 For the decrease in model accuracy, ACC 前 The accuracy rate before optimization, ACC 后 The optimized accuracy; Computational model compression ratio: , Among them, D 压缩率 M represents the model compression ratio. 压缩前 M represents the space occupied by the model before compression. 压缩后 This represents the space occupied by the compressed model.

[0013] To achieve the above objectives, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor is configured to execute the computer program stored in the memory to implement the automated AI algorithm model deployment method for heterogeneous hardware platforms as described above.

[0014] To achieve the above objectives, this application also provides a computer-readable storage medium storing a computer program, which is loaded and executed by a processor to implement the automated AI algorithm model deployment method for heterogeneous hardware platforms as described above.

[0015] The automated AI algorithm model deployment method for heterogeneous hardware platforms provided in this application has the following advantages compared with existing technologies: An innovative set of AI algorithm model deployment rule constraint engine and anomaly handling mechanism is proposed: On the one hand, the deployment rule constraint engine solves the problems of severe performance loss and excessive optimization time in the automated process of AI algorithm model deployment, and avoids the infinite loop of algorithm deployment iteration process; on the other hand, in terms of deployment efficiency and model performance, the algorithm deployment rule constraint engine is built based on rule conditions such as time limit and performance degradation, ensuring that the time of algorithm deployment process is controllable and the performance of deployed models is controllable. On the other hand, the exception handling mechanism overcomes the deployment technology barriers for heterogeneous hardware platforms, strictly controls the model representation during the deployment phase, and abstracts and unifies the deployment process for heterogeneous hardware, achieving automated algorithm deployment with low manual intervention and low barriers to entry. In handling the adaptation issues of heterogeneous hardware platforms, the AI ​​algorithm model to be deployed is strictly constrained to a unified expression method, breaking through the deployment technology barriers between heterogeneous hardware and achieving unified and automated deployment technology engineering. This lowers the technical threshold for algorithm deployment and provides a general solution for algorithm deployment on diverse heterogeneous hardware platforms.

[0016] Other features and advantages of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing this application. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the present application and form part of the specification. Together with the embodiments of the present application, they serve to explain the present application but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of an automated AI algorithm model deployment method for heterogeneous hardware platforms according to an embodiment of this application; Figure 2 This is a flowchart of a model lightweighting and unified representation method based on a rule constraint engine and an exception handling mechanism according to an embodiment of this application. Figure 3 This is a schematic diagram of an electronic device structure according to an embodiment of this application. Detailed Implementation

[0018] The preferred embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application.

[0019] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0020] The term "comprising" and its variations as used in this application are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0021] It should be noted that the terms "first" and "second" may be used in this application only to distinguish different devices, components or parts, and are not used to define the order of functions performed by these devices, components or parts or their interdependence.

[0022] It should be noted that the terms "one" and "more" used in this application are illustrative rather than restrictive, and those skilled in the art should understand that, unless explicitly stated otherwise in the context, they should be understood as "one or more". "More" should be understood as two or more.

[0023] This application proposes an automated AI algorithm deployment system framework. This framework is designed for model deployment processes on heterogeneous hardware platforms. Based on time constraints and performance degradation constraints, and protected by an exception handling mechanism, it ensures model deployment efficiency and accuracy through task construction and parameter setting, model lightweighting, unified model representation, model adaptation, and performance evaluation processes. The aim is to achieve automated algorithm deployment with low manual labor and low barriers to entry.

[0024] This application discloses an automated AI algorithm model deployment method for heterogeneous hardware platforms, comprising: acquiring the AI ​​algorithm model to be deployed and the heterogeneous hardware platform; creating an AI algorithm model deployment task and setting model adaptation task parameters; constructing a lightweight model algorithm library, a rule constraint engine, and establishing an exception handling mechanism; performing model lightweighting and unified representation based on the rule constraint engine and exception handling mechanism to form a recommended AI algorithm model; adapting the recommended AI algorithm model to the heterogeneous hardware platform; and conducting performance evaluation based on the heterogeneous hardware platform testing environment.

[0025] Example 1 Figure 1 The following is a flowchart of an automated AI algorithm model deployment method for heterogeneous hardware platforms according to an embodiment of this application. Figure 1 The embodiments of this application will be described in further detail.

[0026] First, in step 101, the AI ​​algorithm model to be deployed and the heterogeneous hardware platform are obtained.

[0027] In this embodiment of the application, the AI ​​algorithm model to be deployed is deployed to a heterogeneous hardware platform, including: designing a unified algorithm model deployment process, adapting a task construction and parameter setting module, a model lightweighting module, a model adaptation module, and a performance evaluation module.

[0028] In this embodiment of the application, the AI ​​algorithm model to be deployed can be a full-precision Mobilenetv2 model to be deployed, and the heterogeneous hardware platform can be the domestically produced edge chip Huawei 310B4.

[0029] In step 102, create an AI algorithm model deployment task and set the model adaptation task parameters.

[0030] In this embodiment of the application, the AI ​​algorithm model deployment task is created by the adaptation task construction and parameter setting module, and the model adaptation task parameters are set, including the expected speedup ratio, the tolerance for algorithm accuracy decline, the optimization task duration, etc., wherein the algorithm accuracy decline tolerance and the optimization task duration update rules constrain the rule base requirements in the engine.

[0031] Preferably, in this embodiment of the application, the expected speedup is 50%, the tolerance for algorithm accuracy degradation is 5%, and the optimization task duration is 2 hours.

[0032] This application embodiment also includes: Model lightweighting algorithm library construction: In the model lightweighting stage, a model lightweighting algorithm library was built to automatically call different model pruning algorithms, model distillation algorithms, and model quantization algorithms. Based on parameters such as the model type of the deployment task, the target hardware platform, the expected speedup ratio, and the performance loss limit, pruning, quantization, distillation and other algorithms can be freely selected and combined to form an efficient model lightweighting solution. Rule Constraint Engine: In the deployment process, a constraint rule library is designed, including but not limited to deployment time constraints and performance degradation constraints, to build a rule constraint engine. Based on this engine, constraints are applied to the model lightweighting and inference testing phases. Deployment time constraints strictly control deployment time to avoid anomalies such as infinite optimization loops during model lightweighting or excessively long inference times after hardware adaptation. Performance degradation constraints promptly address sudden performance drops in the AI ​​algorithm model during quantization, pruning, and distillation, preventing ineffective model deployment and ensuring the timeliness of algorithm deployment and model effectiveness.

[0033] An exception handling mechanism was established: Based on a unified model representation framework, an exception handling mechanism was designed to perform compatibility checks on operators during model adaptation, dynamically optimize and adjust abnormal operators, and automatically terminate the model adaptation process for cases that cannot be effectively handled, thereby reducing invalid model deployments.

[0034] In step 103, the model is lightweighted and uniformly represented based on the rule constraint engine and the exception handling mechanism to form a recommendation AI algorithm model.

[0035] In this embodiment, the model lightweighting module, under the protection of rule constraints and anomaly handling mechanisms, forms a recommended optimization scheme based on the model quantization, pruning, and distillation algorithms integrated in the model lightweighting algorithm library. It can freely apply method criteria. In this embodiment, three methods, pruning, distillation, and half-precision quantization, are used for full-process optimization. The model input format is pth, and model pruning, model distillation, and model quantization are performed sequentially to output an ONNX format model.

[0036] In step 104, the recommended AI algorithm model is adapted to the heterogeneous hardware platform.

[0037] In this embodiment, the heterogeneous hardware platform executes a model adaptation command to convert the ONNX format recommendation AI algorithm model into a format compatible with the heterogeneous hardware platform.

[0038] Preferably, the model adaptation module automatically calls the deployment code of the Huawei 310B4 edge chip based on the ONNX format Mobilenetv2 model to be deployed, which is optimized and uniformly represented, executes the model adaptation command, and generates an OM format model that is compatible with the heterogeneous hardware platform of the 310B4 edge chip.

[0039] In step 105, a performance evaluation is performed based on a heterogeneous hardware platform test environment.

[0040] In this embodiment, the performance evaluation module performs performance evaluation based on the Huawei 310B4 edge chip test environment, including model inference speedup ratio, model accuracy degradation, and model compression ratio. The specific calculation formulas for these indicators are as follows: , Among them, S 加速比 SP represents the inference speedup ratio of the model, SP' represents the inference efficiency before optimization, and SP' represents the inference efficiency after optimization. , Among them, L 性能损失 For the decrease in model accuracy, ACC 前 The accuracy rate before optimization, ACC 后 The optimized accuracy; , Among them, D 压缩率 M represents the model compression ratio. 压缩前 M represents the space occupied by the model before compression. 压缩后 This represents the space occupied by the compressed model.

[0041] Calculations show that the model inference speedup of this application is 55%, the model accuracy decrease is 4.8%, the model compression rate is 20%, and the total deployment time is 32.5 minutes.

[0042] Figure 2 This is a flowchart illustrating the model lightweighting and unified representation method based on a rule constraint engine and an exception handling mechanism according to an embodiment of this application. The following will combine... Figure 2 The present application provides a more detailed description of the model lightweighting and unified representation method based on the rule constraint engine and exception handling mechanism.

[0043] In this embodiment of the application, under the rule constraints and exception handling mechanism, model pruning, model distillation, precision quantization algorithm, and model pruning are executed sequentially: First, in step 201, anomaly handling is performed on the AI ​​algorithm model to be deployed based on a unified model representation framework.

[0044] In this embodiment, exception handling is based on a unified model representation framework. It performs compatibility checks on operators in the adaptation of the AI ​​algorithm model to be deployed (such as the Mobilenetv2 model), dynamically optimizes and adjusts abnormal operators, and automatically terminates the model adaptation process in cases where it cannot be effectively handled.

[0045] In this embodiment of the application, the format of the AI ​​algorithm model to be deployed includes, but is not limited to, pth, pt, and other formats.

[0046] In step 202, the model pruning algorithm is executed on the AI ​​algorithm model to be deployed.

[0047] In this embodiment, the input AI algorithm model to be deployed is a pth format MobileNetv2 model, the output is a ppruned pth format MobileNetv2 model, and intermediate performance evaluations are performed. The model pruning task took 13.5 minutes, with a model inference speedup of 22%, a model accuracy decrease of 13.5%, and a model compression rate of 25%. The model pruning part is subject to deployment time constraints; when the execution time of the pruning algorithm task exceeds the constraint, the model deployment is automatically terminated.

[0048] In step 203, the model distillation algorithm is executed on the AI ​​algorithm model to be deployed.

[0049] In this embodiment, a pruned pth-format MobileNetv2 model is input. Based on the original MobileNetv2 model, the input lightweight MobileNetv2 model is fine-tuned to output an optimized pth-format MobileNetv2 model. Intermediate performance evaluations are performed, and the model distillation task takes 12.6 minutes, with a 21% speedup for model inference, a 4.4% decrease in model accuracy, and a 25% compression rate. The model distillation process is constrained by deployment time and performance loss. Model deployment is automatically terminated when the pruning algorithm task execution time exceeds the constraints or the model performance loss exceeds the tolerance for accuracy degradation. In step 204, a quantization algorithm is executed on the AI ​​algorithm model to be deployed.

[0050] In the embodiments of this application, the quantization algorithm includes, but is not limited to, a half-precision quantization algorithm, and the output model precision can be FP16 or 8-bit, etc.

[0051] In this embodiment, an optimized pth format MobileNetv2 model is input, a quantization algorithm is performed, and an FP16 pth format MobileNetv2 model is output. The model quantization part is subject to deployment time and performance loss constraints. When the execution time of the quantization algorithm task exceeds the constraints or the model performance loss exceeds the tolerance for accuracy degradation, the model deployment is automatically terminated.

[0052] In step 205, a unified model representation is performed on the AI ​​algorithm model.

[0053] In this embodiment, a fully optimized pth format Mobilenetv2 model is input, exception handling is automatically performed, compatibility checks are performed on operators in the adaptation of the optimized Mobilenetv2 model, abnormal operators are dynamically optimized and adjusted and replaced, and the model adaptation process is automatically terminated in cases where effective handling is not possible. If successful, an onnx format Mobilenetv2 model is output, and the model is prepared for final adaptation and deployment.

[0054] Addressing the challenges of current AI algorithm model deployment and the shortcomings of traditional deployment methods, the inventors propose an automated AI algorithm model deployment method for heterogeneous hardware platforms. Based on classic model optimization algorithms such as quantization, pruning, and distillation, an automated AI algorithm deployment process is designed, and a rule constraint engine and an exception handling mechanism are constructed to achieve efficient automated model deployment with low manual intervention, while avoiding significant performance degradation of the deployed model. 1. Integrate model lightweighting algorithms such as model quantization, pruning, and distillation, construct algorithm deployment rules and constraints, establish an exception handling mechanism, and realize the automated deployment of algorithms on heterogeneous hardware platforms; 2. A set of algorithm deployment constraint rules was constructed, including time limits, performance degradation and other rule conditions, to ensure the controllability of time and performance throughout the algorithm deployment process; 3. An exception handling mechanism was established, and the model representation method was strictly unified during the optimization process of algorithm deployment. A unified adaptation format was set for the deployment of heterogeneous hardware platforms, breaking down the barriers of hardware itself to algorithm deployment technology, and realizing the process-oriented adaptation of heterogeneous hardware platforms including x86 and ARM architecture CPUs, CUDA architecture GPUs, and Huawei NPU, Rockchip NPU, etc. 4. The deployment time of classic image classification, object detection, and semantic segmentation models on various hardware platforms such as x86 architecture and Huawei 310b4 has been reduced to 20% of the traditional manual deployment time, comprehensively improving the efficiency of algorithm deployment; 5. The system supports various lightweight algorithms such as quantization, distillation, and pruning, enabling automated deployment processes, reducing the technical threshold for algorithm deployment, and providing a general solution for algorithm deployment on heterogeneous hardware platforms.

[0055] Example 2 In embodiments of this application, an electronic device is also provided. Figure 3 This is a schematic diagram of the electronic device structure according to an embodiment of this application, such as... Figure 3 As shown, the electronic device of this application includes a processor 301 and a memory 302, wherein, The memory 302 stores a computer program, which, when read and executed by the processor 301, performs the steps described above in the embodiment of the automated AI algorithm model deployment method for heterogeneous hardware platforms.

[0056] Example 3 In the embodiments of this application, a computer-readable storage medium is also provided, which stores a computer program, wherein the computer program is configured to execute the steps in the embodiments of the automated AI algorithm model deployment method for heterogeneous hardware platforms as described above when running.

[0057] In this embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0058] It will be understood by those skilled in the art that the above descriptions are merely preferred embodiments of this application and are not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for deploying automated AI algorithm models on heterogeneous hardware platforms, comprising: Acquire the AI ​​algorithm model to be deployed and the heterogeneous hardware platform; Create an AI algorithm model deployment task and set the model to adapt to the task parameters; Build a lightweight model algorithm library, a rule constraint engine, and establish an exception handling mechanism; The model is lightweighted and uniformly represented based on a rule constraint engine and an exception handling mechanism, forming a recommendation AI algorithm model; Adapt the recommended AI algorithm model to heterogeneous hardware platforms; Performance evaluation was conducted using a heterogeneous hardware platform testing environment.

2. The automated AI algorithm model deployment method for heterogeneous hardware platforms according to claim 1, characterized in that, The model adaptation task parameters also include: expected speedup ratio, tolerance for algorithm accuracy degradation, and optimized task duration.

3. The automated AI algorithm model deployment method for heterogeneous hardware platforms according to claim 1, characterized in that, The steps of constructing a lightweight model algorithm library, a rule constraint engine, and establishing an exception handling mechanism also include: A lightweight model algorithm library is built for model pruning, model distillation, and model quantization algorithms; Design a constraint rule base, including deployment time constraints and performance degradation constraints, and build a rule constraint engine; An exception handling mechanism is established based on a unified model representation framework.

4. The automated AI algorithm model deployment method for heterogeneous hardware platforms according to claim 1, characterized in that, The step of lightweighting and unifying the model based on the rule constraint engine and anomaly handling mechanism to form a recommendation AI algorithm model also includes: Input the AI ​​algorithm model to be deployed under rule constraints and exception handling mechanisms; Based on the model lightweight algorithm library, model pruning, model distillation, and model quantization are performed sequentially on the AI ​​algorithm model to be deployed, and a recommended AI algorithm model in ONNX format is output.

5. The automated AI algorithm model deployment method for heterogeneous hardware platforms according to claim 4, characterized in that, The step of lightweighting and unifying the model based on the rule constraint engine and anomaly handling mechanism to form a recommendation AI algorithm model also includes: According to the aforementioned exception handling mechanism, the operators in the adaptation of the AI ​​algorithm model to be deployed are checked for compatibility, abnormal operators are dynamically optimized and adjusted and replaced, and the model adaptation process is automatically terminated in cases where effective handling is not possible.

6. The automated AI algorithm model deployment method for heterogeneous hardware platforms according to claim 4, characterized in that, The step of lightweighting and unifying the model based on the rule constraint engine and anomaly handling mechanism to form a recommendation AI algorithm model also includes: The model pruning algorithm is executed on the AI ​​algorithm model to be deployed. The input is the AI ​​algorithm model to be deployed, the output is the pruned AI algorithm model to be deployed, and the performance is evaluated in intermediate tests. The model distillation algorithm is executed on the pruned AI algorithm model to be deployed, and the optimized AI algorithm model to be deployed is output. Intermediate performance evaluation is performed on its performance. A quantization algorithm is executed on the optimized AI algorithm model to be deployed, and a quantized AI algorithm model with a preset precision is output. The quantization part of the quantized AI algorithm model is constrained by deployment time and performance loss. A unified model representation is performed. The input is a quantized AI algorithm model with a preset precision. The system automatically performs exception handling according to the aforementioned exception handling mechanism and outputs a recommended AI algorithm model in ONNX format, preparing the model for final adaptation and deployment.

7. The automated AI algorithm model deployment method for heterogeneous hardware platforms according to claim 1, characterized in that, The step of adapting the recommended AI algorithm model to heterogeneous hardware platforms further includes: The heterogeneous hardware platform executes a model adaptation command to convert the ONNX format recommendation AI algorithm model into a format compatible with the heterogeneous hardware platform.

8. The automated AI algorithm model deployment method for heterogeneous hardware platforms according to claim 1, characterized in that, The performance evaluation steps based on the heterogeneous hardware platform test environment also include: Computational model inference speedup: ， Among them, S 加速比 SP represents the inference speedup ratio of the model, SP' represents the inference efficiency before optimization, and SP' represents the inference efficiency after optimization. Calculate the decrease in model accuracy: ， Among them, L 性能损失 For the decrease in model accuracy, ACC 前 The accuracy rate before optimization, ACC 后 The optimized accuracy; Computational model compression ratio: ， Among them, D 压缩率 M represents the model compression ratio. 压缩前 M represents the space occupied by the model before compression. 压缩后 This represents the space occupied by the compressed model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the automated AI algorithm model deployment method for heterogeneous hardware platforms as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the automated AI algorithm model deployment method for heterogeneous hardware platforms as described in any one of claims 1 to 8.

Images