Operator Compilation Separation for Smaller AI Model Files
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI models have large model files due to multiple operators, leading to excessive memory usage and inefficient repeated compilations when operators are updated.
Innovation Solution
Separate deployment of executable files and model files between a host and terminal device, allowing flexible management and update of operators, reducing model file size and avoiding repeated compilations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If multiple operators are compiled into a single model file, then the AI model can be executed, but the model file size becomes excessively large and occupies large memory space
Solution Approach 1:
The patent divides the compilation process into two independent parts: operator compilation and model compilation. Operators are compiled separately into executable files, while the model is compiled into a separate model file that only contains model parameters and structure. This segmentation eliminates the need to include all operator binaries within the model file, significantly reducing model file size while maintaining execution capability through separate operator loading.
2Adaptability or versatility
If the entire AI model and operators are recompiled when one operator is updated, then the model can be updated, but a large amount of repeated compilation occurs resulting in low compilation efficiency
Solution Approach 1:
By separating operator compilation from model compilation, the system enables independent updates of operators without triggering model recompilation. Each operator has its own executable file that can be updated independently, while the model file remains unchanged. This modular approach eliminates redundant compilation operations and significantly improves compilation efficiency.
Solution Approach 2:
The patent extracts operator executable files from the model file structure. Operators are compiled into separate executable files that are loaded dynamically during model execution. This extraction allows operator updates without affecting the model file, enabling efficient updates by only recompiling the specific operator that needs updating rather than the entire model.
3Reliability
If operator executable files are included within the model file, then the model can be self-contained, but the model file size increases and resource waste occurs
Solution Approach 1:
The patent extracts operator executable files from the model file, creating a separate operator loading mechanism. The model file contains only essential model data and structure, while operators are loaded from separate executable files during runtime. This extraction reduces the model file size and eliminates resource waste from storing redundant operator code, while maintaining self-containment through separate operator loading pathways.
Data Source
AI summary
This application provides an operator compilation method and apparatus. The method includes: A first device obtains a first executable file, and sends the first executable file to a second device, where the first executable file is a binary file obtained through compilation of a first operator. In addition, the first device obtains a model file, and sends the model file to the second device, where the model file is a binary file obtained through compilation of an AI model, and the model file includes execution logic of the first executable file. In this way, the first device configures an executable file of an operator to be outside the model file of the AI model, so that a size of the model file of the AI model can be effectively reduced, to help update and manage the operator and the AI model.