Neural Network Accelerator Model Loading with Differential Binaries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies are unable to efficiently load multiple neural network models into neural network accelerators due to the limitations of traditional computing chips, necessitating the development of specialized neural network accelerators but lacking effective methods for model loading.
Innovation Solution
A method involving compiling multiple neural network models into binary files, calculating differences using a preset algorithm, compressing these files using lossless coding, and loading them into a neural network accelerator, allowing for quick switching between models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional computing chips are used for neural network model training and loading, then the system can handle general computing tasks, but it cannot meet the high-performance requirements of neural network algorithms and cannot efficiently load multiple models
Solution Approach 1:
The patent segments multiple neural network models into a base model and differential models. The base model contains common parameters shared by all models, while differential models contain only the unique parameters needed to transform between models. This segmentation enables the system to load one base model and switch between multiple neural network models by applying small differential updates, thereby improving both loading efficiency and multi-model adaptability.
2Loss of time
If model volume is compressed to shorten loading time, then loading speed improves, but the system cannot load multiple neural network models simultaneously
Solution Approach 1:
The patent merges multiple neural network models by extracting and consolidating their common parameters into a single base model. The unique parameters of each model are stored as compact differential data. This merging approach reduces the total storage and loading time while maintaining the ability to support multiple models through parameter differentiation rather than loading complete separate models.
Data Source
AI summary
A method for loading multiple neural network model includes compiling at least two neural network models and generating at least two binary model files corresponding to the at least two neural network models. One of the at least two binary model files is an original model file which is taken as the basic model, and differences between the two files are calculated and recorded using preset difference calculation method. A differences file is generated, and the basic model and the differences file are compressed using a preset compression method, to generate an input file. Such input file is input into a neural network accelerator, the input file being decompressed to obtain the basic model and the differences file. The basic model and the differences file are loaded into the neural network accelerator. An electronic device and a non-volatile storage medium performing the above-described method are also disclosed.