Method for improving AI model edge device reasoning operation
By constructing an inference computing framework for AI models on edge devices, the model is decomposed into multiple sub-models and weighted, enabling synchronous computation of multiple data. This solves the speed limitation of pruning algorithms in existing technologies and improves computational efficiency and accuracy.
Patent Information
- Application Number
- CN202311811786.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-27
- Publication Date
- 2024-11-22
AI Technical Summary
In existing AI model inference methods for edge devices, pruning algorithms have limitations in speeding up inference operations and cannot meet the needs of simultaneous multi-data operations.
An inference framework is constructed, which decomposes the AI model into multiple independently running sub-models. Through weight partitioning and data standard deviation representation, synchronous inference operations of multiple sub-models are achieved, and multiple hardware devices are used to support the synchronous and alternating processing of data.
It enables synchronous inference operations on multiple datasets, improving computational efficiency, shortening overall computation time, avoiding over-pruning or under-pruning that may occur with pruning algorithms, and ensuring that data accuracy is not lost.
Abstract
Citation Information
Patent Citations
Multi-model parallel reasoning method based on AI chip
CN112783650A
Real-time instance segmentation method and system based on calculation unloading mechanism
CN114924874A
