Method for improving AI model edge device reasoning operation

By constructing an inference computing framework for AI models on edge devices, the model is decomposed into multiple sub-models and weighted, enabling synchronous computation of multiple data. This solves the speed limitation of pruning algorithms in existing technologies and improves computational efficiency and accuracy.

CN119005325APending Publication Date: 2024-11-22XIAMEN DNAKE INTELLIGENT TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202311811786.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-12-27
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

In existing AI model inference methods for edge devices, pruning algorithms have limitations in speeding up inference operations and cannot meet the needs of simultaneous multi-data operations.

Method used

An inference framework is constructed, which decomposes the AI ​​model into multiple independently running sub-models. Through weight partitioning and data standard deviation representation, synchronous inference operations of multiple sub-models are achieved, and multiple hardware devices are used to support the synchronous and alternating processing of data.

Benefits of technology

It enables synchronous inference operations on multiple datasets, improving computational efficiency, shortening overall computation time, avoiding over-pruning or under-pruning that may occur with pruning algorithms, and ensuring that data accuracy is not lost.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention discloses a method for improving AI model edge device reasoning operation, which comprises the following steps of: constructing a reasoning operation framework: firstly constructing the AI model edge device reasoning operation framework, and generating a reasoning operation primary framework flow and a reasoning operation secondary framework flow in an AI model edge device reasoning operation process; according to the method, the model is decomposed into a plurality of independent operation sub-models, when the to-be-reasoned operation data is received, the plurality of sub-models operate synchronously, synchronous reasoning operation of the plurality of data can be met, alternate reasoning operation can be performed under the action of the plurality of sub-models, the data cannot influence one another and is not limited by the number of operations, the efficiency is higher, and the method is suitable for large-scale popularization and application. And the plurality of sub-models perform different part operation on the data according to different weights, so that the time of reasoning operation is shorter than that of the whole data, the condition of excessive pruning or insufficient pruning in the existing pruning algorithm is avoided, the data is not lost, and the precision is high.
Need to check novelty before this filing date? Find Prior Art

Citation Information

Patent Citations

  • Multi-model parallel reasoning method based on AI chip

    CN112783650A

  • Real-time instance segmentation method and system based on calculation unloading mechanism

    CN114924874A