Reasoning acceleration method, reasoning acceleration device and storage medium

Abstract

The invention relates to a reasoning acceleration method, a reasoning acceleration device and a storage medium, and the method comprises the steps: obtaining first model information of a to-be-accelerated first network model and second model information of a second network model corresponding to the first network model; inputting the first model information and the second model information into a reasoning acceleration system, performing fixed-point optimization processing on the first network model and the second network model by using a fixed-point optimization module of the reasoning acceleration system, and determining a third network model and fixed-point configuration information of the third network model; performing optimization acceleration on the third network model based on the SIMD instruction by utilizing a single instruction multiple data stream SIMD optimization module of the reasoning acceleration system to obtain SIMD configuration information of the accelerated target network model; and determining target model configuration information of the target network model according to the fixed-point configuration information SIMD configuration information.

Description

technical field [0001] The present disclosure relates to the technical field of information processing, and in particular, to an inference acceleration method, an inference acceleration device, and a storage medium. Background technique [0002] With the development of computer technology, the neural network model has been successfully applied in many fields such as image recognition processing and automatic driving, and with the continuous enrichment of application requirements, there are more and more network layers in the neural network model; the increase of the network layer As a result, the model depth of the neural network model is getting larger and larger, the calculation amount is also significantly increased, and the processing efficiency is greatly reduced. [0003] In the related art, a central processing unit (Central Processing Unit, CPU), an image processing unit (Graphics Processing Unit, GPU), a neural network processing unit (Neural network processing unit...

AI Technical Summary

Problems solved by technology

[0002] With the development of computer technology, neural network models have been successfully applied in many fields such as image recognition processing and automatic driving, and with the continuous enrichment of application requirements, more and more network layers in neural network models; the increase of network layers As a result, the model depth of the neural network model is getting larger and larger, the amount of calculation is also significantly increased, and the processing efficiency is greatly reduced.

[0003] In related technologies, usually through a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU), a neural network processor (Neural network Processing Unit, NPU) and a digital signal processor (Digital Signal Processor, DSP) ) and other hardware to accelerate the reasoning of neural network models; however, using CPU and GPU to run neural network models on mobile edge devices such as mobile phones requires a lot of terminal resources and high power consumption, which cannot be achieved for a long time (such as 24 hours) )run

Images