Deep neural network reasoning acceleration method and system based on multi-operator fusion

Abstract

The invention relates to a deep neural network reasoning acceleration method and system based on multi-operator fusion, and the method specifically comprises the steps: inputting a neural network calculation diagram, obtaining a neural network calculation logic diagram, and obtaining a complete neural network forward calculation symbol expression according to the calculation relation between neural network operators; employing a fusible operator search method, and automatically simplifying the system by means of operator symbol expressions, simplifying a symbol expression of forward calculation of the neural network, obtaining the simplest symbol expression, and achieving multi-operator fusion; according to a multi-operator fusion result and the obtained simplest symbol expression, a new neural network calculation reasoning logic diagram is constructed, the simplest symbol expression is decoupled, offline calculation is performed, new model parameters are stored, and a corresponding neural network model structure is constructed; and finally, loading new model parameters to realize reasoning acceleration. According to the invention, the overhead of operator execution gaps can be reduced, the utilization rate of computing resources of equipment is improved, and the overall network reasoning speed is optimized.

Description

technical field [0001] The invention belongs to the technical field of artificial intelligence, and relates to a deep neural network reasoning acceleration method and system based on multi-operator fusion. Background technique [0002] In the field of artificial intelligence, a deep neural network is usually a very deep neural network that contains many network parameters. For example, ResNet-101 contains a network structure of 101 layers and more than 23 million network parameters. This makes the inference speed of the deep neural network a great challenge in the deployment inference stage. Operator fusion is a neural network inference acceleration technology. Operator fusion is to analyze and optimize the logic of the existing network computing graph, and perform operations such as splitting, reorganizing, and merging the original computing logic to reduce the overhead of operator execution gaps and improve the utilization of device computing resources, so as to realize t...

AI Technical Summary

Problems solved by technology

[0004] In order to solve the above-mentioned technical problems existing in the prior art, the present invention proposes a deep neural network reasoning acceleration method based on multi-operator fusion to solve the problem that the current operator fusion technology lacks automatic reasoning tools, and the operator fusion is not thorough. In-depth questions to improve the speed of model reasoning, the specific technical solutions are as follows:

Images