Forward reasoning method and device for neural network, equipment and storage medium

Abstract

The invention provides a forward reasoning method and device for neural network, equipment and a storage medium The method comprises the following steps of: dividing the target neural network into a plurality of sub-networks, wherein any one sub-network comprises at least one hidden layer of the target neural network, creating inference instances and inference engines corresponding to the plurality of sub-networks respectively on hardware equipment of the inference platform, and performing forward inference on the target neural network based on the inference instances and inference engines corresponding to the plurality of sub-networks respectively. One inference engine is only responsible for a part of hidden layers of the neural network, and multiple data inputs can be executed in different inference engines in parallel at the same time, so that the forward inference method provided by the invention has relatively high inference efficiency and data throughput, and hardware resourcesof an inference platform are fully utilized.

Description

technical field [0001] The present application relates to the technical field of parallel computing, and more specifically, relates to a neural network forward reasoning method, device, equipment and storage medium. Background technique [0002] The forward inference of neural network refers to creating an inference instance and an inference engine on the inference platform for the neural network to be inferred. The inference engine operates on each layer of the neural network based on the input data and inference instances of the input layer of the neural network. [0003] The current inference scheme is: create an inference instance for the neural network to be inferred, and create an inference engine in the inference instance, the inference engine receives input data, and performs operations on each layer of the entire neural network in sequence based on the inference instance, that is, The operation of an input data on different layers is strictly serialized, and differe...

AI Technical Summary

Problems solved by technology

[0005] In view of this, the present application provides a neural network forward reasoning method, device, equipment and readable storage medium to solve the problems of long time-consuming, low efficiency and low utilization of hardware resources in existing reasoning schemes. Its technical scheme is as follows:

Images