Method for realizing neural network model splitting by using multi-core processor and related product

Abstract

The embodiment of the invention discloses a method for realizing neural network model splitting by using a multi-core processor and a related product. When operators capable of being split exist in the neural network model, the operators are split, the optimal splitting combination is selected to obtain the optimal splitting result of the whole neural network model, then the sub-operators corresponding to the optimal splitting result are executed in parallel through multiple cores, and the purpose of reducing resource consumption of computer equipment is achieved.

Description

technical field

[0001] The invention relates to the technical field of deep learning, in particular to a method for splitting a neural network model by using a multi-core processor and related products. Background technique

[0002] In recent years, neural network processors have been continuously proposed, and like general-purpose processors, they are expanding from single-core to multi-core. This extended multi-core structure can support data parallelism in the training phase to improve data throughput and speed up training. However, in the inference phase, deep neural networks have higher requirements for end-to-end latency than throughput, which often determines the availability of accelerators in a certain scenario. Traditional data parallel solutions cannot meet the requirements of small data and low latency for accelerators in inference scenarios. Contents of the invention

[0003] In order to achieve the above purpose, in the first aspect, the embodiment of the p...

Images