Method and device for achieving deep neural network model based on heterogeneous model graph

Abstract

The invention provides a method and a device for achieving a deep neural network model based on a heterogeneous model graph. The method comprises the following steps: executing a splitting operation of splitting a deep neural network model into a plurality of sub-models according to a predetermined splitting standard; according to the performance of the plurality of split sub-models on each processor, merging at least part of the sub-models to obtain merged sub-models and corresponding relationships between the merged sub-models and the processors, the corresponding relationships representingthat the merged sub-models have optimal performance when running on the corresponding processors; constructing a heterogeneous graph which reflects the corresponding relationship and consists of the merged sub-models; and executing the sub-models in the heterogeneous graph for the input task.

Description

technical field [0001] The present application relates to the field of deep learning, and more specifically, to a method and device for a deep neural network model based on a heterogeneous model graph. Background technique [0002] With the improvement of computing power and the development of scientific computing, recognition technologies such as image recognition and speech recognition are increasingly widely used. In order to meet people's growing needs for recognition speed and accuracy, technicians have developed neural networks. At present, the performance of neural networks running on hardware is mainly improved through hardware-based design methods and software-based design methods. [0003] In order to improve the performance of the neural network, the prior art often adopts a hardware-based design method, wherein the hardware-based design method improves the neural network by designing the hardware as a configurable system-on-chip (SOC) or through hardware program...

AI Technical Summary

Problems solved by technology

[0005] However, the above hardware-based design methods and software-based design methods have the following problems: (1) The resource utilization rate is not high: although configurable hardware can improve the execution efficiency of the neural network to some extent or reduce Energy consumption, but in actual execution, a lot of hardware will be idle, which also increases hardware costs

(2) The problem of inflexibility: Since many designs are directly customized for a specific model, but if the model changes, it needs to be reprogrammed

As more and more models become more and more complex, this method will bring a lot of development and maintenance work, greatly reducing work efficiency

(3) Issues that require specific software and hardware support: In hardware-based heterogeneity, specific SOC support is necessary, but at present, mainstream embedded devices are mostly traditional multi-processor architectures

The method based on software design requires software designers to use a cross-platform language like OpenCL, and this method can only be applied to those hardware supported by OpenCL, but new hardware (such as NPU) has special compilers and instructions set, which is completely different from traditional programming methods, so these hardware cannot work with other processors through the existing software architecture

Images