NPU neural network model deployment method and device based on TVM

Abstract

The invention provides a TVM-based NPU neural network model deployment method and apparatus. The method comprises the steps of obtaining a to-be-deployed neural network model and hardware attribute information of a current NPU; according to calculation logic of the neural network model, determining a convolution operator to be deployed to the current NPU; determining the optimal parameter configuration of each convolution operator in the current NPU according to the hardware attribute information of the current NPU and the parameter information of each convolution operator; according to the optimal parameter configuration and the hardware attribute information of the current NPU, adjusting the data layout and the convolution weight of the tir layer, and modifying the parameters of the relay layer to obtain a target TVM adaptive to the current NPU; and based on the target TVM, generating a deployment file of the neural network model suitable for the current NPU. Corresponding parameter adjustment is carried out on the tir layer and the relay layer of the TVM, so that the obtained target TVM can adapt to the NPU.

Description

Technical field [0001] The present application relates to the field of computer technology, and in particular to a TVM-based NPU neural network model deployment method and device. Background technique [0002] With the continuous development of artificial intelligence and big data, more and more dedicated AI chips have emerged. Among them, the embedded neural network processor (Neural-network Processing Unit, referred to as: NPU) adopts a data-driven parallel computing architecture. Good at processing massive multimedia data such as videos and images. [0003] In the existing technology, in order to simply, quickly and efficiently deploy deep learning models under different frameworks to respective NPU hardware platforms, hardware manufacturers or IP vendors need to develop an inference software stack that integrates model optimization, code generation and compilation. As an open source and general machine learning compilation framework, TVM has become the first choice of m...

AI Technical Summary

Problems solved by technology

[0005] This application provides a TVM-based NPU neural network model deployment method and device to solve the defects in the prior art that TVM cannot adapt to NPUs of different hardware manufacturers.

Images