Method for quickly deploying convolutional neural network on FPGA (Field Programmable Gate Array) based on Pytorch framework

Abstract

The invention discloses a method for quickly deploying a convolutional neural network on an FPGA (Field Programmable Gate Array) based on a Pytorch framework. The method comprises the steps of establishing a model quick mapping mechanism, constructing a reconfigurable computing unit and carrying out self-adaptive processing flow based on rule mapping; when a convolutional neural network is definedunder a Pytorch framework, establishing a model fast mapping mechanism through construction of naming rules; making optimization strategy calculation under a hardware resource constraint condition, establishing a template library based on a hardware optimization strategy and creating a reconfigurable calculation unit i at an FPGA end; and finally, decomposing the complex network model file at the FPGA end in the self-adaptive processing flow based on rule mapping, abstracting the network into a directed acyclic graph, and finally generating a neural network accelerator to realize an integrated flow from the model file of the Pytorch framework to FPGA deployment. The directed acyclic graph of the network can be established through a model fast mapping mechanism, the FPGA deployment process can be completed only by inputting hardware design variables in the FPGA deployment process, and the method is simple and high in universality.

Description

technical field [0001] The invention belongs to the technical field of convolutional neural network hardware acceleration, and relates to a rapid deployment method of a convolutional neural network based on a Pytorch framework on an FPGA. Background technique [0002] In recent years, convolutional neural networks have been widely used in natural language processing, computer vision and other fields. A neural network generally includes two stages: training and testing. Training is the process of extracting model parameters from training data and neural network models (ResNet, RNN, etc.) using CPU or GPU. The test is to run the test data with the trained model (neural network model + model parameters) and check the results. And caffe, Pytorch, and tensorflow are to unify and abstract the link data involved in the training process to form a usable framework. [0003] At present, the industry deploys deep learning models on FPGAs mostly by analyzing the caffe prototxt to obt...

AI Technical Summary

Problems solved by technology

Since the Pytorch model does not contain network topology information, the model trained in Pytorch needs to be converted into a caffe model through tools such as onnx before it can be deployed. However, the onnx tool only supports the regular caffe layer. If it is a custom layer in Pytorch, then unable to convert

Images