An automated deployment method for general convolutional neural network accelerator

Abstract

The application discloses a kind of automation deployment methods for general convolutional neural network accelerator, step 1, from the convolutional neural network structure model configuration file of input parsing each layer structure;Step 2, from built-in operator library call floating point operator and the each layer structure obtained in step 1 are matched;Step 3, according to the floating point operator obtained in step 2, parameter quantization processing or directly floating point weight parameter is reordered and exported;Step 4, modeling analysis and optimal solution are carried out;Step 5, the parallelism of FPGA accelerator calculation module is re-adjusted;Step 6, based on step 5, create FPGA accelerator top module, each operator module is instantiated in its interior, and the top module is connected with external control unit, storage module by bus.This application can realize the customization optimization of different network structure, save human resource cost, improve deployment efficiency.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of FPGA devices, and particularly relates to a general FPGA accelerator automation deployment tool design for convolutional neural network inference. BACKGROUND

[0002] Convolutional Neural Network (CNN) is one of the representation algorithms in the field of artificial intelligence, and is widely used in image recognition and target detection fields. The structure of CNN can mainly be divided into an input layer, a convolution layer, a pooling layer, a full connection layer, an activation layer and an output layer. With the increasing size of CNN, the amount of calculation is increasing. In order to meet the use demand of low delay, people begin to try to use various computing platforms to accelerate deployment.

[0003] Field Programmable Gate Array (FPGA) is a kind of semi-custom semiconductor device, which can be configured into different functions according to different use requirements. Implementing CNN accelerator on FPGA can customize special parallel computing units and storage structures based on algorithm characteristics, and achieve better acceleration effect.

[0004] The deployment scheme of traditional CNN on FPGA is to analyze the structure of each layer of neural network manually, then write code to realize each layer calculation module to support neural network inference calculation process, and complete quantization and performance optimization. This scheme requires developers to be familiar with the related knowledge of CNN and FPGA development, which will consume a lot of time, and the finally realized accelerator can only support the inference of a single network. If a new network needs to be deployed and accelerated, the above process needs to be repeated again, and the deployment efficiency is low.

[0005] The proposal of some general CNN inference acceleration modules alleviates the above problems to some extent. For example, NVDLA (NVIDIA Deep Learning Accelerator) proposed by NVIDIA and DPU (Deep Learning Processor Unit) proposed by Xilinx have included common operators such as convolution, pooling and activation in CNN, and users only need to convert CNN model into instruction stream to control general inference acceleration module to call corresponding operators to execute calculation to realize CNN inference process. Although this deployment scheme can realize general support and rapid deployment of CNN, the instruction stream architecture will reduce the calculation efficiency of the accelerator. In addition, the number and type of calculation units in the accelerator are fixed, which cannot be customized and optimized for different network structures, and cannot achieve the best acceleration effect. SUMMARY

[0006] In view of the above technical problems, the present application aims to provide an automatic deployment method for a general-purpose convolutional neural network accelerator, which realizes the customized optimization of a general-purpose FPGA accelerator deployment for a plurality of different network structures of convolutional neural networks.

[0007] The present application realizes the following technical solutions:

[0008] An automatic deployment method for a general-purpose convolutional neural network accelerator, comprising the following steps:

[0009] Step 1: parsing each layer structure from the input convolutional neural network structure model configuration file;

[0010] Step 2: calling a floating-point operator from the built-in operator library and matching it with each layer structure parsed in step 1, the floating-point operator being instantiated into an operation module supporting different dimensions according to different input template parameters when called;

[0011] Step 3: performing parameter quantization processing or directly reordering and exporting the floating-point weight parameters according to the floating-point operator obtained in step 2;

[0012] Step 4: modeling analysis and optimal solution, including building an FPGA accelerator latency model and a resource model:

[0013] The latency model Latency() of the FPGA accelerator is calculated according to the network structure model configuration file input in step 1 and the latency model built in each floating-point operator in step 2, as shown in the following formula:

[0014]

[0015] wherein, n i represents the number of parallel input channels of the input module, n c , n p , n f represent the parallelism of the convolution, pooling and fully connected operators respectively, n o represents the number of parallel output channels of the output module, N represents the total number of convolutional neural network layers, L i (n i ) represents the latency of input module data loading, L c (n c ) represents the latency of convolution module calculation; L p (n p ) represents the latency of pooling module calculation, L f (n f ) represents the latency of fully connected module calculation, L o (n o ) represents the latency of the output module writing data back, Lo represent the remaining latency of the activation module calculation, which is not affected by the parallel parameter;

[0016] The resource model Resource() of the FPGA accelerator is as follows:

[0017] Resource(n i ,n c ,n p ,n f ,n o )=R i (n i )+R c (n c )+R p (n p )+R f (n f )+R o (n o )+R0

[0018] wherein R0 represents the remaining module resource occupation not affected by the parallel parameter, R i (n i ), R c (n c ), R p (n p ), R f (n f ), R o (n o ) represent the resource occupation of the input module, the convolution module, the pooling module, the full connection module and the output module respectively;

[0019] Solving the parallel parameter that makes the inference performance optimal under the specified resource constraint is as follows:

[0020]

[0021] wherein R * represents the maximum limit of the resource occupation of the FPGA accelerator including the total amount of resources of the target FPGA model, and min represents the minimization of the inference latency of the accelerator according to the resource model of the accelerator.

[0022] Step 5, according to the parallel parameter solved in step 4, the parallel degree of the FPGA accelerator calculation module is re-adjusted, and the parallel degree expansion strategy is as follows:

[0023] For the input feature map channel, the number of multipliers is doubled, and the input of the addition tree is also doubled to realize parallel expansion;

[0024] For the pooling module, the single-channel pooling structure is copied multiple times to realize parallel expansion;

[0025] For the input and output module, parallel expansion is realized by expanding the number of AXI bus interfaces;

[0026] For the full connection module, parallel expansion is realized by increasing the number of multipliers by one and the input of the addition tree by one;

[0027] Step 6, based on step 5, create a top-level module of the FPGA accelerator, instantiate each operator module in it, and connect the top-level module with the external control unit and the storage module through the bus.

[0028] Compared with the prior art, the advantages and positive effects of the present application are:

[0029] 1) Customized optimization of different network structures can be realized, manpower resource cost can be saved, and deployment efficiency can be improved;

[0030] 2) A variety of convolutional neural networks can be quickly deployed on the FPGA, and 16-bit quantization can be supported;

[0031] 3) The parallel parameter optimization adjustment of the calculation unit can be realized according to the network structure, and high inference speed can be ensured. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 is a flow chart of an automatic deployment method of a general-purpose convolutional neural network accelerator of the present application;

[0033] Figure 2 is a hardware structure diagram of a convolution calculation module;

[0034] Figure 3 is a hardware structure diagram of a pooling calculation module;

[0035] Figure 4 is a hardware architecture diagram after the accelerator is deployed DETAILED DESCRIPTION

[0036] The technical solutions will be described in detail below with reference to the drawings and examples.

[0037] The specific embodiments of the present application will be described in detail below with reference to the drawings.

[0038] As shown in the figure, it is an automatic deployment system and method flow chart of a general-purpose convolutional neural network accelerator of the present application, specifically including the following steps: Figure 1

[0039] ​Step 1, parse the layer structure from the input convolutional neural network structure model configuration file, including the type of each layer structure, input feature map and input feature map dimension, convolution kernel dimension and convolution step (only convolution layer), pooling step (only pooling layer), activation function type (only activation layer) and other information as well as the connection order of each layer;

[0040] Step 2, call the floating point operator from the built-in operator library to match the layer structure parsed in step 1, the operator library is a pre-written input, output, convolution, pooling, full connection, activation and other module template. When the floating point operator is called, it is instantiated to support different dimension operation modules according to different input template parameters, to realize the general support for different dimension operations;

[0041] Step 3, if parameter quantization is needed, the weight, bias, output feature map and intermediate calculation result and other parameters are quantized according to the following steps; The quantization process follows the formula as follows:

[0042] The value of the unsigned fixed point number is as follows:

[0043]

[0044] Where, bw represents the fixed point bit width, b i represents the binary value of the i-th bit, and E represents the exponent. The purpose of the quantization operation is to obtain the exponent E of the quantized fixed point number. The quantization scheme adopted by the present application is the direct quantization method, and the conversion formula between the floating point number and the fixed point number is as follows:

[0045] v fixed =(int)(v float ×2 bw )

[0046] v float =(int)(v fixed ×2 -bw )

[0047] Since the bit width and the exponent of the fixed point number are fixed, when the floating point number is converted to the fixed point number and then converted to the floating point number, an error will be generated. In order to reduce the error as much as possible, the present application determines an exponent for each layer of parameters. For the weight parameters of a layer, the exponent is as follows:

[0048]

[0049] Where, W float represents the floating point value of any weight of the layer, and W(bw,E w ) represents the determined bit width bw and exponent E wThe floating-point number value after the floating-point number is fixed-pointed and then converted back to the floating-point number, and argmin represents solving the exponent E that minimizes the error w The quantization process of the bias parameters, the output feature map and the intermediate calculation result is similar to the weight parameter quantization step described above, and will not be described again here.

[0050] After quantization is completed, the floating-point operator is replaced with the corresponding fixed-point operator to support the quantized calculation process.

[0051] Because the on-chip storage resource of the FPGA is small, the picture needs to be processed by block calculation, which leads to a situation that the number of memory accesses is large and the amount of data accessed by a single memory access is small when loading the weight, limiting the utilization rate of the communication bandwidth. In order to solve the above problems, the weight is reordered according to the current layer block scheme after the quantization operation is executed, reducing the number of memory accesses, increasing the burst transmission length, and improving the utilization rate of the communication bandwidth.

[0052] Finally, the quantized and reordered parameters are exported for subsequent inference calculation;

[0053] If the quantization operation is not needed to be performed, the tool only reorders and exports the floating-point weight parameters without quantization processing;

[0054] Step 4, modeling analysis and optimization solving, in order to optimize the parallel parameters of the calculation module in the FPGA accelerator circuit, it is necessary to construct the FPGA accelerator delay model and resource model. The delay model is calculated according to the network structure model configuration file input in step 1 and the delay model built in each floating-point operator in step 2. The delay model is a function related to the parallel parameters of each calculation module, as shown in the following formula:

[0055]

[0056] Wherein, n i represents the number of parallel input channels of the input module, n c , n p , n f respectively represent the parallel degree of the convolution, pooling and full connection operators, n o represents the number of parallel output channels of the output module, and N represents the total number of convolutional neural network layers;

[0057] L i (n i ) represents the delay of input module data loading, which is a function of the parallel parameter n i ;

[0058] L c (n c ) represents the delay of convolution module calculation, which is a function of the parallel parameter n c ;

[0059] L p (n p ) represents the latency of the pooling module calculation, which is a function of the parallel parameter n p ;

[0060] L f (n f ) represents the latency of the full connection module calculation, which is a function of the parallel parameter n f ;

[0061] L o (n o ) represents the latency of the output module writing data back, which is a function of the parallel parameter n o ;

[0062] L o represents the latency of the activation module calculation and other remaining latencies that are not affected by the parallel parameter.

[0063] The resource model of the FPGA accelerator is as follows:

[0064] Resource(n i ,n c ,n p ,n f ,n o )=R i (n i )+R c (n c )+R p (n p )+R f (n f )+R o (n o )+R0

[0065] In the above formula, R0 represents the remaining module resource occupation that is not affected by the parallel parameter, R i (n i ), R c (n c ), R p (n p ), R f (n f ), and R o (n o ) represent the resource occupation of the input module, the convolution module, the pooling module, the full connection module, and the output module respectively, and they are all functions of their own parallel parameters.

[0066] After the latency and resource models are constructed, the parallel parameters that make the inference performance optimal under the specified resource constraints are solved, as follows:

[0067]

[0068] Among them, R * This represents the maximum limit on FPGA accelerator resource usage, typically the total resources of the target FPGA model. min indicates minimizing the accelerator's inference latency based on the accelerator's resource model.

[0069] Step 5: Based on the parallel parameters obtained in Step 4, readjust the parallelism of the FPGA accelerator computing module;

[0070] The parallelism of the operator module is adjusted by the instantiation parameters. That is, the resource consumption of the accelerator is calculated based on the built-in operator resource model and the instantiation parallelism, and the total resource of the selected FPGA model is used as a constraint. The inference delay calculated based on the input network structure and operator delay model is used as the objective optimization function. The geometric programming method is used to solve for the parallel parameters that optimize the accelerator performance.

[0071] In this invention, the input feature map channel parallelism and output feature map channel parallelism of the convolution calculation module are configurable. When the input feature map channel parallelism is m and the output feature map channel parallelism is n, the module will include m×n adders, n sets of m / 2 input adder trees, n multiplexers, and n adders. For example... Figure 2 The diagram shows the hardware structure of the convolution calculation module. The solid lines represent the hardware structure of the convolution calculation module when m=2 and n=2, while the dashed lines represent the changes in hardware structure when m is expanded to 4, i.e., the number of multipliers doubles and the input of the addition tree also doubles.

[0072] The input and output feature map channel parallelism of the pooling module are also configurable. Since pooling only changes the width and height dimensions of the feature map and does not change the number of channels, m = n always holds true in the pooling module. Figure 3 The diagram shows the hardware structure of the pooling computation module. Each channel in the pooling module is an independent hardware structure, including a multiplexer, a pooling operator, and a register. The preset value and pooling operator differ depending on the pooling type. For example, in max pooling, the preset value is a pre-defined, relatively small negative number, and the pooling operator is a size comparison unit; in average pooling, the preset value is 0, and the pooling operator is an addition and averaging operation. Parallel expansion of the pooling module is relatively simple; it only requires copying the single-channel pooling structure multiple times.

[0073] The parallel expansion of the fully connected module is similar to that of the convolution module. The parallelism expansion of the input and output modules is achieved by expanding the number of AXI bus interfaces, which will not be elaborated here.

[0074] Step 6, based on step 5, create an accelerator top-level module and instantiate the above-mentioned operators inside it, and connect the top-level module with the CPU, DDR and other external control units, storage modules through the bus. As shown in Figure 4 Fig. 1 is a whole hardware architecture of the accelerator. The input and output modules in the accelerator are interconnected with the DDR through the AXI bus; the interconnection form between the CPU and the DDR and the control module is different according to different target FPGA models, mainly including the PCIE bus and the AXI bus.

[0075] The specific reasoning process of the accelerator is that the CPU first reads the network structure configuration file, then stores the input data in the specified address of the DDR, then writes the current layer parameters, input data address and output data address and the like into the registers of the control module, and finally the CPU sends a start signal to the control module. After receiving the start signal, the control module controls the input module to complete data loading, calls the calculation module required for current layer calculation to complete calculation, then controls the output module to write the calculation result back to the destination address, and finally pulls up the specified register to notify the CPU that the calculation is completed. At the same time, the CPU constantly polls the registers of the accelerator control module, waits for the register to pull up the flag to complete the calculation, and then the CPU starts to repeat the above steps to control the accelerator to complete the next layer calculation, until the reasoning process is completed.

[0076] Step 7, execute the processes of synthesis, layout and routing, and finally generate a deployable bitstream file.

[0077] In summary, the automatic deployment tool for the general-purpose convolutional neural network accelerator of the present application can adjust the types of calculation units inside the accelerator according to different convolutional neural network structures, and realize the general deployment of the FPGA for the convolutional neural network reasoning acceleration. The modeling analysis and geometric planning tool built-in the tool can automatically complete the optimization and adjustment of the parallel parameters of the calculation units, so as to maximize the performance of the accelerator.

[0078] The above only describes the embodiments of the present application, and is not used to limit the range of the present application. For those skilled in the art, any modification, equivalent replacement or variation and the like can be made without departing from the spirit and principle of the present application, and all fall within the protection scope of the present application defined by the appended claims.

Claims

1. A method for automated deployment of a general-purpose convolutional neural network accelerator, comprising: The method comprises the following steps: Step 1, parsing each layer structure from the input convolutional neural network model structure configuration file; Step 2, calling a floating-point operator from the built-in operator library to match each layer structure parsed in step 1, and the floating-point operator is instantiated into an operation module supporting different dimensions according to different input template parameters when called; Step 3, according to the floating-point operator obtained in step 2, performing parameter quantization processing or directly reordering and exporting the floating-point weight parameters; Step 4, modeling analysis and optimization solution, including building an FPGA accelerator delay model and a resource model: Latency model of FPGA accelerator According to the network model structure configuration file input in step 1 and the latency model built in each floating-point operator in step 2, the following formula is calculated: ; wherein, represents the number of parallel input channels of the input module, , , respectively represent the parallelism of the convolution, pooling and fully connected operators, represents the number of parallel output channels of the output module, represents the total number of layers of the convolutional neural network, represents the latency of data loading of the input module, represents the latency of computation of the convolution module; represents the latency of computation of the pooling module, represents the latency of computation of the fully connected module, represents the latency of data writing back of the output module, represents the remaining latency which is not affected by the parallelism parameters; Resource model for fpga accelerators As shown in the following equation: ; wherein, represents the remaining module resource occupation not affected by the parallel parameter, , , , , respectively represent the resource occupation of the input module, the convolution module, the pooling module, the full connection module and the output module. Solving the parallel parameters that make the inference performance optimal under the specified resource constraints, as shown in the following formula: ; wherein, The maximum limit of the FPGA accelerator resource occupation indicates the total amount of resources of the target FPGA model, and min indicates the minimum inference delay of the accelerator according to the resource model of the accelerator. Step 5, according to the parallel parameters solved in step 4, re-adjusting the parallel degree of the FPGA accelerator calculation module, and the parallel expansion strategy is as follows: For the input feature map channel, the number of multipliers is increased by one, and the input of the addition tree is also increased by one to realize parallel expansion; For the pooling module, the single-channel pooling structure is copied multiple times to realize parallel expansion; For the input and output module, the number of AXI bus interfaces is expanded to realize parallel expansion; For the full connection module, the number of multipliers is increased by one, and the input of the addition tree is also increased by one to realize parallel expansion; Step 6, based on step 5, creating a top-level module of the FPGA accelerator, instantiating each operator module in it, and connecting the top-level module with the external control unit and storage module through the bus.

2. The method of claim 1, wherein, The configuration file of the network model structure includes the type of each layer structure, the input feature map and the input feature map dimension, the convolution kernel dimension and the convolution step of the convolution layer, the pooling step of the pooling layer, the activation function type information of the activation layer, and the connection order of each layer structure.

3. The method of claim 2, wherein, The operator library includes pre-written input, output, convolution, pooling, full connection, and activation module templates, and the floating-point operator is instantiated into an operation module supporting different dimensions when called.

4. The method of claim 1, wherein, It also includes a parameter quantization process: Quantizing the weight, bias, output feature map, and intermediate calculation results; after quantization, the floating-point operator is replaced with the corresponding fixed-point operator to support the quantized calculation process; Exporting the quantized and reordered parameters.

5. The method of claim 1, wherein, In step 5, the resource consumption of the accelerator is calculated according to the built-in operator resource model and the instantiation parallel degree, the total amount of resources of the selected FPGA model is taken as the constraint condition, the inference delay calculated according to the input network structure and the operator delay model is taken as the target optimization function, and the geometric programming method is used to solve the parallel parameters that make the accelerator performance optimal.

Images