ZYNQ-based universal CNN accelerator and use method thereof

Abstract

The invention relates to the field of deployment acceleration reasoning of a convolutional neural network on edge end equipment, in particular to a ZYNQ-based universal CNN accelerator and a use method thereof, and the universal CNN accelerator comprises a PS end, a PL end, an AXI bus interface module, a DMA data handling module, a PSDDR memory and a PLDDR memory. According to the method, for different convolutional neural networks, according to the set parameters on the ZYNQ platform, the PL end does not need to be modified, construction and decomposition of the network model only need to be carried out at the PS end, the PL end is controlled to complete forward reasoning of the convolutional neural network, and therefore the efficiency and universality of deployment of the convolutional neural network at the edge end are greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of accelerators, and in particular to a ZYNQ-based general CNN accelerator and a method for using the same. Background Art

[0002] In recent years, with the development of artificial intelligence, deep neural networks have achieved remarkable results in the fields of text recognition, object detection, face recognition, speech recognition, etc. As the application scenarios become more complex, the models of deep neural networks are also increasing, requiring a large amount of storage and computing. Therefore, how to accelerate and implement deep neural network algorithms on hardware has become an important issue in the field of machine learning.

[0003] In the early days, convolutional neural networks were mainly accelerated by CPU (Central Processing Unit) and GPU Graphic Processing Unit (Graphic Processing Unit). The use of CPU to complete algorithms with large data volumes and large computational scales is faced with low speed, while GPU has been widely used in network training and reasoning. However, GPU still has the disadvantages of high power consumption and low performance-to-power ratio, which limits its application in embedded and low-power fields. The main problem of ASIC (Application Specific Integrated Circuit) is that it is only designed for the current mainstream convolutional neural network, but new networks are emerging in an endless stream, and new networks may bring new computational feature operators. If ASIC cannot support many new operators, it will lead to low computing performance when processing such networks. Field Programmable Gate Array (FPGA) is a programmable semi-custom chip that allows users to quickly adjust the circuit according to the required logical functions. Compared with CPU and GPU, the FPGA-based convolutional neural network acceleration implementation has a higher performance-to-power ratio. At the same time, due to the programmable characteristics of FPGA, for the newly emerging convolutional network computing features, the computing features can be remapped on the FPGA by changing the hardware acceleration architecture.

[0004] However, the traditional FPGA-based deep convolutional neural network acceleration method is to perform hardware design based on a specific neural network, and complete the mapping of the network to the FPGA hardware by writing a hardware description language. For example, Chinese patent CN118839737A discloses a compressed YOLOv4-Tiny network accelerator based on ZYNQ. The advantage of this method is that it can fully optimize the design for a specific network, so it has the advantages of high performance, high resource utilization, and low power consumption. However, the disadvantages of this method are also very obvious. The design method based on hardware description language has the disadvantages of long development cycle, large design investment, and high risk. As new networks continue to emerge, once it is decided to replace a new convolutional neural network, the entire hardware circuit will need to be redesigned, which requires a lot of work overhead. Summary of the invention

[0005] Based on the above analysis, it is necessary to propose a general CNN hardware accelerator based on ZYNQ, in which the PL completes the operation of basic operators and the PS side builds the network. This reduces the time cost of mainstream convolutional neural network deployment and reasoning at the edge. The technical problem to be solved by the present invention is to provide a general CNN accelerator based on ZYNQ and its use method.

[0006] In a first aspect of the present invention, the present invention proposes a general CNN accelerator based on ZYNQ, wherein the general CNN accelerator includes a PS end, a PL end, an AXI bus interface module, a DMA data handling module, a PS_DDR memory and a PL_DDR memory; wherein:

[0007] The PS end is used to read external feature data and weight data, store the feature data and weight data in the PS_DDR memory, and transmit them to the PL_DDR memory through the DMA data handling module; perform network decomposition through the network decomposition module to obtain address information, and send the address information and operating parameters to the PL end through the AXI bus interface module;

[0008] The PL end is used to read the feature data and weight data of the PL_DDR memory according to the received address information, perform convolution layer and pooling layer operations through the PE calculation operator module according to the received operating parameters, generate post data, and store the post data in the PL_DDR memory; the PE calculation operator module includes a basic convolution operation operator and an extended convolution operation operator, a maximum pooling operator and an average pooling operator.

[0009] In a second aspect of the present invention, the present invention also proposes a method for using a general CNN accelerator based on ZYNQ, the method comprising:

[0010] Step 1: The PS side obtains external feature data and weight data, stores them in the PS_DDR memory, and transfers them to the PL_DDR memory through the DMA data handling module;

[0011] Step 2: The PS end performs network decomposition through the decomposition network module to obtain address information, and sends the address information and operating parameters to the PL end through the AXI bus interface module; the address information includes the address of the feature data and the address of the weight data;

[0012] Step 3: The PL side reads the feature data and weight data of the PL_DDR memory according to the received address information;

[0013] Step 4: The PL side performs convolution layer and pooling layer operations through the PE calculation operator module according to the received operating parameters, and updates the feature data to the PL_DDR memory; the PE calculation operator module includes a basic convolution operation operator and an extended convolution operation operator, a maximum pooling operator and an average pooling operator;

[0014] Step 5: Repeat steps 2 to 4. When the convolution layer and pooling layer are all completed, transfer the final result to the PS_DDR memory through AXI_DMA for storage.

[0015] The beneficial effects of the present invention are:

[0016] The present invention proposes a general CNN accelerator based on ZYNQ and a method for using the same. A decomposition network is constructed on the PS side to obtain address information and operating parameters, and the information is sent to the PL side to schedule PE operators to complete operations and memory data exchange, thereby completing the accelerated operations of the convolution layer and the pooling layer. Finally, the full connection layer operations and network post-processing operations are completed on the PS side, thereby greatly improving the efficiency and versatility of the deployment of convolutional neural networks on the edge. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the mechanisms shown in these drawings without paying any creative work.

[0018] Figure 1 Schematic diagram of the structure of a general CNN accelerator based on ZYNQ according to an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of the structure of a general CNN accelerator based on ZYNQ according to a preferred embodiment of the present invention;

[0020] Figure 3 Schematic diagram of the structure of a PE calculation operator module according to an embodiment of the present invention;

[0021] Figure 4 This is a flow chart of a method for using a general CNN accelerator based on ZYNQ according to an embodiment of the present invention;

[0022] Figure 5 A schematic diagram of a data storage format according to an embodiment of the present invention;

[0023] Figure 6 This is a schematic diagram of the workflow of a general CNN accelerator based on ZYNQ according to another embodiment of the present invention;

[0024] Figure 7 A schematic diagram of DDR memory partitioning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] Figure 1 FIG. 1 is a schematic diagram of a general CNN accelerator based on ZYNQ according to an embodiment of the present invention. Figure 1 As shown, the general CNN accelerator includes a PS end, a PL end, an AXI bus interface module, a DMA data handling module, a PS_DDR memory and a PL_DDR memory; wherein:

[0027] The PS end is used to read external feature data and weight data, store the feature data and weight data in the PS_DDR memory, and transmit them to the PL_DDR memory through the DMA data handling module; perform network decomposition through the network decomposition module to obtain address information, and send the address information and operating parameters to the PL end through the AXI bus interface module;

[0028] The PL end is used to read the feature data and weight data of the PL_DDR memory according to the received address information, perform convolution layer and pooling layer operations through the PE calculation operator module according to the received operating parameters, generate post data, and store the post data in the PL_DDR memory; the PE calculation operator module includes a basic convolution operation operator and an extended convolution operation operator, a maximum pooling operator and an average pooling operator.

[0029] This embodiment obtains address information and operating parameters by constructing a decomposition network on the PS side, and sends this information to the PL side to schedule PE operators to complete calculations and memory data exchange, thereby completing the accelerated calculations of the convolution layer and the pooling layer, thereby greatly improving the efficiency and versatility of the deployment of convolutional neural networks on the edge.

[0030] Figure 2 FIG. 1 is a schematic diagram of the structure of a general CNN accelerator based on ZYNQ according to a preferred embodiment of the present invention. Figure 2 As shown, the general CNN accelerator includes a PS end, a PL end, an AXI bus interface module, a DMA data handling module, a PS_DDR memory and a PL_DDR memory; wherein:

[0031] The PS end is used to read external feature data and weight data, store the feature data and weight data in the PS_DDR memory, and transmit them to the PL_DDR memory through the DMA data handling module; perform network decomposition through the network decomposition module to obtain address information, and send the address information and operating parameters to the PL end through the AXI bus interface module; the PS end also includes a task processing module, which includes a network pre-processing module, a fully connected layer calculation module, an activation function calculation module, and a network post-processing module.

[0032] The AXI bus interface module connects the FPGA and the CPU and is used for exchanging data between the two parties, specifically exchanging data between the PS-side on-chip storage and the PL-side on-chip storage and between the PL_DDR memory and the PS_DDR.

[0033] Exemplarily, the PL side can be implemented using FPGA, and the FPGA part plays the role of hardware accelerator and programmable logic unit in ZYNQ. It is based on the Xilinx FPGA architecture, and users can define its internal logic circuits through programming to achieve specific hardware functions. The FPGA part communicates and exchanges data with the PS part through the AXI bus. The AXI bus provides a high-bandwidth, low-latency connection, allowing data and control signals to be efficiently transmitted between the PS and PL.

[0034] Exemplarily, the PS side can be implemented using a CPU, which plays the role of an application processor in ZYNQ. It runs the operating system and applications, and is responsible for processing complex computing tasks and managing system resources. The PS part communicates and exchanges data with the PL part through the AXI bus. At the same time, the PS also provides a wealth of peripheral interfaces (such as UART, SPI, I2C, etc.) for connecting and communicating with external devices.

[0035] The PL end is used to read the feature data and weight data of the PL_DDR memory according to the received address information, and perform convolution layer and pooling layer operations through the PE calculation operator module according to the received operating parameters to generate post data, and store the post data in the PL_DDR memory; the PE calculation operator module includes a basic convolution operation operator and an extended convolution operation operator, a maximum pooling operator and an average pooling operator. The PL end also includes a control register, a status register, an address scheduler, a general register, a pre-fetch scheduling module, and a BRAM cache module;

[0036] The control register module is used to receive the address information and operation parameters of the AXI bus interface module, configure the operation parameters into the general register, and send the address information of the data to be scheduled into the address scheduler.

[0037] Exemplarily, the operating parameters include data parallelism P d , channel parallelism P c , DDR read / write burst length Burst_len, whether convolution is filled, convolution step length Stride, pooling method, convolution kernel size and other parameters. These parameters can be configured in general registers, and the address information of the data to be scheduled can be placed in the address scheduler.

[0038] The address scheduler is used to schedule the address information obtained by decomposing the convolutional neural network from the PS side so as to exchange data with the PL_DDR memory;

[0039] The address information in the address scheduler is determined by the operation parameters calculated by the decomposition network module of the PS end through the address mapping scheduling algorithm to decompose the network.

[0040] The status register module is used to indicate whether the current process of the convolution layer operation and the pooling layer operation is completed, and to indicate whether all processes of the convolution layer operation and the pooling layer operation of the convolutional neural network are completed.

[0041] Among them, by querying the status register module, the PS end and the PL end can form a pipeline, thereby greatly improving the operating efficiency of the entire accelerator.

[0042] The general register is used to configure the operating parameters including the data parallelism P d , channel parallelism P c , DDR read / write burst length Burst_len, whether convolution is filled, convolution step length Stride, pooling method, and convolution kernel size;

[0043] The pre-fetch scheduling module is used to schedule the address information in the address scheduler to perform data exchange with the PL_DDR memory.

[0044] The pre-fetch scheduling module can pre-fetch addresses in the read address scheduler, thereby performing data exchange with the PS_DDR memory and the PL_DDR memory.

[0045] The on-chip BRAM buffer area module is used to store input weight data, input feature map data, intermediate weight data, output feature map data, and index information of address information.

[0046] Among them, by caching these feature map data, it is easier to pre-fetch data, thereby completing the accelerated operations of the convolution layer and the pooling layer, greatly improving the efficiency and versatility of the deployment of convolutional neural networks at the edge.

[0047] In a preferred embodiment of the present invention, the PE calculation operator module, in order to be applicable to most convolutional neural networks, includes 1×1, 2×2, 3×3, 5×5, and 7×7 convolution operators, wherein 1×1, 2×2, and 3×3 are basic convolution kernels, and 5×5 and 7×7 convolution kernels are extended convolution kernels, which are composed of 1×1, 2×2, and 3×3 splicing methods. In the 5×5 convolution kernel, positions 1-3, 6-8, and 11-13 are composed of 3×3 convolution kernels, positions 4, 5, 16, and 21 are composed of 1×1 convolution kernels, and positions 9, 10, 14, 15 and 17, 18, 22, 23 and 19, 20, 24, and 25 are composed of 2×2 convolution kernels.

[0048] In the 7×7 convolution kernel, positions 1-5, 8-12, 15-19, 22-26, 29-33 are composed of 5×5 convolution kernels, positions 6-7, 36, 43 are composed of 1×1 convolution kernels, and positions 13-14 and 20-21, 27-28 and 34-35, 37-38 and 44-45, 39-40 and 46-47, 41-42 and 48-49 are composed of 2×2 convolution kernels. Among them, the 5×5 and 7×7 convolution kernels can also undertake the calculation of 3×3, 2×2 and 1×1 convolution, thereby improving the acceleration performance. The specific convolution kernel operators are as follows: Figure 3 As shown, the convolution kernel operator can undertake the work of calculating a*b+c. After receiving the corresponding command, it starts to read the data of the on-chip BRAM buffer module and executes the calculation to generate the part and the calculation result. After the calculation is completed, it returns a completion signal to the state controller module.

[0049] Figure 4 is a flow chart of a method for using a general CNN accelerator based on ZYNQ according to an embodiment of the present invention, such as Figure 4 As shown, the method includes:

[0050] Step 1: The PS side obtains external feature data and weight data, stores them in the PS_DDR memory, and transfers them to the PL_DDR memory through the DMA data handling module;

[0051] In some embodiments, the external feature data can come from external sensors (such as cameras, temperature sensors, etc.), and communicate with the sensors through corresponding interfaces (such as I2C, SPI, UART, etc.); it can also come from external memory (such as SD card, DDR3, NAND Flash, etc.), and the PS side can access these weight data through AXI interfaces (such as AXI HP, AXI GP, etc.).

[0052] In particular, this embodiment uses an SD card as an external memory, reads feature data and weight data from the external memory, and stores them in the PS_DDR memory; the feature data here can be image data, and the weight data here is parameter data of the network model, and these parameters are used to adjust and learn the behavior of the model so as to effectively map the input data and extract useful features.

[0053] Step 2: The PS end performs network decomposition through the decomposition network module to obtain address information, and sends the address information and operating parameters to the PL end through the AXI bus interface module; the address information includes the address of the feature data and the address of the weight data;

[0054] In the embodiment of the present invention, the PS end performs network decomposition by decomposing the network module to obtain address information, including:

[0055] Step 21: The PS terminal sets the operating parameters; the operating parameters at least include the batch data volume N, the data parallelism P d , channel parallelism P c , DDR read / write burst length Burst_len, whether convolution is filled P, convolution step size S, pooling method, convolution kernel K i Size and input feature map size (I H , I W , I C );

[0056] Step 22: The PS side calculates the decomposition convolution parameters according to the operating parameters; the decomposition convolution parameters include the output feature map size (H, W, C), the number of bias burst reads, Convolution burst read count

[0057] In the embodiment of the present invention, the feature map is stored in the memory in the form of row expansion. In order to adapt to the burst mode of the off-chip memory, zeros are added after each row of the feature map and pixel that is less than the burst length to make it an integer multiple of the Burst_len length. Figure 5 shown.

[0058] Step 23, the PS end pre-reads the feature data and weight data stored in PL_DDR in a hierarchical manner according to the decomposed convolution parameters, obtains and updates the address information for fetching the feature data and the address information for fetching the weight data.

[0059] The PS end pre-fetches the feature data and weight data stored in the PL_DDR in a hierarchical manner according to the decomposed convolution parameters, and obtains and updates the address information of the feature data and the address information of the weight data, including:

[0060] The first layer reads a batch of data each time, and reads N batches in total. Each batch of data is executed according to the second to fourth layers respectively;

[0061] The second level, through Burst read operations, each time pre-fetching C convolution bias parameters from the bias parameter address storing weight data, and storing them in the BRAM cache on the PL side; burst read operations, each time prefetching P from the convolution parameter address storing weight data c convolution kernel convolution parameters and store them in the BRAM cache on the PL side; the offset parameter address is the initial offset The convolution parameter address is the initial convolution

[0062] The third level, through Read operations, each time pre-fetching P from the address where the feature data is stored d The characteristic data of the length of Burst_len is stored in the BRAM cache of the PL side; the address of storing the characteristic data is the initial

[0063] The fourth floor, through Burst read operations are performed each time, and feature data of length Burst_len is pre-fetched from the address where the feature data is stored, and stored in the BRAM cache on the PL side; the address where the feature data is stored is the initial feature

[0064] It can be understood that for each convolutional layer, the network can be decomposed according to the first to fourth layers mentioned above, so that the general convolutional layer can be processed to achieve CNN accelerated calculation.

[0065] The embodiment of the present invention can face different convolutional neural networks according to the set operating parameters, without modifying the PL end, and only needs to construct and decompose the network model on the PS end. Any convolutional layer can be decomposed according to the four-layer decomposition, so as to control the PL end to complete the forward reasoning of the convolutional neural network, thereby greatly improving the efficiency and versatility of the deployment of the convolutional neural network at the edge.

[0066] Step 3: The PL side reads the feature data and weight data of the PL_DDR memory according to the received address information;

[0067] Step 4: The PL side performs convolution layer and pooling layer operations through the PE calculation operator module according to the received operating parameters, and updates the feature data to the PL_DDR memory; the PE calculation operator module includes a basic convolution operation operator and an extended convolution operation operator, a maximum pooling operator and an average pooling operator;

[0068] Step 5: Repeat steps 2 to 4. When the convolution layer and pooling layer are all completed, transfer the final result to the PS_DDR memory through AXI_DMA for storage.

[0069] In a preferred embodiment of the present invention, the method further comprises:

[0070] Before step 2, the PS performs network pre-processing;

[0071] After step 5, the PS performs full connection layer calculation, activation function calculation, and network post-processing based on the received operating parameters.

[0072] The embodiment of the present invention completes some network pre-processing before decomposing the network. After decomposing the network, it can complete the full connection layer calculation, activation function calculation and network post-processing, thereby greatly improving the efficiency and versatility of the deployment of convolutional neural networks at the edge.

[0073] Figure 6 is a flow chart of a method for using a general CNN accelerator based on ZYNQ according to an embodiment of the present invention, such as Figure 6 As shown, the method includes:

[0074] Schematic diagram of the workflow of the general CNN accelerator based on ZYNQ. In order to better describe this solution, the workflow of this solution will be explained with specific examples below.

[0075] Assume that the size of an input feature map is 1*28*28*1, the size of the convolution kernel is 6*5*5*1, the Padding is 0, the Stride is 1, the DDR read / write burst length Burst_len is 8, and the number of data quantization bits N is 16.

[0076] 1) Put the model weight data and image feature data into the external storage SD card. The PS side reads the data from the external storage SD card and stores it into PS_DDR after expanding it according to the integer multiple of the burst length.

[0077] Among them, a memory partition of PS_DDR is as follows Figure 7 As shown, it includes the memory area CodeHeap (code heap) for storing or managing codes, the number of channels or the width of each channel C_W (Channel Width), the channel set C_B (Channel Batch) of a batch of images or feature maps, the width F_W (FeatureWidth) of the feature map, a group of feature maps F_B (Feature Batch) processed simultaneously, the pixel value Pixel_in (input pixel) of the input image or data, the feature map Featuremap (feature map), the pixel value Pixel_out (output pixel) of the output image or data, and some form of remixing or adjustment of pixel parameters Pixel_para_remix (pixel parameter remixing), etc. This information covers the basic concepts and operations in image processing, deep learning, and convolutional neural networks. By storing this information, it is convenient to pre-read the feature data and weight data stored in PS_DDR in a hierarchical manner, and obtain and update the address information of the feature data and the address information of the weight data.

[0078] 2) Transfer the data in PS_DDR to PL_DDR through the DMA data transfer module.

[0079] 3) Configure the control registers through the AXI bus to control the general register parameter configuration, for example, Padding is 0, Stride is 1, the convolution kernel size is 6*5*5, the output feature map size is 1*24*24*6, set the data parallelism Pd to 2, and the channel parallelism Pc to 2.

[0080] 4) Build this convolutional layer on the PS side, decompose it according to the decomposition convolutional neural network address mapping scheduling algorithm, and pre-fetch C = 5 convolution biases to the on-chip bram cache Burst reads. Since the data parallelism Pd and channel parallelism Pc are 2, when decomposing this convolution layer, first set the addresses for data fetching and model parameter fetching to 0xA900000 and 0x3100000 respectively. Since data storage is expanded according to integer multiples of the burst length. Therefore, the i=0th pre-fetch model parameter address is 0x3100000+Burst_len*i, until the i=4th pre-fetch model parameter address is 0x3100000+Burst_len*i, completing the reading of the convolution kernel parameters, and so on. Calculating a convolution operation requires 5*5=25 data. Since the data parallelism Pd is 2, one pre-fetch is equivalent to pre-fetching two rows of data. A total of 14 pre-reads are required to complete the calculation of this feature map vertically, and a total of pre-reads are required for horizontal calculations. The data pre-fetch address is 0xA900000+Burst_len*i. The generated intermediate data and pre-fetched data are stored in the on-chip BRAM of the PL side.

[0081] 5) When the generated intermediate feature map data meets the pooling size, the operation is performed according to the set maximum pooling or average pooling. The address for storing the intermediate feature map is set to 0xF100000. After pooling, the feature map data generated is expanded according to the integer multiple of the burst length, and then stored in PL_DDR from PS_DDR through the DMA data transfer module. After the convolution operation is completed, the size of the pooling is 1*12*12*6, and the address mapped to PL_DDR is 0xF100000-0xF100000+6*2*Burst_len*6.

[0082] 6) The address scheduler schedules the above addresses to fetch data, completes the acceleration operation and pooling operation of the convolution calculation, and then stores the data in PL_DDR according to the address.

[0083] 7) Follow the above steps until the convolution and pooling calculations of all channels are completed.

[0084] 8) Transfer data from PL_DDR to PS_DDR via DMA.

[0085] The PS side takes out data from PS_DDR to complete the fully connected layer calculation and the post-processing operations of the convolutional neural network to complete the reasoning of the convolutional neural network.

[0086] Through the above embodiments, network pre-processing can be completed before decomposing the network. After decomposing the network, full connection layer calculation, activation function calculation and network post-processing can be completed, thereby greatly improving the efficiency and versatility of convolutional neural network deployment at the edge.

[0087] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, which can include: ROM, RAM, disk or CD, etc.

[0088] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A general CNN accelerator based on ZYNQ, characterized by: The general CNN accelerator includes a PS end, a PL end, an AXI bus interface module, a DMA data handling module, a PS_DDR memory and a PL_DDR memory; wherein: The PS end is used to read external feature data and weight data, store the feature data and weight data in the PS_DDR memory, and transmit them to the PL_DDR memory through the DMA data handling module; perform network decomposition through the network decomposition module to obtain address information, and send the address information and operating parameters to the PL end through the AXI bus interface module; The PL end is used to read the feature data and weight data of the PL_DDR memory according to the received address information, perform convolution layer and pooling layer operations through the PE calculation operator module according to the received operating parameters, generate post data, and store the post data in the PL_DDR memory; the PE calculation operator module includes a basic convolution operation operator and an extended convolution operation operator, a maximum pooling operator and an average pooling operator.

2. A ZYNQ-based general CNN accelerator according to claim 1, characterized in that: The PS end also includes a task processing module, which includes a network pre-processing module, a fully connected layer calculation module, an activation function calculation module, and a network post-processing module.

3. A ZYNQ-based general CNN accelerator according to claim 1, characterized in that: The PL end also includes a control register, a status register, an address scheduler, a general register, a pre-fetch scheduling module, and a BRAM cache module; The control register module is used to receive the address information and operating parameters of the AXI bus interface module, configure the operating parameters into the general register, and send the address information of the data to be scheduled into the address scheduler; The status register module is used to indicate whether the current process of the convolution layer operation and the pooling layer operation is completed, and whether all processes of the convolution layer operation and the pooling layer operation of the convolutional neural network are completed; The address scheduler is used to schedule the address information obtained by decomposing the convolutional neural network from the PS side so as to exchange data with the PL_DDR memory; The general register is used to configure the operating parameters including the data parallelism P d , channel parallelism P c , DDR read / write burst length Burst_len, whether convolution is filled, convolution step length Stride, pooling method, and convolution kernel size; The pre-fetch scheduling module is used to schedule the address information in the address scheduler to exchange data with the PL_DDR memory; The on-chip BRAM buffer area module is used to store input weight data, input feature map data, intermediate weight data, output feature map data, and index information of address information.

4. A ZYNQ-based general CNN accelerator according to claim 3, characterized in that: The address information in the address scheduler is determined by the operation parameters calculated by the decomposition network module of the PS end through the address mapping scheduling algorithm to decompose the network.

5. A ZYNQ-based general CNN accelerator according to claim 1 or 3, characterized in that: The extended convolution operation operator is obtained by splicing the basic convolution operation operators; the basic convolution operation operators at least include 1×1, 2×2, and 3×3 convolution operation operators, and the extended convolution operation operator includes 5×5 and 7×7 convolution operation operators.

6. A method for using a general CNN accelerator based on ZYNQ, characterized in that: The method comprises: Step 1: The PS side obtains external feature data and weight data, stores them in the PS_DDR memory, and transfers them to the PL_DDR memory through the DMA data handling module; Step 2: The PS end performs network decomposition through the decomposition network module to obtain address information, and sends the address information and operating parameters to the PL end through the AXI bus interface module; the address information includes the address of the feature data and the address of the weight data; Step 3: The PL side reads the feature data and weight data of the PL_DDR memory according to the received address information; Step 4: The PL side performs convolution layer and pooling layer operations through the PE calculation operator module according to the received operating parameters, and updates the feature data to the PL_DDR memory; the PE calculation operator module includes a basic convolution operation operator and an extended convolution operation operator, a maximum pooling operator and an average pooling operator; Step 5: Repeat steps 2 to 4. When the convolution layer and pooling layer are all completed, transfer the final result to the PS_DDR memory through AXI_DMA for storage.

7. The method for using a ZYNQ-based general CNN accelerator according to claim 6, characterized in that: The PS end performs network decomposition by the network decomposition module to obtain address information including: Step 21: The PS terminal sets the operating parameters; the operating parameters at least include the batch data volume N, the data parallelism P d , channel parallelism P c , DDR read / write burst length Burst_len, whether convolution is filled P, convolution step size S, pooling method, convolution kernel K i Size and input feature map size (I H , I W , I C ); Step 22: The PS side calculates the decomposition convolution parameters according to the operating parameters; the decomposition convolution parameters include the output feature map size (H, W, C), the number of bias burst reads, Convolution burst read count Step 23, the PS end pre-reads the feature data and weight data stored in PL_DDR in a hierarchical manner according to the decomposed convolution parameters, obtains and updates the address information for fetching the feature data and the address information for fetching the weight data.

8. The method for using a ZYNQ-based general CNN accelerator according to claim 7, characterized in that: The PS end pre-fetches the feature data and weight data stored in the PL_DDR in a hierarchical manner according to the decomposed convolution parameters, and obtains and updates the address information of the feature data and the address information of the weight data, including: The first layer reads a batch of data each time, and reads N batches in total. Each batch of data is executed according to the second to fourth layers respectively; The second level, through Burst read operations, each time pre-fetching C convolution bias parameters from the bias parameter address storing weight data, and storing them in the BRAM cache on the PL side; burst read operations, each time prefetching P from the convolution parameter address storing weight data c Convolution kernel convolution parameters, and store them in the BRAM cache on the PL side; the offset parameter address is the initial offset parameter address The convolution parameter address is the initial convolution parameter address + The third level, through Read operations, each time pre-fetching P from the address where the feature data is stored d The characteristic data of the length of Burst_len is stored in the BRAM cache of the PL side; the address storing the characteristic data is the initial data address The fourth floor, through Burst read operation, each time pre-fetching feature data of length Burst_len from the address storing feature data, and storing them in the BRAM cache on the PL side; where the address storing feature data is the initial feature data address 9. The method for using a ZYNQ-based general CNN accelerator according to claim 6, characterized in that: The method further comprises: Before step 2, the PS performs network pre-processing; After step 5, the PS performs full connection layer calculation, activation function calculation, and network post-processing based on the received operating parameters.

Images