A zynq-based general cnn accelerator and a method of using the same

Abstract

The application relates to the field of deploying and accelerating reasoning of convolutional neural networks on edge devices, in particular to a general CNN accelerator based on ZYNQ and a use method thereof, the general CNN accelerator comprising a PS end, a PL end, an AXI bus interface module, a DMA data carrying module, a PS_DDR memory and a PL_DDR memory. The application can face different convolutional neural networks, and the ZYNQ platform can be set according to parameters, the PL end does not need to be modified, the network model only needs to be constructed and decomposed on the PS end, and the PL end is controlled to complete forward reasoning of the convolutional neural network, so that the efficiency and universality of the convolutional neural network in edge deployment are greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of accelerators, in particular to a general CNN accelerator based on ZYNQ and a use method thereof. BACKGROUND

[0002] In recent years, with the development of artificial intelligence, deep neural networks have achieved remarkable results in the fields of character recognition, target detection, face recognition, speech recognition, etc. With the complexity of application scenarios, the model of deep neural networks is constantly increasing, requiring a large amount of storage and computation. Therefore, how to accelerate and implement deep neural network algorithms on hardware has become an important problem in the field of machine learning.

[0003] In the early days, the acceleration of convolutional neural networks was mainly completed by CPUs (Central Processing Unit) and GPUs (Graphic Processing Unit). Using CPUs to complete the algorithm with large data volume and computation scale faces the problem of low speed, while GPUs are widely used in network training and inference. However, GPUs still have the disadvantages of high power consumption and low performance-to-power ratio, which limits their application in embedded and low-power fields. The main problem of ASIC (Application Specific Intergrated Circuit) is that it is designed only for the current mainstream convolutional neural network, but new networks are emerging all the time, and new networks may bring new computing feature operators. If ASIC cannot support many new operators, it will result in low computing performance when processing such networks. FPGA (Field Programmable Gate Array) is a programmable semi-custom chip, and users can quickly adjust the circuit according to the required logic function. Compared with CPUs and GPUs, the implementation of convolutional neural networks based on FPGA has a higher performance-to-power ratio. At the same time, due to the programmable characteristics of FPGA, for new convolutional network computing features, the hardware acceleration architecture can be changed to realize the remapping of the computing feature on FPGA.

[0004] But the traditional FPGA-based deep convolutional neural network acceleration method is to design hardware according to a specific neural network, complete the mapping of the network to the FPGA hardware by writing hardware description language, for example, Chinese patent CN118839737A discloses a compressed YOLOv4-Tiny network accelerator based on ZYNQ. The advantage of this method is that it can be fully optimized for a specific network, so it has the advantages of high performance, high resource utilization and low power consumption. However, this method has obvious disadvantages. The design method based on hardware description language has the disadvantages of long development cycle, large design investment and high risk. Since new networks are constantly emerging, once a new convolutional neural network is decided to be replaced, the entire hardware circuit needs to be redesigned, which requires a lot of work overhead. SUMMARY

[0005] Based on the above analysis, it is necessary to propose a ZYNQ-based general CNN hardware accelerator, wherein the PL completes the operation of the basic operator, and the PS end builds the network. Thus, the time cost of deploying and reasoning of mainstream convolutional neural networks on the edge is reduced. The technical problem to be solved by the present application is to provide a ZYNQ-based general CNN accelerator and a use method thereof.

[0006] In a first aspect of the present application, a ZYNQ-based general CNN accelerator is provided, which comprises a PS end, a PL end, an AXI bus interface module, a DMA data carrying module, a PS_DDR memory and a PL_DDR memory; wherein:

[0007] The PS end is used to read external feature data and weight data, store the feature data and the weight data into the PS_DDR memory, and transmit the feature data and the weight data to the PL_DDR memory through the DMA data carrying module; the address information is obtained by decomposing the network module, and the address information and the running parameters are sent to the PL end through the AXI bus interface module;

[0008] The PL end is used to read the feature data and the weight data of the PL_DDR memory according to the received address information, perform convolution layer and pooling layer operation through the PE calculation operator module according to the received running parameters, generate post data, and store the post data into the PL_DDR memory; the PE calculation operator module comprises a basic convolution operation operator and an extended convolution operation operator, a maximum pooling operator and an average pooling operator.

[0009] In a second aspect of the present application, a use method of the ZYNQ-based general CNN accelerator is also provided, which comprises:

[0010] Step 1: The PS end acquires external feature data and weight data and stores them in the PS_DDR memory, and transmits them to the PL_DDR memory through a DMA data carrying module;

[0011] Step 2: The PS end obtains address information through a network decomposition module, and sends the address information and operation parameters to the PL end through an AXI bus interface module; the address information includes the address of the feature data and the address of the weight data;

[0012] Step 3: The PL end reads the feature data and the weight data of the PL_DDR memory according to the received address information;

[0013] Step 4: The PL end performs convolution layer and pooling layer operations through a PE calculation operator module according to the received operation parameters, and updates the feature data to the PL_DDR memory; the PE calculation operator module includes a basic convolution operation operator and an extended convolution operation operator, a maximum pooling operator and an average pooling operator;

[0014] Step 5: Steps 2-4 are repeatedly executed, and when the convolution layer and the pooling layer are all completed, the final result is transmitted to the PS_DDR memory for storage through AXI_DMA.

[0015] The beneficial effects of the present application are:

[0016] The general CNN accelerator based on ZYNQ and the use method thereof proposed in the present application construct a decomposition network in the PS end to obtain address information and operation parameters, send these information to the PL end to schedule PE operators to complete operations and memory data exchange, so as to complete the accelerated operation of the convolution layer and the pooling layer, and finally complete the full connection layer operation and the network post-processing operation in the PS end, thereby greatly improving the efficiency and universality of the convolutional neural network deployed on the edge end. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the drawings shown.

[0018] Figure 1 The structure schematic diagram of the general CNN accelerator based on ZYNQ of the embodiment of the present application;

[0019] Figure 2 The structure schematic diagram of the general CNN accelerator based on ZYNQ of the preferred embodiment of the present application;

[0020] Figure 3 Figure 1 is a schematic diagram of a PE computing operator module structure according to an embodiment of the present application;

[0021] Figure 4 Figure 1 is a schematic diagram of a PE computing operator module structure according to an embodiment of the present application;

[0022] Figure 5 Figure 1 is a schematic diagram of a PE computing operator module structure according to an embodiment of the present application;

[0023] Figure 6 Figure 1 is a schematic diagram of a PE computing operator module structure according to an embodiment of the present application;

[0024] Figure 7 Figure 1 is a schematic diagram of a PE computing operator module structure according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0026] Figure 1 Figure 1 is a schematic diagram of a PE computing operator module structure according to an embodiment of the present application. Figure 1 As shown in Figure 1, the general CNN accelerator includes a PS end, a PL end, an AXI bus interface module, a DMA data carrying module, a PS_DDR memory and a PL_DDR memory; wherein:

[0027] The PS end is configured to read external feature data and weight data, store the feature data and the weight data into the PS_DDR memory, and transmit the feature data and the weight data to the PL_DDR memory through the DMA data carrying module; address information is obtained by decomposing the network through a network decomposition module, and the address information and running parameters are sent to the PL end through the AXI bus interface module.

[0028] The PL end is configured to read the feature data and the weight data of the PL_DDR memory according to the received address information, perform convolution layer and pooling layer operations through a PE computing operator module according to the received running parameters, generate post data, and store the post data into the PL_DDR memory; the PE computing operator module includes a basic convolution operation operator and an extended convolution operation operator, a maximum pooling operator and an average pooling operator.

[0029] The embodiment obtains address information and operation parameters by constructing a decomposition network at the PS end, sends the information to the PL end scheduling PE operator to complete operation and memory data exchange, thereby completing the accelerated operation of the convolution layer and the pooling layer, and greatly improving the efficiency and versatility of the convolutional neural network deployed at the edge.

[0030] Figure 2 A schematic diagram of a general CNN accelerator structure based on ZYNQ according to a preferred embodiment of the application is shown in Figure 2 As shown, the general CNN accelerator includes a PS end, a PL end, an AXI bus interface module, a DMA data transfer module, a PS_DDR memory and a PL_DDR memory; wherein:

[0031] The PS end is configured to read external feature data and weight data, store the feature data and the weight data in the PS_DDR memory, and transmit the feature data and the weight data to the PL_DDR memory through the DMA data transfer module; the address information is obtained through the decomposition network module, and the address information and the operation parameters are sent to the PL end through the AXI bus interface module; the PS end further includes a task processing module, and the task processing module includes a network pre-processing module, a fully connected layer calculation module, an activation function calculation module and a network post-processing module.

[0032] The AXI bus interface module is connected with the FPGA and the CPU, and is configured to exchange data between the two, specifically, exchange data between the on-chip storage of the PS end and the on-chip storage of the PL end, the PL_DDR memory and the PS_DDR.

[0033] Illustratively, the PL end can be implemented by using FPGA, and the FPGA part plays the role of hardware accelerator and programmable logic unit in ZYNQ. It is based on the FPGA architecture of Xilinx, and users can define the internal logic circuit through programming to realize specific hardware functions. The FPGA part communicates and exchanges data with the PS part through the AXI bus. The AXI bus provides high bandwidth and low delay connection, so that the PS and the PL can efficiently transmit data and control signals.

[0034] Illustratively, the PS end can be implemented by using CPU, and the CPU part plays the role of application processor in ZYNQ. It runs the operating system and application program, and is responsible for processing complex computing tasks and managing system resources. The PS part communicates and exchanges data with the PL part through the AXI bus. At the same time, the PS also provides rich peripheral interfaces (such as UART, SPI, I2C, etc.), which are used to connect and communicate with external devices.

[0035] The PL end is used to read the characteristic data and weight data of the PL_DDR memory according to the received address information, to perform convolution layer and pooling layer operations through a PE calculation operator module according to the received operation parameters, to generate post-data, and to store the post-data into the PL_DDR memory; the PE calculation operator module includes a basic convolution operation operator and an extended convolution operation operator, a maximum pooling operator and an average pooling operator. The PL end further includes a control register, a state register, an address scheduler, a general register, a prefetch scheduling module and a BRAM cache area module.

[0036] The control register module is used to receive address information and operation parameters of the AXI bus interface module, to configure the operation parameters into the general register, and to send address information of data to be scheduled into the address scheduler.

[0037] For example, the operation parameters include data parallelism P d , channel parallelism P c , DDR read-write burst length Burst_len, whether convolution is filled, convolution stride, pooling mode, convolution kernel size and the like.

[0038] The address scheduler is used to schedule address information obtained by decomposing a convolutional neural network from the PS end to exchange data with the PL_DDR memory.

[0039] The address information in the address scheduler is determined by operation parameters calculated by a decomposition network module of the PS end through an address mapping scheduling algorithm.

[0040] The state register module is used to indicate whether a current process of convolution layer operation and pooling layer operation is completed, and to indicate whether all processes of convolution layer operation and pooling layer operation of the convolutional neural network are completed.

[0041] By querying the state register module, a pipeline can be formed between the PS end and the PL end, so as to greatly improve the operation efficiency of the entire accelerator.

[0042] The general register is used to configure the operation parameters including data parallelism P d , channel parallelism P c , DDR read-write burst length Burst_len, whether convolution is filled, convolution stride, pooling mode, convolution kernel size and the like.

[0043] The prefetch scheduling module is used to schedule address information in the address scheduler to exchange data with the PL_DDR memory.

[0044] The pre-fetch scheduling module can pre-fetch the address in the read address scheduler, thereby exchanging data with the PS_DDR memory and the PL_DDR memory.

[0045] The on-chip BRAM cache area module is used for storing input weight data, input feature map data, intermediate weight data, output feature map data and address information index information.

[0046] The pre-fetching of the data can be facilitated by caching the feature map data, thereby completing the accelerated operation of the convolution layer and the pooling layer and greatly improving the efficiency and versatility of the convolutional neural network in the edge deployment.

[0047] In the preferred embodiment of the application, the PE calculation operator module includes 1x1, 2x2, 3x3, 5x5 and 7x7 convolution operation operators in order to be applicable to most convolutional neural networks, wherein 1x1, 2x2 and 3x3 are basic convolution kernels, 5x5 and 7x7 are extended convolution kernels, which are composed of 1x1, 2x2 and 3x3 in a splicing manner, 4, 5, 16 and 21 are composed of 1x1 convolution kernels, 9, 10, 14, 15, 17, 18, 22, 23, 19, 20, 24 and 25 are composed of 2x2 convolution kernels in the 5x5 convolution kernel.

[0048] In the 7x7 convolution kernel, 1-5, 8-12, 15-19, 22-26, 29-33 are composed of 5x5 convolution kernels, 6-7 and 36, 43 are composed of 1x1 convolution kernels, 13-14 and 20-21, 27-28 and 34-35, 37-38 and 44-45, 39-40 and 46-47, 41-42 and 48-49 are composed of 2x2 convolution kernels. Figure 3 The convolution kernel operator can undertake the calculation of a*b+c. After receiving the corresponding command, the data in the on-chip BRAM cache area module is read and the calculation result is generated, and the completion signal is returned to the state controller module after the calculation is completed.

[0049] Figure 4 is a use method flow chart of a general CNN accelerator based on ZYNQ according to an embodiment of the application, as shown in Figure 4 The method comprises the following steps.

[0050] Step 1: The PS end acquires external feature data and weight data and stores them in the PS_DDR memory, and transmits them to the PL_DDR memory through a DMA data transfer module;

[0051] In some embodiments, the external feature data can come from an external sensor (such as a camera, a temperature sensor, etc.), and communicate with the sensor through a corresponding interface (such as I2C, SPI, UART, etc.); it can also come from an external memory (such as an SD card, DDR3, NAND Flash, etc.), and the PS end can access these weight data through an AXI interface (such as AXI HP, AXI GP, etc.).

[0052] In particular, the present embodiment uses an SD card as an external memory, reads the feature data and weight data of the external memory, and stores them in the PS_DDR memory; the feature data here can be picture data, and the weight data here is parameter data of a network model, which is used to adjust and learn the behavior of the model to effectively map and extract useful features from input data.

[0053] Step 2: The PS end obtains address information through a network decomposition module, and sends the address information and running parameters to the PL end through an AXI bus interface module; the address information includes the address of the feature data and the address of the weight data;

[0054] In the embodiment of the present application, the address information obtained by the PS end through the network decomposition module includes:

[0055] Step 21, the PS end sets running parameters; the running parameters at least include batch data amount N, data parallelism P d , channel parallelism P c , DDR read-write burst length Burst_len, whether convolution is filled P, convolution step size S, pooling method, convolution kernel K i size and input feature map size (I H , I W , I C );

[0056] Step 22: The PS end calculates decomposition convolution parameters according to the running parameters; the decomposition convolution parameters include output feature map size (H, W, C), bias burst read frequency convolution burst read frequency

[0057] In this embodiment of the invention, the feature map is stored in memory in a row-by-row format. To accommodate the burst mode of off-chip memory, any lengths less than the burst length are padded with zeros after each row of the feature map and pixels to ensure that the length is an integer multiple of the burst_len length. Figure 5 As shown.

[0058] Step 23: The PS end reads the feature data and weight data stored in PL_DDR in layers according to the decomposed convolution parameters, and obtains and updates the address information for retrieving feature data and the address information for retrieving weight data.

[0059] Specifically, the PS end, based on the decomposed convolution parameters, prefetches and reads the feature data and weight data stored in PL_DDR in layers to obtain and update the address information for retrieving feature data and the address information for retrieving weight data, including:

[0060] The first layer reads one batch of data at a time, for a total of N batches. Each batch of data is processed according to the second to fourth layers respectively.

[0061] The second layer, through Each burst read operation prefetches C convolutional bias parameters from the address of the bias parameters storing the weight data and stores them in the BRAM cache at the PL end; through Each burst read operation prefetches P from the convolution parameter address storing the weight data. c The convolution parameters of each convolution kernel are stored in the BRAM cache at the PL end; the bias parameter address is the initial bias. The convolution parameter address is the initial convolution.

[0062] The third layer, through Each read operation prefetches P from the address where the feature data is stored. d The feature data of length Burst_len is processed and stored in the BRAM cache at the PL end; the address where the feature data is stored is the initial address.

[0063] The fourth layer, through Each burst read operation prefetches feature data of length Burst_len from the address where the feature data is stored, and stores it in the BRAM cache at the PL end; where the address where the feature data is stored is the initial feature...

[0064] Understandably, for each convolutional layer, the network can be decomposed according to the first to fourth layers mentioned above. This allows for the processing of general convolutional layers and enables accelerated computation of CNNs.

[0065] The embodiment of the present application can be according to the set running parameters without modifying the PL end when facing different convolutional neural networks, and only the network model construction and decomposition are needed on the PS end, any convolutional layer can be decomposed according to four-layer decomposition, so that the PL end can complete the forward inference of the convolutional neural network, thereby greatly improving the efficiency and universality of the convolutional neural network in the edge deployment.

[0066] Step 3: The PL end reads the feature data and weight data of the PL_DDR memory according to the received address information;

[0067] Step 4: The PL end performs convolutional layer and pooling layer operations through the PE calculation operator module according to the received running parameters, and updates the feature data to the PL_DDR memory; the PE calculation operator module includes a basic convolution operation operator and an extended convolution operation operator, a maximum pooling operator and an average pooling operator;

[0068] Step 5: Steps 2-4 are repeatedly executed, and when the convolutional layer and the pooling layer are all completed, the final result is transmitted to the PS_DDR memory for storage through AXI_DMA.

[0069] In the preferred embodiment of the present application, the method further comprises:

[0070] Before step 2, the PS end performs network preprocessing;

[0071] After step 5, the PS end performs full connection layer calculation, activation function calculation and network post-processing based on the received running parameters.

[0072] The embodiment of the present application completes some network preprocessing before decomposing the network, and can complete full connection layer calculation, activation function calculation and network post-processing after decomposing the network, thereby greatly improving the efficiency and universality of the convolutional neural network in the edge deployment.

[0073] Figure 6 is a use method flowchart of a general CNN accelerator based on ZYNQ according to the embodiment of the present application, as shown in Figure 6 The method comprises:

[0074] The working flowchart of the general CNN accelerator based on ZYNQ. In order to better describe the present scheme, the working flow of the present scheme will be described below in combination with specific examples.

[0075] Suppose that an input feature map size is 1*28*28*1, a convolution kernel size is 6*5*5*1, padding is 0, stride is 1, DDR read-write burst length Burst_len is 8, and data quantization bit number N is 16.

[0076] 1) The model's weight data and the image's feature data are placed in the external storage SD card. The PS end reads the data from the external storage SD card, expands it according to an integer multiple of the burst length, and then stores it in PS_DDR.

[0077] One of the PS_DDR memory partitions is as follows: Figure 7 As shown, this includes the memory region CodeHeap for storing or managing code, the number of channels or the width of each channel C_W (Channel Width), the channel set C_B (Channel Batch), the width of the feature map F_W (Feature Width), a group of feature maps processed simultaneously F_B (Feature Batch), the pixel values ​​of the input image or data Pixel_in, the feature map Featuremap, the pixel values ​​of the output image or data Pixel_out, and some form of remixing or adjustment of pixel parameters Pixel_para_remix. This information covers the basic concepts and operations in image processing, deep learning, and convolutional neural networks. Storing this information facilitates hierarchical prefetching of feature data and weight data stored in PS_DDR, obtaining and updating the address information for retrieving feature data and weight data.

[0078] 2) Transfer the data in PS_DDR to PL_DDR through the DMA data transfer module.

[0079] 3) Configure the general-purpose register parameters through the AXI bus configuration control register. For example, set Padding to 0, Stride to 1, kernel size to 6*5*5, output feature map size to 1*24*24*6, data parallelism to Pd to 2, and channel parallelism to Pc to 2.

[0080] 4) Construct this convolutional layer on the PS side, decompose it according to the decomposed convolutional neural network address mapping scheduling algorithm, and prefetch C=5 convolutional biases to the on-chip BRAM cache. The second burst read, since the data parallelism Pd and the channel parallelism Pc are 2, when the convolution layer is decomposed, the data fetching address and the model parameter fetching address are set to 0xA900000 and 0x3100000 respectively. Since the data storage is expanded by an integer multiple of the burst length, the i = 0th prefetch model parameter address is 0x3100000 + Burst_len * i, and the i = 4th prefetch model parameter address is 0x3100000 + Burst_len * i, and the convolution kernel parameter reading is completed, and so on. One convolution operation requires 5 * 5 = 25 data, since the data parallelism Pd is 2, one prefetching is equivalent to prefetching two rows of data, and a total of 14 times of pre-reading is required for the vertical calculation of the feature map, and a total of 14 times of pre-reading is required for the horizontal calculation of the feature map The second burst read, since the data parallelism Pd and the channel parallelism Pc are 2, when the convolution layer is decomposed, the data fetching address and the model parameter fetching address are set to 0xA900000 and 0x3100000 respectively. Since the data storage is expanded by an integer multiple of the burst length, the i = 0th prefetch model parameter address is 0x3100000 + Burst_len * i, and the i = 4th prefetch model parameter address is 0x3100000 + Burst_len * i, and the convolution kernel parameter reading is completed, and so on. One convolution operation requires 5 * 5 = 25 data, since the data parallelism Pd is 2, one prefetching is equivalent to prefetching two rows of data, and a total of 14 times of pre-reading is required for the vertical calculation of the feature map, and a total of 14 times of pre-reading is required for the horizontal calculation of the feature map

[0081] 5) When the generated intermediate feature map data meets the pooling size, perform the operation according to the set maximum pooling or average pooling. Set the intermediate feature map address to 0xF100000, and after the pooling of the generated feature map data is expanded by an integer multiple of the burst length, store it in the PL_DDR through the DMA data transfer module, complete the convolution operation, and the size of the pooled feature map is 1*12*12*6, which is mapped to the address 0xF100000-0xF100000+6*2*Burst_len*6 in the PL_DDR.

[0082] 6) The above addresses are scheduled by the address scheduler to fetch data, complete the convolution calculation acceleration operation and the pooling operation, and then store the data in the PL_DDR according to the address.

[0083] 7) According to the above operation, the convolution and pooling calculation of all channels are completed.

[0084] 8) The data is transferred from the PL_DDR to the PS_DDR through the DMA.

[0085] The PS end fetches the data from the PS_DDR to complete the full connection layer calculation and the post-processing operation of the convolution neural network, and completes the inference of the convolution neural network.

[0086] Through the above embodiments, the network preprocessing can be completed before the network is decomposed, and after the network is decomposed, the full connection layer calculation, the activation function calculation and the network post-processing can be completed, thereby greatly improving the efficiency and universality of the convolution neural network deployed on the edge end.

[0087] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiments can be completed by programs instructing relevant hardware, and the programs can be stored in a computer readable storage medium, which can include ROM, RAM, magnetic disk or optical disk, etc.

[0088] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A ZYNQ-based universal CNN accelerator, characterized in that: The general CNN accelerator comprises a PS end, a PL end, an AXI bus interface module, a DMA data carrying module, a PS_DDR memory and a PL_DDR memory; wherein: The PS end is configured to read external feature data and weight data, store the feature data and the weight data into the PS_DDR memory, and transmit the feature data and the weight data to the PL_DDR memory through the DMA data carrying module; address information is obtained by decomposing the network through the network decomposition module, and the address information and operation parameters are sent to the PL end through the AXI bus interface module; The PL end is configured to read feature data and weight data of the PL_DDR memory according to the received address information, perform convolution layer and pooling layer operations through the PE calculation operator module according to the received operation parameters, generate post data, and store the post data into the PL_DDR memory; the PE calculation operator module comprises a basic convolution operation operator and an extended convolution operation operator, a maximum pooling operator and an average pooling operator; The address information obtained by decomposing the network through the network decomposition module comprises: Step 1, the PS end sets running parameters; the running parameters at least include batch data volume N, data parallelism P d , channel parallelism P c , DDR read-write burst length Burst_len, whether convolution is filled P, convolution step size S, pooling mode, convolution kernel K i size and input feature map size; Step 2, the PS end calculates the decomposition convolution parameter according to the running parameter; the decomposition convolution parameter includes an output feature map size and a bias burst read frequency Convolution burst read frequency Step 3: The PS end reads the feature data and the weight data stored in the PL_DDR in a hierarchical prefetch manner according to the decomposed convolution parameters, and obtains and updates address information of the feature data and address information of the weight data; The PS end reads the feature data and the weight data stored in the PL_DDR in a hierarchical prefetch manner according to the decomposed convolution parameters, and obtains and updates address information of the feature data and address information of the weight data, which comprises: The first layer reads one batch of data at a time, a total of N batches, and each batch of data is executed according to the second layer to the fourth layer; The second layer is read by The second layer is read by The second layer is read by c The second layer is read by The second layer is read by The third layer, by The second reading operation, each time from the address of the stored feature data to pre-fetch P d The feature data of the row Burst_len length are stored into the BRAM cache at the PL end; wherein the address of the stored feature data is the initial data address The fourth layer is through The next burst read operation, each time from the address of the stored feature data to pre-fetch the length of the Burst_len feature data, and store them in the BRAM cache at the PL end; wherein the address of the stored feature data is the initial feature data address 2.The ZYNQ-based general CNN accelerator of claim 1, wherein: The PS end further comprises a task processing module, and the task processing module comprises a network pre-processing module, a full connection layer calculation module, an activation function calculation module and a network post-processing module. 3.The ZYNQ-based general CNN accelerator of claim 1, wherein: The PL end further comprises a control register, a state register, an address scheduler, a general register, a prefetch scheduling module and a BRAM cache area module; The control register module is configured to receive address information and operation parameters of the AXI bus interface module, configure the operation parameters into the general register, and send address information of data to be scheduled into the address scheduler; The state register module is configured to indicate whether the current process of the convolution layer operation and the pooling layer operation is completed, and indicate whether all processes of the convolution layer operation and the pooling layer operation of the convolutional neural network are completed; The address scheduler is configured to schedule address information obtained by decomposing the convolutional neural network from the PS end to exchange data with the PL_DDR memory; The general register is configured to include the running parameters including data parallelism P d , channel parallelism P c , DDR read-write burst length Burst_len, whether convolution is padded, convolution stride Stride, pooling mode, and convolution kernel size. The prefetch scheduling module is configured to schedule address information in the address scheduler to exchange data with the PL_DDR memory; The BRAM cache area module is configured to store input weight data, input feature map data, intermediate weight data, output feature map data and index information of address information. 4.The ZYNQ-based general CNN accelerator of claim 3, wherein: The address information in the address scheduler is determined by the operation parameters calculated by the address mapping scheduling algorithm of the decomposition network module of the PS end.

5. The ZYNQ-based general CNN accelerator of claim 1 or 3, wherein: The extended convolution operation operator is spliced from the basic convolution operation operator; the basic convolution operation operator at least includes 1*1, 2*2 and 3*3 convolution operation operators, and the extended convolution operation operator includes 5*5 and 7*7 convolution operation operators.

6. A method for using a ZYNQ-based general CNN accelerator, characterized in that, The method comprises: Step 1: the PS end acquires external feature data and weight data and stores them in a PS_DDR memory, and transmits them to a PL_DDR memory through a DMA data carrying module; Step 2: the PS end obtains address information through a network decomposition module and sends the address information and running parameters to the PL end through an AXI bus interface module; the address information comprises addresses of the feature data and the weight data; Step 3: the PL end reads the feature data and the weight data in the PL_DDR memory according to the received address information; Step 4: the PL end performs convolution layer and pooling layer operations through a PE calculation operator module according to the received running parameters, and updates the feature data to the PL_DDR memory; the PE calculation operator module comprises a basic convolution operation operator and an extended convolution operation operator, a maximum pooling operator and an average pooling operator; Step 5: steps 2-4 are repeatedly executed, and when the convolution layer and the pooling layer are completed, the final result is transmitted to the PS_DDR memory for storage through an AXI_DMA; The address information obtained through the network decomposition module comprises: Step 21, the PS end sets running parameters; the running parameters at least include batch data volume N, data parallelism P d , channel parallelism P c , DDR read-write burst length Burst_len, whether convolution is filled P, convolution step size S, pooling mode, convolution kernel K i size and input feature map size; Step 22, the PS side calculates the decomposition convolution parameter according to the running parameter; the decomposition convolution parameter includes an output feature map size and a bias burst read times Convolution burst read times Step 23: the PS end reads the feature data and the weight data stored in the PL_DDR in a hierarchical prefetch manner according to the decomposition convolution parameters, and obtains and updates address information of the feature data and address information of the weight data; The PS end reads the feature data and the weight data stored in the PL_DDR in a hierarchical prefetch manner according to the decomposition convolution parameters, and obtains and updates address information of the feature data and address information of the weight data, which comprises: In the first layer, one batch of data is read at a time, and a total of N batches of data are read, and each batch of data is executed according to the second layer to the fourth layer; The second layer is read by The second layer is read by The second layer is read by c The second layer is read by The second layer is read by The third layer is through The second reading operation, each time from the address of the stored feature data to pre-fetch P d The feature data of the row Burst_len length are stored into the BRAM cache at the PL end; wherein the address of the stored feature data is the initial data address The fourth layer is through The next burst read operation, each time from the address of the stored feature data to pre-fetch the length of the feature data of Burst_len, and store them in the BRAM cache of the PL end; wherein the address of the stored feature data is the initial feature data address 7. The method of claim 6, wherein the method further comprises: The method further comprises: Before step 2, the PS end performs network preprocessing; After step 5, the PS end performs full connection layer calculation, activation function calculation and network post-processing based on the received running parameters.

Images