A FPGA-based CNN accelerator implementation system and method

Abstract

The present application discloses a CNN accelerator implementation system based on an FPGA chip, wherein the CNN accelerator comprises: a process control module, a data broadcast module and a plurality of processing units, wherein the process control module is respectively connected to the data broadcast module and the plurality of processing units; the data broadcast module is respectively connected to each processing unit, the processing unit is used to implement convolution calculation, and wherein the process control module is configured to perform the following operations: read parameter information from a memory, wherein the parameter information comprises the number information of convolution kernels of a convolution layer of a CNN model and / or the size information of a feature image; compare the parameter information with preset threshold information, and determine a parallel computing strategy corresponding to the plurality of processing units according to the comparison result; and control the data broadcast module and the plurality of processing units to perform corresponding operations according to the parallel computing strategy.

Description

Technical Field

[0001] The present application relates to the field of FPGA accelerator design technology, and in particular to a FPGA-based CNN accelerator implementation system and method. Background Art

[0002] Due to the efficient energy consumption and powerful performance of field programmable gate arrays (FPGAs), they have been increasingly used and explored in accelerating convolutional neural networks (CNNs). For low-power CNN edge deployment scenarios, FPGA-based CNN accelerators usually use a spatial expansion architecture for convolution acceleration. For example, the traditional method is to directly input the feature image into the CNN model for feature extraction, while the spatial expansion system design is to expand the channels of the feature image (for example, expand the feature image according to the three channels of R, G, and B), and then use the convolution kernel of the CNN model to perform parallel operations on multiple feature images after the channel expansion. Therefore, the above-mentioned spatial expansion system design can achieve the technical effect of improving computing efficiency and reducing data transmission and memory access delays.

[0003] Nevertheless, since the circuit of the CNN accelerator is fixed, that is, the parallelism of the calculation is the same for different convolutional layers. Furthermore, in order to make the acceleration effect of the CNN accelerator better, the designer will design the parallelism based on the size characteristics of most convolutional layers (that is, smaller size and more channels). However, this will inevitably sacrifice the acceleration performance of the CNN accelerator for convolutional layers with larger size and fewer channels. Therefore, when the input feature image size is large and the channels are few, the acceleration effect of the CNN accelerator is relatively poor.

[0004] Thus, in summary, a simple spatial unfolding architecture is difficult to cope with the diversity of CNN models.

[0005] With respect to the technical problem existing in the above-mentioned prior art of how to improve the adaptability of the CNN accelerator to different convolutional layers in the CNN model, thereby further ensuring the acceleration effect of the CNN accelerator on the convolutional layer, no effective solution has been proposed yet. Summary of the invention

[0006] The embodiments of the present disclosure provide a system and method for implementing an FPGA-based CNN accelerator, so as to at least solve the technical problem in the prior art of how to improve the adaptability of the CNN accelerator to different convolutional layers in the CNN model, thereby further ensuring the acceleration effect of the CNN accelerator on the convolutional layer.

[0007] According to one aspect of an embodiment of the present disclosure, a CNN accelerator implementation system based on an FPGA chip is provided, comprising: a host, an FPGA chip connected to the host, and a memory connected to the FPGA chip, wherein a CNN accelerator is deployed on the FPGA chip, and the CNN accelerator comprises: a process control module, a data broadcast module, and a plurality of processing units, wherein the process control module is respectively connected to the data broadcast module and the plurality of processing units; the data broadcast module is respectively connected to each processing unit, the processing unit is used to implement convolution calculation, and wherein the process control module is configured to perform the following operations: read parameter information from the memory, wherein the parameter information includes the number information of convolution kernels of the convolution layer of the CNN model and / or the size information of the feature image; compare the parameter information with the preset threshold information, and determine the parallel computing strategy corresponding to the plurality of processing units according to the comparison result; and control the data broadcast module and the plurality of processing units to perform corresponding operations according to the parallel computing strategy.

[0008] According to another aspect of an embodiment of the present disclosure, a method for implementing a CNN accelerator based on an FPGA chip is also provided, including: reading parameter information from a memory, wherein the parameter information includes information on the number of convolution kernels of a convolution layer of a CNN model and / or size information of a feature image; comparing the parameter information with preset threshold information, and determining a parallel computing strategy corresponding to a plurality of processing units based on the comparison result, wherein the processing units are deployed on a CNN accelerator and are used to implement convolution computing; and according to the parallel computing strategy, controlling a data broadcast module and a plurality of processing units deployed on the CNN accelerator to perform corresponding operations.

[0009] The present application discloses a CNN accelerator implementation system based on FPGA chip. The system includes a CNN accelerator, and the CNN accelerator includes a process control module, a data broadcast module and a plurality of processing units. The process control module is respectively connected to the data broadcast module and the plurality of processing units. The data broadcast module is respectively connected to each processing unit. The processing unit is used to implement convolution calculation. And the process control module is also configured to perform the following operations: First, the process control module reads parameter information from the memory. Then, the process control module compares the parameter information with the preset threshold information, and determines the parallel computing strategy corresponding to the plurality of processing units according to the comparison result. Finally, the process control module controls the data broadcast module and the plurality of processing units to perform corresponding operations according to the parallel computing strategy.

[0010] Referring to the above content, it can be seen that since the process control module reads parameter information from the memory, and the parameter information includes the number information of the convolution kernels of the convolution layer of the CNN model and / or the size information of the feature image, and further compares the read parameter information with the preset threshold information, and selects the corresponding parallel computing strategy according to the comparison result, the technical solution disclosed in the present application fully considers the number of convolution kernels of the convolution layer, and / or the size of the feature image on the CNN accelerator, so that the CNN accelerator can select the parallel computing strategy that is most suitable for the CNN accelerator under any circumstances, so as to give full play to the acceleration performance of the CNN accelerator.

[0011] Thus, the adaptability of the CNN accelerator to different convolutional layers in the CNN model is improved, and the technical effect of ensuring the acceleration performance of the CNN accelerator is achieved. Furthermore, the technical problem of how to improve the adaptability of the CNN accelerator to different convolutional layers in the CNN model, thereby further ensuring the acceleration effect of the CNN accelerator on the convolutional layer, which exists in the prior art, is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present application. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings:

[0013] Figure 1 is a hardware structure block diagram of a computing device for implementing the method according to Embodiment 1 of the present application;

[0014] Figure 2 is a schematic diagram of an accelerator implementation system based on an FPGA chip according to Embodiment 1 of the present application;

[0015] Figure 3 is a schematic diagram of the hardware structure of the data broadcast module according to Embodiment 1 of the present application;

[0016] Figure 4 It is a flowchart of the CNN accelerator implementation method based on FPGA chip according to Example 1 of the present application;

[0017] Figure 5 is a schematic diagram of multiple processing units performing calculations according to a first parallel strategy according to Embodiment 1 of the present application;

[0018] Figure 6 It is a schematic diagram of organizing and arranging the first feature picture data according to the first parallel strategy by the data broadcast module when the feature picture size is less than the first preset threshold according to Embodiment 1 of the present application;

[0019] Figure 7is a schematic diagram of the data flow design according to Example 1 of the present application;

[0020] Figure 8 is a schematic diagram of multiple processing units performing calculations according to a second parallel strategy according to Embodiment 1 of the present application;

[0021] Fig. 9 It is a schematic diagram of organizing and arranging the third feature image data according to the second parallel strategy by the processing unit when the feature image size is greater than the first preset threshold according to Example 1 of the present application. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only embodiments of a part of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present disclosure.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0024] Example 1

[0025] According to this embodiment, an embodiment of a method for implementing a CNN accelerator based on an FPGA chip is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0026] The method embodiment provided in this embodiment can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computing device for implementing a CNN calculator implementation method based on an FPGA chip. Figure 1 As shown, the computing device may include one or more processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory for storing data, a transmission device for communication functions, and an input / output interface. The memory, the transmission device, and the input / output interface are connected to the processor via a bus. In addition, it may also include: a display, a keyboard, and a cursor control device connected to the input / output interface. A person skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0027] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components of the computing device. As involved in the embodiments of the present disclosure, the data processing circuitry acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0028] The memory can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the CNN accelerator implementation method based on FPGA chips in the embodiment of the present disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, the CNN accelerator implementation method based on FPGA chips for the above-mentioned application program is implemented. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0029] The transmission device is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computing device. In one example, the transmission device includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device can be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet wirelessly.

[0030] The display may be, for example, a touch screen liquid crystal display (LCD) that may enable a user to interact with a user interface of the computing device.

[0031] It should be noted that, in some optional embodiments, the above Figure 1 The computing device shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware elements and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computing devices described above.

[0032] Figure 2 Schematic diagram of an accelerator implementation system based on an FPGA chip according to this embodiment. Figure 2 As shown, the system includes: a host, an FPGA chip and a memory. The host can be, for example, a CPU of an x86 architecture. The host is mainly responsible for parsing the CNN model and sending relevant data information to the memory. The data information can include, for example, the size information of the feature image, the number of channels of the feature image, the convolution step information of the CNN model, the padding size, the feature image data related to the feature image, and the weight data corresponding to the convolution kernel of the convolution layer.

[0033] The FPGA chip is equipped with an XDMA controller and a CNN accelerator. The CNN accelerator and the XDMA controller are interconnected through the PCIE bus. As the terminal of the PCIE bus, the FPGA converts the complex PCIE bus timing into the AXI bus through the XDMA controller, thereby realizing communication connection with the host. The XDMA controller is also connected to the host and the memory respectively, so that the host can write relevant data information to the memory through the XDMA controller, which is convenient for the CNN accelerator to access data from the memory through the data bus controller.

[0034] The FPGA chip is also equipped with a control bus controller. The host sends data information to the CNN accelerator through the control bus controller (i.e., the host stores the data information generated after parsing in the memory, and then sends the address of the above data information in the memory to the CNN accelerator through the control bus controller). The parsed data information includes but is not limited to calculation instructions, data required for calculation, and the address of weight data (including the first weight data or the second weight data), the size information of the feature image, and the number of convolution kernels of the convolution layer of the CNN model. After receiving the above data information sent by the host, the control bus controller sends the above data information to the process control module.

[0035] The FPGA chip is also equipped with a process control module. The process control module reads the data information of the corresponding convolution layer from the memory through the data bus controller according to the address information parsed from the control bus controller, and performs a series of counting controls. Among them, the counting control performed by the process control module includes input channel slice count, output row slice count, and output channel slice count. The process control module contains a complex state machine that can control the pipeline operation of the entire CNN accelerator. The process control module controls data broadcasting, multiple processing units, and memory according to the jumps of the above-mentioned multiple counts and state machines.

[0036] The FPGA chip is also deployed with a configuration module. The configuration module is connected to the process control module, the data broadcast module, the multiple processing units and the output cache module respectively, so that the configuration module can indicate the working status of the process control module, the data broadcast module, the multiple processing units and the output cache module deployed on the FPGA chip according to the state machine.

[0037] The FPGA chip is also equipped with a data broadcast module. The data broadcast module is used to read the first feature image data or the second weight data from the memory through the data bus controller, and expand the first feature image data or the second weight data into multiple copies through multiple storages, and then send each copy to a different processing unit to achieve broadcasting. This achieves the reuse of input data between various processing units.

[0038] The FPGA chip is also equipped with a data bus controller. The CNN accelerator accesses data to the memory through the data bus controller. The data bus controller also converts the user-defined interfaces of each processing unit and data broadcast module into the read and write timing of the memory.

[0039] The FPGA chip is also equipped with multiple processing units. The processing unit contains a DSP array, which is mainly used for parallel computing. And the multiple processing units can also read the first weight data or the third feature image data from the memory through the data bus controller.

[0040] The FPGA chip is also equipped with an output cache module. The output cache module accumulates or connects the data in the memory of each processing unit according to the instructions in the process. And before the output cache module saves the data to the memory through the data bus controller, the output cache module will combine or arrange the data to ensure an organized storage sequence.

[0041] The memory is a DDR chip on the FPGA board. The host can control the memory through the XDMA controller driver, and the FPGA can access the memory by controlling the bus controller.

[0042] Figure 3 Schematic diagram of the hardware structure of the data broadcast module according to Embodiment 1 of the present application. Figure 3 As shown, the data broadcast module is connected to the memory through the data bus controller and can access the memory independently.

[0043] The data broadcast module is provided with a data loading module and an address calculator. The address calculator sends the address information of the first feature image data in the memory or the address information of the second weight data in the memory to the data loading module, and then the data loading module reads the corresponding first feature image data or second weight data from the memory based on the acquired address information.

[0044] Furthermore, the data loading module expands the read first feature image data or second weight data into n copies, and sends the n copies to each register in register group 0. Register group 0 includes register 0, register 1, ..., register n.

[0045] Then, each register in register group 0 expands the received copy and sends the copy generated after expansion to the corresponding register in register group 1. For example, register 0 in register group 0 corresponds to register 0-0, register 0-1 and register 0-2 in register group 1, so that register 0 expands the received copy and sends the copy generated after expansion to register 0-0, register 0-1 and register 0-2 respectively; register 1 in register group 0 corresponds to register 1-0, register 1-1 and register 1-2 in register group 1, so that register 1 expands the received copy and sends the copy generated after expansion to register 1-0, register 1-1 and register 1-2 respectively; and so on; register n in register group 0 corresponds to register n-0, register n-1 and register n-2 in register group 1, so that register n expands the received copy and sends the copy generated after expansion to register n-0, register n-1 and register n-2 respectively.

[0046] Finally, each register in register group 1 sends the received data to the corresponding processing unit. For example, register 0-0 in register group 1 corresponds to processing unit 0, so register 0-0 sends the received copy to processing unit 0; register 0-1 in register group 1 corresponds to processing unit 1, so register 0-1 sends the received copy to processing unit 1; register 0-2 in register group 1 corresponds to processing unit 2, so register 0-2 sends the received copy to processing unit 2; ...; register n-0 in register group 1 corresponds to processing unit 3n-2, so register n-0 sends the received copy to processing unit 3n-2; register n-1 in register group 1 corresponds to processing unit 3n-1, so register n-1 sends the received copy to processing unit 3n-1; register n-2 in register group 1 corresponds to processing unit 3n, so register n-2 sends the received copy to processing unit 3n.

[0047] Thus, reference Figure 3 As can be seen from the above, since the convolution layer with a large number of convolution kernels will repeatedly design the same input features in the calculation, the data broadcast module in this application is a multi-level storage design. Therefore, it is only necessary to expand the read first feature image data or second weight data into n copies, so that multiple processing units can perform convolution operations on the feature map at the same time.

[0048] In addition, it is worth noting that the multi-level accumulation design of the data broadcast module can avoid the timing problem caused by the excessive length of some destinations of the same set of information to a certain extent.

[0049] In the above operating environment, according to the first aspect of this embodiment, a method for implementing a CNN accelerator based on an FPGA chip is provided. The method comprises Figure 2 The process control module implementation shown in . Figure 4 A schematic diagram showing the process of the method is shown in FIG. Figure 4 As shown, the method includes:

[0050] S402: Read parameter information from a memory, wherein the parameter information includes information on the number of convolution kernels of a convolution layer of a CNN model and / or size information of a feature image;

[0051] S404: comparing the parameter information with the preset threshold information, and determining a parallel computing strategy corresponding to the multiple processing units according to the comparison result; and

[0052] S406: According to the parallel computing strategy, control the data broadcast module and multiple processing units to perform corresponding operations.

[0053] Specifically, refer to Figure 2 As shown, first, the process control module reads corresponding parameter information from the memory through the data bus controller according to the address information parsed from the control bus controller (S502). The parameter information includes the number of convolution kernels of the convolution layer of the CNN model and / or the size information of the feature image.

[0054] Then, the process control module compares the parameter information with the preset threshold information, and determines the parallel computing strategy corresponding to the processing unit according to the comparison result (S504). For example, the operator sets the first threshold information and the second threshold information in the process control module in advance through the host. Among them, the first threshold information is used to indicate the size standard corresponding to the feature image, and the second threshold information is used to indicate the quantity standard corresponding to the convolution kernel of the convolution layer. It is worth noting that the above-mentioned first threshold information and the second threshold information can be set by the operator based on actual conditions or historical data, which will not be elaborated here.

[0055] Further for example, when the size of the feature image is smaller than a first preset threshold and / or the number of convolution kernels of the convolution layer is greater than a second preset threshold, multiple processing units and the data broadcasting module execute a first parallel strategy; when the size of the feature image is greater than the first preset threshold and / or the number of convolution kernels of the convolution layer is less than a second preset threshold, multiple processing units and the data broadcasting module execute a second parallel strategy.

[0056] The comparison operation performed by the process control module and the contents related to the first parallel strategy and the second parallel strategy will be described in detail later, so they will not be repeated here. In addition, it is worth noting that this embodiment only takes the first parallel strategy and the second parallel strategy as examples, and the actual situation is not limited to this.

[0057] Finally, the process control module controls the data broadcast module and the multiple processing units to perform corresponding operations according to the parallel computing strategy (S506). For example, after the process control module determines the first parallel strategy based on the comparison result, the data broadcast module and the multiple processing units perform related operations according to the first parallel strategy, or after the process control module determines the second parallel strategy based on the comparison result, the data broadcast module and the multiple processing units perform related operations according to the second parallel strategy. The above content will be described in detail later, so it will not be repeated here.

[0058] As described in the background technology, since the circuit of the CNN accelerator is fixed, that is, for different convolutional layers, the parallelism of the calculation is the same. Furthermore, in order to make the acceleration effect of the CNN accelerator better, the designer will design the parallelism based on the size characteristics of most convolutional layers (that is, smaller size and more channels). However, this will inevitably sacrifice the acceleration performance of the CNN accelerator for convolutional layers with larger size and fewer channels, so that when the input feature image size is large and the channels are fewer, the acceleration effect of the CNN accelerator is relatively poor. Therefore, in summary, a simple spatial expansion architecture is difficult to cope with the diversity of CNN models.

[0059] In view of this, the present application provides a CNN accelerator implementation system based on FPGA chips. And because the process control module reads parameter information from the memory, and the parameter information includes the number of convolution kernels of the convolution layer of the CNN model and / or the size information of the feature image, and further compares the read parameter information with the preset threshold information, and selects the corresponding parallel computing strategy according to the comparison result, the technical solution disclosed in the present application fully considers the number of convolution kernels of the convolution layer, and / or the size of the feature image on the CNN accelerator, so that the CNN accelerator can select the most suitable parallel computing strategy for the CNN accelerator under any circumstances, so as to give full play to the acceleration performance of the CNN accelerator.

[0060] Thus, the adaptability of the CNN accelerator to different convolutional layers in the CNN model is improved, and the technical effect of ensuring the acceleration performance of the CNN accelerator is achieved. Furthermore, the technical problem of how to improve the adaptability of the CNN accelerator to different convolutional layers in the CNN model, thereby further ensuring the acceleration effect of the CNN accelerator on the convolutional layer, which exists in the prior art, is solved.

[0061] Optionally, the parameter information is compared with the preset threshold information, and the operation of the parallel computing strategy corresponding to the multiple processing units is determined based on the comparison result, including: when the size of the feature image is less than the first preset threshold and / or the number of convolution kernels of the convolution layer is greater than the second preset threshold, the first parallel strategy corresponding to the multiple processing units is determined, wherein the first parallel strategy is used to indicate that in the parallel computing process, the convolution kernels assigned to each processing unit are different, and the first feature image data assigned to each processing unit are the same. Further optionally, according to the parallel computing strategy, the operation of controlling the data broadcast module and the multiple processing units to perform corresponding operations includes: the data broadcast module reads multiple first feature image data from the memory, and processes each first feature image data in parallel to generate second feature image data; the data broadcast module transmits the second feature image data to each processing unit respectively; and each processing unit reads the first weight data corresponding to the corresponding convolution kernel from the memory, wherein the first weight data of each processing unit is different.

[0062] Specifically, when the size of the feature image is smaller than the first preset threshold and / or the number of convolution kernels of the convolution layer is greater than the second preset threshold, the process control module selects the first parallel strategy. The first preset threshold and the second preset threshold are standard values ​​determined by the operator based on actual conditions or historical data. The first parallel strategy is used to indicate that in the parallel computing process, the convolution kernels assigned to each processing unit are different, and the first feature image data assigned to each processing unit is the same.

[0063] Further, when the process control module determines the first parallel strategy, the data broadcast module is controlled to read multiple first feature image data from the memory through the data bus controller. Then, the data broadcast module processes the read first feature image data in parallel to generate second feature image data. After that, the process control module controls the data broadcast module to transmit the second feature image data to each processing unit respectively. At the same time, the process control module also controls each processing unit to read the first weight data corresponding to the corresponding convolution kernel from the memory through the data bus controller.

[0064] For example, Figure 5 Schematic diagram of multiple processing units performing calculations according to the first parallel strategy according to Embodiment 1 of the present application. Figure 5 As shown, the number of channels of the feature image is 4, so that the feature image can be split into first feature image data corresponding to channel 1, first feature image data corresponding to channel 2, first feature image data corresponding to channel 3, and first feature image data corresponding to channel 4.

[0065] Further references Figure 5 As shown, the first feature image data corresponding to channel 1 includes f1 to f9, and the first feature image data corresponding to channel 2 includes f 10 ~f 18 , the first feature image data corresponding to channel 3 includes f 19 ~f 27 , the first feature picture data corresponding to channel 4 includes f 28 ~f 36 .

[0066] It is worth noting that the first feature image data corresponding to the same pixel position of each channel correspond to each other. For example, the first feature image data f1 corresponding to channel 1 and the first feature image data f2 corresponding to channel 2 correspond to each other. 10 , the first feature image data f corresponding to channel 3 19 And the first feature image data f corresponding to channel 4 28 Correspond to each other.

[0067] For another example, the first feature image data f2 corresponding to channel 1 and the first feature image data f 11 , the first feature image data f corresponding to channel 3 20 And the first feature image data f corresponding to channel 4 29 Correspond to each other.

[0068] The correspondence between other first feature picture data at the same pixel position corresponding to each channel is the same as the above example, so it will not be repeated here.

[0069] Thus, the data broadcast module splices the first feature picture data obtained from the memory. Figure 5 As shown, for example, the data broadcast module sends the first feature picture data f1 and the first feature picture data f corresponding to channel 2 to the data broadcast module. 10 , the first feature image data f corresponding to channel 3 19 And the first feature image data f corresponding to channel 4 28 Parallel processing, generating second feature picture data, and transmitting the second feature picture data to processing units PU0~PU n-1 .

[0070] Then, each processing unit PU0~PU n-1 The first weight data corresponding to the corresponding convolution kernel is read from the memory by the processing unit PU0. For example, the processing unit PU1 reads the first weight data corresponding to the convolution kernel 2 and the convolution kernel 3 from the memory; and so on. n-1The first weight data corresponding to the convolution kernel n-2 and the convolution kernel n-1 are read from the memory.

[0071] Thus, different processing units PU0~PU n-1 The same first feature image data and different first weight data are obtained. n-1 The internal processing units PU0~PU n-1 They are arranged in parallel according to the output channels.

[0072] Therefore, when the size of the feature image is smaller than the first preset threshold and / or the number of convolution kernels of the convolution layer is greater than the second preset threshold, the selected first parallel strategy is the most suitable parallel computing strategy for the CNN accelerator, thereby ensuring the acceleration effect of the CNN accelerator on the convolution layer.

[0073] Optionally, the data broadcast module reads the first feature image data corresponding to each channel from the memory, and processes each first feature image data in parallel to generate the second feature image data, including: the data broadcast module organizes and arranges the first feature image data corresponding to each channel according to the first addressing mode, thereby generating the second feature image data, wherein in the first addressing mode: the data at the same pixel position is arranged into a group of data according to the channel order; each group of data is arranged in the order of rows; and each row of data is arranged in the order of columns. Further optionally, the data broadcast module transmits the second feature image data to each processing unit respectively, including: each processing unit determines a first multiplier array implemented by multiple multipliers based on the size information of the convolution layer and the number information of the convolution kernel; the data broadcast module broadcasts the second feature image data corresponding to the size of the convolution kernel to the first multiplier array in each processing unit according to the second addressing mode, thereby generating a first data stream corresponding to different rows of the convolution kernel of the convolution layer, wherein in the second addressing mode: each group of data is arranged in the order of columns; and each column of data is arranged in the order of rows.

[0074] Specifically, it can be seen from the above description that the data broadcast module needs to generate the second characteristic picture data before broadcasting the second characteristic picture to each processing unit, that is, the data broadcast module needs to process the first characteristic picture data in parallel to generate the second characteristic picture data.

[0075] Therefore, first, the data broadcast module organizes and arranges the first feature image data corresponding to each channel according to the first addressing mode, thereby generating the second feature image data. The first addressing mode is used to indicate that: first, the data broadcast module arranges the data at the same pixel position into a group of data according to the channel order. Then, the data broadcast module arranges each group of data according to the order of rows. Finally, the data broadcast module arranges each row of data according to the order of columns.

[0076] Figure 6 This is a schematic diagram of the data broadcast module organizing and arranging the first feature picture data according to the first parallel strategy when the feature picture size is less than the first preset threshold according to Embodiment 1 of the present application. Figure 6 As shown, first, the data broadcast module reads all the first characteristic picture data of a channel slice from the memory. For example, the data broadcast module reads the first characteristic picture data F1 to F25 of channel 1, the first characteristic picture data F1' to F25' of channel 2, ..., the first characteristic picture data F1 of channel n n ~F25 n Then, the data broadcast module arranges and organizes the first feature picture data corresponding to each channel into second feature picture data. Figure 6 As shown, for example, according to the first addressing mode, the first feature picture data is arranged and organized as {F1, F1', ..., F1 n},{F2,F2',...,F2 n},...,{F25,F25',...,F25 n}.

[0077] Afterwards, each processing unit determines a first multiplier array implemented by a plurality of multipliers based on the size information of the convolution layer and the number information of the convolution kernels. Figure 6 As shown, each processing unit determines a first multiplier array of 2×2×n based on the size information of the convolution layer and the number information of the convolution kernels. Specifically, at the beginning of designing the processing unit, a plurality of multipliers have been pre-set inside each processing unit. Thus, the processing unit can enable the corresponding number of multipliers based on the size information of the convolution layer and the number information of the convolution kernels, thereby determining the first multiplier array. It is worth noting that each multiplier in the first multiplier array corresponds to a first weight data.

[0078] Referring to the above description, we can see that the above discussion is about the design of different parallel situations between various processing units. For a single processing unit, its work is pipelined. At each moment, a feature image data and a weight data are input into a certain processing unit to complete better calculation. Therefore, only the ideal parallel strategy combined with the data flow method can achieve better acceleration benefits.

[0079] However, in the actual CNN model, since the size of the feature image is usually large, more feature data needs to be traversed to complete the convolution operation. Therefore, the pulsating data flow method designed by the present application fully considers the reusability of the data, and the data that has participated in the calculation does not need to be loaded repeatedly. After the data flow is completed, all multiplication operations have been calculated.

[0080] in, Figure 7 Schematic diagram of data flow design according to an embodiment of the present application. Figure 7 As shown in the figure, this data flow design is divided into three tracks. Track 1 ends at position f1 in the figure, track 2 ends at position f6 in the figure, and track 3 ends at position f31 in the figure. Among them, the shared displacement logic is shown in the black box. Fix w1~w9 at positions f1~f3, f16~f18, and f31~f33 respectively. When the data flows to these positions, multiplication calculations with w1~w9 are realized respectively. And among them, the way of data flow can be calculated. When this length is exceeded, the multiplication result will no longer be valid. Therefore, the control of effective multiplication can be achieved by counting. As for accumulation, in this data pulsation mode, the corresponding products of w1~w9 will be calculated in each cycle. And these products are already cumulative calculations and all elements, which only need to be cached and implemented using addition or accumulation IP.

[0081] Therefore, the data broadcast module needs to broadcast the second feature image data corresponding to the size of the convolution kernel to the first multiplier array in each processing unit according to the second addressing mode, so as to generate a first data stream corresponding to different rows of the convolution kernel of the convolution layer. The second addressing mode is used to indicate that: first, the data broadcast module arranges each group of data in the order of columns. Then, the data broadcast module arranges each column of data in the order of rows.

[0082] For example, refer to Figure 6 As shown, first, the data broadcast module generates two data streams according to the second addressing mode. Data stream 1: {F1, F1', ..., F1 n},{F6,F6',...,F6 n},{F11,F11',...,F11 n},{F16,F16',...,F16 n},{F2,F2',...,F2 n},...,{F19,F19',...,F19 n},...,{F20,F20',...,F20 n}. Data flow 2: {F6, F6', ..., F6 n},{F11,F11',...,F11 n},{F16,F16',...,F16 n},{F21,F21',...,F21 n},{F7,F7',...,F7 n},...,{F24,F24',...,F24 n},...,{F25,F25',...,F25 n}.

[0083] Then, the data broadcast module broadcasts data stream 1 and data stream 2 to the 2×2×n first multiplier arrays in each processing unit. Among them, for the n first weighted data corresponding to the n multipliers in the first row and first column of the first multiplier array, when the first data {F1, F1', ..., F1 n}, the calculation flag w1_valid_flag signal is pulled high, the multiplier input is valid, and the multiplier output result will be sent to the first address of buff1 for temporary storage. At the same time, {F2, F2', ..., F2 in data stream 1 n} is at the position of the n first weight data corresponding to the n multipliers in the first row and second column of the first multiplier array. Therefore, the calculation flag w2_valid_flag signal is pulled high at this time, the multiplier input is valid, and the output result of the multiplier will be sent to the first address of buff2 for temporary storage.

[0084] Similarly, the n first weight data corresponding to the n multipliers in the second row and first column of the first multiplier array, and the n first weight data corresponding to the n multipliers in the second row and second column of the first multiplier array. The multiplication calculation of these four first weight data is performed simultaneously.

[0085] After one cycle, data stream 1 and data stream 2 pulsate by one position. That is, the data {F2, F2', ..., F2 n} pulsates to the positions corresponding to the n multipliers in the first row and first column of the first multiplier array, and the data {F11, F11', ..., F11 in data stream 2 n}pulsates to the positions corresponding to the n multipliers in the first row and first column of the first multiplier array.

[0086] At this time, the output results of the four multipliers in the first multiplier array will be sent to the second address of each corresponding buff. At this time, the first address corresponding to each buff will be added through the addition tree. After another cycle, the register inside the buff is shifted, and the data corresponding to the second address is moved to the first address, and the second address receives the new multiplication result. At this time, the addition tree calculates the accumulation of the data of the second cycle. The result of each addition tree is the result output by the 2×2 convolution kernel, and there is no intermediate result. Therefore, the cache of intermediate results can be reduced and resources can be saved.

[0087] Optionally, the parameter information is compared with the preset threshold information, and the operation of the parallel computing strategy corresponding to the multiple processing units is determined based on the comparison result, including: when the size of the feature image is greater than the first preset threshold and / or the number of convolution kernels of the convolution layer is less than the second preset threshold, determine the second parallel strategy corresponding to the multiple processing units, wherein the second parallel strategy is used to indicate that in the parallel computing process, the third feature image data assigned to each processing unit is different, and the convolution kernels assigned to each processing unit are the same. Further optionally, the CNN accelerator also includes a configuration module, wherein the configuration module is configured to perform the following operations: split the feature image corresponding to each channel by row, and generate multiple different third feature image data; and assign the address of each third feature image data in the memory to each processing unit. Further optionally, according to the parallel computing strategy, the data broadcasting module and multiple processing units are controlled to perform corresponding operations, including: each processing unit reads the corresponding third feature image data from the memory based on the assigned address, and processes the read third feature image data in parallel to generate fourth feature image data; and the data broadcasting module reads the second weight data corresponding to each convolution kernel from the memory, and sends multiple second weight data to each processing unit.

[0088] Specifically, when the size of the feature image is greater than the first preset threshold and / or the number of convolution kernels of the convolution layer is less than the second preset threshold, a second parallel strategy corresponding to the multiple processing units is determined. The second parallel strategy is used to indicate that in the parallel computing process, the third feature image data allocated to each processing unit is different, and the convolution kernels allocated to each processing unit are the same.

[0089] Furthermore, when the process control module determines the second parallel strategy, the control configuration module splits the feature image corresponding to each channel by row, and generates multiple different third feature image data, and the control configuration module allocates the address of each third feature image data in the memory to each processing unit.

[0090] Afterwards, each processing unit reads the corresponding third feature image data from the memory based on the assigned address, and processes the read third feature image data in parallel to generate fourth feature image data. At the same time, the data broadcast module reads the second weight data corresponding to each convolution kernel from the memory, and sends multiple second weight data to each processing unit.

[0091] For example, Figure 8 is a schematic diagram of multiple processing units performing calculations according to the second parallel strategy according to an embodiment of the present application. Figure 8 As shown, the number of channels of the feature image is 2, so that the feature image can be split into third feature image data corresponding to channel 1 and third feature image data corresponding to channel 2.

[0092] Then, the configuration module divides the third feature picture data corresponding to channel 1 and the third feature picture data corresponding to channel 2 according to rows, thereby generating a plurality of fourth feature picture data. Figure 7 As shown, for example, the third characteristic picture data corresponding to channel 1 includes f1 to f225. After the configuration module divides the third characteristic picture data corresponding to channel 1 according to rows, a row slice is generated (wherein the row slice includes 2 rows. And f1 to f30 are the first row slices of channel 1; f31 to f60 are the second row slices of channel 1; ...; f196 to f225 are the last row slices of channel 1. f226 to f255 are the first row slices of channel 2; f256 to f285 are the second row slices of channel 2; ...; f421 to f450 are the last row slices of channel 2.

[0093] Furthermore, the configuration module calculates the addresses of the plurality of row slices and sends the addresses of the corresponding row slices to the corresponding processing units. The processing units obtain different third feature image data from the memory according to the addresses assigned to them, and perform parallel processing on the read third feature image data to generate fourth feature image data. For example, referring to Figure 8 As shown, processing unit PU0 reads f1 and f226 from the memory, processing unit PU1 reads f31 and f2256 from the memory, and so on. n-1f196 and f450 will be read from the memory. Then, the processing unit PU0 processes the third characteristic picture data of the first row slice read in parallel and generates the fourth characteristic picture data; the processing unit PU1 processes the third characteristic picture data of the second row slice read in parallel and generates the fourth characteristic picture data; ...; the processing unit PU n-1 The read third feature picture data is processed in parallel to generate fourth feature picture data.

[0094] Finally, the data broadcast module reads the second weight data from the memory and broadcasts the second weight data to all processing units. Figure 8 As shown, the data broadcast module broadcasts the second weight data corresponding to convolution kernel 0, the second weight data corresponding to convolution kernel 1, ..., the second weight data corresponding to convolution kernel n-2, and the second weight data corresponding to convolution kernel n-1 to processing units PU0 to PU n-1 .

[0095] Thus, different processing units PU0~PU n-1 The same second weight data and different fourth feature picture data are obtained. n-1 The internal part is parallelized according to the channel, and each processing unit PU0~PU n-1 They are arranged in parallel in rows.

[0096] Therefore, when the size of the feature image is greater than the first preset threshold and / or the number of convolution kernels of the convolution layer is less than the second preset threshold, the selected second parallel strategy is the parallel computing strategy most suitable for the CNN accelerator, thereby ensuring the acceleration effect of the CNN accelerator on the convolution layer.

[0097] Optionally, each processing unit processes the read third feature image data in parallel to generate the fourth feature image data, including: each processing unit organizes and arranges the third feature image data corresponding to each channel according to the first addressing mode, thereby generating the fourth feature image data. Further optionally, it also includes: each processing unit determines a second multiplier array implemented by multiple multipliers based on the size of the convolution kernel and the number of convolution kernels; and each processing unit sends the fourth feature image data corresponding to the size of the convolution kernel to the internal second multiplier array according to the second addressing mode, thereby generating a second data stream corresponding to different rows of the convolution kernel of the convolution layer.

[0098] Specifically, it can be seen from the above description that before performing calculations, each processing unit needs to organize and arrange the third feature picture data corresponding to each channel read from the memory according to the first addressing mode, so as to generate fourth feature picture data.

[0099] Fig. 9 This is a schematic diagram of the processing unit organizing and arranging the third feature picture data according to the second parallel strategy when the feature picture size is greater than the first preset threshold according to Embodiment 1 of the present application. Fig. 9 As shown, each processing unit reads the corresponding third feature picture data from the memory. For example, processing unit PU0 reads the third feature picture data of the first row slice, processing unit PU1 reads the third feature picture data of the second row slice, ..., processing unit PU n-1 The third characteristic picture data of the nth row slice is read. Then, each processing unit arranges and organizes the third characteristic picture data corresponding to each channel into fourth characteristic picture data. The manner in which each processing unit arranges and organizes the third characteristic picture data corresponding to each channel into fourth characteristic picture data is the same as the manner in which the data broadcasting module arranges and organizes the first characteristic picture data corresponding to each channel into second characteristic picture data, so it will not be described in detail here.

[0100] Afterwards, each processing unit determines a second multiplier array implemented by a plurality of multipliers based on the size information of the convolution layer and the number information of the convolution kernels. Fig. 9 As shown, each processing unit determines a 2×2×n second multiplier array based on the size information of the convolution layer and the number information of the convolution kernels. It is worth noting that each multiplier in the second multiplier array corresponds to a second weight data.

[0101] Further, each processing unit sends fourth feature picture data corresponding to the size of the convolution kernel to multiple multipliers in the second multiplier array therein according to the second addressing mode, thereby generating a second data stream corresponding to different rows of the convolution kernel of the convolution layer. The manner in which each processing unit broadcasts the fourth feature picture data to multiple multipliers in its respective second multiplier array according to the second addressing mode is the same as the manner in which the above-mentioned data broadcasting module broadcasts the second feature picture data to multiple multipliers in the first multiplier array according to the second addressing mode, and therefore will not be described in detail here.

[0102] Finally, when each processing unit completes the convolution calculation corresponding to its own row slice, the output cache module splices the output result data according to the ID of each processing unit and transfers the spliced ​​output result data to the memory.

[0103] In addition, it is worth noting that the number of multipliers determined in the above-mentioned first multiplier array and the number of multipliers determined in the second multiplier array are determined based on the size information of the convolution layer and the number information of the convolution kernels. Therefore, the above only takes the size information of the convolution layer as 2×2 as an example, and the actual situation is not limited to this.

[0104] Therefore, according to the first aspect of the embodiment of the present application, the technical effect of improving the adaptability of the CNN accelerator to different convolutional layers in the CNN model and ensuring the acceleration performance of the CNN accelerator is achieved.

[0105] The technical solution of this application has the following beneficial effects:

[0106] 1. This application proposes a new computing architecture that divides the acceleration part in the FPGA into multiple processing units. Each processing unit has the function of independently accessing the memory outside the FPGA chip, and each processing unit contains a DSP array inside, which is mainly used for parallel computing. Therefore, under this system, different parallel strategies are designed for different convolutional layers;

[0107] 2. This application designs an efficient characteristic data pulsation method, the operation data arrangement is simple and effective, the multiplier utilization rate is high, and the data broadcast module is used to broadcast the data stream to each processing unit;

[0108] 3. The CNN accelerator in this application adopts a parameterized design. The number of multipliers in each processing unit is configurable, and the number of PUs is also configurable. It can maximize the use of computing resources according to the FPGA resource situation and adapt to various deployment environments.

[0109] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0110] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0111] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0112] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0113] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0114] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0115] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0116] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0117] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A CNN accelerator implementation system based on FPGA chip, comprising: A host, an FPGA chip connected to the host, and a memory connected to the FPGA chip, wherein the FPGA chip is deployed with a CNN accelerator, characterized in that the CNN accelerator comprises: a process control module, a data broadcast module and a plurality of processing units, wherein the process control module is respectively connected to the data broadcast module and the plurality of processing units; the data broadcast module is respectively connected to each processing unit, the processing unit is used to implement convolution calculation, and wherein the process control module is configured to perform the following operations: Reading parameter information from the memory, wherein the parameter information includes the number information of convolution kernels of the convolution layer of the CNN model and / or the size information of the feature image; Comparing the parameter information with preset threshold information, and determining a parallel computing strategy corresponding to the plurality of processing units according to the comparison result; and According to the parallel computing strategy, the data broadcast module and the multiple processing units are controlled to perform corresponding operations.

2. The system according to claim 1, characterized in that The parameter information is compared with preset threshold information, and the operation of the parallel computing strategy corresponding to the multiple processing units is determined according to the comparison result, including: When the size of the feature image is smaller than a first preset threshold and / or the number of convolution kernels of the convolution layer is greater than a second preset threshold, a first parallel strategy corresponding to the multiple processing units is determined, wherein the first parallel strategy is used to indicate that in a parallel computing process, the convolution kernels allocated to the respective processing units are different, and the first feature image data allocated to the respective processing units are the same.

3. The system according to claim 2, characterized in that According to the parallel computing strategy, the operation of controlling the data broadcast module and the plurality of processing units to perform corresponding operations includes: The data broadcast module reads a plurality of the first characteristic picture data from the memory, and processes each of the first characteristic picture data in parallel, thereby generating second characteristic picture data; The data broadcast module transmits the second feature picture data to each of the processing units respectively; and Each of the processing units reads first weight data corresponding to the corresponding convolution kernel from the memory, wherein the first weight data of each of the processing units are different.

4. The system according to claim 3, characterized in that The data broadcast module reads the first feature picture data corresponding to each channel from the memory, and processes each first feature picture data in parallel to generate the second feature picture data, including: The data broadcast module organizes and arranges the first feature picture data corresponding to each channel according to the first addressing mode, thereby generating the second feature picture data, wherein in the first addressing mode: Arrange the data at the same pixel position into a group of data in channel order; Arrange each set of data in row order; and Arrange the rows of data in column order.

5. The system according to claim 4, characterized in that The operation of the data broadcasting module transmitting the second feature picture data to each of the processing units respectively includes: Each processing unit determines a first multiplier array implemented by a plurality of multipliers based on the size information of the convolution layer and the number information of the convolution kernels; The data broadcasting module broadcasts the second feature picture data corresponding to the size of the convolution kernel to the first multiplier array in each processing unit according to the second addressing mode, thereby generating a first data stream corresponding to different rows of the convolution kernel of the convolution layer, respectively, wherein in the second addressing mode: Arranging the groups of data in order of columns; and Arrange the data in each column in the order of rows.

6. The system according to claim 1, characterized in that The parameter information is compared with preset threshold information, and the operation of the parallel computing strategy corresponding to the multiple processing units is determined according to the comparison result, including: When the size of the feature image is greater than a first preset threshold and / or the number of convolution kernels of the convolution layer is less than a second preset threshold, a second parallel strategy corresponding to the multiple processing units is determined, wherein the second parallel strategy is used to indicate that during the parallel computing process, the third feature image data allocated to each processing unit is different, and the convolution kernels allocated to each processing unit are the same.

7. The system according to claim 6, characterized in that The CNN accelerator further includes a configuration module, wherein the configuration module is configured to perform the following operations: Splitting the feature images corresponding to each channel by row, and generating a plurality of different third feature image data; and The addresses of each third feature picture data in the memory are allocated to each processing unit.

8. The system according to claim 7, characterized in that According to the parallel computing strategy, the operation of controlling the data broadcast module and the plurality of processing units to perform corresponding operations includes: Each of the processing units reads the corresponding third feature picture data from the memory based on the allocated address, and performs parallel processing on the read third feature picture data, thereby generating fourth feature picture data; The data broadcast module reads the second weight data corresponding to each convolution kernel from the memory, and sends a plurality of the second weight data to each processing unit, wherein The operation of each processing unit processing the read third feature picture data in parallel to generate fourth feature picture data includes: The respective processing units organize and arrange the third feature picture data corresponding to the respective channels according to the first addressing mode, thereby generating the fourth feature picture data.

9. The system according to claim 8, characterized in that Also includes: Each of the processing units determines a second multiplier array implemented by a plurality of multipliers based on the size of the convolution kernel and the quantity information of the convolution kernel; as well as Each of the processing units sends fourth feature image data corresponding to the size of the convolution kernel to the second multiplier array inside according to the second addressing mode, thereby generating a second data stream corresponding to different rows of the convolution kernel of the convolution layer.

10. A method for implementing a CNN accelerator based on an FPGA chip, characterized in that: include: The process control module reads parameter information from the memory, wherein the parameter information includes the number information of convolution kernels of the convolution layer of the CNN model and / or the size information of the feature image; The process control module compares the parameter information with preset threshold information, and determines a parallel computing strategy corresponding to a plurality of processing units according to the comparison result, wherein the processing units are deployed in a CNN accelerator and are used to implement convolution computing; and The process control module controls the data broadcast module deployed on the CNN accelerator and the multiple processing units to perform corresponding operations according to the parallel computing strategy.

Images