Efficient reconfigurable hardware accelerator for convolutional neural network training

Abstract

The application relates to the technical field of convolutional neural network, and provides an efficient reconfigurable hardware accelerator for convolutional neural network training, which comprises a PE array, the PE array comprises K PE clusters, each PE cluster comprises three input rows, an addition tree and a partial sum cache, a plurality of PE basic units are arranged in each input channel, summation between the three input rows is completed by the addition tree, and intermediate calculation results obtained by the addition tree are stored in the partial sum cache. In actual application, the application can support multiple different calculation modes in different calculation stages in the DNN training process, can realize parallel calculation in different dimensions, thereby improving processing speed, can eliminate invalid calculation of the convolutional layer in different stages, and improve calculation utilization; in a flexible memory access scheme, data reuse in different levels is realized, data access amount is reduced, and time and power consumption overheads are reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of convolutional neural network, and particularly relates to an efficient reconfigurable hardware accelerator for convolutional neural network training. BACKGROUND

[0002] With the continuous development of artificial intelligence (AI), artificial intelligence has evolved from early artificial feature engineering to learning from massive data, and has made major breakthroughs in machine vision, speech recognition and natural language processing. For example, DNN (Deep Neural Network) is more and more favored in the field of artificial intelligence. However, as the network structure becomes larger and more complex, it leads to the need for a large amount of computing resources in the use process of DNN, and the training of deep neural network in a high-end GPU server cluster.

[0003] In recent years, with the increasingly wide application of DNN in actual production and life, especially the rise of online learning, incremental learning and federated learning, at the same time, more and more attention is paid to the protection of data privacy in the application process of DNN, so high energy-efficient DNN training on the end-side device has gradually become an urgent demand.

[0004] In the process of DNN training, a large amount of labeled data needs to be input into the DNN until the iteration result converges. The iteration process includes forward propagation (FP), back propagation (BP) and weight gradient generation (WG), which leads to the following challenges in the design of the current accelerator for training DNN:

[0005] (1) In different stages, the calculation mode shows high diversity. In the FP, BP and WG stages, the convolution (CONV) layer can be formed in the convolution operation, and their characteristics change greatly, which easily leads to underutilization of the calculation units (PEs) due to lack of flexibility.

[0006] (2) Most previous works only consider the traditional CONV layer, but the current popular model usually uses CONV layer with a step of 2 to down-sample features, which brings more calculation diversity to the training process and brings a lot of invalid calculations.

[0007] (3) Different stages of training involve different data access modes, and a single memory access scheme cannot efficiently meet the needs, resulting in a large number of data access times and thus increasing power consumption and time overhead. Especially for weights, the reverse calculation faces the problems of channel transposition and convolution kernel flipping. SUMMARY

[0008] In order to provide a high energy-efficient training accelerator for DNN on an end-side device, the application provides a high-efficiency reconfigurable hardware accelerator for convolutional neural network training.

[0009] A high-efficiency reconfigurable hardware accelerator for convolutional neural network training comprises a PE array, the PE array comprises K PE clusters, each PE cluster comprises three input rows, an addition tree and a partial sum buffer, a plurality of PE basic units are arranged in each input channel, summation between the three input rows is completed by the addition tree, and the intermediate calculation result of the addition tree is stored in the partial sum buffer.

[0010] The PE basic unit comprises an input module, an input control module, a weight control module and an output control module, the input module is connected to the output control module through the input control module, and the weight control module is connected to the output control module.

[0011] The input module comprises a first first-in-first-out buffer and a second first-in-first-out buffer, which are used to buffer odd-position elements and even-position elements of input features respectively; the output control module comprises three multipliers (M0, M1 and M2) and three accumulation units, the three accumulation units are connected in sequence and are connected to the three multipliers (M0, M1 and M2) respectively.

[0012] The input control module is used to transmit input data to corresponding multipliers according to different calculation modes.

[0013] The weight control module is used to transmit weight data to corresponding multipliers according to different calculation modes.

[0014] The multiplier is used to perform multiplication operation on input weight data and input data, and the accumulation unit is used to accumulate and output the output structure of the multiplier.

[0015] Optionally, the input control module comprises a first multiplexer MUX0, a second multiplexer MUX1 and a register D0.

[0016] Two input ends of the first multiplexer MUX0 are connected to the first first-in-first-out buffer and the second first-in-first-out buffer respectively, and an output end is connected to the multiplier (M1); one input end of the second multiplexer MUX1 is directly connected to the first first-in-first-out buffer, the other input end is connected to the first first-in-first-out buffer through the register D0, and an output end is connected to the multiplier (M2).

[0017] Optionally, the weight control module comprises three multiplexers (MUX2, MUX3 and MUX4) and three registers (D1, D2 and D3), which are connected to the three multipliers (M0, M1 and M2) of the output control module respectively.

[0018] Optionally, the accumulation unit comprises a plurality of selectors (MUX5, MUX6, MUX7) connected in sequence, adders (A0, A1, A2) and registers (D4, D5, D6).

[0019] Optionally, the accumulation unit further comprises a storage unit, wherein the storage unit comprises an off-chip DRAM and an on-chip SRAM, and the on-chip SRAM is configured to receive complete data stored in the off-chip DRAM and perform block division on the complete data.

[0020] Optionally, the storage unit is configured to store data on input channels as a basic storage unit in K channels, and preferentially store data on the input channels to obtain complete output feature maps, and perform block division on the output feature maps in a row direction and in K channels in an output channel direction.

[0021] Optionally, the accumulation unit further comprises a weight routing network, wherein the weight routing network comprises a first-level weight network, and the first-level weight network comprises two transmission modes.

[0022] In the FP phase, the weight data adopts a direct transmission mode.

[0023] In the BP phase, transposition is implemented between the input channels and the output channels.

[0024] Optionally, the weight routing network further comprises a second-level weight network, and the second-level weight network is configured to organize a transmission sequence of one row of weights of a PE basic unit.

[0025] In the FP phase, the weights are directly propagated in the original sequence.

[0026] In the BP phase, if the convolution step length is equal to 1, the weight column sequence is transformed from 012 to 210, so as to complete the flipping of the weights; if the convolution step length is equal to 2, the weight column sequence is transformed from 012 to 201.

[0027] Optionally, the number of input channels in each PE cluster is C.

[0028] From the above technical solutions, the application provides a high-efficiency reconfigurable hardware accelerator for convolutional neural network training, which comprises a PE array, the PE array comprises K PE clusters, each PE cluster comprises three input threes, an addition tree and a partial sum buffer, a plurality of PE basic units are arranged in each input channel, summation between the three input threes is completed by the addition tree, and an intermediate calculation result obtained by the addition tree is stored in the partial sum buffer. The PE basic unit comprises an input module, an input control module, a weight control module and an output control module, the input module is connected to the output control module through the input control module, and the weight control module is connected to the output control module. The input module comprises a first first-in-first-out buffer and a second first-in-first-out buffer, which are used to buffer odd-position elements and even-position elements of input features respectively; the output control module comprises three multipliers (M0, M1 and M2) and three accumulation units, the three accumulation units are connected in sequence and connected to the three multipliers (M0, M1 and M2) respectively.

[0029] In actual application, a unified reconfigurable computing unit is provided, which supports to complete different computing modes in different computing stages in the training process; parallel computing can be realized in different dimensions, so that the processing speed is improved; invalid calculation of the convolutional layer with stride=2 in different stages can be eliminated, and the calculation utilization rate is improved; in a flexible memory access scheme, data reuse in different levels is realized, the data access amount is reduced, and the time and power consumption overheads are reduced. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the application, the drawings needed in the embodiments will be briefly introduced as follows, and obviously, other drawings can also be obtained by those skilled in the art without creative labor on the premise of the drawings.

[0031] Figure 1 The structural schematic diagram of the PE basic unit provided by the embodiment of the application is shown in the figure.

[0032] Figure 2 The working principle schematic diagram of the PE basic unit provided by the embodiment of the application in different computing modes is shown in the figure.

[0033] Figure 3 The structural schematic diagram of the PE cluster provided by the embodiment of the application is shown in the figure.

[0034] Figure 4 The working principle schematic diagram of the storage unit provided by the embodiment of the application is shown in the figure.

[0035] Figure 5 The working principle schematic diagram of the first-level weight network provided by the embodiment of the application is shown in the figure.

[0036] Figure 6 The working principle schematic diagram of the second-level weight network provided by the embodiment of the present application is shown. DETAILED DESCRIPTION

[0037] In order to provide a high energy-efficient training accelerator for DNN on an end-side device, the embodiment of the present application provides a high-efficiency reconfigurable hardware accelerator for convolutional neural network training. Specifically, it includes a PE array, the PE array includes K PE clusters, each PE cluster includes three input rows, an addition tree and a partial sum buffer, a plurality of PE basic units are arranged in each input channel, the summation between the three input rows is completed by the addition tree, and the intermediate calculation result obtained by the addition tree is stored in the partial sum buffer. As shown in the figure, the structure schematic diagram of the PE basic unit provided by the embodiment of the present application is shown. The PE basic unit includes an input module, an input control module, a weight control module and an output control module, the input module is connected to the output control module through the input control module, the weight control module is connected to the output control module, and in the embodiment of the present application, the number of input channels in one PE cluster is C. Figure 1

[0038] The input module includes a first first-in-first-out buffer FIFO-H and a second first-in-first-out buffer FIFO-L, which are used to buffer the odd-position elements and the even-position elements of the input features respectively; the output control module includes three multipliers (M0, M1, M2) and three accumulation units, the three accumulation units are connected in sequence and are connected to the three multipliers (M0, M1, M2) respectively.

[0039] The input control module is used to transmit the input data to the corresponding multiplier according to different calculation modes.

[0040] The weight control module is used to transmit the weight data to the corresponding multiplier according to different calculation modes.

[0041] The multiplier is used to perform multiplication operation on the input weight data and the input data, and the accumulation unit is used to accumulate and output the output structure of the multiplier.

[0042] At the PE level, each convolutional layer is calculated in a row-by-row manner. The PE can support the calculation of different calculation stages (FP, BP, WG) in the training process through flexible configuration, and efficiently supports the calculation of different convolutional strides, reducing the redundant calculation under the condition of stride=2.

[0043] ​Specifically, for different calculation modes of different convolution types in FP, BP and WG, a multiplexer is used to control the data path therein, so as to configure the calculation unit into different working modes. Taking a 1*3 convolution as an example, how the PE basic unit processes different calculation modes in the convolution is introduced in detail according to a row-by-row calculation data flow.

[0044] When the stride is equal to 1, the calculation modes of FP and BP stages in the spatial dimension are the same, as shown in (a) and (b) of FIG. 1, and the working principle of the PE is similar to that of a typical one-dimensional convolution in a fixed weight data flow. Figure 2

[0045] When the stride is equal to 2, in the FP stage, using a one-dimensional convolution will cause unnecessary calculation overhead, and in the embodiments of the present application, the input data is divided into two groups, i.e., an even input group and an odd input group, and is stored in two parts of the input module FIFO and is sent to different input control modules in parallel, so as to eliminate the useless operation, as shown in (c) and (d) of FIG. 1; in the BP calculation stage, a zero insertion operation is needed, and in order to skip the zero input operation, in the embodiments of the present application, the output data is divided into two groups according to the output coordinates, i.e., an even output group and an odd output group, and the data of the input control module is received in parallel, as shown in (e) and (f) of FIG. 1. Figure 2 Figure 2

[0046] When the stride is equal to 1, the weight kernel of the WG calculation process is large, and the size of the weight kernel is different in different layers, and the gradient and the weight size are the same, so in the embodiments of the present application, the PE basic unit is configured in a fixed output mode as shown in (g) and (h) of FIG. 1. Figure 2

[0047] When the stride is equal to 2, the WG also needs a zero insertion operation, so the input data is grouped to avoid the zero input operation, as shown in (i) and (j) of FIG. 1. Figure 2

[0048] wherein W is a weight, e is an error, i is an input, o is an output, and g is a gradient. Figure 2

[0049] ​​​​​​The embodiment of the application provides a high-efficiency reconfigurable hardware accelerator for convolutional neural network training. In actual use, the PE basic unit performs convolution calculation in rows, and can implement different types of calculation in the convolutional neural network training process in multiple different data flows; including three stages of forward propagation, backward propagation and gradient calculation, full connection layer, 3*3 convolution and 1*1 convolution. Invalid redundant calculation caused by input skipping in forward calculation and zero insertion calculation in reverse calculation of the convolution layer with stride=2 can be avoided, so that the utilization rate of the calculation unit is improved.

[0050] In the embodiment of the application, the PE array of the entire reconfigurable training accelerator is formed based on the PE basic unit. The PE array effectively utilizes the parallelism at each level, greatly improves the data multiplexing times, helps to reduce power consumption and reduce calculation delay, wherein the structure of a PE array cluster is as shown in Figure 3

[0051] In a PE cluster, first, the calculation in a 3*3 convolution kernel is performed simultaneously, so that the calculation of a complete convolution kernel is completed by 3 1*3 PE basic units in parallel. The data between the rows and columns of the input feature map is effectively multiplexed. In addition, C parallel calculation is performed in the input channel dimension, and the summation between the input channels is completed by an addition tree, and the intermediate calculation result is stored in a partial sum buffer. When the next batch of data enters, the intermediate result is taken out from the partial sum buffer and accumulated again in the addition tree.

[0052] The entire PE array includes K PE clusters, and each PE cluster is responsible for processing the calculation of one output channel. That is, the PE array performs K parallel calculation in the output channel dimension. Therefore, the parallelism of the PE array is 3*3*C*K.

[0053] If the parallelism is further improved, R parallel calculation can also be performed in the output row dimension, so that the parallelism of the PE array is 3*3*C*K*R, so that the multiplexing of weight data can be further increased, and the PE array can be calculated in parallel in multiple dimensions, including the row dimension, the column dimension, the input channel and the output channel; so as to realize multiple different types of data service, including input multiplexing, convolution kernel multiplexing and output partial sum multiplexing.

[0054] Further, as Figure 1 ​As shown in the figure, in some embodiments of the present application, the input control module includes a first multiplexer MUX0, a second multiplexer MUX1 and a register D0. Two inputs of the first multiplexer MUX0 are connected to the first FIFO buffer and the second FIFO buffer respectively, and the output is connected to the multiplier (M1). One input of the second multiplexer MUX1 is directly connected to the first FIFO buffer, and the other input is connected to the first FIFO buffer through the register D0, and the output is connected to the multiplier (M2).

[0055] Further, as shown in the figure, Figure 1 As shown in the figure, in some embodiments of the present application, the weight control module includes three multiplexers (MUX2, MUX3, MUX4) and three registers (D1, D2, D3), which are connected to the three multipliers (M0, M1, M2) of the output control module respectively.

[0056] Further, as shown in the figure, Figure 1 As shown in the figure, in some embodiments of the present application, the accumulation unit includes multiplexers (MUX5, MUX6, MUX7), adders (A0, A1, A2) and registers (D4, D5, D6) connected in sequence.

[0057] Further, as shown in the figure, Figure 4 As shown in the figure, the working principle of the storage unit provided in the embodiments of the present application is shown. In some embodiments of the present application, the storage unit includes off-chip DRAM and on-chip SRAM. The on-chip SRAM is used to receive complete data stored in the off-chip DRAM and to block the complete data. For input data and weight data, K channels are used as basic storage units, and data on the input channel is preferentially stored to obtain a complete output feature map. The output feature map is blocked in the row direction, and K channels are blocked in the output channel direction.

[0058] The embodiments of the present application design a flexible storage scheme for the designed PE array to support the access of the training accelerator to the storage, efficiently provide data for calculation in different calculation stages, and realize flexible data distribution according to different calculation requirements. Due to the limitation of the on-chip storage capacity, it is difficult to store a large number of parameters required by a modern deep convolutional neural network, so off-chip DRAM is introduced as a large-capacity storage. However, DRAM has the characteristic of long access time, and its typical bandwidth is about 4GBps, which cannot meet the access bandwidth of high-speed calculation of large-scale PE arrays, and thus an on-chip SRAM cache scheme is introduced.

[0059] In the embodiments of the present application, the on-chip SRAM cache divides the complete data stored in the DRAM into blocks to meet the limited on-chip storage capacity. For input feature maps and weights, K input channels are taken as the basic storage unit, and all elements on the input channel are stored preferentially to obtain complete output feature map elements, avoiding the need for larger partial and storage SRAM capacity. The output feature map is divided in the row direction, and K basic units are divided in the output channel direction of the kernel. Considering that the number of layers of the deep neural network is deepening, the size of the output feature map is becoming smaller, therefore, the batch dimension is introduced to expand the feature map and improve the degree of reuse of kernel data. For the output feature map, after meeting certain row and output channel size, it is written into the off-chip DRAM by the DMA, and the block size depends on the on-chip SRAM capacity, the output feature map and the kernel block condition.

[0060] Further, in some embodiments of the present application, the efficient reconfigurable hardware accelerator for convolutional neural network training further comprises a weight routing network, the weight routing network comprising a first-level weight network and a second-level weight network.

[0061] As shown in Figure 5 , it is a schematic diagram of the working principle of the first-level weight network provided in the embodiments of the present application, the first-level weight network has two transmission modes, in the FP phase, the weight data adopts a direct transmission mode (thick solid line and thin solid line); in the BP phase, the transposition between the input channel and the output channel is realized (thin dotted line and thin solid line). It should be noted that in Figure 5 , 00, 01, 02, 10, 11, 12, 20, 21 and 22 in the first row respectively represent the output channel coordinates, and 00, 10, 20, 01, 11, 21, 02, 12 and 22 in the second row respectively represent the input channel coordinates.

[0062] As shown in Figure 6 , it is a schematic diagram of the working principle of the second-level weight network provided in the embodiments of the present application, the second-level weight network is used to organize the transmission order of a row of weights of a PE basic unit, in the FP phase, the weights are directly propagated according to the original order (direct output); in the BP phase, if the convolution step is equal to 1, the weight column order is transformed from 012 to 210 (even-odd mixing mode), so as to complete the flipping of the weights; if the convolution step is equal to 2, the weight column order is transformed from 012 to 201 (odd-even mixing mode). It should be noted that Figure 6 , 0, 1 and 2 in the above table represent the column coordinates of a row of weights.

[0063] The weight routing network is used to distribute the weight data from the SRAM to the PE array, and it can flexibly transmit data for different computing data streams.

[0064] The embodiment of the present application proposes a two-stage weight routing network. The first stage weight network, as shown in Figure 5 There are two transmission modes, respectively facing the data access requirements in forward propagation and reverse propagation. In forward propagation, all data adopts direct transmission mode, as shown by the thick dashed line and the thin solid line. In reverse calculation, as shown by the thin dashed line and the thin solid line, the transposition between the input channel and the output channel is implemented.

[0065] The second stage weight routing network, as shown in Figure 6 is used to organize the transmission order of a row of weights in a PE. There are three working modes:

[0066] Direct transmission mode: in forward propagation, the weights are directly propagated in the original order.

[0067] Even shuffle mode: in reverse propagation, when the step length is equal to 1, the weight column order is transformed from 012 to 210, so as to complete the flipping of the weights.

[0068] Odd shuffle mode: in reverse propagation, when the step length is equal to 2, the weight column order is transformed from 012 to 201.

[0069] In order to further illustrate the working principle of the efficient reconfigurable hardware accelerator provided by the present application for the training of the convolutional neural network, in the embodiment of the present application, the working process of the accelerator to complete the DNN training is as follows:

[0070] Step 1: start training, set the network model and the maximum number of iterations. Let the current number of iterations be 1.

[0071] Step 2: start from the first layer and calculate layer by layer.

[0072] Step 3: input the configuration information of the current layer (the size of the input, output and weight, the convolution step length, the training stage, etc.), and configure the calculation channel of the accelerator according to the information, and select the best data access scheme (input multiplexing / output multiplexing / weight multiplexing).

[0073] Step 4: load the weight data block and the input data block from the external DRAM to the on-chip SRAM.

[0074] Step 5: read the data from the on-chip SRAM and distribute the data to the PE array through the data routing unit.

[0075] Step 6: the PE array calculates according to the layer configuration and stores the intermediate results in the partial sum cache.

[0076] Step 7: repeat steps 5 to 6 until the current data block is calculated.

[0077] Step 8: after the current data block is calculated, write the result from the partial sum cache to the external DRAM.

[0078] Step 9: Repeat steps 4 to 8 until the current layer is calculated.

[0079] Step 10: After the current layer is calculated, the layer number is added by 1, and steps 3 to 9 are repeated until all layers are calculated, that is, one iteration is completed, and the iteration number is added by 1.

[0080] Step 11: Repeat steps 2 to 10 until the iteration number reaches the maximum iteration number, and the DNN training is completed.

[0081] From the above technical solutions, the embodiment of the application provides an efficient reconfigurable hardware accelerator for convolutional neural network training, which comprises a PE array, the PE array comprises K PE clusters, each PE cluster comprises three input rows, an addition tree and a partial sum buffer, a plurality of PE basic units are arranged in each input channel, summation between the three input rows is completed by the addition tree, and the intermediate calculation result obtained by the addition tree is stored in the partial sum buffer. The PE basic unit comprises an input module, an input control module, a weight control module and an output control module, the input module is connected to the output control module through the input control module, and the weight control module is connected to the output control module. The input module comprises a first first-in-first-out buffer and a second first-in-first-out buffer, which are used to buffer odd-position elements and even-position elements of input features respectively; the output control module comprises three multipliers (M0, M1 and M2) and three accumulation units, the three accumulation units are connected in sequence and are connected to the three multipliers (M0, M1 and M2) respectively.

[0082] In actual application, a unified reconfigurable computing unit (PE basic unit) is provided, which supports completion of multiple different calculation modes in different calculation stages in the training process; parallel calculation can be realized in different dimensions, so as to improve the processing speed; invalid calculation of the convolutional layer with stride=2 in different stages can be eliminated, and the calculation utilization rate is improved; a flexible memory access scheme is used to realize data reuse at different levels, reduce the data access amount, and thus reduce the time and power consumption overhead.

[0083] The application is described in detail above in combination with the specific embodiments and exemplary examples, but these descriptions cannot be understood as limitations of the application. Those skilled in the art understand that the technical solutions and embodiments of the application can be variously replaced, modified or improved without departing from the spirit and scope of the application, and these all fall within the scope of the application. The protection scope of the application is subject to the appended claims.

Claims

1. A highly efficient and reconfigurable hardware accelerator for training convolutional neural networks, characterized in that, It includes a PE array and a weighted routing network. The PE array includes K PE clusters. Each PE cluster includes three input rows, an addition tree, and a partial sum buffer. Each input channel is equipped with C PE basic units. The summation between the three input rows is completed by the addition tree. The intermediate calculation results obtained by the addition tree are stored in the partial sum buffer. The PE basic unit includes: an input module, an input control module, a weight control module, and an output control module. The input module is connected to the output control module through the input control module, and the weight control module is connected to the output control module. The input module includes a first first-in-first-out buffer and a second first-in-first-out buffer to buffer odd-position elements and even-position elements of the input features, respectively; the output control module includes three multipliers (M0, M1, M2) and three accumulators, the three accumulators being connected in sequence and respectively connected to the three multipliers (M0, M1, M2). The input control module is used to transmit input data to the corresponding multiplier according to different calculation modes; The weight control module is used to transmit weight data to the corresponding multiplier according to different calculation modes. The multiplier is used to perform multiplication operations on the input weight data and the input data, and the accumulator is used to accumulate the output structure of the multiplier and output it. The weighted routing network includes a first-level weighted network and a second-level weighted network. The first-level weighted network includes two transmission modes. In the FP phase, weighted data is transmitted directly. In the BP phase, the transposition between the input and output channels is achieved; The second-level weight network is used to organize the transmission order of a row of weights in a PE basic unit; In the FP phase, weights are propagated directly in their original order. In the backpropagation (BP) phase, if the convolution stride is equal to 1, the weight column order changes from 012 to 210, thus completing the weight flip; if the convolution stride is equal to 2, the weight column order changes from 012 to 201.

2. The efficient and reconfigurable hardware accelerator for training convolutional neural networks according to claim 1, characterized in that, The input control module includes a first multiplexer MUX0, a second multiplexer MUX1, and a register D0; The two inputs of the first multiplexer MUX0 are connected to the first first-in-first-out buffer and the second first-in-first-out buffer, respectively, and the output is connected to the multiplier (M1). One input of the second multiplexer MUX1 is directly connected to the first first-in-first-out buffer, and the other input is connected to the first first-in-first-out buffer through the register Do. The output is connected to the multiplier (M2).

3. The efficient and reconfigurable hardware accelerator for training convolutional neural networks according to claim 1, characterized in that, The weight control module includes three multiplexers (MUX2, MUX3, MUX4) and three registers (D1, D2, D3), which are respectively connected to the three multipliers (M0, M1, M2) of the output control module.

4. The efficient and reconfigurable hardware accelerator for training convolutional neural networks according to claim 1, characterized in that, The accumulator unit includes multiplexers (MUX5, MUX6, MUX7), adders (A0, A1, A2), and registers (D4, D5, D6) connected in sequence.

5. The efficient and reconfigurable hardware accelerator for training convolutional neural networks according to claim 1, characterized in that, It also includes a storage unit, which includes off-chip DRAM and on-chip SRAM. The on-chip SRAM is used to receive the complete data stored in the off-chip DRAM and to divide the complete data into blocks. Specifically, for input data and weight data, K channels are used as the basic storage unit. Data on the input channels are stored first to obtain a complete output feature map. The output feature map is then divided into blocks in the row direction and into blocks in the output channel direction using K channels.

6. The efficient and reconfigurable hardware accelerator for training convolutional neural networks according to claim 1, characterized in that, Each PE cluster contains C input channels.

Images