Neural network accelerator, method and electronic chip

Abstract

The invention discloses a neural network accelerator, a method and an electronic chip, and the neural network accelerator comprises a direct memory access module, a global control module, an operation module and an on-chip storage module. The direct memory access module is used for loading feature data, weight data and instruction data; the direct memory access module reads the feature data and the weight data from the on-chip storage module, distributes the feature data and the weight data to the operation module, and writes an operation result back to the on-chip storage module; the global control module is used for distributing stored instruction data to the operation module to perform control information configuration on the operation module when a work starting instruction is received; the operation module is used for controlling and executing neural network operation based on a Winograd algorithm according to the instruction data distributed by the global control module; and the on-chip storage module is used for storing the loaded feature data, the weight data, the operation intermediate data and the operation result data. The method is suitable for efficiency improvement and power consumption reduction of neural network operation.

Description

Technical Field

[0001] The present invention relates to the field of integrated circuit technology, and in particular to a neural network accelerator, method and electronic chip. Background Art

[0002] Convolutional neural networks (CNNs) have developed rapidly in various fields such as image processing due to their good feature extraction and generalization capabilities.

[0003] The architecture of a convolutional neural network generally includes convolution layers and pooling layers. Since convolution operations include a large number of multiplication and addition operations, convolution operations take up most of the processing time of the neural network. The long time taken by convolution operations not only has low computational efficiency, but also consumes a lot of power for the hardware.

[0004] Therefore, in the process of hardware implementation of neural networks, it is necessary to optimize the convolution operation of neural networks to improve computing efficiency and reduce power consumption. Summary of the invention

[0005] The present invention provides a neural network accelerator, method and electronic chip, which can overcome the technical problems existing in the above-mentioned prior art to achieve improved computing efficiency and reduced power consumption.

[0006] A neural network accelerator provided by the present invention is characterized by comprising: a direct memory access module, a global control module, a calculation module and an on-chip storage module;

[0007] The direct memory access module is connected to the external memory through the first bus, and is used to load feature data, weight data and instruction data from the external memory, the loaded feature data and weight data are stored in the on-chip storage module, and the loaded instruction data is stored in the global control module; the direct memory access module is also used to read the feature data and weight data from the on-chip storage module and distribute them to the operation module, and write the operation results back to the on-chip storage module;

[0008] The global control module is used to distribute the stored instruction data to the operation module to configure the control information of the operation module when receiving the start-up work instruction;

[0009] The operation module is used to control the execution of neural network operations based on the Winograd algorithm according to the instruction data distributed by the global control module;

[0010] The on-chip storage module is used to store the loaded feature data, weight data, intermediate calculation data and calculation result data.

[0011] The present invention also provides a neural network acceleration method, comprising:

[0012] Step 1: The direct memory access module loads feature data, weight data and instruction data from the external memory through the first bus; the loaded feature data and weight data are stored in the on-chip storage module, and the loaded instruction data is stored in the global control module;

[0013] Step 2: When the global control module receives the start-up work instruction, it distributes the instruction data to the direct memory access module and the operation module to configure the control information of the operation module;

[0014] Step 3, the direct memory access module reads the feature data and weight data from the on-chip storage module according to the instruction data and distributes them to the operation module;

[0015] Step 4, the operation module controls the execution of a neural network operation based on the Winograd algorithm according to the instruction data;

[0016] Step 5: The direct memory access module writes the calculation result of the calculation module back to the on-chip storage module according to the instruction data.

[0017] The present invention also provides an electronic chip, comprising: the above-mentioned neural network accelerator.

[0018] In the embodiment of the present invention, an on-chip storage module is configured to achieve effective data reuse while reducing external storage access, thereby improving operation efficiency. At the same time, by dividing the on-chip storage module into an input storage module and an output storage module, and adopting a ping-pong storage structure, when performing the current operation, the direct memory access module will store the next data in advance. Such a design prevents the operation unit from being idle due to waiting for data transfer, and further improves the operation efficiency. The neural network operation based on the Winograd algorithm effectively reduces the multiplication operation in the neural network, and also improves the operation efficiency of the neural network and reduces power consumption. Moreover, the above-mentioned accelerator can support a variety of neural networks and input image sizes. Before use, the CPU only needs to generate corresponding instruction data according to the neural network type and input image size to perform subsequent operation operations, so it also has greater versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0020] Figure 1 A schematic diagram of the structure of a neural network accelerator provided by an embodiment of the present invention;

[0021] Figure 2 for Figure 1 Schematic diagram of the structure of the convolution module;

[0022] Figure 3 for Figure 1 Schematic diagram of the structure of the pooling module;

[0023] Figure 4 for Figure 1 A schematic diagram of the structure of the filling module;

[0024] Figure 5 It is a schematic diagram of arrangement and scanning of feature data or weight data in the present invention;

[0025] Figure 6 A flowchart of a method for accelerating a neural network is also provided for an embodiment of the present invention;

[0026] Figure 7 A flowchart of another neural network accelerator provided by an embodiment of the present invention; DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] As mentioned above, in the existing convolutional neural network, since the convolution operation occupies most of the neural network processing time, the long convolution operation not only has low computational efficiency, but also consumes a lot of energy. To this end, the embodiment of the present invention adopts a neural network operation based on the Winograd algorithm to reduce the multiplication operation in the convolution operation, thereby improving the computational efficiency of the convolutional neural network.

[0029] To make the technical solution of the present invention clearer, embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0030] Figure 1 A schematic diagram of the structure of a neural network accelerator provided by an embodiment of the present invention is shown in FIG. Figure 1As shown, the neural network accelerator 100 of this embodiment includes: a direct memory access module (DMA) 10, a global control module 20, an operation module 40 and an on-chip storage module 30; the direct memory access module 10 is connected to the external memory 300 through a first bus for communication, and is used to load feature data, weight data and instruction data from the external memory, and the loaded feature data and weight data are stored in the on-chip storage module 30, and the loaded instruction data is stored in the global control module 20; the direct memory access module 10 is also used to read feature data and weight data from the on-chip storage module and distribute them to the operation module, and write the operation result back to the on-chip storage module; the global control module 20 is used to distribute the stored instruction data to the operation module when receiving the start-up work instruction to configure the control information of the operation module; the operation module 40 is used to control the execution of the neural network operation based on the Winograd algorithm according to the instruction data distributed by the global control module; the on-chip storage module 30 is used to store the loaded feature data, weight data, operation intermediate data and operation result data.

[0031] In the actual application of neural network accelerators, the external CPU pre-calculates the features, weights and all instructions that the neural network accelerator needs to calculate, and then loads them in through DMA, loads the features and weights into the on-chip storage space, and loads the instructions into the instruction RAM in the global control module. After all are loaded, the global control module is started to start the calculation. After the accelerator completes the calculation, it will return an interrupt to the CPU, and then the CPU can read the calculation results in the output storage area.

[0032] In this embodiment, the direct memory access module 10 can be connected to the external memory 300 through the first bus, and under the control of the external CPU, the feature data and weight data are loaded from the external memory to the on-chip storage module, and the instruction data is loaded to the global control module. When the global control module 20 receives the startup work instruction issued by the CPU, it distributes the stored instruction data to the operation module to configure the control information of the operation module. The instruction data contains the configuration information of the enable control signal of the operation module. According to the configuration information of the control signal, the execution of the neural network operation based on the Winograd algorithm can be controlled. Since the Winograd algorithm has the characteristic of reducing the multiplication operation in the neural network, it can improve the operation efficiency of the neural network and reduce power consumption.

[0033] Since convolutional neural networks contain a large number of convolution operations and pooling operations, and the number of multiplications in the convolution operation in the unoptimized convolutional neural network is large, the operation takes a long time, that is, the operation efficiency is low. In order to optimize and improve the operation efficiency of the convolutional neural network, the embodiment of the present invention can effectively improve the operation efficiency of the network by adopting the neural network operation based on the Winograd algorithm.

[0034] In the above embodiment, the operation module includes: a register module, a convolution module, a pooling module and a filling module, wherein: the register module is used to configure the control information of the convolution module, the pooling module and the data filling module according to the instruction data of the global control module; the convolution module is used to perform a neural network convolution operation based on the Winograd algorithm according to the control information configured by the register module; the pooling module is used to perform a neural network data pooling operation according to the control information configured by the register module; the filling module is used to pad the size of the output data to the size of the input data by filling with 0 according to the control information configured by the register module.

[0035] The register module in the operation module includes the enable register, start register, encoding register, length register, weight update register, etc. of each module, which is used to obtain the configuration information of each module from the CPU according to the instruction data.

[0036] In the above embodiment, the global control module includes: an instruction storage unit (instruction RAM) for storing loaded instruction data, so that the global control module can take out the instruction data from the instruction storage unit and distribute it to the operation module when receiving the start work instruction.

[0037] In the above embodiment, each instruction distributed by the global control module 20 will configure the register module in the operation module. The convolution module, pooling module and filling module in the operation module, each module has a corresponding enable register and a corresponding module register. Only when the module is enabled will the corresponding module register be configured. Each instruction is 32 bytes and 256 bits, of which bits 0-15 are the first register address and bits 16-47 are the first register data. Bits 48-63 are the second register address, and bits 64-95 are the second register data; bits 96-111 are the third register address, and bits 112-143 are the third register data; bits 144-159 are the fourth register address, and bits 160-191 are the fourth register data. The selection end of the configuration register from 192 to 195 uses one-hot encoding. When the 192th bit is 1, it indicates that the convolution module is selected. When the 193th bit is 1, it indicates that the pooling module is selected. When the 194th bit is 1, it indicates that the filling module is selected. When the 195th bit is 1, it indicates that the DMA module is selected. The 196th bit indicates whether to read the next instruction. The 197th bit indicates whether to wait for DMA to return information. The 198th bit indicates whether to jump. If jumping, the lower 16 bits are the jump address of the instruction.

[0038] In this embodiment, the convolution module, pooling module and filling module of the operation module are separated and modularized for processing, so that the operation efficiency is higher; and the control information of each module in the operation module is configured through the instruction data distributed by the global control module, thereby further improving the operation efficiency.

[0039] Figure 2 for Figure 1 The structural diagram of the convolution module is as follows: Figure 2 As shown, the convolution module includes: a first input selection module, a first shift module, a first shift control module, a first shift storage module, a dimension transformation module, a Winograd operation module and a first result storage module, wherein: the first input selection module is used to select whether the input data participating in the convolution operation comes from the on-chip storage module or the first result storage module; when the data comes from the first result storage module, it means that the operation result of the previous layer will be used as input data to enter the next layer for operation. The first shift module is used to select the shift output data that needs to participate in the convolution operation through a sliding window operation; the first shift control module is used to control when the shift output data is valid; the first shift storage module is used to store the data participating in the convolution operation output of the first shift module; the dimension transformation module is used to convert the output data of the first shift storage module into a one-dimensional form; the Winograd operation module is used to perform Winograd operation on the output data of the dimension transformation module; the first result storage module is used to store the convolution operation result of the Winograd operation module.

[0040] In the above embodiment, the specific steps of the convolution module receiving the instruction to perform the convolution operation are as follows:

[0041] S11: The direct memory access module reads instruction data from the instruction RAM of the global control module, sends the instruction data to the enable register of the operation module for configuration, and enables the convolution module.

[0042] S21: Configure the enable register, start register, weight update register, encoding register, length register, etc. of each submodule.

[0043] S31: After the first shift module is enabled, the input selection module selects whether the input data comes from the first input storage module or the first result storage module, and after the selection, the input feature data and weight data are obtained from the corresponding module.

[0044] When data comes from the result storage module, it means that the calculation result of the previous layer will enter the next layer as input data for calculation.

[0045] S41: The first shift control module selects the shift output data that needs to be involved in the calculation through a sliding window operation, and stores it in the shift storage module.

[0046] The shift control module is used to control when the shift output is valid. The output is valid when the window is within the valid sliding period. The valid sliding period is determined by the valid sliding start position and the valid sliding end position. The position of the first column of the sliding window is used to determine whether the window is "wrong row". When the window is "wrong row", the shift output is invalid.

[0047] S51: After all data are output, the shifted output data is sent to a dimension conversion module to convert the data into a one-dimensional form.

[0048] S61: Perform Winograd arrangement on the feature data and weight data output by the dimension transformation module, and perform Winograd operation on the arranged data.

[0049] S71: Store the intermediate data of the operation in the first result storage module, store the operation result in the on-chip storage module, and perform the operation of the next layer of network according to the weight update condition.

[0050] Figure 3 for Figure 1 The structural diagram of the pooling module is as follows: Figure 3As shown, the pooling module includes: a second input selection module, a second shift module, a second shift control module, a second shift storage module, a pooling operation module, a mode selection module and a second result storage module, wherein: the second input selection module is used to select whether the input data participating in the pooling operation comes from the on-chip storage module or the result storage module; the second shift module is used to select the shift output data that needs to participate in the pooling operation through a sliding window operation; the second shift control module is used to control when the shift output data is valid; the second shift storage module is used to store the data participating in the pooling operation output of the second shift module; the mode selection module is used to select the maximum pooling, minimum pooling or average pooling for the output data of the second shift storage module according to the control information configured by the register module; the pooling operation module is used to perform corresponding pooling operations on the data according to the selection mode of the mode selection module; the second result storage module is used to store the pooling operation results of the pooling operation module.

[0051] The mode selection module is used to select whether to perform maximum pooling, minimum pooling or average pooling on the data according to the register configuration; specifically, average pooling is performed when the register configuration is 00, maximum pooling is performed when 10, and minimum pooling is performed when 11. The pooling operation module is used to perform corresponding pooling operations on the data according to the selection mode of the mode selection module, and is composed of a comparator and an accumulator. Specifically, when performing maximum pooling, the output data obtained by the second shift module will be sent to the comparator to select the maximum value for output; when performing minimum pooling, the output data obtained by the second shift module will be sent to the comparator to select the minimum value for output; when performing average pooling, the output data obtained by the second shift module will be sent to the accumulator for accumulation and then the average value is calculated for output.

[0052] exist Figure 2 and Figure 3 In the embodiment, since the pooling operation is generally performed after the convolution operation of each layer, for the hardware implementation structure, some modules with the same functions of the two can be shared, and the convolution operation or the pooling operation can be performed by time-sharing multiplexing. In this way, the time-sharing multiplexing of resources can be reasonably used in the utilization of hardware resources, and the hardware structure can be combined with the software optimization algorithm to make the operation more efficient. That is, the second input selection module, the second shift module, the second shift control module, the second shift storage module and the second result storage module of the pooling module and the first input selection module, the first shift module, the first shift control module, the first shift storage module and the first result storage module of the convolution module can be correspondingly implemented by the same module in the hardware circuit, and the operation of the pooling module or the corresponding function of the convolution module can be performed by time-sharing multiplexing, which can save hardware resources.

[0053] In the above embodiment, the filling module includes a data FIFO and a data selection module. Figure 4 for Figure 1The structural diagram of the filling module is as follows: Figure 4 The specific workflow of the filling module is as follows:

[0054] S14: The global control module distributes the instruction to the enable register of the filling module to enable the filling module;

[0055] S24: After the padding module is enabled, register configuration data is received to configure the width of padding 0 and the length information of the length register. The length information of the length register refers to the data information that needs to be skipped when the input data is filled with 0;

[0056] S34: After the register configuration is completed, the filling module receives the start information and sends the data of the configuration length information into the data FIFO;

[0057] S44: The data in the data FIFO is padded with zeros through the data selection module and output. Since both the input and output are in one-dimensional form, the counter is selected to perform the zero padding operation on the data. When the pointers of the row counter and the column counter are both within the valid range of the length information of the data configuration, the data in the data FIFO is selected for output, otherwise the output is 0.

[0058] The on-chip storage module includes: an input storage module and an output storage module. The input storage module and the output storage module adopt a ping-pong storage structure. When each layer of network calculation is completed, the original input storage module is recorded as the output storage module, and the original output storage module is recorded as the input storage module. When performing the current calculation, the direct memory access module will store the next data in advance. This design prevents the calculation unit from being idle due to waiting for data transfer, and can also improve the calculation efficiency.

[0059] The data storage structure can determine the performance of the neural network accelerator. In the embodiment of the present invention, the storage structure of the feature data stored in the chip is arranged in the manner of rows first, columns second, and channels third. The storage structure of the weight data stored in the chip is also arranged in the manner of rows first, columns second, and channels third. Figure 5 It is a schematic diagram of the arrangement and scanning of feature data or weight data in the present invention, such as Figure 5 This arrangement allows both input data and output data to be viewed as one-dimensional inputs, and when performing convolution, pooling, or padding operations, one-dimensional input and one-dimensional output can be achieved, thus unifying the operation data structure.

[0060] In the embodiment of the present invention, an on-chip storage module is configured to achieve effective data reuse while reducing external storage access, thereby improving operation efficiency. At the same time, by dividing the on-chip storage module into an input storage module and an output storage module, and adopting a ping-pong storage structure, when performing the current operation, the direct memory access module will store the next data in advance. Such a design prevents the operation unit from being idle due to waiting for data transfer, and further improves the operation efficiency. The neural network operation based on the Winograd algorithm effectively reduces the multiplication operation in the neural network, and also improves the operation efficiency of the neural network and reduces power consumption. Moreover, the above-mentioned accelerator can support a variety of neural networks and input image sizes. Before use, the CPU only needs to generate corresponding instruction data according to the neural network type and input image size to perform subsequent operation operations, so it also has greater versatility.

[0061] Based on the above Figure 1 The neural network accelerator shown, Figure 6 The present invention also provides a flowchart of a method for accelerating a neural network, such as Figure 6 As shown, the neural network acceleration method in this embodiment includes:

[0062] Step 101, the direct memory access module loads feature data, weight data and instruction data from the external memory through the first bus; the loaded feature data and weight data are stored in the on-chip storage module, and the loaded instruction data is stored in the global control module;

[0063] Step 102: When the global control module receives the start-up work instruction, it distributes the instruction data to the direct memory access module and the operation module to configure the control information of the operation module;

[0064] Step 103, the direct memory access module reads the feature data and weight data from the on-chip storage module according to the instruction data and distributes them to the calculation module;

[0065] Step 104, the operation module controls the execution of a neural network operation based on the Winograd algorithm according to the instruction data;

[0066] Step 105: The direct memory access module writes the operation result of the operation module back to the on-chip storage module according to the instruction data.

[0067] The above operation steps are completed under the control of the pre-generated instructions of the external CPU. The specific process is as follows: the external CPU pre-calculates the features, weights and all instructions that the neural network accelerator needs to calculate, and then loads them in through DMA, loads the features and weights into the on-chip storage space, and loads the instructions into the instruction RAM in the global control module. After all are loaded, the global control module is started to start the calculation. After the accelerator completes the calculation, it will return an interrupt to the CPU, and then the CPU reads the calculation results in the output storage area.

[0068] In the above step 4, the operation module performs Winograd arrangement on the feature data and weight data involved in the operation, and performs a neural network operation of the Winograd algorithm on the arranged data. By adopting the Winograd algorithm and arranging the data suitable for Winograd operation, the multiplication operation in the neural network is reduced.

[0069] Figure 7 A flowchart of another neural network accelerator provided by an embodiment of the present invention, wherein the specific steps are as follows:

[0070] S1: CPU controls the direct memory access module to load feature data and weight data into the on-chip storage module;

[0071] S2: The CPU controls the direct memory access module to load the instruction data into the instruction RAM of the global control module;

[0072] S3: After all data are loaded, the global control module is started to work;

[0073] S4: Read instruction data from the instruction RAM of the global control module and distribute it to the operation module;

[0074] S5: After receiving the instruction, the operation module reads the feature data and weight data from the on-chip storage module through the direct memory access module according to the instruction requirements;

[0075] S6: According to the instruction requirements, the global control module performs register configuration and corresponding data operations on the operation module;

[0076] S7: Operation is completed, and the interrupt is returned to the CPU, otherwise, continue with S4, S5, and S6;

[0077] S8: storing the operation result into the on-chip storage module through the direct memory access module;

[0078] S9: The CPU controls the direct memory access module to read the operation result from the on-chip storage module.

[0079] In this embodiment, the global control module includes an instruction RAM. After receiving the start command, the global control module takes out the instruction data from the instruction RAM and distributes it to the operation module for register configuration. Each module has a corresponding module enable register and module register. Only when the module is enabled will the corresponding module register be configured. The configuration description of each instruction is as described above and will not be repeated here.

[0080] The following example uses the weight data size of 5*5 and the input feature data size of 28*28 to illustrate the workflow of the convolution module. It specifically includes the following steps:

[0081] S21: Set the weight data size to 5*5 and the input feature data size to 28*28.

[0082] S22: Load the feature data and weight data into the on-chip storage module through the direct memory access module. Since the data parallelism is set to 4, data with 4 parallelisms is loaded, that is, 28*28*4 data is loaded for input features and 5*5*4 data is loaded for weights. The data is arranged in the memory in the order of row first, width second, and channel third.

[0083] When both the input channel and the output channel are more than one dimension, such as the size of the input feature is 28*28*8 and the output feature is 5*5*8*4, the input feature is fixed here, and the weights are loaded preferentially according to the output channel, that is, the 28*28*4 input feature data and the 5*5*4 weight data on the first output channel are loaded first, and the weight data on the second output channel is loaded after the operation is completed.

[0084] S23: After the data is loaded, the shift register is configured. Since the input feature is 28*28 and the weight is 5*5, the shift module only needs to enable 140 registers. We configure the last 140 bits of the configuration register to 1 and the other bits to 0, including: the 27th, 55th, 83rd, 111th, and 139th bits of the encoding register are configured to 1, and the other bits are configured to 0. When the shift register is filled, 5 bits of data output can be obtained per cycle.

[0085] S24: After the shift register and the encoding register are configured, the shift buffer enable register is configured. The shift buffer enable register will shift right every cycle, and the lowest bit is used as the enable signal of the shift buffer module. The output valid bit should be judged. The effective sliding start cycle of the window is FMAP_SIZE*KERNEL_SIZE+(KERNEL_SIZE-1), and so on. The last effective sliding cycle is FMAP_SIZE*TAP_NUM+FMAP_SIZE*(FMAP_SIZE-KERNEL_SIZE+1). When the position of the first column of the window is greater than or equal to FMAP_SIZE-(KERNEL_SIZE-1), a row error will occur, and the lowest bit of the enable register should be 1 during the effective output cycle. Where FMAP_SIZE is the size of the input feature, KERNEL_SIZE is the size of the convolution kernel, and since the effective sliding array size is FMAP_SIZE×KERNEL_SIZE, and the first output also requires a KERNEL_SIZE-1 cycle, the starting cycle is FMAP_SIZE×KERNEL_SIZE+(KERNEL_SIZE-1). And so on. The last effective sliding cycle is FMAP_SIZE×KERNEL_SIZE+FMAP_SIZE×(FMAP_SIZE-KERNEL_SIZE+1).

[0086] S25: After the register configuration of S3 and S4 is completed, the start register of the shift module is configured, the shift module starts to work, and the output result is stored in the shift output buffer module. After all the outputs are completed, they are sent to the dimension transformation module for work.

[0087] S26: Rearrange the 5*5 feature data to form the basic components of the Winograd operation with two 5*5 feature arrays. The arrangement is as follows: take the first column of the first 5*5 matrix as K0, the second column as K1, and the third column as K2, and use it as a one-dimensional output; take the first column of the second 5*5 matrix as K1, the second column as K2, and the third column as K3, and use it as a one-dimensional output; since each 5*5 sliding window output has 4 columns belonging to the same columns, the one-dimensional output has the same elements of K1 and K2 between two rows.

[0088] S27: Perform Winograd arrangement on the 5*5 weight data, take the first column of the weight array as W0, the second column as W1, and the third column as W2, then the output matrix of the original matrix operation is [M0+M1+M2, M1-M2-M3], where:

[0089] M0=(K0-K2)*W0,

[0090] M1=(K1+K2)*(W0+W1+W2) / 2

[0091] M2=(K2-K1)*(W0-W1+W2) / 2

[0092] M3=(K1-K3)*W2

[0093] S28: Multiply the fourth and fifth columns of the original two 5*5 feature arrays with the fourth and fifth columns of the weight array respectively and add the results. The results are recorded as S1 and S2. The matrix operation output is [M0+M1+M2+S1, M1-M2-M3+S2].

[0094] The convolution module performs Winograd operations on the results of the dimension transformation module according to the above steps, and then obtains the one-dimensional convolution operation results, which are stored in the result output storage module.

[0095] The embodiment of the present invention further provides an electronic chip, including: the above-mentioned neural network accelerator, which can execute the operation steps of the above-mentioned method embodiment to accelerate the neural network operation. Its working principle and technical effects are similar to those of the above-mentioned neural network accelerator, and will not be repeated here.

[0096] The data in the present invention includes features and weights. The features are stored in the memory in an N-channel arrangement. The features are three-dimensional matrices with a width of WI, a height of HI, and a number of channels of C. It can be applied to CIFAR10, CIFAR100, and handwritten digit recognition. The arrangement order is arranged in N channels, where N is the parallelism of the channel, and the features of each N channels are stored in the memory at continuous addresses, that is, first in the width direction, then in the height direction, and then in the channel direction. Since the total number of channels is C, the data needs to be read C / N times.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A neural network accelerator, characterized in that: include: Direct memory access module, global control module, operation module and on-chip storage module; The direct memory access module is connected to the external memory through the first bus, and is used to load feature data, weight data and instruction data from the external memory, the loaded feature data and weight data are stored in the on-chip storage module, and the loaded instruction data is stored in the global control module; the direct memory access module is also used to read the feature data and weight data from the on-chip storage module and distribute them to the operation module, and write the operation results back to the on-chip storage module; The global control module is used to distribute the stored instruction data to the operation module to configure the control information of the operation module when receiving the start-up work instruction; The operation module is used to control the execution of neural network operations based on the Winograd algorithm according to the instruction data distributed by the global control module; The on-chip storage module is used to store the loaded feature data, weight data, intermediate calculation data and calculation result data.

2. The accelerator according to claim 1, characterized in that The operation module includes: a register module, a convolution module, a pooling module and a filling module, wherein: the register module is used to configure the control information of the convolution module, the pooling module and the filling module according to the instruction data of the global control module; the convolution module is used to perform a neural network convolution operation based on the Winograd algorithm according to the control information configured by the register module; the pooling module is used to perform a neural network data pooling operation according to the control information configured by the register module; the filling module is used to pad the size of the output data to the size of the input data by filling with 0 according to the control information configured by the register module.

3. The accelerator according to claim 2, characterized in that The convolution module includes: a first input selection module, a first shift module, a first shift control module, a first shift storage module, a dimension transformation module, a Winograd operation module and a first result storage module, wherein: The first input selection module is used to select whether the input data participating in the convolution operation comes from the on-chip storage module or the first result storage module; The first shift module is used to select the shift output data that needs to participate in the convolution operation through a sliding window operation; The first shift control module is used to control when the shift output data is valid; The first shift storage module is used to store the data participating in the convolution operation output by the first shift module; The dimension conversion module is used to convert the output data of the first shift storage module into a one-dimensional form; The Winograd operation module is used to perform Winograd operation on the output data of the dimension transformation module; The first result storage module is used to store the convolution operation result of the Winograd operation module.

4. The accelerator according to claim 2, characterized in that: The pooling module includes: a second input selection module, a second shift module, a second shift control module, a second shift storage module, a pooling operation module, a mode selection module and a second result storage module, wherein: The second input selection module is used to select whether the input data participating in the pooling operation comes from the on-chip storage module or the result storage module; The second shift module is used to select the shift output data that needs to participate in the pooling operation through a sliding window operation; The second shift control module is used to control when the shift output data is valid; The second shift storage module is used to store the data participating in the pooling operation output by the second shift module; The mode selection module is used to select maximum pooling, minimum pooling or average pooling for the output data of the second shift storage module according to the control information configured by the register module; The pooling operation module is used to perform corresponding pooling operations on the data according to the selection mode of the mode selection module; The second result storage module is used to store the pooling operation result of the pooling operation module.

5. The accelerator according to claim 4, characterized in that The second input selection module, the second shift module, the second shift control module, the second shift storage module and the second result storage module of the pooling module and the first input selection module, the first shift module, the first shift control module, the first shift storage module and the first result storage module of the convolution module are correspondingly implemented by the same module in the hardware circuit, and the operations of the pooling module or the corresponding functions of the convolution module are executed through time-sharing multiplexing.

6. The accelerator according to any one of claims 1 to 5, characterized in that: The feature data and weight data stored in the on-chip storage module are arranged in a row-first, column-second, and channel-first manner.

7. The accelerator according to claim 1, characterized in that The global control module comprises: an instruction storage unit for storing loaded instruction data, so that the global control module takes out the instruction data from the instruction storage unit and distributes it to the operation module when receiving the start-up work instruction.

8. The accelerator according to claim 1, characterized in that The on-chip storage module includes: an input storage module and an output storage module, and the input storage module and the output storage module adopt a ping-pong storage structure.

9. A neural network acceleration method, characterized in that: include: Step 1: The direct memory access module loads feature data, weight data and instruction data from the external memory through the first bus; the loaded feature data and weight data are stored in the on-chip storage module, and the loaded instruction data is stored in the global control module; Step 2: When the global control module receives the start-up work instruction, it distributes the instruction data to the direct memory access module and the operation module to configure the control information of the operation module; Step 3, the direct memory access module reads the feature data and weight data from the on-chip storage module according to the instruction data and distributes them to the operation module; Step 4, the operation module controls the execution of a neural network operation based on the Winograd algorithm according to the instruction data; Step 5: The direct memory access module writes the calculation result of the calculation module back to the on-chip storage module according to the instruction data.

10. The method according to claim 9, characterized in that In step 4, the operation module performs Winograd arrangement on the feature data and weight data involved in the operation, and performs neural network operation of the Winograd algorithm on the arranged data.

11. An electronic chip, characterized in that: include: A neural network accelerator as claimed in any one of claims 1 to 8.

Images