Neural network accelerator, convolution operation implementation method, device and storage medium

Abstract

The embodiment of the present invention provides a neural network accelerator, a convolution operation implementation method, device and storage medium, which first obtains multiple compressed sub-convolution kernels and weight indexes corresponding to the original convolution kernel, then reads the original data block whose specifications match the sub-convolution kernel specifications from the original data to be processed, and controls the computing unit array to perform an iterative process according to the original data block, the compressed sub-convolution kernel and the weight index, wherein the computing unit array includes multiple multiplication and accumulation calculators; after the iterative process is completed, the output result of the computing unit array is obtained. The neural network convolution operation implementation scheme provided by the embodiment of the present invention reduces the amount of convolution operation by compressing the weights in the convolution kernel and removing the zero-value weights in the original weight convolution kernel. At the same time, because an array-type computing unit array is used to perform convolution operations on data and weights, the data reuse of the convolution operation can be improved, and the bandwidth occupancy and power consumption in the neural network processing process can be reduced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a neural network accelerator, a convolution operation implementation method, a device and a storage medium. Background Art

[0002] In recent years, artificial intelligence technology has developed rapidly around the world. The industry has invested a lot of energy in the research of artificial intelligence technology and has achieved remarkable results, especially in the fields of image detection and recognition and language recognition. The recognition rate of artificial intelligence has exceeded that of humans. Neural network processing is an important processing technology for realizing artificial intelligence. However, the continuous growth of the scale of neural networks has led to the typical neural network having dozens or even hundreds of layers, and the connections between neurons are in the billions. That is, as the performance indicators of the algorithm continue to improve and the amount of calculation continues to increase, the time required to complete the reasoning is also increasing.

[0003] Therefore, how to reduce the processing time and improve the processing efficiency of the hardware accelerator or processor of the neural network is a key technical issue that the industry is currently paying attention to. Summary of the invention

[0004] The neural network accelerator, convolution operation implementation method, device and storage medium provided by the embodiments of the present invention mainly solve the technical problem of how to reduce the processing time of the neural network hardware accelerator or processor and improve the processing efficiency.

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a method for implementing a convolution operation of a neural network, comprising:

[0006] Get the sub-convolution kernel, compressed sub-convolution kernel and weight index corresponding to the original convolution kernel. The sub-convolution kernel is obtained by splitting the original convolution kernel. The compressed sub-convolution kernel and weight index are obtained by removing the zero-value weights in the sub-convolution kernel and rearranging the remaining weight values.

[0007] Reading a raw data block whose specification matches the subconvolution kernel specification from the raw data to be processed;

[0008] Controlling a computing unit array to execute an iterative process according to the original data block, the compressed sub-convolution kernel and the weight index, wherein the computing unit array includes a plurality of multiplication and accumulation calculators;

[0009] After the iteration process is completed, the output results of the calculation unit array are obtained.

[0010] The embodiment of the present invention further provides a device for implementing a neural network convolution operation, comprising:

[0011] The data acquisition module is used to obtain the sub-convolution kernel, compressed sub-convolution kernel and weight index corresponding to the original convolution kernel. The sub-convolution kernel is obtained by splitting the original convolution kernel, and the compressed sub-convolution kernel and weight index are obtained by removing the zero-value weights in the sub-convolution kernel and rearranging the remaining weight values.

[0012] A data reading module, used for reading an original data block whose specifications match the sub-convolution kernel specifications from the original data to be processed;

[0013] An iterative calculation module, used to control the calculation unit array to perform an iterative process according to the original data block, the compressed sub-convolution kernel and the weight index, and the calculation unit array includes a plurality of multiplication and accumulation calculators;

[0014] The result acquisition module is used to obtain the output result of the calculation unit array after the iteration process ends.

[0015] An embodiment of the present invention further provides a neural network accelerator, comprising a processor, a memory, a computing unit array, and a communication bus;

[0016] The communication bus is used to realize the connection and communication between the processor and the memory;

[0017] The processor is used to execute one or more programs stored in the memory to implement the steps of the above-mentioned neural network convolution operation implementation method.

[0018] An embodiment of the present invention also provides a storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the above-mentioned neural network convolution operation implementation method.

[0019] The beneficial effects of the present invention are:

[0020] According to the neural network accelerator, convolution operation implementation method, device and storage medium provided by the embodiment of the present invention, first obtain multiple compressed sub-convolution kernels and weight indexes corresponding to the original convolution kernel, the compressed sub-convolution kernel is obtained by segmenting and compressing the original convolution kernel, the segmentation process includes the sub-convolution kernel obtained by segmenting the original convolution kernel, the compression process includes removing the zero-value weights in the sub-convolution kernel, rearranging the remaining weight values ​​and determining the weight index; then read the original data block whose specifications match the sub-convolution kernel specifications from the original data to be processed, control the calculation unit array to perform an iterative process according to the original data block, the compressed sub-convolution kernel and the weight index, wherein the calculation unit array includes multiple multiplication and accumulation calculators; after the iterative process ends, obtain the output result of the calculation unit array. The neural network convolution operation implementation scheme provided by the embodiment of the present invention reduces the amount of convolution operation by compressing the weights in the convolution kernel and removing the zero-value weights in the original weight convolution kernel. At the same time, because an array-type calculation unit array is used to perform convolution operation on data and weights, the data reuse of the convolution operation can be improved, and the bandwidth occupancy and power consumption in the neural network processing process can be reduced.

[0021] Other features and corresponding beneficial effects of the present invention are described in the latter part of the specification, and it should be understood that at least part of the beneficial effects become obvious from the description in the specification of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A structural schematic diagram of a neural network convolution operation implementation system shown in Embodiment 1 of the present invention;

[0023] Figure 2 A flow chart of a method for implementing a convolution operation of a neural network provided in Embodiment 1 of the present invention;

[0024] Figure 3 A schematic diagram of performing weight compression on an original convolution kernel to obtain a compressed sub-convolution kernel shown in the first embodiment of the present invention;

[0025] Figure 4 A schematic diagram of a computing unit provided in Embodiment 1 of the present invention;

[0026] Figure 5 A schematic diagram of a computing unit array provided in Embodiment 1 of the present invention;

[0027] Figure 6 for Figure 5 A schematic diagram of the convolution operation of the weights and data by the computing unit array in the figure;

[0028] Figure 7 A schematic diagram of the structure of the neural network convolution operation implementation system shown in the second embodiment of the present invention;

[0029] Figure 8 A flow chart of a method for implementing a convolution operation of a neural network provided in Embodiment 2 of the present invention;

[0030] Fig. 9 Another structural schematic diagram of the neural network convolution operation implementation system shown in the second embodiment of the present invention;

[0031] Fig.10 This is a schematic diagram of weight compression performed by the compiler shown in Embodiment 3 of the present invention;

[0032] Fig.11 A schematic diagram of adapting the specification of the computing unit array according to the convolution kernel shown in the third embodiment of the present invention;

[0033] Fig.12 A schematic diagram of a hardware structure of a neural network accelerator provided in Embodiment 4 of the present invention;

[0034] Fig.13 A schematic diagram of the structure of a device for implementing a convolution operation of a neural network provided in Embodiment 4 of the present invention;

[0035] Fig.14 Another structural schematic diagram of the neural network convolution operation implementation device provided in the fourth embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the following is a further detailed description of the embodiments of the present invention through specific implementation methods combined with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0037] Embodiment 1:

[0038] In order to improve the processing efficiency of neural network processing and reduce the amount of calculation of neural network convolution operation, this embodiment provides a neural network convolution operation implementation method. Before introducing the neural network convolution operation implementation method, this embodiment first introduces a neural network convolution operation implementation device that can implement the neural network convolution operation implementation method. Figure 1 :

[0039] The neural network convolution operation implementation system 1 includes a CPU 11, a global cache 12 and a neural network accelerator 10, wherein the neural network accelerator 10 includes an input and output cache 13, a command parsing unit 14, a computing control unit 15 and a computing unit array 16.

[0040] Among them, the CPU 11 is connected in communication with the command parsing unit 14, and the CPU 11 can send control instructions to the command parsing unit 14, and the command parsing unit 14 is connected in communication with the computing control unit 15 and the input and output cache 13, which can parse the control instructions sent by the CPU 11, and then send the operation strategy obtained according to the control instructions to the computing control unit 15, so that the computing control unit 15 configures the computing unit array 16; at the same time, after receiving the control instructions sent by the CPU 11, the command parsing unit 14 can also obtain the convolution calculation preparation strategy, and the command parsing unit 14 can send the convolution preparation strategy to the input and output cache 13, so that the input and output cache reads the information required for the convolution operation from the global cache 12. The information stored in the global cache 12 includes the original data to be inferred, the compressed sub-convolution kernel, and the weight index. In some examples of this embodiment, the global cache 12 can also store control instructions. In some other examples of this embodiment, the global cache 12 can also be used to store the intermediate results of the computing unit array 16.

[0041] Combine the following Figure 1 The neural network convolution operation implementation system introduces the implementation method of neural network convolution operation, see Figure 2 , Figure 2 What is shown is a flow chart of a method for implementing a convolution operation of a neural network:

[0042] S202: The neural network accelerator obtains the sub-convolution kernel, the compressed sub-convolution kernel, and the weight index corresponding to the original convolution kernel.

[0043] In this embodiment, the original convolution kernel refers to the convolution kernel that originally needs to participate in the convolution operation, and the compressed sub-convolution kernel is the convolution kernel obtained by segmenting and compressing the original convolution kernel.

[0044] In the segmentation process, sub-convolution kernels can be segmented from the original convolution kernel. Here, it is assumed that the specifications of the original convolution kernel are R*S*CH, where R is the column width of the original convolution kernel, S is the row width of the original convolution kernel, and CH is the number of channels of the original convolution kernel. The specifications of the sub-convolution kernel obtained by the segmentation process are R*S*C, where C≤CH. It can be seen that the column width and row width of the sub-convolution kernel are consistent with the column width and row width of the original convolution kernel, such as Figure 3 ,exist Figure 3 In the figure, the original convolution kernel 31 and a sub-convolution kernel 32 obtained by dividing the original convolution kernel 31 only differ in the number of channels.

[0045] After the segmentation process, compression processing is also required. Compression processing refers to the process of removing the zero-value weights in each sub-convolution kernel obtained by the segmentation process, retaining the non-zero weights, and rearranging the non-zero weights. It should be understood that after the non-zero weights are reordered, in order to characterize the sub-convolution kernel before compression by the compressed sub-convolution kernel, the weight index corresponding to the compressed sub-convolution kernel can be determined according to the absolute position of the zero-value weights in the sub-convolution kernel before compression, the absolute position of the non-zero weights, or the relative position between the zero-value weights and the zero-value weights, and the relative position between the non-zero weights and the non-zero weights. The sub-convolution kernel before compression can be restored through the weight index, the compressed sub-convolution kernel, and the rule for determining the weight index.

[0046] For convolutional neural networks, convolution kernels are generally three-dimensional structures. The weights in the convolution kernels can be pruned into a state where the same channel C has zero weights through training. The weight indexes corresponding to each compressed sub-convolution kernel are consistent, that is, there is only one weight index in the entire weight index. This can further reduce the weight compression index table and improve compression efficiency. The following is an introduction to several methods for determining weight indexes proposed in this embodiment for reference with examples:

[0047] Method 1: Determine the weight index corresponding to the compressed sub-convolution kernel according to the absolute position of the zero-value weight in the sub-convolution kernel.

[0048] For example, the sub-convolution kernel is {1,0,3,0,0,0,0,9}, a total of 8 numbers, then the compressed sub-convolution kernel after compression is {1,3,9}, and the corresponding weight index is {0,2,8}, that is, the weight index represents the absolute position of the weight retained after compression in the sub-convolution kernel. Specifically, for the weight value "1" retained after compression, its position number in the sub-convolution kernel is 0, and for the weight value "3" retained after compression, its position number in the sub-convolution kernel is 2, and for the weight value "9" retained after compression, its position number in the sub-convolution kernel is 8. It can be understood that method 1 is more suitable for scenarios with higher weight compression rates, that is, scenarios with more zero-value weights in the sub-convolution kernel.

[0049] Method 2: Determine the weight index corresponding to the compressed sub-convolution kernel according to the number of zero-valued weights between the non-zero weights in the sub-convolution kernel and the previous non-zero weights.

[0050] Continuing to assume that the sub-convolution kernel is {1,0,3,0,0,0,0,9}, the compressed sub-convolution kernel is {1,3,9}, and the corresponding weight index is {0,1,4}. The weight index represents the number of zero-valued weights between the non-zero weights in the sub-convolution kernel relative to the previous non-zero weight. For example, for the weight value "1" retained after compression, there is no zero-valued weight in the middle relative to the previous non-zero weight, that is, the number of zero-valued weights in the interval is 0 (there is no non-zero weight in front of it); for the weight value "3" retained after compression, there is 1 zero-valued weight in the middle of the sub-convolution kernel relative to the previous non-zero weight; for the weight value "9" retained after compression, there are 4 zero-valued weights in the middle of the sub-convolution kernel relative to the previous non-zero weight.

[0051] It should be understood that when the weight index is determined according to method 2, the fewer the number of weight values ​​retained in the compressed sub-convolution kernel, the simpler the weight index is, and the simpler it is to determine the weight index. Therefore, method 2 is also more suitable for scenarios with higher compression rates.

[0052] Method 3: Determine the weight index corresponding to the compressed sub-convolution kernel according to the absolute position of the non-zero weight in the sub-convolution kernel.

[0053] Assume that the sub-convolution kernel is {1, 4, 3, 0, 7, 7, 0, 9}, a total of 8 numbers, and the compressed sub-convolution kernel after weight compression is {1, 4, 3, 7, 7, 9}. Since in this compression scenario, the compressed sub-convolution kernel retains more weight values ​​than the sub-convolution kernel before compression, that is, the compression rate is lower. Therefore, if the scheme provided by the aforementioned method 1 or method 2 is continued to be used to determine the weight index, the work of determining the weight index is relatively large, and there will be more types of weight indexes. Therefore, here is a method for determining the weight index suitable for scenarios with low compression rates: determine the weight index corresponding to the compressed sub-convolution kernel according to the absolute position of the non-zero weight in the sub-convolution kernel. The weight index that can be determined by this compression method is {3,7}, where "3" represents the relative position of the zero-value weight in the sub-convolution kernel before compression, that is, the position number (position number starts from 0) is 3, therefore, the first zero-value weight in the sub-convolution kernel is after "3"; correspondingly, "7" indicates that the position number of the second zero-value weight in the sub-convolution kernel is 7, therefore, the second zero-value weight appears in front of 9.

[0054] Method 4: Determine the weight index corresponding to the compressed sub-convolution kernel according to the number of non-zero weights between the zero-valued weight in the sub-convolution kernel and the previous zero-valued weight.

[0055] Assuming that the subconvolution kernel before compression is {1,4,3,0,7,7,0,9}, the compressed subconvolution kernel after compression is {1,4,3,7,7,9}, and the corresponding weight index is {3,2}. Because this method determines the weight index according to the number of non-zero weights between the zero-valued weights in the subconvolution kernel and the previous zero-valued weight, the weight index "3" represents that the number of non-zero weights between the first zero-valued weight in the subconvolution kernel and the previous zero-valued weight is 3, so the first zero-valued weight appears at position number 3 in the subconvolution kernel; and "2" means that the number of non-zero weights between the second zero-valued weight in the subconvolution kernel and the previous zero weight is 2.

[0056] It is understandable that, compared with the scenario with a high compression rate, the method of determining the weight index proposed in method 4 is more suitable for application in the scenario with a low compression rate.

[0057] S204: The neural network accelerator reads an original data block whose specifications match the sub-convolution kernel specifications from the original data to be processed.

[0058] The so-called "original data block" refers to the original data block whose specifications match the sub-convolution kernel specifications read from the original data to be inferred or processed. Here, "matching" can mean that the specifications of the original data block and the sub-convolution kernel are exactly the same, but they can also be different. In the convolution preparation strategy, the input and output caches are instructed how to read the original data blocks from the global cache, that is, the specifications of the original data blocks are indicated. This specification is determined based on the size specifications of the sub-convolution kernel and the size of the MAC array.

[0059] In this embodiment, the neural network accelerator can receive control instructions from the CPU of the terminal, and then parse the control instructions to determine the operation strategy and convolution preparation strategy for performing the neural network convolution operation.

[0060] In this embodiment, the control instruction is an instruction for indicating the operation strategy of the neural network convolution operation and the data preparation work before the neural network convolution operation. Therefore, by parsing the control instruction, the operation strategy and the convolution preparation strategy can be obtained.

[0061] The computing unit array is configured according to the operation strategy, and the original data block whose specifications match the sub-convolution kernel specifications is read from the original data according to the convolution preparation strategy, and the original data block, the compressed sub-convolution kernel and the weight index are stored in the input and output buffers.

[0062] In some examples of the present embodiment, the operation strategy is actually information for indicating the functional configuration of the computing unit array. The operation strategy can indicate the specifications of the computing unit array. Because in some examples of the present embodiment, the specifications of the computing unit array can be adapted according to the computing requirements of the original data to be processed or inferred. In this way, when the amount of data in the original data that needs to participate in the convolution operation is not large, it is possible to consider using a computing unit array with a smaller specification size, that is, using a computing unit array with a smaller number of multiplication and accumulation units. This can avoid the problem of low computing efficiency of the computing unit array due to the large computing unit array but the small amount of computing data.

[0063] In addition, the operation strategy can also indicate how the computing unit array performs convolution operations on the weight value and the data. For example, the operation strategy can indicate the number of iterations in the calculation process. In this embodiment, the computing unit MAC is actually a multiply accumulate (MAC) unit. Figure 4 The schematic diagram of the multiplication and accumulation unit is shown: the multiplication and accumulation unit 40 includes a multiplier 41 and an accumulator 42. Data and weights are continuously input into the multiplier 41 in each beat. After the multiplication operation of the multiplier 41, they are sent to the accumulator 42 to complete the result of the corresponding convolution operation. Figure 5 A schematic diagram of a computing unit array is shown in Figure 5 Among them, the computing unit array 50 includes a plurality of computing units, and these computing units form a matrix.

[0064] The convolution preparation strategy is used to instruct the original data block, compressed sub-convolution kernel and weight index to be read before the convolution calculation, and then these contents are stored in the input and output cache. Because when performing convolution operations, inputting weight values ​​and data from the input and output cache to the computing unit array is faster than temporarily obtaining data and weights from the global cache, etc., which reduces the number of times the original data block or compressed convolution kernel is read and written from the global cache, reduces the data movement process, and thus effectively reduces bandwidth and power consumption.

[0065] After parsing the control instructions to determine the operation strategy and convolution preparation strategy, the computing unit array used in this neural network convolution operation can be functionally configured according to the operation strategy, and the original data blocks with specifications matching the sub-convolution kernel specifications can be read from the global cache according to the convolution preparation strategy, as well as the compressed sub-convolution kernel and weight index, and these contents can be stored in the input and output cache to perform preparations before the convolution operation.

[0066] S206: The neural network accelerator controls the computing unit array to execute an iterative process according to the original data block, the compressed sub-convolution kernel, and the weight index.

[0067] In this embodiment, the computing unit array can perform convolution operations according to the compressed sub-convolution kernel, the weight index and the original data block. Figure 6 Shows Figure 5 A schematic diagram of convolution operation on the computing unit array in :

[0068] The weight values ​​in the compressed sub-convolution kernels (A1, A2...Ai; B1, B2...Bi; C1, C2...Ci, etc.) can be gradually input from the first side of the computing unit array according to the computing beat of the computing unit array 50, and the data read from the original data block (a1, a2...aj; b1, b2...bj; c1, c2...cj, etc.) can be gradually input from the second side of the computing unit array according to the computing beat of the computing unit array 50. The weight values ​​entering the computing unit array 50 will gradually flow from the first side to the third side opposite to the first side with the computing beat, and the data entering the computing unit array 50 will gradually flow from the second side to the fourth side with the computing beat. Since the computing unit array 50 is a matrix array, the computing unit array 50 includes four sides, the first side is opposite to the third side, and the second side is opposite to the fourth side, so if the first side is the long side of the computing unit array 50, the second side is the wide side of the computing unit array 50; if the first side is the wide side of the computing unit array 50, the second side is the long side of the computing unit array 50. Figure 5 Among them, the weight value in the compressed sub-convolution kernel is input into the computing unit array from the upper edge of the computing unit array 50, and the data read from the original data block is input into the computing unit array from the left edge of the computing unit array 50.

[0069] It can be understood that, in the first calculation beat, the weight values ​​A1, B1, C1...N1 in the compressed subconvolution kernel are input into the first row of the calculation unit array 50, and the data a1, b1, c1...m1 in the original data block are input into the first column of the calculation unit array 50. In the next calculation beat, the weight values ​​A1, B1, C1...N1 will enter the second row of the calculation unit array 50, and the weight values ​​A2, B2, C2...N2 will be input into the first row of the calculation unit array 50; and the data a1, b1, c1...m1 in the original data block are input into the second column of the calculation unit array 50, and the data a2, b2, c2...m2 in the original data block will be input into the first column of the calculation unit array 50... and so on until the calculation unit array 50 obtains the output result.

[0070] It should be noted that the process of performing convolution operation by the computing unit array is an iterative process. Therefore, after the computing unit array obtains the output result, the output result will be used as the original data block for the next iteration.

[0071] In addition, it is worth noting that since the convolution operation performed by the computing unit array uses a compressed sub-convolution kernel, the compressed sub-convolution kernel has been compressed and the zero-value weights have been removed. However, the size specification of the original data block is a data block that matches the size specification of the sub-convolution kernel before compression. Therefore, not all data in the original data block will be input into the computing unit array. When reading data from the input and output cache, the computing unit array can determine which data to read based on the weight index corresponding to the compressed sub-convolution kernel. Optionally, for any compressed sub-convolution kernel, the computing unit array determines the data in the original data block corresponding to each weight in the compressed sub-convolution kernel based on the weight index, and then inputs the determined data from the second side of the computing unit array.

[0072] S208: After the iteration process is completed, the neural network accelerator obtains the output results of the computing unit array.

[0073] After the iteration process is finished, the output result of the last iteration of the computing unit array can be obtained, which is the convolution result of the original data and the original convolution kernel.

[0074] In this embodiment, a neural network convolution operation implementation system and a neural network convolution operation implementation method are provided. Through the computing unit array, a "pipeline" convolution operation is performed on the original data block and the weight value in the compressed sub-convolution kernel according to the computing rhythm. The computing speed is fast and the computing efficiency is high, which can shorten the time spent on convolution operation in the neural network processing process.

[0075] At the same time, in this embodiment, because the weights participating in the convolution operation of the computing unit array are compressed, many zero-value weights in the original convolution kernel are removed. Therefore, this reduces the storage resources required for the global cache to store the convolution kernel. More importantly, when performing a convolution operation, the computing unit array does not need to calculate the zero-value weights and the weight values ​​corresponding to the zero-value weights in the original data block. Therefore, this greatly reduces the computational complexity of the convolution operation.

[0076] Embodiment 2:

[0077] This embodiment will continue to introduce the neural network convolution operation implementation method and device provided by the present invention. Figure 7 :

[0078] exist Figure 7The neural network convolution operation implementation system 7 shown also includes a CPU 71, a global cache 72 and a neural network accelerator 70. However, unlike the first embodiment, in this embodiment, the neural network accelerator 70 includes not only an input and output cache 73, a command parsing unit 74, a calculation control unit 75 and a calculation unit array 76, but also a Winograde conversion unit 77. The Winograde conversion unit 77 is arranged between the input and output cache 73 and the calculation unit array 76, and is used to convert the compressed sub-convolution kernel and the original data block to the Winograde domain after determining that the compressed sub-convolution kernel is a convolution kernel supporting the Winograde algorithm, and then input the compressed sub-convolution kernel transferred to the Winograde domain and the data in the original data block to the calculation unit array 76, so that the calculation unit array 76 performs the convolution operation between the compressed sub-convolution and the original data block according to the configuration under the Winograde domain. The Winograde conversion unit 77 is also used to convert the output result of the last iteration of the calculation unit array into the digital domain to obtain the final result of this neural network convolution operation.

[0079] Combine the following Figure 8 The flowchart shown is used to illustrate the neural network convolution operation implementation method provided in this embodiment:

[0080] S802: The global cache obtains multiple compressed sub-convolution kernels and weight indexes corresponding to the original convolution kernel.

[0081] In this embodiment, the process of compressing the original convolution kernel can be implemented by other devices other than the neural network convolution operation implementation system 7. The global cache 72 can obtain the compressed sub-convolution kernel and the corresponding weight index corresponding to the original convolution kernel from other devices, and store the compressed sub-convolution kernel and the weight index.

[0082] Typically, the global cache can also store raw data to be inferred and control instructions. Those skilled in the art can understand that the global cache can store control instructions, and when the CPU needs to send control instructions to the command parsing unit, the CPU extracts the control instructions from the global cache; however, in some other examples of this embodiment, the global cache may not store control instructions, so that the CPU can generate control instructions in real time when sending control instructions to the command parsing unit.

[0083] S804: The CPU sends a control instruction to the command parsing unit.

[0084] It is understandable that the global cache 72 can store not only the compressed sub-convolution kernels and weight indexes, but also the control instructions sent by the CPU 71 to the command parsing unit 74. Figure 7There is a communication connection between the global cache 72 and the CPU 71 .

[0085] S806: The command parsing unit parses the control instruction to determine the operation strategy and convolution preparation strategy for performing the neural network convolution operation.

[0086] In this embodiment, the control instruction is an instruction for indicating the operation strategy of the neural network convolution operation and the data preparation work before the neural network convolution operation. Therefore, by parsing the control instruction, the operation strategy and the convolution preparation strategy can be obtained.

[0087] S808: The computing control unit configures the computing unit array according to the computing strategy.

[0088] After the control instruction is parsed to determine the operation strategy and convolution preparation strategy, the computing unit array used in the current neural network convolution operation can be functionally configured according to the operation strategy. The operation strategy can indicate the specifications of the computing unit array 76, so the computing control unit 75 can configure the specifications of the computing unit array 76 according to the specifications indicated by the operation strategy.

[0089] S810: The input-output cache reads the original data block whose specifications match the sub-convolution kernel specifications from the original data according to the convolution preparation strategy, and stores the original data block, the compressed sub-convolution kernel, and the weight index into the input-output cache.

[0090] After the command parsing unit 74 parses and obtains the convolution preparation strategy, the input-output cache 73 can read the original data blocks whose specifications match the sub-convolution kernel specifications, as well as the compressed sub-convolution kernel and weight index from the global cache 72 according to the convolution preparation strategy, and cache these contents.

[0091] It can be understood that, in the present embodiment, the process of preparing the input and output cache 73 for the convolution operation is performed after the process of configuring the computing unit array 76 by the computing control unit 75, but those skilled in the art can understand that the process of extracting information such as the original data block from the global cache 72 can also be executed before the configuration process of the computing unit array 76, or the two processes can be performed simultaneously.

[0092] S812: Convert the compressed sub-convolution kernel and the original data block to the Winograde domain.

[0093] Since performing convolution operation in the Winograde domain can reduce the amount of convolution operation calculation, in some examples of this embodiment, before the calculation unit array performs convolution operation on the weight value of the compressed subconvolution terminal and the data in the original data block, the Winograde conversion unit 77 can convert the data of the original data block and the compressed subconvolution to the Winograde domain. The conversion formula of Winograde is as follows:

[0094]

[0095] U=GFG T V=B T IB

[0096] Among them, F is the input weight, In is the input data. T and a transformation of B, used to transform the data into the Winograd domain, where B T is the transposed matrix of B. The weights are G T and a transformation of G, which is used to convert the weights into the Winograd domain, where G T is the transposed matrix of G. After the transformed weights U and data V are convolved, they need to be converted to the data domain and then the result is input. Through optimization, it can be converted to This can further effectively reduce the multiplier operations and thus improve the processing capability.

[0097] It should be understood that the Winograd domain conversion performed by the Winograd conversion unit 77 is conditional, that is, the Winograd conversion unit 77 cannot convert all compressed sub-convolution kernels into the Winograd domain, nor can it convert all original data blocks into the Winograd domain. When performing the Winograd domain conversion, the Winograd conversion unit 77 has requirements on the specifications of the compressed sub-convolution kernels and the original data blocks. The Winograd conversion unit 77 requires that the row width and column width of the compressed sub-convolution kernel are the same, that is, one face of the compressed sub-convolution kernel is a square. If this condition is not met, the Winograd conversion unit 77 cannot perform the Winograd domain conversion on the compressed sub-convolution kernel.

[0098] When the sub-convolution kernel is compressed, the row width and column width between the compressed sub-convolution kernel are guaranteed to be consistent (that is, the row width of the compressed sub-convolution kernel is consistent with that of the sub-convolution kernel, and the column width of the compressed sub-convolution kernel is consistent with that of the sub-convolution kernel), and the specifications of the original data block match the specifications of the sub-convolution kernel, therefore, if the Winograd transformation unit 77 can perform Winograd domain transformation on the compressed sub-convolution kernel, then the Winograd transformation unit 77 can perform Winograd domain transformation on the original data block.

[0099] S814: Control the computing unit array to execute the iterative process according to the configuration.

[0100] It can be understood that after the Winograd domain conversion operation of the Winograd conversion unit 77, the computing unit array 76 performs a convolution operation on the weight value in the compressed sub-convolution kernel and the data in the original data block under the Winograd domain. The computing unit array 76 can input the weight value in the compressed sub-convolution kernel from the upper side, allowing the weight value to gradually flow to the lower side with the calculation beat; at the same time, the computing unit array 76 can read the data in the original data block from the input and output buffer 73 according to the weight index, and input the read data from the left side, allowing the data to flow to the right side with the calculation beat.

[0101] After the computing unit array 76 obtains the output result of the convolution operation, the output result is stored as an intermediate result in the input-output cache 73. In the next iteration, the data in the intermediate result is read and the weight is convolved. It can be understood that the storage capacity of the input-output cache 73 is relatively limited. Therefore, if the storage resources of the input-output cache 73 are insufficient to store the intermediate result of the computing unit array 76, the input-output cache 73 can store the intermediate result in the global cache 72.

[0102] In some examples of this embodiment, a compression and decompression unit may be provided between the input and output buffer 73 and the global buffer 72. Fig. 9 Another structural schematic diagram of the neural network convolution operation implementation system 7 is shown:

[0103] The compression and decompression unit 78 can compress the intermediate results and discard the zero-value data in the intermediate results before storing the intermediate results of each layer of the computing unit array 76 in the global cache 72; then, before the input-output cache 73 reads the intermediate results from the global cache 72, it first extracts the compressed intermediate results from the global cache 72, decompresses the compressed intermediate results, restores the original intermediate results, and then transmits them to the input-output cache 73.

[0104] S816: The Winograd conversion unit obtains the output result of the last iteration of the computing unit array, and converts the output result of the last iteration into the digital domain to obtain the final result of this neural network convolution operation.

[0105] After the iteration process is completed, the Winograd conversion unit 77 can obtain the output result of the last iteration of the computing unit array 76, and then convert the output result from the Winograd domain to the digital domain to obtain the final result of this neural network convolution operation.

[0106] It can be understood that the neural network convolution operation implementation method and device provided in this embodiment only implement the convolution operation in the neural network processing process. For the entire neural network processing process, in addition to the convolution operation, it is also necessary to undergo ReLU (Rectified Linear Unit) activation function operation and pooling operation to obtain the final inference result.

[0107] The neural network convolution operation implementation method and device provided in the embodiment of the present invention can not only reduce the computational complexity of the convolution operation by compressing the convolution kernel, but also further reduce the computational complexity of the convolution operation by Winograd domain conversion, thereby improving the efficiency of the convolution operation processing and reducing the occupation of processing resources by the convolution operation processing.

[0108] Embodiment three:

[0109] This embodiment will combine Figure 7 or Figure 8 The neural network convolution operation implementation system and some examples shown continue to introduce the neural network convolution operation implementation provided in the aforementioned embodiments:

[0110]

Weight Conversion

[0111] Fig.10 A schematic diagram of using a compiler for weight compression is shown:

[0112] The compiler is divided into a segmentation module 101 and a compression module 102. After being processed by the two modules, the original convolution kernel becomes a compressed sub-convolution kernel and a weight index.

[0113] For convolutional neural networks, since convolution kernels are generally three-dimensional structures, the weights in the convolution kernels can be pruned through training so that the same channel C is 0. This can further reduce the content in the weight index and improve the compression efficiency. The following takes the 3×3 original convolution kernel as an example:

[0114] Assume that the size of the original convolution kernel is 3×3×8, and its initial value is:

[0115] {

[0116] {1,0,3,0,0,0,0,9}

[0117] {8,0,4,0,0,0,0,6}

[0118] {1,0,5,0,0,0,0,8}

[0119] {2,0,6,0,0,0,0,5}

[0120] {4,0,3,0,0,0,0,6}

[0121] {6,0,5,0,0,0,0,7}

[0122] {3,0,8,0,0,0,0,5}

[0123] {7,0,0,0,0,0,0,1}

[0124] {9,0,1,0,0,0,0,2}

[0125] }

[0126] The compressed convolution kernel is:

[0127] {

[0128] {1,3,9}

[0129] {8,4,6}

[0130] {1,5,8}

[0131] {2,6,5}

[0132] {4,3,6}

[0133] {6,5,7}

[0134] {3,8,5}

[0135] {7,0,1}

[0136] {9,1,2}

[0137] }

[0138] The weight index is determined according to the method 1 provided in the first embodiment, and the weight index is: {1, 3, 9}.

[0139] The weight index is determined according to the method 1 provided in the first embodiment, and the weight index is: {0, 1, 4}.

[0140]

Command parsing and parameter configuration

[0141] After the computing control unit parses the control instruction sent by the CPU, it can configure the computing unit array.

[0142]

Convolution Preparation

[0143] For a scheduled execution of the computing unit array, the input and output cache can read the original data block of the same size (R*S*C) as the sub-convolution kernel from the global cache for storage. On the other hand, the input and output cache can also read the sub-convolution kernel weight indicator to be processed from the global cache to the indicator cache.

[0144] The calculation control unit can read the weight values ​​in the compressed sub-convolution kernels in sequence, and read the data in the original data block from the input and output buffers according to the weight compression table.

[0145] Winograd conversion

[0146] Winograde's conversion formula is as follows:

[0147]

[0148] U=GFG T V=B T IB

[0149] Among them, F is the input weight, In is the input data. T and a transformation of B, used to transform the data into the Winograd domain, where B T is the transposed matrix of B. The weights are G T and a transformation of G, which is used to convert the weights into the Winograd domain, where G T is the transposed matrix of G. After the transformed weights U and data V are convolved, they need to be converted to the data domain and then the result is input. Through optimization, it can be converted to This can further effectively reduce the multiplier operations and thus improve the processing capability.

[0150]

Multiply and accumulate operation

[0151] For convolutional neural networks, the multiplication and addition processing units are as follows: Figure 4 As shown, the multiplier 41 and the accumulator 42 are used to implement the process. The data and the weight are continuously input into the multiplier 41 in each beat. After the multiplication operation, the data and the weight are sent to the accumulator 42 to complete the result of the corresponding convolution operation.

[0152] Figure 6The values ​​shown are a computing unit array composed of M*N multiplication and accumulation units. It can be seen from the figure that data is input into the computing unit array from left to right at each beat, and is sent to the next column of multiplication and accumulation units through the multiplication and accumulation units to complete other convolution operations; weights are input into the computing unit array from top to bottom at each beat, and are sent to the next row of multiplication and accumulation units through the multiplication and accumulation units to complete other convolution operations.

[0153] For example, in the first beat, data a1, b1, ..., m1 are sent to the first column of multiplication and accumulation units, and weights A1, B1, ..., N1 are sent to the first row of multiplication and accumulation units to complete the first beat operation. In the second beat, data a1, b1, ..., m1 are sent to the second column of multiplication and accumulation units, and weights A1, B1, ..., N1 are sent to the second row of multiplication and accumulation units, and the data in the first column of multiplication and accumulation units are updated to a2, b2, ..., m2, and the weights in the first row of multiplication and accumulation units are updated to A2, B2, ..., N2; the second beat operation is completed, and so on, and finally all convolution operations are completed.

[0154] Since the sizes of different network models are different, such as the size of SSD300 is 300×300, and the size of YOLO V2 or YOLO V3 is 416×416; in addition, the filter sizes of different networks may also be inconsistent, such as 1×1, 3×3, 5×5, 7×7 and fully connected layers. Therefore, the computing unit array is designed as an elastic and scalable structure, that is, when corresponding to different network models, the computing unit array can be matched to the corresponding network structure to maximize the use of computing resources.

[0155] as follows Fig.11 For example, if the convolution kernel is 1×1 or 3×3, or the feature image is 4x4 or less, it can become the following Fig.11 The shape on the left, if the convolution kernel is larger than 3×3 or the feature map is larger than 4×4, the structure of the computing unit array can be changed to Fig.11 Right shape.

[0156] The process of completing neural network processing can be seen in the following example:

[0157] The first step is to compress the trained weight data in advance according to the sparsity situation, and store the compressed weights and weight indexes in the global cache.

[0158] In particular, for data with a weight of 0, since its calculation result is also 0, it does not need to be stored, that is, it is compressed, and subsequent calculations are directly skipped.

[0159] The original data that needs to be inferred is also stored in the global cache in sequence.

[0160] In the second step, the software in the CPU sends corresponding instructions to the neural network accelerator based on the currently processed network model, so that the computing control unit generates the corresponding computing unit array according to the network model configuration.

[0161] At the same time, the software will send control instructions to the hardware, allowing the input and output cache to read the corresponding compressed sub-convolution kernel and original data block from the corresponding global cache. The hardware analyzes the instructions sent by the software and sends them to the corresponding control and input and output cache units respectively.

[0162] The third step is data preparation before operation. According to the software configuration information, the hardware needs to store the weight data and feature data of the current processing progress in the cache unit in advance. The hardware first obtains the weight and weight index from the cache unit, and indexes the corresponding data from the input and output cache units according to the index information (weight position information) corresponding to the weight.

[0163] After obtaining the weights and corresponding data, the weights and data are sent to the corresponding computing unit array according to the corresponding configuration information of the software to complete the corresponding convolution operation.

[0164] In particular, since the result of the convolution operation is still 0, the data with weight 0 is discarded and does not participate in the calculation. That is, it does not need to be stored or read out in the input and output cache, thereby saving storage resources and data read and write bandwidth, and improving resource processing efficiency, thereby achieving the effect of supporting sparse operations.

[0165] The fourth step is Winograd conversion, which converts the data and weights into Winograd, performs convolution multiplication in the Winograd domain, and then accumulates and sums them. T The and A matrices are converted back to the digital domain, and finally the convolution result is obtained.

[0166] The fifth step is the neural network operator calculation. After the data and weights are accumulated, the ReLU activation function and pooling operations are performed, and finally the final operation results of the neural network processing are output.

[0167] Embodiment 4:

[0168] This embodiment provides a storage medium, which can store one or more computer programs that can be read, compiled and executed by one or more processors. In this embodiment, the storage medium can store a neural network convolution operation implementation program, which can be executed by one or more processors to implement any one of the neural network convolution operation implementation methods introduced in the aforementioned embodiments.

[0169] This embodiment also provides a neural network accelerator, such as Fig.12 As shown: the neural network accelerator 120 includes a processor 121, a memory 122, a computing unit array 123, and a communication bus 124 for respectively communicating the memory 122, the computing unit array 123 and the processor 121, wherein the memory 122 may be the storage medium storing the neural network convolution operation implementation program. The processor 121 may read the neural network convolution operation implementation program, compile and execute the process of implementing the neural network convolution operation implementation method described in the above embodiment:

[0170] The processor 121 obtains the sub-convolution kernel, the compressed sub-convolution kernel and the weight index corresponding to the original convolution kernel. The sub-convolution kernel is obtained by segmenting the original convolution kernel. The compressed sub-convolution kernel and the weight index are obtained by removing the zero-value weights in the sub-convolution kernel and rearranging the remaining weight values. Subsequently, the processor 121 reads the original data block whose specifications match the sub-convolution kernel specifications from the original data to be processed, and controls the computing unit array 123 to execute the iterative process according to the original data block, the compressed sub-convolution kernel and the weight index. The computing unit array 123 includes multiple multiplication and accumulation calculators. After the iterative process ends, the processor 121 obtains the output result of the computing unit array 123.

[0171] In some examples of this embodiment, the iterative process includes: the processor 121 gradually inputs the weight values ​​in the compressed sub-convolution kernel from the first side of the computing unit array 123 according to the computing beat of the computing unit array 123, and at the same time reads data from the original data block and inputs it from the second side of the computing unit array 123 according to the indication of the weight index as the computing beat is calculated. The weight values ​​in a compressed sub-convolution kernel will gradually flow to the third side opposite to the first side according to the computing beat, and the data will gradually flow to the fourth side opposite to the second side according to the computing beat; the output result of the computing unit array 123 is used as the original data block for the next iteration.

[0172] In some examples of this embodiment, before the processor 121 controls the computing unit array 123 to perform an iterative process according to the original data block, the compressed sub-convolution kernel and the weight index, it will first determine whether the compressed sub-convolution kernel is a convolution kernel that supports the Winograde algorithm. If so, the compressed sub-convolution kernel and the original data block are converted to the Winograde domain, and then the computing unit array 123 is controlled to perform a convolution operation between the compressed sub-convolution and the original data block according to the weight index under the Winograde domain. After obtaining the output result of the computing unit array 123, the processor 121 will also convert the obtained output result into the digital domain to obtain the final result of this neural network convolution operation.

[0173] In some examples of this embodiment, the processor 121 can gradually input the weight values ​​in the compressed sub-convolution kernel into the computing unit array 123 from the upper edge of the computing unit array 123 according to the computing beat, and read data from the original data block according to the indication of the weight index and input it into the computing unit array 123 from the left edge of the computing unit array 123 along the computing beat.

[0174] Optionally, for any compressed sub-convolution kernel, the processor 121 determines the data corresponding to each weight in the compressed sub-convolution kernel in the original data block according to the weight index, and then inputs the determined data from the second side of the computing unit array 123.

[0175] The processor 121 determines the weight index in any of the following ways:

[0176] Determine the weight index corresponding to the compressed subconvolution kernel according to the absolute position of the zero-value weight in the subconvolution kernel;

[0177] Determine the weight index corresponding to the compressed subconvolution kernel according to the number of zero-valued weights separated by the non-zero weights in the subconvolution kernel relative to the previous non-zero weights;

[0178] Determine the weight index corresponding to the compressed subconvolution kernel according to the absolute position of the non-zero weight in the subconvolution kernel;

[0179] The weight index corresponding to the compressed sub-convolution kernel is determined according to the number of non-zero weights between the zero-valued weight in the sub-convolution kernel and the previous zero-valued weight.

[0180] In some examples of this embodiment, before the processor 121 reads the original data block whose specifications match the sub-convolution kernel specifications from the original data to be processed, it will first parse the control instructions sent by the terminal CPU; then, according to the parsing results of the control instructions, the computing unit array 123 used in this neural network convolution operation is configured.

[0181] For specific details of the implementation method of the neural network accelerator 120 performing neural network convolution operations, please refer to the introduction of the aforementioned embodiment, which will not be repeated here.

[0182] This embodiment provides a neural network convolution operation implementation device that can implement the aforementioned neural network convolution operation implementation method, see Fig.13 :

[0183] The neural network convolution operation implementation device 13 includes: a data acquisition module 130, a data reading module 132, an iterative calculation module 134 and a result acquisition module 136, wherein the data acquisition module 130 is used to obtain the sub-convolution kernel, the compressed sub-convolution kernel and the weight index corresponding to the original convolution kernel, wherein the sub-convolution kernel is obtained by segmenting the original convolution kernel, and the compressed sub-convolution kernel and the weight index are obtained by removing the zero-value weights in the sub-convolution kernel and rearranging the remaining weight values; the data reading module 132 is used to read the original data block whose specifications match the sub-convolution kernel specifications from the original data to be processed; the iterative calculation module 134 is used to control the computing unit array to execute the iterative process according to the original data block, the compressed sub-convolution kernel and the weight index, and the computing unit array includes multiple multiplication and accumulation calculators; the result acquisition module 136 is used to obtain the output result of the computing unit array after the iterative process ends.

[0184] The iterative calculation module 134 can gradually input the weight values ​​in the compressed sub-convolution kernel from the first side of the computing unit array according to the calculation rhythm of the computing unit array, and at the same time read data from the original data block according to the indication of the weight index and input it from the second side of the computing unit array as the calculation rhythm. The weight values ​​in a compressed sub-convolution kernel will gradually flow to the third side opposite to the first side according to the calculation rhythm, and the data will gradually flow to the fourth side opposite to the second side according to the calculation rhythm; the output result of the computing unit array is used as the original data block for the next iteration.

[0185] In some other examples of this embodiment, the neural network convolution operation implementation device 13 also includes a Winograde conversion module, see Fig.14 The Winograde conversion module 138 is used to convert the compressed sub-convolution kernel and the original data block to the Winograde domain after determining that the compressed sub-convolution kernel is a convolution kernel that supports the Winograde algorithm, and then input the compressed sub-convolution kernel transferred to the Winograde domain and the data in the original data block to the computing unit array, so that the computing unit array is placed in the Winograde domain to perform the convolution operation between the compressed sub-convolution and the original data block; the Winograde conversion module 138 is also used to convert the final output result of the computing unit array into the digital domain to obtain the final result of this neural network convolution operation.

[0186] In this embodiment, the neural network convolution operation implementation device 13 can be deployed on a neural network accelerator, wherein the functions of the data acquisition module 130, the data reading module 132, the result acquisition module 136 and the Winograde conversion module 138 can all be implemented by the processor 121 of the neural network accelerator, and the function of the iterative calculation module 134 can be implemented by the computing unit array 123 in the neural network accelerator.

[0187] The neural network convolution operation implementation device provided in this embodiment, the neural network accelerator reduces the amount of convolution operation by compressing the weights in the convolution kernel and removing the zero-value weights in the original weight convolution kernel. At the same time, because an array-type computing unit array is used to perform convolution operations on data and weights, the data reuse of the convolution operation can be improved, and the bandwidth occupancy and power consumption in the neural network processing process can be reduced.

[0188] Obviously, those skilled in the art should understand that all or some steps, systems, and functional modules / units in the above disclosed methods can be implemented as software (which can be implemented with program code executable by a computing device), firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component can have multiple functions, or a function or step can be performed by several physical components in cooperation. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, executed by a computing device, and in some cases, the steps shown or described can be performed in a different order than herein, and the computer-readable medium can include a computer storage medium (or a non-transitory medium) and a communication medium (or a transient medium). As is well known to those of ordinary skill in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information (such as computer readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium. Therefore, the present invention is not limited to any particular combination of hardware and software.

[0189] The above contents are further detailed descriptions of the embodiments of the present invention in combination with specific implementation methods, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A method for implementing a convolution operation of a neural network, comprising: Obtain a subconvolution kernel, a compressed subconvolution kernel, and a weight index corresponding to the original convolution kernel, wherein the subconvolution kernel is obtained by segmenting the original convolution kernel, and the compressed subconvolution kernel and the weight index are obtained by removing zero-value weights in the subconvolution kernel and rearranging the remaining weight values; Reading an original data block whose specification matches the subconvolution kernel specification from the original data to be processed; Controlling a computing unit array to execute an iterative process according to the original data block, the compressed sub-convolution kernel and the weight index, wherein the computing unit array includes a plurality of multiplication and accumulation calculators; After the iteration process is finished, the output result of the computing unit array is obtained; The iterative process includes the computing unit array performing a convolution operation according to the original data block, the compressed sub-convolution kernel and the weight index, and the output result of the computing unit array is used as the original data block for the next iteration.

2. The method for implementing a neural network convolution operation as claimed in claim 1, characterized in that: The iterative process includes: gradually inputting the weight values ​​in the compressed sub-convolution kernel from the first side of the computing unit array according to the computing rhythm of the computing unit array, and at the same time, reading data from the original data block according to the indication of the weight index and inputting it from the second side of the computing unit array as the computing rhythm, each weight value in the compressed sub-convolution kernel will gradually flow to the third side opposite to the first side according to the computing rhythm, and the data will gradually flow to the fourth side opposite to the second side according to the computing rhythm.

3. The method for implementing a neural network convolution operation as claimed in claim 2, characterized in that: Before the control computing unit array performs the iterative process according to the original data block, the compressed sub-convolution kernel and the weight index, the control computing unit array further includes: Determining that the compressed subconvolution kernel is a convolution kernel supporting the Winograde algorithm; Converting the compressed subconvolution kernel and the original data block to Winograde domain; The controlling the computing unit array to perform an iterative process according to the original data block, the compressed subconvolution kernel and the weight index includes: controlling the computing unit array to perform a convolution operation between the compressed subconvolution and the original data block according to the weight index under the Winograde domain; After obtaining the output result of the computing unit array, the method further includes: The obtained output result is converted into the digital domain to obtain the final result of this neural network convolution operation.

4. The method for implementing a neural network convolution operation as claimed in claim 2, wherein: The step of gradually inputting the weight values ​​in the compressed sub-convolution kernel from the first side of the computing unit array according to the computing beat of the computing unit array, and reading data from the original data block according to the indication of the weight index and inputting data from the second side of the computing unit array as the computing beat progresses comprises: The weight values ​​in the compressed sub-convolution kernel are gradually input into the computing unit array from the upper edge of the computing unit array according to the computing beat, and data is read from the original data block according to the indication of the weight index and input into the computing unit array from the left edge of the computing unit array along the computing beat.

5. The method for implementing a neural network convolution operation as claimed in claim 2, characterized in that: The step of reading data from the original data block and inputting data from the second side of the computing unit array according to the indication of the weight index along with the computing beat comprises: For any compressed sub-convolution kernel, determining data in the original data block corresponding to each weight in the compressed sub-convolution kernel according to the weight index; The determined data is input from the second side of the computing unit array.

6. The method for implementing a convolution operation of a neural network according to any one of claims 1 to 5, characterized in that: The weight index may be determined in any of the following ways: Determine the weight index corresponding to the compressed subconvolution kernel according to the absolute position of the zero-value weight in the subconvolution kernel; Determine the weight index corresponding to the compressed subconvolution kernel according to the number of zero-valued weights separated by the non-zero weights in the subconvolution kernel relative to the previous non-zero weights; Determine the weight index corresponding to the compressed subconvolution kernel according to the absolute position of the non-zero weight in the subconvolution kernel; The weight index corresponding to the compressed sub-convolution kernel is determined according to the number of non-zero weights separated by a zero-valued weight in the sub-convolution kernel relative to a previous zero-valued weight.

7. The method for implementing a convolution operation of a neural network according to any one of claims 1 to 5, characterized in that: Before reading the original data block whose specification matches the sub-convolution kernel specification from the original data to be processed, the method further includes: Parse the control instructions sent by the terminal CPU; The computing unit array used in this neural network convolution operation is configured according to the analysis result of the control instruction.

8. A device for implementing a neural network convolution operation, comprising: A data acquisition module, used to acquire a sub-convolution kernel, a compressed sub-convolution kernel, and a weight index corresponding to the original convolution kernel, wherein the sub-convolution kernel is obtained by segmenting the original convolution kernel, and the compressed sub-convolution kernel and the weight index are obtained by removing zero-value weights in the sub-convolution kernel and rearranging the remaining weight values; A data reading module, used for reading an original data block whose specifications match the specifications of the sub-convolution kernel from the original data to be processed; An iterative calculation module, used for controlling a calculation unit array to perform an iterative process according to the original data block, the compressed sub-convolution kernel and the weight index, wherein the calculation unit array includes a plurality of multiplication and accumulation calculators; A result acquisition module, used for acquiring the output result of the computing unit array after the iteration process ends; The iterative process includes the computing unit array performing a convolution operation according to the original data block, the compressed sub-convolution kernel and the weight index, and the output result of the computing unit array is used as the original data block for the next iteration.

9. A neural network accelerator comprising a processor, a memory, a computing unit array, and a communication bus; The communication bus is used to realize the connection and communication between the processor and the memory; The processor is used to execute one or more programs stored in the memory to implement the steps of the neural network convolution operation implementation method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the neural network convolution operation implementation method as described in any one of claims 1 to 7.

Images