A neural network acceleration processing method and related device

Abstract

The present application discloses a neural network acceleration processing method, including: the acceleration device obtains feature data from the on-chip feature cache module; obtains weight data from the on-chip weight cache module; uses multiple processing combinations in the point-by-point processing array to perform parallel convolution calculations on the feature data and weight data in a pipeline manner to obtain point-by-point convolution results; uses the accumulation module to perform accumulation calculation processing on the point-by-point convolution results to obtain point-by-point accumulation processing results; uses the channel-by-channel processing array to perform parallel convolution calculations on the point-by-point accumulation processing results and the corresponding weight data in the on-chip weight cache module in a pipeline manner to obtain channel-by-channel convolution results, thereby improving the effect of accelerating the convolution calculation, and being compatible with processing depth-separable convolution and ordinary convolution. The present application also discloses a neural network acceleration processing device, an acceleration system, and a computer-readable storage medium, which have the above beneficial effects.

Description

Technical Field

[0001] The present application relates to the field of neural network computing technology, and in particular to a neural network acceleration processing method, a neural network acceleration processing device, an acceleration system, and a computer-readable storage medium. Background Art

[0002] With the continuous development of information technology, CNN (Convolutional Neural Networks) has been widely used, and the requirements for accuracy have become higher and higher, which has made the CNN algorithm model larger and more complex, so the amount of CNN network calculations has inevitably increased significantly. On the other hand, due to the increasing real-time requirements in fields such as autonomous driving and industrial Internet, more and more CNN reasoning needs to be completed at the edge, so reasoning needs to be accelerated.

[0003] In related technologies, most neural network accelerators focus on convolution acceleration, but due to the limitation of memory bandwidth, the parallelism of convolution calculation cannot be too high, which limits the acceleration effect. In addition, some neural network accelerators use depthwise separable convolution to replace some standard convolution layers to accelerate the network. Their architecture is specially designed for depthwise separable convolution, and is not compatible with the standard convolution calculation process, which affects the acceleration performance.

[0004] Therefore, how to improve the effect of accelerating convolution calculations is a key issue that technical personnel in this field are concerned about. Summary of the invention

[0005] The purpose of this application is to provide a neural network acceleration processing method, a neural network acceleration processing device, an acceleration system and a computer-readable storage medium to improve the effect of accelerating convolution calculations.

[0006] In order to solve the above technical problems, the present application provides a neural network acceleration processing method, comprising:

[0007] The acceleration device obtains feature data from the on-chip feature cache module;

[0008] Get weight data from the on-chip weight cache module;

[0009] Using multiple processing combinations in a point-by-point processing array to perform parallel convolution calculations on the feature data and the weight data in a pipeline manner to obtain point-by-point convolution results; wherein each of the processing combinations includes multiple processing cores; wherein each of the processing cores includes multiple multiplication units and a group of addition trees;

[0010] Using an accumulation module to perform accumulation calculation processing on the point-by-point convolution results to obtain a point-by-point accumulation processing result;

[0011] A channel-by-channel processing array is used to perform parallel convolution calculation on the point-by-point accumulation processing result and the corresponding weight data in the on-chip weight cache module in a pipeline manner to obtain a channel-by-channel convolution result.

[0012] Optionally, multiple processing combinations in the point-by-point processing array are used to perform parallel convolution calculations on the feature data and the weight data in a pipeline manner to obtain point-by-point convolution results, including:

[0013] The feature data and weight data corresponding to each channel are received in parallel from the N input channels of the point-by-point processing array;

[0014] Each processing core in each of the processing combinations performs parallel convolution calculations on the feature data and weight data of the corresponding channel to obtain a point-by-point convolution result corresponding to each output channel.

[0015] Optionally, an accumulation module is used to perform accumulation calculation processing on the point-by-point convolution result to obtain a point-by-point accumulation processing result, including:

[0016] The accumulation module is used to accumulate the point-by-point convolution results of each output channel to obtain accumulated data;

[0017] The accumulated data is processed to obtain the point-by-point accumulation processing result; wherein the data processing includes at least one or more of quantization processing, activation integration processing, and residual processing.

[0018] Optionally, a channel-by-channel processing array is used to perform parallel convolution calculation on the point-by-point accumulation processing result and the corresponding weight data in the on-chip weight cache module in a pipeline manner to obtain a channel-by-channel convolution result, including:

[0019] When performing depth-wise separable convolution calculation, writing the point-by-point accumulation processing result into a point-by-point feature buffer;

[0020] Get channel-by-channel weight data from the on-chip weight cache module;

[0021] A plurality of channel-by-channel processing cores of a channel-by-channel processing array are used to perform parallel convolution calculation on the point-by-point accumulation processing result and the channel-by-channel weight data in a pipeline manner to obtain the channel-by-channel convolution result; wherein each of the channel-by-channel processing cores includes a plurality of multiplication units and a group of addition trees.

[0022] Optionally, also include:

[0023] When performing depth-wise separable convolution calculation, the channel-by-channel convolution result is integrated into the activation process, and the processing result is quantized to obtain the channel-by-channel processing result;

[0024] When performing a 3x3 convolution calculation, a 3x3 accumulation module is used to accumulate the channel-by-channel convolution results, and data processing is performed on the accumulated calculation results to obtain a 3x3 convolution calculation result; wherein the data processing includes at least one or more of quantization processing, activation integration processing, and residual processing.

[0025] Optionally, also include:

[0026] The on-chip feature cache module is used to store feature data based on a preset format; wherein the on-chip feature cache module is constructed by three groups of cache units.

[0027] Optionally, also include:

[0028] Performing pooling calculation on the feature data in the on-chip feature cache module through a pooling module to obtain a pooling result;

[0029] The pooling result is written into the on-chip feature cache module.

[0030] The present application also provides a neural network acceleration processing device, comprising:

[0031] A feature data acquisition module, used to acquire feature data from an on-chip feature cache module;

[0032] A weight data acquisition module, used to obtain weight data from an on-chip weight cache module;

[0033] A point-by-point calculation module, used to perform parallel convolution calculation on the feature data and the weight data in a pipeline manner using multiple processing combinations in a point-by-point processing array to obtain a point-by-point convolution result; wherein each of the processing combinations includes multiple processing cores; wherein each of the processing cores includes multiple multiplication units and a group of addition trees;

[0034] A 1x1 accumulation calculation module, used for performing accumulation calculation processing on the point-by-point convolution result to obtain a point-by-point accumulation processing result;

[0035] The channel-by-channel calculation module is used to perform parallel convolution calculation on the point-by-point accumulation processing result and the corresponding weight data in the on-chip weight cache module in a pipeline manner to obtain a channel-by-channel convolution result.

[0036] The present application also provides an acceleration system, comprising:

[0037] Memory for storing computer programs;

[0038] A processor is used to implement the steps of the neural network acceleration processing method as described above when executing the computer program.

[0039] The present application also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the neural network acceleration processing method as described above are implemented.

[0040] The present application provides a neural network acceleration processing method, comprising: an acceleration device obtains feature data from an on-chip feature cache module; obtains weight data from an on-chip weight cache module; uses multiple processing combinations in a point-by-point processing array to perform parallel convolution calculations on the feature data and the weight data in a pipeline manner to obtain point-by-point convolution results; wherein each of the processing combinations includes multiple processing cores; wherein each of the processing cores includes multiple multiplication units and a group of addition trees; uses an accumulation module to perform accumulation calculation processing on the point-by-point convolution results to obtain point-by-point accumulation processing results; uses a channel-by-channel processing array to perform parallel convolution calculations on the point-by-point accumulation processing results and the corresponding weight data in the on-chip weight cache module in a pipeline manner to obtain channel-by-channel convolution results.

[0041] The feature data and weight data in the cache are calculated through the point-by-point processing array, and the multiple processing combinations and processing cores in the point-by-point processing array are used for parallel calculation. Then, the channel-by-channel processing array is used to process the data in a pipeline manner, realizing the pipeline parallel convolution calculation process, improving the parallelism of the calculation process, and improving the calculation efficiency of the neural network. It is also compatible with processing deep separable convolution and ordinary convolution at the same time.

[0042] The present application also provides a neural network acceleration processing device, an acceleration system and a computer-readable storage medium, which have the above beneficial effects and are not elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0044] Figure 1 A flowchart of a neural network acceleration processing method provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of the structure of a neural network accelerator provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of the PE core structure of a neural network acceleration processing method provided in an embodiment of the present application;

[0047] Figure 4 A schematic diagram of a point-by-point PE combination structure of a neural network acceleration processing method provided in an embodiment of the present application;

[0048] Figure 5 A schematic diagram of a point-by-point PE array structure of a neural network acceleration processing method provided in an embodiment of the present application;

[0049] Figure 6 A schematic diagram of a 1x1 accumulation module structure of a neural network acceleration processing method provided in an embodiment of the present application;

[0050] Figure 7 A schematic diagram of a channel-by-channel PE core structure of a neural network acceleration processing method provided in an embodiment of the present application;

[0051] Figure 8 A schematic diagram of a channel-by-channel PE array structure of a neural network acceleration processing method provided in an embodiment of the present application;

[0052] Fig. 9 A schematic diagram of the 3x3 accumulation module structure of a neural network acceleration processing method provided in an embodiment of the present application;

[0053] Fig.10 A schematic diagram of the structure of an on-chip feature cache module of a neural network acceleration processing method provided in an embodiment of the present application;

[0054] Fig.11 A schematic diagram of the structure of a neural network acceleration processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] The core of this application is to provide a neural network acceleration processing method, a neural network acceleration processing device, an acceleration system and a computer-readable storage medium to improve the effect of accelerating convolution calculations.

[0056] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0057] In related technologies, most neural network accelerators focus on convolution acceleration, but due to the limitation of memory bandwidth, the parallelism of convolution calculation cannot be too high, which limits the acceleration effect. In addition, some neural network accelerators use depthwise separable convolution to replace some standard convolution layers to accelerate the network. Their architecture is specially designed for depthwise separable convolution, and is not compatible with the standard convolution calculation process, which affects the acceleration performance.

[0058] Therefore, the present application provides a neural network acceleration processing method, which calculates the feature data and weight data in the cache through a point-by-point processing array, and performs parallel calculations through multiple processing combinations and processing cores in the point-by-point processing array, and then uses a channel-by-channel processing array to process data in a pipeline manner, thereby realizing a pipeline parallel convolution calculation process, improving the parallelism of the calculation process, and improving the calculation efficiency of the neural network. And it is compatible with processing depth separable convolution and ordinary convolution at the same time.

[0059] The following is an example of an embodiment to illustrate a neural network acceleration processing method provided by the present application.

[0060] Please refer to Figure 1 , Figure 1 A flowchart of a neural network acceleration processing method provided in an embodiment of the present application.

[0061] In this embodiment, the method may include:

[0062] S101, the acceleration device obtains feature data from an on-chip feature cache module;

[0063] It can be seen that this optional solution is intended to accelerate the device to obtain feature data from the on-chip feature cache module.

[0064] Among them, the on-chip feature cache module temporarily stores the feature data in the CNN network calculation process, and can also be used for data buffering in the input and output process. The temporarily stored feature data includes the initial first-layer input feature data, convolution layer calculation results, pooling layer calculation results, etc.

[0065] S102, obtaining weight data from an on-chip weight cache module;

[0066] It can be seen that this optional solution is intended to accelerate the device to obtain weight data from the on-chip weight cache module.

[0067] Among them, the on-chip weight cache is used to store weight data and provide weight data to the point-by-point processing array and the channel-by-channel processing array in the required format and order. When the neural network weight parameters are not very large and the FPGA (Field-Programmable Gate Array) / ASIC (Application Specific Integrated Circuit) on-chip storage resources are sufficient, all weight data can be stored on-chip; if the neural network weight parameters are relatively large and the on-chip storage resources are relatively tight, the weight data can be stored in an off-chip memory, and the required data can be read from the off-chip during calculations, and then output to the point-by-point processing array or the channel-by-channel processing array.

[0068] S103, using multiple processing combinations in the point-by-point processing array to perform parallel convolution calculations on the feature data and the weight data in a pipeline manner to obtain point-by-point convolution results; wherein each processing combination includes multiple processing cores; wherein each processing core includes multiple multiplication units and a group of addition trees;

[0069] Based on S101 and S102, this step aims to use multiple processing combinations in the point-by-point processing array to perform parallel convolution calculations on the feature data and the weight data to obtain a point-by-point convolution result.

[0070] Furthermore, this step may include:

[0071] Step 1, receiving feature data and weight data corresponding to each channel in parallel from N input channels of the point-by-point processing array;

[0072] Step 2: Perform parallel convolution calculations on the feature data and weight data of the corresponding channel through each processing core in each processing combination to obtain a point-by-point convolution result corresponding to each output channel.

[0073] It can be seen that this optional solution mainly explains how to perform parallel convolution calculations. In this optional solution, feature data and weight data corresponding to each channel are received in parallel from the N input channels of the point-by-point processing array, and each processing core in each processing combination performs parallel convolution calculations on the feature data and weight data of the corresponding channel to obtain the point-by-point convolution result corresponding to each output channel. Among them, N can be 8.

[0074] S104, using an accumulation module to perform accumulation calculation processing on the point-by-point convolution results to obtain a point-by-point accumulation processing result;

[0075] On the basis of S103, this step aims to use the accumulation module to perform accumulation calculation processing on the point-by-point convolution results to obtain the point-by-point accumulation processing results.

[0076] Among them, the point-by-point processing array is used for 1x1 convolution calculation, and the accumulation module is also used as a 1x1 accumulation module, which is mainly used to complete the accumulation of the partial sum of each input channel and the quantization of the accumulation result. In addition, activation and residual functions are also integrated. A selector is set after the 1x1 accumulation module. When calculating the depth-separable convolution, the calculation result of the 1x1 accumulation module is output to the feature cache module for channel-by-channel convolution calculation; when only calculating 1x1 convolution, the calculation result of the accumulation module is output to the on-chip feature cache module.

[0077] Furthermore, this step may include:

[0078] Step 1, using the accumulation module to accumulate the point-by-point convolution results of each output channel to obtain accumulated data;

[0079] Step 2, performing data processing on the accumulated data to obtain a point-by-point accumulation processing result; wherein the data processing includes at least one or more of quantization processing, activation integration processing, and residual processing.

[0080] It can be seen that this optional solution mainly explains how to perform accumulation calculation. In this optional solution, an accumulation module is used to accumulate the point-by-point convolution results of each output channel to obtain accumulated data, and the accumulated data is processed to obtain a point-by-point accumulation processing result; wherein the data processing includes at least one or more of quantization processing, activation integration processing, and residual processing.

[0081] S105, using a channel-by-channel processing array to perform parallel convolution calculations on the point-by-point accumulation processing results and the corresponding weight data in the on-chip weight cache module in a pipeline manner to obtain a channel-by-channel convolution result.

[0082] Based on S104, this step aims to use a channel-by-channel processing array to perform parallel convolution calculations on the point-by-point accumulation processing results and the corresponding weight data in the on-chip weight cache module in a pipeline manner to obtain a channel-by-channel convolution result.

[0083] Among them, the channel-by-channel processing array can be used to calculate both channel-by-channel convolution and standard convolution. Similar to the 1x1 accumulation module, the 3x3 accumulation module can integrate quantization, activation and residual functions in addition to completing the input channel part and accumulation. When calculating channel-by-channel convolution, the channel-by-channel processing array receives data from the point-by-point feature cache, and the calculation result is the final convolution layer output, which is directly output to the on-chip feature cache module; when calculating standard convolution (3x3 convolution), the channel-by-channel processing array reads feature data from the on-chip feature cache module for calculation, and the calculation result also needs to pass through the 3x3 accumulation module to complete the sum of each input channel part, and then go through quantization, activation, residual calculation and other operations to obtain the final result and then output it to the on-chip feature cache module.

[0084] Furthermore, this step may include:

[0085] Step 1, when performing depth-wise separable convolution calculation, write the point-by-point accumulation processing results into the point-by-point feature buffer;

[0086] Step 2, obtain channel-by-channel weight data from the on-chip weight cache module;

[0087] Step 3, using multiple channel-by-channel processing cores of the channel-by-channel processing array to perform parallel convolution calculations on the point-by-point accumulation processing results and the channel-by-channel weight data in a pipeline manner to obtain channel-by-channel convolution results; wherein each channel-by-channel processing core includes multiple multiplication units and a group of addition trees.

[0088] It can be seen that this optional solution also states that after the point-by-point processing array is processed, the channel-by-channel array can also be used for processing and calculation to achieve accelerated calculation of the separable convolution network. In this optional solution, when performing a depth-separable convolution calculation, the processing result is written into the point-by-point feature cache, the channel-by-channel weight data is obtained from the on-chip weight cache module, and multiple channel-by-channel processing cores of the channel-by-channel processing array are used to perform parallel convolution calculations on the point-by-point cumulative processing results and the channel-by-channel weight data in a pipeline manner to obtain the channel-by-channel convolution result; wherein each channel-by-channel processing core includes multiple multiplication units and a set of addition trees.

[0089] In addition, this embodiment may also include:

[0090] Step 1: When performing depth-separable convolution calculation, the channel-by-channel convolution result is integrated into the activation process, and the processing result is quantized to obtain the channel-by-channel processing result;

[0091] Step 2: When performing 3x3 convolution calculation, a 3x3 accumulation module is used to accumulate the channel-by-channel convolution results, and the accumulated calculation results are processed to obtain the 3x3 convolution calculation results; wherein the data processing includes at least one or more of quantization processing, activation integration processing, and residual processing.

[0092] Based on the previous optional solution, the simultaneous compatibility of depth-wise separable convolution and 3x3 convolution is achieved, thereby improving the applicability of the acceleration device.

[0093] In addition, this embodiment may also include:

[0094] An on-chip feature cache module is used to store feature data based on a preset format; wherein the on-chip feature cache module is constructed by three groups of cache units.

[0095] In addition, this embodiment may also include:

[0096] Step 1, performing pooling calculation on the feature data in the on-chip feature cache module through the pooling module to obtain a pooling result;

[0097] Step 2: Write the pooling results into the on-chip feature cache module.

[0098] That is, the pooling module reads data from the on-chip feature cache module and writes it back after the calculation is completed; in addition, the data operation module is used to complete other operations, such as channel data splicing.

[0099] In summary, this embodiment calculates the feature data and weight data in the cache through a point-by-point processing array, and performs parallel calculations through multiple processing combinations and processing cores in the point-by-point processing array, and then uses a channel-by-channel processing array to process data in a pipeline manner, thereby realizing a pipeline parallel convolution calculation process, improving the parallelism of the calculation process, and improving the calculation efficiency of the neural network. It is also compatible with processing depth separable convolution and ordinary convolution at the same time.

[0100] The following is a specific example to further illustrate the neural network accelerator of the neural network acceleration processing method provided by the present application.

[0101] Please refer to Figure 2 , Figure 2 A schematic diagram of the structure of a neural network accelerator provided in an embodiment of the present application.

[0102] In this embodiment, the neural network accelerator is mainly used to accelerate the CNN network. The accelerator is mainly composed of an input (on-chip) filter (weight) cache module, an on-chip feature cache module, a point-by-point convolution processing unit array (hereinafter referred to as a point-by-point PE (Processing elements, processing unit) array), a 1x1 accumulation module, a channel-by-channel convolution processing unit array (hereinafter referred to as a channel-by-channel PE array), a 3x3 accumulation module, a pooling module, etc. Figure 2 This is the overall structural block diagram of the accelerator.

[0103] Among them, the neural network accelerator is generally deployed on FPGA or ASIC, and the CPU sends the pre-processed feature data directly to the on-chip feature cache; when the amount of weight data is not large, the CPU can choose to send the weight data to the input (on-chip) filter cache storage; when the CNN network parameters are large, the CPU can choose to send the weight data to the off-chip memory, and the input (on-chip) cache module is used as the first-level data cache. After reading the data from the off-chip memory, it is transmitted to the point-by-point PE array and the channel-by-channel PE array.

[0104] The point-by-point PE array is used for 1x1 convolution calculations. The 1x1 accumulation module is mainly used to complete the accumulation of the partial sums of each input channel and the quantization of the accumulation results. In addition, it also integrates activation and residual functions. A selector is set after the 1x1 accumulation module. When calculating the depth-separable convolution, the calculation result of the 1x1 accumulation module is output to the feature cache module for channel-by-channel convolution calculation; when only calculating the 1x1 convolution, the calculation result of the accumulation module is output to the on-chip feature cache module.

[0105] The channel-by-channel PE array can be used to calculate both channel-by-channel convolution and standard convolution. Similar to the 1x1 accumulation module, the 3x3 accumulation module not only completes the input channel part and accumulation, but also integrates quantization, activation and residual functions. When calculating channel-by-channel convolution, the channel-by-channel PE array receives data from the point-by-point feature cache, and the calculation result is the final convolution layer output, which is directly output to the on-chip feature cache module; when calculating standard convolution (3x3 convolution), the channel-by-channel PE array reads feature data from the on-chip feature cache module for calculation, and the calculation result also needs to pass through the 3x3 accumulation module to complete the sum of each input channel part, and then go through quantization and other operations (optionally perform activation and residual calculations) to get the final result and then output it to the on-chip feature cache module.

[0106] The pooling module reads data from the on-chip feature cache module and writes it back after calculation; the data operation module is used to complete other operations, such as channel data splicing.

[0107] The neural network accelerator of this embodiment is mainly used for CNN network acceleration, and the main modules thereof are introduced below.

[0108] Please refer to Figure 3 , Figure 3 A schematic diagram of the PE core structure of a neural network acceleration processing method provided in an embodiment of the present application.

[0109] The PE core structure in the point-by-point PE array is as follows Figure 3 As shown in the figure, each PE core contains 8 multiplication units and a set of addition trees. d0-d8 are the feature data of 8 input channels, w1-w8 are the weight data of the corresponding 8 channels, and the 8 multiplication units complete the 1x1 convolution multiplication operation of the 8 channels respectively. After the addition tree completes the addition of the partial sum of the 8 input channels, it outputs the result. It can be seen that a single PE core can complete the parallel calculation of the features of 8 input channels.

[0110] Please refer to Figure 4 , Figure 4 A schematic diagram of a point-by-point PE combination structure of a neural network acceleration processing method provided in an embodiment of the present application.

[0111] The point-by-point PE combination consists of 8 point-by-point PE cores, such as Figure 4 As shown, they are PE1-PE8, and each PE core completes the multiplication and accumulation calculation of 8 features and 8 filter data. Among them, D1 represents the feature data of the 8 input channels corresponding to position 1 on the feature map (8 in total), and D2 represents the feature data of the 8 input channels corresponding to position 2 on the feature map, and so on. The 8-input channel filter is the weight data corresponding to the 8 input channels mentioned above (8 in total). A point-by-point PE combination shares a set of weight data (8), which can complete the calculation of 8-way feature data under 8 input channels in parallel.

[0112] Please refer to Figure 5 , Figure 5 A schematic diagram of a point-by-point PE array structure of a neural network acceleration processing method provided in an embodiment of the present application.

[0113] like Figure 5 As shown, the point-by-point PE array consists of 32 point-by-point PE combinations, and each processing combination calculates an output channel. Feature input 1 represents 8 groups of 8 input channel features corresponding to output channel 1, feature input 2 represents 8 groups of 8 input channel features corresponding to output channel 2, feature input 3-feature input 32 and so on. Weight input F1 represents the 8 input channel weight data corresponding to output channel 1, weight input F1 represents the 8 input channel weight data corresponding to output channel 2, weight input F3-weight input F32 and so on.

[0114] So far, it can be seen that the point-by-point PE array can perform parallel calculations in three dimensions: feature map data, input channels, and output channels. This embodiment is described by taking 8-way feature data, 8 input channels, and 32 output channels as an example, where both feature data and filter data are quantized to INT8 type.

[0115] Please refer to Figure 6 , Figure 6 A schematic diagram of the 1x1 accumulation module structure of a neural network acceleration processing method provided in an embodiment of the present application.

[0116] The 1x1 accumulation module receives the partial sum results of the point-by-point PE array output and completes operations such as accumulation calculation, quantization, and activation. Its structure is as follows Figure 6As shown. The 1x1 accumulation module performs parallel calculations on 32 output channels, matching the point-by-point PE array. In the figure, partial sum 1-partial sum 32 are partial sums of output channels 1-32. The partial sums need to be quantized after accumulation. Quantization is to convert the accumulated result (INT32) back to INT8 format, so that the data bit width of 32 channels parallel calculation is 8*32*8bit=2048bit. This data bit width is too high, which puts a lot of pressure on subsequent data processing. Therefore, FIFO (First Input First Output) is used to convert it into 8 features * 8 output channels = 512bit.

[0117] According to the network structure, you can choose whether to perform activation operations and residual calculations in the 1x1 accumulation module. If the residual calculation is followed by the point-by-point convolution layer, the corresponding data can be read from the on-chip cache unit while the 1x1 accumulation calculation is performed, and then written back to the on-chip feature cache module after the residual calculation is completed. In this embodiment, the order of feature data written back to the on-chip feature cache module is the same each time, so data can be read from the on-chip cache module in sequence, and the accumulation calculation and residual operation can form a pipeline, which can not only reduce a large number of memory access actions, but also save processing time. The final calculation result of the 1x1 accumulation module passes through the selector and is output to the channel-by-channel convolution array or the on-chip feature cache module.

[0118] Furthermore, when the neural network accelerator is calculating the depth-separable convolution, the processing results of the point-by-point PE array first enter the point-by-point feature cache. The point-by-point feature cache mainly changes the order and port format of the cached data as required and transmits it to the channel-by-channel PE array.

[0119] Please refer to Figure 7 , Figure 7 A schematic diagram of a channel-by-channel PE core structure of a neural network acceleration processing method provided in an embodiment of the present application.

[0120] The channel-by-channel PE array consists of 32 channel-by-channel PE cores, which can complete the 3x3 convolution calculation of 32 output channels in parallel. The structure of the channel-by-channel PE core is as follows: Figure 7 As shown, the PE core contains 9 multiplication units and a set of addition trees. Each multiplication unit completes a feature data ( Figure 7 d0-d8) and a weight data ( Figure 7 In the example above, the addition tree completes the summation of the 9 multiplication results. The addition tree contains 4 levels, which is suitable for pipeline calculation. The addition tree is also connected to the quantization and activation operation components. If the separable convolution is calculated, the quantization and activation operations are required; if the standard convolution is calculated, the addition tree accumulation result is directly output.

[0121] Please refer to Figure 8 , Figure 8 A schematic diagram of a channel-by-channel PE array structure of a neural network acceleration processing method provided in an embodiment of the present application.

[0122] The structure of the channel-by-channel PE array is as follows Figure 8 As shown, the array contains a total of 32 channel-by-channel processing cores (PE1-PE32), and each PE core completes the 3x3 convolution calculation of one output channel. Feature channel 1-feature channel 32 are the feature data corresponding to the 1st to 32nd output channels, and each channel contains 9 feature values. Weight channel 1-weight channel 32 are the weight data corresponding to the 1st to 32nd output channels, and each channel contains 9 weight values. In order to ensure that the data format and storage order of the channel-by-channel convolution calculation results are consistent with the point-by-point convolution results, a FIFO is set after each PE core, and when the calculation results accumulate to 8, they are output together. In addition, the 8-feature * 32 channel data output by 32 PE cores in parallel also needs to pass through the FIFO, converted into 8-feature * 8 channels (=512bit) and then output. Finally, after passing through the selector, it is either output to the 3x3 accumulation module or output to the on-chip feature cache module.

[0123] Please refer to Fig. 9 , Fig. 9 A schematic diagram of the 3x3 accumulation module structure of a neural network acceleration processing method provided in an embodiment of the present application.

[0124] When calculating standard convolution, the results of channel-by-channel array calculations need to be accumulated to obtain the final result. The neural network accelerator of this embodiment includes a 3x3 accumulation module, whose structure is basically the same as the 1x1 accumulation module. Fig. 9 As shown in the figure, the 3x3 accumulation module receives the partial sum results output by the point-by-point PE array and completes the accumulation calculation, quantization, activation and other operations. The 3x3 accumulation module performs parallel calculations on 32 output channels to match the channel-by-channel PE array. In the figure, partial sum 1-partial sum 32 are the partial sums of output channels 1-32, and the partial sums need to be quantized after accumulation. Quantization is to convert the accumulated result (INT32) back to INT8 format, so that the data bit width completed by 32 channels in parallel is 8*32*8bit=2048bit. This data bit width is too high, which puts a lot of pressure on subsequent data processing. Therefore, FIFO is used to convert it into 8 features*8 output channels=512bit.

[0125] Similar to the 1x1 accumulation module, depending on the network structure, you can choose whether to perform activation operations and residual calculations in the 3x3 accumulation module. If the convolution layer is followed by residual calculations, the corresponding data can be read from the on-chip cache unit while the 3x3 accumulation calculation is being performed, and then written back to the on-chip feature cache module after the residual calculation is completed. Similarly, the order of feature data written back to the on-chip feature cache module is the same each time, so data can be read from the on-chip cache module in sequence. Unlike the 1x1 accumulation module, the results of the 3x3 accumulation module are directly written back to the on-chip feature cache module.

[0126] Among them, the on-chip filter cache module is used to store weight data and provide filter data for the point-by-point PE array and the channel-by-channel PE array in the required format and order. When the neural network weight parameters are not very large and the on-chip storage resources of the FPGA / ASIC used are sufficient, all weight data can be stored on the chip; if the neural network weight parameters are relatively large and the on-chip storage resources are relatively tight, the weight data can be stored in an off-chip memory, and the required data can be read from the off-chip during calculation, and then output to the point-by-point PE array or the channel-by-channel PE array.

[0127] Please refer to Fig.10 , Fig.10 A schematic diagram of the structure of an on-chip feature cache module of a neural network acceleration processing method provided in an embodiment of the present application.

[0128] The on-chip feature cache module temporarily stores feature data during the CNN network calculation process and can also be used for data buffering during the input and output process. The temporarily stored feature data includes the initial first-layer input feature data, convolution layer calculation results, pooling layer calculation results, etc. Fig.10 As shown in the figure, the on-chip feature cache module is mainly composed of three storage blocks, namely storage block A, storage block B and storage block C. The structures of the three storage blocks are exactly the same, and the port width is 512 bits. They are all composed of simple dual-port RAMs, which can be read or written. Choosing simple dual-port RAM can reduce the consumption of RAM resources. Each storage block has independent read and write control logic. The module as a whole contains three groups of interfaces, namely A, B and C. The module uses the storage block selection control logic to decide which (or which ones) need to be read or written. Three storage blocks are set, and any two storage blocks can form a ping-pong structure, which is suitable for pipeline processing requirements such as convolution calculation and residual calculation, and meets the calculation of the following scenarios:

[0129] When calculating convolution, assuming that the feature data to be calculated is temporarily stored in storage block A, the feature data can be read from storage block A, and the convolution array can be pipelined and the result can be written back to storage block B.

[0130] When calculating the convolution layer and the residual, assuming that the calculation result of the previous convolution layer m is temporarily stored in storage block A, and the feature data of the convolution layer that needs to be calculated is temporarily stored in storage block B, then when calculating, the feature data is read from storage block B for convolution calculation. When the accumulation module is calculating, the data to be residual is read from storage block A at the same time. After the residual data calculation is completed, the final result is written to storage block C;

[0131] When performing pooling calculations, assuming that the feature data currently to be calculated is temporarily stored in storage block A, the data is read from storage block A and written back to storage block B after the calculation is completed.

[0132] In the above scenarios, the intermediate results (calculation results of convolutional layers, residuals, etc.) in the CNN network are written to the on-chip feature cache module for storage, and the data is also read from the on-chip feature cache module when the next layer is calculated, avoiding access to the off-chip memory. This not only breaks the limitation of insufficient bandwidth for accessing off-chip memory, but also reduces the time for reading and writing data. In addition, by setting up three storage blocks, residual calculations can be performed while calculating convolutions, and the entire residual operation can be integrated into the pipeline, which can speed up the calculation of the entire network.

[0133] Pooling operation is one of the most common calculations in CNN network. The pooling module in this embodiment is used to complete the pooling calculation. The pooling module reads data from the on-chip feature cache module and writes it back to the on-chip feature cache module after the calculation is completed. Other data operation modules can complete operations such as data splicing and can be flexibly designed according to the network structure and the available resources of FPGA / ASIC.

[0134] It can be seen that this embodiment can support the point-by-point PE array design for parallel computing in three dimensions: the structural design of the point-by-point PE array enables it to perform parallel computing in three dimensions: feature map data, input channels, and output channels, so that the degree of computational parallelism can be greatly increased. All intermediate computation results of the CNN network are temporarily stored on-chip, avoiding frequent access to off-chip memory and eliminating the impact of insufficient memory bandwidth on acceleration performance; through the design of three groups of storage blocks, the residual calculation is embedded in the accumulation module to achieve pipeline processing, reducing the number of accesses and accelerating the network computing process; the design of three groups of independent read-write storage blocks, any two groups can form a ping-pong structure, making it suitable for pipeline processing; in addition, the three groups of storage blocks can also ensure the pipeline processing of residual calculations.

[0135] The following is an introduction to the neural network acceleration processing device provided in an embodiment of the present application. The neural network acceleration processing device described below and the neural network acceleration processing method described above can be referenced to each other.

[0136] Please refer to Fig.11 , Fig.11A schematic diagram of the structure of a neural network acceleration processing device provided in an embodiment of the present application.

[0137] In this embodiment, the device may include:

[0138] The feature data acquisition module 100 is used to acquire feature data from the on-chip feature cache module;

[0139] The weight data acquisition module 200 is used to acquire weight data from the on-chip weight cache module;

[0140] A point-by-point calculation module 300, configured to perform parallel convolution calculation on the feature data and the weight data in a pipeline manner using a plurality of processing combinations in a point-by-point processing array to obtain a point-by-point convolution result; wherein each of the processing combinations includes a plurality of processing cores; wherein each of the processing cores includes a plurality of multiplication units and a group of addition trees;

[0141] A 1x1 accumulation calculation module 400 is used to perform accumulation calculation processing on the point-by-point convolution result to obtain a point-by-point accumulation processing result;

[0142] The channel-by-channel calculation module 500 is used to perform parallel convolution calculation on the point-by-point accumulation processing result and the corresponding weight data in the on-chip weight cache module in a pipeline manner to obtain a channel-by-channel convolution result.

[0143] Optionally, the point-by-point calculation module 300 is specifically used to receive the feature data and weight data corresponding to each channel in parallel from the N input channels of the point-by-point processing array; perform parallel convolution calculations on the feature data and weight data of the corresponding channel through each processing core in each of the processing combinations, to obtain the point-by-point convolution result corresponding to each output channel.

[0144] Optionally, the 1x1 accumulation calculation module 400 is specifically used to use the accumulation module to perform accumulation processing on the point-by-point convolution results of each output channel to obtain accumulated data; perform data processing on the accumulated data to obtain the point-by-point accumulation processing result; wherein the data processing includes at least one or more of quantization processing, activation integration processing, and residual processing.

[0145] Optionally, the channel-by-channel calculation module 500 is specifically used to write the point-by-point accumulation processing result into the point-by-point feature cache when performing depth-separable convolution calculation; obtain the channel-by-channel weight data from the on-chip weight cache module; use multiple channel-by-channel processing cores of the channel-by-channel processing array to perform parallel convolution calculation on the point-by-point accumulation processing result and the channel-by-channel weight data in a pipeline manner to obtain the channel-by-channel convolution result; wherein each of the channel-by-channel processing cores includes multiple multiplication units and a group of addition trees.

[0146] Optionally, the device may further include:

[0147] A 3x3 accumulation calculation module is used to integrate activation processing into the channel-by-channel convolution result and quantize the processing result to obtain the channel-by-channel processing result when performing depth-separable convolution calculation; when performing 3x3 convolution calculation, a 3x3 accumulation module is used to accumulate the channel-by-channel convolution result, and data processing is performed on the accumulated calculation result to obtain the 3x3 convolution calculation result; wherein the data processing includes at least one or more of quantization processing, integration with activation processing, and residual processing.

[0148] Optionally, the device may further include:

[0149] The feature data storage module is used to use the on-chip feature cache module to store feature data based on a preset format; wherein the on-chip feature cache module is constructed by three groups of cache units.

[0150] Optionally, the device may further include:

[0151] The pooling module is used to perform pooling calculation on the feature data in the on-chip feature cache module to obtain a pooling result; and write the pooling result into the on-chip feature cache module.

[0152] The embodiment of the present application further provides an acceleration system, characterized in that it includes:

[0153] Memory for storing computer programs;

[0154] A processor, configured to implement the steps of the neural network acceleration processing method as described in any one of claims 1 to 7 when executing the computer program.

[0155] An embodiment of the present application also provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the neural network acceleration processing method as described in any one of claims 1 to 7 are implemented.

[0156] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0157] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0158] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0159] The above is a detailed introduction to a neural network acceleration processing method, a neural network acceleration processing device, an acceleration system and a computer-readable storage medium provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A neural network acceleration processing method, characterized in that: include: The acceleration device obtains feature data from the on-chip feature cache module; Get weight data from the on-chip weight cache module; Using multiple processing combinations in a point-by-point processing array to perform parallel convolution calculations on the feature data and the weight data in a pipeline manner to obtain point-by-point convolution results; wherein each of the processing combinations includes multiple processing cores; wherein each of the processing cores includes multiple multiplication units and a group of addition trees; Using an accumulation module to perform accumulation calculation processing on the point-by-point convolution results to obtain a point-by-point accumulation processing result; Using a channel-by-channel processing array to perform parallel convolution calculation on the point-by-point accumulation processing result and the corresponding weight data in the on-chip weight cache module in a pipeline manner to obtain a channel-by-channel convolution result; The feature data and the weight data are parallelly convolved in a pipeline manner using a plurality of processing combinations in a point-by-point processing array to obtain a point-by-point convolution result, including: The feature data and weight data corresponding to each channel are received in parallel from the N input channels of the point-by-point processing array; the feature data and weight data corresponding to the corresponding channel are subjected to parallel convolution calculation by each processing core in each processing combination to obtain a point-by-point convolution result corresponding to each output channel; The accumulation module is used to perform accumulation calculation processing on the point-by-point convolution result to obtain a point-by-point accumulation processing result, including: The accumulation module is used to perform accumulation processing on the point-by-point convolution results of each output channel to obtain accumulated data; the accumulated data is processed to obtain the point-by-point accumulation processing result; wherein the data processing includes at least one or more of quantization processing, activation integration processing, and residual processing; A channel-by-channel processing array is used to perform parallel convolution calculation on the point-by-point accumulation processing result and the corresponding weight data in the on-chip weight cache module in a pipeline manner to obtain a channel-by-channel convolution result, including: When performing depth-separable convolution calculation, the point-by-point accumulation processing result is written into the point-by-point feature cache; the channel-by-channel weight data is obtained from the on-chip weight cache module; multiple channel-by-channel processing cores of the channel-by-channel processing array are used to perform parallel convolution calculation on the point-by-point accumulation processing result and the channel-by-channel weight data in a pipeline manner to obtain the channel-by-channel convolution result; wherein each of the channel-by-channel processing cores includes multiple multiplication units and a group of addition trees; The neural network acceleration processing method further includes: When performing depth-separable convolution calculation, the channel-by-channel convolution result is integrated with activation processing, and the processing result is quantized to obtain the channel-by-channel processing result; when performing 3x3 convolution calculation, a 3x3 accumulation module is used to accumulate the channel-by-channel convolution result, and the accumulated calculation result is subjected to data processing to obtain the 3x3 convolution calculation result; wherein the data processing includes at least one or more of quantization processing, integration with activation processing, and residual processing; Wherein, the neural network acceleration processing method further includes: The point-by-point feature cache changes the order and port format of the cached data according to preset requirements and transmits it to the channel-by-channel processing array.

2. The neural network acceleration processing method according to claim 1, characterized in that: Also includes: The on-chip feature cache module is used to store feature data based on a preset format; wherein the on-chip feature cache module is constructed by three groups of cache units.

3. The neural network acceleration processing method according to claim 2, characterized in that: Also includes: Performing pooling calculation on the feature data in the on-chip feature cache module through a pooling module to obtain a pooling result; The pooling result is written into the on-chip feature cache module.

4. A neural network acceleration processing device, characterized in that: include: A feature data acquisition module, used to acquire feature data from an on-chip feature cache module; A weight data acquisition module, used to obtain weight data from an on-chip weight cache module; A point-by-point calculation module, used to perform parallel convolution calculation on the feature data and the weight data in a pipeline manner using multiple processing combinations in a point-by-point processing array to obtain a point-by-point convolution result; wherein each of the processing combinations includes multiple processing cores; wherein each of the processing cores includes multiple multiplication units and a group of addition trees; A 1x1 accumulation calculation module, used for performing accumulation calculation processing on the point-by-point convolution result to obtain a point-by-point accumulation processing result; A channel-by-channel calculation module, used to perform parallel convolution calculation on the point-by-point accumulation processing result and the corresponding weight data in the on-chip weight cache module in a pipeline manner to obtain a channel-by-channel convolution result; Point-by-point calculation module, specifically used for: The feature data and weight data corresponding to each channel are received in parallel from the N input channels of the point-by-point processing array; the feature data and weight data corresponding to the corresponding channel are subjected to parallel convolution calculation by each processing core in each processing combination to obtain a point-by-point convolution result corresponding to each output channel; 1x1 accumulation calculation module, specifically used for: The accumulation module is used to perform accumulation processing on the point-by-point convolution results of each output channel to obtain accumulated data; the accumulated data is processed to obtain the point-by-point accumulation processing result; wherein the data processing includes at least one or more of quantization processing, activation integration processing, and residual processing; Channel-by-channel calculation module, specifically used for: When performing depth-separable convolution calculation, the point-by-point accumulation processing result is written into the point-by-point feature cache; the channel-by-channel weight data is obtained from the on-chip weight cache module; multiple channel-by-channel processing cores of the channel-by-channel processing array are used to perform parallel convolution calculation on the point-by-point accumulation processing result and the channel-by-channel weight data in a pipeline manner to obtain the channel-by-channel convolution result; wherein each of the channel-by-channel processing cores includes multiple multiplication units and a group of addition trees; The neural network acceleration processing device is specifically used for: When performing depth-separable convolution calculation, the channel-by-channel convolution result is integrated with activation processing, and the processing result is quantized to obtain the channel-by-channel processing result; when performing 3x3 convolution calculation, a 3x3 accumulation module is used to accumulate the channel-by-channel convolution result, and the accumulated calculation result is subjected to data processing to obtain the 3x3 convolution calculation result; wherein the data processing includes at least one or more of quantization processing, integration with activation processing, and residual processing; The neural network acceleration processing device is specifically used for: The point-by-point feature cache changes the order and port format of the cached data according to preset requirements and transmits it to the channel-by-channel processing array.

5. An acceleration system, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the neural network acceleration processing method as described in any one of claims 1 to 3 when executing the computer program.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the neural network acceleration processing method according to any one of claims 1 to 3 are implemented.

Images