Neural network accelerator for unstructured sparse dynamic matching non-zero values

Abstract

The application discloses a neural network accelerator for unstructured sparse dynamic matching non-zero value, and belongs to the technical field of calculation, reckoning or counting. The accelerator architecture comprises a processing module, a load balancing module, a post-processing module, a sparse coding module, a control unit, an input buffer module, a weight buffer module and an output buffer module. The input activation tensor is on-chip coded through the sparse coding module, so that storage and transportation of invalid data are avoided, and the overall energy efficiency of the accelerator is improved. The load balancing module dynamically allocates the sparse tensor after marking, and effectively alleviates the problem of uneven load of the calculation unit caused by irregular input activation tensor. The non-zero value matching unit of the PE efficiently matches non-zero pairs through bitmap index, simultaneously utilizes the sparsity of weights and activations, skips invalid calculation participated by zero values, and increases processing efficiency, so that efficient inference of the unstructured sparse neural network is realized.

Description

TECHNICAL FIELD

[0001] The application discloses a neural network accelerator for unstructured sparse dynamic matching non-zero value, and belongs to the technical field of calculation, estimation or counting. BACKGROUND

[0002] Today, deep learning (DL) models have been widely applied in multiple important fields such as computer vision-based target classification and detection, natural language processing, media generation, recommendation systems, medical diagnosis, large-scale scientific computing, embedded systems, mobile and edge processing, and even for designing or optimizing hardware and software systems. However, the computation and memory requirements of deep learning models applied in these fields increase dramatically. The size of deep learning models increases with the increase of the number of layers and the improvement of the number of parameters of deep neural networks, and the learning accuracy of the models is effectively improved, but there is significant redundancy in these deep learning models containing a large number of parameters. Therefore, the compression and acceleration technology of deep neural networks is proposed. By summarizing and utilizing the sparsity of deep learning models, size reduction operations and quantization operations, the tensors can be effectively compressed, the calculation of the model can be accelerated, and the energy consumption can be saved. With the continuous improvement of computing, storage and communication requirements, the model compression technology is of great significance.

[0003] One of the popular techniques for model compression is pruning weights in neural networks, also known as sparse neural networks. Existing research has shown that a large portion of weights can be pruned to zero without loss of accuracy. By using pruning techniques, more than 30% of neurons and 50% of synapses in VGG16 can be zero-valued, and since the activation data of the intermediate layers in convolutional networks is also sparse, more than 65% of the computations are unnecessary. However, the current mainstream neural network accelerator is designed to perform structured computation with regular memory access and data movement, and the regular dense CNN accelerator is almost unable to benefit from the reduction in computation and data, and zero values are still input into the accelerator to perform unnecessary "virtual" computation. CNN accelerators do not have special support for sparse tensors, they get all the data from memory and input the data to the PE, which wastes execution time, and the non-uniform distribution of non-zero elements or non-zero blocks in the input tensor data presents non-structured sparsity, which leads to irregular processing. Therefore, to take advantage of the compression efficiency of input data sparsity requires additional hardware and data flow to store, extract, communicate, compute and load balance the non-zero elements of the input data. The goal of utilizing input data sparsity is to utilize all forms of sparsity to greatly reduce computation, communication and storage while avoiding the increase in power consumption and area overhead. Effective use of sparsity depends on the encoding and extraction of sparse data and the design of data flow, memory structure, interconnection and write-back mechanism. The existing mainstream neural network accelerator lacks support for the sparsity of the input data of the deep neural network, and still involves zero values in the activation and weight data in computation, communication and storage.

[0004] In order to better utilize the sparsity of the input data of the neural network, the existing technology proposes a convolutional neural network accelerator that utilizes the sparsity of the weight data, but this accelerator only utilizes the sparsity of the weight side and does not specially process the zero values on the activation side, and does not consider the load imbalance problem caused by the irregularity of the sparse tensor, and still has the defects of large amount of computation and low efficiency. On the other hand, the unstructured sparsity of the deep neural network causes the number of zero values in the weight data received by each PE to be different, and if the zero value data of the weight data received by all PEs is not preprocessed before the convolution operation of the current channel, the utilization rate of the PE will be reduced, and the overall efficiency of the processing module will be reduced.

[0005] Therefore, it is of great significance to design a high-energy-efficiency neural network accelerator for unstructured sparse to effectively utilize the sparsity of the activation data and the weight data of the deep neural network. SUMMARY

[0006] The application aims at the deficiencies of the prior art, and provides a neural network accelerator for unstructured sparse dynamic matching of non-zero values.

[0007] The application achieves the above-mentioned application purposes by adopting the following technical scheme.

[0008] The neural network accelerator for unstructured sparse dynamic matching of non-zero values comprises a sparse coding module, an input buffer module, a weight buffer module, a load balancing module, a calculation processing module, an output buffer module, a post-processing module and a control unit.

[0009] The sparse coding module is used for reading an input activation tensor from an external storage or receiving a current channel convolution operation result output by the post-processing module, performing sparse coding in bitmap format on the received data point by point according to channels, and outputting a current channel input activation tensor containing valid non-zero data and bitmap indexes.

[0010] The input buffer module is used for buffering the current channel input activation tensor output by the sparse coding module, and transmitting the current channel input activation tensor to the load balancing module according to a convolution window order.

[0011] The weight buffer module is used for buffering a sparse coded weight tensor read from the external storage, and outputting a current channel weight tensor, i.e., convolution kernel data, to the calculation processing module in a broadcast mode.

[0012] The load balancing module is used for labeling and grouping the received current channel input activation tensor according to a convolution window label, marking the sparsity of each group of current channel input activation tensors, inputting the current channel input activation tensors with marked sparsity to the calculation processing module in groups, selecting a group of current channel input activation tensors with the lowest sparsity, i.e., the largest calculation amount, from the groups of current channel input activation tensors not input to the calculation processing module after receiving a calculation completion feedback signal transmitted by the processing unit, and transmitting the group of current channel input activation tensors with the lowest sparsity to the processing unit that first completes the convolution operation.

[0013] The computing processing module is configured to perform convolution operation on the current channel activation tensor and the current channel weight tensor, each processing unit performs convolution operation on the non-zero pair data according to the bitmap index of a group of current channel input activation tensors, and after the processing unit completes the operation corresponding to one convolution window, the processing unit outputs the convolution operation result to the output buffer module according to the convolution window mark corresponding to the processed current channel activation tensor, the processing unit that first completes the convolution operation transmits a calculation completion feedback signal to the load balancing module, and after the processing unit that first completes the convolution operation receives the group of current channel input activation tensors with the lowest sparsity, the processing unit performs convolution operation on the next group of current channel input activation tensors and the current channel weight tensor, until the processing unit completes the convolution operation on each group of current channel input activation tensors and the current channel weight tensor, and then reads the next channel activation tensor and the next channel weight tensor to start the convolution operation of the next channel, and the convolution operation result of the current channel is output.

[0014] The output buffer module is configured to cache the convolution operation result of the current channel according to the convolution window mark.

[0015] The post-processing module is configured to output the convolution operation result of the current channel after post-processing, and output the calculation result to an external storage after the entire neural network completes the convolution operation.

[0016] The control unit interacts with the top-level state machine information of the load balancing module, the computing processing module, the post-processing module and the sparse coding module, and is configured to control the operation of the load balancing module, the computing processing module, the post-processing module and the sparse coding module.

[0017] As a further optimization scheme of the neural network accelerator for dynamically matching non-zero values of unstructured sparse, the processing unit comprises a non-zero value matching circuit and a multiply-accumulate circuit; the non-zero value matching circuit performs bitwise AND operation on the weight tensor bitmap index and the input activation tensor bitmap index, for the non-zero pair data with the AND operation result of 1, the number of the first 1 in the non-zero data bitmap index is determined to determine the non-zero pair data access address, and the multiply-accumulate circuit outputs the convolution operation result of one grid point of the current channel after reading the non-zero pair data according to the non-zero pair data access address.

[0018] As a further optimization scheme of the neural network accelerator for dynamically matching non-zero values of unstructured sparse, the post-processing module performs post-processing on the convolution operation result of the current channel, including but not limited to batch normalization operation and rectified linear operation.

[0019] The data flow of the neural network accelerator for dynamically matching non-zero values of unstructured sparse mainly includes the following 7 steps.

[0020] Step 1, the unstructured sparse network weight tensor obtained by training is encoded in bitmap format by channel, and the encoded single-layer network weight is input to the weight cache module.

[0021] Step 2, the input activation tensor is encoded in bitmap format by point and channel on-chip by the sparse encoding module, thereby obtaining valid non-zero data and bitmap index, and the input activation tensor obtained after sparse encoding is input to the load balancing module in the order of convolution window through the input cache module.

[0022] Step 3, the network weight tensor is input to the value calculation processing module in a broadcast manner, and each PE in the calculation processing module shares the same weight data; the load balancing module labels the input order of the input activation tensor obtained after sparse encoding, that is, the received input activation data obtained after sparse encoding is labeled as the corresponding convolution window sequence number, and then the input activation data obtained after sparse encoding is grouped according to the convolution window sequence number, and the sparsity of each group of input activation data obtained after sparse encoding is marked, and the input activation data after marking the sparsity is input to each PE unit of the calculation processing module in groups, thereby mapping the calculation of one convolution window to one processing unit.

[0023] Step 4, each PE unit in the calculation processing module calculates the received weight data and input activation data, first matches the weight data and input activation data according to the bitmap index to extract the non-zero pairs that need to be calculated, and skips the invalid calculation of 0 value, and transmits the non-zero pair data that needs to be calculated to the MAC unit for multiplication and accumulation calculation. Since the calculation participated by one convolution kernel corresponds to the output of one channel in the output feature map, and the calculation participated by one convolution window corresponds to the output of one grid point in the output feature map, the weight data in the PE does not change before the calculation of one channel output corresponding to all convolution windows is completed. After the processing unit completes the calculation of one convolution window, the output is stored in the cache at the corresponding address according to the label of the input activation data in this group. Since the sparsity of the input activation data corresponding to the convolution window received by each calculation processing unit is different, the calculation time is different. In order to reduce the decline of the acceleration effect caused by load imbalance, the processing unit that completes the calculation first will send a calculation completion feedback signal to the load balancing module, and the load balancing module will input the activation data corresponding to the convolution window with the most non-zero values in the convolution window that has not been calculated in the current channel output corresponding to all convolution windows, thereby improving the overall calculation efficiency of the calculation processing module. After the calculation corresponding to a single convolution kernel is completed, the next convolution kernel data is read from the weight cache module, and the next channel data of the output activation is calculated.

[0024] Step 5, the calculated result is stored in the output buffer module and sent to the post-processing module for processing calculation including BN, ReLU, for linear rectification and pooling processing of the partial sum accumulated by the processing unit, so as to be used by the next layer network calculation, and the calculation result is stored and transmitted to the outside after the current layer neural network completes the calculation.

[0025] Step 6, the sparse coding module performs bitmap format sparse coding on the output of the single layer network after processing point by channel, and the index data in bitmap format can be obtained through non-zero value judgment, and only the non-zero active value that may participate in the next layer network calculation is transmitted to the input buffer module.

[0026] Step 7, repeat steps 3 to 6 until the last layer network output convolution operation result.

[0027] The technical scheme of the present application has the following beneficial effects:

[0028] (1) The present application proposes a neural network accelerator architecture and data flow for unstructured sparse, which innovatively completes sparse coding of input activation tensor on chip, and the sparse tensor participating in operation on chip is encoded in bitmap format, so that only non-zero value participates in the calculation process, and the sparsity of both sides of the deep neural network weight and activation is utilized, thereby effectively reducing the calculation amount and storage consumption.

[0029] (2) The present application proposes a neural network accelerator architecture and data flow for unstructured sparse, which obtains phase data index by performing or operation according to channel, arranges non-zero data point by point in the order of channel priority, and maps the output of a single convolution window to a PE, thereby effectively multiplexing the weight data and partial sum of convolution operation, and reducing useless data transfer.

[0030] (3) The present application proposes a neural network accelerator architecture and data flow for unstructured sparse,

[0031] By using the broadcast mode to transmit the weight tensor into the processing unit, and combining the marking of the input activation tensor, dynamic load balancing is realized, and the problem of acceleration effect decline caused by irregular input tensor is reduced. DETAILED DESCRIPTION

[0032] Figure 1 is the schematic diagram of the architecture of the present application.

[0033] Figure 2 is the sparse coding format used by the present application.

[0034] Figure 3 is the data arrangement order of the present application

[0035] Figure 4It is a PE processing unit structure of the present application.

[0036] Figure 5 It is a sparse tensor point product calculation schematic diagram of the present application.

[0037] Figure 6 It is a load balancing mode of the present application. DETAILED DESCRIPTION

[0038] In order to more clearly illustrate the inventive purpose, technical scheme and technical effect of the present application, the technical scheme of the present application will be described in detail below in combination with the drawings and specific embodiments.

[0039] The present application provides a kind of unstructured sparse neural network accelerator architecture and data flow, architecture and data flow as shown in Figure 1 The deep neural network selects VGG-19 network trained, pruned and quantized through CIFAR-100 data set, and the network has unstructured sparsity, and image classification task is carried out.

[0040] For single network inference calculation, it includes the following six steps:

[0041] Step one, the unstructured sparse network weight tensor obtained by training is sparsely coded, and bitmap format sparse coding is carried out according to the order of channel-row-column, and input external storage, and the accelerator stores the network weight after sparse coding into on-chip weight cache layer by layer.The bitmap format coding adopted is as shown in Figure 2 Assuming that the original data is a 9*16bit sparse tensor, of which there are 5 values of 0, and the sparsity is 5 / 9.After sparse coding, 9*1bit index data and 4*16bit value data are obtained.9*1bit index data represents whether each element of the original data is a non-zero value, 1 represents that the position element is a non-zero value, and 0 represents that the position element is zero.

[0042] Step two, the input activation tensor is on-chip sparse coded through sparse coding module, as shown in Figure 3 Bitmap format coding is carried out according to the data arrangement order of channel-row-column.The specific coding mode is as shown in Figure 2 First, the index of the position data is obtained by performing bitwise or operation on each 16bit data.Then, according to the index of each 16bit, the non-zero value is selected and input to the input cache module, so that the compressed sparse tensor format is obtained, and the compressed sparse tensor is input to the load balancing module in the order of convolution window.

[0043] Step three, the same weight data is input to each processing unit in the calculation processing module through broadcast mode, and the processing module structure is as shown in Figure 4As shown, the sharing of weight data makes the sparsity of weight data side in each processing element (PE) in the processing module the same, which is beneficial to load balancing. The input activation data is preprocessed by the load balancing module before entering the calculation processing module for calculation. The load balancing module marks the input activation data according to the input order corresponding to the convolution window number, marks the storage location of the output result corresponding to the input data, and then allocates it to the idle calculation processing unit for calculation.

[0044] Step four, input the compressed sparse tensor in bitmap format into the calculation processing unit for calculation. The input activation value is grouped according to the convolution window number and then input into the processing unit, so as to map the output of one convolution window to one processing unit. The first step is to extract the non-zero pair to be calculated according to the bitmap index, so as to skip the invalid calculation of 0 value participation. The second step is to access the corresponding non-zero value and transmit it into the MAC unit for multiplication and accumulation calculation. The first step is to find the matching item by performing AND operation on the corresponding bit of the tensor index, as shown in Figure 5 The second step is to read the corresponding data of the AND operation result with the value of 1 into the FIFO by using the AND operation result. In order to find the position of the next matching pair, the matching logic will calculate the number of 1 before the value of the current AND operation result with the value of 1, that is, count the 1 from the first position. The result of the count provides the address offset required to access the matching pair data. As shown in Figure 5 As shown, 6 and 7 of the original tensor should participate in MAC operation. After bitmap encoding, there are 3 1s before the index value of 6 and 1 1 before the index value of 7. Therefore, the data addresses of 6 and 7 are 3 and 1 respectively (the first address is 0). After extracting the non-zero data of the matching pair, it is transmitted to the FIFO (as shown in Figure 4 The MAC unit reads it out and multiplies the two values, and adds the product to the partial sum saved locally in the unit. After all data operations are completed, the single-point calculation result of one convolution window can be obtained. The calculation corresponding to one convolution kernel produces the output of one channel, and the calculation corresponding to one convolution window produces the output of one grid point. Before the calculation of all convolution windows corresponding to one channel output is completed, the weight data in the PE does not change. After the processing unit completes the calculation of one convolution window, the output is stored in the cache at the corresponding address according to the label of the group of activation data.

[0045] Because the non-structured sparse zero-value distribution is irregular, the input activation tensor sparsity obtained by each processing element (PE) is different, and thus the calculation time is different. In order to reduce the decline of the acceleration effect caused by the load imbalance, the processing element that first completes the calculation will send a calculation completion feedback signal to the load balancing module, and the load balancing module will input the activation data corresponding to the non-zero value with the largest number in all the convolution windows that have not been calculated in the convolution window corresponding to the current channel output, so as to improve the utilization of the processing element (PE) and the overall calculation efficiency of the calculation processing module. Figure 6 For example, the vertical axis is each PE, and the horizontal axis is the time required for calculation, which is in clock cycles, and the result is determined by the number of non-zero elements in the tensor data. Assuming that 8 groups of data are input to four PEs for calculation, if the next batch is calculated after waiting for the first batch of 4 groups of data to be calculated, the PE utilization rate will be reduced, and the overall efficiency of the processing module will be reduced. After dynamic allocation using the load balancing module, the PE4 that first completes the calculation will be allocated the data 7 with the largest calculation amount in the remaining sparse tensor. Similarly, the PE1 that completes the first round of calculation secondly will be allocated the data 5, the PE2 that completes the first round of calculation thirdly will be allocated the data 8, and the PE3 that completes the first round of calculation lastly will be allocated the data 6 with the smallest calculation amount in the second round of data. Because the convolution window number corresponding to the output of each PE will lose regularity, it is necessary to add a mark of the corresponding convolution window number to the sparse tensor in the load balancing module, and write the corresponding storage address after the PE completes the calculation. After the calculation corresponding to a single convolution kernel is completed, that is, the weight data is completely used, the next convolution kernel sparse tensor data is read from the weight cache, and the next channel data of the output activation is calculated.

[0046] Step five, after all the convolution kernels of the single-layer network complete the calculation, the calculated results are stored in the output cache and sent to the post-processing module for post-processing calculation. The post-processing calculation includes batch normalization (BN), rectified linear unit (ReLU), etc., which are used for activation and pooling processing of the accumulated partial sum, so as to be used by the next layer network for calculation.

[0047] Step six, the single-layer network output is encoded in bitmap format point by point by channel, and the index data of the bitmap can be obtained by judging the non-zero value, and only the non-zero activation value is stored.

[0048] Steps three to six are repeated until the output of the last layer network is obtained, and the accelerator completes the forward inference operation of the sparse neural network.

[0049] The application discloses a kind of unstructured sparse-oriented neural network accelerator architecture and data flow, and the sparse tensor of on-chip participation operation is encoded using bitmap format, and the non-zero value matching unit of PE is efficiently matched with non-zero pair by bitmap index, and the sparsity of weight and activation is used, and the invalid calculation of zero value participation is skipped to increase processing efficiency.Through on-chip sparse coding module, the storage and handling of invalid data are avoided, thereby improving the energy efficiency of the overall accelerator. The sparse tensor is marked and dynamically allocated to PE using the load balancing module, which can effectively alleviate the load imbalance problem of the calculation unit caused by the irregular input tensor, thereby realizing efficient inference of unstructured sparse neural networks.

[0050] The above examples only illustrate the technical idea of the present application and cannot limit the protection scope of the present application. Any modification made according to the technical idea of the present application on the basis of the technical solution falls within the protection scope of the present application.

Claims

1. A neural network accelerator for unstructured sparse dynamic matching non-zero values, characterized in that, The application relates to a neural network processing system and method. The application comprises: a sparse coding module for reading an input activation tensor from external storage or receiving a current channel convolution operation result output by a post-processing module, performing sparse coding on the received data point by point in a bitmap format according to channels, and outputting a current channel input activation tensor containing valid non-zero data and bitmap indexes; an input buffer module for buffering the current channel input activation tensor output by the sparse coding module, and transmitting the current channel input activation tensor to a load balancing module according to a convolution window sequence; a weight buffer module for buffering a sparse coded weight tensor read from external storage, and outputting the weight tensor of the current channel to a calculation processing module in a broadcast mode; a load balancing module for marking and grouping the received current channel input activation tensor according to a convolution window index, marking the sparsity of each group of current channel input activation tensors, inputting the current channel input activation tensors with marked sparsity to the calculation processing module in groups, selecting a group of current channel input activation tensors with the lowest sparsity, i.e. the largest calculation amount, from the groups of current channel input activation tensors not input to the calculation processing module after receiving a calculation completion feedback signal transmitted by a processing unit, and transmitting the group of current channel input activation tensors with the lowest sparsity to the processing unit which first completes the convolution operation; a calculation processing module for performing convolution operation on the current channel activation tensor and the current channel weight tensor, wherein each processing unit performs convolution operation on non-zero data according to the bitmap index of a group of current channel input activation tensors, and the processing unit outputs the convolution operation result to an output buffer module according to the convolution window index corresponding to the processed current channel activation tensor after completing the operation corresponding to one convolution window, the processing unit which first completes the convolution operation transmits a calculation completion feedback signal to the load balancing module, and the processing unit which first completes the convolution operation performs convolution operation on the next group of current channel input activation tensors and the current channel weight tensor after receiving the group of current channel input activation tensors with the lowest sparsity, until the processing unit completes the convolution operation on each group of current channel input activation tensors and the current channel weight tensor, the next channel convolution operation is started after reading the next channel activation tensor and the next channel weight tensor, and the convolution operation result of the current channel is output; an output buffer module for buffering the convolution operation result of the current channel according to the convolution window index; a post-processing module for outputting after processing the convolution operation result of the current channel, and outputting the calculation result to external storage after completing the convolution operation of the whole neural network; and a control unit for interacting with the top state machine information of the load balancing module, the calculation processing module, the post-processing module and the sparse coding module, and controlling the running of the load balancing module, the calculation processing module, the post-processing module and the sparse coding module.

2. The neural network accelerator for unstructured sparse dynamic matching of non-zero values of claim 1, wherein, The processing unit comprises a non-zero value matching circuit and a multiply-accumulate circuit; the non-zero value matching circuit performs a bitwise AND operation on a weight tensor bitmap index and an input activation tensor bitmap index, and for non-zero pair data with an AND operation result of 1, determines a non-zero pair data access address according to the number of the first 1 in the non-zero data bitmap index, and the multiply-accumulate circuit reads the non-zero pair data according to the non-zero pair data access address and then outputs a convolution operation result of one grid point in the current channel.

3. The neural network accelerator of claim 1, wherein, The post-processing module performs post-processing on the convolution operation result of the current channel, including but not limited to batch normalization operation and rectified linear operation.

4. The method of claim 1 to 3, wherein the neural network accelerator for unstructured sparse, dynamically matching non-zero values implements a sparse neural network forward inference operation. The method comprises the following steps: Step 1: performing bitmap format sparse coding on the unstructured sparse network weight tensor obtained by training according to the channel, and caching the sparse coded current channel weight tensor to the weight cache module; Step 2: performing bitmap format on-chip sparse coding on the input activation tensor point by point according to the channel, and caching the current channel input activation tensor containing valid non-zero data and bitmap index to the input cache module; Step 3: transmitting the current channel weight tensor to the computing processing module in a broadcast manner, transmitting the current channel input activation tensor to the load balancing module in the order of the convolution window according to the input cache module, marking the received current channel input activation tensor with the corresponding convolution window label by the load balancing module, grouping the current channel input activation tensor according to the label and marking the sparsity, and distributing each group of current channel input activation tensor to the idle processing unit according to the sparsity mark; Step 4: each processing unit in the computing processing module performs convolution operation on the non-zero pair data according to the bitmap index of one group of current channel input activation tensor received, and outputs the convolution operation result to the output cache module according to the convolution window label corresponding to the processed current channel activation tensor after completing the operation of the corresponding convolution window, and the processing unit that first completes the convolution operation transmits a calculation completion feedback signal to the load balancing module, and the processing unit that first completes the convolution operation receives one group of current channel input activation tensor with the lowest sparsity to perform convolution operation on the next group of current channel input activation tensor and the current channel weight tensor, until the processing unit completes the convolution operation of each group of current channel input activation tensor and the current channel weight tensor, and then reads the next channel activation tensor and the next channel weight tensor to start the convolution operation of the next channel, and outputs the convolution operation result of the current channel; Step 5: caching the convolution operation result of the current channel to the output cache module and sending it to the post-processing module, and transmitting the current channel convolution operation result after post-processing to the sparse coding module by the post-processing module, and transmitting the calculation result to the external storage after completing the convolution operation of the entire network; Step 6: the sparse coding module performs bitmap format sparse coding on the current channel convolution operation result after post-processing point by point according to the channel, and caches the next channel input activation tensor containing valid non-zero data and bitmap index to the input cache module; Step 7: repeating steps 3 to 6, and the device outputs the convolution operation result of the last channel.

Images