A block-by-block vector-zero-value sparsity-aware convolutional neural network accelerator
By designing a block-by-block vector-zero-value sparse-perceptual convolutional neural network accelerator, using the sparseness of sparse feature maps and weights, efficient feature map calculation is realized, solving the calculation efficiency of existing accelerators when processing sparse neural networks. Low and resource waste issues improve hardware performance.
Patent Information
- Application Number
- CN202411268674.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-11
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-09-11
AI Technical Summary
Existing neural network accelerators are difficult to effectively utilize sparseness, resulting in low computing efficiency and waste of resources, especially when dealing with sparse feature maps and weights, they cannot perform efficient block-by-block calculations.
A block-by-block calculation vector-zero-value sparse-sensing convolutional neural network accelerator is designed to determine the mapping relationship of feature vectors through feature search units and input feature management units, and use zero-value compression algorithm and matching-aggregated feature reading method. Avoid redundant computing and optimize storage and loading of feature blocks through memory management units.
It significantly improves feature block storage and loading efficiency, reduces the complexity of feature vector search, effectively utilizes sparseness, and improves the computing power performance of the accelerator.
Smart Images

Figure CN118798276B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of neural network technology, and in particular to a block-by-block calculated vector-zero value sparse-aware convolutional neural network accelerator. Background Art
[0002] Convolutional neural network is a powerful algorithm that extracts features through a series of convolutional layers and then sends the features to specific task heads to complete tasks such as classification and detection. Convolutional neural networks have achieved very good results on image data. In recent years, convolutional neural networks have begun to be used to extract effective information from point cloud data. Given an input feature map A feature vector in and weight , the basic convolution operation can be expressed as:
[0003]
[0004] in, Indicates the number of input channels, The input feature map is The size of the direction, The convolution kernel is The size of the direction, Describes the number of output channels, The output feature vector values, Indicates rounding down. Indicates that the weight or feature is In existing neural networks, the commonly used convolution kernel size is ), then, the output features of each layer are used as the input features of the next layer after batch normalization and activation function, and the weights Through training, here All are integers.
[0005] Convolutional neural networks widely use ReLU as an activation function, and the formula is as follows:
[0006]
[0007] The results of the convolution calculation in formula (1) are both positive and negative. After the ReLU activation function is calculated, all negative numbers will become 0, which leads to a large amount of zero-value sparsity in the neural network feature map. For weights, there is no explicit zero-value sparsity, that is, there are few or almost no explicit zeros in the weights. But in general, there is a lot of redundancy in the weights in the neural network. Using network pruning technology to remove these redundant weights can produce strong sparsity while keeping performance indicators such as model accuracy almost unchanged. In addition, the distribution range of point clouds in space is very large (the typical laser radar scanning range is 50-100m), and there are only a few locations in space. This leads to the input feature map There are only a few positions with feature vectors, and most positions are empty (that is, there are no feature vectors). This is called vector sparsity, and the corresponding feature map is also called a sparse feature map. In contrast, image data has no explicit vector sparsity. Specifically, given a picture, the picture size is fixed, and each position in the picture corresponds to a pixel, but usually a large part of the pixels in the picture are background, and removing them has little effect on the accuracy of specific classification, detection and other tasks. In this case, the distribution of pixels in the picture will also become sparse, and the corresponding feature map becomes a sparse feature map, which contains a lot of vector sparsity.
[0008] At the same time, the calculation of neural networks consumes a lot of resources. In order to enable the deployment of neural networks in resource-constrained environments, such as mobile devices and embedded systems, it is necessary to design dedicated neural network accelerators. If zero-value sparsity and vector sparsity can be used to reduce unnecessary calculations, the performance of the hardware can be improved. However, after considering sparsity, feature maps and weights will become irregular, and their loading, calculation, and storage methods will become complicated. General-purpose AI processors such as GPUs cannot take advantage of sparsity because they can only process regular data.
[0009] Sparse compression coding is a method that uses the sparsity of zero values to reduce storage requirements. Common sparse data compression methods include run length coding (RLC), frequent value compression (FVC), and zero value compression (ZVC). When the features or weights of run length coding are irregular, the coding length is uncontrollable, and a storage method of splitting and assembling needs to be designed, which makes data reading difficult and requires additional split position information records. Frequent value compression uses Huffman tree coding, which also has the problem of data storage with unfixed coding length. In addition, the compression and decompression process of Huffman coding is relatively complex and takes a long time, which is not suitable for real-time data transmission. Zero value compression uses 0-1 indexes to represent the positions of zero and non-zero values in the feature vector. The implementation is simple and efficient, does not require a separate code table, and is conducive to hardware implementation.
[0010] In addition, in general, neural network feature maps are large, far exceeding the storage capacity of an accelerator. One approach is to first divide the feature map into multiple small blocks, then load the features onto the chip in blocks and perform the corresponding convolution operations, such as Figure 1 As shown. When dividing the blocks, if some blocks have no feature vectors, they can be directly deleted, thereby improving the working efficiency of the accelerator. However, dividing the feature map into blocks has the following problems: (1) Convolution calculations have data dependence. The calculation of the output features of the current position requires the use of multiple adjacent input features. This results in the need to use the input features of the adjacent blocks when calculating the output features of the block boundary. After the accelerator calculates the output feature map of each block, it needs to update the boundary features to the adjacent feature blocks at the same time; (2) For sparse neural network feature maps, how many features each block has and whether there are features at the boundary are random and cannot be known in advance. These problems make block calculation difficult. Without targeted consideration, it is easy to cause low utilization of the accelerator's storage and computing resources and large computing delays. Therefore, there is currently no neural network accelerator that can well support block-by-block calculations of sparse neural network feature maps. Summary of the invention
[0011] In order to solve the above problems, the present invention provides a vector-zero value sparse-aware convolutional neural network accelerator with block-by-block calculation. The accelerator can better support the block-by-block calculation of sparse neural network feature maps, and can effectively utilize the vector sparsity and zero value sparsity in the sparse feature maps to improve the computational efficiency of the feature maps, thereby reducing resource and energy consumption.
[0012] The present invention provides a block-by-block vector-zero-value sparse-aware convolutional neural network accelerator, comprising: an overall controller, a feature search unit, an input feature management unit, a weight management unit, a convolution calculation array, an output feature management unit and a memory management unit, wherein the overall controller is responsible for controlling the operation of the entire accelerator; the feature search unit is used to determine the mapping relationship between the output feature vector and the input feature, that is, which feature vectors are required to participate in calculating each output feature vector; the input feature management unit is used to temporarily store the input feature vector to be used, and convert the corresponding feature vector into a memory management unit according to the mapping relationship searched by the feature search unit. The non-zero values in the quantity are loaded into the convolution calculation array to perform specific convolution operations; the convolution calculation array includes multiple convolution calculation units, each of which includes a multiplier, an adder tree, an accumulator and a batch normalization unit to perform multiplication and addition operations in the convolution operation, as well as subsequent batch normalization operations; the weight management unit is used to temporarily store weights and output the weights to the convolution calculation array as needed; the output feature management unit temporarily stores the output feature vector and compresses the feature vector using a zero value compression algorithm; the memory management unit writes the compressed features back to an off-chip memory, such as DDR4, to complete the update of the feature block.
[0013] The feature search unit includes a coordinate buffer and a feature searcher. The coordinate buffer is used to store the coordinates of each feature vector in the feature block; the feature searcher is used to search for valid features. First, the feature block is converted into a spatial bitmap, and then the convolution kernel is used to traverse the spatial bitmap from left to right and from top to bottom to obtain the mapping relationship between the input features and the output features. In this way, the accelerator avoids redundant calculations related to vector sparsity.
[0014] The input feature management unit includes a feature decoder, a coordinate buffer, a 0-1 index buffer, a non-zero value buffer, a coordinate matching unit and a feature aggregator; the feature decoder first reads the compressed feature vector from the off-chip memory and decodes the read feature vector to obtain the coordinates of the feature vector after decoding. , 0-1 index and non-zero value, and store these three parts in the corresponding buffer; then, the coordinate matching unit reads the coordinate buffer and determines whether the coordinates of the read feature vector match the search results of the feature search unit. If they match, the feature aggregator will check whether the weight corresponding to the feature vector has been pruned. If not, the feature aggregator loads the non-zero value and the corresponding weight in the feature vector into the convolution calculation array. The specific method is: read the 0-1 index buffer, traverse the index and count, and each time the index 1 is traversed, a non-zero value is read from the non-zero value buffer and sent to the convolution calculation array to perform the convolution operation. The count value will be converted into the read address of the convolution weight corresponding to the feature, and sent to the weight management unit to read the corresponding weight value. Through this matching-aggregation feature reading method, the accelerator can effectively utilize the zero-value sparsity in the feature vector and weight, avoid zero-value related convolution operations, and help improve hardware performance.
[0015] The weight management unit includes a pruning flag register and a weight buffer. Given the weight data in the neural network, the network pruning technology is used to remove the redundant parts of the weights. Specifically, the weights are sorted in descending order according to their absolute values, and the first R% of the weights that have a greater impact on network performance are stored in the weight buffer. R is set according to actual needs. At the same time, the weight pruning flag is stored in the pruning flag register. The so-called weight pruning flag is used to indicate which weights in the weight data have been pruned and which weights are stored in the flag bits of the weight buffer.
[0016] The convolution calculation array includes 32 convolution calculation units, each of which contains 8 multipliers, an addition tree, an accumulator and a batch normalization unit. The convolution calculation array can calculate 32 output channels at a time. Each convolution calculation unit corresponds to an output channel. These 32 output channels are called a group. If the number of output channels is , it will be divided into Groups, here Indicates rounding up. Each convolution calculation unit can calculate the multiplication of 8 input channels in one cycle and accumulate the multiplication results. After batch normalization and ReLU function activation, the accumulated results (i.e., output features) will be sent to the output buffer of the output feature management unit for temporary storage and waiting for further processing. In order to improve the on-chip weight reuse rate and reduce the storage resource consumption of the weight management unit, the coordinates are first The input features are traversed dimensionally, and after calculating a set (32) of output channels for all features, the weights are updated and the next set of output channels are calculated.
[0017] The output feature management unit includes an output buffer, a feature encoder, a coordinate buffer, a 0-1 index buffer, a non-zero value buffer, a feature assembler and a boundary detection unit. The output buffer is used to buffer the output result of the convolution calculation array, which has not been compressed. The feature encoder performs zero-value compression encoding on the output feature and stores the encoding results in the corresponding coordinate buffer, 0-1 index buffer and non-zero value buffer respectively. The feature assembler assembles the compressed encoding results into a standard frame of fixed length. The boundary detection unit is used to detect whether the current output feature vector is located at the block boundary and send the detection result to the memory management unit.
[0018] In order to facilitate the storage and reading of feature blocks in the off-chip memory, the off-chip memory is first segmented, and each feature block occupies a segment of the off-chip memory exclusively. The interaction between the accelerator and the off-chip memory is completed by the memory management unit. The memory management unit includes a memory read-write controller and a feature block storage table. The memory read-write controller is used to perform specific off-chip memory read and write operations, and the feature block storage table is used to record the storage address of each feature block in the memory and complete the feature block update. The feature block storage table contains four items: block label, the starting address of the memory segment corresponding to the block, the space used in each segment, and the number of feature vectors in each feature block. When the memory management unit writes the output feature vector back to the off-chip memory, it will synchronously update the third and fourth columns of the corresponding feature block in the table, that is, the used space and the number of feature vectors. In addition, the memory management unit will also detect whether the output feature vector is a block boundary. If so, the feature vector will be written into multiple feature blocks containing the boundary feature vector at the same time, and the corresponding items in the feature block storage table will be synchronously updated, thereby completing the update of the feature vector in each block.
[0019] In practical applications, first the main processor, such as the CPU, blocks and compresses the input feature map of the first layer of the neural network, stores the compressed feature blocks in the memory, and initializes the feature block storage table, and then the accelerator starts working. The workflow of the accelerator is as follows: first, the feature vector coordinates, weights, feature vectors compressed by blocks and other data are loaded from the off-chip memory to the chip, where the coordinates will be stored in the feature search unit, the weights will be stored in the weight management unit, and the feature vector will be stored in the input feature management unit after decompression. The compressed feature vector also contains the coordinates; then the feature search unit searches for a valid input feature vector, and then the feature aggregator first determines whether the weight corresponding to the feature vector is pruned. If not, the searched input feature vector is read, and the zero value in the feature vector is filtered out by traversing the 0-1 index, and only the non-zero value is loaded into the convolution calculation array; then the weight required for the convolution calculation is read, and the convolution calculation is performed; then the convolution calculation result is compressed and encoded, and the output feature vector is recompressed back to the coordinates, 0-1 index, and non-zero value, and the compressed features are assembled into a standard frame of fixed length, such as 512 bits; then the compressed standard frame is written back to the off-chip memory; finally, the feature block storage table is updated.
[0020] Compared with the prior art, the present invention has the following beneficial technical effects:
[0021] (1) The off-chip memory is segmented, each segment stores a feature block, and a memory management method based on the feature block storage table is used to update the data in the feature block. In this way, only the input features of the first layer of the neural network need to be explicitly segmented (completed by a general-purpose processor such as the CPU). After the accelerator calculates the output feature vector, it only needs to update the data in each feature block (completed by the accelerator), and no explicit re-segmentation operation is required, which significantly improves the storage and loading efficiency of feature blocks.
[0022] (2) By converting the sparse feature map after block division into a spatial bitmap, the bitmap is traversed to search for the input feature vector required for calculation, which greatly reduces the search complexity of the feature vector and indirectly improves the efficiency of convolution calculation;
[0023] (3) A zero-value compression algorithm is used to reduce storage requirements, and a matching-aggregation input feature reading method is used to effectively utilize the zero-value sparsity in feature vectors and weights, avoiding redundant data operations related to vector sparsity and zero-value sparsity, greatly improving the computing power of the accelerator. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0025] Figure 1 Schematic diagram of the block operation involved in the background technology of the present invention, wherein (a) is a schematic diagram of a sparse feature map, and (b) is a schematic diagram of a feature map after block division;
[0026] Figure 2 A schematic diagram of the structure of a vector-zero-value sparse-aware convolutional neural network accelerator for block-by-block calculation provided by an embodiment of the present invention;
[0027] Figure 3 A schematic diagram of the working mode of the feature search unit provided in an embodiment of the present invention, wherein (a) is a schematic diagram of a feature block to be searched, (b) is a schematic diagram of a bitmap corresponding to the feature block, and (c) is a schematic diagram of a mapping relationship between a searched input feature vector and an output feature vector. Assume that the convolution kernel size is ;
[0028] Figure 4 A schematic diagram of a zero-value compression algorithm and a process of assembling a standard frame provided by an embodiment of the present invention;
[0029] Figure 5 A schematic diagram showing the storage of feature blocks provided in an embodiment of the present invention;
[0030] Figure 6 A schematic diagram of the working process of a neural network accelerator provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0032] The data processed by the accelerator of the present invention is a sparse feature map. As described in the background technology, a sparse feature map may exist in: (1) a neural network for processing point clouds; (2) a neural network that can perceive background pixels in an image. The accelerator is applicable to both types of networks. Given a sparse feature map, the feature map is divided into blocks according to a specific size, such as Figure 1As shown, the initial segmentation of the feature map is performed on a general-purpose processor such as a CPU, and the segmented feature map is stored in a DDR. The accelerator calculates and outputs feature vectors by block, and writes the calculated feature vectors back to the corresponding blocks in the DDR, including multiple feature blocks corresponding to the boundary feature vectors.
[0033] Figure 2 A schematic diagram of a vector-zero-value sparse-aware convolutional neural network accelerator for block-by-block calculation provided by the present invention. As shown in the figure, the accelerator includes: an overall controller, a feature search unit, an input feature management unit, a weight management unit, a convolution calculation array, an output feature management unit and a memory management unit. The overall controller is responsible for controlling the operation of the entire accelerator; the feature search unit is used to determine the mapping relationship between the output feature vector and the input feature, that is, which feature vectors are required to participate in calculating each output feature vector; the input feature management unit is used to temporarily store the input feature vector to be used, and according to the mapping relationship searched by the feature search unit, The non-zero values in the corresponding feature vector are loaded into the convolution calculation array to perform specific convolution operations; the convolution calculation array includes multiple convolution calculation units, each of which includes a multiplier, an adder tree, an accumulator and a batch normalization unit to perform multiplication and addition operations in the convolution operation, as well as subsequent batch normalization operations; the weight management unit is used to temporarily store weights and output the weights to the convolution calculation array as needed; the output feature management unit temporarily stores the output feature vector and compresses the feature vector using a zero value compression algorithm; the memory management unit writes the compressed features back to an off-chip memory, such as DDR4, to complete the update of the feature block.
[0034] The feature search unit includes a coordinate buffer and a feature searcher. The coordinate buffer is used to store the coordinates of each feature vector in the feature block; the feature searcher is used to search for valid features, such as Figure 3 As shown in the figure, the feature block is first converted into a spatial bitmap, and then the convolution kernel is used to traverse the spatial bitmap from left to right and from top to bottom to obtain the mapping relationship between the input features and the output features. In this way, the accelerator avoids redundant calculations related to vector sparsity.
[0035] In this accelerator, all buffers are implemented using dual-port RAM. Compared with ping-pong RAM, the benefits of using dual-port RAM are: for sparse feature maps, the number of feature vectors in each feature block is not fixed. Using dual-port RAM can reduce the calculation pause problem that may be caused by block updates (that is, after one block is calculated, the next block needs to be calculated).
[0036] The input feature management unit includes a feature decoder, a coordinate buffer, a 0-1 index buffer, a non-zero value buffer, a coordinate matching unit and a feature aggregator; the feature decoder first reads the compressed feature vector from the off-chip memory and decodes the read feature vector to obtain the coordinates of the feature vector after decoding. , 0-1 index and non-zero value, and store these three parts in the corresponding buffer; then, the coordinate matching unit reads the coordinate buffer and determines whether the coordinates of the read feature vector match the search results of the feature search unit. If they match, the feature aggregator will check whether the weight corresponding to the feature vector has been pruned. If not, the feature aggregator loads the non-zero value and the corresponding weight in the feature vector into the convolution calculation array. The specific method is: read the 0-1 index buffer, traverse the index and count, and each time the index 1 is traversed, a non-zero value is read from the non-zero value buffer and sent to the convolution calculation array to perform the convolution operation. The count value will be converted into the read address of the convolution weight corresponding to the feature, and sent to the weight management unit to read the corresponding weight value. Through this matching-aggregation feature reading method, the accelerator can effectively utilize the zero-value sparsity in the feature vector and weight, avoid zero-value related convolution operations, and help improve hardware performance.
[0037] The weight management unit includes a pruning flag register and a weight buffer. Given the weight data in the neural network, the network pruning technology is used to remove the redundant parts of the weights. Specifically, the weights are sorted in descending order according to their absolute values, and the first R% of the weights that have a greater impact on network performance are stored in the weight buffer. R is set according to actual needs, for example, it can be 50. At the same time, the weight pruning flag is stored in the pruning flag register. The so-called weight pruning flag is used to indicate which weights in the weight data have been pruned and which weights are stored in the flag bits of the weight buffer.
[0038] The convolution calculation array includes 32 convolution calculation units, each of which contains 8 multipliers, an addition tree, an accumulator and a batch normalization unit. The convolution calculation array can calculate 32 output channels at a time. Each convolution calculation unit corresponds to an output channel. These 32 output channels are called a group. If the number of output channels is , it will be divided into Groups, here Indicates rounding up. Each convolution calculation unit can calculate the multiplication of 8 input channels in one cycle and accumulate the multiplication results. Assuming that the input feature map has 256 channels, it takes 32 (256 / 8) clock cycles to calculate the complete output feature. After batch normalization and ReLU function activation, the accumulated result (i.e., output feature) will be sent to the output buffer of the output feature management unit for temporary storage and awaiting further processing. In order to improve the on-chip weight reuse rate and reduce the storage resource consumption of the weight management unit, first The input features are traversed dimensionally, and after calculating a set (32) of output channels for all features, the weights are updated and the next set of output channels are calculated.
[0039] The output feature management unit includes an output buffer, a feature encoder, a coordinate buffer, a 0-1 index buffer, a non-zero value buffer, a feature assembler and a boundary detection unit. The output buffer is used to buffer the output result of the convolution calculation array, which has not been compressed. The feature encoder performs zero-value compression encoding on the output feature and stores the encoding results in the corresponding coordinate buffer, 0-1 index buffer and non-zero value buffer respectively. The feature assembler assembles the compressed encoding results into a standard frame of fixed length. The boundary detection unit is used to detect whether the current output feature vector is located at the block boundary and send the detection result to the memory management unit.
[0040] To facilitate the storage and reading of feature blocks in the off-chip memory, the off-chip memory is first segmented. Each feature block occupies a segment of the off-chip memory. The interaction between the accelerator and the off-chip memory is completed by the memory management unit. The memory management unit contains a memory read-write controller and a feature block storage table. The memory read-write controller is used to perform specific off-chip memory read and write operations. The feature block storage table is used to record the storage address of each feature block in the memory and complete the feature block update. The feature block storage table contains four items: block label, the starting address of the memory segment corresponding to the block, the space used in each segment, and the number of feature vectors in each feature block, such as Figure 5 As shown, when the memory management unit writes the output feature vector back to the off-chip memory, it will synchronously update the third and fourth columns of the corresponding feature block in the table, that is, the used space and the number of feature vectors. In addition, the memory management unit will also detect whether the output feature vector is a block boundary. If so, the feature vector will be written to multiple feature blocks containing the boundary feature vector at the same time, and the corresponding items in the feature block storage table will be synchronously updated to complete the update of the feature vector in each block. In this way, we only need to perform explicit block operations on the input features of the first layer of the neural network (completed by general processors such as the CPU). After the accelerator calculates the output feature vector, it only needs to update the data in each feature block (completed by the accelerator), and no explicit re-blocking operations are required, which significantly improves the storage and loading efficiency of feature blocks.
[0041] In practical applications, the main processor, such as the CPU, first divides and compresses the input feature map of the first layer of the neural network, stores the compressed feature blocks in the memory, and initializes the feature block storage table, and then the accelerator starts working. The workflow of the accelerator is as follows: Figure 6 As shown: first, the feature vector coordinates, weights, feature vectors compressed by blocks and other data are loaded from the off-chip memory to the chip, where the coordinates are stored in the feature search unit, the weights are stored in the weight management unit, the feature vectors are decompressed and stored in the input feature management unit, and the compressed feature vectors also contain coordinates; then the feature search unit searches for valid input feature vectors, and then the feature aggregator first determines whether the weight corresponding to the feature vector is cut off. If not, the searched input feature vector is read, and the zero value in the feature vector is filtered out by traversing the 0-1 index, and only the non-zero value is loaded into the convolution calculation array; then the weight required for the convolution calculation is read, and the convolution calculation is performed; then the convolution calculation result is compressed and encoded, and the output feature vector is recompressed back to the coordinates, 0-1 index, and non-zero value, and the compressed features are assembled into a standard frame of fixed length, such as 512 bits; then the compressed standard frame is written back to the off-chip memory; finally, the feature block storage table is updated.
[0042] The zero-value compression method used by the feature encoder in this accelerator is as follows: Figure 4 Given a feature ,here is the coordinate of the feature vector in the feature block, is the number of channels. When compressing, the features are first traversed along the channel dimension to generate a 0-1 index, that is, 0 is used to indicate that the feature value of the channel is zero, and 1 is used to indicate that the feature value of the output channel is non-zero. Non-zero values are stored separately. The compressed features are divided into three parts: (1) Feature coordinates , (2) 0-1 index (length equal to the number of channels ), (3) Index The non-zero value corresponding to 1 is recorded as . Assume that the number of channels of the feature vector is , the eigenvalue bit width is 8 bits. If all 32 eigenvalues in the vector are 0, the feature vector can be compressed into a 32-bit all-zero mask (8x compression ratio). If all 32 eigenvalues are 1, a 32-bit all-1 mask will be generated (12.5% additional storage overhead because non-zero values must be saved). The more zero values in the feature vector, the better the compression effect. If 60% of the features are zero values, the overall compression ratio is 1.91 times. In this way, all features in the feature block are compressed until the end. Then, the compressed features are cascaded and sliced into standard frames of fixed length (such as 512 bits) for efficient storage in off-chip memory.
[0043] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A vector-zero-value sparse-aware convolutional neural network accelerator supporting block-by-block calculation, characterized in that: The accelerator includes: an overall controller, a feature search unit, an input feature management unit, a weight management unit, a convolution calculation array, an output feature management unit and a memory management unit, and the overall controller is responsible for controlling the operation of the entire accelerator; The feature search unit is used to determine the mapping relationship between the output feature vector and the input feature, that is, to calculate which feature vectors are required to participate in each output feature vector. The feature search unit includes a coordinate buffer and a feature searcher. The coordinate buffer is used to store the coordinates of each feature vector in the feature block. The feature searcher is used to search for valid features. First, the feature block is converted into a spatial bitmap, and then the convolution kernel is used to traverse the spatial bitmap from left to right and from top to bottom to obtain the mapping relationship between the input feature and the output feature. The input feature management unit is used to temporarily store the input feature vector to be used, and load the non-zero value in the corresponding feature vector into the convolution calculation array according to the mapping relationship searched by the feature search unit to perform a specific convolution operation. The input feature management unit includes a feature decoder, a coordinate buffer, a 0-1 index buffer, a non-zero value buffer, a coordinate matching unit and a feature aggregator; the feature decoder first reads the compressed feature vector from the off-chip memory, and decodes the read feature vector, and obtains the coordinates (x, y), 0-1 index and non-zero value of the feature vector after decoding, and stores these three parts into the corresponding buffer; then, the coordinate matching unit reads the coordinate buffer, and determines whether the coordinates of the read feature vector match the search results of the feature search unit. If they match, the feature aggregator will check whether the weight corresponding to the feature vector has been cut off. If not, the feature aggregator loads the non-zero value and the corresponding weight in the feature vector into the convolution calculation array; The weight management unit is used to temporarily store weights and output the weights to the convolution calculation array as needed. The weight management unit includes a pruning flag register and a weight buffer. Given the weight data in the neural network, the network pruning technology is used to remove the redundant parts of the weights. The convolution calculation array includes a plurality of convolution calculation units, each of which includes a multiplier, an addition tree, an accumulator, and a batch normalization unit for performing multiplication and addition operations in the convolution operation, and a subsequent batch normalization operation; The output feature management unit temporarily stores the output feature vector and compresses the feature vector using a zero-value compression algorithm. The output feature management unit includes an output buffer, a feature encoder, a coordinate buffer, a 0-1 index buffer, a non-zero value buffer, a feature assembler and a boundary detection unit. The output buffer is used to buffer the output result of the convolution calculation array, which has not been compressed. The feature encoder performs zero-value compression encoding on the output feature and stores the encoding results in the corresponding coordinate buffer, 0-1 index buffer and non-zero value buffer respectively. The feature assembler assembles the compressed encoding results into a standard frame of a fixed length. The boundary detection unit is used to detect whether the current output feature vector is located at the block boundary, and sends the detection result to the memory management unit; The memory management unit writes the compressed features back to the off-chip memory to complete the update of the feature block. The interaction between the accelerator and the off-chip memory is completed by the memory management unit. The memory management unit includes a memory read-write controller and a feature block storage table. The memory read-write controller is used to perform specific off-chip memory read and write operations. The feature block storage table is used to record the storage address of each feature block in the memory and complete the feature block update.
2. The accelerator according to claim 1, characterized in that The specific implementation method of the feature aggregator in the input feature management unit to load features and weights into the convolution calculation array is as follows: read the 0-1 index buffer, traverse the index buffer and count, and each time it traverses to index 1, read a non-zero value from the non-zero value buffer and send it to the convolution calculation array to perform the convolution operation. The count value will be converted into the read address of the convolution weight corresponding to the feature vector, and sent to the weight management unit to read the corresponding weight value.
3. The accelerator according to claim 1, characterized in that The weight management unit is specifically implemented as follows: sort the weights in descending order according to their absolute values, take the first R% of the weights that have a great impact on network performance and store them in the weight buffer, where R is set according to actual needs, and at the same time, store the weight pruning flag in the pruning flag register.
4. The accelerator according to claim 1, characterized in that The convolution calculation array includes 32 convolution calculation units, each of which contains 8 multipliers, an addition tree, an accumulator and a batch normalization unit. The convolution calculation array calculates 32 output channels at a time. Each convolution calculation unit corresponds to an output channel. These 32 output channels are called a group. If the number of output channels is C out , it will be divided into Groups, It means rounding up. Each convolution calculation unit calculates the multiplication of 8 input channels in one cycle and accumulates the multiplication results. The accumulated results, i.e. the output features, are sent to the output buffer of the output feature management unit for temporary storage after batch normalization and ReLU function activation, waiting for further processing.
5. The accelerator according to claim 1, characterized in that: The feature block storage table contains four items: block label, the starting address of the memory segment corresponding to the block, the used space of each segment, and the number of feature vectors in each feature block. When the memory management unit writes the output feature vector back to the off-chip memory, it will synchronously update the third and fourth columns of the corresponding feature block in the table, that is, the used space and the number of feature vectors. In addition, the memory management unit will also detect whether the output feature vector is a block boundary. If so, the feature vector will be written to multiple feature blocks containing the boundary feature vector at the same time, and the corresponding items in the feature block storage table will be synchronously updated to complete the update of the feature vector in each block.
6. The accelerator according to claim 1, characterized in that The workflow of the entire accelerator is as follows: first, the feature vector coordinates, weights, and feature vectors compressed by blocks are loaded from the off-chip memory to the chip, where the coordinates are stored in the feature search unit, the weights are stored in the weight management unit, and the feature vectors are decompressed and stored in the input feature management unit. The compressed feature vectors also contain coordinates; then the feature search unit searches for valid input feature vectors, and then the feature aggregator determines whether the weights corresponding to the input feature vectors are pruned. If not, the searched input feature vectors are read, and the zero values in the feature vectors are filtered out by traversing the 0-1 index, and only non-zero values are loaded into the convolution calculation array; then the weights required for the convolution calculation are read, and the convolution calculation is performed; then the convolution calculation results are compressed and encoded, and the output feature vectors are recompressed back to the coordinates, 0-1 indexes, and non-zero values, and the compressed features are assembled into a standard frame of fixed length; then the compressed standard frame is written back to the off-chip memory; finally, the feature block storage table is updated.
Citation Information
Patent Citations
System and method for compact and efficient sparse neural networks
CN111247537A
Image processing acceleration method, image processing model storage method and corresponding devices
CN111831254A
Convolutional neural network accelerator based on feature map sparsity
CN113077047A
Efficient deep convolutional neural network pruning method
CN113610227A
Adaptive pruning model compression algorithm based on grouping attention mechanism
CN114118402A
