Run-length encoding accelerator and method for sparse CNN neural network model

Abstract

The present invention discloses a run-length encoding accelerator and method for a sparse CNN neural network model. The accelerator includes: a top-level controller, which is used to identify the input and output data types and perform group processing, and transmit weight data and excitation data to a run-length encoding module; a run-length encoding module, which is used to compress the result data output by the calculation module, and transmit the compressed and encoded calculation results to an external memory; a run-length decoding module, which is used to decompress the data read from the external memory and transmit the data to a data gating; the data gating is used to identify the zero value of the input excitation data and the weight data, and skip the multiplication and addition operation for the zero value; and a calculation module, which is used to perform multiplication and addition operations on the weight data and excitation data transmitted by the data gating and give the calculation results. The present invention reduces the storage space and power consumption required for the CNN model based on data compression and data gating, and has the advantages of high energy efficiency and no need to access a large amount of memory.

Description

Technical Field

[0001] This invention relates to a run-length encoding accelerator and method for sparse CNN neural network models, belonging to the field of digital signal processing and hardware-accelerated neural network algorithms. Background Technology

[0002] Neural networks are currently a hot research topic in the field of artificial intelligence. LeNet was one of the earliest convolutional neural networks and propelled the development of deep learning. However, due to limitations in computer performance, although LeNet achieved good results in image classification, it did not attract much attention. CNNs (Convolutional Neural Networks) deepen the network structure based on LeNet, learning richer and higher-dimensional image features. CNNs use a deeper network structure, namely convolutional layers + convolutional layers + pooling layers, to extract image features, and replace the previous sigmoid activation function with ReLU. CNNs achieved remarkable results at ImageNet LSVRC-2010 in 2010.

[0003] However, due to the relatively large size of CNN models and the massive amount of data required for convolution operations, they consume a great deal of energy in data transmission and run slowly on traditional computers. Common hardware acceleration optimization methods include increasing the parallelism in the hardware computation process to improve the computation speed, but this consumes excessive hardware resources. Another existing hardware optimization method is approximation processing of data, which involves using approximate functions to process complex activation functions. This reduces computational cost but lowers accuracy.

[0004] In-memory computing (CIM) improves energy efficiency by reducing memory accesses through parallel multiply-accumulate (MAC) operations and on-chip caching. However, the capacity required by large-scale neural networks far exceeds the capacity of on-chip memory. Therefore, processors based on in-memory computing (CIM) still encounter memory bottlenecks.

[0005] Model optimization is a common solution. During training, size-based weight pruning gradually reduces the model weights to zero, achieving model sparsity, and sparse models are easy to compress. Summary of the Invention

[0006] To overcome the problem that in-memory computation of neural networks still requires a large amount of memory access and generates a large amount of energy, this invention proposes an accelerator and acceleration method using run-length encoding for sparse CNN models.

[0007] The technical solution adopted by the accelerator of this invention is as follows:

[0008] Run-length encoding accelerators for sparse CNN neural network models include:

[0009] The top-level controller is used to identify the input and output data types and perform grouping processing, transmitting weight data and stimulus data to the run-length encoding module;

[0010] The run-length encoding module is used to compress the output data of the calculation module and transmit the compressed calculation results to external storage.

[0011] The run-length decoding module is used to decompress data read from external memory and transmit the data to the data gating.

[0012] Data gating is used to identify zero values ​​in the input stimulus and weight data and skip multiply-accumulate operations for zero values.

[0013] The calculation module is used to perform multiply-accumulate operations on the weighted and incentive data transmitted by the data gating system and to provide the calculation results.

[0014] The present invention utilizes the above-mentioned run-length encoding accelerator to accelerate a method comprising the following steps: the top-level controller processes the input image, transmits the data to be compressed to the run-length encoding module, and transmits the received data to be decompressed to the run-length decoding module; the run-length encoding module compresses the data transmitted from the top-level controller and stores the compressed result data in an external memory; the run-length decoding module decompresses the input stimulus data and weight data written from the external memory to the calculation module, and then transmits the obtained data to the data gating module; the data gating module records the zero values ​​in the decompressed input stimulus data and weight data, and skips subsequent multiplication and addition operations for zero values.

[0015] Furthermore, the specific method for data compression by the run-length encoding module is as follows:

[0016] 1) Record consecutive zero values ​​as "runs". The maximum length of a "run" is 32, so the number of consecutive zero values ​​can be represented by a 5-bit binary number;

[0017] 2) Represent the non-zero number after the zero value as a 16-bit binary number;

[0018] 3) Repeat step 1) to continue counting zero-value run-length codes;

[0019] 4) Repeat step 2) to continue encoding non-zero values;

[0020] 5) Each run of 5-bit zero values ​​is followed by a 16-bit non-zero number, which is an array of 5+16 bits. A coded data packet has three arrays of this 21-bit type. The last bit of the three arrays is a 1-bit number to indicate whether the data packet is the last data packet of the entire encoding. If it is, it is represented as "1", otherwise it is "0". That is, each coded data packet is a 3*(5+16)+1, which is a 64-bit binary array.

[0021] 6) Except for the data initially input into the convolutional layer, all other feature map data are stored in external memory in the format obtained by the methods in steps 1) to 5).

[0022] Furthermore, the run-length decoding module does not decompress the first layer of the CNN model. For convolutional layers other than the first layer of the CNN model, the specific decompression method of the run-length decoding module is as follows:

[0023] 1) For each 64-bit compressed data packet received, split it into three "runs" to obtain three 5+16-bit sets and one bit to determine whether the transmission has ended;

[0024] 2) Record three sets of 5-bit "runs", expand them into three non-zero numbers, and pass them into the data gating in sequence;

[0025] 3) The data gating receives the number of consecutive zero values ​​and records them. The corresponding zero values ​​are located in the feature map after it is expanded into a one-dimensional array or a multi-dimensional vector. The multiplication and addition operations are skipped for the pixels at these locations, while non-zero values ​​are written into the calculation module for calculation.

[0026] The accelerator method of the present invention can reduce the data size by using a run-length encoding module, thereby reducing access to DRAM and reducing storage bandwidth requirements. Furthermore, by using data gating, it can skip the calculation of zero values, saving a lot of power consumption and computing power. It has the advantages of high energy efficiency, small size, and no need for frequent access to off-chip storage. Attached Figure Description

[0027] Figure 1 This is a block diagram of an accelerator hardware structure for CNN models according to the present invention;

[0028] Figure 2 This is a flowchart of an accelerator computation method for CNN models according to the present invention;

[0029] Figure 3 This is a schematic diagram of the accelerator encoding format in an embodiment of the present invention. Detailed Implementation

[0030] The present invention will now be described in detail with reference to the accompanying drawings.

[0031] like Figure 1 The diagram illustrates a run-length encoding accelerator architecture for sparse CNN neural network models. The accelerator includes: a top-level controller for identifying and grouping input / output data types, passing inputs such as weight data and activation data to a run-length encoding module; a run-length encoding module for compressing the output data from the computation module and transmitting the compressed and encoded computation results to external memory (DRAM); a run-length decoding module for decompressing data read from external memory and transmitting the data to a data gating module; a data gating module for identifying zero values ​​in the decoded input activation data and weight data, recording the position of the zero values ​​in the expanded vector array, and skipping multiply-accumulate operations for them; and a computation module for performing multiply-accumulate operations on the weight data and activation data transmitted by the data gating module and providing the computation results. The top-level controller is connected to the decoding module, encoding module, and data gating module; the encoding module is connected to the computation module, DRAM, and top-level controller; the decoding module is connected to the top-level controller, data gating module, and computation module; and the data gating module is connected to the gating module of the top-level controller, the gating module of the decoding module, and the computation module.

[0032] like Figure 2 The diagram illustrates the operation process of a convolutional network. The top-level controller controls the signal flow, controlling the input data as it is decoded by the decoding module, expanded by the data gating module, and the storage and distribution of weight data and input stimuli. For input data sent from external memory, the weight and stimulus data, along with their layer number, are sent from the host computer to the accelerator via a 64-bit interface. First, the top-level controller determines whether the input has been encoded. Specifically, the first layer of the convolutional layer is unencoded, while the remaining layers are encoded. For encoded data, the decoding module decodes it and sends it to the data gating module for expansion and zero-jumping operations.

[0033] The decoding process involves the following steps: For a 64-bit data packet, it is first truncated into three groups of "runs" and non-zero combinations. Specifically, bits [63:59] represent the number of consecutive zero values ​​in the first group, [58:43] represent the first non-zero number, [42:38] represent the number of consecutive zero values ​​in the second group, [37:32] represent the second non-zero number, [31:17] represent the number of consecutive zero values ​​in the third group, and [16:1] represent the third non-zero number. Bit 0 is the end-of-data transmission flag; when this bit is 1, it indicates the end of the decoding process. The data gating module receives the three groups of "runs" and non-zero combinations sent by the decoding module. The non-zero numbers are transmitted by the data gating module to the calculation module for subsequent multiplication and accumulation operations, while the "runs" are used to determine the position of the non-zero numbers sent to the calculation module.

[0034] The specific implementation of the zero-skipping operation by data gating is as follows: The calculation module contains multiple calculation units, each performing a single multiplication operation on a weight and an excitation. The data gating system, based on the received "run" (i.e., the number of consecutive zero values), pulls down the enable signals of the corresponding consecutive calculation units, meaning that the unit does not perform any operation. Non-zero numbers received by the data gating system are then transmitted to the next calculation unit after the skipped unit for operation. The calculation module accumulates all multiplication results, performs an activation function operation, and then transmits the result to the encoding module.

[0035] The calculation results sent from the calculation module to the encoding module contain a large number of zero values ​​after the activation function operation. Therefore, they are encoded using run-length encoding. The steps are as follows: For the calculation results output from the calculation module, a parallel-in, serial-out operation is performed, that is, all calculation results are encoded multiple times into 64-bit run-length encoded form and then transmitted to the external memory. Specifically:

[0036] 1) Scan each intermediate result output by the calculation module sequentially. If it is zero, record it as a "run" length. Record the number of consecutive zero values ​​as the run length. The maximum length of the "run" is 32. The "run" is represented by a 5-bit binary number. If there are more than 32 consecutive zero values, only the run length of the first 32 consecutive zero values ​​will be recorded as "11111". For example: 0, 0, 0, 0, 0, 13, after encoding, the run length is 5'b00101; 0, 0...0 (33 zeros), after encoding, the run length is 5'b11111.

[0037] 2) After a run of consecutive zero values ​​is recorded, for the subsequent non-zero value or the 33rd zero value, the number is represented as a 16-bit binary number. That is, if the number is non-zero, it is represented as its own binary representation; if the number is still zero, it is recorded as zero. For example: 0, 0, 0, 0, 0, 13, after encoding, the run is 001010000000000001101; 0, 0...0 (33 zeros), after encoding, the run is 0000...00 (21 zeros).

[0038] 3) Repeat steps 1) and 2) twice to continue counting the number of consecutive zero values ​​and encoding them.

[0039] 4) After three encodings, the new data stream will consist of three groups of 5 + 16 bits = 63 bits of data packets. If this group of data is the last calculation result given by the calculation module, then "1" is placed after this 63-bit data packet to indicate that this group of data is the last bit of all data packets and the encoding is over. Otherwise, if the calculation result of the calculation module is not yet completed, then "1" is placed after the 63-bit array to indicate that the array is not over.

[0040] like Figure 3 The diagram illustrates the encoding method of the encoding module. For the output data processed by the activation function of the calculation module, the first count is of the number of "0" values. The number of consecutive "0" values ​​is recorded as a 5-bit number. The non-zero value following a consecutive "0" value, or the 33rd "0" value, is recorded as its 16-bit binary number. The count of "0" values ​​continues after this value. Each group of consecutive "0" values ​​and non-zero values ​​is a 21-bit group. Every three such 21-bit groups are followed by a 1-bit final judgment bit. That is, 21*3+1=64 bits constitute an encoding array. The last bit indicates whether this group is the last group of data; if so, it is represented as "1"; otherwise, it is represented as "0". For example, 0, 0, 0, 0, 25, 0, 0, 68, 0, 0, 71 is encoded as the binary number "4, 25, 2, 68, 2, 71", where 4, 2, 2 represent the runs of "0", i.e., the number of consecutive zero values. Except for the data initially input to the convolutional layer, all other data is stored in external memory in a run-length encoded compressed format. The accelerator reads the encoded input feature map from the external memory, decompresses it through the decoding module, and then sends it to the computation module via data gating. The computation results from the computation module are compressed by the encoding module and then transferred to the external memory.

[0041] In addition to the input image, the output feature maps of each layer of the CNN model are stored in compressed format in external memory after being encoded using the above method. Reading the feature maps from external memory and returning them to the computation module requires decompressing the data. The specific methods for data decompression are as follows: (1) For the first layer of the CNN model, the top-level controller directly transmits the original input image to the subsequent modules; (2) For convolutional layers or fully connected layers other than the first layer of the CNN model, the top-level controller controls the input feature maps to be decoded by the decoding module. The specific methods for decoding are as follows:

[0042] For a 64-bit data packet, it is first truncated into three groups of "runs" and non-zero combinations, specifically: [63:59] bits represent the number of consecutive zero values ​​in the first group, [58:43] represents the first non-zero number, [42:38] represents the number of consecutive zero values ​​in the second group, [37:32] represents the second non-zero number, [31:17] represents the number of consecutive zero values ​​in the third group, and [16:1] represents the third non-zero number. Bit 0 is the end-of-data transmission flag. When this bit is 1, it indicates that the current decoding has ended.

[0043] The data gating module receives three sets of "runs" and combinations of non-zero numbers sent by the decoding module. The non-zero numbers will be transmitted by the data gating module to the calculation module for subsequent multiplication and accumulation operations, while the "runs" are used to determine the position where the non-zero numbers are sent to the calculation module.

[0044] In summary, the hardware accelerator for CNN models proposed in this invention can effectively reduce memory access requirements, save hardware resources, and improve computing speed. The use of the encoding module ensures low memory access and a small footprint. The overall structure enables the system to achieve higher throughput while saving energy, effectively accelerating neural networks through hardware.

Claims

1. A run-length encoding accelerator for sparse CNN neural network models, characterized in that: include: The top-level controller is used to identify the input and output data types and perform group processing, and transmit the weight data and excitation data to the run-length encoding module; The run-length encoding module is used to compress the result data output by the calculation module and transmit the compressed and encoded calculation results to the external memory; A run-length decoding module is used to decompress the data read from the external memory and transmit the data to the data gate; Data gating, which is used to identify zero values ​​of input excitation data and weight data and skip multiplication and addition operations for zero values; A calculation module, used to perform multiplication and addition operations on the weight data and the excitation data transmitted by the data gating and give the calculation results; The specific method of data compression by the run-length encoding module is as follows: 1) Record consecutive zero values ​​as "runs", the maximum length of a "run" is 32, so the number of consecutive zero values ​​can be represented by a 5-bit binary number; 2) Represent the non-zero number after the zero value as a 16-bit binary number; 3) Repeat the operation of step 1) to continue counting zero-value run-length codes; 4) Repeat step 2) to continue non-zero value encoding; 5) Each group of 5-bit zero value "run" is followed by a group of 16-bit non-zero numbers, which is a 5+16-bit array. A coded data packet has three groups of 21-bit arrays. The last bit of the three arrays is a 1-bit number indicating whether the data packet is the last data packet of all the encodings. If it is, it is represented as "1", otherwise it is "0", that is, each group of coded data packets is a 3*(5+16)+1, i.e., a 64-bit binary array; 6) Except for the data input to the convolutional layer for the first time, the remaining feature map data are stored in the external memory in the format obtained by the method of steps 1) to 5); The run-length decoding module does not decompress the first layer of the CNN model. For convolutional layers other than the first layer of the CNN model, the specific method of decompression by the run-length decoding module is as follows: 1) Every time a group of 64-bit compressed data packets is received, the three groups of "runs" are split respectively, that is, three groups of 5+16 bits and a flag to determine whether the transmission is completed; 2) Record three groups of 5-bit "runs", expand three non-zero numbers, and pass them into the data gate in sequence; 3) Data gating receives the number of bits of consecutive zero values ​​and records them, corresponding to the positions of zero values ​​after the feature map is expanded into a one-dimensional array or a multi-dimensional vector, skipping the multiplication and addition operations for the pixels at these positions, while non-zero values ​​are written into the calculation module for calculation.

2. The acceleration method of the run-length encoding accelerator for the sparse CNN neural network model according to claim 1 is characterized in that: The top-level controller processes the input image, transmits the data to be compressed to the run-length encoding module, and transmits the received data to be decompressed to the run-length decoding module; the run-length encoding module compresses the data transmitted by the top-level controller, and stores the compressed result data in the external memory; The run-length decoding module decompresses the input excitation data and weight data written from the external memory to the calculation module, and then transmits the obtained data to the data gating, which records the zero values ​​in the decompressed input excitation data and weight data, and skips the subsequent multiplication and addition operations for zero values.

3. The acceleration method according to claim 2, characterized in that: The specific method of data compression by the run-length encoding module is as follows: 1) Record consecutive zero values ​​as "runs", the maximum length of a "run" is 32, so the number of consecutive zero values ​​can be represented by a 5-bit binary number; 2) Represent the non-zero number after the zero value as a 16-bit binary number; 3) Repeat the operation of step 1) to continue counting zero-value run-length codes; 4) Repeat step 2) to continue non-zero value encoding; 5) Each group of 5-bit zero value "run" is followed by a group of 16-bit non-zero numbers, which is a 5+16-bit array. A coded data packet has three groups of 21-bit arrays. The last bit of the three arrays is a 1-bit number indicating whether the data packet is the last data packet of all the encodings. If it is, it is represented as "1", otherwise it is "0", that is, each group of coded data packets is a 3*(5+16)+1, i.e., a 64-bit binary array; 6) Except for the data input to the convolutional layer for the first time, the remaining feature map data are stored in the external memory in the format obtained by the method of steps 1) to 5).

4. The acceleration method according to claim 3, characterized in that: The run-length decoding module does not decompress the first layer of the CNN model. For convolutional layers other than the first layer of the CNN model, the specific method of decompression by the run-length decoding module is as follows: 1) Every time a group of 64-bit compressed data packets is received, the three groups of "runs" are split respectively, that is, three groups of 5+16 bits and a flag to determine whether the transmission is completed; 2) Record three groups of 5-bit "runs", expand three non-zero numbers, and pass them into the data gate in sequence; 3) Data gating receives the number of bits of consecutive zero values ​​and records them, corresponding to the positions of zero values ​​after the feature map is expanded into a one-dimensional array or a multi-dimensional vector, skipping the multiplication and addition operations for the pixels at these positions, while non-zero values ​​are written into the calculation module for calculation.

Images