Sparse binary neural network hardware accelerator for gesture recognition

Abstract

The present invention belongs to the field of integrated circuit technology, and specifically is a sparse binary neural network hardware accelerator for gesture recognition. The accelerator of the present invention includes: an input cache module, a weight cache module, a data transmission on-chip network, 32 convolution calculation cores, a hierarchical accumulation tree, a subsequent processing unit, an HDMI interface, a UART interface, and a value prediction and sparse activation map compression / decompression module. The accelerator accelerates gesture recognition of sparse binary networks. The two-level value prediction method can skip the repeated calculations existing in the binary network, thereby accelerating the calculation cycle and reducing power consumption; the channel-level sparse activation compression and decompression method is used to reduce the required data storage volume; the present invention has the characteristics of low power consumption and low latency, effectively reducing hardware resources and improving the energy efficiency of the hardware.

Description

Technical Field

[0001] The present invention belongs to the technical field of integrated circuits, and in particular relates to a sparse binary neural network hardware accelerator for gesture recognition. Background Art

[0002] Natural, barrier-free, highly effective, and contactless new intelligent human-computer interaction systems are becoming an inevitable trend in information development. Gestures, as one of the most important channels in human-computer interaction, have the advantages of wide application, simple operation, and high frequency of use. However, currently, gesture recognition mostly uses convolutional neural networks. However, convolutional neural network models are generally parameter-intensive and computationally intensive, resulting in high power consumption and high latency in hardware implementation. Convolutional neural network-based gesture recognition has high resource requirements and correspondingly high power consumption, making it unsuitable for deployment in mobile devices, IoT, and wearable devices. Binary networks, similar to convolutional neural networks, use 1-bit inputs and weights. Using XNOR-PopCount instead of the original convolutional neural network's multiplication-accumulation operations reduces data and computational complexity, making them more suitable for low-power gesture recognition hardware. Given that the first layer of a binary network typically uses multiplication-accumulation due to RGB input, using sparse gesture edges ensures that the first layer also uses XNOR-PopCount. Furthermore, sparse gesture edges result in a large number of identical vectors in the binary network's activation image. These vectors, composed of 1-bit activation data from all channels, are present in large quantities in every layer of the binary network. The calculation results of these vectors can be predicted by software, resulting in a large amount of redundant computation, unnecessary power consumption, and wasted computing cycles. Therefore, designing an energy-efficient sparse binary neural network hardware accelerator for gesture recognition is urgently needed to achieve ultra-efficient gesture recognition hardware. Summary of the Invention

[0003] In order to overcome the shortcomings of the existing technology, the purpose of the present invention is to propose a high-energy-efficiency sparse binary neural network hardware accelerator for gesture recognition with low resource consumption, low power consumption, and fast computing speed, so as to accelerate sparse binary network calculations and improve hardware energy efficiency.

[0004] The high-energy-efficiency sparse binary neural network hardware accelerator for gesture recognition provided by the present invention comprises: an input cache module, a weight cache module, a data transmission on-chip network, 32 convolution calculation cores, a hierarchical accumulation tree, a subsequent processing unit, an HDMI interface, a UART interface, and a value prediction and sparse activation map compression / decompression module; wherein:

[0005] The UART interface is used to receive weight data transmitted by the host and transmit it to the weight cache module. The weight cache module has a size of 15.83KB and is responsible for storing the weight data required by the gesture recognition algorithm. After completing the full connection calculation, the subsequent processing unit sends the recognition result to the host through the UART interface;

[0006] The HDMI interface is used to transmit the edge gesture image to the input buffer module for storage; when the complete image transmission is completed, the input data and weight data transmission process is started; in addition, the input buffer module also temporarily stores the restored data of the value prediction and coefficient activation image compression / decompression module;

[0007] After the calculation is started, the data transmission on-chip network transmits the input data of the input cache module and the weight data of the weight cache module to the computing unit array in the convolution calculation core according to the convolution mapping algorithm; wherein the data transmission on-chip network consists of two parts: an input transmission path and a weight transmission path, and each path supports a unicast-multicast data transmission process;

[0008] After receiving the input data and weight data, the convolution calculation core makes a judgment based on its routing and saves the corresponding data; the convolution calculation core is composed of 16 computing units, each of which can perform a 3x3 binary network convolution operation, namely a 3x3 XNOR-PopCount; 32 convolution calculation cores can perform up to 4608 (32×16×3×3) XNOR-PopCount operations in each cycle; the results calculated by each computing unit are transmitted to the hierarchical accumulation tree to calculate the final result of the convolution operation;

[0009] The hierarchical accumulation tree, after receiving the results of each computing unit, performs accumulation calculation on these results according to the convolution mapping algorithm; the result of the first layer of convolution is the output of each computing unit, and the number of outputs is 32x16; the result of the second layer of convolution is the accumulation result of all the computing units of each convolution calculation kernel, and the number of outputs is 32; the result of the third layer of convolution is the further accumulation of the accumulation results of every two convolution calculation kernels, and the number of outputs is 16; the result of the last fully connected layer is the accumulation result of the accumulation results of all the convolution calculation kernels, and the number of outputs is 1; the output result of each layer can be directly transmitted to the subsequent processing module;

[0010] The subsequent processing unit supports absolute value, pooling, comparison, activation and normalization operations; if the subsequent processing unit receives the convolution calculation results of each layer of the hierarchical accumulator tree, the subsequent processing unit performs absolute value, pooling and comparison operations in sequence to generate a binary activation vector and transmits it to the value prediction and coefficient activation map compression / decompression module. The activation vector is a combination of pixels at the same position of the output activation images of multiple channels; if the subsequent processing unit receives the fully connected layer result of the hierarchical accumulator tree module, it performs activation and normalization operations to obtain the final gesture classification result, and transmits it to the host through the UART interface;

[0011] The subsequent processing unit includes a pre-storage unit for storing the sparse activation vector predicted in advance by the software; the pre-storage unit transmits the corresponding prediction result to the value prediction and coefficient activation map compression / decompression module while transmitting the binarized activation image to the value prediction and coefficient activation map compression / decompression module;

[0012] The value prediction and sparse activation map compression / decompression module is composed of an activation image compression unit, a compressed data storage unit, and a value prediction and decompression unit; the compressed data storage unit includes a valid activation vector register file, a prior distribution register file, and a register file for the first valid activation vector address in each row:

[0013] The activation image compression unit is composed of a prior distribution generator, a valid activation vector counter and a sparse activation vector filter; after the sparse activation vector filter receives the binarized activation vector and the sparse activation vector sent by the subsequent processing unit, the two vectors are compared. If they are equal, it means that the activation vector is a sparse activation vector, and a value of 0 is sent to the prior distribution generator and the valid activation vector counter; if not, it means that the activation vector is a valid activation vector, and a value of 1 is sent to the prior distribution generator and the valid activation vector counter, and the valid activation vector is saved to the valid activation vector register stack of the compressed data storage unit; the valid activation vector counter accumulates the data transmitted from the sparse activation vector filter, and saves the accumulated result to the first valid activation vector address register stack of each row of the compressed data storage unit at the end of a row of data transmission; the prior distribution generator splices the data transmitted by the sparse activation vector filter, and sends the spliced data to the prior distribution register stack at the end of a row of data transmission;

[0014] The value prediction and decompression unit performs the value prediction and decompression process after the convolution of the current layer is completed; the value prediction and decompression unit is composed of a value prediction unit, an address decoder and a data restorer; when the value prediction unit performs the calculation of the next layer, it first takes out 4 rows of prior distribution from the prior distribution register stack, slides with a 4x4 convolution kernel size step of 2, and takes out a 4x4 prior distribution block each time; first judge whether the 4x4 prior distribution block is all 0 values, if so, the output result must be the sparse activation vector of the next layer, and all calculations are skipped; if not, it is divided into 4 3x3 prior distribution blocks, and it is judged whether each 3x3 prior distribution block is all 0: if so, the result of the convolution calculation can be predicted, and the value is also saved in the pre-storage unit of the subsequent processing module , in preparation for subsequent pooling operations; if not, it means that the calculation result is unpredictable, and the 3x3 prior distribution block is sent to the address decoder to calculate the effective activation vector address; the address decoder reads the address data in the first effective activation vector address register stack in each row, and generates 9 addresses according to the 3x3 prior distribution block; wherein the 0 value in the 3x3 prior distribution block represents a sparse activation vector, and the address points to the sparse activation vector, and the 1 value in the 3x3 prior distribution block represents a valid activation vector, and the address points to the valid activation vector in the valid activation vector register stack; the data restorer reads the effective activation vector and the sparse activation vector respectively according to the 9 addresses, restores them to the input data involved in the calculation, and saves the restored input data to the input cache module.

[0015] Compared with the existing technology, the technical features and advantages of the present invention are mainly as follows:

[0016] (1) The present invention introduces new sparsity into the activation image of the binary network. By using sparse gesture edges as the input of the binary network, a large number of identical vectors can be introduced into the activation image of each layer of the binary network. The calculation results of these vectors at each layer can be predicted by software. Therefore, the binary network can compress the activation image and skip a large number of redundant convolution calculations.

[0017] (2) The activation image encoding unit can losslessly compress the sparse activation image of each layer of the binary network and transmit the original activation image to the accelerator in the form of an activation vector. The accelerator can save the activation image as a valid vector, label value, and number of valid vectors based on the known sparse vector, thereby achieving a higher compression ratio, reducing the amount of data stored, and reducing the number of data reads and writes, thereby optimizing hardware power consumption;

[0018] (3) The value prediction unit predicts the binary network calculation process on the prior distribution map, and can skip redundant calculations based on the software prediction results, thereby reducing the binary network calculation power consumption and delay, and further optimizing the binary network energy efficiency;

[0019] (4) The accelerator uses two data transmission on-chip networks, supports unicast and multicast data transmission, and can quickly and efficiently send data to all computing units;

[0020] (5) The hierarchical accumulation tree accumulates downward layer by layer to reuse some computing resources, realize partial sharing of resources, and reduce resource consumption;

[0021] (6) The sparse binary neural network hardware accelerator skips the repeated calculations in the sparse binary network convolution calculations through value prediction technology and data compression calculations, compresses the output data to reduce the resource consumption of data storage, and ultimately achieves an improvement in computing speed, reduces power consumption, and optimizes energy efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Energy-efficient sparse binary neural network hardware accelerator.

[0023] Figure 2 It is a sparse edge binary network model.

[0024] Figure 3 On-chip networks and their transmission methods.

[0025] Figure 4 The process of generating sparse activation vectors and effective activation vectors in convolution calculations.

[0026] Figure 5 Activates the image compression unit.

[0027] Figure 6 The compressed data composition.

[0028] Figure 7 The value prediction unit generates two skipping technology tables: Group-level and Filter-level Skipping tables.

[0029] Figure 8 Value prediction skips the entire process of convolution calculation.

[0030] Figure 9 Compressed data decompression circuit.

[0031] Figure 10 Decompressed data.

[0032] Figure 11 Compression ratio data.

[0033] Figure 12 Speed up comparison data. DETAILED DESCRIPTION

[0034] The present invention provides a high-energy-efficiency sparse binary neural network hardware accelerator for the field of gesture recognition, such as Figure 1 As shown, the structure includes value prediction and sparse activation map compression / decompression modules, input cache module, weight cache module, data transmission on-chip network, 32 convolution calculation cores, hierarchical accumulation tree, subsequent processing unit, HDMI interface, UART interface, etc.; among them:

[0035] The UART interface receives the weight data transmitted by the host and transmits it to the weight cache module. The buffer size is 15.83KB and is responsible for storing the weight data required by the gesture recognition algorithm. The subsequent processing unit will receive the weight data through the UART after completing the full connection calculation.

[0036] The interface sends the identification results to the host;

[0037] like Figure 2 As shown, the sparse binary network structure consists of three binarized convolutional layers and one binarized fully connected layer. All convolution calculations are performed using XNOR-PopCount, and the weights and input data involved in the convolution calculations are all 1-bit. Each convolutional layer is followed by an absolute value activation (abs), an average pooling layer (avgpooling), a Pre-ReLU activation, a normalization layer (BN), and a binary hard tanh operation. During forward inference, the Pre-ReLU activation, the normalization layer (BN), and the binary hard tanh operation are converted into comparison operations. The input to the binary network is a 64x64 black and white gesture image, which provides sparsity for each layer of the binary network.

[0038] The HDMI interface transmits the edge gesture image to the input buffer module for storage. When the complete image transmission is completed, the input data and weight data transmission process is started. In addition, the input buffer module also temporarily stores the restored data of the value prediction and coefficient activation image compression / decompression module.

[0039] After the calculation is started, the data transmission on-chip network transmits the input data of the input cache module and the weight data of the weight cache module to the computing unit array in the convolution calculation core according to the convolution mapping algorithm. The data transmission on-chip network consists of two parts: the input transmission path and the weight transmission path. Each path supports the single-multicast data transmission process. Figure 3 As shown, the on-chip network includes data transmission in two directions, where Y_BUS refers to data transmission in the Y-axis direction and X_BUS refers to data transmission in the X-axis direction. and Represents the unicast transmission path, and Represents the multicast transmission path, thereby transmitting data to each computing unit.

[0040] After receiving the input data and weight data, the convolution calculation core makes a judgment based on its routing and saves the corresponding data in; the convolution calculation core is composed of 16 computing units, each of which can perform a 3x3 binary network convolution operation, namely 3x3 XNOR-PopCount; Figure 4 As shown in the figure, (a) shows the convolution operation of the first layer, which is divided into the sparse vector generation process and the effective vector generation process; (b) shows the convolution operation of the intermediate layer, as well as the sparse activation vector generation process and the effective activation vector generation process of each layer. The 32 convolution computation cores can perform up to 4608 (32×16×3×3) XNOR-PopCount operations in each cycle; the results of each computation unit are transmitted to the hierarchical accumulation tree to calculate the final result of the convolution operation.

[0041] After the hierarchical accumulation tree module receives the results of each calculation unit, it accumulates these results according to the convolution mapping algorithm, such as Figure 1 As shown in the figure; the result of the first layer of convolution is the output of each computing unit, and the number of outputs is 32x16; the result of the second layer of convolution is the cumulative result of all computing units of each convolution calculation kernel, and the number of outputs is 32; the result of the third layer of convolution is the further accumulation of the cumulative results of every two convolution calculation kernels, and the number of outputs is 16; the result of the last fully connected layer is the cumulative result of the cumulative results of all convolution calculation kernels, and the number of outputs is 1; the output result of each layer can be directly transmitted to the subsequent processing module.

[0042] The subsequent processing unit supports absolute value, pooling, comparison, activation and normalization operations; if the subsequent processing unit receives the convolution calculation results of each layer of the hierarchical accumulator tree, the subsequent processing unit performs absolute value, pooling and comparison operations in sequence to generate a binary activation vector and transmit it to the value prediction and coefficient activation map compression / decompression module. The activation vector is a combination of pixels at the same position of the output activation image of multiple channels; if the subsequent processing unit receives the fully connected layer result of the hierarchical accumulator tree module, it performs activation and normalization operations to obtain the final gesture classification result and transmits it to the host through the UART interface; the subsequent processing unit includes a pre-storage unit for storing sparse activation vectors predicted in advance by the software; the pre-storage unit transmits the corresponding prediction result to the value prediction and coefficient activation map compression / decompression module while transmitting the binary activation image to the value prediction and coefficient activation map compression / decompression module.

[0043] The value prediction and sparse activation map compression / decompression module is composed of an activation image compression unit, a compressed data storage unit, and a value prediction and decompression unit; the activation image compression unit is composed of a prior distribution generator, a valid activation vector counter, and a sparse activation vector filter; Figure 5 As shown, after the sparse activation vector filter receives the binarized activation vector and the sparse activation vector sent by the subsequent processing unit, it compares the two vectors. If they are equal, it means that the activation vector is a sparse activation vector, and a value of 0 is sent to the prior distribution generator and the effective activation vector counter; if not, it means that the activation vector is a valid activation vector, and a value of 1 is sent to the prior distribution generator and the effective activation vector counter, and the effective activation vector is saved to the effective activation vector register stack of the compressed data storage unit; the effective activation vector counter accumulates the data transmitted from the sparse activation vector filter, and saves the accumulated result to the first effective activation vector address register stack of each row of the compressed data storage unit at the end of a row of data transmission; the prior distribution generator splices the data transmitted by the sparse activation vector filter, and sends the spliced data to the prior distribution register stack at the end of a row of data transmission; the final effective activation image is composed of the first effective activation vector address of each row, the prior distribution and the effective activation vector. The specific encoding process is as follows Figure 6 As shown in the figure, the original activation image can be regarded as a two-dimensional rectangle represented by the effective activation vector (EAV) and the sparse vector. Then the vector can be regarded as composed of the effective activation vector value and its location. Using CSR compression coding, CSR divides the sparse image into effective value, first effective value address and column coordinate. The CSR compression coding deformation is to divide the original activation image into effective activation vector, first effective vector address (EAVA) and coding, and use coding instead of column coordinates to facilitate the convolution calculation process. Figure 6Taking the original activation image in [1] as an example, the first valid value address is the number of EAVs preceding the first EAV on the leftmost side of each row of raw data; therefore, the first row has 0, the second row has 3 EAVs preceding the first EAV, and so on. Labels represent the EAVs and sparse vectors in the original activation image. Each label represents a row of the original activation image, with a value of 0 representing a sparse vector and a value of 1 representing an EAV. The valid activation vector values are the set of all EAVs, arranged from left to right and top to bottom according to the original activation image. The column coordinates are replaced by the prior distribution map (PDM), and the address of the first EAV is saved. When entering the next BConv layer, the compressed sparse activation map (SAM) is decompressed to provide input data for the convolution computation. During the decompression operation, the efficient PDM selected by the value prediction represents a non-repeated convolution computation. The data decompression operation reads the EAV data based on the efficient PDM block and the first EAV address, then recombines the input data with the predicted SAM. The compressed data storage unit includes a valid vector register stack, an output tag register stack, and a first valid vector address register stack. The valid vector register stack receives valid vectors sent by the activation image encoding unit and stores them in sequence. The output tag register stack receives tag values sent by the activation image encoding unit and stores them in sequence. The first valid vector address register stack receives the number of valid vectors sent by the activation image encoding unit and stores them in sequence. Compression of sparse data can effectively reduce the storage resources required for the original activation image. In addition, calculations performed on the encoded data can skip some repeated calculations, thereby optimizing computational speed.

[0044] The value prediction and decompression unit will perform the value prediction and decompression process after the convolution of the current layer is completed; the value prediction and decompression unit is composed of a value prediction unit, an address decoder and a data restorer; Figure 7 As shown, when the value prediction unit performs the calculation of the next layer, it will first take out 4 rows of prior distribution from the prior distribution register stack, slide with a 4x4 convolution kernel size step of 2, and take out a 4x4 prior distribution block each time; first determine whether the 4x4 prior distribution block is all 0 values, and generate a Group-level Skipping table to indicate whether to skip all calculations: if so, the output result must be the sparse activation vector of the next layer, skipping all calculations; if not, it is divided into 4 3x3 prior distribution blocks, and each 3x3 prior distribution block is determined to be all 0. A Filter-level Skipping table is generated to indicate whether some calculations can be skipped: if so, the result of the convolution calculation can be predicted, and the value is also saved in the pre-storage unit of the subsequent processing module for subsequent pooling operations; if not, it means that the calculation result is unpredictable, and the 3x3 prior distribution block is sent to the address decoder for calculating the effective activation vector address; as shown Figure 8 As shown in the table, a value of 0 in the Group-level Skipping table indicates that all subsequent operations are skipped, including decompression, convolution calculation, accumulation, and subsequent processing, and the output result is directly the sparse activation vector of the next layer; a value of 0 in the Filter-level Skipping table indicates that there is a calculation that can be skipped in the four 3x3 convolution calculations. The result of the convolution calculation can be predicted, but subsequent operations are still required to obtain the output of a valid activation vector. Figure 9 As shown, the address decoder reads the address data in the first valid activation vector address register file in each row and generates 9 addresses based on the 3x3 prior distribution block; the 0 value in the 3x3 prior distribution block represents a sparse activation vector, and the address points to the sparse activation vector, and the 1 value in the 3x3 prior distribution block represents a valid activation vector, and the address points to the valid activation vector in the valid activation vector register file; as shown Figure 10 As shown, the data restorer reads the effective activation vector and the sparse activation vector according to 9 addresses, restores them to the input data involved in the calculation, and saves the restored input data to the input cache module.

[0045] In order to verify the optimization effect of the present invention, four common gesture datasets: RGBDASL, RGBDGES, RGBASL, and GRAYASL were used to conduct experiments on the uncompressed binary neural network accelerator DBA and the energy-efficient sparse binary neural network hardware accelerator SBA of the present invention. In the experiments, the largest edge gesture image LGE and the smallest edge gesture image SME in each dataset were used to represent the worst compression effect (worst acceleration effect) and the best compression effect (best acceleration effect). Figure 11 As shown in , the accelerator achieves a compression ratio of more than 1.72-3.45 times in four commonly used gesture datasets; Figure 12 As shown in the figure, the SBA acceleration effect reaches 1.03-1.83 times. This invention speeds up the calculation speed, reduces the calculation cycle, saves computing resources, and further optimizes the power consumption, reducing the power consumption from 0.9 to 0.8W, and improving the energy efficiency by 1.35-1.69 times.

Claims

1. A sparse binary neural network hardware accelerator for gesture recognition, characterized in that: It includes an input cache module, a weight cache module, a data transmission on-chip network, 32 convolution computing cores, a hierarchical accumulation tree module, a subsequent processing unit, an HDMI interface, a UART interface, a value prediction and sparse activation map compression / decompression module; among which: The UART interface receives the weight data transmitted by the host and transmits it to the weight cache module, which is responsible for storing the weight data required by the gesture recognition algorithm; the UART interface also receives the recognition result after the subsequent processing unit completes the full connection calculation and sends it to the host; The HDMI interface is used to transmit the edge gesture image to the input buffer module for storage; when the complete image transmission is completed, the input data and weight data transmission process is started; the input buffer module is also used to temporarily store the restored data of the value prediction and sparse activation map compression / decompression module; After the calculation is started, the data transmission on-chip network transmits the input data of the input cache module and the weight data of the weight cache module to the computing unit array in the convolution calculation core according to the convolution mapping algorithm; wherein the data transmission on-chip network consists of two parts: an input transmission path and a weight transmission path, and each path supports a unicast-multicast data transmission process; After receiving the input data and weight data, the convolution calculation core makes a judgment based on its routing and saves the corresponding data; the convolution calculation core is composed of 16 computing units, each of which performs a 3x3 binary network convolution operation, namely a 3x3 XNOR-PopCount; 32 convolution calculation cores perform 32×16×3×3 XNOR-PopCount operations in each cycle; the results calculated by each computing unit are transmitted to the hierarchical accumulation tree module to calculate the final result of the convolution operation; The hierarchical accumulation tree module, after receiving the results of each computing unit, performs accumulation calculation on these results according to the convolution mapping algorithm; the result of the first layer of convolution is the output of each computing unit, and the number of outputs is 32x16; the result of the second layer of convolution is the accumulation result of all the computing units of each convolution calculation kernel, and the number of outputs is 32; the result of the third layer of convolution is the further accumulation of the accumulation results of every two convolution calculation kernels, and the number of outputs is 16; the result of the last fully connected layer is the accumulation result of the accumulation results of all the convolution calculation kernels, and the number of outputs is 1; the output result of each layer is directly transmitted to the subsequent processing unit; The value prediction and sparse activation map compression / decompression module is composed of an activation image compression unit, a compressed data storage unit, and a value prediction and decompression unit; the compressed data storage unit includes a valid activation vector register file, a prior distribution register file, and a register file for the first valid activation vector address in each row; The activation image compression unit is composed of a prior distribution generator, a valid activation vector counter and a sparse activation vector filter; The value prediction and decompression unit consists of a value prediction unit, an address decoder and a data restorer.

2. The sparse binary neural network hardware accelerator for gesture recognition according to claim 1, characterized in that: The subsequent processing unit supports absolute value, pooling, comparison, activation and normalization operations; if the subsequent processing unit receives the convolution calculation results of each layer of the hierarchical accumulator tree module, the subsequent processing unit performs absolute value, pooling and comparison operations in sequence to generate a binary activation vector and transmits it to the value prediction and sparse activation map compression / decompression module. The activation vector is a combination of pixels at the same position of the output activation images of multiple channels; If the subsequent processing unit receives the fully connected layer result of the hierarchical cumulative tree module, it performs activation and normalization operations to obtain the final gesture classification result, and transmits it to the host through the UART interface; The subsequent processing unit includes a pre-storage unit for storing the sparse activation vector predicted in advance by the software; the pre-storage unit transmits the corresponding prediction result to the value prediction and sparse activation map compression / decompression module while transmitting the binarized activation image to the value prediction and sparse activation map compression / decompression module.

3. The sparse binary neural network hardware accelerator for gesture recognition according to claim 2, characterized in that: After receiving the binarized activation vector and the sparse activation vector sent by the subsequent processing unit, the sparse activation vector filter compares the two vectors. If they are equal, it means that the activation vector is a sparse activation vector, and a value of 0 is sent to the prior distribution generator and the valid activation vector counter; if not, it means that the activation vector is a valid activation vector, and a value of 1 is sent to the prior distribution generator and the valid activation vector counter, and the valid activation vector is saved to the valid activation vector register stack of the compressed data storage unit; the valid activation vector counter accumulates the data transmitted from the sparse activation vector filter, and saves the accumulated result to the first valid activation vector address register stack of each row of the compressed data storage unit at the end of a row of data transmission; the prior distribution generator splices the data transmitted by the sparse activation vector filter, and sends the spliced data to the prior distribution register stack at the end of a row of data transmission; After the convolution of the current layer is completed, the value prediction and decompression unit performs the value prediction and decompression process; when performing the calculation of the next layer, the value prediction unit first takes out 4 rows of prior distribution from the prior distribution register stack, slides with a 4x4 convolution kernel size step of 2, and takes out a 4x4 prior distribution block each time; first judge whether the 4x4 prior distribution block is all 0 values, if so, the output result must be the sparse activation vector of the next layer, and all calculations are skipped; if not, it is divided into 4 3x3 prior distribution blocks, and it is judged whether each 3x3 prior distribution block is all 0: if so, the result of the convolution calculation can be predicted, and the value is also saved in the pre-storage unit of the subsequent processing unit for subsequent pooling operation; if not, it indicates The calculation result is unpredictable, and the 3x3 prior distribution block is sent to the address decoder to calculate the effective activation vector address; the address decoder reads the address data in the first effective activation vector address register stack in each row, and generates 9 addresses based on the 3x3 prior distribution block; the 0 value in the 3x3 prior distribution block represents a sparse activation vector, and the address points to the sparse activation vector, and the 1 value in the 3x3 prior distribution block represents a valid activation vector, and the address points to the valid activation vector in the valid activation vector register stack; the data restorer reads the effective activation vector and the sparse activation vector respectively according to the 9 addresses, restores them to the input data involved in the calculation, and saves the restored input data to the input cache module.

Images