Binary neural network compression method based on weight sensitivity

Abstract

The invention discloses a binary neural network compression method based on weight sensitivity. The method comprises the following steps of training by using a binary neural network to acquire weightmatrixes and original accuracy; assessing the sensitivity of any weight matrix; presetting a sensitivity threshold, and dividing a sensitive set and a non-sensitive set of the weight matrixes; assessing the sensitivity of the non-sensitive set of the weight matrixes; adjusting the sensitivity threshold to acquire the best non-sensitive set of the weight matrixes, wherein the sensitivity of the best non-sensitive set is equal to a preset maximum accuracy loss value; and storing the best non-sensitive set into a novel storage or a traditional storage using a nearly threshold / sub-threshold voltage technology. According to the scheme, the method provided by the invention has the advantages of low power consumption, high recognition rate, good universality and low cost, and has the wide marketprospect in the field of hardware compression technologies.

Description

technical field [0001] The invention relates to the technical field of hardware compression, in particular to a binary neural network hardware compression method based on weight sensitivity. Background technique [0002] At present, in order to reduce the resource overhead and power consumption required for neural network hardware implementation, the mainstream methods adopted include hardware architecture optimization, neural network compression, binary neural network, etc. Among them, hardware architecture optimization is to design a more efficient way to implement neural networks at the hardware level, reduce the memory resources occupied by data, and reduce the redundancy of data in memory reading and writing and computing methods, so as to achieve the purpose of reducing resource overhead and power consumption. The neural network compression is to realize the compression of the network model by reducing the number of weights and quantization bits in the neural network, ...

AI Technical Summary

Problems solved by technology

Although hardware architecture optimization and neural network compression can save storage and power consumption to a certain extent, they are not as simple as binary neural networks.

[0006] Second, the recognition accuracy of the binary neural network is low; among various binary neural networks, in terms of classification tasks, both BinaryConnect and BNN can only complete the classification tasks well for some smaller data sets, such as Handwritten digit set MNIST, common object recognition data set CIFAR, real-world street house number recognition data set SVHN, etc., when replaced with a very large data set such as ImageNet, BinaryConnect and BNN will seriously reduce the recognition accuracy

[0007] Third, the traditional compression method uses a storage device such as 6T SRAM. This storage device causes relatively large hardware resource overhead and power consumption, which limits the scale of the neural network implemented by the chip. Although the binary neural network has good performance on traditional hardware, But its fault tolerance is not fully utilized

However, near-threshold / sub-threshold voltage technologies still face some challenges, with uncertainty or variability issues

At low supply voltages, circuits are susceptible to disturbances that can cause errors in the weights stored in traditional memory devices

If the entire traditional memory device adopts near-threshold / sub-threshold voltage technology, it will greatly affect the recognition accuracy of the neural network

Images