Neural network weight discretization method and system

Abstract

The invention relates to a neural network weight discretization method and system. The method comprises the following steps: obtaining a weight value range and a number of discrete weight states; acquiring a previous time step weight state and a current time step weight increment; obtaining a state transition direction by using a direction function according to the weight increment of the currenttime step; obtaining a current time step weight state according to the previous time step weight state, the current time step weight increment, the state transition direction, the weight value range,and the number of discrete weight states. The invention ensures that the weight value is always constrained in the same discrete state space without storing additional virtual continuous state implicit weights, and greatly reduces the consumption of storage space and the computational complexity under the premise of ensuring the computational performance of the neural network.

Description

technical field [0001] The invention relates to the technical field of neural networks, in particular to a method and system for discretizing neural network weights. Background technique [0002] In the past few years, large data sets, different learning models, and GPUs (General Purpose Graphic Processing Units) have made deep neural networks more popular in artificial intelligence fields such as computer vision, speech recognition, natural language processing, and human-computer Go games. One result after another has been achieved. However, behind these striking results, there is also a huge overhead of hardware resources, training time overhead and power consumption required to calculate them. Therefore, GPU-based deep learning systems are difficult to embed in portable devices. However, many solutions with high power consumption utility often require a large loss in performance. Therefore, there is a trade-off between performance and computational complexity. [0003...

AI Technical Summary

Problems solved by technology

[0005] Based on this, it is necessary to provide a neural network weight discretization method and system for how to balance the performance and computational complexity of the neural network. The method includes:

Images