Neural network weight discretization method and system

Abstract

The present invention relates to a neural network weight discretization method and system. The method includes: obtaining the weight value range and the number of discrete weight states; obtaining the weight state of the last time step and the weight increment of the current time step; Time step weight increment, use direction function to obtain state transition direction; according to the last time step weight state, the current time step weight increment, the state transition direction, the weight value range, the discrete The number of weight states, get the current time step weight state. The invention ensures that the weight value is always constrained in the same discrete state space, without storing additional virtual continuous state implicit weights, and greatly reduces the consumption of storage space and reduces the complexity of calculation on the premise of ensuring the calculation performance of the neural network Spend.

Description

technical field [0001] The invention relates to the technical field of neural networks, in particular to a method and system for discretizing neural network weights. Background technique [0002] In the past few years, large data sets, different learning models, and GPUs (General Purpose Graphic Processing Units) have made deep neural networks more popular in artificial intelligence fields such as computer vision, speech recognition, natural language processing, and human-computer Go games. One result after another has been achieved. However, behind these striking results, there is also a huge overhead of hardware resources, training time overhead and power consumption required to calculate them. Therefore, GPU-based deep learning systems are difficult to embed in portable devices. However, many solutions with high power consumption utility often require a large loss in performance. Therefore, there is a trade-off between performance and computational complexity. [0003...

AI Technical Summary

Problems solved by technology

[0005] Based on this, it is necessary to provide a neural network weight discretization method and system for how to balance the performance and computational complexity of the neural network. The method includes:

Images