Neural network optimization method and related equipment

Abstract

The embodiment of the invention discloses a neural network optimization method and related equipment, which can be applied to the field of computer vision (such as image super-resolution reconstruction) and the like in the field of artificial intelligence, and comprises the following steps: binarizing a weight matrix / feature representation (or called a feature map and an activation value) of a neural network through a new quantization model, wherein the first quantization model is used for obtaining a second weight matrix of the mth layer of the neural network according to the m first weight matrixes of the first layer to the mth layer of the neural network, and the second quantization model is used for obtaining a second feature representation of the mth layer of the neural network according to the m first feature representations of the first layer to the mth layer, according to the optimization mode, the value of the weight matrix / feature representation of each layer is not only related to itself, but also related to the weight matrix / feature representation of other layers, the quantization error is reduced, the training and use of the neural network are more efficient, and meanwhile, compared with the existing binary neural network, the image information processing precision is improved.

Description

technical field [0001] The present application relates to the field of machine learning, in particular to a neural network optimization method and related equipment. Background technique [0002] Neural network is a neural network that simulates the human brain in order to achieve artificial intelligence-like machine learning technology. It is the basis of deep learning. Current neural networks generally use floating-point calculations, which require a large amount of storage space and calculations, which seriously hinders Applications on edge devices (such as cameras) and end-side devices (such as mobile phones). Due to its potential advantages of high model compression rate and fast calculation speed, binary neural network has become a popular research direction of deep learning in recent years. [0003] The binary neural network (BNN) is based on the neural network, and binarizes each weight in the weight matrix of the neural network to 1 or -1. Through the binarization...

AI Technical Summary

Problems solved by technology

[0004] At present, the binarization method of the neural network is to binarize the weight matrix of a certain layer alone, that is to say, this binarization method only quantizes the weight matrix of each layer of the neural network separately, so that large quantization error

Images