Hybrid neural network training method, system and device and storage medium

Abstract

The invention discloses a hybrid neural network training method, system and device and a storage medium, and belongs to the field of machine learning algorithms. In the training scheme provided by the invention, a gradient calculation method and an error transmission method of a gradient descent-based training method are used, and a self-adaptive variable-step-size Manhattan rule-based training method in the training process is adopted to update the network weight, so that the network prediction accuracy and the convergence speed of the network are considered; compared with a Manhattan-based neural network training algorithm, the method can achieve a higher convergence speed, and compared with a training algorithm based on stochastic gradient descent, the method can achieve a higher accuracy rate. A hybrid neural network training mode provided by the invention is low in complexity and high in convergence speed, can ensure the network prediction accuracy of a trained neural network, and can be more suitable for online training of a neural network calculation circuit based on a resistive random access memory ReRAM.

Description

technical field [0001] The invention belongs to the field of machine learning algorithms, and in particular relates to a hybrid neural network training method, system, equipment and storage medium. Background technique [0002] A neural network computing circuit based on a new type of non-volatile memory device, such as a neural network computing circuit based on a resistive random access memory ReRAM, due to the use of a memory-computing integrated architecture based on a resistive random access memory ReRAM array, compared to a neural network computing circuit based on a resistive random access memory ReRAM array. The traditional CMOS (Complementary Metal-Oxide-Semiconductor) neural network computing circuit has the advantages of faster computing speed, higher integration density, and higher energy efficiency. However, in the existing resistive random access memory ReRAM, even through repeated SET / RESET operations (SET refers to applying a positive write voltage pulse to t...

AI Technical Summary

Problems solved by technology

However, in the existing resistive random access memory ReRAM, even through repeated SET / RESET operations (SET refers to applying a positive write voltage pulse to the resistive random access memory ReRAM to reduce the resistance of the device; RESET refers to the Resistive random access memory ReRAM applies a negative write voltage pulse to increase the resistance of the device

Different voltage pulses have different changes in the resistance of the device), and the way of writing, reading and verifying, the resistance of the device representing the weight of the network cannot be accurately modulated to the target value, and because the current technology is not mature, the array In some resistive random access memory ReRAM, the device resistance value is not adjustable (open circuit / short circuit fault), the device resistance value changes with the ambient temperature and circuit noise, the network weight will have a large error, and the neural network calculation will have a large error. error, which ultimately makes the recognition accuracy of the neural network low

Images