Handwritten form recognition method based on self-adaptive band differential gradient optimization

Abstract

The invention discloses a handwritten form recognition method based on self-adaptive band differential gradient optimization. In the BP neural network parameter optimization algorithm for handwrittenform recognition, a common gradient descent algorithm is reintegrated and deformed by combining a traditional control theory thought; then, a differential link is added into a conventional gradient descent algorithm for advanced correction, and the future change trend of an error signal is forecasted through the change rate of errors, so that the precision is improved; and finally, adaptively adjusting the learning rate by using the average value of the stored exponentially attenuated past square gradients so as to accelerate the training rate. According to the method provided by the invention, a differential link is introduced, so that the training rate can be effectively improved, and the future change trend of error signals is forecasted through the change rate of errors. The learning rate can be adaptively adjusted, i.e., when the training is close to the optimal value, the learning rate is reduced due to the increase of the accumulated past square gradient, and the optimal point is prevented from being skipped due to the overlarge learning rate.

Description

technical field [0001] The invention belongs to the field of artificial intelligence and relates to network training and optimization in deep learning and intelligent computing, in particular to a handwriting recognition method based on adaptive band differential gradient optimization. Background technique [0002] Artificial intelligence is a technology that researches and develops to simulate, extend and expand human intelligence. Its main research contents can be summarized into four aspects: machine perception, machine thinking, machine behavior and machine learning. Machine learning is the use of knowledge such as computers, probability theory, statistics, etc., by inputting data into computer programs, so that computers can learn new knowledge and new skills. intelligent. Deep learning is a broader approach to machine learning based on learning features that attempt to learn at multiple levels, where higher level concepts are defined from lower level concepts that hel...

AI Technical Summary

Problems solved by technology

[0005] The optimization of neural networks is a very important issue in the field of deep learning and intelligent computing, especially the learning and correction of neural network weights. Most learning algorithms are based on iterative update methods, and the commonly used gradient-based optimization algorithms are the largest. The difficulty is how to choose the appropriate learning rate and gradient to speed up the training rate and convergence rate

[0006] At present, the commonly used optimization method for handwriting recognition methods is the gradient descent algorithm with momentum. This method mainly has 1) the convergence rate is poor, and the convergence rate is relatively slow

2) The weight update lags behind the actual gradient change

Images