Method of optimizing deep neural network based on learning automata

Abstract

Provided is a method of optimizing a deep neural network based on learning automata (LA). In the training phase of a deep neural network, by starting from a fully connected initial network structure, weak connections in the network are continuously found and removed in the process of iterative parameter update through gradient descent. Thus, a more sparsely connected network structure with smaller generalization error is obtained, and image classification can be carried out on test samples more accurately. The weak connections are judged by LA through continuous interaction with the neural network in the process of training. By using the idea of reinforcement learning for reference, introducing a learning automaton algorithm to improve the traditional back propagation algorithm and removing redundant connections to reduce network parameters, the classification accuracy of test samples is increased, and the method has stronger ability to prevent over-fitting.

Description

Technical field [0001] The present invention relates to a technology in the field of information processing, in particular to a method for removing weak connections in a deep neural network based on Learning Automata (LA). Background technique [0002] Neural network is a traditional machine learning algorithm that can realize non-linear mapping from input to output, and can be applied to tasks such as feature transformation, classification, and recognition. Because of its powerful model expression ability, it has been widely used in pattern recognition, artificial intelligence and other fields. A neural network model usually includes an input layer, an output layer, and a hidden layer. Each layer is composed of a specific number of neurons. Each neuron can be described as y=f(W*x+b), where: x represents the input vector; y represents the output value; the weight vector W and the bias b are trainable parameters, and their set can be represented by θ; f is a non-linear activation...

AI Technical Summary

Problems solved by technology

[0005] Aiming at the problem of too many redundant parameters of the deep neural network, which is easy to fall into overfitting, the present invention proposes a method for removing weak connections in the deep neural network based on learning automata, and introduces LA into the search for connections in the traditional gradient descent iteration process Weak connections, remove redundant connections to reduce network parameters, reduce network calculations, improve classification accuracy on test samples, and make it more capable of preventing overfitting

Images