Method of optimizing deep neural network based on learning automata

A deep neural network and optimization method technology, applied in the field of removing weak connections, can solve problems such as reducing the amount of network calculations, overfitting, etc., and achieve the effects of simple and intuitive models, optimized structures, and improved classification speed

Inactive Publication Date: 2017-07-14
SHANGHAI JIAO TONG UNIV
View PDF0 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Aiming at the problem of too many redundant parameters of the deep neural network, which is easy to fall into overfitting, the present invention proposes a method for removing weak connections in the deep neural network based on learning automata, and introduces LA into the

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method of optimizing deep neural network based on learning automata
  • Method of optimizing deep neural network based on learning automata
  • Method of optimizing deep neural network based on learning automata

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0018] Such as figure 1 As shown, in the training stage of the deep neural network, this embodiment starts from the fully connected initial network structure, and continuously finds and removes weak connections in the network during the process of iteratively updating parameters through gradient descent, thereby obtaining a more sparse Connected, network structure with smaller generalization error.

[0019] Such as image 3 As shown, the image classification process of the test sample based on the optimized deep neural network obtained by the above method is specifically as follows: first, the original input image (such as grayscale or RGB image) is subjected to simple standardized preprocessing: Each dimension subtracts the mean and divides by the variance, and then enters the trained classification model to classify and get a higher-precision result.

[0020] The classification model includes a deep neural network and LA. The deep neural network is a fully connected multilayer ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

Provided is a method of optimizing a deep neural network based on learning automata (LA). In the training phase of a deep neural network, by starting from a fully connected initial network structure, weak connections in the network are continuously found and removed in the process of iterative parameter update through gradient descent. Thus, a more sparsely connected network structure with smaller generalization error is obtained, and image classification can be carried out on test samples more accurately. The weak connections are judged by LA through continuous interaction with the neural network in the process of training. By using the idea of reinforcement learning for reference, introducing a learning automaton algorithm to improve the traditional back propagation algorithm and removing redundant connections to reduce network parameters, the classification accuracy of test samples is increased, and the method has stronger ability to prevent over-fitting.

Description

Technical field [0001] The present invention relates to a technology in the field of information processing, in particular to a method for removing weak connections in a deep neural network based on Learning Automata (LA). Background technique [0002] Neural network is a traditional machine learning algorithm that can realize non-linear mapping from input to output, and can be applied to tasks such as feature transformation, classification, and recognition. Because of its powerful model expression ability, it has been widely used in pattern recognition, artificial intelligence and other fields. A neural network model usually includes an input layer, an output layer, and a hidden layer. Each layer is composed of a specific number of neurons. Each neuron can be described as y=f(W*x+b), where: x represents the input vector; y represents the output value; the weight vector W and the bias b are trainable parameters, and their set can be represented by θ; f is a non-linear activation...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N3/04
CPCG06N3/045
Inventor 李生红郭浩楠马颖华任栩蝶汤璐
Owner SHANGHAI JIAO TONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap