A Dropout regularization method based on the sensitivity of activation values

An activation value, sensitivity technique, applied in the field of pattern recognition, computer vision and deep learning

Inactive Publication Date: 2018-04-10
TIANJIN UNIV
View PDF0 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Therefore, DropConnect also has certain limitations

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A Dropout regularization method based on the sensitivity of activation values
  • A Dropout regularization method based on the sensitivity of activation values
  • A Dropout regularization method based on the sensitivity of activation values

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0022] The following will clearly and completely describe the technical solutions in the embodiments of the present invention, obviously, the described embodiments are only an example of the present invention, not all examples.

[0023] This patent can be applied to image classification tasks, but is not limited to this task. The convolutional neural network can be applied to many tasks of deep learning.

[0024] This patent provides a dropout regularization method based on activation value sensitivity. The convolutional neural network system mainly includes two stages: training stage and testing stage. The invention applies only to the training phase.

[0025] This patent proposes that the difference between dropout based on activation value sensitivity and traditional dropout lies in the probability that each feature point is set to zero. figure 1 and figure 2 They are the traditional Dropout method and the activation-value-sensitive Dropout method proposed in this paten...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a Dropout regularization method based on the sensitivity of activation values. The method is used for image classification. A data training stage of the method comprises the following steps: 1) data preparation: collecting different types of images and marking the image types as labels; 2) structure design: setting a deep convolutional neural network structure; 3) initialization: (1) determining the weight of a convolution filter, initializing the parameters by using a random initialization method and setting the number of times of iteration and (2) setting a probability density function selected in Dropout; 4) forward computing: performing computing layer by layer from the first layer to the last layer, determining the probability of zero setting of each feature point via the probability density function after the max pooling layer, generating a random number between 0 and 1 by using a uniform distribution function, comparing the random number with the probability of zero setting of each feature point, zero-setting the activation value of the feature point if the random number is less than the probability and maintaining the activation value of the featurepoint if the random number is equal to or greater than the probability; 5) back propagation.

Description

technical field [0001] The invention relates to the fields of pattern recognition, computer vision and deep learning, in particular to a random dropout regularization method. Background technique [0002] With the development of artificial intelligence, convolutional neural networks (CNNs, Convolutional Neural Networks) have gained more attention and are widely used in multiple tasks in the field of computer vision, such as image classification, target recognition, object detection Wait. In order to improve the performance of convolutional neural networks, relevant researchers have proposed many deep and wide network structures. However, due to the limited training data, the complex convolutional neural network is very easy to cause over-fitting problems, which affects the generalization ability of the network and affects the performance of the network. [0003] Regularization is a more effective method to solve the overfitting problem and improve network performance. Dro...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/62G06N3/04G06N3/08
CPCG06N3/084G06N3/047G06N3/045G06F18/24
Inventor 庞彦伟侯聪聪
Owner TIANJIN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap