Sequence classification method based on hierarchical multi-scale recurrent neural network

A technology of cyclic neural network and classification method, which is applied in the field of sequence classification based on hierarchical multi-scale cyclic neural network, and can solve problems such as difficult training and gradient disappearance.

Active Publication Date: 2019-08-13
SOUTH CHINA UNIV OF TECH
View PDF7 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Unfortunately, hierarchical RNNs always have a multi-layer structure, whic...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Sequence classification method based on hierarchical multi-scale recurrent neural network
  • Sequence classification method based on hierarchical multi-scale recurrent neural network
  • Sequence classification method based on hierarchical multi-scale recurrent neural network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0055] Such as figure 1 and figure 2 As shown, this embodiment takes the serialized MNIST handwriting recognition data set as a specific example. The MNIST data set includes 10 categories, wherein the size of the training set is 55,000, the size of the verification set is 5,000, and the size of the test set is 10,000. Convert each 28*28 size picture into a 784*1 sequence, input it to the model for training, and test the model with the best result on the verification set on the test set.

[0056] Such as figure 1 As shown, the sequence classification method based on hierarchical multi-scale recurrent neural network includes the following steps:

[0057] Step S1: Input a serialized MNIST sequence X with a length of 784 and a dimension of 1, and divide the sequence X into 16 equal-length subsequences, then the length L of each subsequence is:

[0058]

[0059] So the sequence X can also be expressed as:

[0060] X=[X 1 ,...,X n ,...,X 16 ]

[0061] where X n For the ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a sequence classification method based on a hierarchical multi-scale recurrent neural network, which comprises the following steps: inputting a section of sequence, and dividing the sequence into a plurality of sub-sequences with equal lengths; constructing a plurality of pyramid structures according to the sequence of the subsequences, receiving a section of subsequence asan input by each pyramid, generating a hidden state located at the tower bottom and a hierarchical aggregation state located at each level, and taking the aggregation state of the tower top of each pyramid as an input of the tower bottom of the next subpyramid; iteratively aggregating the aggregation states of all pyramid tops through jump connection to obtain the output of the layer; constructing a multi-layer recurrent neural network by using hierarchical aggregation state sequences with different scales generated by all low-layer pyramids as input of a high layer, and generating output ofeach layer; aggregating the output of each layer to obtain a multi-scale fusion feature; and finally, on the basis of the feature, using a Softmax layer to classify the sequence. According to the method, relatively high accuracy in sequence classification is achieved.

Description

technical field [0001] The invention relates to the technical field of sequence classification in deep learning, in particular to a sequence classification method based on a hierarchical multi-scale recurrent neural network. Background technique [0002] In the field of deep learning, Recurrent Neural Network (RNN) is often used to model sequence data, which can capture time dependence in sequence data by using cyclic feedback connections. It has achieved good performance in many applications, such as time series classification, text classification and speech recognition, etc. [0003] RNN is usually trained using the Back Propagation Trough Time (BPTT) method. However, in practice, it is difficult for RNN to pass BPTT training and the problem of gradient disappearance or gradient explosion will occur for long sequences, and it is difficult to capture sequence data. long-term dependence. The gradient explosion can be alleviated by simple gradient clipping, but the gradient...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/62G06N3/04G06N3/08
CPCG06N3/084G06N3/044G06N3/045G06F18/241
Inventor 马千里林镇溪陈恩欢
Owner SOUTH CHINA UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap