A Sequence Classification Method Based on Hierarchical Multiscale Recurrent Neural Network

A technology of cyclic neural network and classification method, which is applied in the field of sequence classification based on hierarchical multi-scale cyclic neural network, can solve problems such as difficult training and gradient disappearance, and achieve the effect of easy training, alleviating gradient disappearance, and alleviating problems

Active Publication Date: 2021-08-10
SOUTH CHINA UNIV OF TECH
View PDF7 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Unfortunately, hierarchical RNNs always have a multi-layer structure, which is harder to train and more prone to vanishing gradients than single-layer RNNs

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A Sequence Classification Method Based on Hierarchical Multiscale Recurrent Neural Network
  • A Sequence Classification Method Based on Hierarchical Multiscale Recurrent Neural Network
  • A Sequence Classification Method Based on Hierarchical Multiscale Recurrent Neural Network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0055] Such as figure 1 and figure 2 As shown, this embodiment takes the serialized MNIST handwriting recognition data set as a specific example. The MNIST data set includes 10 categories, wherein the size of the training set is 55,000, the size of the verification set is 5,000, and the size of the test set is 10,000. Convert each 28*28 size picture into a 784*1 sequence, input it to the model for training, and test the model with the best result on the verification set on the test set.

[0056] Such as figure 1 As shown, the sequence classification method based on hierarchical multi-scale recurrent neural network includes the following steps:

[0057] Step S1: Input a serialized MNIST sequence X with a length of 784 and a dimension of 1, and divide the sequence X into 16 equal-length subsequences, then the length L of each subsequence is:

[0058]

[0059] So the sequence X can also be expressed as:

[0060] X=[X 1 ,...,X n ,...,X 16 ]

[0061] where X n For the ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a sequence classification method based on a hierarchical multi-scale cyclic neural network, comprising the following steps: inputting a sequence and dividing it into a plurality of equal-length subsequences; constructing a plurality of pyramid structures according to the sequence of subsequences, each A pyramid receives a sub-sequence as input, and generates a hidden state at the bottom of the pyramid and a hierarchical aggregation state at each level. The aggregation state at the top of each pyramid is used as the input at the bottom of the next sub-pyramid; The aggregation state is iteratively aggregated through the skip connection to obtain the output of this layer; the hierarchical aggregation state sequence of different scales generated by all the pyramids in the lower layer is used as the input of the upper layer to construct a multi-layer cyclic neural network to generate the output of each layer; The output of one layer obtains multi-scale fusion features; finally, based on this feature, a Softmax layer is used to classify the sequence. The invention achieves higher accuracy rate in sequence classification.

Description

technical field [0001] The invention relates to the technical field of sequence classification in deep learning, in particular to a sequence classification method based on a hierarchical multi-scale recurrent neural network. Background technique [0002] In the field of deep learning, Recurrent Neural Network (RNN) is often used to model sequence data, which can capture time dependence in sequence data by using cyclic feedback connections. It has achieved good performance in many applications, such as time series classification, text classification and speech recognition, etc. [0003] RNN is usually trained using the Back Propagation Trough Time (BPTT) method. However, in practice, it is difficult for RNN to pass BPTT training and the problem of gradient disappearance or gradient explosion will occur for long sequences, and it is difficult to capture sequence data. long-term dependence. The gradient explosion can be alleviated by simple gradient clipping, but the gradient...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06K9/62G06N3/04G06N3/08
CPCG06N3/084G06N3/044G06N3/045G06F18/241
Inventor 马千里林镇溪陈恩欢
Owner SOUTH CHINA UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap