Text classification method based on attention mechanism

A technology of text classification and attention, applied in text database clustering/classification, neural learning methods, unstructured text data retrieval, etc., can solve problems such as network degradation, network parameter redundancy, etc. Feature extraction ability, remarkable effect

Active Publication Date: 2021-08-03
NANJING UNIV OF POSTS & TELECOMM
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

For the complexity of text data, using DenseNet is only a simple feature extraction of text information by increasing the number of network layers, which will not only lead to redundant network parameters, but even network degradation.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Text classification method based on attention mechanism
  • Text classification method based on attention mechanism
  • Text classification method based on attention mechanism

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0045] Below in conjunction with accompanying drawing and simulation result, a kind of text classification method based on attention mechanism that the present invention proposes is described in detail:

[0046] A text classification method based on attention mechanism, its implementation process is as follows:

[0047] The experimental environment is Windows 10 64bit operating system, the CPU is Intel i7-8700, the GPU is NVIDIA GeForce RTX 2070, and the memory is 16GB. The experiment is implemented on the deep learning framework Tensorflow, and the development language of the experiment is Python.

[0048] The hyperparameter settings of the experiment, the batch is set to 64, the learning rate is set to 1, and the number of iterations is set to 50, using 3 attention-based blocks, and the corresponding filter numbers are 64, 128, 256, based on attention-intensive blocks The convolution kernel size is set to 1×3, the convolution kernel size of the convolution layer in the conve...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a text classification method based on an attention mechanism. According to the method, a neural network model based on DenseNet is used; before a neural network is trained, semantic coding is utilized to initialize the weight of a convolutional filter of DenseNet, so that the network identifies important semantic information before training, and effective position information of each sentence can be captured in a convolutional layer; and feature extraction is performed on the text information through a local attention intensive connection module. Compared with the prior art, the text classification method has higher feature extraction capacity, the content of the text information is reserved, the effect is remarkable when multiple text classification tasks including sentiment classification, theme classification and the like are included, and the classification accuracy is effectively improved.

Description

technical field [0001] The invention belongs to the field of text classification in natural language processing, and proposes a text classification method based on an attention mechanism. Background technique [0002] With the rapid growth of information data, natural language processing has developed rapidly in the wave of artificial intelligence technology. Text classification is the basic task of natural language processing, which is used in various fields of natural language processing, such as information retrieval, information filtering and semantic analysis. However, in the face of the massive short text information emerging in the era of big data, how to divide new fields The category to which it belongs, its training samples are often insufficient. Deep learning (Xu Yilong, Li Wenfa, Zhou Chunjie. A review of natural language processing based on deep learning. Network Application Branch of China Computer Users Association, 2018.] can not only realize the automation...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/35G06F40/284G06N3/04G06N3/08
CPCG06F16/35G06F40/284G06N3/08G06N3/047G06N3/048G06N3/045Y02D10/00
Inventor 于舒娟蔡梦梦吴梦洁毛新涛黄橙徐钦晨张昀王秀梅
Owner NANJING UNIV OF POSTS & TELECOMM
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap