Unlock instant, AI-driven research and patent intelligence for your innovation.

A Text Classification Method Based on Category Gate Mechanism

A text classification and category technology, applied in text database clustering/classification, neural learning methods, text database query, etc., can solve problems such as the inability to accurately predict the meaning of text and the inability to accurately grasp the true meaning of words

Active Publication Date: 2022-01-04
BEIJING INSTITUTE OF TECHNOLOGYGY
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] The purpose of the present invention is to address the technical defects that the existing model using word vectors as the initial input of the neural network in the field of natural language understanding cannot accurately grasp the true meaning expressed by the word, and thus cannot accurately predict the meaning of this text, and proposes A text classification method based on the category gate mechanism to predict the text category

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A Text Classification Method Based on Category Gate Mechanism
  • A Text Classification Method Based on Category Gate Mechanism
  • A Text Classification Method Based on Category Gate Mechanism

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0051] This embodiment describes the reason why the user gate is only used in the user hierarchical network and the product gate is only used in the product hierarchical network when a category classification method based on the category gate mechanism is implemented.

[0052] A vocabulary category gate is placed at the word level of the user-level network and a product-level network, and a sentence category gate is placed at the sentence level.

[0053] Take the product gate as an example to explain why the user gate and product gate are only set at the sentence level:

[0054] Given product glasses, a sentence describing glasses may contain many different words, but only a few words are highly related to product glasses. For example, the word cool is highly related to product glasses, and the product attention mechanism will The word cool is selected, but the user's attention mechanism may not necessarily select the word cool; after the attention mechanism, use the product g...

Embodiment 2

[0116] Embodiment 2 uses contextual information to acquire word categories, that is, part 1.b of the summary of the invention.

[0117] 1.b specific implementation: set the number of hyperparameter categories to n c , the ith word vector and the word vectors of its adjacent words and Adding up, the category distribution of the i-th word is obtained based on the formula (28) through the linear layer

[0118]

[0119] Among them, softmax is the activation function of the linear layer; Represents element-wise addition; is the trainable weight parameter; is cd i the transposition of is the category distribution;

[0120] The category vocabulary vector is denoted as V c , c i is the category vector of the i-th word, n c is the number of categories, d c is the category vector dimension;

[0121] The category vector c of the i-th word i Obtained by (29) from the product of the category distribution and the category vocabulary vector table:

[0122] c i =...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a text classification method based on a category gate mechanism, belonging to the technical field of natural language understanding. Use two independent hierarchical networks, user-level network and product-level network; the former uses user attention mechanism and user gate, and the latter uses product attention mechanism and product gate; the word level of the two-level network is equipped with a vocabulary category gate, sentence A sentence category gate is placed at each level; the two-level network converts the text into a feature vector, and then uses the feature vector to classify the text; thus accurately grasping the true meaning expressed by the word, and then accurately predicting the category of the text. The classification method filters redundant information from different angles, so that the model can understand the true meaning of the text; no matter whether the sentence has polysemy or not, the model has the advantages of accuracy and little fluctuation; the redundant information of vocabulary is cleaned, So that it will not interfere with the update of model parameters; it makes up for the deficiency that the attention mechanism only extracts key information from words.

Description

technical field [0001] The invention relates to a text classification method based on a category gate mechanism, belonging to the technical field of natural language understanding. Background technique [0002] Text classification aims to understand and classify natural language sentences, which is a key task in the field of natural language processing. The text classification method based on neural network has become the current mainstream method because of its high efficiency. Most of these methods embed words into low-dimensional vectors, and then use these vectors as the initial input of the neural network; then, the text is encoded using a well-designed network to obtain a text feature vector; this vector is further used for analysis The information contained in the text, and then predict the category it belongs to. However, most existing methods do not take polysemy into account, that is, a word usually contains multiple meanings and these meanings share the same wor...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/174G06F16/33G06F16/35G06N3/04G06N3/08G06F40/205
CPCG06F16/3344G06F16/35G06F16/174G06N3/08G06N3/048G06N3/044
Inventor 施重阳姜欣雨冯超群郝戍峰张奇
Owner BEIJING INSTITUTE OF TECHNOLOGYGY