Unlock instant, AI-driven research and patent intelligence for your innovation.

Text classification method based on category door mechanism

A text classification and classification technology, applied in text database clustering/classification, neural learning methods, text database query, etc., can solve problems such as the inability to accurately predict the meaning of text and the inability to accurately grasp the true meaning of words.

Active Publication Date: 2020-02-14
BEIJING INSTITUTE OF TECHNOLOGYGY
View PDF5 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] The purpose of the present invention is to address the technical defects that the existing model using word vectors as the initial input of the neural network in the field of natural language understanding cannot accurately grasp the true meaning expressed by the word, and thus cannot accurately predict the meaning of this text, and proposes A text classification method based on the category gate mechanism to predict the text category

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Text classification method based on category door mechanism
  • Text classification method based on category door mechanism
  • Text classification method based on category door mechanism

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0051] This embodiment describes the reason why the user gate is only used in the user hierarchical network and the product gate is only used in the product hierarchical network when a category classification method based on the category gate mechanism is implemented.

[0052] A vocabulary category gate is placed at the word level of the user-level network and a product-level network, and a sentence category gate is placed at the sentence level.

[0053] Take the product gate as an example to explain why the user gate and product gate are only set at the sentence level:

[0054] Given product glasses, a sentence describing glasses may contain many different words, but only a few words are highly related to product glasses. For example, the word cool is highly related to product glasses, and the product attention mechanism will The word cool is selected, but the user's attention mechanism may not necessarily select the word cool; after the attention mechanism, use the product g...

Embodiment 2

[0116] Embodiment 2 uses contextual information to acquire word categories, that is, part 1.b of the summary of the invention.

[0117] 1.b specific implementation: set the number of hyperparameter categories to n c , the ith word vector and the word vectors of its adjacent words and Adding up, the category distribution of the i-th word is obtained based on the formula (1) through the linear layer

[0118]

[0119] Among them, softmax is the activation function of the linear layer; Represents element-wise addition; is the trainable weight parameter; is cd i the transposition of is the category distribution;

[0120] The category vocabulary vector is denoted as V c , c i is the category vector of the i-th word, n c is the number of categories, d c is the category vector dimension;

[0121] The category vector c of the i-th word i Through (2) obtained by the product of category distribution and category vocabulary vector table:

[0122] c i =cd i ·...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a text classification method based on a category door mechanism, and belongs to the technical field of natural language understanding. Two independent hierarchical networks, namely a user hierarchical network and a product hierarchical network, are used; wherein the former uses a user attention mechanism and a user door, and the latter uses a product attention mechanism and a product door; word levels of the two-level network are respectively provided with a vocabulary category door, and sentence levels of the two-level network are respectively provided with a sentencecategory door; wherein the two-level network respectively converts texts into feature vectors, and then text classification is carried out by using the feature vectors; therefore, real meanings expressed by the words can be accurately mastered, and the category of the text can be accurately predicted. According to the classification method, redundant information is filtered from different angles,so that a model can understand the real meaning of a text; no matter whether sentences have one-word polysemy or not. A model has the advantages of accuracy and small fluctuation; cleaning vocabularyredundant information so that the vocabulary redundant information does not interfere with model parameter updating; the defect that an attention mechanism only extracts key information from words isovercome.

Description

technical field [0001] The invention relates to a text classification method based on a category gate mechanism, belonging to the technical field of natural language understanding. Background technique [0002] Text classification aims to understand and classify natural language sentences, which is a key task in the field of natural language processing. The text classification method based on neural network has become the current mainstream method because of its high efficiency. Most of these methods embed words into low-dimensional vectors, and then use these vectors as the initial input of the neural network; then, the text is encoded using a well-designed network to obtain a text feature vector; this vector is further used for analysis The information contained in the text, and then predict the category it belongs to. However, most existing methods do not take polysemy into account, that is, a word usually contains multiple meanings and these meanings share the same wor...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/174G06F16/33G06F16/35G06N3/04G06N3/08G06F40/205
CPCG06F16/3344G06F16/35G06F16/174G06N3/08G06N3/048G06N3/044
Inventor 施重阳姜欣雨冯超群郝戍峰张奇
Owner BEIJING INSTITUTE OF TECHNOLOGYGY