An Improved Chinese Short Text Classification Method for Low-Frequency Words
A classification method and technology of low-frequency words, applied in the direction of neural learning methods, instruments, biological neural network models, etc., can solve the problem of low-frequency word information not being used and noise, and achieve the effect of improving accuracy
Active Publication Date: 2022-05-10
GUANGDONG UNIV OF TECH
View PDF11 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0005] The purpose of the present invention is to overcome the deficiencies in the prior art, and provide an improved Chinese short text classification method for low-frequency words, in order to solve the problem that low-frequency word information is not utilized in the existing text classification research based on word vectors and utilize low-frequency words The problem of too much noise in the information process
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0055] The present invention will be further described below in conjunction with specific examples, but the embodiments of the present invention are not limited thereto.
[0056] Such as figure 1 As shown, a kind of Chinese short text classification method that this embodiment provides is improved for low-frequency words, specifically comprises the following steps:
[0057] S1, obtain the text data set and divide the training set, preprocess the text data in the training set, and obtain the word list set corresponding to the text data; count the total word frequency of each word in all text data, and select the total word frequency Number is formed low-frequency word set less than N, and this low-frequency word set is preserved as the array type that element is word; Described preprocessing comprises noise information removal, word segmentation processing and stop word processing; Described N is adjustable, in the present embodiment The value of N is 10. The text data set is...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses an improved Chinese short text classification method for low-frequency words. Construct a low-frequency word set; construct a corresponding category feature dictionary according to the text data of each category label in the training set; establish a text representation, divide the words in the text into three categories, and use different conversion methods to list the words corresponding to the text data The set is converted into a set of word vector lists corresponding to text that can be recognized by a computer; building the classification model includes an input layer, a word attention layer, a feature extraction network, and an output layer, wherein the input layer is a set of word vector lists, and the word attention The force layer is used to weight the words in the text data. After the weighted word vector output by the word attention layer is processed by the feature extraction network, the classification result of the text data is obtained in the output layer.
Description
technical field [0001] The invention relates to the field of computer natural language processing, in particular to an improved Chinese short text classification method for low-frequency words. Background technique [0002] In recent years, with the development of social networks, people can easily publish and obtain news on social platforms, and social platforms have become new information gathering places. Massive amounts of data are generated on social platforms every day, most of which are in the form of short texts, such as Weibo, chat messages, news topics, opinions and comments, question texts, mobile phone text messages, etc. Accurate classification of these short texts is of great importance. Important theoretical significance and practical application value: It is beneficial to the research and development of downstream tasks such as information extraction and sentiment analysis; the government can use short texts to quickly understand the people's sentiments to ma...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & AuthorityPatents(China)
IPC IPC(8): G06F40/289G06F40/216G06K9/62G06N3/04G06N3/08
CPCG06F40/289G06F40/216G06N3/08G06N3/045G06F18/2415
Inventor罗孝波梁祖红
OwnerGUANGDONG UNIV OF TECH