Unlock instant, AI-driven research and patent intelligence for your innovation.

An FCBF-based user-defined feature dimension text feature selection algorithm

A feature selection and self-defining technology, applied in the direction of unstructured text data retrieval, text database clustering/classification, calculation, etc., can solve problems such as reducing classification accuracy and affecting classification efficiency.

Active Publication Date: 2019-06-14
NANJING UNIV OF POSTS & TELECOMM
View PDF2 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] In order to solve the deficiencies in the prior art, the present invention provides a text feature selection algorithm based on FCBF's self-defined feature dimension, which solves the rapid The problems brought about by the increased number of features affect the classification efficiency and even reduce the classification accuracy

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • An FCBF-based user-defined feature dimension text feature selection algorithm
  • An FCBF-based user-defined feature dimension text feature selection algorithm
  • An FCBF-based user-defined feature dimension text feature selection algorithm

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0051] The present invention will be further described below in conjunction with the accompanying drawings. The following examples are only used to illustrate the technical solution of the present invention more clearly, but not to limit the protection scope of the present invention.

[0052] A FCBF-based custom feature dimension text feature selection algorithm, comprising the following steps:

[0053] Step 1, the vectorized text matrix is ​​X, the text category matrix is ​​C, and T={t is initialized according to the text matrix X 1 ,t 2 ...t m}, let S list ={},S best ;

[0054] for S list Assignment: S list Initially an empty set, for t k ∈T, calculate the kth feature word t of the text k Correlation Corr(t k ,C), when Corr(t k ,C) ≥thresh add t k Into S list , T is the set of all feature words, S list is a set of feature words whose correlation between feature words and categories meets the above requirements, thresh is a decimal number (a number between 0 and...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses an FCBF-based user-defined feature dimension text feature selection algorithm. The algorithm comprises the steps of 1, initializing; 2, screening the feature words in the feature word set further through an FCBF algorithm, and obtaining an initial feature word set; 3, if the dimension of the initial feature word set is smaller than the set dimension, selecting the feature of which the correlation value of the feature word and the category is in the front to complement the initial feature word set until the dimension of the initial feature word set is equal to the set feature dimension; and if the dimension of the initial feature word set is just greater than or equal to the set feature dimension, obtaining the feature words of the user-defined feature dimension fromthe initial feature word set. According to the method, an FCBF original algorithm correlation calculation formula is improved, the text features can be selected more accurately, and the improved algorithm can obtain a user-defined feature dimension.

Description

technical field [0001] The invention relates to the technical field of natural language processing, in particular to an FCBF-based custom feature dimension text feature selection algorithm. Background technique [0002] With the continuous development of the Internet, the continuous increase of text information and its diversity makes the text classification task more and more attention of the research community. As the number of texts increases, the number of features in the text will increase or even reach tens of thousands. Not all features are helpful for text classification, and even some redundant features may greatly reduce the classification accuracy. Therefore, text classification Feature selection in is particularly important. [0003] In order to facilitate practical operation, the text data that is difficult to be processed by the computer must first be converted into structured data that can be processed by the computer. Generally, VSM (vector space model) and ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/35G06K9/62
Inventor 于舒娟张昀徐前川何伟
Owner NANJING UNIV OF POSTS & TELECOMM