Text feature extraction method based on semantic analysis

A feature extraction and semantic analysis technology, applied in the field of semantic network, can solve the problem of not considering the contribution of the semantic status of feature words, and achieve the effect of great use value, high precision and high accuracy

Inactive Publication Date: 2017-05-03
SICHUAN YONGLIAN INFORMATION TECH CO LTD
View PDF2 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the commonly used text feature extraction methods do not consider the sema...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Text feature extraction method based on semantic analysis
  • Text feature extraction method based on semantic analysis
  • Text feature extraction method based on semantic analysis

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0021] In order to solve the problem that the commonly used text feature extraction methods do not consider the semantic status of the feature vocabulary and its contribution to the expression of the text, combined with Figure 1-Figure 4 The present invention has been described in detail, and its specific implementation steps are as follows:

[0022] Step 1: Initialize the text corpus module and preprocess the text W. The specific description process is as follows:

[0023] Comprehensive word segmentation and stop word removal technology, the flow chart of the Chinese text preprocessing process is as follows figure 2 .

[0024] The word segmentation method here uses a Chinese automatic word segmentation algorithm based on information theory, and its specific word segmentation and stop word removal steps are as follows:

[0025] Step 1.1: Use the stop table to process the text to remove stop words.

[0026] Step 1.2: According to the "Word Segmentation Dictionary", find th...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a text feature extraction method based on semantic analysis. The method comprises the following steps: initializing a text corpus, performing word segmentation and unused word removal processing on texts, clustering the texts according to semantic relevance (the formula is as shown in the specification), constructing a lexical semantic network model graph, obtaining the status and the contribution of a vocabulary in the whole text according to the importance of the vocabulary in a lexical semantic network model, setting a proper importance threshold, and extracting a feature vocabulary vector of the text. Compared with the tradition word frequency-inverse document frequency method, the text feature extraction method provided by the invention has the advantages of having higher accuracy, overcoming the defect that only one category of text features can be extracted by the information gain method, having higher application value, accurately calculating the contribution of different vocabularies to the idea of the text and meanwhile providing good theoretical basis for subsequent text clustering.

Description

technical field [0001] The invention relates to the technical field of semantic network, in particular to a text feature extraction method based on semantic analysis. Background technique [0002] Currently commonly used text feature extraction methods include word frequency-inverse document frequency method—TF-IDF, information gain method, mutual information and other methods; the simple structure of TF-IDF cannot effectively reflect the importance of words or phrases and the characteristics of feature values. Distribution, so the accuracy of TF-IDF is not very high. The information gain method is only suitable for extracting text features of one category, but cannot be used for extracting text features of multiple categories. The mutual information method considers the ratio of the probability of category occurrence to the probability of collection, which will cause a defect, that is, the difference in the number of texts in the category collection will greatly affect the...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/27G06F17/30
CPCG06F16/35G06F40/289
Inventor 金平艳
Owner SICHUAN YONGLIAN INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap