Density-based text clustering algorithm

A text clustering algorithm and density technology, applied in the field of keyword extraction and semantic analysis, can solve the problems of inaccurate calculation of similarity between documents, high-dimensional sparseness, and impact on clustering accuracy

Inactive Publication Date: 2017-07-07
CHONGQING UNIV OF POSTS & TELECOMM
View PDF5 Cites 13 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This text representation method assumes that words are independent, and does not analyze the content of documents semantically, so the similarity

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Density-based text clustering algorithm
  • Density-based text clustering algorithm
  • Density-based text clustering algorithm

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0041] refer to figure 1 , a density-based text clustering method proposed by the present invention includes the following steps:

[0042] Step A, according to the data set, the text is segmented and the stop words are removed;

[0043] Step B, according to the obtained text word segmentation, according to the three kinds of parts of speech and word frequency of noun, verb, adjective, the corresponding keywords are extracted to the word segmentation;

[0044] Step C, calculating the keyword similarity of the text by using the improved HowNet vocabulary similarity algorithm according to the obtained keywords;

[0045]Step D, calculate the similarity of the text according to the obtained text keyword similarity;

[0046] Step E, according to the obtained text similarity, the text is clustered by a density-based clustering algorithm;

[0047] In the step A, the NLPIR Chinese lexical analysis system of the Institute of Computing Technology of the Chinese Academy of Sciences, na...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The present invention discloses a density-based text clustering algorithm research method. The method comprises the following steps: using the ICTCLAS word segmentation system to carry out word segmentation on a text in a text set, and extracting corresponding keywords from the word segmentation according to the three parts of speech of the noun, the verb, and the adjective, and the word frequency; using an improved HowNet word similarity algorithm to calculate keyword similarity of the obtained keywords; according to the keyword similarity in the text, calculating text similarity; and according to the obtained text similarity, using the density-based clustering algorithm to carry out clustering on the text, so that the performance of the existing text-related information retrieval technology can be significantly improved.

Description

technical field [0001] The invention relates to the field of computer text information processing, in particular to a method for keyword extraction and semantic analysis. Background technique [0002] In recent years, with the large-scale popularization of the network and the improvement of enterprise informatization, various resources have exploded. However, most of the information is stored in text databases. For such semi-structured or unstructured data, However, the means to obtain specific content information are weak, resulting in difficulty in information search and low utilization of information. As a result, researches on text mining, information filtering and information retrieval have reached an unprecedented climax. Fast and high-quality text clustering technology can form a large amount of text information into a few meaningful clusters, and make the text information in the same cluster have a high degree of similarity, while the text between different clusters...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30G06F17/27
CPCG06F16/35G06F40/289G06F40/30
Inventor 周应华李春婷
Owner CHONGQING UNIV OF POSTS & TELECOMM
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap