Chinese text parallel data mining method based on hierarchy

A text and Chinese technology, applied in the field of information processing, can solve problems such as low mining efficiency and large amount of original data, and achieve the effects of improving calculation speed, high word segmentation accuracy, and high efficiency
CN102662952BActive Publication Date: 2015-04-15UESTC COMSYS INFORMATION
5 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN ยท China
Patent Type
Patents(China)
Current Assignee / Owner
UESTC COMSYS INFORMATION
Publication Date
2015-04-15

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention relates to a Chinese text parallel data mining method based on hierarchy, comprising the steps of: step 1: a establishing vector space model of Chinese texts: performing work segmentation regarding to the entire Chinese text set to obtain a word segmentation form and a feature term set containing all removed duplicated terms in the text set of each text, then using the feature term set to count the term frequency-inverse document frequency (TFIDF) of each text, and establishing the text vector space model according to the TFIDF; step 2: performing dimension reduction regarding to a feature item vector of the text vector space model; and step 3: clustering texts using DCURE algorithm based on hierarchy. The method is efficient in word segmentation of Chinese texts with high accuracy, requires no input of parameters like radius of neighborhood for the clustering process, can mine irregular cluster and is insensitive to noise, employs distributed calculating, has high efficiency in mining mass texts and improves calculating speed of feature weight.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention belongs to the technical field of information processing, specifically relates to the technical field of computer data mining and machine learning, and can be used for improving search results in search engines, personalized content recommendation systems, aggregation of similar questions in question answering systems, and aggregation of news by category on news websites and other information The processing technology field plays a role. Background technique

[0002] With the development of the Internet, the text information on the webpage is increasing rapidly. How to index, retrieve, manage, and mine the massive text information on the webpage has become a huge challenge in the field of computer science. Chinese text clustering technology is also developing and maturing, distributed technology has been more and more widely used, and distributed clustering technology is an important research content in the field of distributed data minin...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More