Data clustering method based on dimensionality reduction and sampling

A data clustering and clustering technology, applied in the field of data processing, can solve problems such as ineffective processing, and achieve the effect of reducing complexity and efficient clustering

Inactive Publication Date: 2016-08-31
NORTH CHINA ELECTRIC POWER UNIV (BAODING)
View PDF0 Cites 12 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, since k-means needs to repeatedly calculate the similarity between each sample and the center, the complexity of the algorithm will increase exponentially with the increase in the dimension of the data set (time complex

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data clustering method based on dimensionality reduction and sampling
  • Data clustering method based on dimensionality reduction and sampling
  • Data clustering method based on dimensionality reduction and sampling

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0021] The present invention will be further described below in conjunction with accompanying drawing.

[0022] The present invention provides a fast large-scale data clustering analysis method with data clustering analysis capability. The method firstly performs dimensionality reduction processing on the data set through the segmented mean method, and secondly constructs a random function from the large-scale clustering data set. Perform random sampling to obtain a smaller working set, and perform traditional k-means clustering on the working set to obtain the cluster center, complete the sampling process, and obtain the sampling result. Then classify the remaining samples by measuring the relationship between the remaining clustered samples and the obtained sampling results. Since this method greatly reduces the scale of problems involved in k-means clustering through random sampling, it effectively improves the clustering efficiency.

[0023] Let data set X={x 1 ,x 2 ,.....

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data clustering method based on dimensionality reduction and sampling, comprising steps of performing dimensionality reduction processing on a data set through a piecewise mean value algorithm, constructing a random function, performing random sampling from a large-scale clustering data set to obtain a working set with a relatively small scale, performing k-means clustering on the working set to obtain a random sampling clustering result, and performing classification on residual samples through measuring a relation between the residual samples and the obtained sampling clustering result. The data clustering method based on dimensionality reduction and sampling adopts the dimensionality reduction and sampling to reduce the number and dimensionalities of data samples which participate iteration, greatly reduces the complexity of the k-means algorithm under the condition that a good clustering effect is maintained, and realizes high efficiency clustering for big-scale data.

Description

technical field [0001] The invention relates to a method capable of efficiently clustering large-scale data, belonging to the technical field of data processing. Background technique [0002] At present, the commonly used data clustering methods include classic k-means, FCM, hierarchical clustering and self-organizing neural mapping, among which k-means is the most classic and widely used partitioning clustering method. The K-means clustering method dynamically iteratively adjusts the cluster center, and iterates continuously according to the similarity between the sample and each sub-cluster center to obtain the clustering result. However, since k-means needs to repeatedly calculate the similarity between each sample and the center, the complexity of the algorithm will increase exponentially with the increase in the dimension of the data set (time complexity: O(tkmn), space complexity : O((m+k)n). Among them, t is the number of iterations, k is the number of clusters, m is...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/62
CPCG06F18/23213
Inventor 张铁峰李中顾明迪
Owner NORTH CHINA ELECTRIC POWER UNIV (BAODING)
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap