An improved algorithm for carrying out abnormal mining on density irregular data based on DBSCAN

Abstract

The invention discloses an improved algorithm for carrying out abnormal mining on density irregular data based on DBSCAN, and relates to the field of data mining. In order to overcome the defect thata DBSCAN algorithm is poor in effect on a non-uniform density data set, the invention provides an improved method combining the K-means algorithm and a DBSCAN. The method comprises the following stepsof firstly improving the K-means algorithm by utilizing a differential evolution method; speeding up the convergence speed of the algorithm to obtain the optimal clustering division and clustering number, and then utilizing the improved K-means algorithm to perform the preliminary division on a data set with non-uniform density, and then performing the abnormal detection on a divided sub-data setby means of the DBSCAN algorithm, and finally analyzing and combining the detection results to obtain a final set. Compared with a traditional improved algorithm, the accuracy is improved, the speedis higher, the efficiency is improved, and the requirement for high efficiency of the abnormal data mining is met.

Description

technical field [0001] The invention belongs to the field of data mining, and in particular relates to an improved algorithm for abnormally mining data with irregular density based on DBSCAN. Background technique [0002] In the information age, the increasing amount of data is the primary problem people face. How to quickly and effectively use these data, or discover the valuable information hidden behind the data, has become an important challenge for modern technology. Data mining is the intersection of artificial intelligence, pattern recognition, machine learning and statistics. It is a new discipline that adapts to the needs of the information society to extract useful knowledge from massive data sets. Data mining is a non-trivial process whose purpose is to obtain useful, novel, potential, and ultimately understandable knowledge and information from a large amount of data. [0003] Anomaly data mining officially entered people's field of vision in 1887, with a paper ...

AI Technical Summary

Problems solved by technology

However, the DBSCAN algorithm has the disadvantage that it is not effective for data with irregular densities.

This is because the algorithm operates on the entire database, but its parameters are single and constant, which makes the selection of parameters not applicable to the data areas of various densities in the database. Therefore, when the spatial clustering density is uneven , or when the distances between clusters are very different, the clustering quality using the DBSCAN algorithm is poor. Aiming at the shortcomings of the DBSCAN algorithm for non-uniform density data sets, the present invention proposes an improved K-means algorithm that is comparable to DBSCAN Combined method, first use the differential evolution method to improve the K-means algorithm, speed up the convergence speed of the algorithm to obtain the optimal cluster division and cluster number, and then use the improved K-means to initially divide the data set with uneven density, and then Then use the DBSCAN algorithm to detect abnormalities in the sub-data sets that have been divided, and finally analyze the detection results and merge them

Images