Dynamic grid optimization-based LOF (Local Outlier Factor) clustering data anomaly point detection method and detection system

Abstract

The invention discloses a dynamic grid optimization-based LOF (Local Outlier Factor) clustering data anomaly point detection method and detection system. The detection method comprises the following steps: 1, obtaining an optimal unit grid vector Mopt according to an initial unit grid vector M0 and a growth vector delta p; 2, dividing data space into a dense region Rd, a sparse region Rs and a transition region Rt according to the optimal unit grid vector Mopt; 3. dividing all grids in the transition region Rt, which is obtained in the step 2, into dense region grids G'd and sparse region grids G's according to grid periphery density ratio vectors F, and adding the sparse region grids G's into the sparse region Rs to form a sparse region Regions of the data space; and 4, applying an LOF algorithm to detect anomaly points of the sparse region Regions of the data space. According to the method, the operation data amount of the LOF algorithm is reduced through the dynamic grids, and the computation time of the LOF algorithm is greatly reduced.

Description

technical field [0001] The invention belongs to the field of data mining, and mainly relates to a data abnormal point detection method and a detection system. Background technique [0002] With the advent of the big data era, how to effectively and efficiently analyze and mine the information contained in these large-scale data sets is of great significance. Among the information contained in large-scale database datasets, data outliers often contain potentially valuable information. Outlier detection is an important research branch of data mining. Its main function is to extract data that is very different from mainstream data and has a very small amount from huge and complex data. At present, researchers have proposed a large number of outlier detection algorithms, mainly including methods based on statistical distribution, methods based on distance, and detection methods based on clustering. However, most of the existing outlier algorithms have high time complexity, and...

AI Technical Summary

Problems solved by technology

However, most of the existing outlier algorithms have high time complexity, and often spend a lot of time and space when dealing with large-scale data.

In addition, most of the current data mining software has a high dependence on the database environment to be detected, and the versatility is poor.

There is no software that can automatically mine data outliers for any basic database, which greatly limits the application of outlier detection

Images