Dynamic grid optimization-based LOF (Local Outlier Factor) clustering data anomaly point detection method and detection system

A data anomaly and dynamic grid technology, applied in structured data retrieval, database model, relational database, etc., can solve the problems of limited application of abnormal point detection, high time complexity, large time and space, etc., to reduce the amount of calculation , reduce the calculation time, and the effect of small amount of calculation

Inactive Publication Date: 2017-10-17
THE 28TH RES INST OF CHINA ELECTRONICS TECH GROUP CORP
View PDF0 Cites 16 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, most of the existing outlier algorithms have high time complexity, and often spend a lot of time and space when dealing with large-scale data.
In addition, most of the current data mining software has a high depen

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Dynamic grid optimization-based LOF (Local Outlier Factor) clustering data anomaly point detection method and detection system
  • Dynamic grid optimization-based LOF (Local Outlier Factor) clustering data anomaly point detection method and detection system
  • Dynamic grid optimization-based LOF (Local Outlier Factor) clustering data anomaly point detection method and detection system

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0045] The present invention discloses a LOF clustering data abnormal point detection method based on dynamic grid optimization and a detection system applying the method. The following takes the detection of abnormal points in a two-dimensional spatial data set as an example, and further illustrates the present invention in conjunction with the accompanying drawings.

[0046] First of all, the present invention discloses a LOF clustering data abnormal point detection method based on dynamic grid optimization, including the following steps:

[0047] (1) According to the initial unit grid vector M 0 (m 1 ,m 2 ) And the growth vector △p(△p 1 ,△p 2 ), get the optimal unit grid vector M opt ;

[0048] According to a selected unit grid vector, the data space can be divided into multiple grids, such as figure 1 Shown is a distribution map of a data set in a two-dimensional space. The unit length division of each dimension in the data space is independent. Take a smaller unit grid vector M...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a dynamic grid optimization-based LOF (Local Outlier Factor) clustering data anomaly point detection method and detection system. The detection method comprises the following steps: 1, obtaining an optimal unit grid vector Mopt according to an initial unit grid vector M0 and a growth vector delta p; 2, dividing data space into a dense region Rd, a sparse region Rs and a transition region Rt according to the optimal unit grid vector Mopt; 3. dividing all grids in the transition region Rt, which is obtained in the step 2, into dense region grids G'd and sparse region grids G's according to grid periphery density ratio vectors F, and adding the sparse region grids G's into the sparse region Rs to form a sparse region Regions of the data space; and 4, applying an LOF algorithm to detect anomaly points of the sparse region Regions of the data space. According to the method, the operation data amount of the LOF algorithm is reduced through the dynamic grids, and the computation time of the LOF algorithm is greatly reduced.

Description

technical field [0001] The invention belongs to the field of data mining, and mainly relates to a data abnormal point detection method and a detection system. Background technique [0002] With the advent of the big data era, how to effectively and efficiently analyze and mine the information contained in these large-scale data sets is of great significance. Among the information contained in large-scale database datasets, data outliers often contain potentially valuable information. Outlier detection is an important research branch of data mining. Its main function is to extract data that is very different from mainstream data and has a very small amount from huge and complex data. At present, researchers have proposed a large number of outlier detection algorithms, mainly including methods based on statistical distribution, methods based on distance, and detection methods based on clustering. However, most of the existing outlier algorithms have high time complexity, and...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/2465G06F16/285
Inventor 金鑫刘晓晖卢明许田丹叶健聪张硕戴楠
Owner THE 28TH RES INST OF CHINA ELECTRONICS TECH GROUP CORP
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap