Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Abnormal data detection method and system and storage medium

An abnormal data and detection method technology, applied in the Internet field, can solve the problems of long consumption time and slow detection speed, and achieve the effect of improving the detection speed, shortening the detection time, and reducing the amount of data storage.

Inactive Publication Date: 2019-07-05
TENCENT TECH (SHENZHEN) CO LTD
View PDF3 Cites 19 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] However, the amount of data in the data set to be detected is usually massive. Inputting a large amount of data to be detected into the same isolated forest model for detection, the detection speed is slow, and the detection process takes a long time

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Abnormal data detection method and system and storage medium
  • Abnormal data detection method and system and storage medium
  • Abnormal data detection method and system and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0032] In order to make the object, technical solution and advantages of the present invention clearer, the implementation manner of the present invention will be further described in detail below in conjunction with the accompanying drawings.

[0033] Related content of the isolation forest model

[0034] Different from the previous detection method of abnormal data based on distance and density, the isolation forest (IsolationForest) algorithm starts from abnormal data to detect abnormal data. The definition of abnormal data in the isolation forest algorithm is:

[0035] 1. A type of data that is a minority in the data set;

[0036] 2. Compared with the eigenvalues ​​of normal data, it is very different.

[0037] The abnormal data in the isolation forest algorithm is some "few and special" data. When detecting abnormal data based on the isolation forest algorithm, it is usually divided into two stages, one stage is the training stage, by establishing at least two isolatio...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses an abnormal data detection method and system and a storage medium, and belongs to the technical field of the Internet. The method comprises the steps that a detection node obtains a pre-established isolated forest model, and the detection node obtains a to-be-detected data subset; the detection node calculates the average path length of each piece of to-be-detected data inthe isolated forest model; the detection node calculates the detection score of each piece of to-be-detected data according to the average path length of each piece of to-be-detected data in the isolated forest model; and the detection node determines a detection result of each piece of to-be-detected data according to the detection score of each piece of to-be-detected data. According to the invention, the to-be-detected data in the to-be-detected data set is distributed to different detection nodes for storage; compared with the prior art, the data storage amount of each detection node is reduced, and the isolated forest model is distributed to different detection nodes, so that abnormal data can be detected on different detection nodes in parallel, the detection speed is increased, andthe detection duration is shortened.

Description

technical field [0001] The invention relates to the technical field of the Internet, in particular to a detection method, system and storage medium of abnormal data. Background technique [0002] Abnormal data refers to data that does not conform to the laws of other data in massive data. The process of finding abnormal data is called abnormal data detection. The detection of abnormal data has a wide range of application scenarios, including attack detection in network security, fraud detection in finance, disease detection in medical care, noise filtering in noise processing, etc. By analyzing the detected abnormal data, various abnormal behaviors can be discovered in time to reduce the probability of risk occurrence. [0003] The isolation forest (Isolation Forest) model is an anomaly detection model established from abnormal data. Based on the established isolation forest model, the existing abnormal data detection method is: to obtain a data set to be detected, the dat...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F11/07
CPCG06F11/0727
Inventor 卢欣
Owner TENCENT TECH (SHENZHEN) CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com