Method and device for abnormity detection of sparse data

A sparse data, anomaly detection technology, applied in the field of anomaly detection, can solve problems such as high time complexity, lack of generality, and inability to apply to large-scale data.

Active Publication Date: 2017-11-24
BEIHANG UNIV
View PDF6 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

First, most of the values ​​in sparse data are 0, and there are only limited non-zero elements in a single data object. It is very challenging to infer the abnormal characteristics of data objects simply using limited non-zero elements, and it is necessary to combine the implicit relationship between attribute values
Many traditional methods, such as distance-based methods and pattern-based methods, cannot obtain the impl

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for abnormity detection of sparse data
  • Method and device for abnormity detection of sparse data
  • Method and device for abnormity detection of sparse data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0055] In order to understand the characteristics and technical contents of the embodiments of the present invention in more detail, the implementation of the embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings. The attached drawings are only for reference and description, and are not intended to limit the embodiments of the present invention.

[0056] In order to facilitate the understanding of the technical solutions of the embodiments of the present invention, several anomaly detection methods are explained below:

[0057] (1) Distance-based anomaly detection method

[0058] The distance-based method is based on the assumption that abnormal points are far away from most normal points. By calculating the distance between other points and a given point, k neighbor nodes of a given point are found. If the point is far away from its neighbor nodes , the point is more likely to be an outlier. In distance-based met...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method and device for abnormity detection of sparse data. The method comprises the steps that characteristic processing is conducted to original data of different types, so the original data of the different types is converted into the sparse data with a uniform format; a factorization machine is used for modeling of the sparse data, so a nonlinear manifold model is obtained; according to the nonlinear manifold model, abnormal value scores of data objects are computed; and according to abnormal value scores of the data objects, whether the data objects are abnormal data is judged.

Description

technical field [0001] The invention relates to the technical field of anomaly detection, in particular to a sparse data anomaly detection method and device based on a factorization machine. Background technique [0002] Anomaly detection is mainly based on Kawkins' definition of anomalies: anomalies are data that are far away from other observed data and thus suspected to be generated by different mechanisms. Efficient and accurate detection of anomalies is of great significance to the fields of intrusion detection, fraud detection and fault detection. Anomaly detection has been extensively studied, most of the methods are for traditional non-sparse data. However, in actual scenarios, many data are sparse: (1) Short text data: In recent years, with the development of social media, the analysis and mining of short text data has received more and more attention. limited, it becomes extremely challenging to infer anomalous properties of text. (2) Categorical data with "big ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/16G06F17/27G06K9/62
CPCG06F17/16G06F40/30G06F18/2136
Inventor 马帅朱孟笑张晖怀进鹏
Owner BEIHANG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap