Unlock instant, AI-driven research and patent intelligence for your innovation.

Abnormal data detection method and system

A technology of abnormal data detection and text data, which is applied in the field of big data of the State Grid, can solve problems such as low accuracy rate and complicated abnormal data detection process, and achieve the effect of improving accuracy rate

Pending Publication Date: 2022-03-08
国家电网有限公司大数据中心
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0009] In order to solve the problem that the detection process of abnormal data is complex and the accuracy rate is low, the present invention provides a method and system for detecting abnormal data, including:

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Abnormal data detection method and system
  • Abnormal data detection method and system
  • Abnormal data detection method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0055] The present invention provides a method for detecting abnormal data, such as figure 1 Shown: includes:

[0056] Step 1: Obtain text data to be classified;

[0057] Step 2: input the text data to be classified into a pre-built classification model to obtain the text data type corresponding to the text data to be classified;

[0058] Wherein, the classification model is constructed based on an n-grams model that preprocesses text data in conjunction with a pre-trained deep neural network model;

[0059] The deep neural network is obtained by using the text data and the n-grams model to preprocess the text data as input, and the type corresponding to the text data as output for training.

[0060] Before step 1 also include:

[0061] The training of the deep neural network includes: constructing a training set by the text data to be classified, the type corresponding to the text data to be classified and the word vector feature generated by the n-grams model preprocessi...

Embodiment 2

[0110] Based on the same inventive concept, the present invention also provides an abnormal data detection system, including:

[0111] A data acquisition module, configured to acquire text data to be classified;

[0112] A classification module, configured to input the text data to be classified into a pre-built classification model to obtain the text data type corresponding to the text data to be classified;

[0113] Wherein, the classification model is constructed based on an n-grams model that preprocesses text data in conjunction with a pre-trained deep neural network model;

[0114] The deep neural network is obtained by using the text data and the n-grams model to preprocess the text data as input, and the type corresponding to the text data as output for training.

[0115] Wherein, the training of described depth neural network comprises:

[0116] Acquiring text data to be classified, and the type corresponding to the text data to be classified;

[0117] Preprocessin...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides an abnormal data detection method and system. The abnormal data detection method comprises the steps of obtaining to-be-classified text data; inputting the to-be-classified text data into a pre-constructed classification model to obtain a text data type corresponding to the to-be-classified text data; wherein the classification model is constructed by combining an n-grams model for preprocessing text data with a pre-trained deep neural network model; the deep neural network is obtained by taking text data and data obtained by preprocessing the text data through an n-grams model as input and taking a type corresponding to the text data as output for training. According to the method, information is rapidly extracted and classified by adopting a text classification technology and an abnormal data detection method, so that the accuracy of abnormal data detection is improved.

Description

technical field [0001] The invention relates to the field of big data of the State Grid, in particular to a method and system for detecting abnormal data. Background technique [0002] The main basis for identifying a device asset is the object type field, but in actual data, spaces, random characters (such as 0, etc.) may be entered in this field, or wrong codes may be entered. In addition, the coding of equipment asset types by different units, different industries and different personnel is not uniform, which makes the problem of inaccurate actual data input more complicated. [0003] The current technical solutions of text classification algorithms include shallow learning to deep learning, and the shallow learning model emphasizes feature extraction and classifier design. Once the text has well-designed features, the classifier can be trained to converge quickly. Without the need for domain knowledge, deep neural networks can automatically perform feature extraction a...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/35G06F16/31G06F40/242G06N3/04G06N3/08
CPCG06F16/35G06F16/316G06F40/242G06N3/04G06N3/08
Inventor 王路涛陈振宇武丽莎杨畅秦明王家凯吕宏伟贾翠玲
Owner 国家电网有限公司大数据中心