Data exception detection method, device and system and storage medium

A technology of data anomaly and detection method, which is applied in the computer field to achieve the effect of improving monitoring efficiency and monitoring accuracy and reducing monitoring cost

Inactive Publication Date: 2019-04-05
RUN TECH CO LTD BEIJING
View PDF4 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

There is no unified data troubleshooting method in the prior art to monitor and troubleshoot abnormal problems such as data loss

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data exception detection method, device and system and storage medium
  • Data exception detection method, device and system and storage medium
  • Data exception detection method, device and system and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0026] figure 1 It is a flowchart of a data anomaly detection method in Embodiment 1 of the present invention. The embodiment of the present invention is suitable for detecting whether there is anomaly in the data received by the data receiving system that receives a large amount of data. The method is executed by a data anomaly detection device, which is implemented by software and / or hardware, and is specifically configured in a processing link device in a data receiving system.

[0027] Such as figure 1 A data anomaly detection method shown includes:

[0028] S110. Receive log data, and count the received amount of the log data.

[0029] Wherein, the log data is the data in the data stream sent to the data receiving system, and the log data includes data packets and / or data records. Wherein, the data packet corresponds to the data unit exchanged and transmitted in the network, that is, the data block sent by the station to the outgoing link device at one time. A data r...

Embodiment 2

[0050] figure 2 It is a flowchart of a data anomaly detection method in Embodiment 2 of the present invention. The embodiments of the present invention carry out subdivision optimization on the basis of the technical solutions of the foregoing embodiments.

[0051] Further, the operation "counting the received amount of the log data" is subdivided into "calculate the log data according to the input attribute of the received log data to obtain the received amount of the log data; wherein, the input Attributes include at least one of the data receiving time period, data type, and data source type”; correspondingly, the operation “counting the output volume of the output data after data processing of the log data” is subdivided into “logging After the data is processed, the output data is counted according to the output attribute to obtain the output volume of the output data; wherein, the output attribute includes at least one of the data generation time period, the data type,...

Embodiment 3

[0072] image 3 It is a structural diagram of a data anomaly detection device in Embodiment 3 of the present invention. The embodiment of the present invention is suitable for detecting whether the data received by the data receiving system receiving a large amount of data is abnormal. The device is implemented by software and / or hardware, and is specifically configured in the processing link equipment in the data receiving system. Such as image 3 The data anomaly detection device shown includes: an input statistics module 310 , an output statistics module 320 and an anomaly determination module 330 .

[0073] The input statistics module 310 is used to receive log data, and count the received amount of the log data;

[0074] An output statistics module 320, configured to count the output volume of the output data after data processing is performed on the log data;

[0075] The abnormality determining module 330 is configured to determine that the current device has data ab...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention discloses a data exception detection method and device, a detection system and a storage medium. The method comprises the steps that log data are received, and the receiving amount of the log data is counted; Counting the output quantity of the output data after the log data is subjected to data processing; And if the data change rate of the output quantity and thereceiving quantity is greater than a set threshold value, determining that the current equipment has data exception. According to the technical scheme, through comparison of the number and / or the flowof the received log data and the output data output after processing the log data, monitoring and troubleshooting of abnormal phenomena such as data loss are achieved, the monitoring cost of the system is reduced, and the monitoring efficiency and the monitoring precision are improved.

Description

technical field [0001] Embodiments of the present invention relate to the field of computer technology, and in particular, to a data anomaly detection method, device, detection system, and storage medium. Background technique [0002] The network is a virtual platform for information transmission, reception, and sharing. Through it, the information of various points, surfaces, and bodies is linked together, so as to realize the sharing of these resources. With the continuous development of technology, the application of the Internet is becoming more and more extensive, and the amount of data generated is increasing day by day. [0003] When warehousing massive Internet data, there is usually a problem of poor data quality due to data loss. Due to the many links of data storage, data loss may occur in each link. In the prior art, there is no unified data troubleshooting method to monitor and troubleshoot abnormal problems such as data loss. Contents of the invention [0...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F11/34
CPCG06F11/3452G06F11/3476
Inventor 万月亮孔庆爱火一莽
Owner RUN TECH CO LTD BEIJING
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap