Abnormal operation determination method and device, equipment and storage medium

A determination method and operation technology, applied in the field of data processing, can solve the problems of low judgment accuracy of abnormal real-time streaming jobs, reduce the automation degree of abnormal real-time streaming job determination process, and low efficiency of real-time streaming job running process, etc. Precision, the effect of improving accuracy and determining efficiency

Pending Publication Date: 2022-03-25
AGRICULTURAL BANK OF CHINA
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the running process of real-time streaming jobs has the characteristics of burstiness and diversity. In the case of occasional network congestion, message processing failures do not need to be regarded as abnormal jobs, because these messages can still be retransmitted for secondary processing
[0004] Therefore, the accuracy of judging abnormal real-time streaming jobs in the existing technology is low, and human intervention in real-time streaming jobs in the case of inaccurate judgments leads to low efficiency in the running process of real-time streaming jobs, reducing the determination process of abnormal real-time streaming jobs degree of automation

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Abnormal operation determination method and device, equipment and storage medium
  • Abnormal operation determination method and device, equipment and storage medium
  • Abnormal operation determination method and device, equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0024] figure 1 It is a schematic flow diagram of a method for determining abnormal operations provided in Embodiment 1 of the present application. This embodiment is applicable to the situation of abnormal monitoring of real-time stream operations. This method can be executed by an abnormal operation determination device, which can use software and / or hardware implementations such as figure 1 As shown, the method specifically includes the following steps:

[0025] S110. Obtain the number of message processing failures and the average message processing delay of the real-time streaming job within a preset time period.

[0026] Real-time streaming jobs can use streaming data processing methods to process received job tasks. During the processing, the data does not land, and can be processed as soon as it comes. For example, job tasks can be transfers or remittances. During the running of a real-time stream job, processing failure messages are usually generated, and some of th...

Embodiment 2

[0038] figure 2 It is a schematic flowchart of a method for determining an abnormal operation provided in Embodiment 2 of the present application. This embodiment is optimized and improved on the basis of the above-mentioned technical solutions.

[0039]Further, the step "determine the probability of abnormal operation based on the number of message processing failures and the average delay of message processing based on the preset fuzzy logic reasoning rules" is refined into "determine according to the number of message processing failures and the preset message failure probability Algorithm to obtain the first set element in the preset probability set of message failure number; determine the algorithm according to the average message processing delay and the preset average delay level to obtain the second set in the preset average delay high and low probability set element; according to the first set element and the second set element, based on the preset abnormal probabili...

Embodiment 3

[0087] image 3 It is a schematic diagram of the detection process of an abnormal operation determination method provided in Embodiment 3 of the present application. The embodiment of the present application provides a preferred implementation mode on the basis of the technical solutions of the foregoing embodiments.

[0088] In this embodiment, fuzzy logic reasoning is used to avoid direct evaluation of parameters of large-scale online real-time streaming operations, thereby realizing a more general method for determining abnormal operations. By analyzing the abnormal real-time streaming jobs, it can be found that: 1. The amount of abnormal message processing at a certain moment gradually expands; 2. The average message processing delay continues to increase. In this embodiment, the number of message failures and the average message processing delay are used to construct a fuzzy system, and then infer the possibility of abnormality of the current real-time streaming job. The...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention discloses an abnormal job determination method and device, equipment and a storage medium. The method comprises the following steps: acquiring a message processing failure number and a message processing average time delay of a real-time stream job in a preset time period; determining an abnormal job probability based on a preset fuzzy logic reasoning rule according to the message processing failure number and the message processing average time delay; and judging whether the abnormal operation probability is equal to or greater than a preset abnormal operation probability threshold, and if so, determining that the real-time stream operation is an abnormal operation. According to the embodiment of the invention, the method achieves the automatic determination of the abnormal operation, and improves the determination accuracy and efficiency of the abnormal operation.

Description

technical field [0001] The embodiments of the present application relate to the technical field of data processing, and in particular, to a method, device, device, and storage medium for determining an abnormal operation. Background technique [0002] With the development of big data technology, enterprises have higher and higher requirements on the timeliness of data processing. More and more scenarios use real-time stream computing to improve the timeliness of data processing. The scale of real-time stream computing continues to grow, and the corresponding , it is becoming more and more important to judge the abnormality of the real-time streaming job running status. [0003] In the prior art, the following processing methods are usually adopted to determine whether the running state of a real-time streaming job is abnormal: if a message processing failure occurs during the running of the real-time streaming job, it is considered that the real-time streaming job is abnorma...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/54
CPCG06F9/546
Inventor 钟新斌
Owner AGRICULTURAL BANK OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap