Batch operation data monitoring method and device, electronic equipment and storage medium

Abstract

The invention provides a batch operation data monitoring method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a plurality of pieces of original operation data in a preset database and a plurality of pieces of operation data transmitted from a preset interface; performing standardization processing on the plurality of original operation data and the plurality of operation data according to a predetermined format to obtain a plurality of pieces of standardized operation information; judging whether the plurality of standardized operation information triggers alarm or not according to an alarm threshold value; in response to the trigger alarm, monitoring and analyzing other standardized operation information having an association relationship with the abnormal standardized operation information corresponding to the trigger alarm; and generating corresponding alarm information in response to the analysis result that the other standardized operation information is abnormal. According to the invention, the problem source and the abnormal influence of the alarm can be efficiently analyzed.

Description

technical field [0001] The invention relates to the field of information processing, in particular to a batch job data monitoring method, device, electronic equipment and storage medium. Background technique [0002] With the vigorous development of the Internet economy and the prosperity of the commercial society, a large amount of transaction data has been generated. In order to meet the summary and statistical needs of massive data, enterprises and departments such as finance, retail, electric power, and communications have adopted a large number of data batch processing methods to process business data, and the results are widely used in regulatory audits, business decisions, daily operations, etc. In important production activities, it is particularly important to monitor and manage the accuracy and timeliness of data batch processing. [0003] At present, the monitoring of data batch processing is usually implemented by the business system itself or by a special data ...

AI Technical Summary

Problems solved by technology

[0004] (1) The data batch monitoring of different business systems is isolated from each other, making it difficult to effectively comprehensively utilize

Generally speaking, for a large organization, there will be multiple business systems at the same time, and there are complex correlations between their data batch processing. If a single data batch processing job is abnormal, it is generally related to the operation results of the upstream system. At the same time, this exception will also affect the operation of downstream associated jobs.

The existing data batch processing monitoring system fails to aggregate and integrate related job operations to support operation and maintenance personnel to automatically and efficiently conduct comprehensive problem root analysis, abnormal impact assessment and Disposal strategy gold decision

[0005] (2) In order to achieve the purpose of analysis, it is necessary to collect relevant data of data batch processing from the business system. The traditional monitoring method needs to carry out invasive program transformation on the data batch operation program and data batch processing scheduling system of the business system

For the business system developed by the organization itself or customized, in order to provide the monitoring system with data batch processing details, abnormal information, etc., it is necessary to adjust the system function and develop a specific data interface, which has a large transformation cost; for closed-source commercial data batch processing Scheduling tools are subject to the vendor's openness to product control, and sometimes it is difficult to monitor the data batches they provide as required

Images