Method, device, equipment, medium and product for processing monitoring data

Abstract

Embodiments of the present application provide a kind of processing method, device, equipment, medium and product of monitoring data, it is related to big data field or other related fields.The method comprises: according to the request task sent by request end, the monitoring data table of monitoring data in preset alarm period is acquired, monitoring data table includes the alarm time, event type and application name of monitoring data;According to the alarm time of monitoring data, determine the alarm number of monitoring data in different alarm sub-period;According to the alarm number of monitoring data in different alarm sub-period, and the dimension information corresponding to event type and / or application name in request task, the monitoring data in monitoring data table is sorted, and monitoring data list is obtained;According to the extraction requirement in request task, extract target monitoring data in monitoring data from monitoring data list, generate and send alarm information of target monitoring data.The method of the present application improves the data processing effect when facing a large number of monitoring data.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of big data, and in particular to a monitoring data processing method and device, equipment, medium and product. BACKGROUND

[0002] In modern IT operation management, monitoring statistical models are crucial for ensuring the stability and reliability of application systems. These models analyze monitoring message information from different channels (such as availability, transactions, etc.), capture system state changes in real time, warn of potential problems, and assist the operation team in rapid response. However, as the system complexity increases, traditional monitoring models face challenges in handling multi-dimensional data and responding to sudden failures.

[0003] Current monitoring statistical models mainly work on a single dimension of "alarm number", and periodically aggregate monitoring messages (such as availability, batch, transaction, and event monitoring) reported by each application server to count the number of alarms within a specific time range. This method quantifies alarm frequency to identify system risks or abnormalities, providing a simple and intuitive reference for operation decision-making, and is currently a common monitoring governance method.

[0004] Although the statistical model based on alarm number can meet the basic monitoring needs, its effectiveness and accuracy are limited when dealing with a large number of short-term alarms caused by sudden failures, increasing the operation burden and leading to poor results in processing monitoring data SUMMARY

[0005] The embodiments of the present application provide a monitoring data processing method, device, equipment, medium and product to improve the processing effect of monitoring data.

[0006] In a first aspect, the embodiments of the present application provide a monitoring data processing method, comprising:

[0007] According to the request task sent by the request end, a monitoring data table of monitoring data in a preset alarm period is obtained, and the monitoring data table includes the alarm time, event type and application name of the monitoring data;

[0008] According to the alarm time of the monitoring data, the number of alarms of the monitoring data in different alarm sub-periods is determined;

[0009] According to the number of alarms of the monitoring data in different alarm sub-periods and the dimension information corresponding to the event type and / or application name in the request task, the monitoring data in the monitoring data table is sorted to obtain a monitoring data list;

[0010] According to the extraction requirement in the request task, target monitoring data in the monitoring data list and alarm information generated and sent by the target monitoring data are extracted.

[0011] In a possible implementation, according to a request task sent by a request end, a monitoring data table of monitoring data in a preset alarm period is acquired, the monitoring data table including alarm time, event type and application name of the monitoring data, and the method includes:

[0012] According to the request task, initial monitoring data in the preset alarm period is acquired;

[0013] The alarm time, event type and application name in the initial monitoring data are written into a preset first database table, and an initial monitoring data table is obtained;

[0014] The initial monitoring data table is copied to a preset second database table, and the initial monitoring data in the second database table is processed to remove duplication, and a monitoring data table is obtained.

[0015] In a possible implementation, the initial monitoring data table is copied to a preset second database table, and the initial monitoring data in the second database table is processed to remove duplication, and a monitoring data table is obtained, and the method includes:

[0016] The initial monitoring data table is copied to a preset second database table;

[0017] According to the application name, event type and alarm time in the initial monitoring data in the second database table, the initial monitoring data in the second database table is processed to remove duplication, and a monitoring data table is obtained.

[0018] In a possible implementation, according to the alarm number of the monitoring data in different alarm sub-periods and the dimension information corresponding to the event type and / or application name in the request task, the monitoring data in the monitoring data table is sorted, and a monitoring data list is obtained, and the method includes:

[0019] According to the alarm number of the monitoring data in different alarm sub-periods, the monitoring data in the monitoring data table is initially sorted, and an initial monitoring data list is obtained;

[0020] According to the dimension information corresponding to the event type and / or application name in the request task, the monitoring data in the initial monitoring data list is adjusted, and a monitoring data list is obtained.

[0021] In a possible implementation, according to the dimension information corresponding to the event type and / or application name in the request task, the monitoring data in the initial monitoring data list with the same alarm number but different corresponding dimension information is adjusted, and a monitoring data list is obtained, and the method includes:

[0022] According to the type of the request end and the dimension information corresponding to the event type and / or application name in the request task, the sorting priority corresponding to different dimension information is acquired;

[0023] According to the sorting priority corresponding to the different dimension information, and according to the dimension information corresponding to the event type and / or application name in the request task, the monitoring data in the initial monitoring data list with the same alarm times but different corresponding dimension information is adjusted and processed to obtain the monitoring data list.

[0024] In a possible implementation, according to the dimension information corresponding to the event type and / or application name in the request task, the monitoring data in the initial monitoring data list with the same alarm times but different corresponding dimension information is adjusted and processed to obtain the monitoring data list, including: obtaining the sorting priority parameters corresponding to different dimension information according to the request task sent by the request end and the dimension information corresponding to the event type and / or application name in the request task.

[0025] According to the sorting priority corresponding to the different dimension information, and according to the dimension information corresponding to the event type and / or application name in the request task, the monitoring data in the initial monitoring data list with the same alarm times but different corresponding dimension information is adjusted and processed to obtain the monitoring data list.

[0026] In a possible implementation, according to the extraction requirement in the request task, the target monitoring data in the monitoring data is extracted from the monitoring data list, and the alarm information of the target monitoring data is generated and sent, including:

[0027] According to the request task sent by the request end, the extraction requirement of the extraction quantity and dimension information in the monitoring data list is determined;

[0028] According to the extraction requirement of the extraction quantity and dimension information in the monitoring data list, the target monitoring data in the monitoring data is extracted from the monitoring data list, and the alarm information of the target monitoring data is generated and sent.

[0029] In a possible implementation, the dimension information includes at least one of the availability monitoring information, the batch monitoring information, the transaction monitoring information, and the event monitoring information.

[0030] In a second aspect, an embodiment of the present application provides a monitoring data processing apparatus, including:

[0031] The acquisition module is configured to acquire a monitoring data table of the monitoring data in a preset alarm period according to the request task sent by the request end, and the monitoring data table includes alarm time, event type and application name of the monitoring data.

[0032] The determination module is configured to determine the alarm times of the monitoring data in different alarm sub-periods according to the alarm time of the monitoring data.

[0033] The obtaining module is configured to sort the monitoring data in the monitoring data table according to the alarm times of the monitoring data in different alarm sub-periods and the dimension information corresponding to the event type and / or the application name in the request task, to obtain a monitoring data list.

[0034] The sending module is configured to extract target monitoring data from the monitoring data list according to the extraction requirement in the request task, and generate and send alarm information of the target monitoring data.

[0035] In a third aspect, an electronic device is provided, including a memory and a processor.

[0036] The memory stores computer-executable instructions.

[0037] The processor executes the computer-executable instructions stored in the memory to implement the method of the present application.

[0038] In a fourth aspect, a computer-readable storage medium is provided, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, the computer-executable instructions are used to implement the first aspect and / or various possible implementation manners of the first aspect.

[0039] In a fifth aspect, a computer program product is provided, which includes a computer program. When the computer program is executed by a processor, the computer program implements the method of the present application.

[0040] The monitoring data processing method, device, equipment, medium and product provided by the present application reduce the data processing amount by obtaining a monitoring data table of monitoring data composed of alarm time, event type and application name in a preset alarm period, sort the monitoring data in the monitoring data table according to the alarm times of the monitoring data in different alarm sub-periods and the dimension information corresponding to the event type and / or the application name in the request task, to obtain a monitoring data list, thereby further simplifying the data amount in the data processing process. Finally, the target monitoring data is selected from the monitoring data list according to the extraction requirement in the request task, and the alarm information is generated and sent. The technical means improves the effectiveness and accuracy when a large number of short-term alarms caused by sudden failures are processed, reduces the operation and maintenance burden, and solves the problem of poor effect when the monitoring data is processed. BRIEF DESCRIPTION OF DRAWINGS

[0041] The accompanying drawings, which are incorporated into and form a part of the specification, illustrate one embodiment consistent with the present application and, together with the description, serve to explain the principles of the application.

[0042] Figure 1 A scene schematic diagram of a monitoring data processing method provided by the present application embodiment.

[0043] Figure 2 A flowchart of a processing method of monitoring data provided for an embodiment of the present application Figure 1 ;

[0044] Figure 3 A flowchart of a processing method of monitoring data provided for an embodiment of the present application Figure 2 ;

[0045] Figure 4 A structural diagram of a processing device of monitoring data provided for an embodiment of the present application

[0046] Figure 5 A structural diagram of an electronic device provided for an embodiment of the present application.

[0047] The specific embodiments of the present application have been shown through the above-described drawings, and will be described in more detail hereinafter. These drawings and written descriptions are not intended to limit the scope of the concept of the present application by any means, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0048] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The same reference numbers in different drawings represent the same or similar elements unless otherwise represented. The embodiments described in the following exemplary embodiments do not represent all the embodiments consistent with the present application. Rather, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0049] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards of relevant countries and regions, take necessary security measures, do not violate public order and good customs, and provide corresponding operation portal for user to choose authorization or refusal.

[0050] And the present application involves big data analysis of user information (including but not limited to personal biological characteristics, identity data, consumption data, asset data, electronic terminal operation data, etc.), and uses artificial intelligence technology for automatic decision-making, and makes technical solutions based on automatic decision-making results that have a significant impact on personal rights and interests, provides corresponding operation portal for users to choose to agree or refuse automatic decision-making results; if the user chooses to refuse, the expert decision-making process is entered.

[0051] It should be noted that the processing method, device, equipment, medium and product provided by the embodiments of the present application can be used in the field of big data, and can also be used in any field other than big data. The application field of the processing method, device, equipment, medium and product of monitoring data in the present application is not limited.

[0052] First, the terms involved in the present application are explained:

[0053] Availability monitoring: sending availability messages at a preset period, triggering an availability alarm if no message is received within the preset period;

[0054] Batch monitoring: sending messages when the task starts and ends, triggering a batch alarm if no message is received within a specified time;

[0055] Transaction monitoring: monitoring the status of transactions during online transactions, sending messages containing transaction codes, return codes, IP addresses, channels, etc. in real time, and triggering a transaction alarm if the processed information exceeds the set threshold;

[0056] Event monitoring: self-defined abnormal monitoring, the self-defined condition contains an abnormal triggering condition, and the event alarm is triggered if the abnormal triggering condition is met.

[0057] In the prior art, the monitoring system usually gives a warning based on a pre-set threshold when an actual fault occurs. However, a large number of alarm information may be caused by data deviation in a short period of time. The intensive alarms will affect the recognition ability of the monitoring system, and the number of alarms as a measure of the monitoring system has lost its reference value. In addition, a large number of alarm information also increases the maintenance cost of the maintenance personnel.

[0058] The processing method of monitoring data provided by the embodiments of the present application, by analyzing the time period to be monitored according to the request task sent by the user at the request end, taking it as a preset alarm period, taking the monitoring data in the preset alarm period as the alarm time, event type and application name corresponding to the request task of the user as the standard for judging the warning, then dividing the preset alarm period into different alarm sub-periods, thereby accurately calculating the number of alarms, combining the number of alarms and the dimension information concerned by the user, sorting the monitoring data, and finally generating the corresponding warning information according to the sorted monitoring data, solves the problem of a large number of alarm information affecting the judgment.

[0059] Figure 1 The scene diagram of the processing method of monitoring data provided by the embodiments of the present application is shown in Figure 1 The execution subject of the method can be a processing system of monitoring data, which can be a server. The server can be a computer, a notebook, a tablet or a mobile phone, etc.

[0060] The implementation of the execution subject is not particularly limited in this embodiment, as long as the execution subject can obtain a monitoring data table of monitoring data in a preset alarm period according to a request task sent by a request end, the monitoring data table including an alarm time, an event type and an application name of the monitoring data; determine the number of alarms of the monitoring data in different alarm sub-periods according to the alarm time of the monitoring data; sort the monitoring data in the monitoring data table according to the number of alarms of the monitoring data in different alarm sub-periods and dimension information corresponding to the event type and / or the application name in the request task, to obtain a monitoring data list; and extract target monitoring data in the monitoring data and generate and send alarm information of the target monitoring data from the monitoring data list according to extraction requirements in the request task.

[0061] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described again in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0062] Figure 2 Flowchart of a monitoring data processing method provided by an embodiment of the present application Figure 1 As shown in Figure 2 , the method comprises:

[0063] S201, obtaining a monitoring data table of monitoring data in a preset alarm period according to a request task sent by a request end, the monitoring data table including an alarm time, an event type and an application name of the monitoring data.

[0064] The request end can refer to a client or a system that initiates a request, which can be any type of software application, service or hardware device. For example, the request end can be a monitoring management platform through which an administrator sets monitoring rules and views alarm information, or an automated script used to periodically check the health status of a system.

[0065] The request task can refer to an instruction or command sent by the request end, used to instruct the operation of the execution demand. The request task can include specific parameters for processing monitoring data, such as the event type, the application name, and how to sort, filter and extract the monitoring data according to the event type and the application name.

[0066] The preset alarm period can refer to a predefined time period, and the data collected within this time period will be used to evaluate whether an alarm condition is triggered. For example, if the preset alarm period is one day, all monitoring data in the past 24 hours will be analyzed to determine whether any abnormal situation has occurred, and whether an alarm needs to be sent accordingly.

[0067] The monitoring data can refer to various indicator information collected from the monitored object, the monitored object can refer to a server, an application program, a network device, and the monitoring data usually reflects the running state or performance of the object, such as CPU utilization, memory usage, network traffic, and error logs.

[0068] The monitoring data table can refer to a structured form of storing monitoring data, which can exist in the form of a table, where each row represents a separate data record, and each column corresponds to different attributes or fields, such as alarm time and alarm location, facilitating the organization and management of a large amount of monitoring data. The monitoring data table can be obtained in various ways, such as directly collected from a monitoring system, many application programs and services come with monitoring functions, which can generate detailed logs and performance indicators, and through built-in tools, the monitoring data can be directly exported or queried and formatted into a monitoring data table. For example, for target application scenarios, scripts are written to automatically collect and organize log data, and then create a monitoring data table.

[0069] In the embodiments of the present application, the monitoring data table includes the alarm time, event type and application name of the monitoring data, wherein the alarm time can refer to the exact time when the system detects an abnormal situation and triggers an alarm, and the alarm time is represented as a timestamp, accurate to seconds or finer time units, used to identify the specific time point when the problem occurs.

[0070] The event type can refer to the classification of different types of events captured, including system errors, resource overload, security intrusions and other types of events. Each event type represents a behavior or state change that may affect the normal operation of the system.

[0071] The application name can refer to a key identifier that distinguishes applications and services, used to distinguish the monitored application programs or services, so as to ensure that the monitoring data can be accurately associated with a specific application.

[0072] In the embodiments of the present application, the data source of the monitoring data table is the monitoring message information uploaded by each application server, such as a business operation management platform or a cross-border clearing platform.

[0073] S202, according to the alarm time of the monitoring data, determine the alarm number of the monitoring data in different alarm sub-periods.

[0074] The alarm sub-period can refer to a time period divided in a preset alarm period, which can be a time unit of different lengths such as minutes, hours, days, etc., and the specific division depends on the monitoring requirements and business characteristics, so as to accurately find abnormal situations in specific time periods such as night.

[0075] The number of alarms can refer to the number of alarm events occurring within a specified time range. The number of alarms can be the number of alarms within a complete preset alarm period, or the number of alarms within an alarm sub-period.

[0076] In the embodiments of the present application, the alarm sub-period can be set according to actual needs to adapt to different monitoring strategies and business scenarios. For example, for the purpose of reducing frequent alarms caused by temporary and repeated problems, a shorter alarm sub-period (such as every 15 minutes or 30 minutes) can be selected. For example, for real-time monitoring of a key transaction system, a finer-grained alarm sub-period (such as every 5 minutes or even shorter) can be selected to ensure that any anomaly can be captured and responded to in a timely manner. For example, for non-critical systems or long-term trend analysis, a longer alarm sub-period (such as every hour or every 4 hours) can be selected.

[0077] In some embodiments, different alarm sub-periods are divided within a preset alarm period, and the number of alarms in different alarm sub-periods is determined, so as to improve the case that scattered multiple alarms are regarded as multiple independent events. For example, in the past 24 hours, the system has produced 10 separate availability alarms due to temporary network fluctuations, each interval is a few minutes, after setting every hour as an alarm sub-period, it is stipulated that multiple alarms of the same type occurring within an alarm sub-period are counted as one, even if there are 10 temporary alarms in a day, but because they all occur within the same hour, only one alarm will be recorded within the entire 24-hour period.

[0078] S203, according to the number of alarms in different alarm sub-periods and the dimension information corresponding to the event type and / or application name in the request task, sorting the monitoring data in the monitoring data table to obtain a monitoring data list.

[0079] The dimension information can refer to the attributes or characteristics of the monitoring data for description or classification, so as to facilitate understanding and analysis of the data and identification of trends or abnormalities under specific conditions. In the embodiments of the present application, the dimension information can be divided into four categories: "availability monitoring", "batch monitoring", "transaction monitoring" and "event monitoring", and the four categories are used for multi-level monitoring of the system.

[0080] The monitoring data list can refer to a set of monitoring data sorted and filtered according to certain rules, which can be presented in the form of a table, each row representing an independent data record, and the columns corresponding to different dimension information or other important indicators. The monitoring data list can sort or filter the data according to different dimension information, and summarize the number of alarms of the same type.

[0081] In the embodiments of the present application, the monitoring data in the monitoring data table is sorted according to the number of alarms of the monitoring data in different alarm sub-periods and the dimension information corresponding to the event type and / or application name in the request task, to obtain a monitoring data list, including:

[0082] The monitoring data in the monitoring data table is initially sorted according to the number of alarms of the monitoring data in different alarm sub-periods, to obtain an initial monitoring data list;

[0083] The monitoring data in the initial monitoring data list is adjusted according to the dimension information corresponding to the event type and / or application name in the request task, to obtain the monitoring data list.

[0084] The initial monitoring data list can refer to a data set obtained by initially sorting the records in the monitoring data table based on the number of alarms of the monitoring data in different alarm sub-periods, which can help to quickly identify which events frequently occur in a specific time period.

[0085] The adjustment process can refer to a process of further filtering and reordering the monitoring data in the initial monitoring data list according to the dimension information specified in the request task, to finally obtain the monitoring data list, ensuring that the final result is more in line with specific needs and improving the relevance and practicality of the data.

[0086] For example, according to the application name and event type grouping, the number of last occurrence dates can be calculated, which can be implemented by using the select statement to arrange the corresponding data entries in descending or ascending order.

[0087] In some embodiments, the monitoring data in the initial monitoring data list with the same number of alarms but different corresponding dimension information is adjusted according to the dimension information corresponding to the event type and / or application name in the request task, to obtain the monitoring data list, including:

[0088] According to the type of the request end and the dimension information corresponding to the event type and / or application name in the request task, the sorting priority corresponding to different dimension information is obtained;

[0089] The monitoring data in the initial monitoring data list with the same number of alarms but different corresponding dimension information is adjusted according to the sorting priority corresponding to different dimension information and the dimension information corresponding to the event type and / or application name in the request task, to obtain the monitoring data list.

[0090] The type of the request end can refer to the kind of user interface or system that initiates the monitoring data request, for further distinguishing the priority of processing, for example, the web end can be used for internal employees to view and manage monitoring data, the internal tools used by the web end can have different priority settings than the public interface used by external users, while combining with specific dimension information to determine which dimension information should be prioritized, thereby corresponding to different sorting priorities.

[0091] The sorting priority is a relatively fixed order, which is usually pre-set by the system or administrator according to common business logic, suitable for most regular scenarios, and provides a stable data sorting method.

[0092] In some embodiments, the monitoring data list is obtained by adjusting the monitoring data with the same number of alarms but different corresponding dimension information in the initial monitoring data list according to the dimension information corresponding to the event type and / or application name in the request task, including: obtaining the sorting priority parameters corresponding to different dimension information according to the request task sent by the request end and the dimension information corresponding to the event type and / or application name in the request task;

[0093] The monitoring data list is obtained by adjusting the monitoring data with the same number of alarms but different corresponding dimension information in the initial monitoring data list according to the sorting priority corresponding to different dimension information and according to the dimension information corresponding to the event type and / or application name in the request task.

[0094] The sorting priority parameter can refer to a specific numerical value or rule for quantifying the importance of different dimension information. The sorting priority parameter is usually pre-configured or dynamically generated according to the request task, and can be a more dynamic and flexible indicator, so that the system can more accurately respond to the individual needs of users, especially when facing complex or special monitoring requirements.

[0095] S204, extracting target monitoring data from the monitoring data list according to the extraction requirements in the request task, and generating and sending alarm information of the target monitoring data.

[0096] The extraction requirement can refer to a series of standards or rules set for filtering data that meets specific conditions from a large amount of monitoring data, which specifies which data needs to be obtained, how to process these data and the final output form. The extraction requirement is usually defined by the request end according to specific business needs or problem diagnosis purposes.

[0097] The target monitoring data can refer to a data set obtained by filtering according to the extraction requirements from a large amount of original monitoring data. The target monitoring data can present the monitoring results in the form of tables, charts, etc., or can be combined with other data sets for further analysis of the monitoring situation.

[0098] Alert information can refer to a notification or warning generated when an abnormal situation is detected during monitoring, containing specific descriptions of the problem, the time of occurrence, the objects involved, and suggested response measures.

[0099] In the embodiments of the present application, the target monitoring data in the monitoring data and the alert information of the target monitoring data are extracted from the monitoring data list according to the extraction requirements in the request task, which includes:

[0100] According to the request task sent by the request end, the extraction requirements of the extraction quantity and dimension information in the monitoring data list are determined;

[0101] According to the extraction requirements of the extraction quantity and dimension information in the monitoring data list, the target monitoring data in the monitoring data and the alert information of the target monitoring data are extracted from the monitoring data list. The extraction quantity in the monitoring data list can refer to the maximum number of data entries that need to be extracted from the monitoring data list, which helps to limit the amount of returned data, improve query efficiency, and prevent too much data from causing analysis difficulties.

[0102] The extraction requirements of the dimension information can refer to specific fields or attributes that users are interested in, which are used for further filtering and refining data.

[0103] For example, according to the request task sent by the request end, all "performance alarm" records of the "WebApp" application in the last 7 days are extracted from the monitoring data list, with a maximum of 100 records. The target monitoring data that meets the conditions is filtered according to these extraction requirements, and detailed alarm information (including alarm time, event type and description) is generated for each record, and then sent to the designated members through email, ensuring that they can respond and handle performance problems in a timely manner.

[0104] The processing method of monitoring data provided by the embodiments of the present application first acquires a monitoring data table within a preset alarm period according to the request task sent by the request end, which contains alarm time, event type and application name. Then, the system counts the alarm times of these monitoring data in different alarm sub-periods. Then, according to the alarm times and the dimension information corresponding to the event type and / or application name specified in the request task, the monitoring data is sorted to generate an optimized monitoring data list. Finally, the target monitoring data is filtered from the monitoring data list according to the extraction requirements in the request task, and the corresponding alarm information is generated and sent to the relevant personnel. Not only the accuracy and timeliness of the monitoring data are ensured, but also the relevance and practicality of the data are improved through personalized sorting and filtering, while ensuring that the relevant maintenance personnel focus on specific applications or event types.

[0105] Figure 3 A flowchart of a processing method of monitoring data provided by an embodiment of the present application Figure 2 As shown in Figure 3 The embodiment is based on Figure 2 The processing method of monitoring data is described in detail. According to the request task sent by the request end, the monitoring data table of the monitoring data in the preset alarm period is obtained, and the monitoring data table includes the alarm time, event type and application name of the monitoring data, which includes:

[0106] S301, according to the request task, obtaining initial monitoring data in a preset alarm period;

[0107] S302, write the alarm time, event type and application name in the initial monitoring data into the preset first database table to obtain the initial monitoring data table.

[0108] The initial monitoring data can refer to the original monitoring information collected directly from various monitoring sources. The initial monitoring data usually contains rich details, but may contain repeated items or redundant information, and has not been processed or optimized, so further processing is needed for efficient analysis and decision-making.

[0109] The preset first database table can refer to a database table used to receive and store specific attributes in the initial monitoring data, which is the starting point of the entire data processing chain and is responsible for saving the original monitoring information without processing or optimization. In the embodiment of the present application, the preset first database table can be the initial monitoring data table, which ensures that all monitoring data is stored in a consistent manner.

[0110] The initial monitoring data table can be directly obtained from the initial monitoring data, and the obtaining method can be through a filtering statement, a command and a formula. In the embodiment of the present application, the data is exported from the database of the monitoring alarm data system by using a table application software and stored in the form of a table as the initial monitoring data. The alarm date is filtered from 7 days ago to the current date, and from 0 o'clock that day to 0 o'clock that day, and the corresponding event type and application name are obtained. Among them, OSCM_DATA is used as the storage name of the initial monitoring data table, and the initial monitoring data table contains 3 fields app_name, evnet_name and event_date, which represent "application name", "event type" and "alarm time" respectively.

[0111] For example, selecting alarm records occurring between 2024-12-03 00:00:00 and 2024-12-10 00:00:00, there can be an availability alarm information of 2024-12-03 03:15, occurring in WebApp, and a transaction failure alarm information of 2024-12-07 19:20, occurring in WebApp.

[0112] S303, copy the initial monitoring data table to the preset second database table, and perform deduplication processing on the initial monitoring data in the second database table to obtain a monitoring data table.

[0113] The preset second database table can refer to receiving data copied from the first database table. The preset second database table is used to store data after preliminary processing, so as to facilitate setting of a new processing area, so that the data before and after processing are separated and completely retained.

[0114] Deduplication processing can refer to a process of removing duplicate data records in the second database table based on specific key fields, so as to ensure that each data entry is unique, thereby improving the accuracy and efficiency of data analysis. Deduplication processing can be implemented in various ways, such as using the filtering function of the table, or using the filtering command editing window of the tool, which is not limited in the present application.

[0115] In the embodiments of the present application, the initial monitoring data table is copied to the preset second database table, and the initial monitoring data in the second database table is deduplicated to obtain a monitoring data table, including:

[0116] Copy the initial monitoring data table to the preset second database table.

[0117] According to the application name, event type and alarm time in the initial monitoring data in the second database table, the initial monitoring data in the second database table is deduplicated to obtain a monitoring data table. Copying the initial monitoring data table to the preset second database table is a direct data migration, which ensures that all fields and records are accurately transferred to the new table.

[0118] The deduplication processing uses the application name, event type and alarm time because these three fields together define the unique identification of the monitoring event, which can effectively distinguish different alarm events. For example, for the data entries stored in the second database table, group them according to the application name, event type and alarm time, and when the number of data in the group exceeds 1, delete the duplicate records and only keep one data entry.

[0119] The processing method of monitoring data provided by the embodiment of the application comprises the following steps: obtaining initial monitoring data in a preset alarm period according to a request task sent by a request end, writing alarm time, event type and application name in the data into a preset first database table to form an initial monitoring data table, copying the initial monitoring data table to a preset second database table, and performing deduplication processing on the data in the second database table based on the application name, the event type and the alarm time to ensure that only one record is retained for each unique monitoring event, and finally generating an optimized monitoring data table. This process not only improves the accuracy and consistency of the data, but also simplifies the subsequent data analysis work.

[0120] Table 1 is a result schematic table 1 of the processing method of monitoring data provided by the embodiment of the application, as shown in Table 1, in the preset alarm period 2024.03.10-2024.03.16, the event monitoring repetition degree statistics are that the same alarm days of the self-service terminal are 5 days, the same alarm days of the natural language processing are 5 days, the same alarm days of the intelligent decision platform are 5 days, and the results of multiple applications such as the self-service terminal, the natural language processing and the intelligent decision platform can also be seen.

[0121]

[0122] Table 2 is a result schematic table 2 of the processing method of monitoring data provided by the embodiment of the application, as shown in Table 2, in the preset alarm period 2024.03.10-2024.03.16, the batch monitoring repetition degree statistics are that the same alarm days of the bank-enterprise service are 6 days, the same alarm days of the global cash management are 6 days, the same alarm days of the bond transaction are 5 days, and the results of multiple applications such as the bank-enterprise service, the global cash management and the bond transaction can also be seen.

[0123]

[0124] Figure 4 The structure schematic diagram of the processing device of monitoring data provided by the embodiment of the application is shown in Figure 4 The processing device 40 of monitoring data provided by the embodiment comprises:

[0125] The obtaining module 401 is configured to obtain a monitoring data table of monitoring data in a preset alarm period according to a request task sent by a request end, and the monitoring data table comprises alarm time, event type and application name of the monitoring data.

[0126] The determining module 402 is configured to determine the alarm times of the monitoring data in different alarm sub-periods according to the alarm time of the monitoring data.

[0127] The obtaining module 403 is configured to sort the monitoring data in the monitoring data table according to the number of alarms of the monitoring data in different alarm sub-periods and the dimension information corresponding to the event type and / or the application name in the request task, to obtain a monitoring data list;

[0128] The sending module 404 is configured to extract target monitoring data from the monitoring data list according to the extraction requirement in the request task, and generate and send alarm information of the target monitoring data.

[0129] In the embodiments of the present application, the obtaining module 401 can be specifically configured to:

[0130] obtain initial monitoring data in a preset alarm period according to the request task;

[0131] write the alarm time, event type and application name in the initial monitoring data into a preset first database table to obtain an initial monitoring data table;

[0132] copy the initial monitoring data table to a preset second database table, and perform a deduplication process on the initial monitoring data in the second database table to obtain a monitoring data table.

[0133] In the embodiments of the present application, the obtaining module 401 can be specifically configured to:

[0134] copy the initial monitoring data table to a preset second database table;

[0135] perform a deduplication process on the initial monitoring data in the second database table according to the application name, event type and alarm time in the initial monitoring data in the second database table to obtain a monitoring data table. In the embodiments of the present application, the obtaining module 403 can be specifically configured to:

[0136] perform an initial sorting on the monitoring data in the monitoring data table according to the number of alarms of the monitoring data in different alarm sub-periods to obtain an initial monitoring data list;

[0137] perform an adjustment process on the monitoring data in the initial monitoring data list according to the dimension information corresponding to the event type and / or the application name in the request task to obtain a monitoring data list.

[0138] In the embodiments of the present application, the obtaining module 403 can be specifically configured to:

[0139] obtain the sorting priority corresponding to different dimension information according to the type of the request end and the dimension information corresponding to the event type and / or the application name in the request task;

[0140] According to the sorting priority corresponding to the different dimension information, and according to the dimension information corresponding to the event type and / or application name in the request task, the monitoring data in the initial monitoring data list with the same alarm number but different corresponding dimension information is adjusted and processed to obtain the monitoring data list.

[0141] In the embodiment of the application, the obtaining module 403 can be specifically used for:

[0142] According to the request task sent by the request end and the dimension information corresponding to the event type and / or application name in the request task, the sorting priority parameters corresponding to the different dimension information are obtained.

[0143] According to the sorting priority corresponding to the different dimension information, and according to the dimension information corresponding to the event type and / or application name in the request task, the monitoring data in the initial monitoring data list with the same alarm number but different corresponding dimension information is adjusted and processed to obtain the monitoring data list.

[0144] In the embodiment of the application, the sending module 404 can be specifically used for:

[0145] According to the request task sent by the request end, the extraction quantity and the extraction requirement of the dimension information in the monitoring data list are determined.

[0146] According to the extraction quantity and the extraction requirement of the dimension information in the monitoring data list, the target monitoring data in the monitoring data is extracted from the monitoring data list, and the alarm information of the target monitoring data is generated and sent. The monitoring data processing device provided in the embodiment can execute the method provided in the above method embodiment, and has similar implementation principles and technical effects. Herein, the embodiment will not be repeated.

[0147] Figure 5 The structure schematic diagram of the electronic device provided in the embodiment of the application is shown in FIG. 5. As shown in FIG. 5, the electronic device 50 provided in the embodiment of the application includes at least one processor 501 and a memory 502. Optionally, the device 50 further includes a communication component 503. The processor 501, the memory 502 and the communication component 503 are connected through a bus 504. Figure 5

[0148] In the specific implementation process, the at least one processor 501 executes the computer execution instructions stored in the memory 502, so that the at least one processor 501 executes the above method.

[0149] The specific implementation process of the processor 501 can refer to the above method embodiment, and has similar implementation principles and technical effects. Herein, the embodiment will not be repeated.

[0150] ​In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in combination with the application can be directly embodied as hardware processor execution, or executed by a combination of hardware and software modules in the processor.

[0151] The memory can include a random access memory (RAM), and can also include a non-volatile memory (NVM), such as at least one disk memory.

[0152] The bus can be an industry standard architecture (ISA) bus, a peripheral component (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application does not limit to only one bus or one type of bus.

[0153] The embodiments of the present application also provide a computer program product, comprising a computer program, which is executed by a processor to implement the above method.

[0154] The embodiments of the present application also provide a computer readable storage medium, which stores computer execution instructions, and when a processor executes the computer execution instructions, the above method is implemented.

[0155] The above readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0156] An example readable storage medium is coupled to the processor such that the processor can read information from the readable storage medium and can write information to the readable storage medium. Of course, the readable storage medium can also be a part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in the device.

[0157] The division of units is only a logical functional division, and in actual implementation, there can be another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0158] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0159] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0160] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the part of the present application that essentially contributes to the prior art or the part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0161] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The foregoing program can be stored in a computer readable storage medium. The program executes to perform the steps of the above-mentioned method embodiments; and the foregoing storage medium includes various media capable of storing program codes, such as ROM, RAM, magnetic disk, or optical disk.

[0162] Finally, it should be noted that other embodiments of the present application will readily occur to those skilled in the art upon consideration of the specification and practice of the present application disclosed herein. The present application is intended to include all such variations as fall within the general scope of the application, and includes the generic principles disclosed and the best mode known to the inventors to be currently practiced as well as variations thereof, without departing from the scope of the present application as defined by the claims. The specification and examples give the best application of the present application as known to at least one of the inventors at the time of the filing of this application. It is to be understood that since numerous modifications and changes will readily occur to those skilled in the art, the application is not to be limited to the exact construction and operation as illustrated and described. Accordingly, all such variations are intended to be included within the scope of the present application as defined in the claims. The application is to be limited only by the claims.

Claims

1. A method of processing data for monitoring, characterized by, The method comprises: According to the request task sent by the request end, the monitoring data table of the monitoring data in the preset alarm period is obtained, and the monitoring data table comprises the alarm time, event type and application name of the monitoring data; According to the alarm time of the monitoring data, the alarm times of the monitoring data in different alarm sub-periods are determined; According to the alarm times of the monitoring data in different alarm sub-periods and the dimension information corresponding to the event type and / or the application name in the request task, the monitoring data in the monitoring data table is sorted to obtain a monitoring data list; According to the extraction requirement in the request task, the target monitoring data in the monitoring data is extracted from the monitoring data list, and the alarm information of the target monitoring data is generated and sent.

2. The method of claim 1, wherein, According to the request task sent by the request end, the monitoring data table of the monitoring data in the preset alarm period is obtained, and the monitoring data table comprises the alarm time, event type and application name of the monitoring data, comprising: According to the request task, the initial monitoring data in the preset alarm period is obtained; The alarm time, event type and application name in the initial monitoring data are written into a preset first database table to obtain an initial monitoring data table; The initial monitoring data table is copied to a preset second database table, and the initial monitoring data in the second database table is de-duplicated to obtain the monitoring data table.

3. The method of claim 2, wherein, The initial monitoring data table is copied to a preset second database table, and the initial monitoring data in the second database table is de-duplicated to obtain the monitoring data table. The initial monitoring data table is copied to a preset second database table, and the initial monitoring data in the second database table is de-duplicated to obtain the monitoring data table. According to the alarm times of the monitoring data in different alarm sub-periods, the monitoring data in the monitoring data table is initially sorted to obtain an initial monitoring data list; 4. The method of claim 1, wherein, According to the alarm times of the monitoring data in different alarm sub-periods, the monitoring data in the monitoring data table is initially sorted to obtain an initial monitoring data list; According to the alarm times of the monitoring data in different alarm sub-periods, the monitoring data in the monitoring data table is initially sorted to obtain an initial monitoring data list; According to the alarm times of the monitoring data in different alarm sub-periods, the monitoring data in the monitoring data table is initially sorted to obtain an initial monitoring data list; 5. The method of claim 4, wherein, According to the request end type and the dimension information corresponding to the event type and / or the application name in the request task, the sorting priority corresponding to different dimension information is obtained; ​ According to the different dimension information corresponding to the sorting priority, and according to the dimension information corresponding to the event type and / or the application name in the request task, the monitoring data with the same alarm times but different corresponding dimension information in the initial monitoring data list is adjusted and processed to obtain a monitoring data list.

6. The method of claim 4, wherein, According to the different dimension information corresponding to the sorting priority, and according to the dimension information corresponding to the event type and / or the application name in the request task, the monitoring data with the same alarm times but different corresponding dimension information in the initial monitoring data list is adjusted and processed to obtain a monitoring data list. According to the different dimension information corresponding to the sorting priority, and according to the dimension information corresponding to the event type and / or the application name in the request task, the monitoring data with the same alarm times but different corresponding dimension information in the initial monitoring data list is adjusted and processed to obtain a monitoring data list.

7. The method of claim 1, wherein, According to the different dimension information corresponding to the sorting priority, and according to the dimension information corresponding to the event type and / or the application name in the request task, the monitoring data with the same alarm times but different corresponding dimension information in the initial monitoring data list is adjusted and processed to obtain a monitoring data list. According to the different dimension information corresponding to the sorting priority, and according to the dimension information corresponding to the event type and / or the application name in the request task, the monitoring data with the same alarm times but different corresponding dimension information in the initial monitoring data list is adjusted and processed to obtain a monitoring data list. The dimension information includes at least one of availability monitoring information, batch monitoring information, transaction monitoring information, and event monitoring information.

8. The method of claim 1, wherein, The device includes:

9. A processing device for monitoring data, characterized by The acquisition module is configured to acquire a monitoring data table of monitoring data in a preset alarm period according to a request task sent by a request end, the monitoring data table including alarm time, event type, and application name of the monitoring data; The determination module is configured to determine alarm times of the monitoring data in different alarm sub-periods according to the alarm time of the monitoring data; The obtaining module is configured to sort the monitoring data in the monitoring data table according to the alarm times of the monitoring data in different alarm sub-periods and dimension information corresponding to the event type and / or the application name in the request task to obtain a monitoring data list; The sending module is configured to extract target monitoring data from the monitoring data list according to extraction requirements in the request task and generate and send alarm information of the target monitoring data. The device includes:

10. An electronic device, comprising: The memory and the processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, so that the processor executes the method of any one of claims 1-8. The memory and the processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, so that the processor executes the method of any one of claims 1-8.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores computer-executable instructions which, when executed by the processor, implement the method of any one of claims 1-7.

12. A computer program product comprising a computer program which, when executed by the processor, implements the method of any one of claims 1-8.

Images