Monitoring data processing method and device, equipment, medium and product

Abstract

The embodiment of the invention provides a monitoring data processing method and device, equipment, a medium and a product, and relates to the field of big data or other related fields. The method comprises the steps that a monitoring data table of monitoring data in a preset alarm period is acquired according to a request task sent by a request end, and the monitoring data table comprises alarm time, event types and application names of the monitoring data; determining the alarm times of the monitoring data in different alarm sub-cycles according to the alarm time of the monitoring data; according to alarm times of the monitoring data in different alarm sub-cycles and dimension information corresponding to event types and / or application names in the request task, sorting the monitoring data in the monitoring data table to obtain a monitoring data list; and extracting target monitoring data in the monitoring data from the monitoring data list according to an extraction requirement in the request task, and generating and sending alarm information of the target monitoring data. According to the method provided by the invention, the data processing effect for a large amount of monitoring data is improved.

Description

Technical Field

[0001] The present application relates to the field of big data, and in particular to a monitoring data processing method, device, equipment, medium and product. Background Art

[0002] In modern IT operation and maintenance management, monitoring statistical models are crucial to ensure the stability and reliability of application systems. These models analyze monitoring message information from different channels (such as availability, transactions, etc.) to capture system status changes in real time, warn of potential problems, and assist the operation and maintenance team to respond quickly. However, as the complexity of the system increases, traditional monitoring models face challenges in processing multi-dimensional data and responding to sudden failures.

[0003] The current monitoring statistical model mainly works based on the single dimension of "number of alarms", which periodically summarizes the monitoring messages reported by each application server (such as availability, batch, transaction and event monitoring) and counts the number of alarms within a specific time range. This method identifies system risks or anomalies by quantifying the frequency of alarms, providing a simple and intuitive reference for operation and maintenance decisions, and is currently a commonly used monitoring and governance method.

[0004] Although the statistical model based on the number of alarms can meet basic monitoring needs, its effectiveness and accuracy are limited when dealing with a large number of short-term alarms caused by sudden failures, which increases the burden of operation and maintenance and leads to poor results in processing monitoring data. Summary of the invention

[0005] The embodiments of the present application provide a monitoring data processing method, device, equipment, medium and product to improve the technical problem of processing monitoring data.

[0006] In a first aspect, an embodiment of the present application provides a method for processing monitoring data, including:

[0007] According to the request task sent by the requesting end, a monitoring data table of monitoring data within a preset alarm period is obtained, where the monitoring data table includes the alarm time, event type and application name of the monitoring data;

[0008] According to the alarm time of the monitoring data, determine the alarm times of the monitoring data in different alarm sub-cycles;

[0009] According to the number of alarms of the monitoring data in different alarm sub-cycles and the dimension information corresponding to the event type and / or application name in the request task, the monitoring data in the monitoring data table is sorted to obtain a monitoring data list;

[0010] According to the extraction requirements in the request task, target monitoring data in the monitoring data is extracted from the monitoring data list, and alarm information of the target monitoring data is generated and sent.

[0011] In a possible implementation, according to the request task sent by the requesting end, a monitoring data table of monitoring data within a preset alarm period is obtained, and the monitoring data table includes the alarm time, event type and application name of the monitoring data, including:

[0012] According to the requested task, obtain the initial monitoring data within the preset alarm period;

[0013] Writing the alarm time, event type and application name in the initial monitoring data into a preset first database table to obtain an initial monitoring data table;

[0014] The initial monitoring data table is copied to a preset second database table, and the initial monitoring data in the second database table is deduplicated to obtain a monitoring data table.

[0015] In a possible implementation, the initial monitoring data table is copied to a preset second database table, and the initial monitoring data in the second database table is deduplicated to obtain a monitoring data table, including:

[0016] Copying the initial monitoring data table to a preset second database table;

[0017] According to the application name, event type and alarm time in the initial monitoring data in the second database table, deduplication processing is performed on the initial monitoring data in the second database table to obtain a monitoring data table.

[0018] In a possible implementation, the monitoring data in the monitoring data table is sorted according to the number of alarms of the monitoring data in different alarm sub-cycles and the dimension information corresponding to the event type and / or application name in the request task to obtain a monitoring data list, including:

[0019] According to the alarm times of the monitoring data in different alarm sub-cycles, the monitoring data in the monitoring data table are initially sorted to obtain an initial monitoring data list;

[0020] According to the dimension information corresponding to the event type and / or application name in the request task, the monitoring data in the initial monitoring data list is adjusted to obtain the monitoring data list.

[0021] In a possible implementation, according to the dimension information corresponding to the event type and / or application name in the request task, the monitoring data with the same alarm times but different corresponding dimension information in the initial monitoring data list are adjusted to obtain a monitoring data list, including:

[0022] According to the type of the request end and the dimension information corresponding to the event type and / or application name in the request task, the sorting priority corresponding to different dimension information is obtained;

[0023] According to the sorting priorities corresponding to different dimensional information and the dimensional information corresponding to the event type and / or application name in the request task, the monitoring data in the initial monitoring data list with the same alarm times but different corresponding dimensional information are adjusted and processed to obtain a monitoring data list.

[0024] In a possible implementation, according to the dimension information corresponding to the event type and / or the application name in the request task, the monitoring data in the initial monitoring data list with the same alarm times but with different corresponding dimension information are adjusted to obtain the monitoring data list, including: according to the request task sent by the request end and the dimension information corresponding to the event type and / or the application name in the request task, obtaining the sorting priority parameters corresponding to the different dimension information;

[0025] According to the sorting priorities corresponding to different dimensional information and the dimensional information corresponding to the event type and / or application name in the request task, the monitoring data in the initial monitoring data list with the same alarm times but different corresponding dimensional information are adjusted and processed to obtain a monitoring data list.

[0026] In a possible implementation, according to the extraction requirement in the request task, extracting target monitoring data in the monitoring data from the monitoring data list, and generating and sending alarm information of the target monitoring data include:

[0027] According to the request task sent by the requesting end, determine the extraction requirements of the extraction quantity and dimension information in the monitoring data list;

[0028] According to the extraction requirements of the extraction quantity and dimension information in the monitoring data list, the target monitoring data in the monitoring data list is extracted from the monitoring data list, and the alarm information of the target monitoring data is generated and sent.

[0029] In a possible implementation, the dimensional information includes at least one of availability monitoring information, batch monitoring information, transaction monitoring information, and event monitoring information.

[0030] In a second aspect, an embodiment of the present application provides a monitoring data processing device, including:

[0031] The acquisition module is used to obtain a monitoring data table of monitoring data within a preset alarm period according to the request task sent by the request end, and the monitoring data table includes the alarm time, event type and application name of the monitoring data;

[0032] A determination module, used to determine the alarm times of the monitoring data in different alarm sub-periods according to the alarm time of the monitoring data;

[0033] A module is used to sort the monitoring data in the monitoring data table according to the number of alarms of the monitoring data in different alarm sub-cycles and the dimension information corresponding to the event type and / or application name in the request task to obtain a monitoring data list;

[0034] The sending module is used to extract target monitoring data in the monitoring data from the monitoring data list according to the extraction requirements in the request task, and to generate and send alarm information of the target monitoring data.

[0035] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0036] Memory stores computer-executable instructions;

[0037] The processor executes the computer-executable instructions stored in the memory to implement the method of the present application.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0039] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements the method of the embodiment of the present application when executed by a processor.

[0040] The monitoring data processing method, device, equipment, medium and product provided in the embodiments of the present application reduce the data processing volume by obtaining a monitoring data table of monitoring data consisting of alarm time, event type and application name within a preset alarm period, and sort the monitoring data in the monitoring data table according to the number of alarms of the monitoring data in different alarm sub-periods determined by the alarm time of the monitoring data and the dimensional information corresponding to the event type and / or application name in the request task to obtain a monitoring data list, thereby further simplifying the amount of data in the data processing process, and finally selecting the target monitoring data from the monitoring data list according to the extraction requirements in the request task, and generating and sending alarm information. Technical means to improve the problem that when processing a large number of short-term alarms caused by sudden failures, the effectiveness and accuracy are limited, the operation and maintenance burden is increased, and the effect of processing the monitoring data is poor. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0042] Figure 1 A schematic diagram of a monitoring data processing method provided in an embodiment of the present application;

[0043] Figure 2 A flow chart of a monitoring data processing method provided in an embodiment of the present application Figure 1 ;

[0044] Figure 3 A flow chart of a monitoring data processing method provided in an embodiment of the present application Figure 2 ;

[0045] Figure 4 A schematic diagram of the structure of a monitoring data processing device provided in an embodiment of the present application;

[0046] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0047] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0048] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0049] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0050] In addition, this application involves conducting big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.), and using artificial intelligence technology to make automated decisions, and making technical solutions that have a significant impact on personal rights and interests based on the results of automated decisions. The application provides users with corresponding operation entrances for them to choose to agree or reject the results of automated decisions; if the user chooses to reject, the expert decision-making process will be entered.

[0051] It should be noted that the monitoring data processing method, device, equipment, medium and product provided in the embodiments of the present application can be used in the field of big data, and can also be used in any field other than big data. The application field of the monitoring data processing method, device, equipment, medium and product in the present application is not limited.

[0052] First, the terms involved in this application are explained:

[0053] Availability monitoring: Availability messages are sent at a preset period. If they are not received within the preset period, an availability alarm is triggered.

[0054] Batch monitoring: Send messages when a task starts and ends. If they are not received within the specified time, a batch alarm is triggered;

[0055] Transaction monitoring: Monitor the status of transactions during online transactions, and send messages containing transaction codes, return codes, IP addresses, channels and other information in real time. If such information is processed and judged to exceed the set threshold, a transaction alarm will be triggered;

[0056] Event monitoring: Customized exception monitoring. The custom conditions include exception triggering conditions. If the exception triggering conditions are met, the event alarm will be triggered.

[0057] In the prior art, the monitoring system usually issues early warnings based on pre-set thresholds when judging actual faults. However, data deviations may lead to a large number of alarm messages in the short term. Intensive alarms will affect the recognition ability of the monitoring system. At the same time, the number of alarms, as a measurement indicator of the monitoring system, has lost its due reference value. In addition, a large amount of alarm messages also increases the maintenance costs of maintenance personnel.

[0058] A monitoring data processing method provided in an embodiment of the present application parses out a time period that needs to be monitored according to a request task sent by a user at a request end, and uses it as a preset alarm period. The monitoring data within the preset alarm period corresponds to the alarm time, event type, and application name according to the user's request task as a standard for judging the warning. The preset alarm period is then divided into different alarm sub-periods to accurately calculate the number of alarms. The monitoring data is then sorted based on the number of alarms and the dimensional information that the user is concerned about. Finally, corresponding warning information is generated based on the sorted monitoring data, thereby solving the problem of a large amount of alarm information affecting the judgment.

[0059] Figure 1 A scenario diagram of a monitoring data processing method provided in an embodiment of the present application, such as Figure 1 As shown, the execution subject of the method can be a monitoring data processing system, which can be a server. The server can be a computer, notebook, tablet or mobile phone.

[0060] This embodiment does not impose any special restrictions on the implementation method of the execution subject, as long as the execution subject can obtain a monitoring data table of monitoring data within a preset alarm period according to the request task sent by the requesting end, the monitoring data table including the alarm time, event type and application name of the monitoring data; determine the number of alarms of the monitoring data in different alarm sub-periods according to the alarm time of the monitoring data; sort the monitoring data in the monitoring data table according to the number of alarms of the monitoring data in different alarm sub-periods and the dimension information corresponding to the event type and / or application name in the request task to obtain a monitoring data list; extract the target monitoring data in the monitoring data from the monitoring data list according to the extraction requirements in the request task, and generate and send the alarm information of the target monitoring data.

[0061] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0062] Figure 2 A flow chart of a monitoring data processing method provided in an embodiment of the present application Figure 1 ,like Figure 2 As shown, the method includes:

[0063] S201. According to the request task sent by the requesting end, a monitoring data table of monitoring data within a preset alarm period is obtained, where the monitoring data table includes the alarm time, event type and application name of the monitoring data.

[0064] The requester can refer to the client or system that initiates the request, and can be any type of software application, service, or hardware device. For example, the requester can be a monitoring management platform, through which the administrator sets monitoring rules and views alarm information, or it can be an automated script that regularly checks the health of the system.

[0065] A request task may refer to an instruction or command sent by a request end to instruct the execution of a required operation. A request task may include specific parameters for processing monitoring data, such as event type, application name, and how to sort, filter, and extract monitoring data based on event type and application name.

[0066] The preset alarm period can refer to a predefined period of time during which the data collected will be used to evaluate whether an alarm condition is triggered. For example, if the preset alarm period is one day, all monitoring data in the past 24 hours will be analyzed to determine whether any abnormal conditions have occurred and whether an alarm needs to be issued.

[0067] Monitoring data can refer to various indicator information collected from the monitored objects. The monitored objects can refer to servers, applications, and network devices. Monitoring data usually reflects the operating status or performance of the objects, such as CPU utilization, memory usage, network traffic, and error logs.

[0068] A monitoring data table can refer to a structured form of storing monitoring data, which can exist in the form of a table, where each row represents a separate data record and each column corresponds to a different attribute or field, such as alarm time or alarm location, which facilitates the organization and management of large amounts of monitoring data. Monitoring data tables can be obtained in a variety of ways, such as directly collecting them from the monitoring system. Many applications and services have built-in monitoring functions that can generate detailed logs and performance indicators. Through built-in tools, monitoring data can be directly exported or queried and formatted into monitoring data tables. For example, for target application scenarios, scripts can be written to automatically collect and organize log data, and then create monitoring data tables.

[0069] In an embodiment of the present application, the monitoring data table includes the alarm time, event type and application name of the monitoring data, wherein the alarm time may refer to the exact moment when the system detects an abnormal situation and triggers an alarm. The alarm time is expressed as a timestamp, accurate to seconds or finer-grained time units, which is used to identify the specific time point when the problem occurs.

[0070] Event type can refer to the classification of different types of events captured, including system errors, resource overloads, security intrusions, etc. Each event type represents a behavior or state change that may affect the normal operation of the system.

[0071] The application name may refer to a key identifier for distinguishing applications and services, and is used to distinguish monitored applications or services, thereby ensuring that monitoring data can be accurately associated with specific applications.

[0072] In an embodiment of the present application, the data source of the monitoring data table is the monitoring message information sent by each application server, such as a business operation management platform or a cross-border clearing platform.

[0073] S202: Determine the alarm times of the monitoring data in different alarm sub-periods according to the alarm time of the monitoring data.

[0074] Among them, the alarm sub-cycle can refer to the time period divided in the preset alarm cycle, which can be time units of different lengths such as minutes, hours, days, etc. The specific division depends on the monitoring needs and business characteristics, so as to accurately detect abnormal situations in specific time periods such as nighttime.

[0075] The number of alarms can refer to the number of alarm events that occur within a specified time range. The number of alarms can be the number of alarms within a complete preset alarm cycle or the number of alarms within an alarm sub-cycle.

[0076] In an embodiment of the present application, the alarm sub-period can be set according to actual needs to adapt to different monitoring strategies and business scenarios. For example, if you want to reduce frequent alarms caused by short-term, repetitive problems, you can choose a shorter alarm sub-period (such as every 15 minutes or 30 minutes). For real-time monitoring of critical trading systems, you can choose a more fine-grained alarm sub-period (such as every 5 minutes or even shorter) to ensure that any anomalies can be captured and responded to in a timely manner. For non-critical systems or long-term trend analysis, you can choose a longer alarm sub-period (such as every hour or every 4 hours).

[0077] In some embodiments, different alarm sub-periods are divided within a preset alarm cycle, and the number of alarms within different alarm sub-periods is determined, thereby improving the situation where multiple dispersed alarms are regarded as multiple independent events. For example, in the past 24 hours, the system has generated 10 separate availability alarms due to short-term network fluctuations, each with a few minutes interval. After setting one hour as an alarm sub-period, it is stipulated that multiple alarms of the same type occurring within one alarm sub-period are only counted as one. Even if 10 short-term alarms occur in one day, because they all occur within the same hour, they will only be recorded as one alarm in the entire 24-hour period.

[0078] S203. Sort the monitoring data in the monitoring data table according to the number of alarms of the monitoring data in different alarm sub-cycles and the dimension information corresponding to the event type and / or application name in the request task to obtain a monitoring data list.

[0079] Among them, dimension information can refer to the attributes or characteristics that describe or classify monitoring data, which facilitates understanding and analysis of data and identification of trends or anomalies under specific conditions. In the embodiment of the present application, dimension information can be divided into four categories: "availability monitoring", "batch monitoring", "transaction monitoring" and "event monitoring", and these four categories are used to monitor the system at multiple levels.

[0080] The monitoring data list can refer to a set of monitoring data that is sorted and filtered according to certain rules. It can be presented in a table format, with each row representing an independent data record and columns corresponding to different dimensional information or other important indicators. The monitoring data list can sort or filter data according to different dimensional information and summarize the number of alarms of the same type.

[0081] Among them, in the embodiment of the present application, the monitoring data in the monitoring data table is sorted according to the number of alarms of the monitoring data in different alarm sub-periods and the dimension information corresponding to the event type and / or application name in the request task to obtain a monitoring data list, including:

[0082] According to the alarm times of the monitoring data in different alarm sub-cycles, the monitoring data in the monitoring data table are initially sorted to obtain an initial monitoring data list;

[0083] According to the dimension information corresponding to the event type and / or application name in the request task, the monitoring data in the initial monitoring data list is adjusted to obtain the monitoring data list.

[0084] The initial monitoring data list may refer to a data set obtained by preliminarily sorting the records in the monitoring data table based on the number of alarms of the monitoring data in different alarm sub-periods, which can help quickly identify which events occur frequently in a specific time period.

[0085] Adjustment processing may refer to the process of further screening and reordering the monitoring data in the initial monitoring data list according to the dimension information specified in the request task, and finally obtaining a monitoring data list, thereby ensuring that the final result better meets specific needs and improving the relevance and practicality of the data.

[0086] For example, if you want to group by application name and event type and calculate the number of the last occurrence date, you can use the selcet statement to sort the corresponding data entries in descending or ascending order.

[0087] In some embodiments, according to the dimension information corresponding to the event type and / or application name in the request task, the monitoring data in the initial monitoring data list with the same alarm times but different corresponding dimension information are adjusted to obtain a monitoring data list, including:

[0088] According to the type of the request end and the dimension information corresponding to the event type and / or application name in the request task, the sorting priority corresponding to different dimension information is obtained;

[0089] According to the sorting priorities corresponding to different dimensional information and the dimensional information corresponding to the event type and / or application name in the request task, the monitoring data in the initial monitoring data list with the same alarm times but different corresponding dimensional information are adjusted and processed to obtain a monitoring data list.

[0090] The type of the request end can refer to the type of user interface or system that initiates the monitoring data request, which is used to further distinguish the processing priority. For example, the Web end can be used for internal employees to view and manage monitoring data. The internal tools used by the Web end and the public interface used by external users may have different priority settings. At the same time, combined with specific dimension information, it is determined which dimension information should be given priority, thereby corresponding to different sorting priorities.

[0091] Sorting priority is a relatively fixed order, usually pre-set by the system or administrator based on common business logic. It is applicable to most common scenarios and provides a stable way to sort data.

[0092] In some embodiments, according to the dimension information corresponding to the event type and / or application name in the request task, the monitoring data in the initial monitoring data list with the same alarm times but different corresponding dimension information are adjusted to obtain the monitoring data list, including: according to the request task sent by the request end and the dimension information corresponding to the event type and / or application name in the request task, obtaining the sorting priority parameters corresponding to the different dimension information;

[0093] According to the sorting priorities corresponding to different dimensional information and the dimensional information corresponding to the event type and / or application name in the request task, the monitoring data in the initial monitoring data list with the same alarm times but different corresponding dimensional information are adjusted and processed to obtain a monitoring data list.

[0094] Sorting priority parameters can refer to specific values ​​or rules used to quantify the importance of information in different dimensions. Sorting priority parameters are usually pre-configured or dynamically generated based on the requested task. They can be more dynamic and flexible indicators, allowing the system to respond more accurately to the personalized needs of users, especially when facing complex or special monitoring requirements.

[0095] S204: extract target monitoring data from the monitoring data list according to the extraction requirements in the request task, and generate and send alarm information of the target monitoring data.

[0096] Extraction requirements can refer to a set of standards or rules set to filter out data that meets specific conditions from a large amount of monitoring data. The specific conditions specify what data to obtain, how to process the data, and the final output format. Extraction requirements are usually defined by the requester based on specific business needs or problem diagnosis purposes.

[0097] Target monitoring data can refer to a data set obtained from a large amount of original monitoring data after screening according to extraction requirements. Target monitoring data can present monitoring results in the form of tables, charts, etc., or can be combined with other data sets to analyze the monitoring situation again.

[0098] Alert information can refer to notifications or warnings generated when an abnormal situation is detected during the monitoring process, including a detailed description of the problem, the time of occurrence, the objects involved, and the recommended response measures.

[0099] Among them, in the embodiment of the present application, according to the extraction requirements in the request task, extracting the target monitoring data in the monitoring data from the monitoring data list, and generating and sending the alarm information of the target monitoring data include:

[0100] According to the request task sent by the requesting end, determine the extraction requirements of the extraction quantity and dimension information in the monitoring data list;

[0101] According to the extraction requirements of the extraction quantity and dimension information in the monitoring data list, the target monitoring data in the monitoring data list is extracted, and the alarm information of the target monitoring data is generated and sent. Among them, the extraction quantity in the monitoring data list can refer to the maximum number of data items that need to be extracted from the monitoring data list, which helps to limit the amount of data returned, improve query efficiency, and prevent excessive data from causing analysis difficulties.

[0102] The extraction requirement of dimensional information can refer to specific fields or attributes that the user is interested in, which can be used to further filter and refine the data.

[0103] For example, according to the request task sent by the requester, all "performance alarm" records of the "WebApp" application in the past 7 days are extracted from the monitoring data list, with a maximum of 100 records. According to these extraction requirements, the target monitoring data that meets the conditions is screened out, and detailed alarm information (including alarm time, event type and description) is generated for each record, and then sent to the designated members via email to ensure that they can respond and handle performance issues in a timely manner.

[0104] A method for processing monitoring data provided by an embodiment of the present application first obtains a monitoring data table within a preset alarm period according to a request task sent by a requesting end, and the table includes an alarm time, an event type, and an application name. Then, the system counts the number of alarms of these monitoring data in different alarm sub-periods. Then, according to the number of alarms and the dimension information corresponding to the event type and / or application name specified in the request task, the monitoring data is sorted to generate an optimized monitoring data list. Finally, according to the extraction requirements in the request task, the target monitoring data is filtered out from the monitoring data list, and the corresponding alarm information is generated and sent to the relevant personnel. Not only the accuracy and timeliness of the monitoring data are ensured, but also the relevance and practicality of the data are improved through personalized sorting and screening, while ensuring that the relevant maintenance personnel focus on specific applications or event types.

[0105] Figure 3 A flow chart of a monitoring data processing method provided in an embodiment of the present application Figure 2 ,like Figure 3 As shown, in this embodiment Figure 2 Based on the embodiment, a method for processing monitoring data is described in detail. According to the request task sent by the request end, a monitoring data table of monitoring data within a preset alarm period is obtained. The monitoring data table includes the alarm time, event type and application name of the monitoring data, including:

[0106] S301, according to the request task, obtaining the initial monitoring data within the preset alarm period;

[0107] S302: Write the alarm time, event type and application name in the initial monitoring data into a preset first database table to obtain an initial monitoring data table.

[0108] Among them, initial monitoring data can refer to the original monitoring information collected directly from various monitoring sources. The initial monitoring data usually contains rich details, but may contain duplicates or redundant information and is not processed or optimized. Therefore, it needs further processing to be used for efficient analysis and decision-making.

[0109] The preset first database table may refer to a database table for receiving and storing specific attributes in the initial monitoring data, which is the starting point of the entire data processing chain and is responsible for storing the original monitoring information that has not been processed or optimized. In an embodiment of the present application, the preset first database table may be an initial monitoring data table to ensure that all monitoring data is stored in a consistent manner.

[0110] The initial monitoring data table can be directly obtained from the initial monitoring data, and the acquisition method can be performed through filtering statements, commands and formulas. In an embodiment of the present application, a table application software is used to export data from the database of the monitoring alarm data system, and the data is stored in the form of a table as the initial monitoring data, from which the alarm date is 7 days from the current date, and the data starts from 0 o'clock on that day and ends at 0 o'clock on the same day, and the corresponding event type and application name are obtained. Among them, OSCM_DATA is used as the storage name of the initial monitoring data table, and the initial monitoring data table contains 3 fields app_name, evnet_name and event_date, which respectively represent "application name", "event type" and "alarm time".

[0111] For example, if you select the alarm records that occurred between 00:00:00 on December 3, 2024 and 00:00:00 on December 10, 2024, there may be availability alarm information at 2024-12-03 03:15, which occurred in WebApp, and there may also be transaction failure alarm information at 2024-12-07 19:20, which occurred in WebApp.

[0112] S303: Copy the initial monitoring data table to a preset second database table, and perform deduplication processing on the initial monitoring data in the second database table to obtain a monitoring data table.

[0113] The preset second database table may refer to the data copied from the first database table. The preset second database table is used to store the data after preliminary processing, so as to facilitate setting a new processing area so that the data before and after processing are separated and completely retained.

[0114] Deduplication may refer to the process of removing duplicate data records in the second database table based on specific key fields, thereby ensuring that each data entry is unique, thereby improving the accuracy and efficiency of data analysis. Deduplication may be implemented in a variety of ways, such as using a table's filtering function or a tool's filtering command editing window, which is not limited in this application.

[0115] In the embodiment of the present application, the initial monitoring data table is copied to a preset second database table, and the initial monitoring data in the second database table is deduplicated to obtain a monitoring data table, including:

[0116] Copying the initial monitoring data table to a preset second database table;

[0117] According to the application name, event type and alarm time in the initial monitoring data in the second database table, the initial monitoring data in the second database table is deduplicated to obtain a monitoring data table. Copying the initial monitoring data table to the preset second database table is a direct data migration, which ensures that all fields and records are accurately transferred to the new table.

[0118] The deduplication process uses application name, event type, and alarm time because these three fields together define the unique identifier of the monitoring event and can effectively distinguish different alarm events. For example, for the data entries stored in the second database table, they are grouped according to application name, event type, and alarm time. When the number of data in a group exceeds one, the duplicate records are deleted and only one data entry is retained.

[0119] The embodiment of the present application provides a method for processing monitoring data. According to the request task sent by the request end, the initial monitoring data within the preset alarm period is first obtained, and the alarm time, event type and application name in these data are written into the preset first database table to form an initial monitoring data table. Next, the initial monitoring data table is copied to the preset second database table, and the data in the second database table is deduplicated based on the application name, event type and alarm time to ensure that only one record is retained for each unique monitoring event, and finally an optimized monitoring data table is generated. This process not only improves the accuracy and consistency of the data, but also simplifies the subsequent data analysis work.

[0120] Table 1 is a schematic diagram of the results of the monitoring data processing method provided in an embodiment of the present application. As shown in Table 1, within the preset alarm period 2024.03.10-2024.03.16, the event monitoring repetition statistics include: the number of similar alarm days for the self-service terminal is 5 days, the number of similar alarm days for the natural language processing is 5 days, and the number of similar alarm days for the intelligent decision-making platform is 5 days. In addition to the results of multiple applications such as the self-service terminal, natural language processing and the intelligent decision-making platform, you can also see the display of results.

[0121]

[0122] Table 2 is a schematic diagram of the results of the monitoring data processing method provided in the embodiment of the present application. As shown in Table 2, within the preset alarm period 2024.03.10-2024.03.16, the batch monitoring repetition statistics are as follows: the number of similar alarm days for bank-enterprise services is 6 days, the number of similar alarm days for global cash management is 6 days, and the number of similar alarm days for bond trading is 5 days. In addition to the result display of multiple applications such as bank-enterprise services, global cash management and bond trading, you can also see.

[0123]

[0124] Figure 4 A schematic diagram of the structure of the monitoring data processing device provided in the embodiment of the present application, such as Figure 4 As shown, the monitoring data processing device 40 provided in this embodiment includes:

[0125] The acquisition module 401 is used to acquire a monitoring data table of monitoring data within a preset alarm period according to the request task sent by the request end, wherein the monitoring data table includes the alarm time, event type and application name of the monitoring data;

[0126] A determination module 402 is used to determine the alarm times of the monitoring data in different alarm sub-periods according to the alarm time of the monitoring data;

[0127] Obtaining module 403, for sorting the monitoring data in the monitoring data table according to the number of alarms of the monitoring data in different alarm sub-periods and the dimension information corresponding to the event type and / or application name in the request task, to obtain a monitoring data list;

[0128] The sending module 404 is used to extract target monitoring data in the monitoring data from the monitoring data list according to the extraction requirements in the request task, and to generate and send alarm information of the target monitoring data.

[0129] In the embodiment of the present application, the acquisition module 401 may also be specifically used for:

[0130] According to the requested task, obtain the initial monitoring data within the preset alarm period;

[0131] Writing the alarm time, event type and application name in the initial monitoring data into a preset first database table to obtain an initial monitoring data table;

[0132] The initial monitoring data table is copied to a preset second database table, and the initial monitoring data in the second database table is deduplicated to obtain a monitoring data table.

[0133] In the embodiment of the present application, the acquisition module 401 may also be specifically used for:

[0134] Copying the initial monitoring data table to a preset second database table;

[0135] According to the application name, event type and alarm time in the initial monitoring data in the second database table, the initial monitoring data in the second database table is deduplicated to obtain a monitoring data table. In the embodiment of the present application, the obtaining module 403 can also be specifically used for:

[0136] According to the alarm times of the monitoring data in different alarm sub-cycles, the monitoring data in the monitoring data table are initially sorted to obtain an initial monitoring data list;

[0137] According to the dimension information corresponding to the event type and / or application name in the request task, the monitoring data in the initial monitoring data list is adjusted to obtain the monitoring data list.

[0138] In the embodiment of the present application, the obtaining module 403 may also be specifically used for:

[0139] According to the type of the request end and the dimension information corresponding to the event type and / or application name in the request task, the sorting priority corresponding to different dimension information is obtained;

[0140] According to the sorting priorities corresponding to different dimensional information and the dimensional information corresponding to the event type and / or application name in the request task, the monitoring data in the initial monitoring data list with the same alarm times but different corresponding dimensional information are adjusted and processed to obtain a monitoring data list.

[0141] In the embodiment of the present application, the obtaining module 403 may also be specifically used for:

[0142] According to the request task sent by the request end and the dimension information corresponding to the event type and / or application name in the request task, the sorting priority parameters corresponding to different dimension information are obtained;

[0143] According to the sorting priorities corresponding to different dimensional information and the dimensional information corresponding to the event type and / or application name in the request task, the monitoring data in the initial monitoring data list with the same alarm times but different corresponding dimensional information are adjusted and processed to obtain a monitoring data list.

[0144] In the embodiment of the present application, the sending module 404 may also be specifically used for:

[0145] According to the request task sent by the requesting end, determine the extraction requirements of the extraction quantity and dimension information in the monitoring data list;

[0146] According to the extraction requirements of the extraction quantity and dimension information in the monitoring data list, the target monitoring data in the monitoring data is extracted from the monitoring data list, and the alarm information of the target monitoring data is generated and sent. The monitoring data processing device provided in this embodiment can execute the method provided in the above method embodiment, and its implementation principle and technical effect are similar, which will not be described in detail in this embodiment.

[0147] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device 50 provided in this embodiment includes: at least one processor 501 and a memory 502. Optionally, the device 50 also includes a communication component 503. The processor 501, the memory 502 and the communication component 503 are connected via a bus 504.

[0148] In a specific implementation process, at least one processor 501 executes the computer-executable instructions stored in the memory 502, so that at least one processor 501 executes the above method.

[0149] The specific implementation process of the processor 501 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0150] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.

[0151] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0152] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0153] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0154] An embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0155] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0156] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0157] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0158] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0159] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0160] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0161] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0162] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for processing monitoring data, characterized in that: The method comprises: According to the request task sent by the requesting end, a monitoring data table of monitoring data within a preset alarm period is obtained, wherein the monitoring data table includes the alarm time, event type and application name of the monitoring data; Determining the number of alarms of the monitoring data in different alarm sub-periods according to the alarm time of the monitoring data; According to the alarm times of the monitoring data in different alarm sub-periods and the dimension information corresponding to the event type and / or the application name in the request task, the monitoring data in the monitoring data table is sorted to obtain a monitoring data list; According to the extraction requirement in the request task, target monitoring data in the monitoring data is extracted from the monitoring data list, and alarm information of the target monitoring data is generated and sent.

2. The method according to claim 1, characterized in that The request task sent by the requesting end is used to obtain a monitoring data table of monitoring data within a preset alarm period, wherein the monitoring data table includes the alarm time, event type and application name of the monitoring data, including: According to the requested task, obtain the initial monitoring data within the preset alarm period; Writing the alarm time, event type and application name in the initial monitoring data into a preset first database table to obtain an initial monitoring data table; The initial monitoring data table is copied to a preset second database table, and the initial monitoring data in the second database table is deduplicated to obtain the monitoring data table.

3. The method according to claim 2, characterized in that The step of copying the initial monitoring data table to a preset second database table and performing deduplication processing on the initial monitoring data in the second database table to obtain the monitoring data table includes: Copying the initial monitoring data table to a preset second database table; According to the application name, event type and alarm time in the initial monitoring data in the second database table, deduplication processing is performed on the initial monitoring data in the second database table to obtain the monitoring data table.

4. The method according to claim 1, characterized in that: The monitoring data in the monitoring data table is sorted according to the alarm times of the monitoring data in different alarm sub-periods and the dimension information corresponding to the event type and / or the application name in the request task to obtain a monitoring data list, including: According to the alarm times of the monitoring data in different alarm sub-periods, the monitoring data in the monitoring data table are initially sorted to obtain an initial monitoring data list; According to the dimension information corresponding to the event type and / or the application name in the request task, the monitoring data in the initial monitoring data list is adjusted to obtain a monitoring data list.

5. The method according to claim 4, characterized in that The monitoring data in the initial monitoring data list with the same alarm times but different corresponding dimensional information are adjusted according to the dimensional information corresponding to the event type and / or the application name in the request task to obtain a monitoring data list, including: According to the type of the request end and the dimension information corresponding to the event type and / or the application name in the request task, obtaining the sorting priority corresponding to different dimension information; According to the sorting priorities corresponding to different dimensional information and according to the dimensional information corresponding to the event type and / or the application name in the request task, the monitoring data in the initial monitoring data list with the same alarm times but different corresponding dimensional information are adjusted and processed to obtain a monitoring data list.

6. The method according to claim 4, characterized in that The step of adjusting and processing the monitoring data in the initial monitoring data list with the same alarm times but corresponding to different dimensional information according to the dimensional information corresponding to the event type and / or the application name in the request task to obtain the monitoring data list includes: obtaining sorting priority parameters corresponding to different dimensional information according to the request task sent by the request end and the dimensional information corresponding to the event type and / or the application name in the request task; According to the sorting priorities corresponding to the different dimensional information and according to the dimensional information corresponding to the event type and / or the application name in the request task, the monitoring data in the initial monitoring data list with the same alarm times but different corresponding dimensional information are adjusted and processed to obtain a monitoring data list.

7. The method according to claim 1, characterized in that The extracting target monitoring data in the monitoring data from the monitoring data list according to the extraction requirement in the request task, and generating and sending alarm information of the target monitoring data, comprises: Determine the extraction quantity in the monitoring data list and the extraction requirement of the dimension information according to the request task sent by the request end; According to the extraction quantity in the monitoring data list and the extraction requirement of the dimension information, the target monitoring data in the monitoring data is extracted from the monitoring data list, and the alarm information of the target monitoring data is generated and sent.

8. The method according to claim 1, characterized in that The dimensional information includes at least one of availability monitoring information, batch monitoring information, transaction monitoring information, and event monitoring information.

9. A monitoring data processing device, characterized in that: The method comprises: An acquisition module, used to acquire a monitoring data table of monitoring data within a preset alarm period according to a request task sent by a requesting end, wherein the monitoring data table includes an alarm time, an event type and an application name of the monitoring data; A determination module, used to determine the alarm times of the monitoring data in different alarm sub-periods according to the alarm time of the monitoring data; An obtaining module is used to sort the monitoring data in the monitoring data table according to the number of alarms of the monitoring data in different alarm sub-cycles and the dimension information corresponding to the event type and / or the application name in the request task to obtain a monitoring data list; The sending module is used to extract the target monitoring data in the monitoring data from the monitoring data list according to the extraction requirements in the request task, and to generate and send the alarm information of the target monitoring data.

10. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

12. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images