Server alarm method, electronic equipment, storage medium and computer program product

Abstract

The invention discloses a server alarm method, electronic equipment, a storage medium and a computer program product. The method belongs to the field of civil aviation services and comprises the steps that a query data table and an alarm rule of a server are obtained, the query data table is used for storing query data which is generated when the server runs and can affect the running state of the server, and the alarm rule is used for determining whether the server can run normally or not; matching the query data stored in the query data table with the alarm rule to obtain a rule matching result; in response to the fact that the rule matching result is that the target data stored in the query data table is abnormal, server alarm information is generated based on the target data; and outputting server alarm information. According to the method and the device, the technical problem of relatively low operation and maintenance efficiency of the server in related technologies is solved.

Description

Technical Field

[0001] The present invention relates to the field of civil aviation services, and in particular to a server alarm method, electronic equipment, storage medium, and computer program product. Background Art

[0002] With the rise of cloud computing, big data, and microservice architecture, the complexity of modern enterprise-level application systems is increasing day by day, and their operation and maintenance work is also facing huge challenges. In distributed systems and microservice architectures, each microservice collaborates with each other through network calls, forming multiple intricate call links. These call links not only involve interactions between microservices, but also include access to the underlying database. The quality of database access performance is directly related to the response time and user experience of the entire system.

[0003] As an important component of the microservice architecture, call chain tracing technology can record the entire process of each request flowing through the system, including the time taken for the request, the call relationship between services, etc. However, in actual application scenarios, due to the large number of SQL (Structured Query Language) statements, traditional call chain systems are difficult to count and analyze in real time, resulting in poor real-time alarms, which reduces the speed of problem location and leads to low operation and maintenance efficiency.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a server alarm method, an electronic device, a storage medium, and a computer program product, so as to at least solve the technical problem of low efficiency in operating and maintaining a server in the related art.

[0006] According to one aspect of an embodiment of the present invention, a server alarm method is provided, comprising: obtaining a query data table and an alarm rule of a server, wherein the query data table is used to store query data generated by the server during operation and which will affect the operating status of the server, and the alarm rule is used to determine whether the server can operate normally; matching the query data stored in the query data table with the alarm rule to obtain a rule matching result; in response to the rule matching result being that there is an abnormality in target data stored in the query data table, generating server alarm information based on the target data; and outputting the server alarm information.

[0007] Furthermore, the query data stored in the query data table is matched with the alarm rules to obtain the rule matching result, including: extracting features of the alarm rules to obtain the rule features of the alarm rules; constructing a data query statement that matches the alarm rules based on the rule features and a preset statement format; based on the data query statement, reading at least one data to be detected from the query data table, wherein at least one data to be detected includes target data; matching at least one data to be detected with the alarm rules to obtain the rule matching result.

[0008] Furthermore, obtaining the query data table of the server includes: reading the full amount of data generated by the server during operation according to a preset method; performing data cleaning on the full amount of data to obtain cleaned data; and integrating the cleaned data to construct a query data table.

[0009] Furthermore, the full data includes: initial query data generated by the server at runtime, and data cleaning is performed on the full data to obtain cleaned data, including: extracting features from the initial query data in a real-time consumption manner to obtain initial features of the initial query data; based on the initial features, filtering out target query data from the initial query data; extracting information from the target query data to obtain query information corresponding to the target query data; and converting the query information into a format according to a preset key-value pair format to obtain cleaned data.

[0010] Furthermore, the cleaned data is integrated to construct a query data table, including: converting the format of the cleaned data according to a preset data format to obtain converted data; extracting fields from the converted data to obtain key fields of the converted data; formatting the key fields based on the field types of the key fields to obtain first formatted fields; and constructing a query data table according to a preset data table format based on the first formatted fields.

[0011] Furthermore, based on the first formatted field, a query data table is constructed in accordance with a preset data table format, including: based on the key field, obtaining configuration parameters of an application that generates conversion data from a preset data source; formatting the configuration parameters to obtain second formatted data; integrating the first formatted data and the second formatted data to obtain target formatted data; and constructing a query data table based on the target formatted data.

[0012] Furthermore, outputting server alarm information includes: obtaining information transmission parameters matching an operation interface, wherein the operation interface is used to display the alarm information; formatting the alarm information based on the information transmission parameters to obtain formatted alarm information; and sending the formatted alarm information to the operation interface.

[0013] According to another aspect of an embodiment of the present invention, a server alarm device is also provided, including: a first acquisition module, used to acquire a query data table and an alarm rule of the server, wherein the query data table is used to store query data generated by the server during operation and which will affect the operating status of the server, and the alarm rule is used to determine whether the server can operate normally; a rule matching module, used to match the query data stored in the query data table with the alarm rule to obtain a rule matching result; an information generation module, used to generate server alarm information based on the target data in response to the rule matching result that the target data stored in the query data table is abnormal; and an information output module, used to output the server alarm information.

[0014] According to another aspect of an embodiment of the present invention, there is further provided an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein the method in each embodiment of the present invention is executed when the program is running.

[0015] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present invention.

[0016] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0017] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0018] According to another aspect of the embodiments of the present invention, a computer program is further provided. When the computer program is executed by a processor, the methods in the embodiments of the present invention are implemented.

[0019] In an embodiment of the present invention, a query data table and an alarm rule of a server are obtained; the query data stored in the query data table are matched with the alarm rule to obtain a rule matching result; in response to the rule matching result that the target data stored in the query data table is abnormal, a server alarm information is generated based on the target data; and the server alarm information is output in a manner that, by obtaining the query data table, various query data generated by the server during operation are queried in real time, thereby ensuring that operation and maintenance personnel can access information that directly reflects the operation status of the server, and by obtaining the alarm rule, the alarm system can automatically identify and determine whether the query data exceeds the normal operation threshold, thereby determining the health status of the server. Based on the above steps, after the query data is matched with the alarm rule, if the data is abnormal, the alarm system can immediately generate and output the alarm information in a timely manner, thereby achieving the purpose of immediate response and rapid positioning of the abnormal situation of the server, thereby achieving the technical effect of improving the operation and maintenance efficiency of the server, and further solving the technical problem of low efficiency in operation and maintenance of the server in the related technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0021] Figure 1 is a flow chart of a server alarm method according to an embodiment of the present invention;

[0022] Figure 2 This is a framework diagram of an optional slow SQL warning system based on call chain according to an embodiment of the present invention;

[0023] Figure 3 is a flow chart of an optional SQL data alarm information processing according to an embodiment of the present invention;

[0024] Figure 4 It is a schematic diagram of an optional SQL data indicator alarm component rule generation process according to an embodiment of the present invention;

[0025] Figure 5 It is a schematic diagram of an optional SQL data indicator alarm component alarm process according to an embodiment of the present invention;

[0026] Figure 6 It is a schematic diagram of a server alarm device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] According to an embodiment of the present invention, an embodiment of a server alarm method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0030] Figure 1 is a flow chart of a server alarm method according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0031] Step S102, obtaining a query data table and an alarm rule of the server, wherein the query data table is used to store query data generated by the server during operation and which may affect the operation status of the server, and the alarm rule is used to determine whether the server can operate normally.

[0032] The query data table may be a database table designed to store query data generated during server operation that may affect the server's operating status. The alarm rule may be a series of predefined logical conditions for automatically monitoring the execution of SQL statements recorded in the query data table to determine whether the server can operate normally.

[0033] In an optional embodiment, considering that a large amount of query data is generated during the operation of the server, which not only reflects the workload of the server, but also may reveal potential performance bottlenecks or signs of failure, by collecting and analyzing the query data generated by the server in real time and storing the query data in a specially designed query data table, the database access behavior can be finely monitored. Further considering that traditional server performance monitoring relies on the experience and manual query analysis of operation and maintenance personnel, which is inefficient and error-prone under a complex service architecture, slow SQL and abnormal SQL can be efficiently identified through pre-set alarm rules, thereby determining whether the server can operate normally, where slow SQL can refer to SQL statements that take too long to execute, and abnormal SQL can refer to SQL statements that have errors during execution.

[0034] Step S104, matching the query data stored in the query data table with the alarm rule to obtain a rule matching result.

[0035] The above rule matching result can be the result of the alarm system judging whether the query data is abnormal or exceeds the expected standard according to the alarm rules. The above rule matching result directly determines whether an alarm notification needs to be triggered, and further determines whether operation and maintenance personnel need to intervene to troubleshoot and handle the fault.

[0036] In an optional embodiment, through the matching mechanism of query data and alarm rules, the alarm system can not only detect the anomaly of a single SQL statement, but also identify the common problems of a series of related SQL statements through aggregation and filtering functions, such as service A frequently calling slow SQL within a period of time, or the SQL execution anomaly rate of application B continues to rise. The above rule matching results can provide operation and maintenance personnel with a deeper performance analysis perspective, thereby helping operation and maintenance personnel make more informed operation and maintenance decisions, thereby improving the stability and operating performance of the server.

[0037] Step S106 , in response to the rule matching result indicating that the target data stored in the query data table is abnormal, server alarm information is generated based on the target data.

[0038] The server alarm information may be a detailed abnormality report generated by the alarm system for notifying operation and maintenance personnel or automated processing mechanisms in response to the rule matching result indicating that the target data in the query data table is abnormal. The server alarm information may include a comprehensive description of the slow SQL or abnormal database access behavior, aiming to help the information recipient quickly understand the nature of the problem and take corresponding corrective measures.

[0039] In an optional embodiment, if the rule matching result indicates that the target data stored in the query data table is abnormal, the alarm system can immediately generate an alarm message based on the target data to notify the operation and maintenance personnel to intervene. This real-time response feature enables the timely generation of alarm information as soon as the abnormal situation occurs, thereby ensuring that the operation and maintenance personnel can take prompt measures to avoid potential performance problems or service interruptions. In addition, the alarm information generated based on the target data can include specific execution details of slow SQL or abnormal SQL statements, such as important fields such as service name, application name, server name, and execution time, which provides the operation and maintenance personnel with specific fault location clues. Compared with traditional manual troubleshooting methods, this automated alarm mechanism improves the efficiency of fault location, reduces the time spent by operation and maintenance personnel on locating problems, and speeds up the response to server performance issues.

[0040] Step S108: output server alarm information.

[0041] In an optional embodiment, the server alarm information can be output not only to application operation and maintenance personnel, but also to component operation and maintenance personnel and R&D personnel to ensure that all relevant teams can be informed of server abnormalities in a timely manner. This cross-departmental information sharing mechanism promotes collaboration between different teams, helps to quickly mobilize the required resources for problem solving, and reduces the common communication barriers and delays in the traditional fault location process.

[0042] In an embodiment of the present invention, a query data table and an alarm rule of a server are obtained; the query data stored in the query data table are matched with the alarm rule to obtain a rule matching result; in response to the rule matching result that the target data stored in the query data table is abnormal, a server alarm information is generated based on the target data; and the server alarm information is output in a manner that, by obtaining the query data table, various query data generated by the server during operation are queried in real time, thereby ensuring that operation and maintenance personnel can access information that directly reflects the operation status of the server, and by obtaining the alarm rule, the alarm system can automatically identify and determine whether the query data exceeds the normal operation threshold, thereby determining the health status of the server. Based on the above steps, after the query data is matched with the alarm rule, if the data is abnormal, the alarm system can immediately generate and output the alarm information in a timely manner, thereby achieving the purpose of immediate response and rapid positioning of the abnormal situation of the server, thereby achieving the technical effect of improving the operation and maintenance efficiency of the server, and further solving the technical problem of low efficiency in operation and maintenance of the server in the related technology.

[0043] Furthermore, the query data stored in the query data table is matched with the alarm rules to obtain the rule matching result, including: extracting features of the alarm rules to obtain the rule features of the alarm rules; constructing a data query statement that matches the alarm rules based on the rule features and a preset statement format; based on the data query statement, reading at least one data to be detected from the query data table, wherein at least one data to be detected includes target data; matching at least one data to be detected with the alarm rules to obtain the rule matching result.

[0044] The above rule features may refer to features obtained after feature extraction of the alarm rule. For example, the above rule features may include at least one or more of the following: aggregation dimension, screening condition, calculation method, etc., but are not limited thereto.

[0045] The above preset statement format may refer to a standardized SQL query statement template for reading data from a query data table. This format may contain placeholders or dynamic parameters to adapt to the specific requirements of different alarm rules. For example, the above preset statement format may be "SELECT * FROM s lowsql WHERE filter condition AND time interval GROUP BY aggregation dimension", but is not limited to this. The above time interval, aggregation dimension and filter condition are all parameters dynamically filled according to the characteristics of the rule.

[0046] The above data query statement can be an actual SQL statement constructed according to the rule characteristics and the preset statement format. The statement can specifically define the query scope, aggregation method, filtering conditions, etc. to read data related to the alarm rule from the query data table.

[0047] The above-mentioned data to be detected may refer to a data set read from a query data table and to be matched with an alarm rule. Such data may be a single record or a group of records filtered according to the aggregation dimension and time interval defined in the rule feature.

[0048] The target data may be specific data points in the data to be detected that are directly related to the alarm rule. In this embodiment, the target data may be indicators such as the execution time and the number of calls of the slow SQL statement.

[0049] In an optional embodiment, the alarm system may first extract features from the alarm rules to understand the requirements and conditions of the rules. The above rule features may include aggregation dimensions (such as services, applications or servers), screening conditions (such as SQL execution time thresholds), threshold conditions (such as the number of times exceeding the threshold), data types (such as SQL statement execution time), etc. The above feature extraction process can convert the rules into an operational parameter set, which is convenient for subsequent construction of query statements and matching operations. Subsequently, the alarm system can construct a data query statement that matches the alarm rule based on the extracted rule features and the preset statement format. The above preset statement format can be an SQL query template, including a basic query structure and dynamic fields. By filling the specific parameters in the rule features (such as time intervals, aggregation dimensions, etc.) into the template, an actual query statement can be generated. The above construction process can realize the intelligent combination of rules and data, so that the alarm system can automatically generate queries for rule requirements, avoiding the complexity and error risk of manually writing query statements, while ensuring the standardization and consistency of queries. Then, the alarm system can read at least one data to be detected from the query data table based on the constructed data query statement. The above data to be detected is screened according to the conditions of the alarm rules, and may include target data, that is, specific records related to slow SQL or abnormal database access. Finally, the alarm system can match the read data to be detected with the alarm rules to obtain the rule matching results. The above steps provide the ability of intelligent alarm, that is, the alarm system can automatically identify potential problems without manual continuous monitoring or post-analysis, thereby improving operation and maintenance efficiency and fault response speed. In addition, accurate matching results ensure the pertinence and accuracy of alarms, avoiding waste of resources or omission of problems due to false alarms or missed alarms.

[0050] Furthermore, obtaining the query data table of the server includes: reading the full amount of data generated by the server during operation according to a preset method; performing data cleaning on the full amount of data to obtain cleaned data; and integrating the cleaned data to construct a query data table.

[0051] The above-mentioned preset method may refer to the data collection and processing rules, policies and processes pre-defined by the alarm system, which are used to guide how to read the full amount of data generated by the server during operation. The above-mentioned full amount of data may refer to all relevant data generated by the server during operation, including but not limited to the golden indicator data in the call chain, detailed information of SQL statements, caller time of the application, total time, etc. These data can be collected from sources such as servers, databases, network devices, etc. in their original state. The above-mentioned cleaned data may refer to the data obtained after preprocessing and quality inspection of the full amount of data. The purpose of data cleaning is to eliminate noise, outliers, duplicates and missing values ​​in the data, ensure the accuracy and consistency of the data, and thus improve the efficiency and quality of data processing and analysis.

[0052] In an optional embodiment, the preset method may include the frequency of data collection (such as real-time collection), data source (i.e., monitored applications and services), data type (such as SQL statements, call time, etc.) and storage location, so as to ensure that the data collection rules cover all key database operations and performance indicators. During the operation of the server, the alarm system can automatically read the full amount of data through the data collector component according to the above preset method. The reading of the above full amount of data ensures that the alarm mechanism covers all relevant data sources, thereby improving the comprehensiveness of problem detection. After reading the above full amount of data, the alarm system can perform data cleaning on the full amount of data to remove or correct the noise, outliers, duplicate records and missing information therein, so as to obtain cleaned data to improve data quality and ensure the accuracy of analysis. Finally, the alarm system can use data stream processing technology to integrate the cleaned data, and construct a query data table according to the pre-set table structure. The above query data table can be used as the data source of the alarm mechanism, which provides strong support for real-time monitoring and automated operation and maintenance, speeds up the operation and maintenance response speed, reduces the fault recovery time, and improves the user experience and system operation efficiency.

[0053] Furthermore, the full data includes: initial query data generated by the server at runtime, and data cleaning is performed on the full data to obtain cleaned data, including: extracting features from the initial query data in a real-time consumption manner to obtain initial features of the initial query data; based on the initial features, filtering out target query data from the initial query data; extracting information from the target query data to obtain query information corresponding to the target query data; and converting the query information into a format according to a preset key-value pair format to obtain cleaned data.

[0054] The above-mentioned initial query data may refer to data read or consumed in real time from a data collector component such as an APM (Application Performance Management) collector. The initial query data may include the full amount of data generated by the server during operation. For example, the above-mentioned initial query data may include at least one or more of the following: information about SQL statements in the call chain, the duration of application requests, database type, SQL execution exception information, etc., but not limited to these. The initial query data is raw data that has not been processed in any way, contains a large amount of information that may be redundant or unfiltered, and is the original input for subsequent data processing and alarm mechanisms.

[0055] The above-mentioned real-time consumption can be a data processing mode. Real-time consumption allows the system to process data immediately after it is generated without waiting for data accumulation or batch processing. In this embodiment, real-time consumption can be manifested as the alarm system continuously monitoring the topic (subject) corresponding to APM in the Kafka (Apache Kafka, a stream processing platform) cluster, that is, topic apm. Once new data is generated, the alarm system can process it immediately. Real-time consumption enables the alarm system to respond to new data instantly, ensure the real-time and efficiency of the alarm mechanism, reduce the delay of fault detection, and improve the accuracy of problem location.

[0056] The above initial features may refer to key information extracted from the initial query data consumed in real time and used for subsequent data screening and processing. For example, the above initial features may include at least one or more of the following: data timestamp, service type, abnormal information, time consumption index, etc., but are not limited to these. The initial features are the basis for defining data cleaning rules and screening conditions.

[0057] The target query data may refer to a data subset directly related to the alarm rule that is extracted from the initial query data through screening conditions based on the initial features.

[0058] The preset key-value pair format may refer to a predefined rule for converting target query data into a key-value pair format for the convenience of storage and query during the data formatting process. The preset key-value pair format helps standardize and uniformly manage data, and facilitates subsequent data analysis and matching of alarm rules.

[0059] In an optional embodiment, real-time consumption can use stream processing technology to monitor and process the full data stream generated by the server to ensure the immediacy and continuity of data processing. Specifically, the real-time consumption mechanism can be implemented through a Kafka cluster or other high-performance message queue system, ensuring low latency from data generation to processing and enhancing the real-time response capability of the alarm system. Feature extraction can ensure that each data packet can be processed in a timely manner without data loss by continuously consuming the main body of the Kafka cluster. The alarm system can extract feature fields related to slow SQL alarms from each data packet, such as database type serviceType, SQL statement description sqlDesc, SQL operation type sqlType, SQL execution exception information sqlExceptionMsg, and time consumption pureTime and wallTime, etc. By judging the values ​​of pureTime and wallTime, the alarm system can preliminarily identify possible slow SQL operations, thereby providing a basis for subsequent data screening. Based on the initial features, the alarm system can further filter out the target query data that matches the slow SQL alarm rules. For example, the alarm system can filter out the database operation data that takes longer than the preset pureTime and wallTime thresholds, and extract key information such as complete SQL statements, execution time, database type, service name, application name, and host name from the filtered target query data as the basis for subsequent alarm logic judgment. After screening and information extraction, the target query data can be converted into a data format according to the preset key-value pair format to generate clean data that can be used for subsequent processing components. Through the above process, the full amount of data is consumed in real time, extracted features, screened for target data, extracted information, and converted into a key-value pair format, and finally clean data is generated. This series of processing steps not only improves the real-time, accuracy, and efficiency of data processing, but also provides a solid data foundation for the slow SQL alarm mechanism based on call chain data, accelerates the process of fault recovery, and improves the overall stability and operation and maintenance efficiency of the server.

[0060] Furthermore, the cleaned data is integrated to construct a query data table, including: converting the format of the cleaned data according to a preset data format to obtain converted data; extracting fields from the converted data to obtain key fields of the converted data; formatting the key fields based on the field types of the key fields to obtain first formatted fields; and constructing a query data table according to a preset data table format based on the first formatted fields.

[0061] The above-mentioned converted data may refer to data obtained by format conversion of the cleaned data after the data cleaning process. This conversion usually converts the data from an unstructured or semi-structured format into a structured data format for easy storage in a relational database.

[0062] The above key fields may refer to data fields in the conversion data that are directly related to the slow SQL alarm logic and are crucial to building the query data table. For example, the above key fields may include at least one or more of the following: database service type, SQL statement description, time consumption of the method caller, total time consumption of request in and out routing, service name, etc., but are not limited thereto.

[0063] The above field type can refer to the data type of each field in the database table, such as string type (varchar), integer type (int), floating point type (double), datetime type (datetime), etc. When building a query data table, the choice of field type is crucial to ensure the correct storage and efficient query of data. For example, the time consumption of the method caller and the total time consumption of the request in and out of the route as time consumption can be specified as floating point type or integer type for numerical comparison and calculation.

[0064] The first formatted field may refer to a data field obtained by formatting according to the field type after the key field is extracted.

[0065] In an optional embodiment, the alarm system can reorganize each field in the cleaned data according to a preset data format. For example, the alarm system can unify the time units of pureTime and wallTime into microseconds to ensure the unit consistency of the field value. Subsequently, the alarm system can convert the reorganized data field into a key-value pair format, and each data field is represented as a key-value pair to facilitate data storage and retrieval. At the same time, the alarm system can convert the field value into a corresponding data type according to the field type. For example, the alarm system can convert the service name dc_app_name into a varchar type to ensure the accuracy of data storage. Through the above steps, the alarm system can complete the format conversion of the cleaned data, thereby obtaining the converted data. Then, the alarm system can further filter out the fields related to the slow SQL alarm, such as serviceType (database type), pureTime (time consumed by the method caller), dc_app_name (service name), application (application name) and hostname (server name), etc. These fields constitute the key fields of the converted data, thereby reducing the number of fields in the query data table, accelerating the data query speed, and improving the system response efficiency. Based on the formatting of the field types of the above key fields, the alarm system can obtain the first formatted field, thereby ensuring the flexibility and compatibility of data processing and being able to adapt to the specific needs of different databases and data types. Finally, the alarm system can build a query data table based on the first formatted field in accordance with the preset data table format. This step ensures that the data can be effectively stored and queried, providing a solid data foundation for slow SQL alarms.

[0066] Furthermore, based on the first formatted field, a query data table is constructed in accordance with a preset data table format, including: based on the key field, obtaining configuration parameters of an application that generates conversion data from a preset data source; formatting the configuration parameters to obtain second formatted data; integrating the first formatted data and the second formatted data to obtain target formatted data; and constructing a query data table based on the target formatted data.

[0067] The above-mentioned preset data source may refer to a centralized storage system for storing and managing application configuration parameters. For example, in the present embodiment, the preset data source may be a configuration management database, a file system or a configuration center service, but is not limited thereto. The above-mentioned configuration parameters may refer to parameter settings related to the application and slow SQL alarm logic defined in the above-mentioned preset data source. The above-mentioned second formatted data may refer to the data after the application configuration parameters are formatted. The above-mentioned target formatted data may refer to the data obtained by integrating the first formatted data (i.e., the data after the key fields of the cleaned data are formatted) with the second formatted data (i.e., the data after the configuration parameters are formatted). This integration process ensures the format consistency and data integrity of all data (including original data and configuration parameters) in the data processing process, which facilitates subsequent data storage and query operations.

[0068] In an optional embodiment, the alarm system can retrieve the configuration parameters related to the slow SQL alarm from the preset data source to ensure that the data processing logic complies with the alarm strategy. If the configuration parameters in the preset data source are updated, the alarm system can obtain the updated parameters in real time to ensure that the data processing logic is consistent with the newer new strategy. By obtaining the configuration parameters from the preset data source, the alarm system can dynamically adjust the threshold and rules of the slow SQL alarm without modifying the original code, thereby enhancing the flexibility and configurability of the alarm system. Considering that the obtained configuration parameters may contain multiple types of data, such as strings, values, etc., in order to ensure that the configuration parameters can be effectively integrated with the data of the first formatted field, the alarm system can format the configuration parameters to obtain the second formatted data. Through field type matching and data standardization, the data compatibility of the configuration parameters and the first formatted field can be ensured, avoiding data processing errors caused by data type mismatch. After obtaining the second formatted data, the alarm system can merge the first formatted field with the fields in the second formatted data to ensure that the data table contains all necessary information related to the slow SQL alarm. In the above merging process, the alarm system can also perform data type checks to ensure that the data types of all fields match the preset data table format. Finally, the alarm system can define the structure of the query data table, including field names, data types, and indexes, to facilitate efficient storage and fast retrieval of data. The alarm system can also import the target formatted data into the query data table to ensure accurate storage and integrity of the data. In order to improve the efficiency of querying data by specified dimensions, the alarm system can create indexes for key fields in the query data table according to the query requirements of slow SQL alarms. In the above steps, the constructed query data table supports fast queries by service, application, or server dimensions, as well as aggregate queries based on time windows, thereby improving the query efficiency and accuracy of slow SQL alarms, reducing alarm delays, and speeding up problem location and fault recovery.

[0069] Furthermore, outputting server alarm information includes: obtaining information transmission parameters matching an operation interface, wherein the operation interface is used to display the alarm information; formatting the alarm information based on the information transmission parameters to obtain formatted alarm information; and sending the formatted alarm information to the operation interface.

[0070] The above-mentioned operation interface may refer to a user interface used to display alarm information in an alarm receiving system. For example, the above-mentioned operation interface may be a Web (World Wide Web) page, a mobile application, etc., but is not limited thereto. The operation interface provides an intuitive and friendly way of displaying alarm information for operation and maintenance personnel, so that they can quickly understand and respond to alarm events. The above-mentioned information transmission parameters may refer to parameters used to ensure the correct transmission and formatting of information during the process of sending alarm information from the alarm system to the operation interface. The above-mentioned formatted alarm information may refer to alarm information converted into a format that can be recognized and displayed by the operation interface based on the information transmission parameters.

[0071] In an optional embodiment, the alarm system can read the information transmission parameters from the configuration management database or the preset data source to ensure the real-time and validity of the parameters. Subsequently, the alarm system can verify the read information transmission parameters to check whether they meet the display requirements of the operation interface, such as whether the data format is supported, whether the encoding method is correct, etc. The alarm system can also make appropriate adjustments to the information transmission parameters according to the actual needs of the operation interface. By obtaining the information transmission parameters that match the operation interface, it can be ensured that the alarm information can be transmitted in the data format and encoding method supported by the operation interface, avoiding display anomalies caused by format incompatibility, and improving the user experience and the display effect of the alarm information. Based on the information transmission parameters, the alarm system can format the alarm information extracted from the slow SQL alarm scenario to obtain formatted alarm information. This process needs to ensure that the format of the alarm information meets the display requirements of the operation interface, while retaining the integrity of key information, so that the operation and maintenance personnel can quickly locate the problem. Specifically, the alarm system can convert the alarm information from the internal storage format to a data format supported by the operation interface, such as JSON (JavaScript Object Notation), XML (Extensible Markup Language) or a custom text format. Subsequently, the alarm system can fill the field values ​​in the alarm information into the display template corresponding to the operation interface according to the field mapping rules in the information transmission parameters, ensuring that all key information (such as slow SQL statements, execution time, associated services, application names and host names, etc.) can be correctly displayed on the operation interface. It should be noted that if the information transmission parameters contain security transmission requirements, the sensitive data in the alarm information is encrypted and the alarm information is sent through a secure transmission protocol to prevent the data from being accessed by unauthorized third parties during the transmission process. After obtaining the formatted alarm information, the alarm system can send the formatted alarm information to the operation interface. For example, the alarm system can send the formatted alarm information as the body of an HTTP (Hypertext Transfer Protocol) or HTTPS (Hypertext Transfer Protocol Secure) request to the operation interface, supporting the display of alarm information on the Web operation interface. For another example, if the operation interface supports real-time updates, such as a Web operation interface, the alarm system can use the WebSocket protocol to push alarm information in real time, thereby realizing instant display of alarm information.

[0072] For ease of understanding, Figure 2 is a framework diagram of an optional slow SQL warning system based on call chain according to an embodiment of the present invention, such as Figure 2As shown in the figure, the framework includes: multiple APM collectors (only three are shown in the figure), a message storage component Kafka cluster, an alarm information processing component, a data formatting processing component, an indicator alarm component, and an alarm receiving component. Among them, the APM collectors are deployed in different applications or services respectively, and are used to capture and collect the call chain data of the application in real time. These collectors can identify and track the call relationships between various services, including the call chain information of relational databases, NOSQL (NOStructured Query Language, non-relational) databases, and distributed databases. The message storage component Kafka cluster includes two topics, namely topic APM and topic alarm. Topic APM is a temporary storage area for storing original call chain data to ensure that the data is accessible for a short time before processing. Topic alarm is the data storage area after processing by the alarm information processing component. The storage time is longer than topic APM, which can provide a data basis for subsequent data formatting and alarms. The alarm information processing component is responsible for real-time consumption of data in topic APM and preprocessing data involving SQL statements. It will determine the SQL type and extract key information, such as the execution time pureTime and wallTime, service type serviceType, and related exception information. The pre-processed data will be sent back to the topic alarm of the Kafka cluster in the flattened KEY:VALUE String format for storage, providing preparation for subsequent data formatting processing. The data formatting processing component can consume the data in the topic alarm in real time, and convert the data into a format suitable for storage and alarm through steps such as data stream processing, field assignment, numerical operation, and logical judgment. In addition, the WEB system function provided by the data formatting processing component allows system administrators to configure the data processing process, including converting data into JSON format, querying related information through Redis fields, extracting key fields and storing them as varchar, int, double, etc., and finally writing the data into the s lowsql data table. The above s lowsql data table is a database table designed for alarm information content, which stores the processed SQL statement details, including service name, application name, host name, etc., and provides a data basis for the indicator alarm component. The indicator alarm component is used to set custom alarm rules, including selecting the software, data table, indicator, aggregation method, aggregation dimension and filter items to be monitored. The enabled rules will be stored in the database table metric_rule for scheduling execution of scheduled tasks in the component.The scheduled task executes the SQL query statement according to the set time interval, compares the query result with the set threshold to determine whether to trigger an alarm. If the alarm condition is met, the indicator alarm component will trigger an alarm and send the alarm details to the alarm receiving component. The alarm receiving component includes three notification channels: SMS, email, and on-duty alarm processing platform. The alarm receiving component is responsible for receiving the alarm information sent by the indicator alarm component and performing necessary data format processing to adapt to the data format requirements of different alarm channels. The alarm receiving component can notify the corresponding operation and maintenance personnel of the alarm information through SMS, email, or on-duty alarm processing platform according to the alarm receiving channel set by the user to ensure the timely transmission of alarm information. Figure 2 It clearly demonstrates the complete workflow from data collection, preprocessing, storage, formatting, indicator alarm to alarm information notification. Each link is closely linked, ensuring the efficiency and accuracy of the slow SQL alarm mechanism.

[0073] Figure 3 is a flowchart of an optional SQL data alarm information processing according to an embodiment of the present invention, such as Figure 3 As shown, at the beginning of the process, the APM collector obtains the call chain method data and determines whether the traversal is complete. If the traversal is complete, the process ends. If the traversal is not complete, the serviceType and sqlExceptionMsg are further obtained, where serviceType is used to identify the database type of this database operation, whether it is a relational database, a NOSQL database or a distributed database, and sqlExceptionMsg records the abnormal situations that may occur during the SQL execution process. After obtaining serviceType and sqlExceptionMsg, it can be determined whether the serviceType is NOSQL or SQL. If the judgment result is no, it returns to the above step of determining whether the traversal is complete. If the judgment result is yes, the alarm information processing component can obtain access pureTime and wallTime, that is, the alarm information processing component can obtain the pureTime of the method caller of the application and the total wallTime of the routing distribution component of the application request entering and exiting the microservice framework. After obtaining the time information, the alarm information processing component can further process the data, and the processed data can be sent back to the Kafka topic in real time. At this point, the process ends.

[0074] Figure 4 is a schematic diagram of an optional SQL data indicator alarm component rule generation process according to an embodiment of the present invention, such as Figure 4As shown in the figure, at the beginning of the process, users can select software and slow SQL data tables. This is the first step in generating alarm rules, which determines the specific applications and data sources to be monitored. After selecting the software and data tables to be monitored, users need to further select indicators, aggregation dimensions, such as execution time pureTime or wallTime, and aggregation dimensions of alarm rules. Aggregation dimensions can be service dc_app_name, application application, or server hostname. The purpose of selecting aggregation dimensions is to aggregate data statistics on specific dimensions so as to more accurately identify slow SQL problems. Subsequently, users need to select a calculation method to process the data of the selected indicator. Common calculation methods include sum, average avg, maximum value, etc. The selection of calculation method is based on the needs of the alarm scenario. For example, if you need to monitor the number of times the SQL execution time exceeds a specific threshold, you can select the sum function to count how many SQL statements exceed the threshold. Then, users can determine whether to use filter items. If you do not use filter items, you can proceed to the steps of setting alarm thresholds and alarm receiving channels. If you use a filter item, you need to select the filter item, and then determine whether the filter item type is varchar. If so, select the operator: equal, not equal, similar. If the filter item type is not varchar, select the operator: greater than, greater than or equal to, less than, less than or equal to, equal. Then, the user needs to set the alarm threshold, alarm receiving channel, etc. After completing the settings, the alarm rules will be saved to the database, and the process ends.

[0075] Figure 5 is a schematic diagram of an optional SQL data indicator alarm component alarm process according to an embodiment of the present invention, such as Figure 5 As shown in the figure, when the process starts, the alarm component can obtain rules from the rule database. Specifically, the alarm component's background scheduled task program retrieves all enabled alarm rules from the database table metric_rule. These rules are in Figure 4 The steps shown are set and saved by the user, including detailed information such as the monitored software, data tables, indicators, aggregation dimensions, filter conditions, operators, and thresholds. After obtaining the rules, the alarm component can determine whether the rules are enabled. If not, the process ends. If the rules are enabled, the alarm component can start a scheduled task for each rule and splice SQL statements according to the rules. Subsequently, the alarm component can call the SQL statement execution at a fixed time and compare the result with the alarm condition. If the execution result does not meet the alarm condition, the process ends. If the execution result meets the alarm condition, an alarm is triggered and the alarm information is sent to the alarm receiving component. At this point, the process ends.

[0076] According to an embodiment of the present invention, an embodiment of a server alarm device is provided. It should be noted that the device can be used to execute the above server alarm method. The specific implementation method and application scenario are the same as those of the above embodiment, and will not be repeated here. Figure 6 is a schematic diagram of a server alarm device according to an embodiment of the present invention. Figure 6 As shown, the device comprises:

[0077] The first acquisition module 602 is used to acquire the query data table and alarm rules of the server, wherein the query data table is used to store the query data generated by the server during operation and which will affect the operation status of the server, and the alarm rules are used to determine whether the server can operate normally.

[0078] The rule matching module 604 is used to match the query data stored in the query data table with the alarm rule to obtain a rule matching result.

[0079] The information generating module 606 is used to generate server warning information based on the target data in response to the rule matching result indicating that the target data stored in the query data table is abnormal.

[0080] The information output module 608 is used to output server alarm information.

[0081] Furthermore, the rule matching module is also used to: extract features of the alarm rules to obtain rule features of the alarm rules; construct a data query statement that matches the alarm rules based on the rule features and a preset statement format; based on the data query statement, read at least one data to be detected from the query data table, wherein at least one data to be detected includes target data; match at least one data to be detected with the alarm rules to obtain a rule matching result.

[0082] Furthermore, the first acquisition module is also used to: read the full amount of data generated by the server during operation according to a preset method; clean the full amount of data to obtain cleaned data; integrate the cleaned data to construct a query data table.

[0083] Furthermore, the full data includes: initial query data generated by the server at runtime. The first acquisition module is also used to: extract features from the initial query data in real-time consumption to obtain initial features of the initial query data; filter out target query data from the initial query data based on the initial features; extract information from the target query data to obtain query information corresponding to the target query data; convert the query information into a format according to a preset key-value pair format to obtain cleaned data.

[0084] Furthermore, the first acquisition module is also used to: convert the format of the cleaned data according to a preset data format to obtain converted data; extract fields from the converted data to obtain key fields of the converted data; format the key fields based on the field type of the key fields to obtain a first formatted field; and construct a query data table based on the first formatted field according to a preset data table format.

[0085] Furthermore, the first acquisition module is also used to: based on the key fields, obtain the configuration parameters of the application that generates the conversion data from the preset data source; format the configuration parameters to obtain the second formatted data; integrate the first formatted data and the second formatted data to obtain the target formatted data; and construct a query data table based on the target formatted data.

[0086] Furthermore, the information output module is also used to: obtain information transmission parameters matching the operation interface, wherein the operation interface is used to display alarm information; format the alarm information based on the information transmission parameters to obtain formatted alarm information; and send the formatted alarm information to the operation interface.

[0087] An embodiment of the present application further provides an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of the present invention when running.

[0088] An embodiment of the present application further provides a computer-readable storage medium, which includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present invention.

[0089] An embodiment of the present application further provides a computer program product, including a computer program, which implements the methods in various embodiments of the present invention when executed by a processor.

[0090] An embodiment of the present application further provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0091] The embodiments of the present application further provide a computer program, which implements the methods in the above-mentioned embodiments of the present invention when executed by a processor.

[0092] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0093] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0094] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0095] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0096] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0097] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A server alarm method, characterized in that: include: Obtaining a query data table and an alarm rule of a server, wherein the query data table is used to store query data generated by the server during operation and which may affect the operation status of the server, and the alarm rule is used to determine whether the server can operate normally; Matching the query data stored in the query data table with the alarm rule to obtain a rule matching result; In response to the rule matching result being that the target data stored in the query data table is abnormal, generating server alarm information based on the target data; Output the server warning information.

2. The method according to claim 1, characterized in that Matching the query data stored in the query data table with the alarm rule to obtain a rule matching result, including: Extracting features of the alarm rule to obtain rule features of the alarm rule; Based on the rule features and the preset statement format, construct a data query statement that matches the alarm rule; Based on the data query statement, read at least one to-be-detected data from the query data table, wherein the at least one to-be-detected data includes the target data; The at least one data to be detected is matched with the alarm rule to obtain the rule matching result.

3. The method according to claim 1, characterized in that Get the query data table of the server, including: Read all data generated by the server during operation according to a preset method; Performing data cleaning on the full amount of data to obtain cleaned data; The cleaned data is integrated to construct the query data table.

4. The method according to claim 3, characterized in that The full amount of data includes: initial query data generated by the server during operation, and the full amount of data is cleaned to obtain cleaned data, including: Extracting features from the initial query data in a real-time consumption manner to obtain initial features of the initial query data; Based on the initial features, filter out target query data from the initial query data; Extracting information from the target query data to obtain query information corresponding to the target query data; The query information is formatted according to a preset key-value pair format to obtain the cleansed data.

5. The method according to claim 4, characterized in that Integrating the cleaned data to construct the query data table includes: Performing format conversion on the cleaned data according to a preset data format to obtain converted data; Performing field extraction on the conversion data to obtain key fields of the conversion data; Based on the field type of the key field, format the key field to obtain a first formatted field; Based on the first formatted field, the query data table is constructed according to a preset data table format.

6. The method according to claim 5, characterized in that Based on the first formatted field, constructing the query data table according to a preset data table format includes: Based on the key field, obtaining configuration parameters of the application that generates the conversion data from a preset data source; Formatting the configuration parameters to obtain second formatted data; Integrating the first formatted data and the second formatted data to obtain target formatted data; The query data table is constructed based on the target formatted data.

7. The method according to claim 1, characterized in that Outputting the server alarm information includes: Acquire information transmission parameters matching an operation interface, wherein the operation interface is used to display the alarm information; Formatting the warning information based on the information transmission parameter to obtain formatted warning information; The formatted alarm information is sent to the operation interface.

8. An electronic device, characterized in that: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 7 when running.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is run, the device where the storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images