A log-based backend service monitoring and alarm method, device and medium

Abstract

The present invention discloses a log-based back-end service monitoring and alarm method, device and medium, which belongs to the technical field of back-end service monitoring, and is used to solve the technical problems that the monitoring and alarm system in the existing back-end service is prone to over-reliance on system integration, and the alarm information is difficult to locate, and the abnormal alarm response is slow. The method includes: based on a preset log collection tool, collecting log data in the back-end business system; transmitting the log data to a Kafka cluster, and performing log parsing on the log data to obtain log parsing data; according to a preset regular matching rule, performing log screening on the log parsing data to determine the log entry data that meets the alarm condition; performing alarm threshold judgment on the log entry data to obtain alarm data; encapsulating and processing the log data corresponding to the alarm data, and sending it to the operation and maintenance center of the back-end business system.

Description

Technical Field

[0001] The present application relates to the field of backend service monitoring, and in particular to a log-based backend service monitoring and alarm method, device and medium. Background Art

[0002] With the rapid development of the Internet, servers have become an indispensable part of modern enterprise operations. The stable operation of servers is crucial to the normal operation of enterprises. Once a server fails or is abnormal, it may cause business interruption or even data loss. In order to detect server problems in time and take corresponding measures, server monitoring and alarm systems came into being.

[0003] Existing backend service monitoring and alarm systems often have the following problems:

[0004] 1. Dependence on complex alarm system integration: Some monitoring alarm solutions require backend services to directly connect to specific alarm systems, which increases the complexity of system integration and maintenance costs.

[0005] 2. Lack of alarm information: Some alarm systems can only provide simple status or indicator abnormality alarms, lack detailed error information, and are not conducive to quickly locating problems.

[0006] 3. Single delivery method: Some alarm systems provide limited alarm notification methods, which may not meet the receiving preferences of different team members and affect the response speed.

[0007] 4. Language restrictions and service intrusion: Some monitoring and alarm solutions have specific requirements for the development language of backend services, or require in-depth modifications to the service code, posing a potential threat to the stability of the existing system. Summary of the invention

[0008] The embodiments of the present application provide a log-based backend service monitoring and alarm method, device and medium, which are used to solve the following technical problems: the monitoring and alarm system in the existing backend service is prone to over-reliance on system integration, and the alarm information is difficult to locate, and the abnormal alarm response is slow.

[0009] The present application embodiment adopts the following technical solutions:

[0010] On the one hand, an embodiment of the present application provides a log-based back-end service monitoring and alarm method, including: based on a preset log collection tool, collecting log data in the back-end business system; transmitting the log data to a Kafka cluster, and performing log parsing on the log data to obtain log parsed data; according to a preset regular matching rule, performing log screening on the log parsed data to determine log entry data that meets the alarm condition; performing alarm threshold judgment on the log entry data to obtain alarm data; packaging the log data corresponding to the alarm data, and sending it to the operation and maintenance center of the back-end business system.

[0011] The embodiment of the present application avoids the workload of docking complex alarm systems and reduces integration costs by making full use of the existing log data of the back-end business system. The alarm information comes directly from the log, including detailed timestamps, log levels, error details, etc., which is conducive to developers to quickly locate the root cause of the fault. It also supports a variety of alarm delivery methods, which can adapt to the receiving habits of different team members, ensure that alarm information is communicated in a timely manner, and improve problem handling efficiency. The log collection module interacts with the business system through a standard interface, is not limited by the programming language used by the back-end service, and has good versatility. It only needs to integrate the log collection component in the business system, without the need to deeply modify the service code, and maintains the original structure and stability of the service.

[0012] In a feasible implementation, based on a preset log collection tool, log data in a backend business system is collected, specifically including: integrating the log collection tool into the backend business system; wherein the log collection tool includes at least: a Logback tool, a Kafka Connect tool, a Fluentd tool, and a Sqoop tool; based on a preset time interval and through the log collection tool, data is collected on the log data stream and the log application stream in the backend business system to obtain the log data.

[0013] In a feasible implementation, the log data is transmitted to a Kafka cluster, and the log data is parsed to obtain log parsed data, specifically including: pushing the log data to the Kafka cluster through a preset distributed message queue system; and subscribing to and consuming the log data through the Kafka cluster; extracting key field data in the log data; wherein the key field data includes at least: timestamp, log level and error details; based on an attribute item table, performing data tabular matching on the key field data to obtain the log parsed data; wherein the attribute item table includes at least: attribute value, attribute prefix and attribute content.

[0014] In a feasible implementation, the log parsing data is screened according to preset regular matching rules to determine log entry data that meets the alarm conditions, specifically including: based on the attribute item table in the log parsing data, the attribute items in the attribute item table are marked to obtain marked attribute items; multiple types of regular matching rules corresponding to each of the marked attribute items are constructed; wherein each type of regular matching rule corresponds to each of the marked attribute items; according to the multiple types of regular matching rules, the log parsing data is traversed to determine abnormal candidate log entries; through the alarm conditions, the attribute conditions of the abnormal candidate log entries are matched to determine the log entry data that meets the alarm conditions.

[0015] In a feasible implementation manner, an alarm threshold judgment is performed on the log entry data to obtain alarm data, specifically including: based on the monitoring alarm configuration in the back-end business system, determining the alarm threshold corresponding to each log entry in the log entry data; counting and judging the number of alarm condition matches of each log entry with the alarm threshold to obtain alarm data; wherein the alarm data includes: alarm triggering data and alarm not-triggered data.

[0016] In a feasible implementation, the log data corresponding to the alarm data is encapsulated and processed, and sent to the operation and maintenance center of the back-end business system, specifically including: performing alarm identification processing on the log data corresponding to the alarm trigger data in the alarm data; and encapsulating the log data to obtain back-end service alarm data; wherein the back-end service alarm data includes: exception timestamp, exception log level, error details and log entry data that triggers the alarm; according to a preset alarm sending method, the back-end service alarm data is formatted and adapted to the relevant data structure, and sent to the operation and maintenance center of the back-end business system, so that different operation and maintenance teams receive the back-end service alarm data.

[0017] In a feasible implementation, after the back-end service alarm data is formatted and adapted to the relevant data structure, the method further includes: using a preset alarm problem template, performing template matching and data analysis on the formatted and adapted back-end service alarm data, and generating initial alarm cause data; sending the initial alarm cause data to the operation and maintenance center of the back-end business system, so that the different operation and maintenance teams can complete the information positioning of the alarm data.

[0018] In a feasible implementation manner, the alarm sending method includes at least: SMS, email, WeChat, DingTalk and Webhook.

[0019] In the second aspect, an embodiment of the present application also provides a log-based back-end service monitoring and alarm device, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor so that the at least one processor can execute a log-based back-end service monitoring and alarm method described in any of the above embodiments.

[0020] In the third aspect, an embodiment of the present application also provides a non-volatile computer storage medium, characterized in that the storage medium is a non-volatile computer-readable storage medium, and the non-volatile computer-readable storage medium stores at least one program, each of which includes instructions, and when the instructions are executed by the terminal, the terminal executes a log-based back-end service monitoring and alarm method described in any of the above-mentioned embodiments.

[0021] The present application provides a log-based backend service monitoring and alarm method, device and medium. Compared with the prior art, the embodiments of the present application have the following beneficial technical effects:

[0022] 1. No need to integrate complex alarm systems: By making full use of the existing log data in the business system, the workload of connecting to complex alarm systems is avoided and the integration cost is reduced.

[0023] 2. Rich alarm information helps to quickly locate problems: Alarm information comes directly from logs and contains detailed timestamps, log levels, error details, etc., which helps developers quickly locate the root cause of the fault.

[0024] 3. Rich alarm delivery methods to improve response speed: Support multiple alarm delivery methods, which can adapt to the receiving habits of different team members, ensure that alarm information is conveyed in time, and improve problem handling efficiency.

[0025] 4. No restrictions on backend service development language: The log collection module interacts with the business system through a standard interface and is not restricted by the programming language used by the backend service, so it has good versatility.

[0026] 5. No intrusion to services: You only need to integrate the log collection component into the business system without making deep modifications to the service code, thus maintaining the original structure and stability of the service. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0028] Figure 1 A flow chart of a log-based backend service monitoring and alarm method provided in an embodiment of the present application;

[0029] Figure 2 A schematic diagram of the structure of a log-based back-end service monitoring and alarm device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0031] The present application embodiment provides a log-based backend service monitoring and alarm method, such as Figure 1 As shown, the log-based backend service monitoring and alarm method specifically includes steps S101-S105:

[0032] S101. Based on a preset log collection tool, log data in a backend business system is collected.

[0033] Specifically, the log collection tool needs to be integrated into the backend business system. The log collection tool includes at least: Logback tool, Kafka Connect tool, Fluentd tool and Sqoop tool.

[0034] Furthermore, based on a preset time interval and through a log collection tool, data collection is performed on the log data flow and the log application flow in the backend business system to obtain log data.

[0035] In one embodiment, a mature log collection tool (such as Logback in the Java technology system) is integrated into the back-end business system to ensure efficient generation and collection of log data. The log collection module should have flexible configuration capabilities, support custom log levels, formats, and output targets (such as files, network interfaces, etc.), and facilitate docking with subsequent log transmission modules.

[0036] S102: The log data is transmitted to the Kafka cluster, and log parsing is performed on the log data to obtain log parsing data.

[0037] Specifically, the log data needs to be pushed to the Kafka cluster through the preset distributed message queue system, and the log data is subscribed and consumed through the Kafka cluster.

[0038] Furthermore, key field data in the log data is extracted, wherein the key field data at least includes: timestamp, log level and error details.

[0039] Furthermore, it is also necessary to match the key field data in a data table based on the attribute item table to obtain log parsing data. The attribute item table at least includes: attribute value, attribute prefix and attribute content.

[0040] In one embodiment, a high-throughput, low-latency, highly reliable distributed message queue system (such as Kafka) is selected as the log transmission module. The logs generated by the back-end business system are pushed to the Kafka cluster via the log collection module, and then subscribed to and consumed by the log processing module. The distributed characteristics of Kafka ensure the stability and scalability of log transmission, and its message persistence function ensures the reliability of data. Then, the attribute item table can be used to match the key field data in data tables, parse the received logs, and extract key fields such as timestamps, log levels, and error details.

[0041] S103: Screen the log parsing data according to the preset regular matching rules to determine the log entry data that meets the alarm condition.

[0042] Specifically, firstly, based on the attribute item table in the log parsing data, the attribute items in the attribute item table are marked to obtain marked attribute items.

[0043] Furthermore, multiple types of regular matching rules corresponding to each tag attribute item are constructed, wherein each type of regular matching rule corresponds to each tag attribute item.

[0044] Furthermore, according to multiple types of regular matching rules, the log parsing data is traversed to determine abnormal candidate log entries.

[0045] Furthermore, the alarm conditions are used to match the attribute conditions of the abnormal candidate log entries to determine the log entry data that meets the alarm conditions.

[0046] In one embodiment, the configuration management module of the backend business system can be used to preset monitoring alarm configurations, including: Log regular matching rules: define regular expressions for specific log patterns to filter out log entries that need to be focused on. Alarm threshold: set the threshold for the number of log matches required to trigger an alarm to avoid frequent alarms or omissions of important issues. Alarm delivery method: specify the channels through which alarm information should be delivered to relevant personnel, which can be flexibly configured according to actual needs.

[0047] As a feasible implementation method, the parsed logs can be screened according to the regular matching rules preset by the configuration management module to identify the log entries that meet the alarm conditions: for example, the attribute prefixes in the attribute item table can be used for screening and matching, and multiple types of regular matching rules corresponding to the attribute prefixes can be constructed, and then the logs can be traversed and screened to determine the abnormal candidate log entries with the attribute prefixes. Then, the preset alarm conditions are used to match the attribute conditions of the abnormal candidate log entries with the attribute prefixes, and finally the abnormal log entry data under the attribute prefixes can be screened.

[0048] S104: Perform alarm threshold judgment on the log entry data to obtain alarm data.

[0049] Specifically, it is also necessary to determine the alarm threshold corresponding to each log entry in the log entry data based on the monitoring alarm configuration in the backend business system.

[0050] Furthermore, the number of alarm condition matches of each log entry is counted and judged with the alarm threshold to obtain alarm data, wherein the alarm data includes: alarm trigger data and alarm non-trigger data.

[0051] In one embodiment, the successfully matched log entries are counted according to the alarm threshold in the monitoring alarm configuration, that is, the alarm threshold corresponding to each log entry in the log entry data is determined. When the number of matches of a certain type of log exceeds the threshold, that is, the number of matches of the alarm condition of each log entry exceeds the alarm threshold, an alarm is triggered and alarm triggering data is generated.

[0052] S105: Encapsulate the log data corresponding to the alarm data and send it to the operation and maintenance center of the backend business system.

[0053] Specifically, the log data corresponding to the alarm trigger data in the alarm data is first processed with an alarm mark, and the log data is encapsulated to obtain the backend service alarm data. The backend service alarm data includes: abnormal timestamp, abnormal log level, error details, and log entry data that triggers the alarm.

[0054] Furthermore, according to the preset alarm sending method, the backend service alarm data is formatted and adapted to the relevant data structure, and sent to the operation and maintenance center of the backend business system, so that different operation and maintenance teams can receive the backend service alarm data. The alarm sending method includes at least: SMS, email, WeChat, DingTalk and Webhook.

[0055] In one embodiment, the alarm information can be received through the alarm module of the backend business system. The alarm module is responsible for receiving the alarm information issued by the log processing module and notifying according to the preset alarm delivery method, including: 1) Multi-channel delivery: Supports multiple alarm delivery methods such as SMS, email, WeChat, DingTalk, Webhook, etc. to meet the receiving habits of different team members and ensure timely delivery of alarm information. 2) Rich alarm information: Contains the timestamp, log level, error details and configuration information of the alarm trigger, helping operation and maintenance personnel quickly understand the cause of the alarm and improve the efficiency of problem location.

[0056] As a feasible implementation method, after the backend service alarm data is formatted and adapted to the relevant data structure, the preset alarm problem template can be used to perform template matching and data analysis on the formatted and adapted backend service alarm data, and generate initial alarm cause data. The initial alarm cause data is sent to the operation and maintenance center of the backend business system so that different operation and maintenance teams can complete the information positioning of the alarm data.

[0057] In addition, the present application embodiment also provides a log-based backend service monitoring and alarm device, such as Figure 2 As shown, the log-based backend service monitoring and alarm device 200 specifically includes:

[0058] At least one processor 201. And a memory 202 in communication with the at least one processor 201. The memory 202 stores instructions that can be executed by the at least one processor 201, so that the at least one processor 201 can execute:

[0059] Based on the preset log collection tool, collect log data from the backend business system;

[0060] The log data is transferred to the Kafka cluster, and the log data is parsed to obtain log parsed data;

[0061] According to the preset regular matching rules, the log parsing data is screened to determine the log entry data that meets the alarm conditions;

[0062] Perform alarm threshold judgment on log entry data to obtain alarm data;

[0063] The log data corresponding to the alarm data is packaged and sent to the operation and maintenance center of the back-end business system.

[0064] The embodiment of the present application avoids the workload of docking complex alarm systems and reduces integration costs by making full use of the existing log data of the back-end business system. The alarm information comes directly from the log, including detailed timestamps, log levels, error details, etc., which is conducive to developers to quickly locate the root cause of the fault. It also supports a variety of alarm delivery methods, which can adapt to the receiving habits of different team members, ensure that alarm information is communicated in a timely manner, and improve problem handling efficiency. The log collection module interacts with the business system through a standard interface, is not limited by the programming language used by the back-end service, and has good versatility. It only needs to integrate the log collection component in the business system, without the need to deeply modify the service code, and maintains the original structure and stability of the service.

[0065] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0066] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0067] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0068] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0069] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0070] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0071] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0072] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0073] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0074] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0075] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A log-based backend service monitoring and alarm method, characterized in that: The method comprises: Based on the preset log collection tool, collect log data from the backend business system; The log data is transmitted to the Kafka cluster, and the log data is parsed to obtain the log parsed data, specifically including: pushing the log data to the Kafka cluster through a preset distributed message queue system, and subscribing and consuming the log data through the Kafka cluster, extracting key field data in the log data, wherein the key field data at least includes: timestamp, log level and error details, and matching the key field data in a data table based on an attribute item table to obtain the log parsed data, wherein the attribute item table at least includes: attribute value, attribute prefix and attribute content; According to the preset regular matching rules, the log parsing data is screened to determine the log entry data that meets the alarm conditions; Performing alarm threshold judgment on the log entry data to obtain alarm data; The log data corresponding to the alarm data is packaged and sent to the operation and maintenance center of the back-end business system.

2. According to a log-based backend service monitoring and alarm method according to claim 1, it is characterized in that: Based on the preset log collection tool, log data in the backend business system is collected, including: Integrate the log collection tool into the backend business system; wherein the log collection tool at least includes: Logback tool, Kafka Connect tool, Fluentd tool and Sqoop tool; Based on a preset time interval and through the log collection tool, data collection is performed on the log data flow and the log application flow in the backend business system to obtain the log data.

3. According to a log-based backend service monitoring and alarm method according to claim 1, it is characterized in that: According to the preset regular matching rules, the log parsing data is screened to determine the log entry data that meets the alarm conditions, including: Based on the attribute item table in the log parsing data, marking the attribute items in the attribute item table to obtain marked attribute items; Constructing multiple types of regular matching rules corresponding to each of the tag attribute items; wherein each type of regular matching rule corresponds to each of the tag attribute items; According to the multiple types of regular matching rules, traverse the log parsing data to determine abnormal candidate log entries; According to the alarm condition, attribute condition matching is performed on the abnormal candidate log entries to determine the log entry data that meets the alarm condition.

4. According to a log-based backend service monitoring and alarm method according to claim 1, it is characterized in that: Performing alarm threshold judgment on the log entry data to obtain alarm data specifically includes: Determining the alarm threshold corresponding to each log entry in the log entry data based on the monitoring alarm configuration in the backend business system; The number of alarm condition matches of each log entry is counted and judged with the alarm threshold to obtain alarm data; wherein the alarm data includes: alarm triggering data and alarm non-triggering data.

5. A log-based backend service monitoring and alarm method according to claim 1, characterized in that: The log data corresponding to the alarm data is packaged and sent to the operation and maintenance center of the back-end business system, specifically including: Perform alarm identification processing on the log data corresponding to the alarm trigger data in the alarm data; and encapsulate the log data to obtain backend service alarm data; wherein the backend service alarm data includes: abnormal timestamp, abnormal log level, error details and log entry data that triggers the alarm; According to a preset alarm sending method, the backend service alarm data is formatted and adapted to the relevant data structure, and sent to the operation and maintenance center of the backend business system, so that different operation and maintenance teams can receive the backend service alarm data.

6. A log-based backend service monitoring and alarm method according to claim 5, characterized in that: After formatting the backend service alarm data to adapt the relevant data structure, the method further includes: Through the preset alarm problem template, the formatted and adapted back-end service alarm data is matched with the alarm problem template and analyzed, and the initial alarm cause data is generated; The initial alarm reason data is sent to the operation and maintenance center of the back-end business system, so that the different operation and maintenance teams can complete the information positioning of the alarm data.

7. A log-based backend service monitoring and alarm method according to claim 5, characterized in that: The alarm sending method includes at least: SMS, email, WeChat, DingTalk and Webhook.

8. A log-based backend service monitoring and alarm device, characterized in that: The device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, so that the at least one processor can execute the log-based back-end service monitoring and alarm method according to any one of claims 1-7.

9. A non-volatile computer storage medium, characterized in that: The storage medium is a non-volatile computer-readable storage medium, and the non-volatile computer-readable storage medium stores at least one program, each of which includes instructions, and when the instructions are executed by the terminal, the terminal executes a log-based back-end service monitoring and alarm method according to any one of claims 1-7.

Images