Service monitoring method, system, equipment and medium

Abstract

The invention provides a service monitoring method, system and device and a medium. The method comprises the steps of obtaining a log of at least one service; according to an analysis rule of each service in the at least one service, analyzing the log of each service to obtain a service field set of each service; an alarm set of each service is determined, the alarm set comprises alarm fields and alarm rules, and the alarm fields and the alarm rules are in one-to-one correspondence; for each service, according to each alarm field in the alarm set, matching with each service field in the service field set; under the condition that the business field is matched with the alarm field, the alarm is performed based on the alarm rule corresponding to the alarm field, so that diversified and personalized configuration of the alarm can be realized, the tedious processing flow of a worker can be saved, and the alarm efficiency is greatly improved.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a business monitoring method, system, device and medium. Background Art

[0002] During the operation of the application system, it is necessary to monitor the business functions of the application system to ensure the stability of the application system and the normal operation of the business.

[0003] When a business function fails, an alert is generally issued for the entire log at the log level. In fact, there are many situations in each business, and different business nodes in a business have different responsible persons. The current method of directly alerting the entire log is not flexible enough and often brings heavy workload. Summary of the invention

[0004] In order to solve the problem of heavy workload caused by the method of issuing an alarm for the entire log in the prior art, the present invention provides a business monitoring method, system, device and medium.

[0005] In a first aspect, an embodiment of the present disclosure provides a service monitoring method, the method comprising:

[0006] Get the log of at least one business;

[0007] Parse the log of each business according to the parsing rule of each business in at least one business to obtain a business field set of each business;

[0008] Determine the alarm set for each service. The alarm set includes alarm fields and alarm rules. The alarm fields correspond to the alarm rules one by one.

[0009] For each service, match each alarm field in the alarm set with each service field in the service field set;

[0010] When the service field matches the alarm field, an alarm is issued based on the alarm rule corresponding to the alarm field.

[0011] In one embodiment, the log includes a service identifier; the service identifier is used to distinguish each service.

[0012] In one embodiment, the parsing rule includes a rule identifier;

[0013] The method also includes:

[0014] Determine the parsing rules for each business; the parsing rules for each business and the logs of each business meet a first preset condition, the first preset condition including that the rule identifier of the parsing rule matches the business identifier of the log.

[0015] In one embodiment, according to the parsing rule of each business in at least one business, before parsing the log of each business to obtain the business field set of each business, the method further includes:

[0016] Determine the log level of each business log; the log level is used to indicate that the log corresponding to the log level is to be parsed when the second preset condition is met.

[0017] In one embodiment, the parsing rule includes at least one parsing field;

[0018] According to the parsing rule of each business in at least one business, the log of each business is parsed to obtain a business field set of each business, including:

[0019] For each business, match each parsed field in at least one parsed field with the log;

[0020] The fields in the log that meet the third preset condition are determined as business fields, and a business field set for each business is obtained; the third preset condition includes a field that matches the parsed field.

[0021] In one embodiment, the alarm rule includes at least one alarm sub-rule, and at least two alarm sub-rules of the at least one alarm sub-rule generate alarms according to a set priority during the alarm process.

[0022] In one embodiment, after obtaining the log of at least one service, the method further includes:

[0023] A log of each service in at least one service is stored.

[0024] In a second aspect, an embodiment of the present disclosure provides a service monitoring system, the system comprising:

[0025] An acquisition module, used to acquire a log of at least one business;

[0026] A parsing module, used to parse the log of each business according to the parsing rule of each business in at least one business, and obtain a business field set of each business;

[0027] The first determination module determines an alarm set for each service, wherein the alarm set includes an alarm field and an alarm rule, and the alarm field and the alarm rule correspond one to one;

[0028] A matching module, used for matching each service field in the service field set with each alarm field in the alarm set for each service;

[0029] The alarm module is used to generate an alarm based on the alarm rule corresponding to the alarm field when the business field matches the alarm field.

[0030] In one embodiment, the log includes a service identifier; the service identifier is used to distinguish each service.

[0031] In one embodiment, the parsing rule includes a rule identifier;

[0032] The system also includes:

[0033] The second determination module is used to determine the parsing rules for each business; the parsing rules for each business and the logs of each business meet a first preset condition, and the first preset condition includes that the rule identifier of the parsing rule matches the business identifier of the log.

[0034] In one embodiment, before executing the parsing module, the system further includes:

[0035] The third determination module is used to determine the log level of each business log; the log level is used to indicate to parse the log corresponding to the log level when the second preset condition is met.

[0036] In one embodiment, the parsing rule includes at least one parsing field;

[0037] The parsing modules include:

[0038] A matching unit, configured to match each parsing field in at least one parsing field with the log for each service;

[0039] The determination unit is used to determine the fields in the log that meet the third preset condition as business fields, and obtain a business field set for each business; the third preset condition includes a field that matches the parsed field.

[0040] In one embodiment, the alarm rule includes at least one alarm sub-rule, and at least two alarm sub-rules of the at least one alarm sub-rule generate alarms according to a set priority during the alarm process.

[0041] In one embodiment, after executing the acquisition module, the method further includes:

[0042] The storage module is used to store the log of each business in at least one business.

[0043] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement a business monitoring method as described in any one of the first aspects.

[0044] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, characterized in that at least one instruction or at least one program is stored in the computer-readable storage medium, and the at least one instruction or at least one program is loaded and executed by a processor to implement a business monitoring method as described in any one of the first aspects.

[0045] The service monitoring method, system, device and medium provided by the embodiments of the present disclosure have the following technical effects:

[0046] In this embodiment, the log is parsed through parsing rules to obtain a business field set including at least one business field. In the process of business monitoring, the field content in the log system can be accurately obtained, and then the alarm field in the alarm set is matched with the business field in the business field set. On the premise that the alarm field matches the business field, an alarm is issued according to the alarm rule corresponding to the alarm field. The alarm field can identify abnormal content in the business field, and different alarm rules are configured for different alarm fields, which can realize diversified and personalized configuration of alarms, save the staff the cumbersome processing procedures, and greatly improve the efficiency of alarms.

[0047] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present disclosure or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 is a schematic diagram of an application environment provided by an embodiment of the present disclosure;

[0050] Figure 2 is a flow chart of a business monitoring method provided by an embodiment of the present disclosure;

[0051] Figure 3 is a flowchart of step S22 of a service monitoring method provided by an embodiment of the present disclosure;

[0052] Figure 4a It is a schematic diagram of a log of a unified acquiring service under general circumstances provided by an embodiment of the present disclosure;

[0053] Figure 4b It is a schematic diagram of a log of a unified acquiring service provided by an embodiment of the present disclosure in a case where an alarm is required;

[0054] Figure 4c is a schematic diagram of an international distribution service log under general circumstances provided by an embodiment of the present disclosure;

[0055] Figure 4d is a schematic diagram of a log of an international distribution service in a situation where an alarm is required provided by an embodiment of the present disclosure;

[0056] Figure 5 is a schematic diagram of an execution framework of a business monitoring method provided by an embodiment of the present disclosure;

[0057] Figure 6 is a module diagram of a business monitoring system provided by an embodiment of the present disclosure;

[0058] Figure 7 It is a structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0060] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0061] See also Figure 1 , Figure 1It is a schematic diagram of an application environment provided by an embodiment of the present application, and the schematic diagram includes a terminal 101 and a server 102, wherein the terminal 101 includes but is not limited to any one or more of a PC, a mobile phone, and a tablet computer. The server 102 is connected to the terminal 101 in communication, and the communication connection mode includes a wireless connection mode and a wired connection mode, etc. The wireless connection mode includes but is not limited to Wi-Fi, Bluetooth, ZigBee (a short-range wireless connection mode), LoRa (long-range wireless), etc., and the wired connection mode includes but is not limited to Ethernet, optical fiber, USB, serial port, parallel port, etc. In an optional embodiment, the server 102 may also include a terminal server set on the terminal 101. In another optional real-time mode, at least one service can be loaded on a terminal 101, and a terminal 101 can collect logs of at least one service. In another optional embodiment, the server 102 includes parsing rules and an alarm set, and the server 102 receives the logs collected by the terminal 101, parses the logs according to the parsing rules, and alarms the erroneous logs according to the alarm set.

[0062] The following describes a specific embodiment of a service monitoring method of the present application. Figure 2 It is a flowchart of a business monitoring method provided by an embodiment of the present application. This specification provides method operation steps such as the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or server product is executed, it can be executed in the order shown in the embodiment or the drawings or in parallel (for example, in a parallel processor or multi-threaded processing environment). Specifically, Figure 2 As shown, the method may include:

[0063] S21. Obtain a log of at least one service.

[0064] Among them, logs are used to record business-related operations, events, and states during the operation of the business system. They can be used to track business processes, diagnose problems, audit operations, and ensure the normal operation of the system. Common business types include: operational business (such as order creation, payment management, user login, etc.), event tasks (such as transaction completion, invoice generation, inventory management, etc.), audit business (operations related to security and compliance in the system), and performance business (performance indicators of business systems, such as response time, processing time, etc.).

[0065] S22. According to the parsing rule of each service in at least one service, parse the log of each service to obtain a service field set of each service.

[0066] Log alert is a monitoring mechanism that detects abnormal information or specific events in the business system in real time and promptly notifies relevant personnel when a problem occurs. One business system can correspond to one business.

[0067] In order to accurately warn of abnormal information in the log and reduce the workload of managers for screening and abnormal determination, in this embodiment, parsing rules can be used to parse the log to obtain a set of business fields contained in the log. Since the formats of logs of different businesses are different, the parsing rules configured for the logs of each business are also different. For the accuracy of log parsing, the parsing rules of the logs of each business need to be matched before parsing. Parsing rules can generally be manually set by managers, or automatically set by the system according to the normal process of the log. No special restrictions are made here, and selection is made according to actual conditions.

[0068] S23. Determine the alarm set for each service.

[0069] The alarm set includes alarm fields and alarm rules, and the alarm fields correspond to the alarm rules one by one.

[0070] The alarm field is used to match the business field in the business field set. When the alarm field matches the business field, it means that the content corresponding to the business field is abnormal. At the same time, in order to accurately and timely alarm for the abnormal business field, set the alarm rule corresponding to the alarm field. The alarm rule can specifically include at least one of the responsible person, alarm name, alarm type, alarm interval, alarm triggering method, alarm processing method, etc., which can be configured according to the actual situation.

[0071] S24. For each service, match each alarm field in the alarm set with each service field in the service field set.

[0072] In the process of matching the alarm field, each alarm field in the alarm set can be used to search each business field in the business field set. When a business field matching the alarm field appears in the business field set, the alarm rule corresponding to the alarm field will be triggered to issue an alarm.

[0073] Specifically, an expression of an alarm field or a business field includes a field name and field content.

[0074] An alarm field may include at least one alarm subfield. Multiple alarm subfields in at least one alarm subfield will be isolated by @@. The specific field name and field content are written in the middle of @@. At the same time, the field name and field content can be separated by ":".

[0075] The matching methods of the alarm field and the service field include any one or more of equal matching, wildcard matching, or matching, not equal matching, and regular matching. The matching methods are generally defined in the expression of the alarm field. In the expression of the alarm field, * usually represents wildcard matching, | represents or matching, and ! represents not equal matching.

[0076] In an example, the alarm field is, for example, @appname:llgp-channel-gw-alipay@, where the field name is appname and the field content is llgp-channel-gw-alipay.

[0077] In an example, multiple warning subfields in at least one warning subfield, such as @appname:llgp-channel-gw-alipay@&@loglvl:ERROR@&@message:*PayService.queryPaymentexception*org.Springframeworkweb.client.Re sourceAccessException:I / O error on GETrequest for*@.

[0078] Among them, @appname:llgp-channel-gw-alipay@ specifies the application name of the log entry, indicating that you are looking for logs from the llgp-channel-gw-alipay application.

[0079] @loglvl:ERROR@ specifies the log level as ERROR, that is, only log entries recorded at the error level are searched.

[0080] @message:*PayService.queryPaymentexception*org.Springframeworkweb.client.ResourceAccessException:I / O error on GET request for*@, specifies to filter the log message content, which is divided into three parts:

[0081] The wildcard * in *PayService.queryPaymentexception* represents any character, and searches for entries containing PayService.queryPaymentexception in the message, regardless of whether there is other content before or after.

[0082] org.springframework.web.client.ResourceAccessException:I / Oerror onGET request for* represents that this part is the exception information, which means to find entries related to org.springframework.web.client.ResourceAccessException, indicating that an I / O error occurred in the GET request. The * wildcard at the end of the field content allows any content to follow, such as a URL or other related information.

[0083] The method for matching the alarm fields in the alarm set with the business field set provided in this embodiment is applicable to complex logs (such as ELK, Splunk, etc.), and can quickly locate specific errors in a large number of logs. In particular, when a specific exception or problem occurs in a business system, the corresponding exception content can be accurately screened out by matching the alarm field with the business field set, so that the exception can be efficiently handled in subsequent processing.

[0084] S25. When the service field matches the alarm field, an alarm is issued based on the alarm rule corresponding to the alarm field.

[0085] Among them, the alarm rules include but are not limited to at least one or more of the business identifier, policy name, alarm responsible person, alarm level, importance, alarm type, alarm interval, alarm triggering method, alarm processing method, alarm notification method, and processing plan, and may also include other content besides the above examples, which is selected according to the actual situation.

[0086] Generally speaking, each alarm field in the alarm set has a corresponding alarm rule to match. When the business field in the business field set matches the alarm field, it means that an abnormality corresponding to the alarm field occurs in the business system. At this time, the alarm rule matching the alarm field can be determined to issue an alarm. Targeted alarms can be issued for each different abnormal content in the business, thereby improving alarm efficiency.

[0087] The following are examples of alarm rules. Before that, it should be noted that the alarm rules can be configured by management personnel or artificial intelligence according to actual application scenarios. Therefore, the alarm rules have a large degree of freedom and are not limited to the following examples:

[0088] The policy name can be configured according to the specific application scenario and is not specifically limited here.

[0089] The alarm responsible person includes the function responsible person and the business responsible person. The function responsible person is the developer corresponding to the part of the log that generates the alarm, and the business responsible person is the person in charge of the corresponding business. During the alarm process, both the function responsible person and the business responsible person will receive the alarm information. There must be at least one function responsible person or business responsible person. When there are multiple function responsible persons or business responsible persons, they can be arranged in ascending order according to their levels.

[0090] Alarm levels include system crash (SYSTEM_CRASH), impact on all transactions (AFFECT_ALL), impact on some transactions (AFFECT_PART), business alarm to be handled in time (BUSINESS_HANDLE_IN_TIME), unclassified message (UNCLEAR), business alarm does not affect transactions (BUSINESS_ALARM), general prompt (TIPS), none (NONE), etc.

[0091] Importance, including IMPORTANT, MODERATELY, LIGHT, EDGE, etc.

[0092] Alarm types include simultaneous alarms for all user table configurations (USER_MEANWHILE), simultaneous alarms for functional managers (LIABLE_MEANWHILE), simultaneous alarms for business managers (LIABLE_APP_MEANWHILE), alarms for functional managers one by one (LIABLE_ONEBYONE), alarms for business managers one by one (APP_ONEBYONE), and alarms for functional application managers one by one (LIABLE_APP_ONEBYONE). Specifically, the alarm method includes first alarming the first person in charge. After a preset period of time, if no feedback is received from the first person in charge that the processing has been completed, the second person in charge is alarmed, and so on, until the highest-level person in charge is alarmed.

[0093] Alarm processing methods include single alarm (SINGLE), alarm according to the most urgent level of the link (TRANCE), and ignore (IGNORE).

[0094] Alarm triggering modes include delayed triggering (DELAY), immediate triggering (IMMEDIATELY) and interval alarm.

[0095] Interval alarms include NONE, which means no interval and an alarm will be triggered if a hit occurs; COUNT:100, which means an alarm will be triggered every 100 items; INTERVAL:100, which means an alarm will be triggered every 100 seconds; COUNT:100&INTERVAL:100, which means an alarm will be triggered only when there are more than 100 items within 100 seconds; +COUNT:100&INTERVAL:100, which means an alarm will be triggered only when there are more than 100 items within 100 seconds, and the first alarm will be triggered (where + means the first alarm, that is, an alarm will be triggered once no matter how a hit occurs).

[0096] Alarm notification methods include but are not limited to email, phone, text message, WeChat (an instant messaging software), Feishu (an office communication software), DingTalk (an office communication software), etc.

[0097] The processing plan can be a processing suggestion sent to the person in charge through an alarm notification when an alarm occurs. The processing plan can be configured by the management personnel; it can be intelligently retrieved from the knowledge base based on the alarm field and automatically generated; it can also be automatically generated when configuring by learning from the knowledge base and historical policy rules based on machine learning and natural language processing models to obtain a trained model.

[0098] According to the alarm rules in the above example, the specific process of alarm is explained: after determining the alarm rules for alarm, a specific alarm task is generated according to the corresponding alarm rules and alarm fields, and sent to the corresponding person in charge via the set alarm notification method. After the person in charge receives the alarm task, he can automatically jump to the corresponding business system for processing.

[0099] In one embodiment, the alarm rule may further include at least one alarm sub-rule, and the contents of the alarm sub-rule may refer to the above example. Each alarm sub-rule may be set with a corresponding priority, and during the alarm process, at least two of the at least one alarm sub-rule may be set to alarm according to the set priority.

[0100] In one embodiment, the log includes a transaction identification.

[0101] Specifically, the business ID is different depending on the business corresponding to the log. The business ID is mainly used to distinguish each business. In the subsequent series of processing processes, such as log parsing and alarm rule matching, each business needs to be isolated through the business ID.

[0102] Since the service monitoring method provided in this embodiment is not only for a single service scenario, but for various types of multi-service scenarios. In this scenario, multiple service logs may be obtained at the same time. If the logs of each service are not isolated, confusion may occur. For example, during the log parsing process, the service fields of service A and service B may be confused, which will seriously affect the accuracy and efficiency of subsequent alarms.

[0103] In one embodiment, isolating log parsing by business identification is described, and the method further includes:

[0104] Determine the parsing rules for each business.

[0105] The parsing rule includes a rule identifier, and the parsing rule of each service and the log of each service meet a first preset condition, wherein the first preset condition includes that the rule identifier of the parsing rule matches the service identifier of the log.

[0106] Before parsing, the parsing rules corresponding to the log can be searched by matching the rule identifier with the business identifier, thereby improving parsing efficiency.

[0107] In one embodiment, before step S22, the method further includes:

[0108] Determine the log level for each service log.

[0109] The log level is used to instruct to parse the log corresponding to the log level when the second preset condition is met.

[0110] Specifically, the log level is used to indicate whether an exception occurs in the log, and the second preset condition includes that the log level indicates that an exception occurs in the log. For example, when the log level is warn or error, it indicates that an exception occurs in the log.

[0111] For this embodiment, before performing log analysis, the logs may be filtered according to the log level. Logs with log levels indicating that no abnormality occurs may not be processed, so as to improve the efficiency of business monitoring.

[0112] In one embodiment, the log parsing process is described in detail. The parsing rule includes at least one parsing field. Figure 3 , step S22 specifically includes:

[0113] S221. For each business, match each parsing field in at least one parsing field with a log.

[0114] S222: Determine the fields in the log that meet the third preset condition as business fields, and obtain a business field set for each business.

[0115] The third preset condition includes a field that matches the parsed field.

[0116] In one embodiment, the service monitoring method further includes:

[0117] Record the alarm.

[0118] Specifically, it records the triggered alarms, including business identification, business-related information, person in charge-related information, triggered alarm rules, specific content of the log, etc.

[0119] In one embodiment, after step S21, the method further includes:

[0120] A log of each service in at least one service is stored.

[0121] To facilitate understanding of the service monitoring method provided in this embodiment, two specific services are used for illustration below:

[0122] The unified acquiring service, whose service identifier is LLGP, can be used to isolate the logs, fields and other related data of the unified acquiring service during the subsequent monitoring process. Figure 4a This is the log of the unified acquiring business under normal circumstances. Figure 4b It is a log of unified acquiring business in case of alarm. Figure 4a and Figure 4b The log includes multiple business fields.

[0123] According to the unified parsing rules of acquiring business, Figure 4a and Figure 4b The logs in are parsed into a business field set of unified acquiring business, where the business field set is composed of the following business fields: logtime, appname, loglvl, pid, pname, tid, spanid, spanExportable, classpath, methodname, codeline, and content.

[0124] Next, determine the unified alarm set for acquiring services, including alarm fields and alarm configurations.

[0125] Alert fields are as follows:

[0126] @appname:llgp-channel-front-service@&@loglvl:WARN@&@clas spath:com.lianlian.llgp.channel.front.core.ChannelCodeStatusConverter@&@methodname:getChannelSerialStatusAndUniteCode@&@content:*channel serial status and unitecode get exception*@.

[0127] Alarm configuration example:

[0128] The alarm level is TIPS, and the alarm notification methods are WeChat and Lark;

[0129] The alarm level is BUSINESS_ALARM, and the alarm notification methods are WeChat, Lark, and email;

[0130] The alarm level is UNCLEAR, and the alarm notification method is Lark and email;

[0131] The alarm level is BUSINESS_HANDLE_IN_TIME, and the alarm notification methods are Lark and email.

[0132] The above alarm configuration is only an example. You can select it according to your needs. The alarm configuration corresponds to the alarm field.

[0133] The international distribution service, whose service identifier is LGPS, can be used to isolate the logs, fields and other related data of the international distribution service during the subsequent monitoring process. Figure 4c For general international distribution business logs, Figure 4d Logs for international distribution of services in case of alarms. Figure 4c and Figure 4d The log includes multiple business fields.

[0134] According to the analysis rules of international distribution business, Figure 4c and Figure 4d The logs in are parsed into a business field set for international distribution business, where the business field set consists of the following business fields: logtime, loglvl, pname, classpath, tid, and content.

[0135] Next, determine the alarm set for the international distribution service, including alarm fields and alarm configuration.

[0136] Alert fields are as follows:

[0137] @loglvl:ERROR@&@classpath:

[0138] com.lianlian.cb.lgps.service.handler.PayoutUpiService

[0139] com.lianlian.cb.cbewcommonservice.core.template.impl.ActionTemplateImpl@&@content:*upi credit verify error*@.

[0140] Alarm configuration example:

[0141] The alarm level is TIPS, and the alarm notification method is Feishu;

[0142] The alarm level is BUSINESS_ALARM, and the alarm notification methods are Lark and email;

[0143] The alarm level is UNCLEAR, and the alarm notification method is Lark and email;

[0144] The alarm level is BUSINESS_HANDLE_IN_TIME, and the alarm notification methods are Lark and email.

[0145] The above alarm configuration is only an example. You can select it according to your needs. The alarm configuration corresponds to the alarm field.

[0146] In one embodiment, see Figure 5 , the execution framework of this embodiment is further described:

[0147] The log of each business can be collected by Filebeat (file transmitter) in the business system in the terminal 101 . Filebeat can be set in the terminal 101 or in the server 102 .

[0148] The collected logs are sent to Kafka. The message queue data source collected in Kafka, that is, the Kafka server address and subscribed Topic, can be configured through the business identifier.

[0149] Among them, Kafka is a distributed stream processing platform, and its main uses include: message queue, which can process large amounts of real-time data streams; log collection, which collects and transmits logs of business systems in a centralized manner; data integration, which serves as a central data bus that can transfer data between different systems.

[0150] Logstash can consume logs in Kafka and generate two links to process logs according to the log level. The first link resends logs with log levels of warn or error to Kafka; the second link sends logs to ElasticSearch for storage. At the same time, Logstash will parse the logs to obtain the business field set of the logs.

[0151] Among them, Logstash is a data collection and processing pipeline tool used to collect, transform and send data from multiple sources to the storage system. ElasticSearch is a distributed search and analysis engine used to store, search and analyze large-scale data.

[0152] For the logs resent to Kafka, they can be consumed by the alarm application, and operations such as matching alarm fields and alarms can be performed. For the alarm application, the alarm fields and alarm rules in the alarm collection can be configured in the alarm application. In the alarm application, this can be achieved without the support of ElasticSearch.

[0153] In this embodiment, the log is parsed through parsing rules to obtain a business field set including at least one business field. In the process of business monitoring, the field content in the log system can be accurately obtained, and then the alarm field in the alarm set is matched with the business field in the business field set. On the premise that the alarm field matches the business field, an alarm is issued according to the alarm rule corresponding to the alarm field. The alarm field can identify abnormal content in the business field, and different alarm rules are configured for different alarm fields, which can realize diversified and personalized configuration of alarms, save the staff the cumbersome processing procedures, and greatly improve the efficiency of alarms.

[0154] An exemplary embodiment of the present disclosure provides a service monitoring system. Figure 6 , the system includes:

[0155] The acquisition module 61 is used to acquire a log of at least one service.

[0156] The parsing module 62 is used to parse the log of each business according to the parsing rule of each business in at least one business to obtain a business field set of each business.

[0157] The first determination module 63 determines an alarm set for each service. The alarm set includes an alarm field and an alarm rule. The alarm field and the alarm rule correspond to each other one by one.

[0158] The matching module 64 is used to match each service field in the service field set according to each alarm field in the alarm set for each service.

[0159] The alarm module 65 is used to generate an alarm based on the alarm rule corresponding to the alarm field when the service field matches the alarm field.

[0160] In one embodiment, the log includes a service identifier; the service identifier is used to distinguish each service.

[0161] In one embodiment, the parsing rule includes a rule identification.

[0162] The system also includes:

[0163] The second determination module is used to determine the parsing rules for each business; the parsing rules for each business and the logs of each business meet a first preset condition, and the first preset condition includes that the rule identifier of the parsing rule matches the business identifier of the log.

[0164] In one embodiment, before executing the parsing module 62, the system further includes:

[0165] The third determination module is used to determine the log level of each business log; the log level is used to indicate to parse the log corresponding to the log level when the second preset condition is met.

[0166] In one embodiment, the parsing rule includes at least one parsing field;

[0167] The parsing module 62 includes:

[0168] A matching unit, configured to match each parsing field in at least one parsing field with the log for each service;

[0169] The determination unit is used to determine the fields in the log that meet the third preset condition as business fields, and obtain a business field set for each business; the third preset condition includes a field that matches the parsed field.

[0170] In one embodiment, the alarm rule includes at least one alarm sub-rule, and at least two alarm sub-rules of the at least one alarm sub-rule generate alarms according to a set priority during the alarm process.

[0171] In one embodiment, after executing the acquisition module 61, the method further includes:

[0172] The storage module is used to store the log of each business in at least one business.

[0173] The device and service monitoring method embodiments in the embodiments of the present disclosure are based on the same disclosed concept.

[0174] An exemplary embodiment of the present disclosure provides an electronic device, such as Figure 7 As shown, the electronic device 700 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 710 (the processor 710 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 730 for storing data, and one or more storage media 720 (such as one or more mass storage devices) for storing application programs 723 or data 722. Among them, the memory 730 and the storage medium 720 can be short-term storage or permanent storage. The storage medium 720 stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the above-mentioned business monitoring method.

[0175] Furthermore, the CPU 710 may be configured to communicate with the storage medium 720 to execute a series of instruction operations in the storage medium 720 on the electronic device 700. The electronic device 700 may also include one or more power supplies 760, one or more wired or wireless network interfaces 70, one or more input and output interfaces 740, and / or one or more operating systems 721, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0176] The input / output interface 740 may be used to receive or send data via a network. The specific example of the network may include a wireless network provided by a communication provider of the electronic device 700. In one example, the input / output interface 740 includes a network adapter (Network Interface Controller, NIC), which may be connected to other network devices via a base station so as to communicate with the Internet. In one example, the input / output interface 740 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0177] It can be understood by those skilled in the art that Figure 7 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 7 More or fewer components as shown, or with Figure 7 Different configurations are shown.

[0178] An embodiment of the present disclosure also provides a computer-readable storage medium, which can be set in a server to store at least one instruction, at least one program, a code set or an instruction set related to a business monitoring method in an embodiment of the business monitoring method. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the above-mentioned business monitoring method.

[0179] Optionally, in this embodiment, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0180] It should be noted that the above-mentioned sequence of the embodiments of the present disclosure is for description only and does not represent the advantages and disadvantages of the embodiments. The above-mentioned specific embodiments of this specification are described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0181] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the business monitoring method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the business monitoring method embodiment.

[0182] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0183] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A business monitoring method, characterized in that: The method comprises: Get the log of at least one business; Parsing the log of each business according to the parsing rule of each business in the at least one business to obtain a business field set of each business; Determine an alarm set for each service, the alarm set comprising an alarm field and an alarm rule, the alarm field corresponding to the alarm rule one by one; For each of the services, matching each alarm field in the alarm set with each service field in the service field set; In the case where the service field matches the alarm field, an alarm is issued based on the alarm rule corresponding to the alarm field.

2. The service monitoring method according to claim 1, characterized in that: The log includes a service identifier; the service identifier is used to distinguish each service.

3. The service monitoring method according to claim 2, characterized in that: The parsing rule includes a rule identifier; The method further comprises: Determine the parsing rule for each business; the parsing rule for each business and the log of each business meet a first preset condition, wherein the first preset condition includes that the rule identifier of the parsing rule matches the business identifier of the log.

4. The service monitoring method according to claim 1, characterized in that: Before parsing the log of each business according to the parsing rule of each business in the at least one business to obtain the business field set of each business, the method further includes: Determine the log level of the log of each business; the log level is used to indicate to parse the log corresponding to the log level when the second preset condition is met.

5. The service monitoring method according to claim 1, characterized in that: The parsing rule includes at least one parsing field; The step of parsing the log of each business according to the parsing rule of each business in the at least one business to obtain the business field set of each business includes: For each of the services, matching each parsed field in the at least one parsed field with the log; The fields in the log that meet the third preset condition are determined as business fields, and a business field set of each business is obtained; the third preset condition includes a field that matches the parsed field.

6. The service monitoring method according to claim 1, characterized in that: The alarm rule includes at least one alarm sub-rule, and at least two alarm sub-rules of the at least one alarm sub-rule generate alarms according to a set priority during the alarm process.

7. The service monitoring method according to claim 1, characterized in that: After obtaining the log of at least one service, the method further includes: The log of each service in the at least one service is stored.

8. A business monitoring system, characterized in that: The system comprises: An acquisition module, used to acquire a log of at least one business; A parsing module, configured to parse the log of each business according to the parsing rule of each business in the at least one business, to obtain a business field set of each business; A first determination module determines an alarm set for each service, wherein the alarm set includes an alarm field and an alarm rule, and the alarm field corresponds to the alarm rule in a one-to-one manner; A matching module, configured to match each service field in the service field set according to each alarm field in the alarm set for each service; The alarm module is used to generate an alarm based on the alarm rule corresponding to the alarm field when the service field matches the alarm field.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement the business monitoring method as described in any one of claims 1-6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by a processor to implement the service monitoring method according to any one of claims 1 to 6.

Images