Abnormal equipment monitoring method, system and device

Abstract

The invention provides an abnormal equipment monitoring method, system and device, and the method comprises the steps: responding to the received log information sent by supervised equipment, and carrying out the analysis of the log information, and obtaining a log analysis result; matching the log analysis result with an alarm rule in a preset rule set; in response to determining that a first alarm rule matched with the log analysis result is matched and the first alarm rule is associated with an exception handling instruction, sending the exception handling instruction for automatically repairing the hidden danger or the fault to the supervised equipment; and in response to a second alarm rule which is determined to be matched with the log analysis result and is not associated with the exception processing instruction, outputting an alarm prompt, and outputting an alarm prompt once in a time period for a plurality of pieces of log information corresponding to the same exception. Through centralized collection and online analysis of logs, hidden dangers and faults in an IT system environment are monitored in real time, faults are reduced as much as possible, the faults are quickly solved after occurring, and stable operation of the system is kept.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, system and device for monitoring abnormal equipment. Background Art

[0002] The reliability of equipment hardware, changes in the computer room environment, bugs (faults) triggered by business scenarios not considered in system design, and bugs in operating systems and platform components can all lead to various anomalies in the operation of IT (Information Technology) systems, making it difficult to ensure that the system always runs stably. Therefore, it is an important task for IT operation and maintenance to strengthen monitoring, discover hidden dangers as early as possible and eliminate them, locate and eliminate faults as soon as possible when faults occur, and minimize the impact of IT system faults on supporting businesses.

[0003] Among them, end-to-end business monitoring, system user feedback, and installation of agent software to monitor server operating status are some of the main measures currently used to monitor and discover IT system failures.

[0004] For end-to-end business monitoring, business operation logs, automatic simulated dialing and other methods can be used to understand business support conditions, proactively discover abnormal situations in the business process, and promptly notify IT operation and maintenance personnel to conduct relevant investigations. However, monitoring business anomalies is only the appearance of IT system anomalies. Locating the network element equipment and servers that cause anomalies requires a series of complex processes.

[0005] IT system front-end user feedback is a passive method in which system users inform system abnormalities. Abnormal events may have already occurred when users give feedback, resulting in poor timeliness in fault discovery, difficulty in locating, and heavy workload in troubleshooting.

[0006] Installing agent software to monitor the server's operating status can proactively discover and report hidden dangers and faults that occur during server operation in a timely manner. It has good timeliness and can directly locate the faulty server. However, agent software needs to be developed separately for different operating system versions. The monitoring scope and capabilities of agent software are also related to the cognition of engineers in the development stage. Some hidden dangers and faults that are not considered are easily overlooked, which has limitations. In addition, agent software is not suitable for network element devices, and it is difficult to monitor abnormal conditions such as switches and network cables in the IT operating environment through agent software. In an environment with many servers, installing and deploying agent software for each server is also a relatively complicated task, which is prone to omissions, especially new equipment is often overlooked, resulting in omissions. In addition, the agent software itself will cause a large resource consumption of the device, and it may also have bugs and induce server abnormalities.

[0007] Since system logs record various events that occur during the operation of servers and network element devices, especially various abnormal events that occur during the operation of devices, they will be recorded in a timely and detailed manner. Therefore, system logs are of great reference significance for fault cause analysis and troubleshooting.

[0008] Real-time analysis of log streams can promptly detect abnormal situations reflected in logs and quickly locate abnormal devices. By analyzing past fault cases, it can be found that many faults generated log content with significant characteristics when they occurred. Before some faults occur, a lot of hidden danger information has been recorded in the system log. Real-time monitoring of log streams can capture hidden dangers in time and deal with them immediately when they occur, avoiding hidden dangers from turning into faults that affect the normal operation of business. However, in a complex IT environment, how to achieve full monitoring and rapid analysis of logs generated by a large number of devices is a major challenge.

[0009] Another problem with using real-time analysis of log streams to monitor hidden dangers and failures is how to solve log storms. For example, if there is insufficient remaining disk space to satisfy a file write request, the process will usually not stop, but will continue to issue file write requests based on business requests, continuously generating log records of write failures. Depending on the business situation, it is likely that dozens of write failure log records will be generated per second. If an alarm is triggered for each write failure record, it will cause great interference to the operation and maintenance work and reduce the efficiency of the operation and maintenance work. And because the network and business dependencies are interrelated, the occurrence of a fault may cause a large number of related devices and related businesses to be abnormal at the same time, and generate a large number of related alarms in a short period of time. This is another manifestation of log storms.

[0010] It can be seen that in the existing IT system fault monitoring, the end-to-end business monitoring method has the disadvantage of complex process of locating abnormal devices; the monitoring method based on system user feedback has the problems of poor timeliness of fault discovery, difficulty in locating, and large workload of troubleshooting; the method of installing agent software for monitoring has great limitations, and the installation process is complicated, and the agent software will cause great resource consumption on the equipment; the existing method of fault monitoring based on analyzing system logs is difficult to monitor the logs generated by multiple devices in full, and log storms will cause great interference to operation and maintenance work, affecting the application effect of using log analysis for fault monitoring. Summary of the invention

[0011] In view of the above problems, embodiments of the present application provide a method, system and apparatus for monitoring abnormal equipment that overcome the above problems or at least partially solve the above problems.

[0012] In a first aspect, an embodiment of the present application provides an abnormal device monitoring method, which is applied to a log collection device, wherein the log collection device is associated with a plurality of supervised devices, and the method includes:

[0013] In response to receiving the log information sent by the supervised device, parsing the log information to obtain a log parsing result;

[0014] Matching the log analysis result with the alarm rules in the preset rule set to perform abnormal monitoring on the supervised device;

[0015] In response to determining a first alarm rule that matches the log parsing result and the first alarm rule being associated with an exception handling instruction, sending the exception handling instruction to the supervised device, the exception handling instruction being used to automatically repair a hidden danger or fault;

[0016] In response to determining a second alarm rule that matches the log parsing result and is not associated with an exception handling instruction, an alarm prompt is output, and an alarm prompt is output once within a time period for multiple log information corresponding to the same exception.

[0017] In a second aspect, an embodiment of the present application provides an abnormal device monitoring system, comprising: a log collection device, a plurality of supervised devices associated with the log collection device, and an alarm device connected to the log collection device;

[0018] The log collection device receives the log information sent by the supervised device, parses the log information, obtains the log parsing result, and matches the log parsing result with the alarm rule in the preset rule set to perform abnormal monitoring on the supervised device;

[0019] In response to determining a first alarm rule that matches the log parsing result and the first alarm rule being associated with an exception handling instruction, the log collection device sends the exception handling instruction to the supervised device, the exception handling instruction being used to automatically repair a hidden danger or fault;

[0020] In response to determining a second alarm rule that matches the log parsing result and is not associated with an exception handling instruction, the log collection device sends an alarm prompt to the alarm device;

[0021] Among them, corresponding to multiple log information of the same exception, the log collection device outputs an alarm prompt once within a time period.

[0022] In a third aspect, an embodiment of the present application provides an abnormal device monitoring device, which is applied to a log collection device, wherein the log collection device is associated with a plurality of supervised devices, and the device includes:

[0023] A parsing and obtaining module, configured to parse the log information and obtain a log parsing result in response to receiving the log information sent by the supervised device;

[0024] A matching monitoring module, used for matching the log parsing result with the alarm rule in the preset rule set to perform abnormal monitoring on the supervised device;

[0025] a sending module, configured to, in response to determining a first alarm rule matching the log parsing result and the first alarm rule being associated with an exception handling instruction, send the exception handling instruction to the supervised device, wherein the exception handling instruction is used to automatically repair a hidden danger or fault;

[0026] The first output module is used to output an alarm prompt in response to determining a second alarm rule that matches the log parsing result and is not associated with an exception handling instruction, and output an alarm prompt once within a time period for multiple log information corresponding to the same exception.

[0027] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the abnormal device monitoring method described in the first aspect.

[0028] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the abnormal device monitoring method described in the first aspect is implemented.

[0029] The technical solution of the embodiment of the present application is that a log collection device establishes communication with multiple supervised devices, centrally collects log information provided by multiple supervised devices, performs online analysis on the acquired log information, matches the log parsing results with the alarm rules in a preset rule set, and when an abnormality is identified in the supervised device, promptly outputs alarm information to notify maintenance personnel to handle it, or, for hidden dangers and faults that are eligible for automatic processing, automatically sends exception processing instructions to be executed on the abnormal device to achieve automatic repair of hidden dangers and faults. Through centralized collection and online analysis of system logs, hidden dangers and faults in a complex IT system environment can be monitored in real time, the occurrence of faults can be minimized, and the faults can be quickly located and resolved after they occur, reducing the impact on the normal development of the business and maintaining the stable operation of the IT system. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram showing a method for monitoring abnormal equipment provided by an embodiment of the present application;

[0031] Figure 2 An example of dynamically configuring alarm rules in a management interface provided by an embodiment of the present application is shown;

[0032] Figure 3A schematic diagram of an abnormal device monitoring system provided in an embodiment of the present application is shown;

[0033] Figure 4 A data processing flow chart showing an abnormal device monitoring system provided by an embodiment of the present application;

[0034] Figure 5 A schematic diagram showing an abnormal equipment monitoring device provided in an embodiment of the present application;

[0035] Figure 6 A structural block diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0037] It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present application. Therefore, the references to "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0038] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0039] The embodiments of the present application realize timely discovery and rapid location of system hidden dangers and faults through centralized collection and online analysis of logs of supervised devices in complex IT systems, thereby reducing the occurrence rate of faults, shortening fault handling time, and improving the quality and efficiency of IT operation and maintenance work.

[0040] The following is an introduction to the abnormal device monitoring method applied to the log collection device provided in the embodiment of the present application, wherein the log collection device, as a device for centralized collection of logs, establishes communication with multiple supervised devices to achieve association with multiple supervised devices; the supervised devices include supervision servers and network element devices (such as switches and routers). When the number of supervised devices is large, due to the large amount of logs to be collected, multiple log collection devices can be deployed to respectively handle the logs of different supervised devices, to ensure that all logs can be processed in a timely and effective manner, and to avoid the omission of hidden dangers and faults.

[0041] like Figure 1 As shown, the abnormal device monitoring method applied to the log collection device provided in the embodiment of the present application includes the following steps:

[0042] Step 101: In response to receiving log information sent by a supervised device, the log information is parsed to obtain a log parsing result.

[0043] The logs corresponding to the supervised devices are stored in local files by default. In the embodiment of the present application, the supervised devices will record various event information in real time according to the Syslog protocol during operation to form system logs. Syslog is a standard log protocol. The log information recorded by the supervised devices includes login, permission change, network anomalies, resource overload, database and application errors, etc. The system log is stored in a specific directory by default, and the log can also be sent to a specified device for processing through settings.

[0044] As an optional embodiment, logs are sent to a designated device by modifying configuration parameters for the supervised device. The configuration content includes the IP (Internet Protocol) of the designated device, the listening port and protocol, the log level, etc. For example, UDP (User Datagram Protocol) is the default protocol and 514 is the default port. In a complex network environment, in order to simplify the configuration and improve compatibility, the default values ​​can be used directly.

[0045] The configuration methods are different for servers of different device types, operating systems, and versions. For example, for a Linux server, if you want to send all logs on the server to port 514 of a remote server (equivalent to a log collection device), you can add the following configuration to the file / etc / rsyslog.conf:

[0046] *.*@192.168.1.7:514 / / The IP is replaced with the actual IP in production deployment

[0047] After setting the configuration parameters, restart the log service (systemctl restart rsyslog) to complete the remote log sending configuration. During debugging, you can use the logger command to send log messages (#logger'test message') for verification.

[0048] For a switch (e.g., a certain type of switch), the configuration method to send log information to a remote server is as follows:

[0049] system-view / / Enter the configuration interface

[0050] info-center source aaa channel 7log level warning / / Set the local port for sending logs and the level of sending logs

[0051] info-center loghost 192.168.1.7 / / Specify remote IP

[0052] It should be noted that, in the case of a large number of supervised devices, a management program can be written to batch modify the log service parameters on each supervised device to achieve remote batch configuration.

[0053] For log collection devices, a streaming working mechanism is used to receive log information sent by each supervised device. In order to avoid port blocking, which causes logs containing faults and hidden dangers to be discarded and abnormal alarm information to be missed, a separate thread is designed for log collection devices. The monitoring thread maintains a working state in real time and efficiently processes data packets sent by peripheral devices; after obtaining the data packets, a new thread is created to analyze the log content. The analysis thread can automatically create multiple parallel executions as needed. The log content analysis and alarm processing threads allow a certain delay and can queue up when resources are insufficient. For the created threads, they can destroy themselves after processing their respective log analysis tasks to release resources.

[0054] The data packet received by the log collection device contains the encapsulation of log information, which is unstructured text data. Taking out the package body and parsing it according to the Syslog protocol can decompose the various parts of the log, such as parsing the device identification, log level, occurrence time, process name, log content, etc. As an example, the program module (Facility) that generates the log and the severity (Severity) of the log are encapsulated in a number contained in a pair of angle brackets, and the other parts of the log information can be extracted one by one through space separators. The Syslog log format is not completely unified. For example, some log time parts include the year, while others do not. In this regard, the parsing program fully considers various possible situations and has sufficient fault tolerance to ensure that every part of the log can be correctly parsed. For example, the log collection device parses the received data packet, such as the received data packet includes the following content: <86> Feb 27 09:51:53h4sshd

[34656] :Accepted password formrftp from 135.32.xxx.xxx port 5923ssh2. Through parsing, the binary characters corresponding to the numbers 8 and 6 can be identified, and then the program module that generates the log and the severity of the log (log level) can be determined based on the binary characters; through parsing, the event represented by Feb 27 09:51:53 can be identified; through parsing, the device name represented by h4 can be determined; through parsing, the process name and process ID represented by sshd

[34656] can be determined.

[0055] Step 102: Match the log analysis result with the alarm rules in the preset rule set to perform abnormal monitoring on the supervised device.

[0056] After obtaining the log analysis results through log analysis, the log analysis results are matched with the alarm rules in the preset rule set. The alarm rules are mainly derived from expert experience, and are continuously analyzed, summarized and optimized in combination with subsequent operation and maintenance cases. The management interface supports dynamic configuration of alarm rules, such as Figure 2 The following is an example of dynamically configuring alarm rules in the management interface.

[0057] By matching the log analysis results with the alarm rules in the preset rule set one by one, the logs reported by the supervised device can be analyzed based on the alarm rules to monitor the supervised device for abnormalities through log analysis. After determining the alarm rules that match the log analysis results, step 103 or step 104 is executed.

[0058] Step 103: In response to determining that a first alarm rule matches the log analysis result and the first alarm rule is associated with an exception handling instruction, the exception handling instruction is sent to the supervised device, and the exception handling instruction is used to automatically repair the hidden danger or fault.

[0059] When the log parsing result is matched with the alarm rules in the preset rule set, and the first alarm rule matching the log parsing result is determined, and the first alarm rule is associated with an exception handling instruction, the exception handling instruction associated with the first alarm rule is sent to the supervised device, and the supervised device automatically repairs the hidden danger or fault based on the exception handling instruction, so as to automatically repair the hidden danger or fault by calling the exception handling instruction.

[0060] For log collection devices, they monitor the log stream in real time, and when hidden dangers or faults are found, they further match the exception handling instructions under the corresponding alarm rules. If they match the pre-configured exception handling instructions, they call the exception handling instructions and synchronize them to the supervised devices to execute the exception handling instructions on the abnormal devices, so as to automatically and promptly eliminate hidden dangers and faults, maintain stable system operation, and avoid affecting the normal development of the supported businesses.

[0061] As an example, the log collection device detects that the key process of the supervised device has exited abnormally based on the matching situation, and can send a restart process command (exception handling instruction) to enable the supervised device to restart the process and achieve automatic repair of the fault. The command interaction between the log collection device and the supervised device is implemented based on the SSH (Secure Shell) remote control channel.

[0062] Step 104: In response to determining that the second alarm rule that matches the log parsing result and is not associated with the exception handling instruction is matched, an alarm prompt is output, and an alarm prompt is output once within a time period for multiple log information corresponding to the same exception.

[0063] When the log parsing result is matched with the alarm rules in the preset rule set, and a second alarm rule that matches the log parsing result and is not associated with an exception handling instruction is determined, it is determined that there is an abnormality in the supervised device that reports the log information. At this time, the log collection device can output an alarm prompt to prompt the device abnormality, so that when an abnormal situation is found, an alarm message can be sent immediately to notify maintenance personnel to handle it.

[0064] The log collection device can output alarm prompts in the form of SMS and / or sound and light alarms. For example, when the log analysis result is detected to match an item in the alarm rule and the item is not configured with an exception handling instruction, an SMS and / or sound and light alarm is immediately issued. The SMS is sent to the mobile phone number reserved in the alarm handling interface, so that the relevant operation and maintenance personnel can receive SMS notifications; the sound and light alarm pushes instant messages to the client through Websocket, triggers an alarm sound on the Web interface, and marks the pending alarm with a striking color until the on-duty personnel handle the alarm and respond to the receipt.

[0065] For some abnormal situations, once they occur, they will continue to recur, forming a log storm. In order to avoid interference to operation and maintenance caused by a large number of repeated abnormal alarms due to a single reason, only one alarm prompt is output for multiple log information corresponding to the same abnormality within a time period. This can effectively eliminate a large number of repeated alarms in a short period of time caused by the same fault source and avoid interference of log storms on operation and maintenance work.

[0066] In the above implementation scheme of the present application, the log collection device establishes communication with multiple supervised devices, centrally collects log information provided by multiple supervised devices, performs online analysis on the obtained log information, matches the log parsing results with the alarm rules in the preset rule set, and when an abnormality is identified in the supervised device, promptly outputs alarm information to notify maintenance personnel to handle it, or, for hidden dangers and faults that are eligible for automatic processing, automatically sends exception processing instructions to execute on the abnormal device to achieve automatic repair of hidden dangers and faults. Through centralized collection and online analysis of system logs, hidden dangers and faults in a complex IT system environment can be monitored in real time, the occurrence of faults can be minimized, and the faults can be quickly located and resolved after they occur, reducing the impact on the normal development of the business and maintaining the stable operation of the IT system.

[0067] It should be noted that time drift is also a common hidden danger that causes system failures and business anomalies. Many programs and services rely on the correct time to run, especially some large and complex systems, which require each associated server to have correct and consistent time to ensure reliable data synchronization and the accuracy of business services, such as big data clusters and billing systems. In daily operation and maintenance, due to the lack of time accuracy monitoring means, it is only possible to conduct manual inspections and spot checks, and it is difficult to ensure that the time deviation of each device is discovered in time. Therefore, failures caused by excessive time deviations are usually only discovered after a system failure occurs or after obvious errors occur in the business. At this time, it will be analyzed whether it is caused by time deviation, and then the time of each machine will be checked one by one. Based on this, the embodiments of the present application provide a reliable and convenient solution for the timely discovery of time drift hazards.

[0068] Since the log parsing result includes the log occurrence time, the method further includes:

[0069] Compare the log generation time with the log information reception time to determine the time difference;

[0070] In response to the time difference being greater than a set threshold, outputting a time drift fault alarm or sending a time synchronization instruction to the supervised device;

[0071] Among them, the time drift fault alarm is used to indicate that there is a system time deviation between the supervised device that sends log information and the log collection device, and the time synchronization instruction is used to instruct to reset the system time.

[0072] The system log includes the occurrence time of the log event. Under the premise of ensuring the accuracy of the time of the log collection device itself, the time difference between the supervised device and the log collection device can be obtained by comparing the parsed log occurrence time with the reception time of the log information. When the calculated time difference exceeds the set threshold, it can be considered that there is a time drift risk, and a time drift fault alarm is immediately issued for handling by the operation and maintenance personnel. Alternatively, the log collection device automatically calls the time synchronization command and sends the command to the supervised device, so that the supervised device resets the system time to eliminate the time drift risk.

[0073] By monitoring the time deviation between the supervised device and the log collection device, and automatically alarming or correcting the system time of the supervised device when the time deviation exceeds the threshold, the occurrence of time drift-related failures caused by clock accuracy, time synchronization service abnormalities, etc. can be effectively avoided.

[0074] In an optional embodiment, the log information reported by the supervised device is a log with analytical value, and all logs corresponding to the supervised device can be divided into levels. When reporting logs, only logs of corresponding levels are reported to avoid resource consumption caused by reporting all logs. Among them, when receiving the log information reported by the supervised device, the log collection device includes:

[0075] Receive log information corresponding to a target level sent by a supervised device, where the target level is a log level set based on an application running on the supervised device and meets log sending requirements.

[0076] The supervised device can reasonably set the log level to be sent according to the application running on the device, for example, the log level has 0-7 levels. The supervised device determines the target level that meets the log sending requirements among multiple log levels, and sends the log information corresponding to the target level to ensure that all logs with analytical value can be sent to avoid the omission of hidden dangers and fault events. At the same time, the sending of log information with little analytical value is reduced to reduce the amount of logs sent and reduce resource consumption.

[0077] The following is an introduction to the process of matching the log parsing results with the alarm rules in the preset rule set. The log parsing results include the log content, log level, and device identifier corresponding to the log sending device. The preset rule set includes a first rule subset corresponding to public attributes and a second rule subset corresponding to private attributes. The first rule subset includes public alarm rules and is applicable to all devices. It is the main matching rule for abnormal monitoring matching. It is loaded into the memory when the monitoring program is started to improve the matching speed; the second rule subset includes private alarm rules set for specific devices, that is, private alarm rules are targeted at specific devices. Some devices run specific applications, and some unique log information related to the application will be generated during the operation. In order to understand the system operation status from this information and discover possible hidden dangers and faults, private alarm rules for specific devices are designed.

[0078] When matching the log parsing results with the alarm rules in the preset rule set, the matching method supports single keyword matching, multi-keyword full matching, fuzzy matching, etc. When identifying abnormal devices based on matching, the following steps are included:

[0079] Identify, according to the device identifier, whether the log sending device belongs to a target device associated with the second rule subset;

[0080] In response to the log sending device belonging to the target device, searching the second rule subset for a private alarm rule associated with the log sending device, matching the log parsing result with the public alarm rule in the first rule subset and the found private alarm rule to identify whether the log sending device is abnormal;

[0081] In response to the log sending device not belonging to the target device, the log parsing result is matched with the common alarm rule in the first rule subset to identify whether there is an abnormality in the log sending device.

[0082] Before matching the log parsing results with the alarm rules, it is necessary to identify whether the log sending device (the supervised device that sends the log information) belongs to the target device of the associated private alarm rule. If it belongs to the target device, then for the log parsing results, in addition to matching the public alarm rules, the corresponding private alarm rules will also be taken out for matching; if it does not belong to the target device, then for the log parsing results, only the public alarm rules need to be matched.

[0083] For the second rule subset, it can be associated with one or more specific devices (target devices), and the second rule subset corresponds to one or more device identifiers (such as device names). When identifying whether the log sending device belongs to the target device of the associated private alarm rule, the device identifier of the log sending device is compared with the device identifier corresponding to the second rule subset to determine whether the log sending device is the target device.

[0084] Since the second rule subset can be associated with one or more target devices, after determining that the log sending device is the target device, the private alarm rule associated with the log sending device is searched in the second rule subset, and then the log parsing result is matched one by one with the public alarm rules in the first rule subset and with the found private alarm rules one by one, so as to identify whether there is an abnormality in the log sending device based on the match with the public alarm rules and the private alarm rules.

[0085] When it is identified that the log sending device does not belong to the target device, the log parsing result is directly matched with the common alarm rules in the first rule subset one by one to determine whether the log sending device is an abnormal device according to the common alarm rules.

[0086] In the above implementation scheme, by identifying whether the log sending device belongs to the target device based on the device identification, the alarm rules adapted by the log sending device can be determined, so as to match the appropriate alarm rules with the log parsing results to ensure accurate abnormal identification of the log sending device.

[0087] It should be noted that the alarm rules can be continuously optimized and updated. By analyzing and self-learning previous fault cases, verifying the identification results, and continuously enriching and optimizing the alarm rules, the identification efficiency can be improved and misjudgments and missed judgments can be reduced.

[0088] In an optional embodiment of the present application, in order to avoid a large number of repeated abnormal alarms caused by a single reason from interfering with the operation and maintenance personnel, an alarm source tracking mechanism is introduced, and an alarm buffer is designed in the log collection device to cache the logs that have issued alarms in the current time period. When the log collection device receives a log message indicating that the supervised device has an abnormality and needs to output a first alarm prompt, it detects whether the alarm buffer has a log message indicating the output of the first alarm prompt. If the alarm buffer has a log message indicating the output of the first alarm prompt, and the first alarm prompt has been output in the current time period, then for the received log message, stop outputting the first alarm prompt again, so as to avoid repeatedly outputting the corresponding alarm prompt for the log message indicating the output of the first alarm prompt that is repeatedly received within a time period, thereby avoiding interference with the operation and maintenance personnel.

[0089] That is, for the log collection device, when it determines that there is an abnormality in the log sending device (supervised device) based on the received log information, it will check whether an alarm has been issued for the same type of fault of the device within the current time period (such as the same day). If an alarm has already been issued, it does not need to be sent again to avoid repeated output of alarms due to the appearance of multiple duplicate log information, which affects the normal work of the operation and maintenance personnel.

[0090] Among them, when log information sent by N supervised devices is received respectively within a preset time period and the alarm prompts corresponding to the N supervised devices are identified as chain alarm prompts based on the N log information, the first supervised device corresponding to the first abnormal event is determined among the N supervised devices; prompt information for repairing the abnormality of the first supervised device is output; N is an integer greater than or equal to 2, and an abnormality occurs in the second supervised device among the N supervised devices after calling the service on the first supervised device.

[0091] For the problem that a fault point may cause multiple related devices to alarm in a short period of time, the log collection device identifies the source of the first event in the chain alarm event based on the time identification mechanism, and handles the first fault first, and then the alarms of other devices will be eliminated. That is, if the log collection device receives log information sent by N supervised devices within a preset time, analyzes the log information sent by the N supervised devices, and identifies the alarm prompt corresponding to the log information sent by the N supervised devices as a chain alarm prompt, the first supervised device corresponding to the first abnormal event is determined in the N supervised devices in chronological order. Here, the first abnormal event is the abnormal event corresponding to the first alarm prompt in the chain alarm prompt. The N supervised devices include the first supervised device and N-1 second supervised devices. The second supervised device is a device that has an abnormality after calling the service on the first supervised device, that is, the abnormality of the second supervised device is caused by calling the service on the first supervised device.

[0092] After determining the first supervised device corresponding to the first abnormal event, the log collection device outputs prompt information for repairing the abnormality of the first supervised device, so that the operation and maintenance personnel can repair the abnormality of the first supervised device based on the prompt information, handle the first fault, and then ensure that the alarms of other devices are eliminated. After determining the first supervised device corresponding to the first abnormal event and outputting the prompt information, the log collection device can also automatically repair the abnormality of the first supervised device to ensure that the chain alarm problem is solved from the root.

[0093] In an optional embodiment of the present application, after sending an exception handling instruction or outputting an alarm prompt based on the matching of the log analysis results and the alarm rules, it also includes: moving the log analysis results to a database to store structured log information in the database; wherein, when the triggering conditions for data migration are met, the log information in the database is moved in batches to the log data platform, and the log data platform centrally stores offline logs to provide the query device with log information that meets the retrieval requirements.

[0094] After receiving the log information, the log collection device parses the log information to obtain a structured log analysis result. After completing the matching process based on the log analysis result, the log analysis result is moved to a database (such as a Postgresql database) to save the structured log data based on the database. For the structured log data stored in the database, when the data migration conditions are met, the log data in the database is moved in batches to the log data platform to store offline logs through the log data platform. For example, the log data in the database is batch-dumped to the log data platform at regular intervals every day for offline analysis. Since the log data platform centrally stores offline logs, it can be used for information retrieval by query devices, providing query devices with log information that meets retrieval requirements.

[0095] The following is an introduction to the log data platform for storing offline data. Problems in system performance, security events (unexpected access), software design deviations, etc. may also cause failures, resulting in system freezes, blockages, business errors, etc. Such failures generally do not appear as significant anomalies in system logs and are not easy to detect through log stream monitoring. However, through normal log analysis, some information with important reference value for troubleshooting and resource optimization can be found. Such as unexpected access, invalid processes running in the server, high-frequency connections, etc. Therefore, in addition to being used for online anomaly monitoring, it is very necessary to store system logs in a centralized manner to provide reference for subsequent assistance in troubleshooting, resource optimization, and security reinforcement. The centralized collection, storage, and analysis of logs of servers and network element devices across the entire network require support from a large-capacity, high-performance platform. After testing and evaluation, the open source high-performance Clickhouse (CK) was used to build the log data platform.

[0096] CK uses columnar storage, high compression rate, and good IO (Input / Output) performance, which can give full play to the multi-core parallel advantages of the CPU (Central Processing Unit). Under the current mainstream hardware environment, it can achieve a millisecond-level response capability for retrieving tens of billions of records, and has significant advantages in query performance; in terms of cost, since CK has a higher compression rate, compared with other traditional log platforms, it requires less storage space to store the same amount of log content. Therefore, using CK cluster as a log data platform for centralized log storage has significant advantages of high performance and low cost. The number of cluster nodes can be flexibly configured according to the daily log volume and the length of time the logs are kept. In the early stage, it can be configured less, and the cluster nodes can be gradually increased according to the business growth, the number of supervised devices, and the complexity of the system.

[0097] CK cluster (log data platform) is similar to most big data platforms. Its advantages are mainly reflected in the storage and rapid retrieval of massive data, but its performance in high-frequency transaction processing is relatively weak. In order to avoid the performance consumption of the entire system caused by frequent data writing, the logs received by the log collection device are not immediately written to the CK cluster. Instead, a separate Postgresql database is used to save real-time log data. For example, the log data of the day is saved, and the log data is batch-dumped to the CK cluster for offline analysis at regular intervals every day. This can give full play to the advantages of cluster big data batch processing, while also reducing the pressure of online log analysis and processing, and improving the overall performance of the system.

[0098] After the logs are centrally stored, the log information of the entire network equipment can be remotely retrieved through the Web interface. Through offline log analysis, security risks, resource usage, performance and other issues in the entire IT operating environment can be discovered, providing sufficient data basis for fault cause analysis, security reinforcement, and capacity expansion construction of the entire platform.

[0099] The above is the overall implementation process of the abnormal device monitoring method provided in the embodiment of the present application. The system log recording configuration parameters are set on the supervised device, and the log is sent to the log collection device for analysis and processing; the log collection device monitors the log stream content through the log analysis program, screens whether it contains abnormal information, and promptly issues an alarm or calls the self-processing program to perform self-healing repair of hidden dangers and faults when hidden dangers and faults are found; a log data platform for centralized log storage is deployed to store the formatted log information in full, providing data support for subsequent troubleshooting, system optimization, expansion, etc.

[0100] Through the centralized collection and online analysis of system logs, real-time monitoring of hidden dangers and faults in complex IT system environments is achieved. It has high practical value in preventing hidden dangers from turning into faults, discovering faults in real time, and quickly locating faults. Some hidden dangers and faults with standard handling actions can be automatically eliminated, so that the system can automatically return to normal, avoiding impact on normal business operations.

[0101] The embodiments of the present application are suitable for scenarios with complex networking, a large number of supervised devices, a large workload for operation and maintenance, and high requirements for IT and network stability, and can be better applied to large and complex IT system operating environments; can effectively avoid continuous alarms caused by log storms, accurately identify time drift, and can minimize the occurrence of faults, discover and locate the faults as soon as possible after they occur, and quickly recover from the faults, which can improve the accuracy and reliability of abnormal monitoring, improve IT operation and maintenance efficiency, and maintain stable operation of IT systems.

[0102] The present application also provides an abnormal device monitoring system. Figure 3As shown, the system includes a log collection device 31, a plurality of supervised devices 32 associated with the log collection device 31, and an alarm device 33 connected to the log collection device;

[0103] The log collection device 31 receives the log information sent by the supervised device 32, parses the log information, obtains the log parsing result, and matches the log parsing result with the alarm rule in the preset rule set to perform abnormal monitoring on the supervised device 32;

[0104] In response to determining that a first alarm rule matches the log parsing result and the first alarm rule is associated with an exception handling instruction, the log collection device 31 sends the exception handling instruction to the supervised device 32, and the exception handling instruction is used to automatically repair the hidden danger or fault;

[0105] In response to determining a second alarm rule that matches the log parsing result and is not associated with an exception handling instruction, the log collection device 31 sends an alarm prompt to the alarm device 33; wherein, for multiple log information corresponding to the same exception, the log collection device 31 outputs an alarm prompt once within a time period.

[0106] The log collection device 31 establishes communication with multiple supervised devices 32 and receives log information provided by the supervised devices 32. The log collection device 31 centrally collects log information provided by multiple supervised devices 32, detects the log stream content, and promptly issues an alarm after an abnormality is found, or, for abnormal situations configured with automatic processing instructions (exception processing instructions), calls automatic processing instructions to achieve self-repair of hidden dangers and faults. The detailed process of the log collection device 31 analyzing log information and processing based on the matching of log data and alarm rules can be found in the introduction of the method embodiment, which will not be elaborated here.

[0107] The alarm device 33 can be a terminal corresponding to the operation and maintenance personnel, such as a mobile phone. The log collection device 31 sends an alarm prompt to the alarm device 33, which can be: sending a notification SMS to the operation and maintenance personnel's mobile phone, so that the operation and maintenance personnel receive the notification in the form of SMS. The alarm device 33 can also be a client. The log collection device 31 sends an alarm to the client, triggers an alarm sound on the client's Web interface, and marks the pending alarm with a striking color. It will not be eliminated until the on-duty personnel handle the alarm and return the order.

[0108] In order to avoid interference to operation and maintenance caused by a large number of repeated abnormal alarms due to a single reason, for multiple log information corresponding to the same abnormality within a time period, the log collection device 31 only outputs an alarm prompt once within a time period, which can effectively eliminate a large number of repeated alarms in a short period of time caused by the same fault source and avoid interference of log storms on operation and maintenance work.

[0109] The system further includes: a database 34 connected to the log collection device 31, and a log data platform 35 connected to the database 34;

[0110] After sending an exception handling instruction or sending an alarm prompt based on the matching of the log analysis result and the alarm rule, the log collection device 31 moves the log analysis result to the database 34 to store structured log information in the database 34;

[0111] When the triggering conditions for data migration are met, the log data in the database 34 is moved in batches to the log data platform 35. The log data platform 35 centrally stores offline logs and provides query devices with log information that meets retrieval requirements.

[0112] After receiving the log information, the log collection device 31 obtains a structured log analysis result by analyzing the log information. After completing the matching process based on the log analysis result, the log analysis result is moved to the database 34 to save the structured log information based on the database 34. For the structured log information stored in the database 34, when the data migration conditions are met, the log data in the database 34 is moved in batches to the log data platform 35, and the offline logs are centrally stored in the log data platform 35. By storing the full amount of formatted log information in the log data platform 35, data support is provided for subsequent troubleshooting, system optimization, expansion, etc.

[0113] The log data platform 35 in this embodiment is a CK cluster, which is the core of data storage and offline analysis and calculation. The specific configuration and deployment is determined according to the log volume of the entire environment and the length of time required for storage. Under normal circumstances, a 5-node mainstream configuration server, a single node stores 20TB, which can meet the centralized storage and offline analysis support of logs for more than 100 devices and 3 months.

[0114] Among them, the log data platform 35 is connected to the application server 36, the application server 36 is connected to the application client 37, and the application client 37 can be used as a query device to perform log query on the log data platform 35, so that the log data platform 35 provides the query device with log information that meets the retrieval requirements.

[0115] like Figure 4As shown, it is a data processing flow chart of the abnormal device monitoring system provided in an embodiment of the present application. The supervised device reports log information to the log collection device. The log collection device parses the received log information, monitors the abnormal device based on the parsing situation, and outputs an alarm prompt or calls the exception handling instruction to repair the abnormality after identifying the abnormal device. The database stores the structured log information provided by the log collection device. When the data migration conditions are met, the log information in the database is batch moved to the log data platform, and the offline logs are centrally stored on the log data platform. The log data platform provides the application server with log information that meets the retrieval requirements. The retrieval requirements are sent to the application server by the application client. The application server obtains the log information that meets the retrieval requirements from the log data platform based on the retrieval request carrying the retrieval requirements, and provides the log information to the application client.

[0116] The above is an abnormal equipment monitoring system provided in an embodiment of the present application, which is similar to the method embodiment. For related contents not elaborated in detail, please refer to the introduction of the method embodiment.

[0117] The following is a specific example to illustrate the process of monitoring abnormal equipment and handling abnormal situations using the solution provided in the embodiment of the present application.

[0118] In the existing technology, various measures are taken to ensure the safe operation of IT systems, such as adopting end-to-end business monitoring measures, carrying out cloud eye tracking measures at important nodes, installing agent programs to monitor the operating status of equipment, regular inspections, standardizing and strictly implementing work plans, etc. These measures effectively reduce the system failure rate. However, due to equipment reliability, business uncertainty, system and platform software bugs, operational errors and other reasons, failures will still occur. Some failures are difficult to quickly determine the cause of the failure and locate the faulty node due to the large number of related devices and related applications, resulting in a long recovery time, which has a great impact on production and operations.

[0119] In order to further reduce the failure rate, shorten the fault handling time, and minimize the impact of network and system failures on production support, a centralized log collection and online analysis system is deployed. By real-time analysis of each log information generated by each device, existing hidden dangers and faults can be discovered in time, and hidden dangers can be dealt with in advance and faults can be handled in a timely manner.

[0120] In the early stage of the system, a high-performance server was used to run the online log processing program to detect the log content. The database was configured to store the daily logs, and a Clickhouse cluster consisting of 5 servers was built to store historical logs. Daily management, including alarm rule setting, historical log query analysis, abnormal event processing, etc., is completed using a browser to access the system. By introducing the above system, several hidden dangers and minor faults were accurately and promptly discovered and quickly repaired through the system. At the same time, based on system log analysis, some invalid services in the system were disabled to reduce invalid resource consumption. Time correction was performed on devices with large time drift, or time synchronization services were configured.

[0121] For example, in actual production applications, there are multiple "sftp-server write failed" abnormal log messages, and the alarm source is the same device and the same reason. One of them triggered the "write failed" alarm in the alarm rule and was pushed to the on-duty mobile phone in a timely manner. The reason was analyzed that during the debugging of the interface program on a newly added signaling collection device, the signaling data processing service was suspended, resulting in a backlog of signaling data pushed by the network side, and the remaining disk space was insufficient to accommodate the newly sent files. After the space was released and the data processing process was restarted, the fault was quickly restored.

[0122] In actual production applications, there is an error-level log "Failed to start dnf makecache" but there is no matching rule, so no alarm is triggered. Further analysis shows that the failure of dnf indicates that the system automatic update failed, which is an invalid service. By disabling the meaningless automatic update of intranet environment devices (systemctl disablednf-makecache.timer), resource consumption can be saved and the overall system performance can be improved.

[0123] The embodiment of the present application also provides an abnormal device monitoring device, which is applied to a log collection device, and the log collection device is associated with multiple supervised devices, such as Figure 5 As shown, the device comprises:

[0124] The parsing and obtaining module 501 is used to parse the log information sent by the supervised device in response to receiving the log information, and obtain the log parsing result;

[0125] A matching monitoring module 502, used to match the log parsing result with an alarm rule in a preset rule set to perform abnormal monitoring on the supervised device;

[0126] A sending module 503, configured to, in response to determining a first alarm rule matching the log parsing result and the first alarm rule being associated with an exception handling instruction, send the exception handling instruction to the supervised device, wherein the exception handling instruction is used to automatically repair a hidden danger or fault;

[0127] The first output module 504 is used to output an alarm prompt in response to determining a second alarm rule that matches the log parsing result and is not associated with an exception handling instruction, and output an alarm prompt once within a time period for multiple log information corresponding to the same exception.

[0128] Optionally, the log parsing result includes the log occurrence time, and the device further includes:

[0129] A comparison and determination module, used to compare the log generation time with the log information reception time to determine the time difference;

[0130] A processing module, configured to output a time drift fault alarm or send a time synchronization instruction to the supervised device in response to the time difference being greater than a set threshold;

[0131] The time drift fault alarm is used to indicate that there is a system time deviation between the supervised device that sends the log information and the log collection device, and the time synchronization instruction is used to instruct to reset the system time.

[0132] Optionally, the device further comprises:

[0133] A receiving module, used for receiving log information corresponding to a target level sent by the supervised device;

[0134] The target level is a log level that is set based on the application running on the supervised device and meets the log sending requirements.

[0135] Optionally, the log parsing result includes log content, log level and device identification corresponding to the log sending device, and the preset rule set includes a first rule subset corresponding to public attributes and a second rule subset corresponding to private attributes;

[0136] The matching monitoring module includes:

[0137] an identification submodule, configured to identify, according to the device identifier, whether the log sending device belongs to a target device associated with the second rule subset;

[0138] A first processing submodule is used for searching the second rule subset for a private alarm rule associated with the log sending device in response to the log sending device belonging to the target device, and matching the log parsing result with the public alarm rule in the first rule subset and the found private alarm rule to identify whether the log sending device is abnormal;

[0139] The second processing submodule is configured to, in response to the log sending device not belonging to the target device, match the log parsing result with the common alarm rule in the first rule subset to identify whether the log sending device is abnormal.

[0140] Optionally, the device further comprises:

[0141] A detection module, configured to detect whether log information indicating output of the first alarm prompt is cached when receiving log information indicating that the supervised device is abnormal and needs to output a first alarm prompt;

[0142] The stopping module is used for stopping outputting the first alarm prompt again in response to the log information indicating the outputting of the first alarm prompt being cached and the first alarm prompt having been output in the current time period.

[0143] Optionally, the device further comprises:

[0144] A determination module, configured to determine a first supervised device corresponding to the first abnormal event among the N supervised devices when log information respectively sent by N supervised devices is received within a preset time period and the alarm prompts corresponding to the N supervised devices are identified as chain alarm prompts based on the N log information;

[0145] A second output module, used to output prompt information for performing abnormal repair on the first supervised device;

[0146] N is an integer greater than or equal to 2, and an exception occurs in a second supervised device among the N supervised devices after invoking a service on the first supervised device.

[0147] Optionally, after sending the exception handling instruction or outputting the alarm prompt based on the matching of the log parsing result and the alarm rule, the device further includes:

[0148] A transfer module, used for transferring the log parsing result to a database to store structured log information in the database;

[0149] Among them, when the triggering conditions for data migration are met, the log information in the database is batch-migrated to the log data platform, and the log data platform centrally stores offline logs to provide the query device with log information that meets the retrieval requirements.

[0150] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0151] On the other hand, an embodiment of the present application further provides an electronic device, comprising a memory, a processor, a bus, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the above-mentioned abnormal device monitoring method when executing the program.

[0152] For example, Figure 6 A schematic diagram of the physical structure of an electronic device is shown.

[0153] like Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630, and the processor 610 is used to execute each process of the abnormal device monitoring method of the embodiment of the present application, which will not be described one by one here.

[0154] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0155] On the other hand, an embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the abnormal device monitoring method provided in the above embodiments.

[0156] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0157] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An abnormal device monitoring method, applied to a log collection device, characterized in that: The log collection device is associated with a plurality of supervised devices, and the method comprises: In response to receiving the log information sent by the supervised device, parsing the log information to obtain a log parsing result; Matching the log analysis result with the alarm rules in the preset rule set to perform abnormal monitoring on the supervised device; In response to determining a first alarm rule that matches the log parsing result and the first alarm rule being associated with an exception handling instruction, sending the exception handling instruction to the supervised device, the exception handling instruction being used to automatically repair a hidden danger or fault; In response to determining a second alarm rule that matches the log parsing result and is not associated with an exception handling instruction, an alarm prompt is output, and an alarm prompt is output once within a time period for multiple log information corresponding to the same exception.

2. The method according to claim 1, characterized in that: The log parsing result includes the log occurrence time, and the method further includes: Compare the log generation time with the log information reception time to determine the time difference; In response to the time difference being greater than a set threshold, outputting a time drift fault alarm or sending a time synchronization instruction to the supervised device; The time drift fault alarm is used to indicate that there is a system time deviation between the supervised device that sends the log information and the log collection device, and the time synchronization instruction is used to instruct to reset the system time.

3. The method according to claim 1, characterized in that Also includes: Receiving log information corresponding to a target level sent by the supervised device; The target level is a log level that is set based on the application running on the supervised device and meets the log sending requirements.

4. The method according to any one of claims 1 to 3, characterized in that: The log parsing result includes log content, log level and device identification corresponding to the log sending device, and the preset rule set includes a first rule subset corresponding to public attributes and a second rule subset corresponding to private attributes; The matching of the log parsing result with the alarm rule in the preset rule set to perform abnormal monitoring on the supervised device includes: identifying, according to the device identifier, whether the log sending device belongs to a target device associated with the second rule subset; In response to the log sending device belonging to the target device, searching the second rule subset for a private alarm rule associated with the log sending device, matching the log parsing result with the public alarm rule in the first rule subset and the found private alarm rule to identify whether the log sending device has an abnormality; In response to the log sending device not belonging to the target device, the log parsing result is matched with the common alarm rule in the first rule subset to identify whether the log sending device has an abnormality.

5. The method according to claim 1, characterized in that in, In the case of receiving log information indicating that the supervised device has an abnormality and needs to output a first alarm prompt, detecting whether log information indicating outputting the first alarm prompt is cached; In response to the log information indicating the output of the first alarm prompt being cached and the first alarm prompt having been output in the current time period, the output of the first alarm prompt is stopped again.

6. The method according to claim 1 or 5, characterized in that: Also includes: In the case where log information respectively sent by N supervised devices is received within a preset time period and the alarm prompts corresponding to the N supervised devices are identified as chain alarm prompts based on the N log information, a first supervised device corresponding to the first abnormal event is determined among the N supervised devices; Outputting prompt information for performing abnormal repair on the first supervised device; N is an integer greater than or equal to 2, and an exception occurs in a second supervised device among the N supervised devices after invoking a service on the first supervised device.

7. The method according to claim 1, characterized in that After sending the exception handling instruction or outputting the alarm prompt based on the matching of the log parsing result and the alarm rule, the method further includes: Moving the log parsing results to a database to store structured log information in the database; Among them, when the triggering conditions for data migration are met, the log information in the database is batch-migrated to the log data platform, and the log data platform centrally stores offline logs to provide the query device with log information that meets the retrieval requirements.

8. An abnormal equipment monitoring system, characterized in that: include: A log collection device, a plurality of supervised devices associated with the log collection device, and an alarm device connected to the log collection device; The log collection device receives the log information sent by the supervised device, parses the log information, obtains the log parsing result, and matches the log parsing result with the alarm rule in the preset rule set to perform abnormal monitoring on the supervised device; In response to determining a first alarm rule that matches the log parsing result and the first alarm rule being associated with an exception handling instruction, the log collection device sends the exception handling instruction to the supervised device, the exception handling instruction being used to automatically repair a hidden danger or fault; In response to determining a second alarm rule that matches the log parsing result and is not associated with an exception handling instruction, the log collection device sends an alarm prompt to the alarm device; Among them, corresponding to multiple log information of the same exception, the log collection device outputs an alarm prompt once within a time period.

9. The system according to claim 8, characterized in that Also includes: A database connected to the log collection device, and a log data platform connected to the database; After sending the exception handling instruction or sending the alarm prompt based on the matching of the log analysis result and the alarm rule, the log collection device moves the log analysis result to the database to store structured log information in the database; When the triggering conditions for data migration are met, the log information in the database is batch-migrated to the log data platform, and the log data platform centrally stores offline logs to provide query devices with log information that meets retrieval requirements.

10. An abnormal device monitoring device, applied to a log collection device, characterized in that: The log collection device is associated with a plurality of supervised devices, and the apparatus comprises: A parsing and obtaining module, configured to parse the log information and obtain a log parsing result in response to receiving the log information sent by the supervised device; A matching monitoring module, used for matching the log parsing result with the alarm rule in the preset rule set to perform abnormal monitoring on the supervised device; a sending module, configured to, in response to determining a first alarm rule matching the log parsing result and the first alarm rule being associated with an exception handling instruction, send the exception handling instruction to the supervised device, wherein the exception handling instruction is used to automatically repair a hidden danger or fault; The first output module is used to output an alarm prompt in response to determining a second alarm rule that matches the log parsing result and is not associated with an exception handling instruction, and output an alarm prompt once within a time period for multiple log information corresponding to the same exception.

11. An electronic device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the abnormal equipment monitoring method as claimed in any one of claims 1 to 7.

Images