A log processing method, apparatus, system, and storage medium

Abstract

This invention discloses a log processing method, apparatus, system, and storage medium. The method, applied to a log processing system, may include: acquiring log information of software to be maintained; standardizing the format of the log information; updating the log information based on the format standardization result; wherein each log information originates from at least one of the software to be maintained, and / or a terminal, server, and network associated with the software; if abnormal information is obtained after processing the log information, the abnormal attributes of the abnormal information are determined, and an abnormal warning is issued based on the abnormal attributes; if so, a warning message is generated based on the abnormal information and / or the log information, and an abnormal warning is issued based on the warning message. The technical solution of this invention can proactively acquire log information of the software to be maintained and issue abnormal warnings for log information with abnormalities, thereby achieving the effect of quickly locating the fault in the software to be maintained.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a log processing method, apparatus, system and storage medium. Background Technology

[0002] With the rapid development of computer technology, network technology, and the IT industry, application software is playing an increasingly important role in people's daily lives, studies, and work. Due to the convenience brought by application software, numerous vendors across various fields offer a wide variety of applications. These applications may differ in their programming languages, technical architectures, and implemented business functions. Furthermore, the number of related applications within a single field is increasing, and there is a growing trend of interconnectivity between applications within the same field and across different fields. This means that the normal operation of a single application software depends not only on its own functions and components but also on one or more external application software programs.

[0003] Most application software vendors typically log information on the servers or terminals where their applications run to facilitate maintenance, troubleshooting, and fault location. These logs generally record runtime environment information, exception information, and event information related to the application software. During application software maintenance, the typical process involves an operations engineer initially locating the fault based on the symptoms after a user reports a problem. If the location is inaccurate or verification fails, the application's own logs are analyzed to find the cause, and the fault is then located based on the findings.

[0004] As application software becomes more feature-rich, daily operations rely on it more heavily, leading to increased complexity and tighter parallel and serial connections between applications. In this environment, existing maintenance methods struggle to quickly pinpoint faults, significantly disrupting normal business operations. Summary of the Invention

[0005] This invention provides a log processing method, apparatus, system, and storage medium, which solves the problem of difficulty in quickly locating faults during the operation and maintenance of application software.

[0006] In a first aspect, embodiments of the present invention provide a log processing method, applied to a log processing system, which may include:

[0007] Obtain log information from the software to be maintained, standardize the format of the log information, and update the log information according to the format standardization result. Each log information comes from at least one of the software to be maintained and / or the terminal, server and network associated with the software to be maintained. If abnormal information is obtained after processing the log information, determine the abnormal attributes of the abnormal information and determine whether to issue an abnormal warning based on the abnormal attributes. If so, generate warning information based on the abnormal information and / or log information, and issue an abnormal warning based on the warning information.

[0008] Optionally, obtaining log information from the software to be maintained may include:

[0009] Log information of the software to be maintained is obtained from log storage files located on terminals, servers, and / or networks; and / or, log information is obtained based on log collection interfaces set in the log processing system, wherein the log information collected by the log collection interface includes information pushed by the software to be maintained to the log collection interface based on application programming interfaces embedded in the software to be maintained; and / or log information is obtained when the log processing system performs test tasks.

[0010] Optionally, if abnormal information is obtained after processing the log information, the abnormal attributes of the abnormal information are determined, which may include:

[0011] The log information is processed to obtain the log attributes, which include the log source, log level, and / or log content. The log attributes are compared with preset attributes. If it is determined that abnormal information should be generated based on the comparison results, the abnormal attributes of the abnormal information are determined. The preset attributes include the attributes of the log information to be given abnormal warnings.

[0012] Optionally, the anomaly attribute includes priority. Determining whether to issue an anomaly warning based on the anomaly attribute includes: determining whether to issue an anomaly warning based on the numerical relationship between the priority and a pre-set anomaly warning threshold.

[0013] Correspondingly, abnormal warnings based on early warning information can include:

[0014] Based on the warning method corresponding to the priority, warning information is sent to provide anomaly warning.

[0015] Optionally, the above log processing method may also include:

[0016] If no abnormal information is received or no abnormal warning is issued based on the abnormal attributes, the log information is cached, and the cached log information is stored in the log storage module set on the log processing system at a preset time.

[0017] Based on this, the above log processing method may also include:

[0018] When a log retrieval trigger event is detected, the log to be retrieved corresponding to the retrieval trigger event is selected from the log information stored in the log storage module; the log to be retrieved is then displayed.

[0019] Optionally, the above log processing method may also include:

[0020] When a processing trigger event for early warning processing is detected, the processing operation information corresponding to the processing trigger event is obtained and stored in the early warning processing module; when an analysis trigger event for early warning analysis is detected, the information to be analyzed corresponding to the analysis trigger event is selected from the processing operation information stored in the early warning processing module; the information to be analyzed is analyzed and the analysis results are displayed.

[0021] Secondly, embodiments of the present invention also provide a log processing apparatus configured in a log processing system, the apparatus comprising:

[0022] The log information acquisition module is used to acquire log information of the software to be maintained, standardize the format of the log information, and update the log information according to the format standardization result. Each log information comes from at least one of the software to be maintained, and / or the terminal, server and network associated with the software to be maintained.

[0023] The anomaly warning and judgment module is used to determine the anomaly attribute of the anomaly information if anomaly information is obtained after processing the log information, and to determine whether to issue an anomaly warning based on the anomaly attribute.

[0024] The anomaly warning execution module is used to generate warning information based on the anomaly information and / or log information if the anomaly is detected, and to issue an anomaly warning based on the warning information.

[0025] Thirdly, embodiments of the present invention also provide a log processing system, which may include:

[0026] One or more processors;

[0027] Memory, used to store one or more programs;

[0028] When one or more programs are executed by one or more processors, the one or more processors implement the log processing method provided in any embodiment of the present invention.

[0029] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the log processing method provided in any embodiment of the present invention.

[0030] The technical solution of this invention standardizes the format of log information from the software under maintenance, which may originate from at least one channel, to obtain uniformly formatted log information. If abnormal information is obtained after processing the log information, it indicates that the software under maintenance may have a fault. In this case, the abnormal attribute of the abnormal information can be determined to accurately judge whether an abnormal warning is needed for the software under maintenance. If so, a warning message can be generated based on the abnormal information and / or log information, and an abnormal warning can be issued based on the warning message. This helps maintenance engineers quickly locate the fault in the software under maintenance. The above technical solution can proactively acquire the log information of the software under maintenance and issue abnormal warnings for abnormal log information, thereby achieving the effect of quickly locating the fault in the software under maintenance. Attached Figure Description

[0031] Figure 1 This is a flowchart of a log processing method according to Embodiment 1 of the present invention;

[0032] Figure 2 This is a flowchart of a log processing method according to Embodiment 2 of the present invention;

[0033] Figure 3 This is a flowchart of a log processing method according to Embodiment 3 of the present invention;

[0034] Figure 4a This is an architecture diagram of an optional example of a log processing method in Embodiment 3 of the present invention;

[0035] Figure 4b This is a flowchart of an optional example of a log processing method in Embodiment 3 of the present invention;

[0036] Figure 5 This is a structural block diagram of a log processing device according to Embodiment 4 of the present invention;

[0037] Figure 6 This is a schematic diagram of the structure of a log processing system according to Embodiment 5 of the present invention. Detailed Implementation

[0038] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0039] Before introducing the embodiments of the present invention, an exemplary application scenario is provided: An organization uses an increasing number of application software programs, daily business operations rely more heavily on these programs, and the connections between these programs become increasingly close. Therefore, a failure in one application software program has a growing impact on daily business operations. It should be noted that as the complexity of application software increases, the log information associated with that application software plays an increasingly important role in the operation and maintenance of the application software.

[0040] However, the log information of each application software and its operating environment is independent and unsystematic. The existing application software operation and maintenance process can only be known by operation and maintenance engineers after users actively report the fault. It is impossible to achieve proactive early warning and proactive troubleshooting, which inevitably leads to the obstruction and delay of daily business. Moreover, when operation and maintenance engineers analyze the log information of the application software, they may be unable to locate the fault due to the incompleteness of the log information recorded by the application software itself.

[0041] To quickly pinpoint application software faults, embodiments of this invention propose a log processing method for use in a log processing system. This system collects log information from various application software and their operating environment, analyzes the log information, and issues timely anomaly warnings when anomalies are detected based on the analysis results. This allows maintenance engineers to detect faults before users report them, and eliminates the need for preliminary fault location based on symptoms. Instead, the fault can be directly located based on the log information associated with the anomaly warning, significantly improving fault location efficiency.

[0042] It should be noted that in the description of the following embodiments, in order to better fit the application scenarios that the embodiments of the present invention may involve, the interaction process between the log processing system and the operations and maintenance engineer is described using an operations and maintenance engineer as an example. Of course, the operations and maintenance engineer is only an example. In addition, there may be interface operators, information analysts, information retrieval personnel, etc., which will not be listed here.

[0043] Example 1

[0044] Figure 1 This is a flowchart of a log processing method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where abnormal situations during the operation of software under maintenance are alerted based on log information. This method can be executed by the log processing device provided in this embodiment of the invention, which can be implemented in software and / or hardware and can be integrated into a log processing system.

[0045] See Figure 1The method of this invention specifically includes the following steps:

[0046] S110. Obtain the log information of the software to be maintained, standardize the format of the log information, and update the log information according to the format standardization result. Each log information comes from at least one of the software to be maintained, and / or the terminal, server and network associated with the software to be maintained.

[0047] The software to be maintained can be an application that may be involved in the maintenance process, such as an application that is currently running within an organization and is designed to meet specific needs. When such an application malfunctions, maintenance engineers can locate and resolve the problem. There can be one, two, or more such applications, which may originate from the same or different application software vendors and interact with each other through a mesh-like interface. Therefore, both the internal structure of the software and the interfaces between the applications can experience failures, all of which require rapid identification.

[0048] Since the log processing system can collect log information from the software under maintenance in real time or periodically, each log entry can be standardized upon acquisition. This means processing the logs according to a pre-defined standard format to obtain uniformly formatted log information. This standard format can include log source, log level, and log content. This is because log information may originate from the software itself, or from at least one of the terminals, servers, or networks associated with it. Therefore, the formats of different log entries may vary. To ensure the effectiveness of subsequent log processing, standardization is performed first to ensure that each standardized log entry has a uniform format.

[0049] It should be noted that the aforementioned terminal can be a terminal on which the software to be maintained is deployed, specifically the hardware or software on the terminal; the server can be a server on which the software to be maintained is deployed, specifically the hardware (such as memory, CPU) or software (such as middleware, database) on the terminal; the network can be a network used for information transmission between the server and the terminal, or it can be the operating environment of the software to be maintained.

[0050] S120. If abnormal information is obtained after processing the log information, the abnormal attributes of the abnormal information are determined, and an abnormal warning is issued based on the abnormal attributes.

[0051] After obtaining log information, it can be processed. The processing results can determine whether the software associated with the log information is faulty. For example, key log information can be extracted and used to determine if the associated software is faulty. If so, associated anomaly information can be obtained. This anomaly information can be a portion of the log information or information generated based on the processing results, etc. In practical applications, some anomaly information requires alerts while others do not. Therefore, when obtaining anomaly information after processing the log information, the anomaly attributes can be determined first. These attributes can help determine whether an anomaly requires an alert. These anomaly attributes reflect the characteristics of the anomaly information, such as the number, priority, severity, and frequency of occurrence.

[0052] Based on this, optionally, one possible implementation of the above steps is: processing the log information to obtain log attributes. In practical applications, each log attribute can correspond to a different part in the standard format. Optionally, the log attribute can include log source, log level, and / or log content. The log source can reflect the source channel of the log information, such as the software to be maintained, server, terminal, network, etc. The log level can be message, debug, warning, error, etc., and the log content can be the specific content of the log information. Further, the log attributes are compared with preset attributes. Since the preset attributes can be pre-set attributes of log information to be anomaly alerted, the comparison result can determine whether there is an anomaly in the log information. For example, if a part of the log attribute matches the anomaly attribute, the corresponding log information is likely to be anomaly. Therefore, if the comparison result determines that the log information is anomaly, anomaly information can be generated. Based on this, to improve the accuracy of anomaly alerts, the anomaly attributes of the anomaly information can be further determined, and anomaly alerts can be issued based on the anomaly attributes.

[0053] S130. If so, generate warning information based on the abnormal information and / or log information, and issue an abnormal warning based on the warning information.

[0054] When it is determined that an anomaly warning is needed, warning information can be generated based on anomaly information and / or log information. For example, the anomaly information and / or log information can be directly used as the warning information, or the processing results of the anomaly information and / or log information can be used as the warning information, etc., without specific limitations. Furthermore, anomaly warnings can be issued based on this warning information, such as displaying and / or sending the warning information in a preset warning format, such as sending it to the recipient (e.g., an operations engineer) via email, SMS, WeChat message, etc., or displaying the warning information directly on a display device, such as a display screen, etc. Since the above warning information can reflect the location of the fault in the software to be maintained, operations engineers can quickly locate the fault before or when the application software malfunctions by receiving and / or displaying the warning information.

[0055] Optionally, the anomaly attribute can include priority, which reflects the anomaly level of the anomaly information corresponding to that attribute. It should be noted that some anomaly information at certain anomaly levels may require anomaly warnings, while others may not. Determining whether to issue an anomaly warning based on the anomaly attribute can include: judging whether to issue an anomaly warning based on the numerical relationship between the priority and a pre-set anomaly warning threshold. For example, anomaly information corresponding to priorities greater than, greater than or equal to, less than, or less than or equal to the anomaly warning threshold can be considered information requiring anomaly warnings. Furthermore, issuing anomaly warnings based on the warning information can include: sending warning information based on the warning method corresponding to the priority. The reason for this setting is that although anomaly information at at least two priorities may require anomaly warnings, the severity of the faults caused by anomaly information at different priorities is different. Therefore, anomaly warnings can be issued based on the warning method corresponding to the priority. For example, the warning method for a priority corresponding to a minor fault can be set to sending an email, a system message, or a message on a warning dashboard, etc., while the warning method for a priority corresponding to a major fault can be set to sending a short message or a WeChat message to the relevant operations engineer, etc. The above technical solution can determine whether to issue an anomaly warning based on the priority of the abnormal information, and when it is determined to issue an anomaly warning, it can send an anomaly warning through the warning method corresponding to the priority. This achieves the effect of issuing corresponding anomaly warnings under various circumstances and improves the distinguishability of anomaly warnings.

[0056] The technical solution of this invention standardizes the format of log information from the software under maintenance, which may originate from at least one channel, to obtain uniformly formatted log information. If abnormal information is obtained after processing the log information, it indicates that the software under maintenance may have a fault. In this case, the abnormal attribute of the abnormal information can be determined to accurately judge whether an abnormal warning is needed for the software under maintenance. If so, a warning message can be generated based on the abnormal information and / or log information, and an abnormal warning can be issued based on the warning message. This helps maintenance engineers quickly locate the fault in the software under maintenance. The above technical solution can proactively acquire the log information of the software under maintenance and issue abnormal warnings for abnormal log information, thereby achieving the effect of quickly locating the fault in the software under maintenance.

[0057] In one optional technical solution, after receiving an anomaly alert, the operations engineer may process the alert. Therefore, the log processing method described above may further include: upon detecting a processing trigger event for alert processing, acquiring the processing operation information corresponding to the trigger event and storing the processing operation information in the alert processing module. The trigger event can be an event triggered by the operations engineer to process the anomaly alert; therefore, the processing operation information corresponding to the trigger event can be acquired and stored in the alert processing module. This processing operation information can be information about the operations engineer's direct processing operations, or information about the log processing system automatically processing operations based on the operations engineer's feedback on the anomaly alert, etc., without specific limitations. Furthermore, the operations engineer can also perform statistical analysis on various faults occurring within a certain time period. Therefore, upon detecting an analysis trigger event for alert analysis, the information to be analyzed corresponding to the analysis trigger event can be filtered from the processing operation information stored in the alert processing module. This information can then be analyzed, and the analysis results can be displayed. The above technical solution can perform statistical analysis on the processing operation information executed after an anomaly warning, which helps operation and maintenance engineers to summarize various fault situations in order to better improve the operation and maintenance software.

[0058] Example 2

[0059] Figure 2This is a flowchart of a log processing method provided in Embodiment 2 of the present invention. This embodiment is based on and optimized from the above-described technical solutions. In this embodiment, optionally, obtaining the log information of the software to be maintained may specifically include: obtaining the log information of the software to be maintained from log storage files set in a terminal, server, and / or network; and / or, obtaining log information based on a log collection interface set in the log processing system, wherein the log information collected by the log collection interface includes information pushed by the software to be maintained to the log collection interface based on an application programming interface embedded in the software to be maintained; and / or, obtaining log information when the log processing system executes a test task. The explanations of terms that are the same as or corresponding to those in the above embodiments will not be repeated here.

[0060] See Figure 2 The method in this embodiment may specifically include the following steps:

[0061] S210. Obtain log information of the software to be maintained from a log storage file located in a terminal, server, and / or network associated with the software to be maintained; and / or, obtain log information based on a log collection interface located in the log processing system, wherein the log information collected by the log collection interface includes information pushed by the software to be maintained to the log collection interface based on an application programming interface embedded in the software to be maintained; and / or, obtain log information when the log processing system performs a test task.

[0062] The log processing system can obtain log information from the software under maintenance in various ways. For example, it can obtain log information from log storage files located in terminals, servers, and / or networks associated with the software. This is an active way to obtain log information. Another example is obtaining log information through a log collection interface set up in the log processing system. The log information collected by this interface can be information pushed to the log collection interface by the software under maintenance through an application programming interface (API) embedded in the software. That is, the log collection interface can require the software under maintenance to transmit log information to the log collection interface through the API when performing log operations. This is a passive way to obtain log information. Yet another example is obtaining log information when the log processing system executes test tasks. This is a test dialing process, which can be achieved by writing a program to continuously call a certain function or simulate a certain IP address to determine whether the network is working properly. Of course, log information can also be obtained through other methods, which are not specifically limited here.

[0063] S220. Standardize the format of the log information and update the log information according to the format standardization result.

[0064] S230. If abnormal information is obtained after processing the log information, the abnormal attributes of the abnormal information are determined, and an abnormal warning is issued based on the abnormal attributes.

[0065] S240. If so, generate warning information based on the abnormal information and / or log information, and issue an abnormal warning based on the warning information.

[0066] The technical solution of this invention obtains the log information of the software to be maintained through multiple methods, thereby achieving the effect of comprehensive collection of log information and ensuring the comprehensiveness of log information.

[0067] Example 3

[0068] Figure 3 This is a flowchart of a log processing method provided in Embodiment 3 of the present invention. This embodiment is an optimization based on the above-described technical solutions. Optionally, in this embodiment, the log processing method may further include: if no abnormal information is obtained or if it is determined based on abnormal attributes that no abnormal warning will be issued, then the log information is cached, and the cached log information is stored in a log storage module configured on the log processing system at a preset time. The explanations of terms that are the same as or corresponding to those in the above embodiments will not be repeated here.

[0069] See Figure 3 The method in this embodiment may specifically include the following steps:

[0070] S310. Obtain the log information of the software to be maintained, standardize the format of the log information, and update the log information according to the format standardization result. Each log information comes from at least one of the software to be maintained, and / or the terminal, server and network associated with the software to be maintained.

[0071] S320. If abnormal information is obtained after processing the log information, the abnormal attributes of the abnormal information are determined, and an abnormal warning is issued based on the abnormal attributes.

[0072] S330. If so, generate warning information based on the abnormal information and / or log information, and issue an abnormal warning based on the warning information.

[0073] S340. If no abnormal information is received or if no abnormal warning is issued based on the abnormal attributes, the log information is cached and stored in the log storage module set on the log processing system at a preset time.

[0074] In addition to the cases described above where abnormal information is obtained and an abnormal warning is required based on the abnormal attributes of the abnormal information, after processing the log information, there may also be cases where no abnormal information is obtained, meaning there is no abnormality in the log information, or where abnormal information is obtained but an abnormality is not required based on its abnormal attributes, meaning the log information contains an abnormality but the abnormality has not yet reached the level requiring an abnormal warning. In these cases, the log information can be cached first, and the cached log information can be stored in the log storage module set on the log processing system within a preset time. That is, the log information is pushed to the log storage module for storage according to a fixed period. This setting method is particularly suitable for situations with a large number of concurrent information, as the method of caching first and then batch submitting increases the concurrency.

[0075] It should be noted that in practical applications, the early warning processing module and the log storage module mentioned above can be merged into one module. That is, the log processing system has a large module, which includes two smaller modules: the early warning processing module and the log storage module. They can store different types of information respectively, and the relevant information of the same software to be maintained can be linked together through a unique identifier. The advantage of this setup is that all the information of the software to be maintained can be retrieved in the large module later, so as to directly view and analyze it from a global perspective.

[0076] For example, taking the log storage module as an example, the above log processing method may further include: when a log retrieval trigger event is detected, filtering out the log to be retrieved corresponding to the retrieval trigger event from the log information stored in the log storage module; and displaying the log to be retrieved. Here, the retrieval trigger event may be an event triggered by an operations engineer to retrieve log information that meets certain conditions. Therefore, the log to be retrieved corresponding to the retrieval trigger event can be filtered out from the log information stored in the log storage module, and then the log to be retrieved can be displayed. The log to be retrieved may be the log information to be retrieved. As another example, taking the early warning processing module as an example, when a log retrieval trigger event is detected, the information to be retrieved corresponding to the retrieval trigger event can also be filtered out from the processing operation information stored in the early warning processing module, and the log to be retrieved can be displayed.

[0077] The technical solution of this invention achieves high data concurrency by caching log information first, without issuing anomaly warnings or based on anomaly attributes, and then storing the cached log information in the log storage module set on the log processing system at a preset time.

[0078] To better understand the specific implementation process of the above steps, the following provides an exemplary description of the architecture diagram and flowchart of the log processing system used in this embodiment, with specific examples. For example, Figure 4a and Figure 4b As shown, a log processing system aimed at intelligent operation and maintenance may include the following modules: a log information collection module, a log information transmission module, a log information processing module, a log information analysis module, a log information storage module, an early warning information notification module, a log information retrieval module, an early warning information processing module, and a statistical analysis module. The specific functions of each module are as follows:

[0079] 1. Log information collection module, used to collect various log information involved in the operation of the software to be maintained, specifically divided into:

[0080] 1) Collect information from servers with deployed maintenance software, using methods such as syslog, WMI, SNMP, LEA, telnet, JMX, SSH, and test dialing to obtain the corresponding log information.

[0081] 2) Collect terminal information for software to be maintained by using methods such as syslog, WMI, SNMP, LEA, telnet, JMX, SSH and test dialing to obtain the corresponding log information.

[0082] 3) Collect network information for the software to be maintained, using methods such as syslog, WMI, SNMP, LEA, telnet, JMX, SSH, and test dialing to obtain the corresponding log information.

[0083] 4) Collect software information about the software to be maintained, publish API interfaces, and obtain log information generated during the operation of the software through the API interfaces embedded in the software to be maintained.

[0084] It should be noted that Syslog and WMI are typical existing log storage files; telnet and SSH test dialing are typical methods for obtaining running status when the log processing system performs test tasks; and the API interface embedded in the software to be maintained can actively push the running information of the software to be maintained to the log processing system.

[0085] 2. Log information transmission module, used to immediately transmit log information to log information processing module after receiving log information pushed by log information collection module;

[0086] 3. The log information processing module, upon receiving log information from the log information transmission module, first standardizes the log information format, then calls the log information analysis module to analyze the standardized log information. Based on the analysis results, two methods can be used: either calling the early warning information notification module to send early warning information to the operations engineer; or calling the log information storage module to store the standardized log information. Specifically...

[0087] 1) Standardize the format of log information. Standardize the log information collected from various channels according to the standard format to obtain log information with a unified format, such as log source, log level, log content, etc.

[0088] 2) Cache log information: Cache the standardized log information.

[0089] 3) Log information processing and scheduling: The log information analysis module is called to analyze the log information. If there is abnormal information in the analysis results that is greater than the abnormal warning threshold set by the user, the warning information notification module is immediately called to issue a warning information. For abnormal information whose analysis results are not abnormal or are less than the abnormal warning threshold set by the user, they are first cached and pushed to the log storage module for storage at fixed intervals.

[0090] 4. Log Information Analysis Module: Called by the log information processing module, this module analyzes log information and determines whether any anomalies exist based on the analysis results. If so, it determines the priority of the anomalies. Specifically:

[0091] 1) Log source settings: A graphical user interface is provided for users to set the source of log information, such as hardware, network, general software to be maintained, business software to be maintained, etc.

[0092] 2) Log level settings: Provides a graphical interface for users to set the level of log information, such as message, debug, warning, error, etc., based on the content of the log information. Each level can also be further subdivided by user-defined priority levels.

[0093] 3) Setting up early warning rules: A graphical user interface is provided for users to set up log information that requires early warning, such as by log source, log level, log content, etc., and set up corresponding early warning methods, such as email, SMS, WeChat, early warning dashboard, etc. (i.e., proactive notification and proactive display). For catastrophic early warning information, the contact information of the early warning notifier can also be set, etc.

[0094] 4) Log information analysis engine: Analyzes log information one by one according to the set early warning rules, and generates corresponding early warning information if there are any abnormalities.

[0095] 5. Log information storage module, which is called by the log information processing module, is used to store log information for later retrieval.

[0096] 6. Early warning information notification module, which is called by the log information processing module, is used to send log information and corresponding abnormal information with a priority greater than the user-set abnormal early warning threshold to the operation and maintenance engineer.

[0097] 7. Log Information Retrieval Module: Provides a graphical user interface for maintenance engineers to view log information. Log information can be filtered and searched based on specific conditions. This log information retrieval module can include all log information of the software under maintenance during its operation and can be viewed directly from a global perspective.

[0098] 8. Early Warning Information Processing Module: Provides a graphical user interface for maintenance engineers to process received early warning information. Processing methods include viewing fault details, confirming fault information, troubleshooting fault information, ignoring fault information, forwarding fault information, reporting fault information, etc. For example, the processing information for fault A is as follows: the occurrence time was 9:25 AM on October 26, 2020; the confirmation time was 9:30 AM on October 26, 2020; the handler was Zhang San; the processing method was updating backend data; the fault cause was an operational error leading to abnormal backend data; and the confirmed repair time was 9:40 AM on October 26, 2020.

[0099] 9. Statistical Analysis Module: Provides a graphical user interface for the operations and maintenance department to perform statistical analysis on abnormal information, fault information, and fault handling information of the software to be maintained within a specified time period. This statistical analysis of fault information and fault handling information can reflect the failure rate, repair rate, and workload of maintenance engineers, etc., and it can also help operations and maintenance engineers improve subsequent operations and maintenance work.

[0100] The aforementioned log processing system, aimed at intelligent operation and maintenance, collects log information from the operating environment of the software under maintenance through servers and terminals deployed with the software to be maintained, and API interfaces embedded in the software. It standardizes the format of this log information, and the log information analysis module analyzes the log information in real time, proactively detecting anomalies and issuing early warnings. Combined with the log information retrieval module and statistical analysis module, it achieves unified management standards, unified collection and proactive analysis of log information, and effective anomaly warnings. Specifically, the aforementioned log information system has the following advantages:

[0101] 1) It has achieved full collection and format standardization of log information in the business processes of servers and terminals with software to be maintained, thereby forming a unified standard for log information management.

[0102] 2) Unified storage of log information has been achieved, thereby forming a unified log information database. Subsequent searches can be performed on massive amounts of log information based on conditions, improving the efficiency of log information retrieval.

[0103] 3) It enables proactive analysis and early warning of log information, which allows maintenance engineers to promptly discover or even detect anomalies in the software itself or its operating environment, thereby reducing or even eliminating the disruption of daily operations caused by software failures of the software to be maintained.

[0104] 4) Fault information and handling methods processed during daily operation and maintenance can be archived, and statistical analysis can be performed on them later to guide and optimize operation and maintenance work.

[0105] In summary, the log processing system described in this embodiment of the invention proposes a management standard for managing log information of software under maintenance, and uniformly collects, stores, and analyzes the generated log information. By extracting key log information, it can promptly detect faults and provide anomaly warnings. In other words, the aforementioned log processing system will largely revolutionize existing maintenance methods, realizing a unified standard for log management of hardware, software, and network, as well as unified collection, processing, and analysis of log information. This allows for real-time and comprehensive monitoring of the operational status of various resources, devices, and software under maintenance in the IT environment, and comprehensively displays log information related to the running software in sequence around fault information. By assisting maintenance engineers in quickly detecting anomalies and assisting in troubleshooting, it improves the maintenance capabilities of maintenance engineers and reduces the adverse impact of software failures on daily operations.

[0106] Example 4

[0107] Figure 5 This is a structural block diagram of a log processing apparatus provided in Embodiment 4 of the present invention. This apparatus is used to execute the log processing method provided in any of the above embodiments. This apparatus and the log processing methods of the above embodiments belong to the same inventive concept. Details not described in detail in the embodiments of the log processing apparatus can be found in the embodiments of the above log processing methods. See also... Figure 5 The device is configured in a log processing system and may include: a log information acquisition module 410, an anomaly warning judgment module 420, and an anomaly warning execution module 430.

[0108] The log information acquisition module 410 is used to acquire the log information of the software to be maintained, standardize the format of the log information, and update the log information according to the format standardization result. Each log information comes from at least one of the software to be maintained, and / or the terminal, server and network associated with the software to be maintained.

[0109] The anomaly warning judgment module 420 is used to determine the anomaly attribute of the anomaly information if anomaly information is obtained after processing the log information, and to determine whether to issue an anomaly warning based on the anomaly attribute.

[0110] The anomaly warning execution module 430 is used to generate warning information based on the anomaly information and / or log information if the anomaly is true, and to issue an anomaly warning based on the warning information.

[0111] Optionally, the log information acquisition module 410 may specifically include:

[0112] The log information acquisition unit is used to acquire log information of the software to be maintained from log storage files set in terminals, servers and / or networks; and / or, to acquire log information based on the log collection interface set in the log processing system, wherein the log information collected by the log collection interface includes information pushed to the log collection interface by the software to be maintained based on the application interface embedded in the software to be maintained; and / or, to acquire log information when the log processing system performs test tasks.

[0113] Optionally, the anomaly warning and judgment module 420 may specifically include:

[0114] The log attribute acquisition unit is used to process log information and obtain the log attributes of the log information. The log attributes include log source, log level and / or log content.

[0115] The abnormal attribute determination unit is used to compare log attributes with preset attributes. If it is determined that abnormal information should be generated based on the comparison result, the abnormal attributes of the abnormal information are determined. The preset attributes include the attributes of the log information to be given abnormal warnings that have been set in advance.

[0116] Optionally, the exception attributes include priority. The exception warning judgment module 420 may specifically include:

[0117] An anomaly warning judgment unit is used to determine whether to issue an anomaly warning based on the numerical relationship between the priority and the pre-set anomaly warning threshold.

[0118] Correspondingly, the anomaly warning execution module 430 may specifically include:

[0119] The anomaly warning execution unit is used to send warning information to provide anomaly warnings based on the warning method corresponding to the priority.

[0120] Optionally, the above-mentioned log processing apparatus may further include:

[0121] The log information storage module is used to cache log information if no abnormal information is received or if it is determined that no abnormal warning will be issued based on the abnormal attributes. The cached log information is then stored in the log storage module set on the log processing system at a preset time.

[0122] Optionally, based on the above-described apparatus, the apparatus may further include:

[0123] The log filtering module is used to filter out the logs to be accessed from the log information stored in the log storage module when a log access trigger event is detected.

[0124] The pending log display module is used to display the pending logs.

[0125] Optionally, the above-mentioned log processing apparatus may further include:

[0126] The processing operation information storage module is used to obtain the processing operation information corresponding to the processing trigger event when the processing trigger event of the early warning processing is detected, and to store the processing operation information in the early warning processing module;

[0127] The information to be analyzed filtering module is used to filter out the information to be analyzed corresponding to the analysis trigger event from the various processing operation information stored in the early warning processing module when the analysis trigger event is detected.

[0128] The analysis results display module is used to analyze the information to be analyzed and display the analysis results.

[0129] The log processing device provided in Embodiment 4 of this invention standardizes the format of log information from the software under maintenance, which may originate from at least one channel, through a log information acquisition module to obtain uniformly formatted log information. If an anomaly is detected after processing the log information by the anomaly warning judgment module, it indicates that the software under maintenance may have a fault. The anomaly attribute of the anomaly information can be determined to accurately determine whether an anomaly warning is needed for the software under maintenance. If so, the anomaly warning execution module generates warning information based on the anomaly information and / or log information, and performs an anomaly warning based on this warning information. This helps maintenance engineers quickly locate the fault in the software under maintenance. The above device can proactively acquire log information from the software under maintenance and provide anomaly warnings for abnormal log information, thereby achieving the effect of quickly locating the fault in the software under maintenance.

[0130] The log processing apparatus provided in the embodiments of the present invention can execute the log processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0131] It is worth noting that in the embodiments of the above-mentioned log processing device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0132] Example 5

[0133] Figure 6 This is a schematic diagram of the structure of a log processing system provided in Embodiment 5 of the present invention, as shown below. Figure 6 As shown, the log processing system includes a memory 510, a processor 520, an input device 530, and an output device 540. The number of processors 520 in the log processing system can be one or more. Figure 6 Taking a processor 520 as an example; the memory 510, processor 520, input device 530, and output device 540 in the log processing system can be connected via a bus or other means. Figure 6 Taking the connection via bus 550 as an example.

[0134] The memory 510, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the log processing method in this embodiment of the invention (e.g., the log information acquisition module 410, the anomaly warning judgment module 420, and the anomaly warning execution module 430 in the log processing device). The processor 520 executes various functional applications and data processing of the log processing system by running the software programs, instructions, and modules stored in the memory 510, thereby implementing the aforementioned log processing method.

[0135] The memory 510 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the log processing system. Furthermore, the memory 510 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 510 may further include memory remotely located relative to the processor 520, which can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0136] Input device 530 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the device. Output device 540 may include display devices such as a display screen.

[0137] Example 6

[0138] Embodiment 6 of the present invention provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform a log processing method. This method, applied to a log processing system, may include:

[0139] Obtain log information from the software to be maintained, standardize the format of the log information, and update the log information according to the format standardization result. Each log information comes from at least one of the software to be maintained and / or the terminal, server and network associated with the software to be maintained. If abnormal information is obtained after processing the log information, determine the abnormal attributes of the abnormal information and determine whether to issue an abnormal warning based on the abnormal attributes. If so, generate warning information based on the abnormal information and / or log information, and issue an abnormal warning based on the warning information.

[0140] Of course, the computer-executable instructions provided in the embodiments of the present invention are not limited to the method operations described above, but can also perform related operations in the log processing method provided in any embodiment of the present invention.

[0141] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. According to this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0142] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A log processing method, characterized in that, Applied to a log processing system, the method includes: Obtain log information of the software to be maintained, standardize the format of the log information, and update the log information according to the format standardization result. Each log information originates from at least one of the software to be maintained, and / or the terminal, server and network associated with the software to be maintained. If abnormal information is obtained after processing the log information, the abnormal attributes of the abnormal information are determined, and an abnormal warning is issued based on the abnormal attributes; wherein, the abnormal attributes include the number, priority, degree and frequency of occurrence of abnormalities; If so, then generate warning information based on the abnormal information and / or the log information, and issue an abnormal warning based on the warning information; The method further includes: If the abnormal information is not obtained or if it is determined that no abnormal warning will be issued based on the abnormal attributes, the log information is cached, and the cached log information is stored in the log storage module set on the log processing system at a preset time.

2. The method according to claim 1, characterized in that, The acquisition of log information of the software to be maintained includes: Obtain log information of the software to be maintained from log storage files located in the terminal, the server, and / or the network; and / or, The log information is obtained based on the log collection interface set in the log processing system, wherein the log information collected by the log collection interface includes information pushed to the log collection interface by the software to be maintained based on the application programming interface embedded in the software to be maintained; and / or, The log information is obtained when the log processing system performs a test task.

3. The method according to claim 1, characterized in that, If abnormal information is obtained after processing the log information, the abnormal attributes of the abnormal information are determined, including: The log information is processed to obtain the log attributes of the log information, wherein the log attributes include log source, log level and / or log content; The log attributes are compared with preset attributes. If it is determined that abnormal information is to be generated based on the comparison result, the abnormal attributes of the abnormal information are determined. The preset attributes include the attributes of the log information to be given an abnormal warning that are set in advance.

4. The method according to claim 1, characterized in that, The step of determining whether to issue an anomaly warning based on the anomaly attribute includes: determining whether to issue an anomaly warning based on the numerical relationship between the priority and the pre-set anomaly warning threshold; Accordingly, the abnormal warning based on the warning information includes: sending the warning information to issue an abnormal warning based on the warning method corresponding to the priority.

5. The method according to claim 1, characterized in that, Also includes: When a log access trigger event is detected, the log to be accessed corresponding to the access trigger event is selected from the log information stored in the log storage module. Display the logs to be accessed.

6. The method according to claim 1, characterized in that, Also includes: When a processing trigger event for early warning processing is detected, the processing operation information corresponding to the processing trigger event is obtained and stored in the early warning processing module; When an analysis trigger event for early warning analysis is detected, the information to be analyzed corresponding to the analysis trigger event is filtered from the processing operation information stored in the early warning processing module. The information to be analyzed is analyzed, and the analysis results are displayed.

7. A log processing device, characterized in that, Configured in a log processing system, the device includes: The log information acquisition module is used to acquire log information of the software to be maintained, standardize the format of the log information, and update the log information according to the format standardization result. The log information originates from at least one of the software to be maintained, and / or the terminal, server and network associated with the software to be maintained. An anomaly warning judgment module is used to determine the anomaly attribute of the anomaly information if anomaly information is obtained after processing the log information, and to determine whether to issue an anomaly warning based on the anomaly attribute; wherein, the anomaly attribute includes the number, priority, severity and frequency of occurrence of anomalies; An anomaly warning execution module is used to generate warning information based on the anomaly information and / or the log information if the anomaly is true, and to issue an anomaly warning based on the warning information. The device further includes: The log information storage module is used to cache the log information if no abnormal information is obtained or if it is determined not to issue an abnormal warning based on the abnormal attributes, and to store the cached log information in the log storage module set on the log processing system at a preset time.

8. A log processing system, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the log processing method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the log processing method as described in any one of claims 1-6.

Images