Log anomaly identification methods, devices, equipment and storage media

Abstract

This invention discloses a method, apparatus, device, and storage medium for log anomaly identification, belonging to the field of computer technology. The method includes: receiving log data to be identified; establishing a full-text index of the log data; and obtaining log anomaly identification results based on preset indicator thresholds and anomaly judgment rules, according to the full-text index. By using preset indicator thresholds and anomaly judgment rules, log anomaly identification results are obtained. This achieves log anomaly identification and solves the technical problem of poor accuracy in monitoring transaction systems. Compared with existing technologies, it has the advantage of higher accuracy.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, device, and storage medium for identifying log anomalies. Background Technology

[0002] Existing transaction systems log a large amount of alarm information. However, these alarms are interspersed with noise (inaccurate alarms, legitimate business errors, etc.), and these log messages are generally not directly related, resulting in isolated information and excessive redundancy, which hinders anomaly identification. Identified anomalies cannot be applied to the analysis and identification of other similar logs. There is a lack of backtracking mechanisms for system anomalies, making it difficult to quickly and accurately analyze the large volume of logs generated in real time. Summary of the Invention

[0003] The main objective of this invention is to provide a log anomaly identification method, apparatus, device, and storage medium, aiming to solve the problem of inaccurate identification of abnormal information in logs.

[0004] To achieve the above objectives, the present invention provides a log anomaly identification method, which includes the following steps:

[0005] Receive log data to be identified;

[0006] Based on the log data to be identified, a full-text index of the logs is established;

[0007] Based on preset indicator thresholds and anomaly judgment rules, log anomaly identification results are obtained according to the full-text log index.

[0008] Optionally, the step of establishing a full-text index of the logs based on the log data to be identified includes:

[0009] The log data to be identified is processed to obtain the first log data;

[0010] Store the first log data in the full-text index analysis engine;

[0011] The full-text indexing analysis engine is used to create a full-text index of the log data based on preset important fields in the first log data.

[0012] Optionally, the step of processing the log data to be identified to obtain the first log data includes:

[0013] The log data to be identified is preprocessed to obtain preprocessed log data to be identified;

[0014] According to preset annotation conditions, the preprocessed log data to be identified is annotated to obtain annotated log data to be identified;

[0015] The labeled log data to be identified is classified according to the preset data priority classification conditions to obtain the first log data.

[0016] Optionally, the step of storing the first log data in the full-text indexing and analysis engine includes:

[0017] Based on preset hierarchical storage conditions, the first log data is stored hierarchically using the full-text indexing and analysis engine.

[0018] Optionally, the step of obtaining log anomaly identification results based on the preset indicator thresholds and anomaly judgment rules, according to the full-text log index, includes:

[0019] Based on the full-text log index, the link-related data in the first log data is analyzed to obtain the log anomaly range value;

[0020] Based on the full-text index of the log, the interface-related data in the first log data is evaluated to obtain the log anomaly level value;

[0021] Based on the aforementioned indicator thresholds and anomaly determination rules, the log anomaly identification results are obtained according to the log anomaly range values ​​and log anomaly severity values.

[0022] Optionally, the step of obtaining the log anomaly identification result based on the indicator threshold and anomaly judgment rules, according to the log anomaly range value and log anomaly severity value, includes:

[0023] Based on the log anomaly range value and the log anomaly severity value, generate the log anomaly impact value;

[0024] Determine whether the log anomaly impact value is greater than the indicator threshold;

[0025] If the log anomaly impact value is greater than the indicator threshold, then the log anomaly identification result is generated based on the anomaly determination rule.

[0026] Optionally, after the step of obtaining the log anomaly identification result based on the full-text log index according to the preset indicator threshold and anomaly judgment rules, the method further includes:

[0027] Based on the log anomaly identification results, send an anomaly alert message to the terminal;

[0028] Receive the anomaly identification result feedback sent by the terminal;

[0029] Based on the feedback of the anomaly identification results, the indicator thresholds and anomaly judgment rules are corrected to obtain key target thresholds and key anomaly judgment rules.

[0030] Furthermore, to achieve the above objectives, the present invention also provides a log anomaly identification device, the log anomaly identification device comprising:

[0031] The receiving module is used to receive log data to be identified;

[0032] The index building module is used to build a full-text index of the logs based on the log data to be identified;

[0033] The anomaly identification module is used to obtain log anomaly identification results based on preset indicator thresholds and anomaly judgment rules, according to the full-text index of the log.

[0034] This invention also proposes a terminal device, which includes a memory, a processor, and a log anomaly identification program stored in the memory and executable on the processor. When the log anomaly identification program is executed by the processor, it implements the steps of the log anomaly identification method as described above.

[0035] This invention also proposes a computer-readable storage medium storing a log anomaly identification program, which, when executed by a processor, implements the steps of the log anomaly identification method described above.

[0036] This invention proposes a log anomaly identification method, apparatus, device, and storage medium. The method involves receiving log data to be identified; establishing a full-text index based on the log data; and obtaining log anomaly identification results based on preset indicator thresholds and anomaly judgment rules, according to the full-text index. By using preset indicator thresholds and anomaly judgment rules, log anomaly identification results are obtained. This achieves log anomaly identification and solves the technical problem of poor accuracy in monitoring transaction systems. Compared to existing technologies, it has the advantage of higher accuracy.

[0037] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the functional modules of the terminal device to which the log anomaly identification device of the present invention belongs;

[0039] Figure 2This is a flowchart illustrating a first exemplary embodiment of the log anomaly identification method of the present invention;

[0040] Figure 3 This is a flowchart illustrating a second exemplary embodiment of the log anomaly identification method of the present invention;

[0041] Figure 4 This is a flowchart illustrating a third exemplary embodiment of the log anomaly identification method of the present invention;

[0042] Figure 5 This is a flowchart illustrating a fourth exemplary embodiment of the log anomaly identification method of the present invention;

[0043] Figure 6 This is a flowchart illustrating a fifth exemplary embodiment of the log anomaly identification method of the present invention;

[0044] Figure 7 This is a flowchart illustrating a sixth exemplary embodiment of the log anomaly identification method of the present invention;

[0045] Figure 8 This is a flowchart illustrating a seventh exemplary embodiment of the log anomaly identification method of the present invention;

[0046] Figure 9 This is a schematic diagram of the overall process of another exemplary embodiment of the log anomaly identification method of the present invention.

[0047] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0048] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0049] The main solution of this invention is as follows: receiving log data to be identified; establishing a full-text index of the log data; and obtaining log anomaly identification results based on preset indicator thresholds and anomaly judgment rules, according to the full-text index of the log data. By using preset indicator thresholds and anomaly judgment rules, log anomaly identification results are obtained. Thus, log anomaly identification is achieved, solving the technical problem of poor accuracy in monitoring transaction systems. Compared with existing technologies, it has the advantage of higher accuracy.

[0050] The embodiments of the present invention take into account that the logs of related technical solutions have poor real-time performance and low accuracy when performing anomaly identification.

[0051] Based on this, this invention proposes a solution that can accurately identify log anomalies, helping system developers and users discover system and business anomalies, solving the accuracy problem of system monitoring, and improving the accuracy of log anomaly identification.

[0052] Specifically, refer to Figure 1 , Figure 1 This is a schematic diagram of the functional modules of the terminal device to which the log anomaly identification device of the present invention belongs. This log anomaly identification device can be an independent device capable of identifying log anomalies, and it can be implemented on the terminal device in hardware or software form. The terminal device can be a smart mobile device with data processing capabilities, such as a mobile phone or tablet computer, or it can be a fixed terminal device or server with data processing capabilities.

[0053] In this embodiment, the terminal device to which the log anomaly identification device belongs includes at least an output module 110, a processor 120, a memory 130, and a communication module 140.

[0054] The memory 130 stores the operating system and a log anomaly identification program. The log anomaly identification device can receive log data to be identified; establish a full-text index of the log data; and obtain the log anomaly identification result based on the full-text index, according to preset indicator thresholds and anomaly judgment rules. The log anomaly identification program preprocesses the log data to be identified, generating log anomaly identification results and other information, which are stored in the memory 130. The output module 110 can be a display screen, etc. The communication module 140 can include a WIFI module, a mobile communication module, and a Bluetooth module, etc., and communicates with external devices or servers through the communication module 140.

[0055] When the log exception detection program in memory 130 is executed by the processor, it performs the following steps:

[0056] Receive log data to be identified;

[0057] Based on the log data to be identified, a full-text index of the logs is established;

[0058] Based on preset indicator thresholds and anomaly judgment rules, log anomaly identification results are obtained according to the full-text log index.

[0059] Furthermore, when the log exception detection program in memory 130 is executed by the processor, it also performs the following steps:

[0060] The log data to be identified is processed to obtain the first log data;

[0061] Store the first log data in the full-text index analysis engine;

[0062] The full-text indexing analysis engine is used to create a full-text index of the log data based on preset important fields in the first log data.

[0063] Furthermore, when the log exception detection program in memory 130 is executed by the processor, it also performs the following steps:

[0064] The log data to be identified is preprocessed to obtain preprocessed log data to be identified;

[0065] According to preset annotation conditions, the preprocessed log data to be identified is annotated to obtain annotated log data to be identified;

[0066] The labeled log data to be identified is classified according to the preset data priority classification conditions to obtain the first log data.

[0067] Furthermore, when the log exception detection program in memory 130 is executed by the processor, it also performs the following steps:

[0068] Based on preset hierarchical storage conditions, the first log data is stored hierarchically using the full-text indexing and analysis engine.

[0069] Furthermore, when the log exception detection program in memory 130 is executed by the processor, it also performs the following steps:

[0070] Based on the full-text log index, the link-related data in the first log data is analyzed to obtain the log anomaly range value;

[0071] Based on the full-text index of the log, the interface-related data in the first log data is evaluated to obtain the log anomaly level value;

[0072] Based on the aforementioned indicator thresholds and anomaly determination rules, the log anomaly identification results are obtained according to the log anomaly range values ​​and log anomaly severity values.

[0073] Furthermore, when the log exception detection program in memory 130 is executed by the processor, it also performs the following steps:

[0074] Based on the log anomaly range value and the log anomaly severity value, generate the log anomaly impact value;

[0075] Determine whether the log anomaly impact value is greater than the indicator threshold;

[0076] If the log anomaly impact value is greater than the indicator threshold, then the log anomaly identification result is generated based on the anomaly determination rule.

[0077] Furthermore, when the log exception detection program in memory 130 is executed by the processor, it also performs the following steps:

[0078] Based on the log anomaly identification results, send an anomaly alert message to the terminal;

[0079] Receive the anomaly identification result feedback sent by the terminal;

[0080] Based on the feedback of the anomaly identification results, the indicator thresholds and anomaly judgment rules are corrected to obtain key target thresholds and key anomaly judgment rules.

[0081] This embodiment, through the above-described scheme, specifically involves receiving log data to be identified; establishing a full-text index of the log data; and obtaining log anomaly identification results based on preset indicator thresholds and anomaly judgment rules, according to the full-text index. By using preset indicator thresholds and anomaly judgment rules, log anomaly identification results are obtained. Thus, log anomaly identification is achieved, solving the technical problem of poor accuracy in monitoring transaction systems. Compared to existing technologies, it has the advantage of higher accuracy.

[0082] Based on, but not limited to, the terminal device architecture described above, embodiments of the method of the present invention are proposed.

[0083] Reference Figure 2 , Figure 2 This is a flowchart illustrating a first exemplary embodiment of the log anomaly identification method of the present invention. The log anomaly identification method includes:

[0084] Step S101: Receive the log data to be identified;

[0085] The execution subject of the method in this embodiment is a log anomaly analysis and identification service system, or a log anomaly analysis and identification device, or a log anomaly analysis and identification equipment. This embodiment takes a log anomaly analysis and identification device as an example. The log anomaly analysis and identification device can be integrated into a device with data processing function.

[0086] To address the issue of poor accuracy in monitoring the trading system, improving the accuracy of identifying log anomalies is crucial. The following steps are taken to achieve accurate log anomaly identification:

[0087] First, a special log anomaly analysis and identification device is integrated into a device with data processing capabilities. This device can identify anomalies in the logs.

[0088] Finally, the log anomaly analysis and identification device receives log data to be identified, which includes log data and anomaly data. This data is sent actively by the trading system or automatically collected periodically. After the trading system preprocesses the data and adds necessary information, it is sent to the message middleware. The log anomaly analysis and identification device receives the data through the message middleware, which serves as the data transmission channel, thus decoupling the trading system from the log anomaly analysis and identification device.

[0089] Step S102: Based on the log data to be identified, establish a full-text index of the logs;

[0090] After the log anomaly analysis and identification device receives the log data to be identified, it establishes a full-text index of the log data to improve data retrieval speed. This is achieved through the following steps:

[0091] First, the log data to be identified is processed to obtain the first log data;

[0092] Finally, the log anomaly analysis and identification device uses a full-text index analysis engine to build a full-text index of the logs based on important fields in the first log data.

[0093] Step S103: Based on preset indicator thresholds and anomaly judgment rules, obtain log anomaly identification results according to the full-text log index.

[0094] After the full-text index of the logs is established, the log anomaly analysis and identification device generates log anomaly identification results based on preset indicator thresholds and anomaly judgment rules, according to the first log data and the full-text index of the logs, by taking the following steps:

[0095] First, the log anomaly analysis and identification service device obtains the log anomaly range and log anomaly severity values ​​through the link-related data and interface-related data in the first log data;

[0096] Finally, based on the indicator thresholds and anomaly judgment rules, log anomaly identification results are generated according to the log anomaly range and log anomaly severity values.

[0097] This embodiment, through the above-described scheme, receives log data to be identified; establishes a full-text index of the log data; and obtains log anomaly identification results based on preset indicator thresholds and anomaly judgment rules, according to the full-text index of the log data. By using preset indicator thresholds and anomaly judgment rules, log anomaly identification results are obtained. Thus, log anomaly identification is achieved, solving the technical problem of poor accuracy in monitoring transaction systems. Compared to existing technologies, it has the advantage of higher accuracy.

[0098] Reference Figure 3 , Figure 3This is a flowchart illustrating a second exemplary embodiment of the log anomaly identification method of the present invention.

[0099] Based on the above Figure 2 In the embodiment shown, step S102, establishing a full-text index of the logs based on the log data to be identified, may include:

[0100] Step S1021: Process the log data to be identified to obtain the first log data;

[0101] Step S1022: Store the first log data in the full-text index analysis engine;

[0102] Step S1023: Using the full-text index analysis engine, a log full-text index is established based on preset important fields in the first log data.

[0103] Specifically, after the log anomaly analysis and identification device receives the log data to be identified, it establishes a full-text index of the log based on the log data and the first log data to improve data retrieval speed. This is achieved through the following steps:

[0104] First, the log anomaly analysis and identification device processes the log data to be identified to obtain the first log data;

[0105] Then, the log anomaly analysis and identification device stores the first log data into the full-text index analysis engine;

[0106] Finally, the log anomaly analysis and identification device uses a full-text index analysis engine to build a full-text index of the logs based on preset important fields in the first log data.

[0107] This embodiment, through the above-described scheme, specifically involves processing the log data to be identified to obtain first log data; storing the first log data in a full-text indexing analysis engine; and using the full-text indexing analysis engine to establish a log full-text index based on preset important fields in the first log data. Based on the log data to be identified, a log full-text index is established. This results in a log full-text index, improving data retrieval speed and providing support for log anomaly identification.

[0108] Reference Figure 4 , Figure 4 This is a flowchart illustrating a third exemplary embodiment of the log anomaly identification method of the present invention.

[0109] Based on the above Figure 3 In the embodiment shown, step S1021, which involves processing the log data to be identified to obtain the first log data, includes:

[0110] Step S10211: Preprocess the log data to be identified to obtain preprocessed log data to be identified;

[0111] Step S10212: According to the preset annotation conditions, the preprocessed log data to be identified is annotated to obtain the annotated log data to be identified.

[0112] Step S10213: According to the preset data priority classification conditions, classify the labeled log data to be identified to obtain the first log data.

[0113] Specifically, after the log anomaly analysis and identification device receives the log data to be identified, it processes the log data to obtain the first log data, which is achieved by the following steps:

[0114] First, the log anomaly analysis and identification device preprocesses the log data to be identified and obtains the preprocessed log data to be identified.

[0115] Then, the log anomaly analysis and identification device annotates the preprocessed log data to be identified according to the preset annotation conditions, and obtains the annotated log data to be identified. The preset annotation conditions include product identification and scene identification.

[0116] Finally, the log anomaly analysis and identification device classifies the labeled log data to be identified according to preset data priority classification conditions to obtain the first log data.

[0117] This embodiment, through the above-described scheme, specifically involves preprocessing the log data to be identified to obtain preprocessed log data; labeling the preprocessed log data according to preset labeling conditions to obtain labeled log data; and classifying the labeled log data according to preset data priority classification conditions to obtain the first log data. By processing the log data to be identified, the first log data is obtained. This provides support for anomaly identification in logs.

[0118] Reference Figure 5 , Figure 5 This is a flowchart illustrating a fourth exemplary embodiment of the log anomaly identification method of the present invention.

[0119] Based on the above Figure 3 In the embodiment shown, step S1022, storing the first log data in the full-text indexing and analysis engine, includes:

[0120] Step S1022: Based on the preset hierarchical storage conditions, the first log data is stored hierarchically using the full-text indexing analysis engine.

[0121] Specifically, after generating the first log data, the first log data is stored in a hierarchical manner according to the preset hierarchical storage conditions, using a full-text indexing analysis engine. This is achieved through the following steps:

[0122] First, the log anomaly analysis and identification device uploads the first log data to the full-text index analysis engine;

[0123] Finally, the log anomaly analysis and identification device performs hierarchical storage of the first log data according to the preset hierarchical storage conditions and through the full-text index analysis engine.

[0124] This embodiment, through the above-described scheme, specifically by using the full-text indexing and analysis engine to perform hierarchical storage of the first log data according to preset hierarchical storage conditions. The first log data is stored hierarchically through the full-text indexing and analysis engine. This achieves hierarchical storage of the first log data, providing support for anomaly identification in the logs.

[0125] Reference Figure 6 , Figure 6 This is a flowchart illustrating a fifth exemplary embodiment of the log anomaly identification method of the present invention.

[0126] Based on the above Figure 2 In the embodiment shown, step S103, based on preset indicator thresholds and anomaly judgment rules, and according to the full-text log index, obtains the log anomaly identification result, including:

[0127] Step S1031: Based on the full-text log index, analyze the link-related data in the first log data to obtain the log anomaly range value;

[0128] Step S1032: Based on the full-text log index, evaluate the interface-related data in the first log data to obtain the log anomaly level value;

[0129] Step S1033: Based on the indicator threshold and anomaly judgment rules, obtain the log anomaly identification result according to the log anomaly range value and log anomaly degree value.

[0130] Specifically, based on preset indicator thresholds and anomaly judgment rules, and according to the first log data and the full-text index of the log, log anomaly identification results are generated, which is achieved through the following steps:

[0131] First, the log anomaly analysis and identification device analyzes the link-related data in the first log data based on the full-text index of the log to obtain the log anomaly range value. The link-related data includes the link ID associated data, upstream and downstream links, and error stack.

[0132] Then, the log anomaly analysis and identification device evaluates the interface-related data in the first log data based on the full-text index of the log, and obtains the log anomaly degree value. The interface-related data includes the interface name, interface performance, and the number of interface requests per unit time.

[0133] Finally, based on the indicator thresholds and anomaly judgment rules, the log anomaly identification results are obtained according to the log anomaly range and log anomaly severity values.

[0134] This embodiment, through the above-described scheme, specifically analyzes the link-related data in the first log data based on the full-text log index to obtain the log anomaly range value; evaluates the interface-related data in the first log data based on the full-text log index to obtain the log anomaly severity value; and obtains the log anomaly identification result based on the indicator threshold and anomaly judgment rules, according to the log anomaly range value and the log anomaly severity value. By using the link-related data and interface-related data in the first log data, the log anomaly identification result is obtained. Thus, log anomaly identification is achieved, solving the technical problem of poor accuracy in monitoring the transaction system. Compared with existing technologies, it has the advantage of higher accuracy.

[0135] Reference Figure 7 , Figure 7 This is a flowchart illustrating a sixth exemplary embodiment of the log anomaly identification method of the present invention.

[0136] Based on the above Figure 6 In the embodiment shown, step S1033, based on the indicator threshold and anomaly determination rules, and according to the log anomaly range value and log anomaly severity value, obtains the log anomaly identification result, including:

[0137] Step S10331: Generate log anomaly impact value based on the log anomaly range value and the log anomaly severity value;

[0138] Step S10332: Determine whether the log anomaly impact value is greater than the indicator threshold;

[0139] Step S10333: If the log anomaly impact value is greater than the indicator threshold, then the log anomaly identification result is generated based on the anomaly determination rule.

[0140] Specifically, after determining the log anomaly range and severity values, based on indicator thresholds and anomaly judgment rules, a log anomaly identification result is generated using a log anomaly analysis and identification device. This is achieved through the following steps:

[0141] First, the log anomaly analysis and identification device calculates the impact value of the log anomaly based on the log anomaly range value and the log anomaly severity value.

[0142] Then, the log anomaly analysis and identification device determines whether the log anomaly impact value is greater than the indicator threshold;

[0143] Finally, if the log anomaly impact value is greater than the indicator threshold, the log anomaly analysis and identification device generates a log anomaly identification result based on the anomaly determination rules. If the log anomaly impact value is not greater than the indicator threshold, the first log data is determined to be normal.

[0144] This embodiment, through the above-described scheme, specifically generates a log anomaly impact value based on the log anomaly range value and the log anomaly severity value; determines whether the log anomaly impact value is greater than the indicator threshold; if the log anomaly impact value is greater than the indicator threshold, then generates the log anomaly identification result based on the anomaly judgment rule. By calculating the log anomaly range value and the log anomaly severity value, the log anomaly identification result is obtained. Thus, log anomaly identification is achieved, solving the technical problem of poor accuracy in monitoring the transaction system. Compared to existing technologies, it has the advantage of higher accuracy.

[0145] Reference Figure 8 , Figure 8 This is a flowchart illustrating a seventh exemplary embodiment of the log anomaly identification method of the present invention.

[0146] Based on the above Figure 2 In the embodiment shown, after obtaining the log anomaly identification result based on the log full-text index according to the preset indicator threshold and anomaly judgment rules in step S103, the method further includes:

[0147] Step S104: Based on the log anomaly identification result, send an anomaly prompt message to the terminal;

[0148] Step S105: Receive the anomaly identification result feedback sent by the terminal;

[0149] Step S106: Based on the feedback of the anomaly identification results, the indicator thresholds and anomaly determination rules are corrected to obtain key target thresholds and key anomaly determination rules.

[0150] Specifically, after obtaining the log anomaly identification results, the log anomaly analysis and identification device sends anomaly alert information to the terminal, and adjusts the indicator thresholds and anomaly judgment rules based on the received anomaly identification results. This is achieved through the following steps:

[0151] First, the log anomaly analysis and identification device integrates and records the related data based on the log anomaly identification results, and sends anomaly prompt information to the terminal through various communication methods;

[0152] Then, the log anomaly analysis and identification device receives the anomaly identification result feedback sent by the terminal. The user determines whether the anomaly prompt information is a system problem or business data problem that needs to be handled. The user calls back the relevant interface of this log anomaly analysis and identification device through the feedback interface in the anomaly prompt information.

[0153] Finally, the log anomaly analysis and identification device adjusts the indicator thresholds or adds additional anomaly judgment rules based on the feedback of anomaly identification results, and accumulates key target thresholds and key anomaly judgment rules through user feedback of anomaly identification results.

[0154] This embodiment, through the above-described scheme, specifically involves sending anomaly alerts to the terminal based on the log anomaly identification results; receiving anomaly identification result feedback from the terminal; and revising the indicator thresholds and anomaly judgment rules based on the anomaly identification result feedback to obtain key target thresholds and key anomaly judgment rules. By revising the indicator thresholds and anomaly judgment rules based on the anomaly identification result feedback from the terminal, a closed-loop process for log anomaly identification is completed, achieving positive feedback in log anomaly identification and improving the accuracy of anomaly identification.

[0155] As another exemplary embodiment, this embodiment can achieve log anomaly identification through a log anomaly analysis and identification device. The flow of the log anomaly identification method is as follows: Figure 9 As shown:

[0156] Taking a log anomaly analysis and identification device as an example, the log data to be identified is log data, the full-text index of the log is a full-text index, and the anomaly prompt information is an anomaly reminder.

[0157] First, the log anomaly analysis and identification device receives log data collected and sent by the transaction system through a message middleware;

[0158] Secondly, the log anomaly analysis and identification device preprocesses the log data, tags it with product and scenario identifiers, and adopts a hierarchical storage strategy to distinguish the priority of abnormal data and obtain the first log data.

[0159] Then, the log anomaly analysis and identification device stores the first log data into the full-text index analysis engine;

[0160] Then, the log anomaly analysis and identification device uses a full-text index analysis engine to build a full-text index based on important fields in the first log data;

[0161] Then, the log anomaly analysis and identification device uses streaming detection or timed polling to monitor all log data, analyze the data, and identify anomalies.

[0162] Then, the log anomaly analysis and identification device vertically associates the upstream and downstream links of the data with the error stack through the link ID in the first log data, analyzes the scope of the anomaly impact, and obtains the log anomaly range value.

[0163] Then, the log anomaly analysis and identification device horizontally assesses the degree of anomaly impact by considering the interface name, interface performance, and the number of interface requests per unit time, and obtains a log anomaly degree value.

[0164] Then, the log anomaly analysis and identification device calculates the log anomaly impact value based on the log anomaly range value and the log anomaly severity value.

[0165] Then, the log anomaly analysis and identification device determines whether the log anomaly impact value is greater than the indicator threshold;

[0166] Then, if the log anomaly impact value is greater than the indicator threshold, the log anomaly analysis and identification device directly records the anomaly according to the rules and generates a log anomaly identification result. If the log anomaly impact value is not greater than the indicator threshold, the first log data is determined to be normal.

[0167] Then, the log anomaly analysis and identification device integrates and records the related data based on the log anomaly identification results, and sends anomaly alerts to relevant users through various communication methods;

[0168] Then, the user determines whether the abnormal alert is a system problem or business data problem that needs to be addressed. If the user determines that the abnormal alert is not a system problem or business data problem that needs to be addressed, the user calls back the relevant interface of this log abnormality analysis and identification device through the feedback interface in the abnormal alert information to correct the abnormality identification results, indicator thresholds and rules.

[0169] Finally, the log anomaly analysis and identification device accumulates key judgment thresholds and rules through user feedback, thereby improving the monitoring accuracy for specific scenarios and products.

[0170] This embodiment, through the above-described scheme, specifically involves receiving log data to be identified; establishing a full-text index of the log data; and obtaining log anomaly identification results based on preset indicator thresholds and anomaly judgment rules, according to the full-text index. By using preset indicator thresholds and anomaly judgment rules, log anomaly identification results are obtained. Thus, log anomaly identification is achieved, solving the technical problem of poor accuracy in monitoring transaction systems. Compared to existing technologies, it has the advantage of higher accuracy.

[0171] Furthermore, the present invention also provides a log anomaly identification device, the log anomaly identification device comprising:

[0172] The receiving module is used to receive log data to be identified;

[0173] The index building module is used to build a full-text index of the logs based on the log data to be identified;

[0174] The anomaly identification module is used to obtain log anomaly identification results based on preset indicator thresholds and anomaly judgment rules, according to the full-text index of the log.

[0175] The principle behind log anomaly identification in this embodiment is explained in the above embodiments and will not be repeated here.

[0176] Furthermore, this embodiment of the invention also proposes a terminal device, which includes a memory, a processor, and a log anomaly identification program stored in the memory and executable on the processor. When the log anomaly identification program is executed by the processor, it implements the steps of the log anomaly identification method as described above.

[0177] Since the log anomaly identification program employs all the technical solutions of all the aforementioned embodiments when executed by the processor, it possesses at least all the beneficial effects brought about by all the technical solutions of all the aforementioned embodiments, which will not be elaborated upon here.

[0178] This invention also proposes a computer-readable storage medium storing a log anomaly identification program, which, when executed by a processor, implements the steps of the log anomaly identification method described above.

[0179] Since this log anomaly identification program employs all the technical solutions of all the aforementioned embodiments when executed by the processor, it possesses at least all the beneficial effects brought about by all the technical solutions of all the aforementioned embodiments, which will not be elaborated upon here.

[0180] Compared to existing technologies, the log anomaly identification scheme proposed in this invention receives log data to be identified; establishes a full-text index based on the log data; and obtains the log anomaly identification result based on the full-text index according to preset indicator thresholds and anomaly judgment rules. This solves the problem of inaccurate identification of log anomalies in trading systems. Based on this invention, starting from the real-world application problem of inaccurate identification of log anomalies in trading systems, a log anomaly identification scheme is designed, and the effectiveness of the log anomaly identification method of this invention is verified on the anomaly identification of log data to be identified. Finally, the accuracy of log anomaly identification using the method of this invention is significantly improved.

[0181] Compared with existing technologies, the solutions of the embodiments of the present invention have the following advantages:

[0182] 1. Multi-dimensional data associations were established.

[0183] 2. The accuracy of anomaly identification in logs is higher.

[0184] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0185] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0186] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the methods of each embodiment of the present invention.

[0187] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for identifying log anomalies, characterized in that, The log anomaly identification method includes the following steps: Receive log data to be identified; Based on the log data to be identified, a full-text index of the logs is established; Based on preset indicator thresholds and anomaly judgment rules, log anomaly identification results are obtained according to the full-text log index. The step of obtaining log anomaly identification results based on preset indicator thresholds and anomaly judgment rules, and according to the full-text log index, includes: Based on the full-text log index, the link-related data in the first log data is analyzed to analyze the scope of the anomaly and obtain the log anomaly range value. The link-related data includes the upstream and downstream links of the data associated with the link ID and the error stack. Based on the full-text index of the log, the interface-related data in the first log data is evaluated to assess the degree of impact of the anomaly and obtain the log anomaly degree value. The interface-related data includes the interface name, interface performance, and the number of interface requests per unit time. Based on the log anomaly range value and the log anomaly severity value, generate the log anomaly impact value; Determine whether the log anomaly impact value is greater than the indicator threshold; If the log anomaly impact value is greater than the indicator threshold, then the log anomaly identification result is generated based on the anomaly determination rule.

2. The log anomaly identification method as described in claim 1, characterized in that, The step of establishing a full-text index of the logs based on the log data to be identified includes: The log data to be identified is processed to obtain the first log data; Store the first log data in the full-text index analysis engine; The full-text indexing analysis engine is used to create a full-text index of the log data based on preset important fields in the first log data.

3. The log anomaly identification method as described in claim 2, characterized in that, The step of processing the log data to be identified to obtain the first log data includes: The log data to be identified is preprocessed to obtain preprocessed log data to be identified; According to preset annotation conditions, the preprocessed log data to be identified is annotated to obtain annotated log data to be identified; The labeled log data to be identified is classified according to the preset data priority classification conditions to obtain the first log data.

4. The log anomaly identification method as described in claim 2, characterized in that, The step of storing the first log data into the full-text indexing and analysis engine includes: Based on preset hierarchical storage conditions, the first log data is stored hierarchically using the full-text indexing and analysis engine.

5. The log anomaly identification method as described in any one of claims 1-4, characterized in that, The step of obtaining log anomaly identification results based on the preset indicator thresholds and anomaly judgment rules, according to the full-text log index, further includes: Based on the log anomaly identification results, send an anomaly alert message to the terminal; Receive the anomaly identification result feedback sent by the terminal; Based on the feedback of the anomaly identification results, the indicator thresholds and anomaly judgment rules are corrected to obtain key target thresholds and key anomaly judgment rules.

6. A log anomaly identification device, characterized in that, The log anomaly identification device includes: The receiving module is used to receive log data to be identified; The index building module is used to build a full-text index of the logs based on the log data to be identified; The anomaly detection module is used to obtain log anomaly detection results based on the preset indicator thresholds and anomaly judgment rules, according to the full-text log index. The anomaly detection module is also used for: Based on the full-text log index, the link-related data in the first log data is analyzed to analyze the scope of the anomaly and obtain the log anomaly range value. The link-related data includes the upstream and downstream links of the data associated with the link ID and the error stack. Based on the full-text index of the log, the interface-related data in the first log data is evaluated to assess the degree of impact of the anomaly and obtain the log anomaly degree value. The interface-related data includes the interface name, interface performance, and the number of interface requests per unit time. Based on the log anomaly range value and the log anomaly severity value, generate the log anomaly impact value; Determine whether the log anomaly impact value is greater than the indicator threshold; If the log anomaly impact value is greater than the indicator threshold, then the log anomaly identification result is generated based on the anomaly determination rule.

7. A terminal device, characterized in that, The terminal device includes a memory, a processor, and a log anomaly identification program stored in the memory and executable on the processor. When the log anomaly identification program is executed by the processor, it implements the steps of the log anomaly identification method as described in any one of claims 1-5.

8. A calculator-readable storage medium, characterized in that, The computer-readable storage medium stores a log anomaly identification program, which, when executed by a processor, implements the steps of the log anomaly identification method as described in any one of claims 1-5.

Images