Abnormal log identification method and device, electronic equipment and storage medium

Abstract

The application provides an abnormal log identification method and device, electronic equipment and a storage medium. The method comprises the following steps: obtaining log data corresponding to a to-be-identified service; preprocessing the log data according to a pre-stored preprocessing rule to obtain structured log data; inputting the structured log data into a pre-trained abnormal log identification model for identification to obtain an abnormal log identification result; and sending the abnormal log identification result to a terminal device for display. The application improves the objectivity and accuracy of abnormal log determination, and also improves the efficiency of abnormal log determination, thereby ensuring the normal operation of the service.

Description

Technical Field

[0001] This application relates to the field of big data technology, and in particular to an abnormal log identification method, device, electronic device and storage medium. Background Technology

[0002] As the application scenarios of big data clusters continue to expand, more and more business scenarios have deployed big data clusters. During the operation of a big data cluster, operational anomalies may occur. Therefore, anomaly logs can be obtained from the logs generated during the operation of the big data cluster, and then the anomaly information can be located based on the anomaly logs.

[0003] In existing technologies, log collection typically involves a log processing terminal sending a log collection request to a log server. The log server then searches for the logs to be collected based on the content of the request and sends the collected log files to the log processing terminal. After receiving the logs, the log processing terminal can display them in the order they were received and then manually identify any abnormal logs.

[0004] However, manually identifying abnormal logs requires operations and maintenance personnel to review each log entry and then filter out the abnormal ones. This relies too heavily on the experience of operations and maintenance personnel, making the identification of abnormal logs highly subjective, reducing the accuracy of the identification process, increasing manpower consumption, reducing the efficiency of abnormal log identification, and ultimately affecting the normal operation of the business. Summary of the Invention

[0005] This application provides an anomaly log identification method, apparatus, electronic device, and storage medium to improve the efficiency and accuracy of anomaly log identification.

[0006] Firstly, this application provides a method for identifying abnormal logs, including:

[0007] Obtain the log data corresponding to the business to be identified;

[0008] The log data is preprocessed according to the pre-stored preprocessing rules to obtain structured log data;

[0009] The structured log data is input into a pre-trained abnormal log recognition model for recognition, and the abnormal log recognition result is obtained.

[0010] The abnormal log identification results are sent to the terminal device for display.

[0011] Optionally, after inputting the structured log data into a pre-trained anomaly log recognition model for recognition and obtaining the anomaly log recognition result, the method further includes:

[0012] The abnormal log identification results are input into a pre-trained emotion recognition model for identification, and the emotion recognition results are obtained.

[0013] Optionally, the log data is audit log data and / or operation log data, and the step of inputting the abnormal log identification result into a pre-trained emotion recognition model for identification to obtain the emotion recognition result includes:

[0014] If the abnormal log identification result is that the number of abnormal log data exceeds a preset threshold, the abnormal log identification result is input into a pre-trained emotion recognition model for identification, so as to obtain the operator emotion abnormality identification result corresponding to the audit log data and / or the operation log data.

[0015] If the abnormal log identification result is that the number of abnormal log data does not exceed the preset number threshold, then the abnormal log identification result is input into the pre-trained emotion recognition model for identification, to obtain the operator's normal emotion identification result corresponding to the audit log data and / or the operation log data.

[0016] Optionally, the step of preprocessing the log data according to pre-stored preprocessing rules to obtain structured log data includes:

[0017] Based on pre-stored preprocessing rules, different types of log data are converted into structured log data containing table identifiers.

[0018] Optionally, before obtaining the log data corresponding to the service to be identified, the method further includes:

[0019] Obtain training log data;

[0020] The training log data is preprocessed according to the preprocessing rules to obtain structured training log data;

[0021] The structured training log data is input into the first network model for training to obtain the initial abnormal log recognition model;

[0022] Acquire test log data and input the test log data into the initial anomaly log recognition model for recognition to obtain the test anomaly log recognition result;

[0023] Determine whether the accuracy rate of abnormal log identification in the test abnormal log identification results exceeds a preset accuracy threshold;

[0024] If the accuracy of the abnormal log identification exceeds the accuracy threshold, then the initial abnormal log identification model is determined as the abnormal log identification model.

[0025] Optional, also includes:

[0026] If the accuracy of the abnormal log identification does not exceed the accuracy threshold, then the parameters of the initial abnormal log identification model are adjusted.

[0027] The training log data is input into the adjusted initial anomaly log recognition model for training until the accuracy of the adjusted initial anomaly log recognition model exceeds the accuracy threshold.

[0028] Optionally, after preprocessing the training log data according to the preprocessing rules to obtain structured training log data, the method further includes:

[0029] Sentiment annotation is performed on the structured training log data based on the operation time, connection address, connection user, operation type, operation content, and operation frequency corresponding to the training log data, to obtain annotated structured training log data;

[0030] The labeled structured training log data is input into the second network model for training to obtain the emotion recognition model.

[0031] Secondly, this application provides an anomaly log identification device, comprising:

[0032] The acquisition module is used to acquire log data corresponding to the business to be identified;

[0033] The processing module is used to preprocess the log data according to the pre-stored preprocessing rules to obtain structured log data;

[0034] The processing module is also used to input the structured log data into a pre-trained abnormal log recognition model for recognition, and obtain abnormal log recognition results;

[0035] The processing module is also used to send the abnormal log identification results to the terminal device for display.

[0036] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0037] The memory stores computer-executed instructions;

[0038] The processor executes computer execution instructions stored in the memory to implement the abnormal log identification method as described in any of the first aspects.

[0039] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the abnormal log identification method as described in any of the first aspects.

[0040] This application provides an abnormal log identification method, apparatus, electronic device, and storage medium. Using the above scheme, log data corresponding to the business to be identified can be obtained first. Then, the log data is preprocessed according to pre-stored preprocessing rules to obtain structured log data. After obtaining the structured log data, it can be input into a pre-trained abnormal log identification model for identification to obtain the abnormal log identification result. The abnormal log identification result is then sent to a terminal device for display. By automatically identifying log data based on a pre-trained abnormal log identification model, the objectivity and accuracy of abnormal log identification are improved, as is the efficiency of abnormal log identification, thereby ensuring the normal operation of the business. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A schematic diagram of the application system architecture for the abnormal log identification method provided in the embodiments of this application;

[0043] Figure 2 A flowchart illustrating the abnormal log identification method provided in this application embodiment;

[0044] Figure 3 This is an application diagram illustrating the model training process provided in the embodiments of this application;

[0045] Figure 4 A flowchart illustrating an anomaly log identification method provided in another embodiment of this application;

[0046] Figure 5 This is a schematic diagram of the structure of the abnormal log identification device provided in the embodiments of this application;

[0047] Figure 6 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0048] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0049] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can also include other sequential examples besides those illustrated or described. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0050] Existing big data clusters and Kubernetes fault monitoring functions all rely on log collection. Log collection typically involves using open-source tools to simply gather logs together. For example, a log processing terminal sends a log collection request to a log server. The log server then searches for the required logs based on the request and sends the collected log files back to the log processing terminal. After receiving the logs, the terminal displays them in the order they were received, and then manually identifies any abnormal logs. However, manually identifying abnormal logs requires operations personnel to sift through the collected logs one by one. It's possible that the same log entry might be classified as abnormal by one operator while another classifies it as normal. This over-reliance on operator experience leads to high subjectivity in abnormal log identification, reducing accuracy and increasing manpower consumption, thus impacting normal business operations.

[0051] Based on the aforementioned technical issues, this application achieves the technical effect of automatically identifying log data and obtaining abnormal log data by using a pre-trained abnormal log identification model. This improves both the objectivity and accuracy of abnormal log identification, as well as the efficiency of abnormal log identification, thereby ensuring the normal operation of business.

[0052] Figure 1 A schematic diagram of the application system architecture for the abnormal log recognition method provided in the embodiments of this application is shown below. Figure 1As shown, the application system may include a server 101, a terminal device 102, and a database 103. The database 103 may store log data for different services. The server 101 can obtain the log data corresponding to the service to be processed from the database 103, and then further process the log data corresponding to the service to be processed to finally obtain the abnormal log identification result, and send the abnormal log identification result to the terminal device 102 for display.

[0053] In addition, the log data corresponding to the pending business can also be generated in real time during the operation of the pending business.

[0054] Among them, the terminal device 102 can be a smartphone, tablet, personal computer, wearable smart device or monitor, or other device that can display the abnormal log identification results.

[0055] The technical solutions of this application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0056] Figure 2 This is a flowchart illustrating the abnormal log identification method provided in an embodiment of this application. The method in this embodiment can be executed by server 101. Figure 2 As shown, the method in this embodiment may include:

[0057] S201: Obtain the log data corresponding to the business to be identified.

[0058] In this embodiment, the logs of the big data peripheral components exhibit certain standardization and correlation. Each log line is generated from the output statements of the source code. During the execution of the big data component program, logs printed by a single line of source code at the underlying level are of the same type, and this code is called the log key. A log entry often depends on the previous one or several log entries; adjacent or nearby logs have a high degree of correlation. When this logical consistency is broken by a particular log entry, it indicates that log execution has encountered an anomaly. Therefore, log data can be used to determine whether there are any anomalies in business operations or device operation.

[0059] The log data can include audit log data, operation log data, etc.

[0060] S202: Preprocess the log data according to the pre-stored preprocessing rules to obtain structured log data.

[0061] In this embodiment, the log data may contain parameters of different dimensions. For example, the log data may contain time information and level information. In order to improve the accuracy of identification, the log data can be preprocessed according to the pre-stored preprocessing rules to obtain structured log data.

[0062] Furthermore, the log data is preprocessed according to pre-stored preprocessing rules to obtain structured log data, which may specifically include:

[0063] Based on pre-stored preprocessing rules, different types of log data are converted into structured log data containing table identifiers.

[0064] Specifically, log data can be uniformly represented in the form of data tables. Therefore, structured log data can be log data containing table identifiers, which are used to represent specific parameters in the log data.

[0065] In addition, log data preprocessing can be achieved using existing methods. For example, log data preprocessing can be implemented using LogParser.

[0066] S203: Input the structured log data into the pre-trained abnormal log recognition model for recognition, and obtain the abnormal log recognition result.

[0067] In this embodiment, after obtaining the structured log data, the structured log data can be input into a pre-trained abnormal log recognition model to identify the abnormal logs and obtain the abnormal log recognition results.

[0068] The abnormal log identification results may include abnormal log data, and the number of abnormal log data can be 0 (i.e., no abnormal log data) or 1 or more.

[0069] S204: Send the abnormal log identification results to the terminal device for display.

[0070] In this embodiment, after obtaining the abnormal log identification results, in order to facilitate the operation and maintenance personnel to understand the abnormal log identification situation in a timely manner, the abnormal log identification results can be sent to the terminal device for display.

[0071] By adopting the above scheme, the log data corresponding to the business to be identified can be obtained first. Then, the log data can be preprocessed according to the pre-stored preprocessing rules to obtain structured log data. After obtaining the structured log data, the structured log data can be input into a pre-trained abnormal log recognition model for recognition to obtain the abnormal log recognition result. Then, the abnormal log recognition result is sent to the terminal device for display. By automatically identifying log data based on the pre-trained abnormal log recognition model, the objectivity and accuracy of abnormal log determination are improved, as well as the efficiency of abnormal log determination, thereby ensuring the normal operation of the business.

[0072] based on Figure 2In addition to the method described herein, this specification also provides some specific implementation schemes of the method, which will be described below.

[0073] Furthermore, in another embodiment, after inputting structured log data into a pre-trained anomaly log recognition model for recognition and obtaining anomaly log recognition results, the method may further include:

[0074] The abnormal log identification results are input into a pre-trained emotion recognition model for identification, and the emotion recognition results are obtained.

[0075] In this embodiment, during cluster management, most issues require operations and maintenance (O&M) personnel to operate and handle the cluster. These O&M personnel typically possess special permissions, such as cluster management and data operation permissions. Therefore, the actions and operations of O&M personnel significantly impact the cluster and the services running within it. Simple permission management cannot prevent intentional or unintentional actions, while simply establishing procedures is overly cumbersome. Therefore, intelligent judgment is needed to determine which operations require interception and alerts. Furthermore, some O&M personnel's actions may sometimes be related to personality traits, so their personality and behavior also need to be considered.

[0076] Therefore, after obtaining the abnormal log identification results, the abnormal log identification results can be input into a pre-trained emotion recognition model for identification to obtain the emotion recognition results.

[0077] Furthermore, the log data includes audit log data and / or operation log data. The step of inputting the abnormal log identification results into a pre-trained emotion recognition model for identification to obtain emotion recognition results may specifically include:

[0078] If the abnormal log identification result is that the number of abnormal log data exceeds a preset threshold, the abnormal log identification result is input into a pre-trained emotion recognition model for identification, so as to obtain the operator emotion abnormal identification result corresponding to the audit log data and / or the operation log data.

[0079] If the abnormal log identification result is that the number of abnormal log data does not exceed the preset number threshold, the abnormal log identification result is input into the pre-trained emotion recognition model for identification, and the operator's emotion recognition result corresponding to the audit log data and / or operation log data is obtained.

[0080] Specifically, the preset threshold number can be customized according to the actual application scenario. For example, the preset threshold number can be zero. That is, if there are abnormal logs, the operator's emotional abnormality identification results corresponding to the audit log data and / or operation log data can be obtained. The operator needs to be paid extra attention to prevent the operator from performing certain incorrect operations that affect the normal operation of the business.

[0081] Furthermore, in another embodiment, before obtaining the log data corresponding to the service to be identified, the method may further include:

[0082] Obtain training log data.

[0083] The training log data is preprocessed according to the preprocessing rules to obtain structured training log data.

[0084] The structured training log data is input into the first network model for training to obtain the initial abnormal log recognition model.

[0085] Acquire test log data and input the test log data into the initial anomaly log recognition model for recognition to obtain the test anomaly log recognition result.

[0086] Determine whether the accuracy rate of abnormal log identification in the test abnormal log identification results exceeds a preset accuracy threshold.

[0087] If the accuracy of the abnormal log identification exceeds the accuracy threshold, then the initial abnormal log identification model is determined as the abnormal log identification model.

[0088] In this embodiment, the first network model can be an LSTM (Long Short-Term Memory) model. Training based on structured training log data can specifically include: semantic recognition, which involves numbering and classifying all possible log displays contained in the current log, then encoding them into template vectors. A structured log dataset is generated by extracting template vectors and converting template sequences into semantic vector sequences. Then, template vectors are designed and substituted into the training set using a sliding window sampling method to generate sequence vectors, counting vectors, and semantic vectors. The desired combination of feature vectors is then selected and substituted into the LSTM to train the initial abnormal log recognition model. The initial abnormal log recognition model is then tested using a test set. The model's output is sorted according to probability values, and the g log templates with the highest probability values ​​are selected. If a newly generated log template m is among these g log templates, m is considered a normal log; otherwise, m is considered an abnormal log. The accuracy of the initial abnormal log recognition model is determined based on the test results. If the accuracy of abnormal log recognition exceeds an accuracy threshold, the initial abnormal log recognition model is confirmed as an abnormal log recognition model.

[0089] The accuracy threshold can be customized according to the actual application scenario; specifically, it can be any value between 85% and 95%.

[0090] In addition, the method may also include:

[0091] If the accuracy of the abnormal log identification does not exceed the accuracy threshold, the parameters of the initial abnormal log identification model are adjusted.

[0092] The training log data is input into the adjusted initial anomaly log recognition model for training until the accuracy of the adjusted initial anomaly log recognition model exceeds the accuracy threshold.

[0093] Furthermore, in another embodiment, after preprocessing the training log data according to the preprocessing rules to obtain structured training log data, the method may further include:

[0094] Sentiment annotation is performed on the structured training log data based on the operation time, connection address, connection user, operation type, operation content, and operation frequency corresponding to the training log data, resulting in annotated structured training log data.

[0095] The labeled structured training log data is input into the second network model for training to obtain the emotion recognition model.

[0096] In this embodiment, a sentiment recognition model can be constructed by training existing log data with sentiment annotation using a second network model. Then, the abnormal logs obtained above are substituted into the model to determine the user's current emotion, or to infer the operator's personality. This allows the model to determine whether the operator's personality is suitable for the position, or whether the operator is currently experiencing a volatile, impatient, irritable, or error-prone emotional state, thus reminding the operator to manage their emotions. The second network model can be a BERT (Bidirectional Encoder Representations from Transformer) model.

[0097] For example, Figure 3 This is an application diagram illustrating the model training process provided in the embodiments of this application, such as... Figure 3 As shown, in this embodiment, log data, including audit logs and operation logs, can be acquired and stored first. After storing a certain amount of log data, it can prepare for subsequent training and recognition based on classification. Specifically, the log data can be parsed and transformed into a structured log dataset, and the BERT model can be used for personality and sentiment annotation. This involves annotating the existing operation logs and audit logs with personality and sentiment data, considering factors such as operation time, connection address, connected user, operation type, operation content, and operation frequency to reflect the operator's emotional state. The log recognition problem is essentially a multi-classification problem, which can be transformed into an algorithm modeling problem. Model generation requires a large amount of data support; therefore, the logs need to be parsed and transformed into a structured log dataset. Then, the BERT and LSTM models are trained based on the structured log dataset to obtain an abnormal log recognition model and a sentiment recognition model.

[0098] also, Figure 4 A flowchart illustrating an anomaly log identification method provided in another embodiment of this application is shown below. Figure 4 As shown in this embodiment, log data can be collected first, and then an abnormal log recognition model and an emotion recognition model can be trained on the log data. The two models can be applied respectively to obtain the emotion output of the operation and maintenance personnel.

[0099] Based on the same idea, this specification also provides an apparatus corresponding to the above method. Figure 5 This is a schematic diagram of the structure of the abnormal log identification device provided in the embodiments of this application, as shown below. Figure 5 As shown, the apparatus provided in this embodiment may include:

[0100] The acquisition module 501 is used to acquire the log data corresponding to the business to be identified.

[0101] The processing module 502 is used to preprocess the log data according to the pre-stored preprocessing rules to obtain structured log data.

[0102] In this embodiment, the processing module 502 is further configured to:

[0103] Based on pre-stored preprocessing rules, different types of log data are converted into structured log data containing table identifiers.

[0104] The processing module 502 is further configured to input the structured log data into a pre-trained abnormal log recognition model for recognition, and obtain abnormal log recognition results.

[0105] The processing module 502 is also used to send the abnormal log identification result to the terminal device for display.

[0106] Furthermore, in another embodiment, the processing module 502 is also used for:

[0107] The abnormal log identification results are input into a pre-trained emotion recognition model for identification, and the emotion recognition results are obtained.

[0108] In this embodiment, the log data is audit log data and / or operation log data, and the processing module 502 is further configured to:

[0109] If the abnormal log identification result is that the number of abnormal log data exceeds a preset threshold, the abnormal log identification result is input into a pre-trained emotion recognition model for identification, so as to obtain the operator emotion abnormality identification result corresponding to the audit log data and / or operation log data.

[0110] If the abnormal log identification result is that the number of abnormal log data does not exceed the preset number threshold, then the abnormal log identification result is input into the pre-trained emotion recognition model for identification, to obtain the operator's normal emotion identification result corresponding to the audit log data and / or the operation log data.

[0111] Furthermore, in another embodiment, the processing module 502 is also used for:

[0112] Obtain training log data.

[0113] The training log data is preprocessed according to the preprocessing rules to obtain structured training log data.

[0114] The structured training log data is input into the first network model for training to obtain the initial abnormal log recognition model.

[0115] Acquire test log data and input the test log data into the initial anomaly log recognition model for recognition to obtain the test anomaly log recognition result.

[0116] Determine whether the accuracy rate of abnormal log identification in the test abnormal log identification results exceeds a preset accuracy threshold.

[0117] If the accuracy of the abnormal log identification exceeds the accuracy threshold, then the initial abnormal log identification model is determined as the abnormal log identification model.

[0118] In addition, the processing module 502 is also used for:

[0119] If the accuracy of the abnormal log identification does not exceed the accuracy threshold, the parameters of the initial abnormal log identification model are adjusted.

[0120] The training log data is input into the adjusted initial anomaly log recognition model for training until the accuracy of the adjusted initial anomaly log recognition model exceeds the accuracy threshold.

[0121] In addition, the processing module 502 is also used for:

[0122] Sentiment annotation is performed on the structured training log data based on the operation time, connection address, connection user, operation type, operation content, and operation frequency corresponding to the training log data, resulting in annotated structured training log data.

[0123] The labeled structured training log data is input into the second network model for training to obtain the emotion recognition model.

[0124] The apparatus provided in this application embodiment can achieve the above-mentioned... Figure 2 The methods in the embodiments shown are similar in principle and technical effect, and will not be described again here.

[0125] Figure 6 A schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application, such as... Figure 6 As shown, the device 600 provided in this embodiment includes a processor 601 and a memory communicatively connected to the processor. The processor 601 and the memory 602 are connected via a bus 603.

[0126] In the specific implementation process, the processor 601 executes the computer execution instructions stored in the memory 602, causing the processor 601 to execute the abnormal log identification method in the above method embodiment.

[0127] The specific implementation process of processor 601 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0128] In the above Figure 6 In the illustrated embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0129] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage.

[0130] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0131] This application also provides a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the abnormal log identification method of the above-described method embodiments.

[0132] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the abnormal log identification method described above.

[0133] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0134] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0135] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An anomaly log identification method, characterized in that, include: Obtain the log data corresponding to the business to be identified. The log data includes audit log data and operation log data. The log data is preprocessed according to the pre-stored preprocessing rules to obtain structured log data; The structured log data is input into a pre-trained abnormal log recognition model for recognition, and the abnormal log recognition result is obtained. The abnormal log identification results are sent to the terminal device for display. If the abnormal log identification result is that the number of abnormal log data exceeds a preset threshold, the abnormal log identification result is input into a pre-trained emotion recognition model for identification, and the operator emotion abnormality identification result corresponding to the audit log data and the operation log data is obtained. The emotion recognition model can be obtained using the BERT model. If the abnormal log identification result is that the number of abnormal log data does not exceed the preset number threshold, then the abnormal log identification result is input into the pre-trained emotion recognition model for identification, and the operator's emotion recognition result corresponding to the audit log data and the operation log data is obtained.

2. The method according to claim 1, characterized in that, The step of preprocessing the log data according to pre-stored preprocessing rules to obtain structured log data includes: Based on pre-stored preprocessing rules, different types of log data are converted into structured log data containing table identifiers.

3. The method according to claim 1, characterized in that, Before obtaining the log data corresponding to the service to be identified, the following steps are also included: Obtain training log data; The training log data is preprocessed according to the preprocessing rules to obtain structured training log data; The structured training log data is input into the first network model for training to obtain the initial abnormal log recognition model; Acquire test log data and input the test log data into the initial anomaly log recognition model for recognition to obtain the test anomaly log recognition result; Determine whether the accuracy rate of abnormal log identification in the test abnormal log identification results exceeds a preset accuracy threshold; If the accuracy of the abnormal log identification exceeds the accuracy threshold, then the initial abnormal log identification model is determined as the abnormal log identification model.

4. The method according to claim 3, characterized in that, Also includes: If the accuracy of the abnormal log identification does not exceed the accuracy threshold, then the parameters of the initial abnormal log identification model are adjusted. The training log data is input into the adjusted initial anomaly log recognition model for training until the accuracy of the adjusted initial anomaly log recognition model exceeds the accuracy threshold.

5. The method according to claim 3, characterized in that, After preprocessing the training log data according to the preprocessing rules to obtain structured training log data, the method further includes: Sentiment annotation is performed on the structured training log data based on the operation time, connection address, connection user, operation type, operation content, and operation frequency corresponding to the training log data, to obtain annotated structured training log data; The labeled structured training log data is input into the second network model for training to obtain the emotion recognition model.

6. An anomaly log identification device, characterized in that, include: The acquisition module is used to acquire the log data corresponding to the business to be identified. The log data includes audit log data and operation log data. The processing module is used to preprocess the log data according to the pre-stored preprocessing rules to obtain structured log data; The processing module is also used to input the structured log data into a pre-trained abnormal log recognition model for recognition, and obtain abnormal log recognition results; The processing module is also used to send the abnormal log identification result to the terminal device for display; The processing module is further configured to input the abnormal log identification result into a pre-trained emotion recognition model for identification if the number of abnormal log data exceeds a preset threshold, so as to obtain the operator emotion abnormality identification result corresponding to the audit log data and / or the operation log data. The emotion recognition model can be obtained using the BERT model. If the abnormal log identification result is that the number of abnormal log data does not exceed the preset number threshold, then the abnormal log identification result is input into the pre-trained emotion recognition model for identification, to obtain the operator's normal emotion identification result corresponding to the audit log data and / or the operation log data.

7. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the abnormal log identification method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by the processor, implement the abnormal log identification method as described in any one of claims 1 to 5.

Images