A method, device, computer equipment, and intelligent system for detecting operation and maintenance logs.

Abstract

This invention provides a method, apparatus, computer device, and intelligent system for detecting operation and maintenance logs. The method includes acquiring log data to be tested; determining whether the log data is structured data; if the log data is not structured data, inputting the log data into a pre-stored detection model; and performing anomaly detection on the log data based on the detection model to obtain a first anomaly detection result. The method provided by this invention can effectively cope with changes in log formats in different projects and development stages, improve the fault identification rate, efficiency, and coverage of operation and maintenance log data, and improve the accuracy and reliability of detection.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, computer equipment, and intelligent system for detecting operation and maintenance logs. Background Technology

[0002] Log data can cover operational information for various systems. Developers can use logs for anomaly detection and system monitoring, enabling timely adjustments and responses. Artificial Intelligence for ITOperation (AIOps) systems can automatically resolve various operational anomalies and problems by combining large amounts of log data with machine learning.

[0003] Intelligent operation and maintenance systems require a large amount of log data to perform automatic detection. However, in practical applications, log data is complex and variable, which cannot meet the needs of automatic detection and troubleshooting. Summary of the Invention

[0004] To address the problem that existing smart home system operation and maintenance logs are complex and diverse, making automatic detection difficult, this application provides an operation and maintenance log detection method, device, computer equipment, and intelligent system, which can improve the accuracy and reliability of automatic operation and maintenance log detection.

[0005] On the one hand, a method for detecting operation and maintenance logs is provided, the method including:

[0006] Obtain the log data to be tested and determine whether the log data to be tested is structured data;

[0007] If the log data to be tested is not structured data, then the log data to be tested is input into the pre-stored detection model;

[0008] Anomaly detection is performed on the log data to be tested based on the detection model to obtain the first anomaly detection result.

[0009] In some embodiments, obtaining the log data to be tested includes:

[0010] The log data to be tested is acquired, and source detection is performed on the log data to be tested based on a pre-stored source database to obtain a second anomaly detection result.

[0011] In some embodiments, obtaining the log data to be tested includes:

[0012] Obtain the log data to be tested and preprocess the log data to be tested;

[0013] The preprocessing includes regularization and parsing.

[0014] In some embodiments, the method further includes:

[0015] If the log to be tested is structured data, then anomaly detection is performed on the log data based on the pre-stored structured database.

[0016] In some embodiments, the detection model includes a BERT-based bidirectional LSTM-CRF model.

[0017] Obtain historical log data and train the detection model based on the historical log data.

[0018] On the other hand, an operation and maintenance log detection device is provided, the device comprising:

[0019] The test log data acquisition module is used to acquire the test log data and determine whether the test log data is structured data.

[0020] The structured data judgment module is used to input the log data to be tested into a pre-stored detection model if the log data to be tested is not structured data.

[0021] The first anomaly detection module is used to perform anomaly detection on the log data to be tested based on the detection model, and obtain the first anomaly detection result.

[0022] In some embodiments, the test log data acquisition module is specifically used for:

[0023] The log data to be tested is acquired, and source detection is performed on the log data to be tested based on a pre-stored source database to obtain a second anomaly detection result.

[0024] In some embodiments, the test log data acquisition module is specifically used for:

[0025] Obtain the log data to be tested and preprocess the log data to be tested;

[0026] The preprocessing includes regularization and parsing.

[0027] In some embodiments, the apparatus further includes a second anomaly detection module, configured to:

[0028] If the log to be tested is structured data, then anomaly detection is performed on the log data based on the pre-stored structured database.

[0029] In some embodiments, the apparatus further includes a model training module for:

[0030] Obtain historical log data and train the detection model based on the historical log data.

[0031] In some embodiments, the detection model includes a BERT-based bidirectional LSTM-CRF model.

[0032] On the other hand, a computer device is provided, which includes a processor and a memory. The memory stores at least one instruction, at least one program, code set, or instruction set. The processor can load and execute at least one instruction, at least one program, code set, or instruction set to implement the operation and maintenance log detection method provided in the above-mentioned application embodiments.

[0033] On the other hand, an intelligent system is provided, including the computer device described above.

[0034] On the other hand, a computer-readable storage medium is provided, which stores at least one instruction, at least one program, code set or instruction set, and a processor can load and execute at least one instruction, at least one program, code set or instruction set to implement the operation and maintenance log detection method provided in the embodiments of this application.

[0035] On the other hand, a computer program product or computer program is provided, which includes computer program instructions stored in a computer-readable storage medium. A processor reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the operation and maintenance log detection method described in any of the above embodiments.

[0036] The beneficial effects of the technical solution provided in this application include at least the following: Embodiments of this invention provide a method, apparatus, computer device, and intelligent system for detecting operation and maintenance logs. The method includes acquiring log data to be tested, determining whether the log data to be tested is structured data; if the log data to be tested is not structured data, inputting the log data to be tested into a pre-stored detection model; and performing anomaly detection on the log data to be tested based on the detection model to obtain a first anomaly detection result. The method provided by embodiments of this invention can effectively cope with changes in log formats in different projects and different development stages, improve the fault identification rate, efficiency, and coverage of operation and maintenance log data, and improve the accuracy and reliability of detection. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1This illustration shows a schematic diagram of the implementation process of an operation and maintenance log detection method provided in an exemplary embodiment of this application;

[0039] Figure 2 This illustration shows another implementation flow diagram of an operation and maintenance log detection method provided in an exemplary embodiment of this application;

[0040] Figure 3 This invention provides a structural diagram of an operation and maintenance log detection device according to an exemplary embodiment of the present application.

[0041] Figure 4 This illustration shows a schematic diagram of the computer device corresponding to an operation and maintenance log detection method provided in an exemplary embodiment of this application. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0043] The operation and maintenance log detection method provided in this application can improve the accuracy and reliability of automatic operation and maintenance log detection.

[0044] Example 1

[0045] Log data can cover operational information of various systems. Developers can use logs for anomaly detection and system monitoring, enabling timely adjustments and responses. Intelligent operation and maintenance systems can automatically resolve various abnormal situations and problems that occur during system operation by combining large amounts of log data with machine learning. Intelligent operation and maintenance systems require a large amount of log data to support automatic detection; however, in practical applications, log data is complex and variable, making it insufficient for automatic detection and troubleshooting.

[0046] Figure 1 The diagram illustrates the implementation flow of an operation and maintenance log detection method provided by an embodiment of the present invention.

[0047] See Figure 1 The operation and maintenance log detection method provided in this embodiment of the invention may include steps 101 to 103.

[0048] Step 101: Obtain the log data to be tested and determine whether the log data to be tested is structured data.

[0049] In some embodiments, step 101 specifically includes:

[0050] The log data to be tested is obtained, and the source of the log data to be tested is detected based on the pre-stored source database to obtain a second anomaly detection result.

[0051] In some embodiments, step 101 specifically includes:

[0052] Obtain the log data to be tested and preprocess the log data to be tested;

[0053] The preprocessing includes regularization and parsing.

[0054] In some embodiments, step 101 uses front-end regularization to clean and classify log data, thereby standardizing the complex and changing logs.

[0055] Furthermore, the method provided in this embodiment of the invention utilizes regularization to filter manually set structured log data and adds detection rules.

[0056] In some embodiments, standardization of semi-structured log text data can lay the foundation for log anomaly detection. Since log data contains information such as timestamps, time sources, log levels, IP parameters, URL parameters, and PATH parameters, a large amount of redundant content can affect subsequent model judgments; therefore, log data standardization is necessary.

[0057] The embodiments of the present invention can realize the templated nature of logs, detect abnormal situations in logs by increasing the analysis of log content, and at the same time, the standardized log structure facilitates the system to be viewed and maintained at any time.

[0058] Step 102: If the log data to be tested is not structured data, then input the log data to be tested into the pre-stored detection model.

[0059] In some embodiments, the method further includes:

[0060] If the log to be tested is structured data, then anomaly detection is performed on the log data based on the pre-stored structured database.

[0061] Step 103: Perform anomaly detection on the log data to be tested based on the detection model to obtain the first anomaly detection result.

[0062] In some embodiments, the method further includes:

[0063] Obtain historical log data and train the detection model based on the historical log data.

[0064] In some embodiments, the detection model includes a BERT-based bidirectional LSTM-CRF model.

[0065] The method provided in this embodiment of the invention uses the ERNIE pre-trained model as the semantic feature extraction model, and adds CRF to the backend to constrain the training results, which can improve the accuracy of detection.

[0066] In some embodiments, the log pattern may change when the system malfunctions. This method can automatically identify and judge faults such as changes in sequence order, changes in log time interval, and changes in events in the face of complex changes in logs, thereby enhancing the robustness of anomaly detection to cope with various complex situations.

[0067] The method provided in this embodiment of the invention performs offline data modeling and online data analysis simultaneously, and the log data detection model is continuously updated to achieve anomaly detection.

[0068] The operation and maintenance log detection method provided in this invention can effectively cope with changes in log formats in different projects and development stages. The front end uses regularization rules to clean abnormal data and updates the rules in real time to reduce training categories and increase the model's recognition rate. Faced with different types of logs, the machine learning model can re-label and retrain the data to improve fault recognition rate, efficiency, and case coverage. On the other hand, the ERINE model is used to pre-train the log data, maintaining input modeling based on word features, so that the model does not need to rely on other information during learning, and has stronger versatility and scalability.

[0069] Example 2

[0070] Figure 2 This diagram illustrates another implementation flow of the operation and maintenance log detection method provided in this embodiment of the invention.

[0071] See Figure 2 In a specific example, the implementation process of the operation and maintenance log detection method provided in this embodiment of the invention is as follows.

[0072] First, modeling is performed based on historical logs from the offline terminal.

[0073] During anomaly detection, the first step is to retrieve new logs from the online client and feed them in. This process involves two aspects: firstly, detecting the log source and performing anomaly detection based on that source; and secondly, performing regularization on the log data.

[0074] In a specific example, the log source detection process establishes different detection rules for logs from different sources.

[0075] Optionally, a log source database can be established based on the log data format. For log data in typical formats, other processing steps can be skipped and rapid anomaly detection can be performed directly.

[0076] Regularization is a general term for methods used to suppress overfitting. It reduces the complexity of the model by dynamically adjusting the values ​​of model parameters. This is because when the values ​​of some parameters are sufficiently small, the influence of the corresponding attributes on the results is negligible, which effectively eliminates the influence of irrelevant attributes.

[0077] In a specific example, log regularization includes regularizing both historical and new log data. Regularization involves initialization, log parsing, and vectorization. This step removes redundant parts of the log data, achieving cleaning and standardization. Furthermore, word segmentation rules can be established as needed to segment the log data into words.

[0078] The log data after regularization is parsed to further determine whether it is an error log. If it is determined to be an error log, an exception is thrown.

[0079] In a specific example, log parsing involves classifying logs based on the level field that appears in the logs. First, the effective words in the log data are classified, and then the log semantic information is classified to build a dataset training pattern, thus achieving dictionaryization.

[0080] If it is determined that it is not an error log, then it is determined whether it is a structured log. If it is a structured log, then log anomaly detection is performed directly.

[0081] Structured log data indicates that this portion of the log data originates from a relatively large and stable system, resulting in a more uniform log format and description method. A unified log description method reduces the vocabulary required for log anomaly detection, as out-of-vocabulary (OOV) words may appear. Therefore, anomaly detection is performed directly on the structured log data using pre-defined rules.

[0082] If the log data is determined to be unstructured, then log parsing and log pattern learning are performed sequentially.

[0083] Optionally, a data acquisition system can be used to obtain log information, and intermediate logs and business logs can be extracted, deduplicated, and manually labeled. The classification model can include, but is not limited to, binary classification algorithms, deep learning algorithms, etc.

[0084] Specifically, binary classification algorithms can be based on CART regression trees. GBDT is trained in each iteration based on the negative gradient values ​​of the model from the previous iteration. This algorithm requires that in each iteration, the difference between the true label and the output of the weak classifier is meaningful, i.e., the residual is meaningful. If the weak classifier used is a classification tree, the subtraction of categories is meaningless. This problem can be solved in two ways: one is to use an exponential loss function, which degenerates GBDT into Adaboost, thus solving the classification problem; the other is to use a log-likelihood loss function similar to logistic regression, which uses the difference between the resulting probability value and the true probability value as the residual for fitting the model.

[0085] In a specific example, the log model learning process includes encoding text sentences and intermediate time features as sentences using a BERT (Bidirectional Encoder Representation from Transformers) model, while simultaneously incorporating time features as additional features into the vector encoding. After sentence embedding, the log information is encoded as semantic and temporal information concatenated to obtain the encoded vector. Finally, based on historical logs from offline devices, a bidirectional LSTM-CRF model is used to learn log sequence patterns, completing the model training.

[0086] Specifically, the BERT model further improves upon the GPT language model by predicting the current word using the left and right contexts and predicting the next sentence using the current sentence. The pre-trained BERT representation can be fine-tuned with only one additional output layer, and can create state-of-the-art models for many tasks without making significant modifications to the task-specific architecture.

[0087] Finally, log anomaly detection is performed, and after various log anomaly detections, an anomaly result is finally thrown.

[0088] In a specific example, the log anomaly detection process involves using cleaned and parsed log data to perform anomaly detection through a trained model, and generating corresponding anomaly identifiers based on the anomaly type to complete the detection process.

[0089] Optionally, the trained model may include, but is not limited to, the BERT bidirectional LSTM-CRF model.

[0090] In summary, compared with traditional principal component analysis, invariant mining-based methods, workflow-based methods, and deep learning-based methods, the method provided in this embodiment of the invention can effectively detect abnormal data in the complex log data of real-world production environments, and significantly reduce the impact of diverse log formats on anomaly detection.

[0091] Example 3

[0092] Figure 3 A schematic diagram of the operation and maintenance log detection device provided in an embodiment of the present invention is shown.

[0093] See Figure 3 The operation and maintenance log detection device provided in this embodiment of the invention may include:

[0094] The test log data acquisition module 201 is used to acquire the test log data and determine whether the test log data is structured data;

[0095] The structured data judgment module 202 is used to input the log data to be tested into a pre-stored detection model if the log data to be tested is not structured data.

[0096] The first anomaly detection module 203 is used to perform anomaly detection on the log data to be tested based on the detection model, and obtain a first anomaly detection result.

[0097] In some embodiments, the test log data acquisition module 201 is specifically used for:

[0098] The log data to be tested is acquired, and source detection is performed on the log data to be tested based on a pre-stored source database to obtain a second anomaly detection result.

[0099] In some embodiments, the test log data acquisition module 201 is specifically used for:

[0100] Obtain the log data to be tested and preprocess the log data to be tested;

[0101] The preprocessing includes regularization and parsing.

[0102] In some embodiments, the apparatus further includes a second anomaly detection module, configured to:

[0103] If the log to be tested is structured data, then anomaly detection is performed on the log data based on the pre-stored structured database.

[0104] In some embodiments, the apparatus further includes a model training module for:

[0105] Obtain historical log data and train the detection model based on the historical log data.

[0106] In some embodiments, the detection model includes a BERT-based bidirectional LSTM-CRF model.

[0107] In summary, the apparatus provided by this invention can effectively handle changes in log formats across different projects and development stages. The front end uses regularization rules to clean abnormal data and updates the rules in real time to reduce training categories and increase the model's recognition rate. Faced with different types of logs, the machine learning model can re-label and retrain the data, improving fault identification rate, efficiency, and case coverage. Furthermore, by using the ERINE model to pre-train the log data and maintaining character-feature-based input modeling, the model does not need to rely on other information during learning, resulting in stronger versatility and scalability.

[0108] Example 4

[0109] Figure 4This application shows a schematic diagram of the structure of a computer device provided in an exemplary embodiment, the computer device comprising:

[0110] The processor 301 includes one or more processing cores. The processor 301 executes various functional applications and data processing by running software programs and modules.

[0111] The receiver 302 and transmitter 303 can be implemented as a communication component, which can be a communication chip. Optionally, this communication component can include signal transmission functionality. That is, the transmitter 303 can be used to transmit control signals to the image acquisition device and the scanning device, and the receiver 302 can be used to receive corresponding feedback commands.

[0112] The memory 304 is connected to the processor 301 via the bus 305.

[0113] The memory 304 can be used to store at least one instruction, and the processor 301 is used to execute the at least one instruction to implement the following steps in the above embodiment of the operation and maintenance log detection method.

[0114] The method includes:

[0115] Obtain the log data to be tested and determine whether the log data to be tested is structured data;

[0116] If the log data to be tested is not structured data, then the log data to be tested is input into the pre-stored detection model;

[0117] Anomaly detection is performed on the log data to be tested based on the detection model to obtain the first anomaly detection result.

[0118] In some embodiments, obtaining the log data to be tested includes:

[0119] The log data to be tested is acquired, and source detection is performed on the log data to be tested based on a pre-stored source database to obtain a second anomaly detection result.

[0120] In some embodiments, obtaining the log data to be tested includes:

[0121] Obtain the log data to be tested and preprocess the log data to be tested;

[0122] The preprocessing includes regularization and parsing.

[0123] In some embodiments, the method further includes:

[0124] If the log to be tested is structured data, then anomaly detection is performed on the log data based on the pre-stored structured database.

[0125] In some embodiments, the detection model includes a BERT-based bidirectional LSTM-CRF model.

[0126] Obtain historical log data and train the detection model based on the historical log data.

[0127] Those skilled in the art will understand that Figure 4 This is merely an example of a computer device and does not constitute a limitation on the computer device. It may include more or fewer components than shown, or combine certain components, or different components. For example, the computer device may also include network access devices, etc.

[0128] The processor 301 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0129] The memory 304 can be an internal storage unit of the computer device, such as a hard drive or RAM. The memory 304 can also be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 304 can include both internal and external storage units. The memory 304 is used to store the computer program and other programs and data required by the terminal device. The memory 304 can also be used to temporarily store data that has been output or will be output.

[0130] Example 5

[0131] This application also provides an intelligent system, including the computer device described above.

[0132] In some embodiments, the intelligent system includes, but is not limited to, a smart home system, and the operation and maintenance logs include, but are not limited to, log data generated by various smart home devices in the smart home system.

[0133] Example 6

[0134] This application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, which can be loaded and executed by a processor to implement the following steps of the above-described operation and maintenance log detection method.

[0135] The method includes:

[0136] Obtain the log data to be tested and determine whether the log data to be tested is structured data;

[0137] If the log data to be tested is not structured data, then the log data to be tested is input into the pre-stored detection model;

[0138] Anomaly detection is performed on the log data to be tested based on the detection model to obtain the first anomaly detection result.

[0139] In some embodiments, obtaining the log data to be tested includes:

[0140] The log data to be tested is acquired, and source detection is performed on the log data to be tested based on a pre-stored source database to obtain a second anomaly detection result.

[0141] In some embodiments, obtaining the log data to be tested includes:

[0142] Obtain the log data to be tested and preprocess the log data to be tested;

[0143] The preprocessing includes regularization and parsing.

[0144] In some embodiments, the method further includes:

[0145] If the log to be tested is structured data, then anomaly detection is performed on the log data based on the pre-stored structured database.

[0146] In some embodiments, the detection model includes a BERT-based bidirectional LSTM-CRF model.

[0147] Obtain historical log data and train the detection model based on the historical log data.

[0148] Optionally, the computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. The random access memory may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM).

[0149] Example 7

[0150] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the following steps of the operation and maintenance log detection method described in any of the above embodiments.

[0151] The method includes:

[0152] Obtain the log data to be tested and determine whether the log data to be tested is structured data;

[0153] If the log data to be tested is not structured data, then the log data to be tested is input into the pre-stored detection model;

[0154] Anomaly detection is performed on the log data to be tested based on the detection model to obtain the first anomaly detection result.

[0155] In some embodiments, obtaining the log data to be tested includes:

[0156] The log data to be tested is acquired, and source detection is performed on the log data to be tested based on a pre-stored source database to obtain a second anomaly detection result.

[0157] In some embodiments, obtaining the log data to be tested includes:

[0158] Obtain the log data to be tested and preprocess the log data to be tested;

[0159] The preprocessing includes regularization and parsing.

[0160] In some embodiments, the method further includes:

[0161] If the log to be tested is structured data, then anomaly detection is performed on the log data based on the pre-stored structured database.

[0162] In some embodiments, the detection model includes a BERT-based bidirectional LSTM-CRF model.

[0163] Obtain historical log data and train the detection model based on the historical log data.

[0164] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the implementation.

[0165] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0166] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0167] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0168] The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0169] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. An operation and maintenance log detection method, characterized by, The method includes: Obtain the log data to be tested and determine whether the log data to be tested is structured data; If the log data to be tested is not structured data, then the log data to be tested is input into the pre-stored detection model; Based on the detection model, anomaly detection is performed on the log data to be tested to obtain a first anomaly detection result; The method further includes: Obtain historical log data and train the detection model based on the historical log data; The detection model includes a BERT-based bidirectional LSTM-CRF model and a log model; The training process of the detection model includes: for unstructured log data, log parsing and log pattern learning are performed sequentially. The log parsing includes classification based on the level field appearing in the logs. First, the effective words in the log data are classified, and then classification is performed based on the log semantic information to construct a dataset training pattern, achieving dictionary-like representation. The log pattern learning process includes encoding text sentences and intermediate time features as sentence encodings, performing text encoding through the BERT model, and adding time features as additional features to the vector encoding. After sentence embedding, the log information encoding is concatenated as semantic and time information to obtain the encoding vector. Finally, based on the historical logs from the offline terminal, a bidirectional LSTM-CRF model is used to learn the log sequence pattern, completing the model training. If the log data to be tested is structured data, then anomaly detection is performed on the log data to be tested based on the pre-stored structured database.

2. The method according to claim 1, characterized in that, The acquisition of the log data to be tested includes: The log data to be tested is obtained, and the source of the log data to be tested is detected based on the pre-stored source database to obtain a second anomaly detection result.

3. The method according to claim 1, characterized in that, The acquisition of the log data to be tested includes: Obtain the log data to be tested and preprocess the log data to be tested; The preprocessing includes regularization and parsing.

4. An operation and maintenance log detection device for implementing the operation and maintenance log detection method according to any one of claims 1-3, characterized in that, The device includes: The test log data acquisition module is used to acquire the test log data and determine whether the test log data is structured data. The structured data judgment module is used to input the log data to be tested into a pre-stored detection model if the log data to be tested is not structured data. The first anomaly detection module is used to perform anomaly detection on the log data to be tested based on the detection model, and obtain the first anomaly detection result.

5. A computer device, characterized in that, The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set, or instruction set, and the at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor to implement the operation and maintenance log detection method as described in any one of claims 1 to 3.

6. An intelligent system, characterized in that, Includes the computer device as described in claim 5.

7. A computer-readable storage medium, characterized in that, The readable storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the operation and maintenance log detection method as described in any one of claims 1 to 3.

Images