Log anomaly detection method, device, equipment and storage medium

Abstract

The present application discloses a log anomaly detection method, apparatus, device and storage medium, relating to the field of artificial intelligence technology. The method comprises: extracting feature data from log data, the feature data comprising: log sequence features and log semantic features; inputting the log sequence features into a pre-trained policy network to obtain prediction results at the log sequence level; inputting the log semantic features into a fine-tuned large language model to obtain inference results at the log semantic level; generating a reward signal based on the prediction results at the log sequence level and the inference results at the log semantic level; using the reward signal to perform reinforcement learning fine-tuning on the policy network to obtain an anomaly detection model; performing anomaly detection on the log data based on the anomaly detection model, and outputting the anomaly detection results.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a log anomaly detection method, apparatus, device, and storage medium. Background Art

[0002] Anomaly detection is a key technology for identifying patterns or instances that significantly deviate from normal data, playing an indispensable role in various real-world applications. Log-based anomaly detection, in particular, involves analyzing system-generated logs to identify anomalies, making it a significant research area in data mining and software engineering. Logs typically capture dynamic information related to the runtime of system applications or software, including detailed information about system status and ongoing events. When a system failure occurs, logs record the anomaly, allowing operations personnel or experts to query the logs for detailed information. This helps systematically trace and locate the source of the failure, facilitating troubleshooting and resolution. However, the evolution of IT services has led to more complex and extensive system components, resulting in the generation of vast amounts of log data. Consequently, manually examining log data for signs of system anomalies has become impractical. Against this backdrop, AI for IT Operations (AIOps) has emerged as a new trend that leverages artificial intelligence to address tasks such as anomaly detection and root cause analysis.

[0003] Anomaly detection methods based on both traditional machine learning and deep learning have been widely studied. However, these methods only produce predictions without accompanying explanations, which poses a challenge for human analysts to understand and act upon the results. Therefore, some work has leveraged large language models (LLMs) to provide interpretability to enhance log-based anomaly detection. However, using LLMs for anomaly detection often results in false alarms, misidentifying many normal logs as anomalies and thus reducing accuracy. Summary of the Invention

[0004] The purpose of this application is to provide a log anomaly detection method, device, equipment and storage medium.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a log anomaly detection method, comprising:

[0007] Extracting feature data from log data, wherein the feature data includes: log sequence features and log semantic features;

[0008] Input the log sequence features into the pre-trained policy network to obtain prediction results at the log sequence level;

[0009] Inputting the log semantic features into the fine-tuned large language model to obtain reasoning results at the log semantic level;

[0010] generating a reward signal based on the prediction result at the log sequence level and the reasoning result at the log semantic level;

[0011] Using the reward signal to perform reinforcement learning fine-tuning on the policy network to obtain an anomaly detection model;

[0012] Anomaly detection is performed on the log data based on the anomaly detection model, and an anomaly detection result is output.

[0013] Optionally, before the step of extracting feature data from the log data, the method further includes:

[0014] Get time series information of log data;

[0015] Dividing the log data into multiple time steps according to the time series information, each time step containing a log event;

[0016] The log events of each time step are marked to obtain the log sequence data and the log semantic data.

[0017] Optionally, the step of extracting feature data from log data includes:

[0018] Performing serialization processing on the log sequence data to obtain the log sequence features;

[0019] Performing word segmentation processing on the log semantic data to obtain the log semantic features;

[0020] The log sequence features and the log semantic features are normalized to obtain standardized feature data.

[0021] Optionally, the step of inputting the log sequence features into a pre-trained policy network to obtain a prediction result at the log sequence level includes:

[0022] Inputting the log sequence features into the policy network, the policy network outputting the predicted log events for each time step;

[0023] According to the consistency between the predicted log events and the actual log events, the prediction results at the log sequence level are calculated.

[0024] Optionally, the step of inputting the log semantic features into the fine-tuned large language model to obtain an inference result at the log semantic level includes:

[0025] Inputting the log semantic features into the large language model, and the large language model outputting the reasoning result at the log semantic level;

[0026] Determine whether the log data has semantic anomalies based on the inference result, and generate a corresponding semantic reward signal.

[0027] Optionally, the step of generating a reward signal based on the prediction result at the log sequence level and the inference result at the log semantic level includes:

[0028] Generate a sequential reward signal based on the prediction results at the log sequence level;

[0029] generating a semantic reward signal according to the reasoning result at the semantic level of the log;

[0030] The sequence reward signal and the semantic reward signal are weighted and summed to obtain a reward signal.

[0031] Optionally, the step of using the reward signal to perform reinforcement learning fine-tuning on the policy network to obtain an anomaly detection model includes:

[0032] Inputting the reward signal into the policy network, wherein the policy network adjusts its parameters according to the reward signal;

[0033] Through iterative training, the prediction ability of the policy network at the log sequence and log semantic levels is optimized to obtain an anomaly detection model.

[0034] In a second aspect, the present application provides a log anomaly detection device, comprising:

[0035] An extraction module is used to extract feature data from log data, wherein the feature data includes: log sequence features and log semantic features;

[0036] A training module is used to input the log sequence features into a pre-trained policy network to obtain prediction results at the log sequence level;

[0037] Inputting the log semantic features into the fine-tuned large language model to obtain reasoning results at the log semantic level;

[0038] generating a reward signal based on the prediction result at the log sequence level and the reasoning result at the log semantic level;

[0039] Using the reward signal to perform reinforcement learning fine-tuning on the policy network to obtain an anomaly detection model;

[0040] An output module is used to perform anomaly detection on the log data based on the anomaly detection model and output an anomaly detection result.

[0041] Optionally, the extraction module is further configured to:

[0042] Get time series information of log data;

[0043] Dividing the log data into multiple time steps according to the time series information, each time step containing a log event;

[0044] The log events of each time step are marked to obtain the log sequence data and the log semantic data.

[0045] Optionally, the extraction module is further configured to:

[0046] Performing serialization processing on the log sequence data to obtain the log sequence features;

[0047] Performing word segmentation processing on the log semantic data to obtain the log semantic features;

[0048] The log sequence features and the log semantic features are normalized to obtain standardized feature data.

[0049] Optionally, the training module is further used to:

[0050] Inputting the log sequence features into the policy network, the policy network outputting the predicted log events for each time step;

[0051] According to the consistency between the predicted log events and the actual log events, the prediction results at the log sequence level are calculated.

[0052] Optionally, the training module is further used to:

[0053] Inputting the log semantic features into the large language model, and the large language model outputting the reasoning result at the log semantic level;

[0054] Determine whether the log data has semantic anomalies based on the inference result, and generate a corresponding semantic reward signal.

[0055] Optionally, the training module is further used to:

[0056] Generate a sequential reward signal based on the prediction results at the log sequence level;

[0057] generating a semantic reward signal according to the reasoning result at the semantic level of the log;

[0058] The sequence reward signal and the semantic reward signal are weighted and summed to obtain a reward signal.

[0059] Optionally, the training module is further used to:

[0060] Inputting the reward signal into the policy network, wherein the policy network adjusts its parameters according to the reward signal;

[0061] Through iterative training, the prediction ability of the policy network at the log sequence and log semantic levels is optimized to obtain an anomaly detection model.

[0062] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any one of the above-described log anomaly detection methods.

[0063] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the above-mentioned log anomaly detection methods.

[0064] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any one of the above-mentioned log anomaly detection methods.

[0065] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0066] The present application provides a log anomaly detection method, apparatus, device, and storage medium. By learning strategies directly from expert data, the need for manually crafted reward functions is avoided, thereby eliminating the problem of suboptimal reward functions caused by improper manual design. The large language model is fine-tuned through LoRA to reduce the model's hallucination problem and make it better adapted to the log anomaly detection task. The fine-tuned large language model is then used to generate rewards, and the policy network is further fine-tuned through reinforcement learning, so that the model can integrate sequence and semantic knowledge, thereby improving the accuracy and reliability of anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0068] Figure 1 A flow chart of a log anomaly detection method provided in one embodiment of the present application;

[0069] Figure 2 This is a schematic diagram of the principle of a log anomaly detection method provided in one embodiment of the present application;

[0070] Figure 3 This is a second schematic diagram of the principle of a log anomaly detection method provided in one embodiment of the present application;

[0071] Figure 4 A schematic diagram of the functional modules of a log anomaly detection device provided in one embodiment of the present application;

[0072] Figure 5 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0073] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0074] like Figure 1 As shown, some embodiments of the present application provide a log anomaly detection method, which includes the following steps 101 to 106. Among them:

[0075] Step 101: extract feature data from log data, where the feature data includes log sequence features and log semantic features.

[0076] In the embodiments of the present application, log data refers to the record files generated during the operation of a system or application, containing detailed information about the system status and event occurrences. Feature data refers to a collection of data extracted from the raw log data that can represent the log's characteristics and is used for subsequent analysis and processing. Log sequence features reflect the chronological arrangement of log events. Log semantic features reflect the meaning and contextual information of the log text content.

[0077] The system first reads stored log data and then parses and analyzes it using feature extraction algorithms, such as natural language processing (NLP). This process allows the system to identify and extract the log's sequence and semantic features, which serve as input for subsequent steps.

[0078] Step 102: Input the log sequence features into a pre-trained policy network to obtain a prediction result at the log sequence level.

[0079] In the embodiment of the present application, the pre-trained policy network refers to a network model that has been trained in some way (such as inverse reinforcement learning) and can preliminarily understand the log sequence and make predictions.

[0080] The system uses the extracted log sequence features as input and passes them to the pre-trained policy network. The policy network uses the knowledge it learned during the pre-training phase to process and analyze the input log sequence features and generate prediction results at the log sequence level. These prediction results can include the predicted next log event and the probability distribution of log events.

[0081] Specifically, such as Figure 2 As shown, inverse reinforcement learning pre-training includes a policy network and a discriminator , policy network The time step Status As the input of the policy network, output the action that should be taken in this state , that is, predicting the next upcoming log event; and the discriminator The state-action pair As input, it outputs a score between 0 and 1, representing the discriminator Consider this state-action pair to be from an expert strategy Or agent strategy , the closer the score is to 1, the higher the probability that the discriminator regards this state-action pair as generated by the expert strategy.

[0082] Policy Network The goal is to generate interactive trajectories Can discriminate Mistaken for expert trajectory , so we can use the discriminator The output of is used as a reward function to train the policy network , that is, the reward is set to .

[0083] Finally, in the process of continuous adversarial training, the policy network The generated trajectory distribution will become closer and closer to the trajectory distribution of the expert strategy, and finally the pre-trained strategy network will be obtained. .

[0084] Step 103: Input the log semantic features into the fine-tuned large language model to obtain an inference result at the log semantic level.

[0085] In the embodiment of the present application, the fine-tuned large language model refers to a large language model that can better understand log semantics and provide reasoning results after fine-tuning data for a specific task (such as log anomaly detection).

[0086] The system inputs the extracted log semantic features into the fine-tuned large language model. The large language model uses the knowledge it learned about log anomaly detection during the fine-tuning phase to reason and analyze the input log semantic features, generating inference results at the log semantic level. These inference results may include whether the log contains an anomaly, the specific type of anomaly, and the cause.

[0087] Step 104: Generate a reward signal based on the prediction result at the log sequence level and the reasoning result at the log semantic level.

[0088] In the embodiment of the present application, the reward signal is a signal used to evaluate the quality of the behavior of the agent (here, the policy network) in reinforcement learning, and is usually generated based on the interaction between the agent's behavior and the environment (here, the log data).

[0089] The system combines predictions at the log sequence level with inferences at the log semantic level to design a reward mechanism. When the predictions align with the inferences and correctly identify log anomalies, the system provides higher rewards. When the predictions or inferences are incorrect, the system provides lower rewards or penalties. In this way, the system generates a series of reward signals that are used in the subsequent reinforcement learning fine-tuning process.

[0090] Step 105: Use the reward signal to perform reinforcement learning fine-tuning on the policy network to obtain an anomaly detection model.

[0091] In the embodiment of the present application, reinforcement learning fine-tuning is a process of further training a pre-trained policy network under the reinforcement learning framework to optimize its performance.

[0092] The system uses the generated reward signals to fine-tune the policy network through reinforcement learning. During fine-tuning, the system continuously adjusts the policy network's parameters to maximize the total reward. Through continuous iteration and optimization, the policy network gradually learns to more accurately predict log sequences and understand log semantics, thereby improving the accuracy of anomaly detection.

[0093] Specifically, the policy network A large language model is needed to provide rewards for further fine-tuning. Before this, the large language model needs to be fine-tuned through LoRA to alleviate the hallucination problem. LoRA is a lightweight network structure introduced to reduce the training cost of large language models. Studies have found that large language models are over-parameterized, they have smaller intrinsic dimensions, and will exhibit low-rank characteristics after pre-training. For a large language model with a heavy pre-training weight matrix , its weight is updated to Since the weight matrix It has low rank properties and can be decomposed into two matrices and The product of is the following formula (1):

[0094] (1)

[0095] in, , , and order When fine-tuning a large language model, freeze the original weight matrix , only for two matrices and Update parameters. It is usually a small value, so it can significantly reduce the number of parameters that need to be updated in large language models. Initialized using a Gaussian distribution, Initialize the matrix with 0.

[0096] This paper collects data for LoRA fine-tuning of the large language model at the log semantic level, which includes instructions, inputs and outputs, and manually designs prompts for log anomaly detection, so that the large language model can further understand the log anomaly detection task after fine-tuning and reduce hallucinations.

[0097] First, insert the pre-designed thought chain prompt into the front end of the log text and use it as the input of the model together with the log text. The word segmenter of the large language model will segment the input and obtain a word segmentation sequence. , then input it into the model and finally output a word segmentation sequence The large language model optimizes a conditional probability distribution during training. , it is given For each possible output sequence Assign a probability. Formally, given The output sequence The probability of is calculated as follows (2):

[0098] (2)

[0099] in express From position 0 to The prefix sequence of Indicates a given and Post-generative participle The probability of . The formula is equivalent to minimizing the loss function as shown in the following formula (3):

[0100] (3)

[0101] After fine-tuning the large language model, it is used to provide rewards and further fine-tune the policy network. .

[0102] Policy networks at the log sequence and semantic levels Make further fine-tuning.

[0103] Log sequence level rewards The design is as follows:

[0104] (4)

[0105] in, Predict the size of the set for log events, For the actual log event, Indicates action middle, The corresponding index When there are multiple actions (predicted log events) that are consistent with the incoming log event, the minimum index is taken. Therefore, when the predicted log event is consistent with the incoming log event and is ranked higher, the reward is greater.

[0106] The large language model infers whether the log is abnormal at the semantic level based on the prompt and log text, and gives a corresponding explanation. The inference results are manually marked based on the output of the large language model. , in the process of reinforcement learning fine-tuning, if the action Inference results with large language models If they are consistent, a reward greater than 0 will be given , and a higher reward is given when it is inferred to be abnormal, otherwise the reward is 0, that is, the following formula (5):

[0107] (5)

[0108] in, Predict the size of the set for log events, is the reward coefficient and , Indicates that the large language model inference is abnormal, Indicates that the large language model is inferred to be normal. The final reward is the following formula (6)

[0109] (6)

[0110] in is a hyperparameter used to balance the two reward components. Finally, we use the Proximal Policy Optimization (PPO) algorithm to fine-tune the policy network , which integrates the anomaly detection capabilities of log sequence and semantic level.

[0111] Step 106: Perform anomaly detection on the log data based on the anomaly detection model and output an anomaly detection result.

[0112] In the embodiment of the present application, the anomaly detection model refers to the final model that can be used for log anomaly detection after pre-training, fine-tuning and reinforcement learning training.

[0113] The system uses a trained anomaly detection model to detect anomalies in raw log data. The model determines whether the log contains anomalies based on the data's characteristics (including sequence and semantic features). Ultimately, the system outputs anomaly detection results, including detailed information about the anomaly, the anomaly type, and the cause, for analysis and resolution by operations personnel or experts.

[0114] Specifically, such as Figure 3 As shown. Defined in time step Action , which includes the time step The actions at the log sequence and semantic level are recorded as and At the log sequence level, it is defined at the time step Action To predict the next possible set of log events When the actual log event arrives Not in the strategic network Among the predicted log events, , it is considered as a sequence anomaly, otherwise it is normal. At the log semantic level, it is defined at the time step Action To predict whether the current log event is abnormal at the semantic level.

[0115] The embodiments of the present application avoid the need for manually crafted reward functions by learning strategies directly from expert data, thereby eliminating the problem of suboptimal reward functions caused by improper manual design. The large language model is fine-tuned through LoRA to reduce the model's hallucination problem and make it better adapted to the log anomaly detection task. The fine-tuned large language model is then used to generate rewards, and the policy network is further fine-tuned through reinforcement learning, so that the model can integrate sequence and semantic knowledge, thereby improving the accuracy and reliability of anomaly detection.

[0116] Optionally, before step 101, the method further includes:

[0117] Step 201: Obtain time series information of log data.

[0118] In the embodiment of the present application, time series information refers to the information in which each log event in the log data is arranged in chronological order, reflecting the sequential relationship between the log events.

[0119] The system first reads the stored log data and then uses time series analysis technology to extract the timestamps or time tags of each log event from the log data. These timestamps or time tags constitute the time series information of the log data, providing the basis for subsequent log data segmentation and event tagging.

[0120] Step 202: Divide the log data into multiple time steps according to the time series information, each time step containing a log event.

[0121] In the embodiment of the present application, a time step is a number of consecutive time periods into which log data is divided according to time series information in log data analysis, and each time period contains one or more log events.

[0122] Based on the extracted time series information, the system divides the log data into multiple consecutive time steps. The length of each time step can be set based on actual needs. For example, it can be a fixed period (such as 1 minute, 5 minutes, etc.) or a period that is dynamically adjusted based on the density of log events. During the division process, the system ensures that each time step contains at least one log event, so that the log events in each time step can be marked and analyzed later.

[0123] Step 203: Mark the log events of each time step to obtain the log sequence data and the log semantic data.

[0124] In the present embodiments, log sequence data refers to a data set that contains the order of log events at each time step, arranged according to time series information. Log semantic data refers to the data obtained through semantic analysis of log events, reflecting the meaning and context of the log events. Tagging, in this context, refers to the process of classifying, labeling, or interpreting log events to facilitate subsequent analysis and processing.

[0125] The system labels log events within each time step. This labeling process can involve classifying log events (e.g., error, warning, information) or performing semantic analysis (e.g., extracting key information from events and determining their nature). Through this labeling process, the system generates log sequence data and log semantic data. Log sequence data preserves the chronological order of log events, while log semantic data provides a deeper understanding and description of log events. This data provides an important foundation for subsequent anomaly detection and analysis.

[0126] Optionally, step 101 includes:

[0127] Step 1011: perform serialization processing on the log sequence data to obtain the log sequence features.

[0128] In the embodiments of this application, the system employs a series of technical means, such as converting log events into numerical codes, extracting time interval features between events, and constructing a transition probability matrix between events, in order to convert log sequence data into a format that can be recognized by machine learning algorithms. Through these processes, the system obtains a series of log sequence features that can reflect the sequential relationships and patterns of log events, providing a foundation for subsequent analysis and processing.

[0129] Step 1012: perform word segmentation processing on the log semantic data to obtain the log semantic features.

[0130] In the embodiments of this application, the system uses a word segmentation algorithm from natural language processing to segment the text content in the log semantic data into individual words or phrases. Through this word segmentation process, the system can better understand the meaning and context of the log events, and then extract a series of log semantic features that can reflect this information. These features are crucial for subsequent anomaly detection and analysis.

[0131] Step 1013 : normalize the log sequence features and the log semantic features to obtain standardized feature data.

[0132] In an embodiment of the present application, the system normalizes the log sequence features and log semantic features. During this process, the system may adopt different normalization methods, such as minimum-maximum normalization, Z-score normalization, etc., to ensure that each feature has the same scale and range. Through these normalization processes, the system obtains standardized feature data, which has better comparability and interpretability, providing an important basis for subsequent analysis and processing. At the same time, normalization also helps to improve the convergence speed and performance of machine learning algorithms.

[0133] Optionally, step 102 includes:

[0134] Step 1021: Input the log sequence features into the policy network, and the policy network outputs the predicted log event for each time step.

[0135] In the embodiment of the present application, predicting log events refers to the prediction of log events that may occur at each time step by the policy network based on the input log sequence features.

[0136] The system first passes the log sequence features as input to the policy network. The policy network performs deep learning and feature extraction on the input log sequence features through a series of complex computations, such as convolution and full connectivity. Ultimately, the policy network outputs predicted log events for each time step. These predicted events reflect the system's prediction of future log events based on historical log data.

[0137] Step 1022: Calculate the prediction result at the log sequence level based on the consistency between the predicted log event and the actual log event.

[0138] In an embodiment of the present application, the system then compares the predicted log events output by the policy network with the actual log events. During this process, the system may use a series of evaluation indicators, such as accuracy, recall rate, F1 score, etc., to quantify the consistency between the predicted log events and the actual log events. Through the calculation of these evaluation indicators, the system can obtain the prediction results at the log sequence level, which reflects the prediction performance of the policy network on the entire log sequence. If the prediction results are highly consistent with the actual events, it means that the prediction ability of the policy network is strong; otherwise, it is necessary to further adjust and optimize the structure or parameters of the policy network.

[0139] Optionally, step 103 includes:

[0140] Step 1031: Input the log semantic features into the large language model, and the large language model outputs the reasoning result at the log semantic level.

[0141] In this embodiment, the system first passes log semantic features as input to a large language model. The large language model uses a complex neural network structure to perform deep learning and language understanding on the input log semantic features. It utilizes technologies such as attention mechanisms and transformer structures to perform detailed analysis and processing of log semantic features. Ultimately, the large language model outputs reasoning results at the log semantic level, which reflect the model's deep understanding and reasoning capabilities of the log data.

[0142] Step 1032: Determine whether the log data has semantic anomalies based on the inference result, and generate a corresponding semantic reward signal.

[0143] In the present embodiment, semantic anomalies refer to situations where log data is abnormal or inconsistent with expectations at the semantic level, which can manifest as confusing semantic relationships between log events or unclear implicit intent. Semantic reward signals refer to reward or penalty signals generated based on semantic anomalies in log data, and are used to guide the system's processing or optimization of log data.

[0144] The system then analyzes and judges the semantics of the log data based on the inference results output by the large language model. The system may set a series of rules or thresholds to detect semantic anomalies in the log data. If the inference results indicate that the log data's semantic relationships are confusing, the implicit intent is unclear, or there is a conflict with other log events, the system determines that the log data has a semantic anomaly. At this point, the system generates a corresponding semantic reward signal, which can be a negative reward signal, indicating punishment or dissatisfaction with the anomalous log data. Conversely, if the semantics of the log data meet expectations, the system can generate a positive reward signal, indicating recognition or encouragement of normal log data. These semantic reward signals can be used in subsequent log data processing, optimization, or decision-making.

[0145] Optionally, step 104 includes:

[0146] Step 1041: Generate a sequence reward signal based on the prediction result at the log sequence level.

[0147] In an embodiment of the present application, the sequence reward signal is a reward or penalty signal generated based on the prediction results at the log sequence level, and is used to evaluate the system's prediction performance for log sequence data.

[0148] The system first analyzes the prediction results at the log sequence level. This result typically includes information comparing the predicted log events with the actual log events, such as accuracy and recall. Based on these evaluation metrics and pre-set reward rules, the system generates a sequence reward signal. If the prediction accuracy is high, the system can generate a positive sequence reward signal, indicating approval of the prediction performance; if the prediction accuracy is low, the system can generate a negative sequence reward signal, indicating dissatisfaction with the prediction performance or a penalty.

[0149] Step 1042: Generate a semantic reward signal based on the reasoning result at the semantic level of the log.

[0150] In the embodiment of the present application, the semantic reward signal is a reward or penalty signal generated based on the reasoning results at the log semantic level, and is used to evaluate the system's ability to understand and process log semantics.

[0151] The system then analyzes the semantic reasoning results of the logs. This result reflects the system's semantic understanding and reasoning capabilities of the log data, such as whether the semantic relationships between log events are clear and the implicit intent is clear. Based on these reasoning results and preset semantic reward rules, the system generates a semantic reward signal. If the reasoning result is accurate and meets expectations, the system can generate a positive semantic reward signal; if the reasoning result contains errors or does not meet expectations, the system can generate a negative semantic reward signal.

[0152] Step 1043: Perform a weighted summation on the sequence reward signal and the semantic reward signal to obtain a reward signal.

[0153] In this embodiment of the present application, the system finally performs a weighted summation of the sequence reward signal and the semantic reward signal. During this process, the system weights the sequence reward signal and the semantic reward signal according to a preset weighting rule. The weighted signals are summed to produce a composite reward signal. This reward signal comprehensively reflects the system's predictive performance for log sequence data and its ability to understand and process log semantics, and can be used to guide the system's further optimization or processing decisions for log data.

[0154] Optionally, step 105 includes:

[0155] Step 1051: Input the reward signal to the policy network, and the policy network adjusts its parameters according to the reward signal.

[0156] In an embodiment of the present application, the system first inputs a reward signal into the policy network. This reward signal is a comprehensive evaluation based on the prediction results of the policy network at the log sequence and log semantic levels. After receiving the reward signal, the policy network adjusts its internal parameters according to this signal. Specifically, if the reward signal indicates that the prediction result of the policy network is good, then the policy network may fine-tune its parameters to maintain or improve performance; if the reward signal indicates that the prediction result is poor, the policy network will significantly adjust its parameters to improve the prediction ability. This parameter adjustment based on reward signals is a common practice in machine learning, especially reinforcement learning, which aims to optimize model performance through feedback mechanisms.

[0157] In step 1052 , through iterative training, the prediction capability of the policy network at the log sequence and log semantic levels is optimized to obtain an anomaly detection model.

[0158] In an embodiment of the present application, the system then optimizes the predictive ability of the policy network through iterative training. In each iteration, the system uses a set of log data to train the policy network and generates a reward signal based on the training results. The policy network adjusts its parameters based on the reward signal to improve its predictive ability at the log sequence and log semantic levels. Through multiple iterative training, the parameters of the policy network gradually tend to be optimal, and its predictive ability on log data also reaches the best state. Ultimately, the system obtains a fully trained anomaly detection model that can accurately detect abnormal events in log data, providing strong support for the system's anomaly monitoring and troubleshooting.

[0159] Based on the same inventive concept, embodiments of the present application also provide a log anomaly detection device for implementing the aforementioned log anomaly detection method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations in one or more of the following embodiments of the log anomaly detection device can be found in the limitations of the log anomaly detection method above and will not be further elaborated here.

[0160] In an exemplary embodiment, Figure 4 As shown, a log anomaly detection device 30 is provided, comprising:

[0161] Extraction module 301, used to extract feature data from log data, the feature data including: log sequence features and log semantic features;

[0162] A training module 302 is configured to input the log sequence features into a pre-trained policy network to obtain prediction results at the log sequence level;

[0163] Inputting the log semantic features into the fine-tuned large language model to obtain reasoning results at the log semantic level;

[0164] generating a reward signal based on the prediction result at the log sequence level and the reasoning result at the log semantic level;

[0165] Using the reward signal to perform reinforcement learning fine-tuning on the policy network to obtain an anomaly detection model;

[0166] The output module 303 is configured to perform anomaly detection on the log data based on the anomaly detection model and output an anomaly detection result.

[0167] Optionally, the extraction module 301 is further configured to:

[0168] Get time series information of log data;

[0169] Dividing the log data into multiple time steps according to the time series information, each time step containing a log event;

[0170] The log events of each time step are marked to obtain the log sequence data and the log semantic data.

[0171] Optionally, the extraction module 301 is further configured to:

[0172] Performing serialization processing on the log sequence data to obtain the log sequence features;

[0173] Performing word segmentation processing on the log semantic data to obtain the log semantic features;

[0174] The log sequence features and the log semantic features are normalized to obtain standardized feature data.

[0175] Optionally, the training module 302 is further configured to:

[0176] Inputting the log sequence features into the policy network, the policy network outputting the predicted log events for each time step;

[0177] According to the consistency between the predicted log events and the actual log events, the prediction results at the log sequence level are calculated.

[0178] Optionally, the training module 302 is further configured to:

[0179] Inputting the log semantic features into the large language model, and the large language model outputting the reasoning result at the log semantic level;

[0180] Determine whether the log data has semantic anomalies based on the inference result, and generate a corresponding semantic reward signal.

[0181] Optionally, the training module 302 is further configured to:

[0182] Generate a sequential reward signal based on the prediction results at the log sequence level;

[0183] generating a semantic reward signal according to the reasoning result at the semantic level of the log;

[0184] The sequence reward signal and the semantic reward signal are weighted and summed to obtain a reward signal.

[0185] Optionally, the training module 302 is further configured to:

[0186] Inputting the reward signal into the policy network, wherein the policy network adjusts its parameters according to the reward signal;

[0187] Through iterative training, the prediction ability of the policy network at the log sequence and log semantic levels is optimized to obtain an anomaly detection model.

[0188] The embodiments of the present application avoid the need for manually crafted reward functions by learning strategies directly from expert data, thereby eliminating the problem of suboptimal reward functions caused by improper manual design. The large language model is fine-tuned through LoRA to reduce the model's hallucination problem and make it better adapted to the log anomaly detection task. The fine-tuned large language model is then used to generate rewards, and the policy network is further fine-tuned through reinforcement learning, so that the model can integrate sequence and semantic knowledge, thereby improving the accuracy and reliability of anomaly detection.

[0189] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store log anomaly detection data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a log anomaly detection method is implemented.

[0190] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0191] In an exemplary embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0192] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0193] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0194] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0195] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0196] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0197] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0198] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A log anomaly detection method, characterized in that: The log anomaly detection method includes: Extracting feature data from log data, wherein the feature data includes: log sequence features and log semantic features; Input the log sequence features into the pre-trained policy network to obtain prediction results at the log sequence level; Inputting the log semantic features into the fine-tuned large language model to obtain reasoning results at the log semantic level; generating a reward signal based on the prediction result at the log sequence level and the reasoning result at the log semantic level; Using the reward signal to perform reinforcement learning fine-tuning on the policy network to obtain an anomaly detection model; Anomaly detection is performed on the log data based on the anomaly detection model, and an anomaly detection result is output.

2. The log anomaly detection method according to claim 1, characterized in that: Before the step of extracting feature data from the log data, the method further includes: Get time series information of log data; Dividing the log data into multiple time steps according to the time series information, each time step containing a log event; The log events at each time step are marked to obtain log sequence data and log semantic data.

3. The log anomaly detection method according to claim 2, characterized in that: The step of extracting feature data from log data includes: Performing serialization processing on the log sequence data to obtain the log sequence features; Performing word segmentation processing on the log semantic data to obtain the log semantic features; The log sequence features and the log semantic features are normalized to obtain standardized feature data.

4. The log anomaly detection method according to claim 1, characterized in that: The step of inputting the log sequence features into the pre-trained policy network to obtain a prediction result at the log sequence level includes: Inputting the log sequence features into the policy network, the policy network outputting the predicted log events for each time step; According to the consistency between the predicted log events and the actual log events, the prediction results at the log sequence level are calculated.

5. The log anomaly detection method according to claim 1, characterized in that: The step of inputting the log semantic features into the fine-tuned large language model to obtain the reasoning result at the log semantic level includes: Inputting the log semantic features into the large language model, and the large language model outputting the reasoning result at the log semantic level; Determine whether the log data has semantic anomalies based on the inference result, and generate a corresponding semantic reward signal.

6. The log anomaly detection method according to claim 1, characterized in that: The step of generating a reward signal based on the prediction result at the log sequence level and the reasoning result at the log semantic level includes: Generate a sequential reward signal based on the prediction results at the log sequence level; generating a semantic reward signal according to the reasoning result at the semantic level of the log; The sequence reward signal and the semantic reward signal are weighted and summed to obtain a reward signal.

7. The log anomaly detection method according to claim 1, characterized in that: The step of using the reward signal to perform reinforcement learning fine-tuning on the policy network to obtain an anomaly detection model includes: Inputting the reward signal into the policy network, wherein the policy network adjusts its parameters according to the reward signal; Through iterative training, the prediction ability of the policy network at the log sequence and log semantic levels is optimized to obtain an anomaly detection model.

8. A log anomaly detection device, characterized in that: The log anomaly detection device includes: An extraction module is used to extract feature data from log data, wherein the feature data includes: log sequence features and log semantic features; A training module is used to input the log sequence features into a pre-trained policy network to obtain prediction results at the log sequence level; Inputting the log semantic features into the fine-tuned large language model to obtain reasoning results at the log semantic level; Based on the prediction results at the log sequence level and the inference results at the log semantic level, a reward signal is generated; and the policy network is fine-tuned through reinforcement learning using the reward signal to obtain an anomaly detection model. An output module is used to perform anomaly detection on the log data based on the anomaly detection model and output an anomaly detection result.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the log anomaly detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the log anomaly detection method according to any one of claims 1 to 7 are implemented.

Images