Log anomaly detection method and device, equipment and storage medium

Abstract

The invention discloses a log anomaly detection method and device, equipment and a storage medium, and relates to the technical field of artificial intelligence, the method comprises the following steps: extracting feature data from log data, the feature data comprising log sequence features and log semantic features; inputting the log sequence features into a pre-trained strategy network to obtain a prediction result of a log sequence level; inputting the log semantic features into a fine-tuned large language model to obtain a reasoning result of a log semantic level; generating a reward signal based on the prediction result of the log sequence level and the reasoning result of the log semantic level; performing reinforcement learning fine tuning on the strategy network by using the reward signal to obtain an anomaly detection model; and performing anomaly detection on the log data based on the anomaly detection model, and outputting an anomaly detection result.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a log anomaly detection method, device, equipment and storage medium. Background Art

[0002] Anomaly detection is a key technology for identifying patterns or instances that significantly deviate from normal data, and plays an indispensable role in various real-world applications. In particular, log-based anomaly detection requires the analysis of system-generated logs to determine the occurrence of anomalies, which constitutes an important research area in data mining and software engineering. Logs usually capture dynamic information related to the runtime of system applications or software, including detailed information about system status and ongoing events. When a system fails, the role of the log is to record the abnormal information, making it easier for operation and maintenance personnel or experts to obtain detailed information about the failure by querying the log. This helps to systematically track and locate the source of the failure, and assists in the troubleshooting and resolution process. However, the development of IT services has led to system components becoming more complex and extensive, resulting in the generation of larger-scale log data. Therefore, it has become impractical to manually check log data for signs of system anomalies. In this context, AI for IT Operations (AIOps) has emerged as a new trend that uses artificial intelligence to solve tasks such as anomaly detection and root cause analysis.

[0003] Currently, anomaly detection methods based on both traditional machine learning and deep learning have been widely studied. However, these methods only produce predictions without accompanying explanations, which poses a challenge for human analysts in understanding and taking action based on the results. Therefore, some works have enhanced log-based anomaly detection by leveraging large language models (LLMs) that provide interpretability. Nevertheless, utilizing LLMs for anomaly detection often results in false alarms, with many normal logs being mistaken for anomalies, thus reducing accuracy. Summary of the invention

[0004] The purpose of this application is to provide a log anomaly detection method, device, equipment and storage medium.

[0005] To achieve the above objectives, this application provides the following solutions: In a first aspect, the present application provides a log anomaly detection method, comprising: Extracting feature data from log data, wherein the feature data includes: log sequence features and log semantic features; Inputting the log sequence features into the pre-trained policy network to obtain prediction results at the log sequence level; Inputting the log semantic features into the fine-tuned large language model to obtain the reasoning results at the log semantic level; Generate a reward signal based on the prediction result at the log sequence level and the reasoning result at the log semantic level; Using the reward signal to perform reinforcement learning fine-tuning on the policy network to obtain an anomaly detection model; Anomaly detection is performed on the log data based on the anomaly detection model, and an anomaly detection result is output.

[0006] Optionally, before the step of extracting feature data from log data, the method further includes: Get time series information of log data; Dividing the log data into a plurality of time steps according to the time series information, each time step including a log event; The log events at each time step are marked to obtain the log sequence data and the log semantic data.

[0007] Optionally, the step of extracting feature data from log data includes: Performing serialization processing on the log sequence data to obtain the log sequence features; Performing word segmentation processing on the log semantic data to obtain the log semantic features; The log sequence features and the log semantic features are normalized to obtain standardized feature data.

[0008] Optionally, the step of inputting the log sequence features into a pre-trained policy network to obtain a prediction result at the log sequence level includes: Inputting the log sequence features into the policy network, the policy network outputting the predicted log events at each time step; According to the consistency between the predicted log events and the actual log events, the prediction results at the log sequence level are calculated.

[0009] Optionally, the step of inputting the log semantic features into the fine-tuned large language model to obtain the reasoning result at the log semantic level includes: Inputting the log semantic features into the large language model, and the large language model outputs the reasoning result at the log semantic level; Whether the log data has semantic anomalies is determined according to the inference result, and a corresponding semantic reward signal is generated.

[0010] Optionally, the step of generating a reward signal based on the prediction result at the log sequence level and the reasoning result at the log semantic level includes: Generate a sequence reward signal according to the prediction results at the log sequence level; Generate a semantic reward signal according to the reasoning result at the semantic level of the log; The sequence reward signal and the semantic reward signal are weightedly summed to obtain a reward signal.

[0011] Optionally, the step of using the reward signal to perform reinforcement learning fine-tuning on the policy network to obtain an anomaly detection model includes: Inputting the reward signal into the policy network, wherein the policy network adjusts its parameters according to the reward signal; Through iterative training, the prediction ability of the policy network at the log sequence and log semantic levels is optimized to obtain an anomaly detection model.

[0012] In a second aspect, the present application provides a log anomaly detection device, comprising: An extraction module, used to extract feature data from log data, wherein the feature data includes: log sequence features and log semantic features; A training module, used for inputting the log sequence features into a pre-trained policy network to obtain a prediction result at the log sequence level; Inputting the log semantic features into the fine-tuned large language model to obtain the reasoning results at the log semantic level; Generate a reward signal based on the prediction result at the log sequence level and the reasoning result at the log semantic level; Using the reward signal to perform reinforcement learning fine-tuning on the policy network to obtain an anomaly detection model; The output module is used to perform anomaly detection on the log data based on the anomaly detection model and output anomaly detection results.

[0013] Optionally, the extraction module is further used to: Get time series information of log data; Dividing the log data into a plurality of time steps according to the time series information, each time step including a log event; The log events at each time step are marked to obtain the log sequence data and the log semantic data.

[0014] Optionally, the extraction module is further used to: Performing serialization processing on the log sequence data to obtain the log sequence features; Performing word segmentation processing on the log semantic data to obtain the log semantic features; The log sequence features and the log semantic features are normalized to obtain standardized feature data.

[0015] Optionally, the training module is further used to: Inputting the log sequence features into the policy network, the policy network outputting the predicted log events at each time step; According to the consistency between the predicted log events and the actual log events, the prediction results at the log sequence level are calculated.

[0016] Optionally, the training module is further used to: Inputting the log semantic features into the large language model, and the large language model outputs the reasoning result at the log semantic level; Whether the log data has semantic anomalies is determined according to the inference result, and a corresponding semantic reward signal is generated.

[0017] Optionally, the training module is further used to: Generate a sequence reward signal according to the prediction results at the log sequence level; Generate a semantic reward signal according to the reasoning result at the semantic level of the log; The sequence reward signal and the semantic reward signal are weightedly summed to obtain a reward signal.

[0018] Optionally, the training module is further used to: Inputting the reward signal into the policy network, wherein the policy network adjusts its parameters according to the reward signal; Through iterative training, the prediction ability of the policy network at the log sequence and log semantic levels is optimized to obtain an anomaly detection model.

[0019] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any one of the above-mentioned log anomaly detection methods.

[0020] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the log anomaly detection methods described above.

[0021] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any one of the log anomaly detection methods described above.

[0022] According to the specific embodiments provided in this application, this application discloses the following technical effects: The present application provides a log anomaly detection method, apparatus, device and storage medium, which avoids the need to manually craft a reward function by learning strategies directly from expert data, thereby eliminating the problem of suboptimal reward functions caused by improper manual design, and fine-tunes the large language model through LoRA to reduce the model's hallucination problem and make it better adapted to the log anomaly detection task. The fine-tuned large language model is then used to generate rewards, and the policy network is further fine-tuned through reinforcement learning, so that the model can integrate sequence and semantic knowledge, thereby improving the accuracy and reliability of anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 A flow chart of a log anomaly detection method provided in one embodiment of the present application; Figure 2 One of the principle schematic diagrams of a log anomaly detection method provided in one embodiment of the present application; Figure 3 A second schematic diagram of a log anomaly detection method provided in an embodiment of the present application; Figure 4 A schematic diagram of the functional modules of a log anomaly detection device provided in one embodiment of the present application; Figure 5 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] like Figure 1 As shown, some embodiments of the present application provide a log anomaly detection method, which includes the following steps 101 to 106. Among them: Step 101: extract feature data from log data, where the feature data includes: log sequence features and log semantic features.

[0027] In the embodiments of the present application, log data refers to the record files generated during the operation of the system or application, which contain detailed information about the system status and event occurrence. Feature data refers to a set of data extracted from the original log data that can represent the characteristics of the log and is used for subsequent analysis and processing. Log sequence features are features that reflect the chronological arrangement of log events. Log semantic features are features that reflect the meaning and contextual information of the log text content.

[0028] The system first reads the stored log data, and then uses feature extraction algorithms, such as natural language processing (NLP) technology, to parse and analyze the log data. In this process, the system can identify and extract the sequence features and semantic features of the logs, which will serve as input for subsequent steps.

[0029] Step 102: input the log sequence features into a pre-trained policy network to obtain a prediction result at the log sequence level.

[0030] In the embodiment of the present application, the pre-trained policy network refers to a network model that has been trained in some way (such as inverse reinforcement learning) and can preliminarily understand the log sequence and make predictions.

[0031] The system takes the extracted log sequence features as input and passes them to the pre-trained policy network. The policy network uses the knowledge it has learned in the pre-training phase to process and analyze the input log sequence features and generate prediction results at the log sequence level. These prediction results can include the predicted next log event, the probability distribution of log events, etc.

[0032] Specifically, Figure 2 As shown, inverse reinforcement learning pre-training includes a policy network and a discriminator , policy network The time step Status on As the input of the policy network, output the action that should be taken in this state , that is, predicting the next upcoming log event; while the discriminator The state-action pair As input, it outputs a score between 0 and 1, indicating that the discriminator Assume that this state-action pair is from an expert strategy Or agent strategy , the closer the score is to 1, the higher the probability that the discriminator regards this state-action pair as generated by the expert strategy.

[0033] Policy Network The goal is to generate interactive trajectories Can be discriminated Mistaken for expert track , so we can use the discriminator The output of is used as a reward function to train the policy network , that is, the reward is set to .

[0034] Finally, in the process of continuous adversarial training, the policy network The generated trajectory distribution will become closer and closer to the trajectory distribution of the expert strategy, and finally the pre-trained strategy network will be obtained. .

[0035] Step 103: input the log semantic features into the fine-tuned large language model to obtain the reasoning results at the log semantic level.

[0036] In the embodiment of the present application, the fine-tuned large language model refers to a large language model that can better understand log semantics and provide reasoning results after fine-tuning data for a specific task (such as log anomaly detection).

[0037] The system inputs the extracted log semantic features into the fine-tuned large language model. The large language model uses the knowledge about log anomaly detection learned during the fine-tuning phase to infer and analyze the input log semantic features and generate inference results at the log semantic level. These inference results may include whether the log is abnormal, the specific type or cause of the abnormality, etc.

[0038] Step 104: Generate a reward signal based on the prediction result at the log sequence level and the reasoning result at the log semantic level.

[0039] In the embodiment of the present application, the reward signal is a signal used to evaluate the quality of the behavior of the agent (here, the policy network) in reinforcement learning, and is usually generated based on the interaction results between the agent's behavior and the environment (here, the log data).

[0040] The system combines the prediction results at the log sequence level and the reasoning results at the log semantic level to design a reward mechanism. When the prediction results are consistent with the reasoning results and the log anomalies are correctly identified, the system gives a higher reward; when the prediction results or reasoning results are wrong, the system gives a lower reward or penalty. In this way, the system generates a series of reward signals for the subsequent reinforcement learning fine-tuning process.

[0041] Step 105: Use the reward signal to perform reinforcement learning fine-tuning on the policy network to obtain an anomaly detection model.

[0042] In the embodiment of the present application, reinforcement learning fine-tuning is the process of further training a pre-trained policy network under the reinforcement learning framework to optimize its performance.

[0043] The system uses the generated reward signal to fine-tune the policy network through reinforcement learning. During the fine-tuning process, the system continuously adjusts the parameters of the policy network to maximize the total reward obtained. Through continuous iteration and optimization, the policy network gradually learns how to more accurately predict log sequences and understand log semantics, thereby improving the accuracy of anomaly detection.

[0044] Specifically, the policy network A large language model is needed to provide rewards for further fine-tuning. Prior to this, the large language model needs to be fine-tuned through LoRA to alleviate the hallucination problem. LoRA is a lightweight network structure introduced to reduce the training cost of large language models. Studies have found that large language models are over-parameterized, they have smaller intrinsic dimensions, and will exhibit low-rank characteristics after pre-training. For a large language model with a heavy pre-training weight matrix , its weight is updated to Since the weight matrix It has low rank properties and can be decomposed into two matrices and The product of is the following formula (1): (1) in, , , and the order When fine-tuning a large language model, freeze the original weight matrix , only for two matrices and Update the parameters. This is usually a small value, so it can significantly reduce the number of parameters that need to be updated in large language models. Initialized using a Gaussian distribution, Initialize the matrix with 0.

[0045] The present invention collects data for LoRA fine-tuning a large language model at the log semantic level, which includes instructions, inputs and outputs, and manually designs prompts for log anomaly detection, so that the large language model can further understand the log anomaly detection task and reduce hallucinations after fine-tuning.

[0046] First, insert the pre-designed thought chain prompt into the front end of the log text and use it as the input of the model together with the log text. The word segmenter of the large language model will segment the input and obtain a word segmentation sequence , and then input it into the model, and finally output a word segmentation sequence Large language models optimize a conditional probability distribution during training , it is given For each possible output sequence Assign a probability. Formally, given The output sequence The probability is calculated as follows: (2) in express From position 0 to The prefix sequence of Indicates a given and Post-generative participle The formula is equivalent to minimizing the loss function as shown in the following formula (3): (3) After the large language model is fine-tuned, it is used to provide rewards and further fine-tune the policy network .

[0047] Policy Networks at the Log Sequence and Semantic Levels Make further fine-tuning.

[0048] Log sequence level rewards The design is as follows: (4) in, predict the size of the set for log events, For the actual log event, Indicates in action middle, The corresponding index , when there are multiple actions (predicted log events) that are consistent with the incoming log events, the minimum index is taken. Therefore, when the predicted log event is consistent with the incoming log event and the higher it is in the sorting position, the greater the reward.

[0049] The large language model infers whether the log is abnormal at the semantic level based on the prompt and log text, and gives a corresponding explanation. The inference results are manually marked based on the output of the large language model , in the process of reinforcement learning fine-tuning, if the action Inference results with large language models If they are consistent, a reward greater than 0 will be given , and a higher reward is given when it is inferred to be abnormal, otherwise the reward is 0, that is, the following formula (5): (5) in, predict the size of the set for log events, is the reward coefficient and , Indicates that the large language model inference is abnormal, Indicates that the large language model is inferred to be normal. The final reward is the following formula (6) (6) in is a hyperparameter used to balance the two reward components. Finally, we use the Proximal Policy Optimization (PPO) algorithm to fine-tune the policy network , which integrates the anomaly detection capabilities at the log sequence and semantic levels.

[0050] Step 106: Perform anomaly detection on the log data based on the anomaly detection model, and output anomaly detection results.

[0051] In the embodiment of the present application, the anomaly detection model refers to the final model that can be used for log anomaly detection after pre-training, fine-tuning and reinforcement learning training.

[0052] The system uses the trained anomaly detection model to perform anomaly detection on the original log data. The model determines whether the log has anomalies based on the characteristics of the log data (including sequence characteristics and semantic characteristics). Finally, the system outputs the anomaly detection results, including detailed information of the abnormal log, the type of anomaly or the cause, etc., for analysis and processing by operation and maintenance personnel or experts.

[0053] Specifically, Figure 3 As shown. Defined in time step Action , which includes the time step The actions at the log sequence and semantic level are recorded as and At the log sequence level, the definition is in time step Action To predict the next possible set of log events When the actual log event arrives Not in the strategic network In the predicted log events, , it is considered as a sequence anomaly, otherwise it is normal. At the log semantic level, it is defined at the time step Action To predict whether the current log event is abnormal at the semantic level.

[0054] The embodiments of the present application avoid the need to manually craft reward functions by learning strategies directly from expert data, thereby eliminating the problem of suboptimal reward functions caused by improper manual design. The large language model is fine-tuned through LoRA to reduce the model's hallucination problem and make it better suited to the log anomaly detection task. The fine-tuned large language model is then used to generate rewards, and the policy network is further fine-tuned through reinforcement learning, so that the model can integrate sequence and semantic knowledge, thereby improving the accuracy and reliability of anomaly detection.

[0055] Optionally, before step 101, the method further includes: Step 201: Obtain time series information of log data.

[0056] In the embodiment of the present application, time series information refers to information in which each log event in the log data is arranged in chronological order, reflecting the chronological relationship between the log events.

[0057] The system first reads the stored log data, and then uses time series analysis technology to extract the timestamps or time tags of each log event from the log data. These timestamps or time tags constitute the time series information of the log data, providing a basis for subsequent log data division and event marking.

[0058] Step 202: Divide the log data into multiple time steps according to the time series information, each time step including a log event.

[0059] In the embodiment of the present application, a time step is a number of continuous time periods into which log data is divided according to time series information in log data analysis, and each time period contains one or more log events.

[0060] The system divides the log data into multiple continuous time steps based on the extracted time series information. The length of each time step can be set according to actual needs. For example, it can be a fixed time period (such as 1 minute, 5 minutes, etc.), or a time period dynamically adjusted according to the density of log events. During the division process, the system ensures that each time step contains at least one log event, so that the log events of each time step can be marked and analyzed later.

[0061] Step 203: Mark the log event at each time step to obtain the log sequence data and the log semantic data.

[0062] In the embodiments of the present application, log sequence data refers to a data set that is arranged according to time series information and contains the order information of log events at each time step. Log semantic data refers to the data obtained after semantic analysis of log events, reflecting the meaning and context information of log events. Tagging here refers to the process of classifying, marking or interpreting log events for subsequent analysis and processing.

[0063] The system marks the log events in each time step. The marking process can involve classifying log events (such as errors, warnings, information, etc.) or performing semantic analysis on them (such as extracting key information of events, determining the nature of events, etc.). Through the marking process, the system can obtain log sequence data and log semantic data. Log sequence data retains the time order information of log events, while log semantic data provides a deeper understanding and description of log events. These data provide an important basis for subsequent anomaly detection and analysis.

[0064] Optionally, the step 101 includes: Step 1011, serialize the log sequence data to obtain the log sequence features.

[0065] In the embodiment of the present application, the system will use a series of technical means, such as converting log events into numerical codes, extracting time interval features between events, constructing a transition probability matrix between events, etc., in order to convert log sequence data into a format that can be recognized by the machine learning algorithm. Through these processes, the system obtains a series of log sequence features that can reflect the order relationship and pattern of log events, providing a basis for subsequent analysis and processing.

[0066] Step 1012: perform word segmentation processing on the log semantic data to obtain the log semantic features.

[0067] In the embodiment of the present application, the system will use the word segmentation algorithm in natural language processing to divide the text content in the log semantic data into independent words or phrases. Through these word segmentation processes, the system can better understand the meaning and contextual information of the log event, and then extract a series of log semantic features that can reflect this information. These features are crucial for subsequent anomaly detection and analysis.

[0068] Step 1013: normalize the log sequence features and the log semantic features to obtain standardized feature data.

[0069] In an embodiment of the present application, the system normalizes the log sequence features and log semantic features. In this process, the system may use different normalization methods, such as minimum-maximum normalization, Z-score normalization, etc., to ensure that each feature has the same scale and range. Through these normalization processes, the system obtains standardized feature data, which has better comparability and interpretability, and provides an important basis for subsequent analysis and processing. At the same time, normalization also helps to improve the convergence speed and performance of machine learning algorithms.

[0070] Optionally, the step 102 includes: Step 1021: input the log sequence features into the policy network, and the policy network outputs the predicted log event for each time step.

[0071] In the embodiment of the present application, predicting log events refers to the prediction of log events that may occur at each time step by the policy network based on the input log sequence features.

[0072] The system first passes the log sequence features as input to the policy network. The policy network performs deep learning and feature extraction on the input log sequence features through a series of complex calculations, such as convolution and full connection. Finally, the policy network outputs the predicted log events for each time step. These predicted events reflect the system's prediction of future log events based on historical log data.

[0073] Step 1022: Calculate the prediction result at the log sequence level according to the consistency between the predicted log event and the actual log event.

[0074] In an embodiment of the present application, the system then compares the predicted log events output by the policy network with the real log events. During this process, the system may use a series of evaluation indicators, such as accuracy, recall, F1 score, etc., to quantify the consistency between the predicted log events and the real log events. Through the calculation of these evaluation indicators, the system can obtain the prediction results at the log sequence level, which reflects the prediction performance of the policy network on the entire log sequence. If the prediction results are highly consistent with the real events, it means that the prediction ability of the policy network is strong; otherwise, it is necessary to further adjust and optimize the structure or parameters of the policy network.

[0075] Optionally, the step 103 includes: Step 1031: input the log semantic features into the large language model, and the large language model outputs the reasoning result at the log semantic level.

[0076] In the embodiment of the present application, the system first passes the log semantic features as input to the large language model. The large language model uses a complex neural network structure to perform deep learning and language understanding on the input log semantic features. It can use technical means such as attention mechanism and transformer structure to perform detailed analysis and processing of log semantic features. Finally, the large language model outputs reasoning results at the log semantic level, which reflect the model's in-depth understanding and reasoning ability of log data.

[0077] Step 1032: determine whether the log data has semantic anomalies according to the inference result, and generate a corresponding semantic reward signal.

[0078] In the embodiments of the present application, semantic anomaly refers to abnormal or unexpected situations in log data at the semantic level, which can be manifested as confusion in the semantic relationship of log events, unclear implicit intentions, etc. Semantic reward signal refers to a reward or penalty signal generated according to the semantic anomaly of log data, which is used to guide the system to process or optimize log data.

[0079] The system then analyzes and judges the semantic level of the log data based on the inference results output by the large language model. The system can set a series of rules or thresholds to detect whether there are semantic anomalies in the log data. If the inference results show that the semantic relationship of the log data is confusing, the implicit intention is unclear, or there is a contradiction with other log events, the system determines that the log data has semantic anomalies. At this time, the system will generate a corresponding semantic reward signal, which can be a negative reward signal, indicating punishment or dissatisfaction with the abnormal log data. On the contrary, if the semantic level of the log data meets expectations, the system can generate a positive reward signal to indicate recognition or encouragement of normal log data. These semantic reward signals can be used in subsequent log data processing, optimization or decision-making processes.

[0080] Optionally, the step 104 includes: Step 1041: Generate a sequence reward signal according to the prediction result at the log sequence level.

[0081] In an embodiment of the present application, the sequence reward signal is a reward or penalty signal generated based on the prediction results at the log sequence level, and is used to evaluate the system's prediction performance for log sequence data.

[0082] The system first analyzes the prediction results at the log sequence level. This result usually contains the comparison information between the predicted log events and the real log events, such as accuracy, recall and other evaluation indicators. The system generates a sequence reward signal based on these evaluation indicators and the preset reward rules. If the prediction result is accurate, the system can generate a positive sequence reward signal to indicate recognition of the prediction performance; if the prediction result is accurate, the system can generate a negative sequence reward signal to indicate dissatisfaction or punishment for the prediction performance.

[0083] Step 1042: Generate a semantic reward signal according to the reasoning result at the log semantic level.

[0084] In the embodiment of the present application, the semantic reward signal is a reward or penalty signal generated according to the reasoning result at the log semantic level, and is used to evaluate the system's ability to understand and process log semantics.

[0085] The system then analyzes the reasoning results at the semantic level of the log. This result reflects the system's semantic understanding and reasoning ability of the log data, such as whether the semantic relationship of the log event is clear and whether the implicit intention is clear. The system generates a semantic reward signal based on these reasoning results and the preset semantic reward rules. If the reasoning result is accurate and meets expectations, the system can generate a positive semantic reward signal; if the reasoning result has errors or does not meet expectations, the system can generate a negative semantic reward signal.

[0086] Step 1043: Perform a weighted summation on the sequence reward signal and the semantic reward signal to obtain a reward signal.

[0087] In the embodiment of the present application, the system finally performs weighted summation of the sequence reward signal and the semantic reward signal. In this process, the system performs weighted processing on the sequence reward signal and the semantic reward signal according to the preset weight distribution rules. The weighted signals are added to obtain a comprehensive reward signal. This reward signal comprehensively reflects the system's prediction performance for log sequence data and its understanding and processing capabilities for log semantics, and can be used to guide the system's further optimization or processing decisions for log data.

[0088] Optionally, the step 105 includes: Step 1051: input the reward signal to the policy network, and the policy network adjusts its parameters according to the reward signal.

[0089] In an embodiment of the present application, the system first inputs a reward signal into the policy network. This reward signal is a comprehensive evaluation based on the prediction results of the policy network at the log sequence and log semantic levels. After receiving the reward signal, the policy network adjusts its internal parameters according to this signal. Specifically, if the reward signal indicates that the prediction result of the policy network is good, then the policy network may fine-tune its parameters to maintain or improve performance; if the reward signal indicates that the prediction result is not good, the policy network will significantly adjust its parameters to improve the prediction ability. This parameter adjustment based on reward signals is a common practice in machine learning, especially reinforcement learning, which aims to optimize model performance through feedback mechanisms.

[0090] Step 1052, through iterative training, the prediction ability of the policy network at the log sequence and log semantic levels is optimized to obtain an anomaly detection model.

[0091] In an embodiment of the present application, the system then optimizes the predictive ability of the policy network through iterative training. In each iteration, the system uses a set of log data to train the policy network and generates a reward signal based on the training results. The policy network adjusts its parameters according to the reward signal to improve the predictive ability at the log sequence and log semantic levels. Through multiple iterative training, the parameters of the policy network gradually tend to be optimal, and its predictive ability on log data also reaches the best state. In the end, the system obtains a fully trained anomaly detection model that can accurately detect abnormal events in log data, providing strong support for the system's abnormal monitoring and troubleshooting.

[0092] Based on the same inventive concept, the embodiment of the present application also provides a log anomaly detection device for implementing the log anomaly detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more log anomaly detection device embodiments provided below can refer to the limitations of the log anomaly detection method above, and will not be repeated here.

[0093] In an exemplary embodiment, Figure 4 As shown, a log anomaly detection device 30 is provided, comprising: The extraction module 301 is used to extract feature data from the log data, wherein the feature data includes: log sequence features and log semantic features; A training module 302 is used to input the log sequence features into a pre-trained policy network to obtain a prediction result at the log sequence level; Inputting the log semantic features into the fine-tuned large language model to obtain the reasoning results at the log semantic level; Generate a reward signal based on the prediction result at the log sequence level and the reasoning result at the log semantic level; Using the reward signal to perform reinforcement learning fine-tuning on the policy network to obtain an anomaly detection model; The output module 303 is used to perform anomaly detection on the log data based on the anomaly detection model and output anomaly detection results.

[0094] Optionally, the extraction module 301 is further used for: Get time series information of log data; Dividing the log data into a plurality of time steps according to the time series information, each time step including a log event; The log events at each time step are marked to obtain the log sequence data and the log semantic data.

[0095] Optionally, the extraction module 301 is further used for: Performing serialization processing on the log sequence data to obtain the log sequence features; Performing word segmentation processing on the log semantic data to obtain the log semantic features; The log sequence features and the log semantic features are normalized to obtain standardized feature data.

[0096] Optionally, the training module 302 is further used to: Inputting the log sequence features into the policy network, the policy network outputting the predicted log events at each time step; According to the consistency between the predicted log events and the actual log events, the prediction results at the log sequence level are calculated.

[0097] Optionally, the training module 302 is further used to: Inputting the log semantic features into the large language model, and the large language model outputs the reasoning result at the log semantic level; Whether the log data has semantic anomalies is determined according to the inference result, and a corresponding semantic reward signal is generated.

[0098] Optionally, the training module 302 is further used to: Generate a sequence reward signal according to the prediction results at the log sequence level; Generate a semantic reward signal according to the reasoning result at the semantic level of the log; The sequence reward signal and the semantic reward signal are weightedly summed to obtain a reward signal.

[0099] Optionally, the training module 302 is further used to: Inputting the reward signal into the policy network, wherein the policy network adjusts its parameters according to the reward signal; Through iterative training, the prediction ability of the policy network at the log sequence and log semantic levels is optimized to obtain an anomaly detection model.

[0100] The embodiments of the present application avoid the need to manually craft reward functions by learning strategies directly from expert data, thereby eliminating the problem of suboptimal reward functions caused by improper manual design. The large language model is fine-tuned through LoRA to reduce the model's hallucination problem and make it better suited to the log anomaly detection task. The fine-tuned large language model is then used to generate rewards, and the policy network is further fine-tuned through reinforcement learning, so that the model can integrate sequence and semantic knowledge, thereby improving the accuracy and reliability of anomaly detection.

[0101] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store log anomaly detection data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a log anomaly detection method is implemented.

[0102] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0103] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0104] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0105] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0106] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0107] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0108] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0109] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0110] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A log anomaly detection method, characterized in that: The log anomaly detection method comprises: Extracting feature data from log data, wherein the feature data includes: log sequence features and log semantic features; Inputting the log sequence features into the pre-trained policy network to obtain prediction results at the log sequence level; Inputting the log semantic features into the fine-tuned large language model to obtain the reasoning results at the log semantic level; Generate a reward signal based on the prediction result at the log sequence level and the reasoning result at the log semantic level; Using the reward signal to perform reinforcement learning fine-tuning on the policy network to obtain an anomaly detection model; Anomaly detection is performed on the log data based on the anomaly detection model, and an anomaly detection result is output.

2. The log anomaly detection method according to claim 1, characterized in that: Before the step of extracting feature data from log data, the method further includes: Get time series information of log data; Dividing the log data into a plurality of time steps according to the time series information, each time step including a log event; The log events at each time step are marked to obtain the log sequence data and the log semantic data.

3. The log anomaly detection method according to claim 2, characterized in that: The step of extracting feature data from log data includes: Performing serialization processing on the log sequence data to obtain the log sequence features; Performing word segmentation processing on the log semantic data to obtain the log semantic features; The log sequence features and the log semantic features are normalized to obtain standardized feature data.

4. The log anomaly detection method according to claim 1, characterized in that: The step of inputting the log sequence features into a pre-trained policy network to obtain a prediction result at the log sequence level includes: Inputting the log sequence features into the policy network, the policy network outputting the predicted log events at each time step; According to the consistency between the predicted log events and the actual log events, the prediction results at the log sequence level are calculated.

5. The log anomaly detection method according to claim 1, characterized in that: The step of inputting the log semantic features into the fine-tuned large language model to obtain the reasoning result at the log semantic level includes: Inputting the log semantic features into the large language model, and the large language model outputs the reasoning result at the log semantic level; Whether the log data has semantic anomalies is determined according to the inference result, and a corresponding semantic reward signal is generated.

6. The log anomaly detection method according to claim 1, characterized in that: The step of generating a reward signal based on the prediction result at the log sequence level and the reasoning result at the log semantic level comprises: Generate a sequence reward signal according to the prediction results at the log sequence level; Generate a semantic reward signal according to the reasoning result at the semantic level of the log; The sequence reward signal and the semantic reward signal are weightedly summed to obtain a reward signal.

7. The log anomaly detection method according to claim 1, characterized in that: The step of using the reward signal to perform reinforcement learning fine-tuning on the policy network to obtain an anomaly detection model includes: Inputting the reward signal into the policy network, wherein the policy network adjusts its parameters according to the reward signal; Through iterative training, the prediction ability of the policy network at the log sequence and log semantic levels is optimized to obtain an anomaly detection model.

8. A log anomaly detection device, characterized in that: The log anomaly detection device comprises: An extraction module, used to extract feature data from log data, wherein the feature data includes: log sequence features and log semantic features; A training module, used for inputting the log sequence features into a pre-trained policy network to obtain a prediction result at the log sequence level; Inputting the log semantic features into the fine-tuned large language model to obtain the reasoning results at the log semantic level; Generate a reward signal based on the prediction result at the log sequence level and the reasoning result at the log semantic level; Using the reward signal to perform reinforcement learning fine-tuning on the policy network to obtain an anomaly detection model; The output module is used to perform anomaly detection on the log data based on the anomaly detection model and output anomaly detection results.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the log anomaly detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the log anomaly detection method described in any one of claims 1 to 7 are implemented.

Images