A deep learning driven server log anomaly detection method

Abstract

The application discloses a kind of deep learning driven server log anomaly detection methods, it is related to server operation and maintenance technical field, the method includes log data preprocessing, isolated forest dynamic sample selection, online contrast learning semantic self-adaption, deep anomaly detection model training and anomaly detection and output five steps, first to original unstructured log is parsed and standardized processing, again through isolated forest algorithm screening and resampling sample, generate dynamic balanced training dataset;Log template vector is updated using online contrast learning continuously, so that it adapts to system dynamic change, and both form closed loop optimization through collaborative mechanism;Finally, based on deep sequence model, learn normal log sequence mode, carry out anomaly detection to new log sequence and output result, the present application does not need frequent manual retraining, reduce operation and maintenance cost, improve long-term robustness and accuracy of anomaly detection simultaneously, suitable for intelligent operation and maintenance scene of various server clusters.

Description

Technical Field

[0001] This invention relates to the field of server operation and maintenance technology, specifically to a deep learning-driven method for detecting server log anomalies. Background Technology

[0002] With the widespread application of cloud computing and distributed systems, server systems are constantly expanding in scale, resulting in a significant increase in the log data generated during their operation. This log data records in detail the server's operating status, user operations, and system events, serving as crucial evidence for fault diagnosis, security monitoring, and performance optimization. Against this backdrop, intelligent operations and maintenance (O&M) has emerged, utilizing artificial intelligence technology to analyze O&M data to achieve automatic detection and early warning of system anomalies. Because log data directly and comprehensively reflects the system's operational status, log-based anomaly detection has become one of the key technologies in intelligent O&M.

[0003] Currently, log-based anomaly detection methods can be mainly divided into two categories: those based on traditional machine learning and those based on deep learning. In traditional machine learning methods, the Isolation Forest algorithm is widely used for log anomaly detection due to its high efficiency and lack of need for labeling. For example, patent CN110958222A discloses a server log anomaly detection method and system based on the Isolation Forest algorithm. This method uses the Isolation Forest algorithm to automatically detect anomalies in server logs, discovering abnormal patterns and improving detection efficiency. However, such methods typically use the Isolation Forest as an independent detector, and its detection performance largely depends on manual feature engineering, making it difficult to capture complex temporal dependencies and semantic information in log data. Another type of method is based on deep learning models. For example, the DeepLog framework uses Long Short-Term Memory networks to model log sequences, learning normal log sequence patterns to identify anomalies. While these methods can capture long-term dependencies in sequences, their models are usually trained on static historical data. When the system is updated, business logic changes, or new anomalies occur, the distribution of log data dynamically changes, causing the performance of the pre-trained model to rapidly decline and making it difficult to adapt to new operating environments. In addition, some studies have attempted to parse logs into templates and extract variable parameters, and combine them with the sequence of events before and after to comprehensively determine anomalies in order to improve the accuracy of the determination. However, such methods also face the challenge of insufficient model adaptability when dealing with massive log streams that are constantly changing.

[0004] In summary, existing server log anomaly detection methods, whether based on traditional machine learning algorithms like Isolation Forest or deep learning sequence models, all suffer from a common technical problem: they struggle to effectively handle dynamic changes in data distribution caused by continuous system operation and updates. This leads to rapid model failure, decreased accuracy, and the need for frequent manual retraining, resulting in high maintenance costs. Specifically, the main shortcomings of existing technologies are: while Isolation Forest-based methods offer high detection efficiency, their ability to learn complex temporal patterns and deep semantic features is limited, and the models themselves lack adaptive update mechanisms. Deep learning-based methods, although capable of learning sequence patterns, typically rely on fixed log template vector representations, failing to adapt to changes in log semantics as the system iterates. Most existing methods employ isolated detection strategies or simple model concatenation, lacking a collaborative and continuously evolving technical framework to address concept and semantic drift.

[0005] Therefore, there is a need in this field for a server log anomaly detection method that can adapt to dynamic changes in the system environment and has continuous learning capabilities, in order to reduce model maintenance costs and improve the long-term robustness and accuracy of the detection system in real and complex environments. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a deep learning-driven server log anomaly detection method. By integrating the dynamic sample selection of isolated forests with the online comparative learning semantic adaptive mechanism, it can effectively cope with the dynamic changes in data distribution in the system operating environment, achieve continuous adaptive anomaly detection, significantly improve the long-term robustness and detection accuracy of the model in complex real-world environments, and reduce the cost of manual intervention and model maintenance.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: On one hand, a deep learning-driven method for detecting server log anomalies includes the following steps: Step 1: Server log data preprocessing, used to parse the raw unstructured server logs, convert each log into a log template and parameter sequence, and process the log sequence into fixed-length segmented data through a sliding window mechanism; Step 2: Dynamic sample selection for isolated forest. This step is used to perform isolated forest algorithm analysis on the preprocessed log data, calculate the isolation path depth of each log data sample, and classify and resample the samples based on the isolation path depth. Simple normal samples with longer isolation path depths are downsampled, while boundary samples with shorter isolation path depths are oversampled to generate a dynamically balanced training dataset. Step 3: Online contrastive learning semantic adaptation is used to receive a dynamically balanced training dataset and continuously update the vector representation of the log template using an online contrastive learning mechanism. Instances of the same log template in different contexts are used as positive sample pairs, and instances of different log templates are used as negative sample pairs. By optimizing the contrastive learning loss function, the vector representations of positive sample pairs are made closer in semantic space, while the vector representations of negative sample pairs are made further apart, thereby obtaining a log template vector that is adapted to the current system operating environment. Step 4: Training the deep anomaly detection model. This step involves training a deep sequence model to learn normal log sequence patterns using a dynamically balanced training dataset and an adaptive log template vector. Step 5, Anomaly Detection and Output, is used to input the new log sequence to be detected into the trained deep anomaly detection model after passing through Step 1 and Step 3, calculate the anomaly score and compare it with a preset threshold, and finally output the anomaly detection result.

[0008] Furthermore, step one specifically includes: The log parsing unit is used to separate the dynamic parameter part from the static text part in the original log entry using a template-based parsing method, and extract a unified log template. A sequence construction unit is used to process variable-length log sequences using an improved sliding window, wherein the sliding window slides along the log sequence at a preset length and uses the log template sequence within the window as an input sample for the model; The dynamic zero-padding unit is used to pad log sequence segments that are shorter than the preset window length with zeros to ensure that all input data are standardized to a fixed dimension. The zero-padding operation is performed at the end of the sequence, and the zero-padding position is masked in subsequent model calculations to avoid introducing noise.

[0009] Furthermore, step two specifically includes: The isolated forest model building unit is used to build multiple isolated trees based on the input log data to form an isolated forest, where each isolated tree grows by randomly selecting features and split points; The sample path depth calculation unit is used to calculate the number of edges traversed from the root node to the leaf node of each log data sample, which is defined as the isolation path depth of the sample. The average of the path depths calculated in all isolated trees is taken as the final isolation path depth of the sample. The sample classification and resampling unit is used to perform classification and resampling operations based on the final isolation path depth of the samples. Its specific process is defined by the following mathematical formulas and rules:

[0010]

[0011] in, Indicates sample In the Path depth in an isolated tree This represents the total number of isolated trees. Indicates sample Average isolation path depth To determine the path depth threshold for simple samples, To determine the path depth threshold for boundary samples, This is the oversampling factor applied to boundary samples. This is a downsampling factor applied to simple, normal samples; The optimized training set output unit is used to resample the samples in the original training dataset according to the sampling weights, generate a dynamically balanced training dataset that focuses on boundary samples, and output it to step three.

[0012] Furthermore, step three specifically includes: The log template vector initialization unit is used to generate an initial semantic vector representation for each unique log template using a pre-trained language model; The context-aware sample pair building unit is used to collect instances of each log template in different context environments from the log stream generated by the system in real time. The context environment includes different time periods, different server instances, and different user sessions. Instances of the same template in different contexts are built into positive sample pairs, and instances of randomly selected different templates are built into negative sample pairs. The online contrastive loss calculation and vector update unit is used to optimize the vector representation of the log template through online contrastive learning of the loss function. The loss function is configured to increase the similarity of the vector representations between positive sample pairs and decrease the similarity of the vector representations between negative sample pairs through the optimization process, while maximizing the distance between negative sample pairs. The specific form of the loss function is as follows:

[0013] in, This indicates the number of sample pairs in the training batch. This indicates the number of negative sample pairs. Indicates sample Vector representation after encoder network Indicates and The sample vectors that constitute positive sample pairs Indicates the first A vector representation of a negative sample. This represents the temperature coefficient, used to adjust the sharpness of the probability distribution; The adaptive vector output unit is used to provide the updated log template vector to step four to ensure that the semantic representation used by the deep anomaly detection model is consistent with the current system state.

[0014] Furthermore, step four specifically includes: The model selection and initialization unit is used to select the Long Short-Term Memory (LSTM) network as the base sequence model and initialize its parameters. An adaptive vector input unit is used to take the log template vector updated in step three as the input feature of the deep sequence model. The sequence pattern learning unit is used to learn normal behavior patterns in log sequences through a deep sequence model, where the model takes the vectors of one or more previous log templates as input and predicts the conditional probability distribution of the next log template vector. Anomaly score definition unit is used to define anomaly scores for new log sequences after model training is completed, based on the accuracy of model prediction. The anomaly score is calculated based on the log-likelihood value of the actual log template in the conditional probability distribution predicted by the model.

[0015] Furthermore, steps two and three are coordinated through a shared semantic space and a feedback mechanism, specifically including: A sample priority passing mechanism is used to mark the boundary samples identified in step two as high-priority samples and pass their list to step three. A context-enhanced contrastive learning mechanism is used in step three to assign higher sampling probabilities to high-priority samples and to mine more diverse context instances for high-priority samples when constructing positive sample pairs, so as to accelerate the calibration of their vector representations. The feature space collaborative update mechanism is used to feed back the updated vector representation to step two after updating the log template vector in step three, which affects the calculation of the sample isolation path depth in the next round of pre-screening and forms a closed-loop optimization.

[0016] On the other hand, a deep learning-driven server log anomaly detection system, applicable to the aforementioned deep learning-driven server log anomaly detection method, the system comprising: The data acquisition and preprocessing module is used to collect raw log data from the server cluster in real time and perform log parsing, sequence construction and standardization operations. The dynamic sample selection module is used to run the Isolation Forest algorithm, realize the dynamic selection and resampling of training samples, and output the optimized training set. The semantic adaptive module is used to perform online contrastive learning and continuously update the vector representation of the log template to adapt to changes in the system environment; The model training module is used to train and update the deep anomaly detection model using the optimized training set and adaptive vectors. The real-time detection and alarm module is used to receive new server log streams, use the trained model to perform anomaly detection, and generate alarm information when an anomaly is detected. The system control and scheduling module is used to coordinate the data flow and control flow between various modules and manage the timing and dependencies of the entire anomaly detection process.

[0017] In another aspect, a computer-readable storage medium is provided for the aforementioned deep learning-driven server log anomaly detection method, wherein a computer program is stored thereon, which, when executed by a processor, implements steps one through five.

[0018] In another aspect, a computer device is provided, applicable to the aforementioned deep learning-driven server log anomaly detection method, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement steps one to five.

[0019] Compared with existing technologies, this deep learning-driven server log anomaly detection method has the following advantages: I. This invention classifies and resamples log data through isolated forest dynamic sample selection, focusing on boundary samples and weakening the redundant influence of simple normal samples. Combined with online contrastive learning semantic adaptive continuous updating of log template vectors, the semantic representation adapts to the dynamic changes of the system's operating environment. At the same time, through sample priority propagation, context-enhanced contrastive learning, and feature space collaborative update mechanisms, the two achieve collaborative closed-loop optimization, thereby effectively solving the problem that existing methods are unable to cope with the rapid failure of models caused by dynamic changes in data distribution. It eliminates the need for frequent manual retraining of the model, significantly reduces operation and maintenance costs, and improves the long-term robustness of the detection system in complex environments.

[0020] Second, this invention achieves standardized preprocessing of log data through template-based log parsing, improved sliding windows, and dynamic zero padding, providing high-quality input for subsequent model training. By leveraging deep sequence models to learn the temporal dependencies and semantic patterns of normal log sequences, and combining dynamically balanced training datasets and adaptive semantic vectors, the model's ability to learn complex temporal patterns and deep semantic features is enhanced, thereby effectively improving the accuracy of anomaly detection. In particular, it has a good recognition effect on boundary samples and easily confused log sequences, ensuring the reliability of detection results and providing strong support for maintenance personnel to quickly locate faults.

[0021] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0023] Figure 1 This is a flowchart of the steps of the present invention; Figure 2 This is a schematic diagram of the dynamic sample selection and semantic adaptive collaborative mechanism of the present invention; Figure 3 This is a flowchart of the training and inference process of the deep anomaly detection model of the present invention. Detailed Implementation

[0024] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0025] Example 1 like Figures 1 to 3 As shown, this embodiment provides a specific implementation scheme for a deep learning-driven server log anomaly detection method. This scheme is applicable to scenarios requiring real-time monitoring of log anomalies, such as large-scale distributed server clusters, cloud computing data centers, and e-commerce platform servers. It can effectively address the problem of dynamic changes in data distribution during system operation and improve the long-term robustness and accuracy of anomaly detection. The technical solution of this invention will be described in detail below with reference to the specific implementation process.

[0026] In this embodiment, the core purpose of server log data preprocessing is to convert the raw unstructured logs into standardized model input data, including three key steps: log parsing, sequence construction, and dynamic zero padding. The specific implementation is as follows: Log parsing: The log parsing unit employs a template-based parsing method, combining string matching and regular expression strategies to separate dynamic parameters from static text. In practical applications, raw logs typically contain information such as timestamps, server identifiers, event descriptions, and dynamic parameters. The dynamic parameters are uncertain, while the static text reflects the core event type of the log.

[0027] The specific implementation process is as follows: First, the collected raw logs are cleaned to remove invalid characters and duplicate log entries; then, by statistically analyzing the repetition patterns of strings in the logs, high-frequency text fragments are initially selected as candidate static texts; next, regular expressions are used to optimize the candidate static texts, defining parameter matching rules, marking the parts of the logs that match the parameter rules as dynamic parameters, and the remaining parts as static texts; finally, logs corresponding to the same static texts are grouped into one category to form a unified log template.

[0028] Example description: The original log entry is "2024-06-1514:30:22Server-89Process-1001Successfullyread2048bytesfromfile / data / logs / app.log" After parsing, the static text is "ServerProcessSuccessfullyreadbytesfromfile" Dynamic parameters are "89" "1001" "2024-06-1514:30:22" "2048" " / data / logs / app.log" The corresponding log template is "ServerProcessSuccessfullyreadbytesfromfile".

[0029] The purpose of this step is to strip away the variable information in the logs and extract log templates with semantic consistency, laying the foundation for subsequent sequence modeling and semantic learning.

[0030] Sequence Construction: The sequence construction unit employs an improved sliding window mechanism to handle variable-length log sequences. The preset length of the sliding window needs to be determined based on the statistical characteristics of the log sequences in the actual scenario. Specifically, the length distribution of normal log sequences in historical log data is statistically analyzed, and the median of the distribution is taken as the preset window length, ensuring that this length can cover most normal log behavior sequences. The sliding step size is set to 1, meaning that the window slides one log entry sequentially along the log sequence to ensure the continuity and integrity of the log sequence.

[0031] The specific implementation process is as follows: the parsed log templates are sorted according to the time sequence of log generation to form an ordered log template sequence; the sliding window starts from the beginning of the sequence and each time a log template fragment of a preset length is extracted as an input sample; when the window slides to the end of the sequence, if the number of remaining log templates is less than the preset window length, the remaining part is extracted and then normalized by dynamic zero padding.

[0032] For example, if the preset window length is 10 and a certain log template sequence contains 15 entries, then the sliding window will generate a total of 6 input samples. The first 5 samples each contain 10 log templates, and the 6th sample contains 5 log templates.

[0033] The purpose of this step is to convert the variable-length log sequence into a fixed-length input sample, thus meeting the requirement of deep sequence models for consistency of input dimensions.

[0034] Dynamic zero-padding: The dynamic zero-padding unit performs zero-padding on log sequence segments that are shorter than the preset window length, with the padding position located at the end of the sequence. To avoid the noise introduced by zero-padding affecting model training, a masking mechanism is used to mask the zero-padding position after padding.

[0035] The specific implementation process is as follows: For each input sample, a mask vector with the same length as the sample is assigned. The elements of the mask vector are either 1 or 0, where the position corresponding to the original log template is 1 and the position corresponding to the zero padding is 0. The mask vector is input into the model along with the input sample. In the model's embedding layer and subsequent computation layers, the gradient calculation and feature extraction at the position with a value of 0 are ignored through the mask operation, and only the position with a value of 1 is effectively calculated.

[0036] The purpose of this step is to ensure that all input samples have the same dimensionality, while avoiding the negative impact of zero-padding noise on model performance.

[0037] In this embodiment, the core objective of the dynamic sample selection in isolated forest is to select representative samples using the isolated forest algorithm, generate a dynamically balanced training dataset through resampling, and highlight the training weights of boundary samples. The specific implementation is as follows: Isolation Forest Model Construction: The isolation forest model construction unit builds multiple isolation trees based on the preprocessed log data, forming an isolation forest. The number of isolation trees... The number of isolated trees required depends on the size of the training samples. A larger sample size necessitates more isolated trees to ensure model stability and detection accuracy. In practical applications, when the training sample size is between 10,000 and 100,000... The value ranges from 100 to 200; when the sample size exceeds 100,000, The value ranges from 200 to 300.

[0038] The construction process of each isolated tree is as follows: randomly select a portion of the training samples as the training data for the isolated tree; randomly select a log template feature as the segmentation feature; randomly select a segmentation point within the value range of the feature to divide the sample set into two parts, which are the left subtree and right subtree of the current node, respectively; repeat the above segmentation process until each leaf node contains only one sample or the preset tree depth limit is reached.

[0039] The purpose of this step is to construct an isolated forest by using multiple isolated trees, and to use the sensitivity of the isolated forest to abnormal samples to achieve preliminary screening of samples.

[0040] Sample path depth calculation: For each log data sample, the sample path depth calculation unit calculates its isolation path depth in each isolated tree and takes the average value as the final isolation path depth.

[0041] The path depth of an isolated tree is defined as the number of edges a sample traverses from the root node to a leaf node. The specific calculation process is as follows: For each sample, traverse every isolated tree in the isolated forest. Starting from the root node, based on the relationship between the sample's feature values ​​and the split point, sequentially enter the left or right subtree until a leaf node is reached. Record the number of edges traversed during this process; this is the path depth of the sample in that isolated tree. After traversing all isolated trees, the average isolated path depth of the samples is calculated using the following formula. :

[0042] in, It is an abbreviation for Average Isolated Path Depth, representing the average path depth of a sample across all isolated trees in an isolated forest. Indicates sample In the The path depth in an isolated tree is a non-negative integer. The smaller the path depth, the more likely the sample is to be isolated, and the more likely it is to be an outlier or boundary sample. This represents the total number of isolated trees in an isolated forest, and is a preset hyperparameter determined based on the sample size. Indicates sample The average isolation path depth reflects the degree of isolation of a sample within the entire isolated forest. The smaller the sample size, the higher the degree of isolation.

[0043] The purpose of this step is to quantify the isolation level of samples through path depth, providing a basis for sample classification and resampling.

[0044] Sample classification and resampling: Sample classification and resampling units are based on the average isolation path depth. The samples are classified, and sampling weights are assigned based on the classification results to achieve resampling.

[0045] First, determine two key thresholds: the path depth threshold for judging simple samples. and the path depth threshold for determining boundary samples The threshold is determined by statistically analyzing all samples in the training set. Distribution, calculate the 80th quantile of the distribution as The 20th percentile as .Right now The sample is a simple normal sample. The samples in the middle are considered boundary samples, and the samples in between are considered normal samples.

[0046] The sampling weights are determined by the following formula:

[0047] in, This is an oversampling factor applied to boundary samples, ranging from 1.5 to 2.0. It increases the proportion of boundary samples in the training set, improving the model's learning ability on boundary samples. This is a downsampling factor applied to simple normal samples, with a value ranging from 0.3 to 0.5. It is used to reduce the number of simple normal samples and prevent the model from overfitting to typical normal samples. The sampling weights for ordinary samples are set to ensure that this type of sample maintains its original proportion in training. To supplement the conditional keywords, cover all sample cases not included in the aforementioned conditions, and ensure that each sample has a unique sampling weight, Indicates sample The sampling weights are determined by the sample size; the larger the weight, the higher the probability that the sample will be selected during the resampling process. These are keywords used for conditional judgments and weight allocation rules for different sample types.

[0048] The specific implementation process of resampling is as follows: Based on the sampling weight of each sample, samples are drawn from the original training set using a weighted random sampling method to form a new training dataset. For boundary samples, due to their sampling weights... The probability of being selected is higher than the original proportion; for simple normal samples, due to their sampling weights The probability of being selected is lower than the original proportion; the sampling weight of ordinary normal samples is 1, keeping the original proportion unchanged.

[0049] The purpose of this step is to generate a dynamically balanced training dataset through resampling, highlight the training priority of boundary samples, and improve the model's ability to identify abnormal and boundary samples.

[0050] Optimize training set output: The optimized training set output unit outputs the resampled training dataset to the online contrastive learning semantic adaptation stage, ensuring that subsequent semantic learning and model training can be based on representative samples.

[0051] In this embodiment, the core objective of online contrastive learning semantic adaptation is to continuously update the vector representation of the log template through online contrastive learning, enabling it to adapt to the dynamic changes in the system's operating environment. The specific implementation is as follows: Log template vector initialization: The log template vector initialization unit generates an initial semantic vector representation for each unique log template using a pre-trained language model. The selected pre-trained language model is the BERT-base model, which can effectively capture the contextual semantic information of the text and is suitable for learning the semantic representation of log templates.

[0052] The specific implementation process is as follows: the static text of each log template is used as input and fed into the pre-trained BERT-base model. The output vector of the last hidden layer of the model is extracted as the initial semantic vector of the log template. The dimension of the vector is set to 768, which is consistent with the output dimension of the BERT-base model to ensure the integrity of the semantic information.

[0053] The purpose of this step is to provide an initial semantic vector representation for the log template, laying the foundation for subsequent online comparative learning.

[0054] Context-aware sample pair construction: The context-aware sample pair construction unit collects instances from the real-time log stream generated by the system under different contexts for each log template, and constructs positive and negative sample pairs. The context includes different time periods, different server instances, and different user sessions: different time periods refer to the time intervals in which logs are generated, different server instances refer to different physical or virtual server nodes in the cluster, and different user sessions refer to operation sessions initiated by different users.

[0055] The specific implementation process is as follows: Positive sample pair construction: For each log template, collect its log instances in different contexts within a recent period, with each instance corresponding to a log template vector; form positive sample pairs by combining any two instances of the same log template in different contexts to ensure semantic consistency of the positive sample pairs.

[0056] Negative sample pair construction: For each log template instance, randomly select instances of other different log templates as negative samples. Each positive sample pair corresponds to multiple negative samples, forming a negative sample pair. The selection of negative samples must ensure diversity and avoid repeatedly selecting instances of the same type of log template.

[0057] For example, if the instance of log template A under time period T1, server S1, and user session U1 is A1, and the instance under time period T2, server S2, and user session U2 is A2, then A1 and A2 form a positive sample pair; randomly select the instance B1 of log template B and the instance C1 of log template C as negative samples, and form negative sample pairs (A1, B1) and (A1, C1) with A1.

[0058] The purpose of this step is to construct sample pairs with contextual discrimination to provide training data for contrastive learning.

[0059] Online contrastive loss calculation and vector update: The online contrastive loss calculation and vector update unit optimizes the contrastive learning loss function and adjusts the vector representation of the log template to increase the vector similarity of positive sample pairs and decrease the vector similarity of negative sample pairs.

[0060] The specific form of the contrastive learning loss function is:

[0061] in, This represents the number of sample pairs in the training batch, ranging from 256 to 512, determined based on GPU memory capacity and training efficiency. This represents the number of negative sample pairs corresponding to each positive sample pair, with a value ranging from 1024 to 2048, ensuring the diversity of negative samples. Indicates sample The vector representation after passing through the encoder network, which uses a two-layer fully connected neural network with an input dimension of 768 and an output dimension of 256, employs ReLU as the activation function. Indicates and The sample vectors that constitute positive sample pairs Indicates the first A vector representation of a negative sample. This represents the temperature coefficient, ranging from 0.05 to 0.5, and is used to adjust the sharpness of the probability distribution. The smaller the value, the sharper the probability distribution, and the higher the model's ability to distinguish between samples of similarity. This represents the contrastive learning loss value. The smaller the loss value, the higher the similarity of positive sample pairs and the lower the similarity of negative sample pairs, indicating better semantic discriminative power in the vector representation. The natural logarithm function (in) (Based on the base), used to convert probability ratios into optimizable loss values. The natural exponential function ( The exponentiation operation is used to map vector similarity to the positive real number domain and amplify the differences. The pointer transpose operation makes the column vector satisfy the dimension requirements of the dot product operation.

[0062] The specific optimization process is as follows: input the constructed sample pairs into the encoder network to obtain the vector representation of each sample; calculate the loss value according to the above loss function; use the Adam optimizer to update the parameters of the encoder network and the vector representation of the log template, with the learning rate set to... The number of iterations is determined based on the update frequency of the real-time log stream, with one iteration update performed every 1000 new logs received.

[0063] The role of this step is to continuously optimize the loss function and update the vector representation of the log template so that it can adapt to the dynamic changes in the system's operating environment and maintain the accuracy of the semantic representation.

[0064] Adaptive Vector Output: The adaptive vector output unit outputs the updated log template vector to the deep anomaly detection model training stage, ensuring that the semantic vector used by the model is consistent with the current system state.

[0065] In this embodiment, the core objective of training the deep anomaly detection model is to train a deep sequence model using a dynamically balanced training dataset and adaptive log template vectors to learn normal log sequence patterns. The specific implementation is as follows: Model selection and initialization: The Long Short-Term Memory (LSTM) network was selected as the basic sequence model for model selection and initialization. LSTM can effectively capture long-term dependencies in log sequences and is suitable for time series data modeling.

[0066] The model parameters are initialized as follows: The input layer has a dimension of 256, which is consistent with the dimension of the adaptive log template vector; The hidden layer is set to two layers, with 256 hidden units in each layer; The output layer dimension is consistent with the total number of log templates, and is used to output the probability distribution of the next log template; Activation functions: tanh function is used in the hidden layer, and softmax function is used in the output layer; The forget gate bias is initialized to 1 to help the model remember long-term dependencies in the early stages of training. Other parameters are initialized using the Xavier method to avoid gradient vanishing or gradient exploding during training.

[0067] The purpose of this step is to build a basic sequence model, providing a model architecture for sequence pattern learning.

[0068] Adaptive Vector Input: The adaptive vector input unit uses the updated log template vector learned online as the input feature of the LSTM model. Specifically, the log template vector corresponding to each input sample is arranged in order to form an input matrix of dimension , which is then input into the LSTM model.

[0069] The purpose of this step is to provide the model with adaptive semantic input features, thereby improving the model's ability to adapt to changes in the system environment.

[0070] Sequence Pattern Learning: The sequence pattern learning unit learns the behavioral patterns of normal log sequences using an LSTM model. The training objective of the model is to learn from the previous... The vector representation of the log template is used to predict the th log template. Conditional probability distribution of log templates ( (Minus 1 from the preset window length).

[0071] The specific training process is as follows: the dynamically balanced training dataset is divided into a training set and a validation set in a 7:3 ratio; the input matrix of the training set is input into the LSTM model, and the model calculates the probability distribution of the output layer through forward propagation; the cross-entropy loss function is used to calculate the predicted value and the true value (the first value is the first value). The loss value of each log template is calculated; the model parameters are updated using the Adam optimizer, with the learning rate set to [value missing]. Iterate through training until the loss value on the validation set stabilizes (the loss value decreases by less than 1% over 5 consecutive iterations). ).

[0072] The purpose of this step is to enable the model to learn the temporal dependencies and semantic patterns of normal log sequences, providing a basis for anomaly detection.

[0073] Anomaly Score Definition: The anomaly score definition unit defines anomaly scores for new log sequences based on the accuracy of model predictions. The specific calculation method is as follows: For a new log sequence, it is converted into a standardized input matrix and then input into a trained LSTM model to obtain the conditional probability distribution of the next log template output by the model; the probability value of the actual next log template in this probability distribution is extracted, and the log-likelihood value of this probability value is calculated (i.e., ...). (Probability value); the negative value of the log-likelihood is taken as the outlier score, i.e., the outlier score. (Predicted probability of real log template).

[0074] The physical meaning of the anomaly score is as follows: the higher the anomaly score, the lower the prediction probability of the true log template, the greater the deviation of the log sequence from the normal pattern learned by the model, and the more likely it is to be an anomaly sequence; conversely, the lower the anomaly score, the more the log sequence conforms to the normal pattern.

[0075] The purpose of this step is to quantify the degree of anomaly in the log sequence, providing quantitative indicators for anomaly detection.

[0076] In this embodiment, the core purpose of anomaly detection and output is to determine the anomalies in the new log sequence and output the detection results. The specific implementation is as follows: First, a preset threshold for anomaly scores is determined. This threshold is calculated by taking the 95th percentile of all normal log sequences from the validation set used during training, and then using this threshold as the preset threshold. This threshold setting ensures that the false positive rate for normal log sequences is low while maximizing the coverage of anomaly log sequences.

[0077] The specific detection process is as follows: New log streams from the server are collected in real time and preprocessed according to the method in step one to obtain standardized input samples; the log template vector in the current system environment is obtained through the online contrastive learning semantic adaptive module in step three, and an input matrix is ​​constructed; the input matrix is ​​input into the trained deep anomaly detection model to calculate the anomaly score; the anomaly score is compared with a preset threshold. If the anomaly score is greater than or equal to the preset threshold, the log sequence is determined to be abnormal; if the anomaly score is less than the preset threshold, the log sequence is determined to be normal.

[0078] The output of the detection results includes: the abnormal status of the log sequence, the abnormal score, the generation time of the log sequence, the corresponding server instance identifier, and the log template sequence. When an anomaly is detected, the system generates an alarm message and notifies the operations and maintenance personnel via SMS, email, or the operations and maintenance platform to facilitate timely troubleshooting.

[0079] The purpose of this step is to enable real-time anomaly detection in log sequences and provide clear anomaly information to operations and maintenance personnel to assist in fault location.

[0080] In this embodiment, steps two and three work together through a shared semantic space and a feedback mechanism to form a closed-loop optimization, as specifically implemented as follows: Sample priority passing mechanism: The sample priority passing mechanism marks the boundary samples identified in step two as high-priority samples and records the identification information of these samples to form a list of high-priority samples, which is then passed to step three.

[0081] Specifically, the implementation is as follows: In the optimized training set output unit of step two, a priority marking module is added to add high priority markings to the resampled boundary samples; the list of high priority samples is transmitted in real time to the context-aware sample pair construction unit of step three through the data interface to ensure that step three can quickly obtain high priority sample information.

[0082] The purpose of this mechanism is to enable step three to focus on the semantic learning of boundary samples, thereby improving the accuracy of boundary sample vector representation.

[0083] Context-enhanced contrastive learning mechanism: In step three, the context-enhanced contrastive learning mechanism assigns higher sampling probabilities to high-priority samples and mines more diverse context instances.

[0084] Specifically, in the context-aware sample pair construction unit, a sampling weight coefficient is set, with the sampling weight of high-priority samples being twice that of ordinary samples, ensuring that high-priority samples account for a higher proportion in the training batch; when collecting context instances, for high-priority samples, the coverage of the context environment is expanded to mine more instances in different scenarios and construct more diverse positive sample pairs.

[0085] The purpose of this mechanism is to accelerate the calibration of high-priority sample vector representations and improve the model's ability to semantically distinguish boundary samples.

[0086] Feature space collaborative update mechanism: After updating the log template vector in step three, the feature space collaborative update mechanism feeds back the updated vector representation to step two, affecting the calculation of the sample isolation path depth in the next round.

[0087] Specifically, in the adaptive vector output unit of step three, a vector feedback interface is added to feed back all updated log template vectors to the isolated forest model construction unit of step two in real time. When constructing the isolated forest in the next round of step two, the fed-back log template vectors are used as the feature input of the samples to replace the original features, and the isolation path depth and sampling weight of each sample are recalculated.

[0088] The role of this mechanism is to achieve collaborative optimization of the feature space in steps two and three, enabling sample selection and semantic learning to mutually promote each other and improve the adaptive capability of the entire system.

[0089] This embodiment achieves anomaly detection in server logs through the above five steps and collaborative mechanism. Compared with the prior art, the beneficial effects of this embodiment are as follows: By using isolated forest dynamic sample selection, the training weight of boundary samples is highlighted, and redundant information of simple normal samples is reduced, making the training dataset more representative and improving the model's ability to identify boundary samples. Online contrastive learning semantic adaptation can continuously update the vector representation of log templates, enabling semantic features to adapt to the dynamic changes in the system's operating environment and preventing the model from failing due to changes in data distribution. The collaborative mechanism of steps two and three forms a closed-loop optimization, with sample selection and semantic learning mutually promoting each other, further improving the model's adaptability and detection accuracy. The entire method does not require frequent manual retraining of the model, reducing operation and maintenance costs.

[0090] Example 2 like Figures 1 to 3As shown, this embodiment, as a supplementary optimization to Embodiment 1, adapts each step of the method to different scenarios with varying server cluster sizes, log generation rates, and the intensity of system dynamic changes, while maintaining the core technical framework. This further enhances the flexibility, accuracy, and engineering feasibility of anomaly detection. This embodiment remains applicable to scenarios with high requirements for real-time performance and robustness in log anomaly detection, such as large-scale distributed server clusters, cloud computing data centers, and core financial business servers. The following detailed explanation of the implementation process further elaborates on this concept.

[0091] This basic implementation method corresponds to the core technical solution of claims 1 to 6. It supplements key implementation details based on embodiment one to ensure the integrity and reproducibility of the technical solution.

[0092] Server log data preprocessing: When the log parsing unit uses a template-based parsing method, an adaptive string alignment mechanism is introduced to address scenarios with slight differences in log format. Specifically, the original log is first segmented, and fixed fields such as timestamps and server identifiers are extracted as anchors. Then, the log text is aligned based on these anchors to reduce parsing errors caused by minor fluctuations in log format. For example, in logs generated by different server instances, when some static text has part-of-speech variations or minor word order adjustments, anchor alignment can still accurately separate dynamic parameters from static text and extract a unified log template.

[0093] The sliding window used in the sequence construction unit has an initial preset length determined based on the median length of historical log sequences, and is further calibrated in conjunction with the business scenario in which the logs were generated. For example, for transaction servers, log sequences are usually strongly correlated with the transaction process and have a relatively fixed length, so the preset window length can be appropriately reduced; for operation and maintenance monitoring servers, log sequences may fluctuate significantly due to changes in monitoring frequency, so the preset window length can be appropriately increased to ensure complete coverage of behavioral patterns.

[0094] After performing zero-padding, the dynamic zero-padding unit not only masks the zero-padding positions using a masking mechanism, but also normalizes the zero-padding sequence. Specifically, it maps the element values ​​of each log template vector in the sequence to the [0,1] interval to avoid gradient imbalance during model training due to differences in the original vector value ranges, further reducing the indirect impact of zero-padding noise. Dynamic Sample Selection for Isolation Forests: When determining the total number of isolated trees, the construction unit of the isolation forest model introduces a sample complexity evaluation mechanism. This can be understood as the sample complexity being jointly determined by the diversity of log templates and the volatility of parameter sequences. By statistically analyzing the number of unique log templates and the variance of parameter values, sample complexity is categorized into low, medium, and high levels. For low-complexity sample sets, the total number of isolated trees is 100; for medium-complexity sample sets, it is 150; and for high-complexity sample sets, it is 250, ensuring the adaptability of the isolation forest to data of different complexities.

[0095] When calculating path depth, the sample path depth calculation unit introduces a path depth truncation mechanism for isolated trees with excessive depth. Specifically, a maximum path depth threshold is set. When the path depth of a sample in an isolated tree exceeds this threshold, the threshold is directly used as the path depth of the sample in that isolated tree, thus avoiding distortion in the calculation of average isolated path depth due to individual abnormal isolated trees.

[0096] In the sample classification and resampling unit, the threshold and The determination of the isolation path depth is based not only on the quantiles of the average isolation path depth of the samples, but also on iterative calibration using historical anomaly detection results. For example, if there are some misclassified normal samples in the historical data, and their average isolation path depth is close to... Then adjust appropriately. If there are some missed abnormal samples, and their average isolation path depth is close to... Then adjust it appropriately. This improves the accuracy of sample classification.

[0097] Online comparative learning for semantic adaptation: The pre-trained language model used for the log template vector initialization unit, in addition to the BERT-base model, can also be selected based on the length of the log text. For scenarios where the static text of the log template is short, the DistilBERT model can be used to improve computational efficiency while ensuring semantic representation. For scenarios where the log template contains complex business descriptions and is long, the BERT-large model can be used to enhance the ability to capture semantic information.

[0098] The context-aware sample pair construction unit introduces a time window constraint when collecting log instances from different contexts. Specifically, only log instances generated within the last 72 hours are collected as the data source for sample pair construction, avoiding excessive differences in context due to a large time span and ensuring the semantic consistency of positive sample pairs. Simultaneously, to avoid sample pair duplication, a sample pair cache pool is established; used sample pairs are not reused within 24 hours, ensuring the diversity of training data.

[0099] In the online contrastive loss calculation and vector update unit, the encoder network's activation function, besides ReLU, can be replaced with the LeakyReLU function in scenarios with sparse input data distribution to avoid neuron failure. Simultaneously, the optimizer's learning rate is dynamically adjusted using a cosine annealing strategy, with an initial learning rate set to... The learning rate is reduced to half of its current value every 500 iterations to balance the model's convergence speed and generalization ability.

[0100] Deep anomaly detection model training: In model selection and initialization, in addition to the LSTM model, the Gated Recurrent Unit (GRU) can also be selected as the basic sequence model. The GRU model has fewer parameters than the LSTM model, and in scenarios with limited server resources, it can reduce computational overhead while maintaining the ability to capture temporal dependencies. During model initialization, the number of hidden layer units can be dynamically adjusted according to the total number of log templates. The more log templates there are, the more hidden layer units are added to ensure the model's expressive power.

[0101] The sequence pattern learning unit uses a time-series partitioning method, rather than random partitioning, when dividing the training and validation sets. Specifically, the preprocessed log data is sorted chronologically, with the first 70% used as the training set and the last 30% as the validation set. This better aligns with the application logic of training models based on historical data and detecting future data in real-world scenarios, thus improving the actual deployment performance of the model.

[0102] In the anomaly score definition unit, in addition to calculating the anomaly score based on the log-likelihood value, a moving average mechanism is also introduced to smooth the anomaly score. Specifically, when calculating the anomaly score of the current log sequence, the average of the anomaly scores of the previous 5 adjacent log sequences is taken to avoid false detections caused by accidental fluctuations in a single log sequence and to improve the stability of anomaly detection.

[0103] Anomaly Detection and Output: The determination of the preset threshold for anomaly scores is based not only on the quantiles of anomaly scores in the normal log sequences of the validation set, but also adjusted according to the false positive rate requirements of the business scenario. For example, for core financial business servers, the false positive rate is required to be below 0.1%, so the preset threshold is adjusted to the 99.9th quantile of the anomaly scores in the normal log sequences of the validation set; for ordinary office servers, the false positive rate can be relaxed to 1%, so the threshold is adjusted to the 99th quantile, balancing detection accuracy and operational costs.

[0104] When outputting the detection results, in addition to basic information such as abnormal status and abnormal score, the results of abnormal feature analysis will also be provided. Specifically, the log template with the largest deviation from the normal pattern in the abnormal log sequence is extracted, and information such as the frequency of occurrence and context of the template is marked to assist maintenance personnel in quickly locating the cause of the fault.

[0105] The collaborative mechanism between steps two and three: In the sample priority passing mechanism, in addition to marking boundary samples as high priority, the abnormal tendency coefficient of the boundary samples is also recorded. This coefficient is calculated based on the degree of fluctuation of the sample's parameter sequence and the similarity with historical abnormal samples. The higher the coefficient, the closer the sample is to the abnormal pattern, and the higher the priority weight it receives when passing to step three.

[0106] In the context-enhanced contrastive learning mechanism, business scenario labels are introduced when mining diverse context instances for high-priority samples. Specifically, scenarios are divided according to the business type corresponding to the logs, ensuring that the context instances collected for high-priority samples cover the main scenarios of their respective businesses, thereby improving the business adaptability of vector representations.

[0107] In the feature space collaborative update mechanism, the feedback frequency of the log template vector is dynamically adjusted according to the system update frequency. For example, when the server system is in a stable operation phase without version updates or business changes, the feedback frequency is set to once every 5,000 new logs received; when the system is in an update and iteration phase, the feedback frequency is increased to once every 1,000 new logs received, ensuring the collaborative timeliness of sample selection and semantic learning.

[0108] Optimized Implementation Method 1: Dynamic Sliding Window Adjustment For scenarios where the log generation rate varies significantly across different time periods, the sequence construction unit in step one is optimized to dynamically adjust the sliding window length, thereby improving the adaptability of sequence modeling.

[0109] The specific implementation process is as follows: A log activity evaluation metric is introduced, defined as the number of log entries generated by the server per unit of time. Log activity is statistically analyzed in real time using a sliding time window, with three activity thresholds: low activity threshold, medium activity threshold, and high activity threshold. When the statistical log activity is below the low activity threshold, it indicates that few logs are being generated, and the sliding window length is adjusted to 1.5 times the base length to ensure each sample contains sufficient behavioral information. When the log activity is between the low and medium activity thresholds, the base window length remains unchanged. When the log activity is above the high activity threshold, it indicates that log generation is dense, and the sliding window length is adjusted to 0.7 times the base length to avoid a single sample containing too much redundant information, thus improving model training and detection efficiency.

[0110] The purpose of this optimized implementation is to adapt the sliding window length to the dynamic changes in the log generation rate, ensure the integrity of sample information when the logs are sparse, improve processing efficiency when the logs are dense, and further cover the flexible application scenarios of the "sliding window mechanism" in claim 1.

[0111] Optimized Implementation Method 2: Optimization of Isolation Tree Feature Selection For scenarios with high feature dimensions in log data, the isolated forest model building unit in step two is optimized to improve the efficiency of isolated tree construction and the accuracy of path depth calculation.

[0112] The specific implementation process is as follows: In the feature selection stage of isolated tree construction, a feature importance evaluation mechanism is introduced. First, variance analysis is performed on the features of each dimension of the log template vector to calculate the variance contribution of each feature. The variance contribution reflects the feature's ability to distinguish sample differences. Then, only the features with the top 30% variance contribution are selected as candidate features for isolated tree construction, replacing the method of randomly selecting all features. In the subsequent selection of split points, random selection is performed only within the value range of the candidate features.

[0113] The purpose of this optimized implementation is to reduce the interference of irrelevant features on the construction of the isolated tree, improve the ability of the isolated tree to distinguish sample differences, make the calculation of the sample isolation path depth more accurate, and thus improve the classification accuracy of boundary samples and simple normal samples, covering the optimized implementation scenario of "isolated forest algorithm analysis" in claim 1.

[0114] Optimized Implementation Method 3: Hard Negative Sample Mining To address the scenario where semantically similar log templates are easily confused, the context-aware sample building unit in step three is optimized by introducing a hard negative sample mining strategy to improve the discriminative power of semantic vectors.

[0115] The specific implementation process is as follows: In the negative sample pair construction stage, instead of randomly selecting instances of different log templates, the semantic similarity between candidate negative samples and positive samples is calculated first; a semantic similarity threshold is set, and candidate negative samples with semantic similarity higher than the threshold are selected as hard negative samples; in the negative sample pairs corresponding to each positive sample pair, hard negative samples account for no less than 40%, and the remaining negative samples are supplemented by random selection. In this way, the contrastive learning process focuses more on log templates that are semantically similar but different in category, enhancing the model's ability to distinguish subtle semantic differences.

[0116] The purpose of this optimized implementation is to improve the training efficiency of contrastive learning, make the vector representation of log templates more distinguishable in semantic space, especially suitable for server environments with a large number of semantically similar log templates, and further cover the optimization scenario of constructing "negative sample pairs" in claim 1.

[0117] Optimized Implementation Method 4: Dynamic Adjustment of Cooperative Feedback Frequency For scenarios with varying intensity of dynamic changes in the system, the collaborative mechanism between steps two and three is optimized to achieve dynamic adjustment of the vector feedback frequency and improve the timeliness of closed-loop optimization.

[0118] The specific implementation process is as follows: A system change intensity assessment index is introduced, which is calculated based on a combination of server version update records, business configuration change records, and the number of new / deleted log templates. When a server version update or business configuration change is detected, the system change intensity is marked as high; when the number of new / deleted log templates exceeds a preset threshold, the system change intensity is marked as medium; otherwise, it is marked as low. Different vector feedback frequencies are set for different system change intensities: under high change intensity, vector feedback is triggered once every 500 new logs received; under medium change intensity, it is triggered once every 2000 new logs received; and under low change intensity, it is triggered once every 5000 new logs received.

[0119] The purpose of this optimized implementation is to make the collaborative optimization of sample selection and semantic learning more in line with the dynamic changes of the system, accelerate the closed-loop optimization speed when the system changes drastically, reduce the computational overhead when the system is stable, improve the engineering practicality of the whole method, and cover the optimization implementation scenario of "shared semantic space and feedback mechanism" in claim 6.

[0120] This embodiment ensures the integrity and reproducibility of the core technical solution of the claims through a basic implementation method, and covers the flexible adaptation requirements under different application scenarios through four optimized implementation methods, further verifying the versatility of the technical solution of the present invention. Compared with the prior art, this embodiment not only maintains the core advantages of adaptive system environment dynamic changes and continuous learning, but also improves engineering feasibility and scenario adaptability through detailed optimization, effectively reducing the deployment difficulty and operation and maintenance costs in different server environments, and significantly improving the accuracy and stability of anomaly detection.

[0121] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A deep learning-driven method for detecting server log anomalies, characterized in that, Includes the following steps: Step 1: Server log data preprocessing, used to parse the raw unstructured server logs, convert each log into a log template and parameter sequence, and process the log sequence into fixed-length segmented data through a sliding window mechanism; Step 2: Dynamic sample selection for isolated forest. This step is used to perform isolated forest algorithm analysis on the preprocessed log data, calculate the isolation path depth of each log data sample, and classify and resample the samples based on the isolation path depth. Simple normal samples with longer isolation path depths are downsampled, while boundary samples with shorter isolation path depths are oversampled to generate a dynamically balanced training dataset. Step 3: Online contrastive learning semantic adaptation is used to receive a dynamically balanced training dataset and continuously update the vector representation of the log template using an online contrastive learning mechanism. Instances of the same log template in different contexts are used as positive sample pairs, and instances of different log templates are used as negative sample pairs. By optimizing the contrastive learning loss function, the vector representations of positive sample pairs are made closer in semantic space, while the vector representations of negative sample pairs are made further apart, thereby obtaining a log template vector that is adapted to the current system operating environment. Step 4: Training the deep anomaly detection model. This step involves training a deep sequence model to learn normal log sequence patterns using a dynamically balanced training dataset and an adaptive log template vector. Step 5, Anomaly Detection and Output, is used to input the new log sequence to be detected into the trained deep anomaly detection model after passing through Step 1 and Step 3, calculate the anomaly score and compare it with a preset threshold, and finally output the anomaly detection result.

2. The deep learning-driven server log anomaly detection method according to claim 1, characterized in that, Step one specifically includes: The log parsing unit is used to separate the dynamic parameter part from the static text part in the original log entry using a template-based parsing method, and extract a unified log template. A sequence construction unit is used to process variable-length log sequences using an improved sliding window, wherein the sliding window slides along the log sequence at a preset length and uses the log template sequence within the window as an input sample for the model; The dynamic zero-padding unit is used to pad log sequence fragments that are shorter than the preset window length with zeros to ensure that all input data are standardized to a fixed dimension. The zero-padding operation is performed at the end of the sequence.

3. The deep learning-driven server log anomaly detection method according to claim 1, characterized in that, Step two specifically includes: The isolated forest model building unit is used to build multiple isolated trees based on the input log data to form an isolated forest, where each isolated tree grows by randomly selecting features and split points; The sample path depth calculation unit is used to calculate the number of edges traversed from the root node to the leaf node of each log data sample, which is defined as the isolation path depth of the sample. The average of the path depths calculated in all isolated trees is taken as the final isolation path depth of the sample. The sample classification and resampling unit is used to perform classification and resampling operations based on the final isolation path depth of the samples. Its specific process is defined by the following mathematical formulas and rules: in, Indicates sample In the Path depth in an isolated tree This represents the total number of isolated trees. Indicates sample Average isolation path depth To determine the path depth threshold for simple samples, To determine the path depth threshold for boundary samples, This is the oversampling factor applied to boundary samples. This is a downsampling factor applied to simple, normal samples; The optimized training set output unit is used to resample the samples in the original training dataset according to the sampling weights, generate a dynamically balanced training dataset that focuses on boundary samples, and output it to step three.

4. The deep learning-driven server log anomaly detection method according to claim 1, characterized in that, Step three specifically includes: The log template vector initialization unit is used to generate an initial semantic vector representation for each unique log template using a pre-trained language model; The context-aware sample pair building unit is used to collect instances of each log template in different context environments from the log stream generated by the system in real time. The context environment includes different time periods, different server instances, and different user sessions. Instances of the same template in different contexts are built into positive sample pairs, and instances of randomly selected different templates are built into negative sample pairs. The online contrastive loss calculation and vector update unit is used to optimize the vector representation of the log template through online contrastive learning of the loss function. The loss function is configured to increase the similarity of the vector representations between positive sample pairs and decrease the similarity of the vector representations between negative sample pairs through the optimization process, while maximizing the distance between negative sample pairs. The specific form of the loss function is as follows: in, This indicates the number of sample pairs in the training batch. This indicates the number of negative sample pairs. Indicates sample Vector representation after encoder network Indicates and The sample vectors that constitute positive sample pairs Indicates the first A vector representation of a negative sample. This represents the temperature coefficient, used to adjust the sharpness of the probability distribution; An adaptive vector output unit is used to provide the updated log template vector to step four.

5. The deep learning-driven server log anomaly detection method according to claim 1, characterized in that, Step four specifically includes: The model selection and initialization unit is used to select the Long Short-Term Memory (LSTM) network as the base sequence model and initialize its parameters. An adaptive vector input unit is used to take the log template vector updated in step three as the input feature of the deep sequence model. The sequence pattern learning unit is used to learn normal behavior patterns in log sequences through a deep sequence model, where the model takes the vectors of one or more previous log templates as input and predicts the conditional probability distribution of the next log template vector. Anomaly score definition unit is used to define anomaly scores for new log sequences after model training is completed, based on the accuracy of model prediction. The anomaly score is calculated based on the log-likelihood value of the actual log template in the conditional probability distribution predicted by the model.

6. The deep learning-driven server log anomaly detection method according to claim 1, characterized in that, Steps two and three are coordinated through a shared semantic space and a feedback mechanism, specifically including: A sample priority passing mechanism is used to mark the boundary samples identified in step two as high-priority samples and pass their list to step three. A context-enhanced contrastive learning mechanism is used to assign higher sampling probabilities to high-priority samples in step three. The feature space collaborative update mechanism is used to feed back the updated vector representation to step two after updating the log template vector in step three, which affects the calculation of the sample isolation path depth in the next round of pre-screening.

7. A deep learning-driven server log anomaly detection system, applicable to the deep learning-driven server log anomaly detection method according to any one of claims 1 to 6, characterized in that, The system includes: The data acquisition and preprocessing module is used to collect raw log data from the server cluster in real time and perform log parsing, sequence construction and standardization operations. The dynamic sample selection module is used to run the Isolation Forest algorithm, realize the dynamic selection and resampling of training samples, and output the optimized training set. The semantic adaptive module is used to perform online contrastive learning and continuously update the vector representation of the log template to adapt to changes in the system environment; The model training module is used to train and update the deep anomaly detection model using the optimized training set and adaptive vectors. The real-time detection and alarm module is used to receive new server log streams, use the trained model to perform anomaly detection, and generate alarm information when an anomaly is detected. The system control and scheduling module is used to coordinate the data flow and control flow between various modules.

8. A computer-readable storage medium, applicable to the deep learning-driven server log anomaly detection method according to any one of claims 1 to 6, wherein a computer program is stored thereon, characterized in that, When the computer program is executed by the processor, it performs steps one through five.

9. A computer device, applicable to the deep learning-driven server log anomaly detection method according to any one of claims 1 to 6, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it performs steps one through five.

Images