Log abnormal behavior detection method and system based on periodic pattern mining and incremental learning

Abstract

The application discloses a log abnormal behavior detection method and system based on period pattern mining and incremental learning, and relates to the technical field of system anomaly monitoring. The method comprises the following steps: obtaining original log data and performing a preprocessing operation; extracting main period frequency components to form a period set through time frequency analysis, dividing period stable part sequences and transient change part sequences, and modeling by using a period modeling mechanism and a transient modeling mechanism respectively; constructing a joint prediction model, introducing Monte Carlo Dropout to estimate uncertainty, and generating a prediction value by using Bayesian weighting; determining an anomaly through period residual, transient residual and overall residual, and identifying an abnormal causal path by combining with transfer entropy analysis; performing incremental training by using a memory sample driven playback and distillation mechanism, and dynamically adjusting modeling structure parameters. The application has the capabilities of period rule modeling, non-steady state behavior expression, prediction fusion, abnormal causal identification and model continuous learning.

Description

Technical Field

[0001] This invention relates to the field of system anomaly monitoring technology, and in particular to a method and system for detecting log anomaly behavior based on periodic pattern mining and incremental learning. Background Technology

[0002] Logs, as crucial records of system operations, reflect key information such as system operations, abnormal states, and fault evolution, and have become a core data source for ensuring stable system operation and supporting intelligent operation and maintenance decisions. Actual log data exhibits characteristics such as high frequency, large scale, and strong heterogeneity. Furthermore, abnormal behaviors often manifest as a combination of periodic patterns and abrupt, non-periodic changes, making traditional static rule matching, single-model detection, or general reconstruction methods insufficient to meet the multiple demands of complex system logs, including periodic dynamics, anomaly drift, behavioral diversity, and semantic interpretation.

[0003] On the one hand, logs contain numerous periodic operation patterns (such as scheduled tasks and batch processing workflows). The stability and interference of these periodic behaviors have a significant impact on anomaly identification results. If periodic patterns cannot be accurately extracted and modeled, false positives or false negatives are highly likely. On the other hand, sudden anomalies are often accompanied by nonlinear transient evolution, with diverse behavioral forms, complex impact paths, and susceptibility to concept drift and changes in contextual conditions. Traditional static models struggle to adapt to new anomaly patterns in a timely manner and lack support for model updates, parameter adjustments, and structural reconstruction.

[0004] In addition, existing anomaly detection schemes generally suffer from insufficient interpretability, failing to provide a clear causal path for identified anomalies, thus limiting the system's response efficiency to complex faults. Summary of the Invention

[0005] To address issues such as inaccurate log behavior modeling, untimely anomaly detection, unclear alarm localization, and difficulty in adapting models to continuously changing environments, this invention proposes a log anomaly behavior detection method and system based on periodic pattern mining and incremental learning. Targeting the massive log data generated by market supervision platforms, this method extracts periodic patterns and applies incremental learning strategies to achieve accurate detection of anomalies and adaptive model evolution.

[0006] The present invention achieves the above objectives through the following technical solutions:

[0007] A log anomaly behavior detection method based on periodic pattern mining and incremental learning, the method comprising:

[0008] Obtain raw log data from the market supervision platform and perform preprocessing operations such as timestamp standardization, log template normalization, and key field extraction to construct a log indicator sequence arranged in chronological order as the input sequence;

[0009] Periodic pattern mining is performed on the input sequence. The frequency components of the first N energy spectrum peaks are extracted using time-frequency analysis to form a periodic set of normal behavior. Based on the periodic set, the input sequence is divided into a periodically stable part and a transient part.

[0010] The periodically stable part of the sequence is encoded using a periodic modeling mechanism, which employs a linear encoder and introduces an adaptive linear constraint driven by a periodic stability score to enhance consistency across time periods.

[0011] The transient change sequence is modeled using a transient modeling mechanism, which includes performing variational mode decomposition on the transient change sequence to extract several intrinsic mode function components, selecting high-frequency unsteady components as inputs to a fixed-dimensional Koopman dynamic encoder-decoder structure, introducing Koopman linear operators to model state evolution in the state space output by the encoder, and performing online estimation through dynamic mode decomposition.

[0012] The periodic modeling mechanism and the transient modeling mechanism are combined into a joint prediction model. Monte Carlo Dropout is introduced to estimate the prediction uncertainty, and the final prediction value is generated based on Bayesian weighting.

[0013] The real-time log indicator sequence is input into the joint prediction model to generate predicted values. The periodic residual, transient residual and overall residual are calculated and compared with the preset anomaly threshold. If the anomaly threshold is exceeded, an anomaly alarm is output and residual propagation analysis is performed. The transfer entropy is used to identify the anomaly causal path and potential root cause.

[0014] Preferably, the method of extracting the frequency components of the first N energy spectral peaks using time-frequency analysis to form a periodic set of normal behavior includes:

[0015] The input sequence is divided into slices using a sliding window of fixed length, and a fast Fourier transform is performed on the log subsequence within each window to obtain the corresponding frequency domain amplitude spectrum.

[0016] In the frequency domain amplitude spectrum corresponding to each window, the top N frequency components are selected according to their amplitude to form a local candidate frequency set.

[0017] Frequency alignment is performed on the local candidate frequency sets extracted within all windows. The frequency of each frequency value in all sliding windows and the standard deviation of the amplitude change of the frequency value in different windows are calculated to measure the frequency energy stability.

[0018] A comprehensive significance scoring function is constructed based on the occurrence frequency of each frequency and its corresponding energy stability, expressed as:

[0019] S(f i )=α·Norm(c i )+β·(1-Norm(σ i ));

[0020] In the formula, f i Let S(f) be the i-th frequency in the local candidate frequency set. i () represents the frequency f i The overall significance score; c i For frequency f i Frequency of occurrence in all sliding windows; σ i For frequency f i Standard deviation of spectral energy in different windows; Norm(·) represents the normalization operation; α,β∈[0,1] are adjustable weight coefficients, and α+β=1;

[0021] Sort the scores in descending order and select the top M frequency values ​​based on the overall significance score to form the final period set.

[0022] Preferably, the input sequence is divided into a periodically stable portion and a transient portion based on the period set, specifically including:

[0023] For each frequency value in the period set, based on the drift range of the frequency value in each sliding window or the entropy value of the spectral energy distribution, a bandwidth interval centered on the frequency value is set as a frequency matching window to divide the frequency domain components.

[0024] Perform a global fast Fourier transform on the complete input sequence to obtain all frequency components of the input sequence in the frequency domain, and determine whether the frequency value of each frequency component falls within any frequency matching window:

[0025] If the frequency value of a frequency component belongs to any frequency matching window, then the frequency component is assigned to the frequency set corresponding to the periodically stable part of the sequence.

[0026] If the frequency value of a frequency component does not belong to all frequency matching windows, then the frequency component is assigned to the frequency set corresponding to the transient change sequence.

[0027] Frequency domain filtering and inverse transform operations are performed on the frequency sets corresponding to the periodically stable part of the sequence and the frequency sets corresponding to the transient part of the sequence, respectively, to obtain the reconstructed periodically stable part and transient part of the sequence.

[0028] Preferably, the periodically stable portion of the sequence is encoded using a periodic modeling mechanism, the method specifically including:

[0029] The periodically stable partial sequence is input into a linear encoder with time-series modeling capability. The linear encoder performs a linear transformation on the current input time slice to generate a representation vector, and constructs a recursive encoding result by additively combining it with the historical representation vector.

[0030] The comprehensive significance score calculated from each frequency value in the period set is used as the period stability score. The regularization strength or residual retention ratio in the linear coding process is adjusted based on the period stability score. If the period stability score is greater than the set stability threshold, the regularization strength of the linear term is increased; if the period stability score is lower than the stability threshold, the linear constraint is reduced.

[0031] A mapping relationship between key fields and periodic frequency values ​​is constructed, and a symbolic periodic label is generated based on the mapping result. The symbolic periodic label is then fused with the numerical representation of the periodically stable part of the sequence after vector encoding.

[0032] The original time index corresponding to each frequency value is recorded in the period set as the period start reference point, and the original time index corresponding to each time step is retained during the reconstruction of the period stable part of the sequence. The relative displacement is calculated based on the difference between the current time step index and the period start reference point. The relative displacement is used to generate a phase displacement vector to input the period position encoding process.

[0033] Preferably, the transient change portion of the sequence is modeled using a transient modeling mechanism, the method specifically including:

[0034] Variational mode decomposition is performed on the transient change sequence to obtain multiple intrinsic mode function components;

[0035] For each intrinsic mode function component, the dominant frequency is calculated, and all intrinsic mode function components with dominant frequencies higher than a preset minimum frequency threshold are screened out by frequency spectrum analysis to construct a set of high-frequency unsteady components;

[0036] The set of high-frequency unsteady components is input into a fixed-dimensional Koopman dynamic encoder-decoder structure. The set of high-frequency unsteady components is mapped to the Koopman observation space through encoding operations. A fixed-dimensional Koopman linear operator is constructed in the Koopman observation space to model the temporal evolution path of the state in the Koopman observation space.

[0037] The state transition structure of the Koopman linear operator is estimated online using the dynamic mode decomposition method, and the parameters of the Koopman linear operator are updated in real time according to the temporal change law of the observed state within the sliding time window.

[0038] In the modeling process of the Koopman linear operator, Lyapunov stability constraints are introduced to enhance the stability and convergence of the state evolution path by controlling the spectral radius of the state transition structure or constructing a Lyapunov function that is monotonically decreasing on the state trajectory.

[0039] The state modeling results generated in the Koopman observation space are output as the modeling representation of the transient modeling mechanism, and used for fusion and anomaly detection in the subsequent joint prediction model.

[0040] Preferably, the periodic modeling mechanism and the transient modeling mechanism are combined to form a joint prediction model, including:

[0041] The periodically stable part of the sequence generates corresponding prediction outputs based on the periodic modeling mechanism, while the transient part of the sequence generates corresponding prediction outputs based on the transient modeling mechanism.

[0042] Monte Carlo Dropout is introduced into the prediction process of both the periodic modeling mechanism and the transient modeling mechanism to perform multiple forward propagations, generating multiple prediction output samples. Based on the multiple prediction output samples, the prediction mean and prediction variance of the periodic modeling mechanism and the transient modeling mechanism are calculated for uncertainty estimation.

[0043] A Bayesian weighted strategy is constructed based on the prediction variance using both periodic and transient modeling mechanisms. The weights are inversely proportional to the prediction variance to generate the final predicted value. The fusion formula is as follows:

[0044]

[0045] in, and These are the prediction outputs of the periodic modeling mechanism and the transient modeling mechanism, respectively. and The prediction variances are for the periodic modeling mechanism and the transient modeling mechanism, respectively.

[0046] A confidence gating mechanism is introduced in the prediction fusion process. When the difference between the prediction variances of the periodic modeling mechanism and the transient modeling mechanism exceeds the set difference threshold, Bayesian weighting is skipped, and the prediction output of the modeling mechanism with the smaller prediction variance is directly selected as the final prediction value.

[0047] The joint prediction model continuously performs dynamic prediction output, uncertainty estimation, and Bayesian weighted fusion during operation to support the calculation of periodic residuals, transient residuals, and global residuals.

[0048] Preferably, the method for calculating the periodic residual, transient residual, and overall residual and comparing them with a preset anomaly threshold includes:

[0049] Align the periodically stable sequence with the prediction output of the periodic modeling mechanism by time, and calculate the periodic residual at each time point, using the following formula:

[0050] Align the transient variation sequence with the prediction output of the transient modeling mechanism by time, and calculate the transient residuals at the corresponding time points, using the following formula:

[0051] Align the complete input sequence with the final predictions of the joint prediction model by time, and calculate the overall residuals at the corresponding time points, using the following formula:

[0052] In the formula, δ (t) These are the periodic residual, transient residual, and global residual at time point t, respectively. y (t) These are the actual observed values ​​of the periodically stable portion of the sequence, the transient portion of the sequence, and the log index sequence at time point t, respectively. The predicted outputs of the periodic modeling mechanism and the transient modeling mechanism at time point t are respectively. This represents the predicted value output by the joint prediction model at time point t.

[0053] For the periodic residual, transient residual, and global residual, calculate the absolute value of the residual within a sliding window of fixed length. |δ (t) | Moving average sliding standard deviation and the slope of the residual change And construct a multidimensional residual feature vector R (t) ;

[0054] A direction labeling function D is constructed based on the positive and negative directions at each time point of the periodic residual, transient residual, and global residual. (t) ;

[0055] Define the residual directionality enhancement anomaly scoring function as follows:

[0056]

[0057] In the formula, λ j , λ 13 These are the preset weighting coefficients; R is the multidimensional residual eigenvector. (t) The j-th dimension component;

[0058] Anomaly score value A (t) If the error is compared with a preset anomaly threshold, an anomaly alarm is triggered and the corresponding residual dimension and directionality are recorded.

[0059] Preferably, the method for performing residual propagation analysis, which uses transfer entropy to identify anomalous causal paths and potential root causes, includes:

[0060] Identify abnormal time points where the abnormal score value exceeds the preset abnormal threshold, and form an abnormal residual trigger event sequence;

[0061] Using the abnormal time point as the anchor window, extract segments from all log indicator sequences that overlap with or lag behind the abnormal time point to construct a candidate set of residual variables;

[0062] For the candidate residual variables (X,Y) in the candidate residual variable set, the time-delay-sensitive transfer entropy calculation method is used within a set lag time window τ∈[τ min ,τ max Within [the specified area], search for the maximum transition entropy value:

[0063] In the formula, This represents the transfer entropy of X to Y with a time lag of τ units; τ min τ max These are the minimum and maximum values ​​of the lag time window, respectively; the maximum transfer entropy value... Used for causal strength, a residual causal graph model is constructed, where nodes in the graph are log metrics, edge weights are maximum transition entropy values, and edge attributes record propagation lag.

[0064] The maximum allowed depth of the propagation path is limited in the causal graph, and a link credibility score C is calculated for each causal edge. X→Y :

[0065]

[0066] In the formula, γ1 and γ2 are the residual values ​​of the source index X at abnormal time points, and γ1 and γ2 are the weighting coefficients.

[0067] Calculate the root cause priority scoring function for each node, using the following formula:

[0068]

[0069] In the formula, Q m D is the root cause score for node m. m Let m be the out-degree of node m; τ is the maximum transition entropy value from node m to node n; m→n For propagation delay; C m→n Link credibility score; η1, η2, η3, and η4 are weighting coefficients;

[0070] Root cause score Q m Exceeding the preset threshold Q thThe nodes were identified as potential root causes;

[0071] The node identifiers of potential root causes, the propagation path structure, and the key transition entropy values ​​are output as part of the anomaly alarm results to support operation and maintenance response and causal analysis decisions.

[0072] Preferably, after generating the prediction output, incremental data is received and incremental training based on playback and distillation is performed to update the model parameters of the periodic modeling mechanism and the transient modeling mechanism, and adjust the structural configuration, specifically including:

[0073] The training dataset is composed of log samples already stored in the memory bank and incremental data. The time decay factor is calculated based on the sample generation time, and weights are assigned to the memory samples.

[0074] When performing incremental training, knowledge-based constraints are applied, specifically: recording the predicted output or intermediate representation of the previous version model for the memorized samples, minimizing the difference between the current model's output for the corresponding samples and the recorded values; identifying important parameters in the periodic modeling mechanism and the transient modeling mechanism, and applying weighted regularization terms to these parameters or keeping their values ​​unchanged.

[0075] The calculation periodic modeling mechanism assesses the modeling confidence of each frequency component in the periodic set and adjusts the number of frequency components retained in the periodic set.

[0076] The dimensions of the Koopman state space are dynamically adjusted based on the magnitude of the prediction error change within the sliding window in the transient modeling mechanism.

[0077] Evaluate the convergence performance of the regularization term within the current window and dynamically update the regularization strength;

[0078] Whether to perform the above structural adjustment operations is determined based on whether the change in verification error or the variance of prediction uncertainty exceeds a preset threshold.

[0079] A log anomaly detection system based on periodic pattern mining and incremental learning, applied to the log anomaly detection method based on periodic pattern mining and incremental learning as described above, the system comprising:

[0080] The log collection and preprocessing module is used to acquire raw log data from the market supervision platform, perform preprocessing operations such as timestamp standardization, log template normalization, and key field extraction, and generate a log indicator sequence arranged in chronological order as the input sequence.

[0081] The periodic pattern mining module is used to perform a fast Fourier transform on the input sequence, extract the frequency components of the first N energy spectrum peaks to form a normal behavior periodic set, and divide the input sequence into a periodically stable part sequence and a transient change part sequence.

[0082] The periodic modeling module is used to model the periodically stable parts of the sequence. It includes a linear encoder driven by the periodic stability score to generate the predictive output of the periodic modeling mechanism.

[0083] The transient modeling module is used to model the transient variation part of the sequence. It includes a variational mode decomposition module and a Koopman encoder-decoder structure, and generates the prediction output of the transient modeling mechanism.

[0084] The joint prediction module is used to estimate the uncertainty of the prediction outputs of the periodic modeling module and the transient modeling module through Monte Carlo Dropout, and then fuse them based on the Bayesian weighting mechanism to generate the final prediction value.

[0085] The anomaly detection module is used to calculate the periodic residual, transient residual and global residual based on the error between the predicted value and the actual observed value, extract multidimensional residual feature vectors, generate anomaly scores, and determine whether to trigger an anomaly alarm.

[0086] The causal identification module is used to perform residual propagation analysis after an abnormal event occurs, construct a residual causal graph based on transfer entropy, and identify potential root cause paths and nodes.

[0087] The memory management module is used to build a memory bank based on prediction entropy or reconstruction error, select samples with high uncertainty and high diversity, and update the memory content.

[0088] The incremental training module is used to perform incremental training that combines replay and distillation based on memory samples and incremental data. It updates the model parameters of the periodic modeling mechanism and the transient modeling mechanism, and dynamically adjusts the number of frequency components, the dimension of the Koopman state space, and the configuration of regularization terms according to the structural confidence and error fluctuation.

[0089] The beneficial effects of this invention are as follows: By introducing a periodic pattern mining mechanism based on frequency analysis, the periodic patterns and non-periodic changes in log indicator sequences are separated, effectively reducing the interference of periodic noise on anomaly identification; by constructing a dual-channel structure of periodic modeling and transient modeling mechanisms, the ability to perceive sudden anomalies is enhanced while maintaining the expressive power of periodic patterns; by introducing adaptive linear constraints driven by periodic stability scores in the periodic modeling mechanism, the model can flexibly adjust the modeling intensity according to the confidence level of periodic behavior, enhancing the ability to model the consistency of periodic structures across time periods; and by introducing variational mode decomposition and a Koopman encoder-decoder structure in the transient modeling mechanism, combined with dynamic mode decomposition methods to analyze the state... Evolutionary operators are used for online estimation, improving the accuracy and scalability of modeling non-steady-state log anomaly behavior. A Monte Carlo Dropout and Bayesian weighting mechanism are introduced through a joint prediction model, implementing a dynamic fusion strategy based on model prediction uncertainty. This adaptively selects the most reliable prediction path under periodic disturbances or transient mutations, enhancing the model's robustness to anomaly pattern drift and structural uncertainty. By decomposing prediction errors into periodic, transient, and global residuals, and analyzing and judging them separately, the granularity and sensitivity of anomaly identification are improved. Furthermore, by combining residual propagation analysis and transfer entropy methods, causal paths and potential root causes among log indicators are identified, enhancing the interpretability and traceability of anomaly alerts. Attached Figure Description

[0090] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0091] in:

[0092] Figure 1 This is a flowchart of the method in an embodiment of the present invention;

[0093] Figure 2 This is a system structure block diagram in an embodiment of the present invention. Detailed Implementation

[0094] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0095] like Figure 1 As shown, this is an embodiment of the present invention, which provides a method for detecting abnormal log behavior based on periodic pattern mining and incremental learning, including the following steps:

[0096] S1: Obtain the raw log data from the market supervision platform and perform preprocessing operations such as timestamp standardization, log template normalization, and key field extraction to construct a log indicator sequence arranged in chronological order as the input sequence.

[0097] S2: Perform periodic pattern mining on the input sequence, and use time-frequency analysis to extract the frequency components of the first N energy spectrum peaks to form a periodic set of normal behavior. Based on the periodic set, the input sequence is divided into a periodically stable part of the sequence and a transient part of the sequence.

[0098] In one embodiment, the method for implementing step S2 includes:

[0099] S2A1: The input sequence is divided into segments using a sliding window of fixed length. Assuming the window length is L and the step size is S, several overlapping or non-overlapping log subsequences are obtained. Fast Fourier Transform (FFT) is performed on the log subsequences within each window to obtain the corresponding frequency domain amplitude spectrum.

[0100] S2A2: In the frequency domain amplitude spectrum corresponding to each window, select the top N frequency components according to their amplitude to form a local candidate frequency set.

[0101] S2A3: Perform frequency alignment on the local candidate frequency set extracted within all windows (classified based on frequency resolution), calculate the frequency of each frequency value in all sliding windows, and the standard deviation of the amplitude change of the frequency value in different windows, to measure frequency energy stability.

[0102] S2A4: Construct a comprehensive significance scoring function based on the occurrence frequency of each frequency and the corresponding energy stability, expressed as:

[0103] S(f i )=α·Norm(c i )+β·(1-Norm(σ i ));

[0104] In the formula, f i Let S(f) be the i-th frequency in the local candidate frequency set. i () represents the frequency f i The overall significance score; c i For frequency f i Frequency of occurrence in all sliding windows; σ i For frequency f iStandard deviation of spectral energy in different windows; Norm(·) represents the normalization operation; α,β∈[0,1] are adjustable weighting coefficients used to balance the importance of frequency occurrence and energy stability, and α+β=1;

[0105] S2A5: Sort the scores in descending order and select the frequency values ​​of the top M in the comprehensive significance score to form the final period set.

[0106] After obtaining the periodic set, this embodiment further divides and reconstructs the log indicator sequence into periodic and aperiodic components in the frequency domain, specifically including:

[0107] S2B1: For each frequency value in the period set The bandwidth range of the frequency is calculated based on the drift range (e.g., maximum and minimum difference) of the frequency value within each sliding window or the entropy value of the spectral energy distribution. Where δ k The bandwidth factor can be set to a fixed value or determined adaptively based on the drift. The bandwidth range centered on this frequency value serves as the frequency matching window, used to divide the frequency domain components.

[0108] S2B2: Perform a global fast Fourier transform on the complete input sequence to obtain its frequency domain representation F(ω), where ω is the discrete frequency coordinate;

[0109] S2B3: For each frequency component ω i ∈F(ω), determine whether it falls within any bandwidth interval W. k (where k represents the k-th frequency), if satisfying The frequency component is then classified as a periodically stable component; otherwise, it is classified as a transient component. This represents the bandwidth interval W1, W2, ..., W corresponding to all M periodic frequencies. M The union of all frequency ranges considered to belong to the periodic frequency;

[0110] Assume the periodic set has three frequency components:

[0111] f1 * =0.1Hz, corresponding to bandwidth W1=[0.08,0.12];

[0112] The corresponding bandwidth W2 = [0.22, 0.28];

[0113] The corresponding bandwidth W3 = [0.48, 0.52];

[0114] but:

[0115] Now there is a frequency component ω i =0.26, since 0.26∈[0.22,0.28]=W2, therefore we have That is, this frequency component is a periodically stable component;

[0116] S2B4: The periodic stable component and the transient component after division are subjected to frequency domain filtering, retaining only the corresponding frequency components and setting the rest to zero. The inverse FFT operation is then performed on the processed spectrum to reconstruct the periodic stable part sequence and the transient part sequence, respectively.

[0117] S3: The periodically stable part of the sequence is encoded using a periodic modeling mechanism. The periodic modeling mechanism uses a linear encoder and introduces an adaptive linear constraint driven by the periodic stability score to enhance consistency across time periods.

[0118] The periodic modeling mechanism aims to model and encode the structural consistency and semantic interpretability of periodically stable sequences, thereby enhancing the model's generalization ability to periodic behavior and its robustness in anomaly detection. In one embodiment, step S3 is implemented by:

[0119] S31: Input the periodically stable part of the sequence into a linear encoder with time-series modeling capabilities. The linear encoder adopts a recursive encoding strategy, which generates a representation vector by performing a linear transformation on the current input time slice and additively combining it with the encoding results of the historical time slices to form a state representation sequence for capturing the evolutionary features of periodic behavior.

[0120] S32: A periodic stability score is introduced to regulate the regularization strength or residual retention ratio in the linear encoding process. The periodic stability score is derived from the comprehensive significance score calculated in the previous periodic pattern mining process. In the linear encoding stage, a stability threshold is set to distinguish periodic segments with different stability levels. If the periodic stability score is greater than the set stability threshold, the regularization strength of the linear term is increased to limit the degree of freedom of the model, thereby maintaining the consistency and reconfigurability of the representation. If the periodic stability score is lower than the stability threshold, the linear constraints are reduced, allowing the local structure to retain slight variation features.

[0121] S33: To enhance the model's semantic understanding of periodic patterns, a symbolic periodic pattern is introduced into the periodic modeling mechanism to assist the modeling path. Specifically, during the log preprocessing stage, periodic-related keywords (such as scheduled tasks, batch processing scripts, etc.) in the log template are identified through key field extraction operations. The mapping relationship between keywords and periodic frequency values ​​is constructed in combination with the frequency analysis results. Based on this mapping, corresponding symbolic periodic labels are generated for the periodic stable part of the sequence. After numerical encoding using the embedding vector method, the labels are fused with the numerical representation of the original periodic stable part of the sequence in the input layer or intermediate representation layer to enhance the semantic interpretability of the model for periodic structures.

[0122] S34: To enable the periodic modeling mechanism to have periodic stage perception capability, the original time index corresponding to each frequency value is recorded as the periodic start reference point during the periodic set generation stage. During the reconstruction of the periodic stable part sequence, the original time index corresponding to each time step is retained. The relative displacement is calculated based on the difference between the current time step index and the periodic start reference point. This relative displacement is used to generate a phase displacement vector as an additional input to the encoder to model the stage characteristics inside the period.

[0123] By leveraging the synergistic effects of periodic stability scoring to regulate the modeling path, symbolic periodic semantic fusion, and periodic displacement sensing mechanism, the embodiments of this invention achieve structural consistency maintenance, cross-time period robust modeling, and enhanced interpretability of periodic behavior, effectively improving the sensitivity and accuracy of periodic behavior anomaly identification.

[0124] S4: The transient change part of the sequence is modeled using a transient modeling mechanism, including performing variational mode decomposition on the transient change part of the sequence to extract several intrinsic mode function components, selecting high-frequency non-steady-state components as inputs to a fixed-dimensional Koopman dynamic encoder-decoder structure, introducing Koopman linear operators to model state evolution in the state space of the encoder output, and performing online estimation through dynamic mode decomposition.

[0125] This embodiment provides a specific implementation process for modeling transient behavior in log data from a market supervision platform, used to construct a transient modeling mechanism in a log abnormal behavior detection system based on periodic pattern mining and incremental learning, including the following steps:

[0126] S41: The transient change sequence extracted from the preprocessed log indicator sequence is used as input data. The periodic stable component has been removed by frequency domain analysis. It mainly contains short-term unstable fluctuations in system operation, such as sudden failures, configuration changes or non-periodic load disturbances. Variational mode decomposition (VMD) is performed on the transient change sequence to obtain multiple intrinsic mode function components.

[0127] In this embodiment, the decomposition order of VMD is set to K=6, the penalty parameter α=2000, and 6 intrinsic mode function (IMF) components are retained, which respectively represent the oscillation components of different frequency bands in the transient sequence;

[0128] S42: Calculate the dominant frequency for each intrinsic mode function component. The dominant frequency is extracted based on the Fourier transform results of each component, calculating the power spectral density function, and selecting the frequency corresponding to the maximum energy as the dominant frequency value. In this embodiment, the minimum threshold for the dominant frequency is set to f. min =0.1Hz, filtering out all dominant frequencies higher than f min The IMF components are used to construct a set of high-frequency non-steady-state components to characterize the mutation behavior or non-steady-state disturbances in the log system.

[0129] S43: Input the set of high-frequency non-steady-state components into a fixed-dimensional Koopman dynamic encoder-decoder structure. This structure includes a two-layer LSTM encoder with a hidden dimension of 64, which is responsible for encoding the input sequence into a state vector of length 16, forming the Koopman observation space.

[0130] S44: Construct a fixed-dimensional Koopman linear operator in the Koopman observation space to model the temporal evolution path of the state in the Koopman observation space; the Koopman linear operator is in the form of a 16×16 state transition matrix, and the initial values ​​are obtained by least squares fitting;

[0131] S45: In subsequent operations, the Dynamic Mode Decomposition (DMD) method is used to update the state transition structure of the Koopman linear operator online. The DMD method uses the encoded state sequence within a sliding time window of T = 30 steps to calculate the state evolution relationship in batches and adjust the value of the state transition matrix in real time accordingly to improve the adaptability to non-stationary dynamic changes.

[0132] S45: During the construction of the state transition structure, Lyapunov stability constraints are introduced. These constraints are implemented by spectral radius control. That is, after each DMD update, the eigenvalues ​​of the state transition matrix are truncated by spectral radius, forcing their maximum modulus to be less than 1, thereby ensuring that the state sequence does not diverge. This operation effectively suppresses system instability caused by high-frequency disturbances.

[0133] Finally, the completed Koopman observation space state evolution results are used as the modeling representation output of the transient modeling mechanism, and are then fused with the outputs of the subsequent joint prediction model and periodic modeling mechanism to participate in the generation of predicted values ​​and anomaly detection and judgment.

[0134] Through the transient modeling mechanism in this embodiment, the system can accurately model short-term high-frequency disturbances in the logs and has structural stability and online adaptability, which significantly improves the detection accuracy of non-periodic abnormal behavior.

[0135] S5: Combine the periodic modeling mechanism and the transient modeling mechanism into a joint prediction model, introduce Monte Carlo Dropout to estimate the prediction uncertainty, and generate the final prediction value based on Bayesian weighting.

[0136] In one specific embodiment, the method for implementing step S5, which constructs a joint prediction model to generate the final predicted value, includes:

[0137] The periodically stable part of the sequence generates corresponding prediction outputs based on the periodic modeling mechanism, while the transient part of the sequence generates corresponding prediction outputs based on the transient modeling mechanism.

[0138] Monte Carlo Dropout is introduced into the prediction process of both the periodic modeling mechanism and the transient modeling mechanism. Multiple forward propagations are performed during the inference phase to generate multiple prediction output samples. The prediction mean and prediction variance of the periodic modeling mechanism and the transient modeling mechanism are calculated based on the multiple prediction output samples for uncertainty estimation.

[0139] A Bayesian weighted strategy is constructed based on the prediction variance using both periodic and transient modeling mechanisms. The weights are inversely proportional to the prediction variance, with those having smaller prediction variances receiving higher weights in the final prediction, thus generating the final predicted value. The fusion formula is as follows:

[0140]

[0141] in, and These are the prediction outputs of the periodic modeling mechanism and the transient modeling mechanism, respectively. and The prediction variances are for the periodic modeling mechanism and the transient modeling mechanism, respectively.

[0142] A confidence gating mechanism is introduced in the prediction fusion process. When the difference between the prediction variances of the periodic modeling mechanism and the transient modeling mechanism exceeds the set difference threshold, Bayesian weighting is skipped, and the prediction output of the modeling mechanism with the smaller prediction variance is directly selected as the final prediction value.

[0143] Furthermore, a covariance adjustment term can be introduced to correct the correlation between the periodic modeling mechanism and the transient modeling mechanism in historical prediction errors, thereby improving the stability of the fusion result and the overall prediction accuracy.

[0144] The joint prediction model continuously performs dynamic prediction output, uncertainty estimation, and Bayesian weighted fusion during operation to support the calculation of periodic residuals, transient residuals, and global residuals.

[0145] S6: Input the real-time log indicator sequence into the joint prediction model to generate predicted values, calculate the periodic residual, transient residual and overall residual and compare them with the preset anomaly threshold. If the anomaly threshold is exceeded, an anomaly alarm is output and residual propagation analysis is performed. The transfer entropy is used to identify the anomaly causal path and potential root cause.

[0146] In one embodiment, step S6 includes:

[0147] S6A1: Align the periodically stable portion of the sequence with the prediction output of the periodic modeling mechanism by time, and calculate the periodic residual at each time point. The formula is as follows:

[0148] S6A2: Align the transient variation sequence with the prediction output of the transient modeling mechanism by time, and calculate the transient residuals at the corresponding time points. The formula is as follows:

[0149] S6A3: Align the complete input sequence with the final predictions of the joint prediction model by time, and calculate the overall residuals at the corresponding time points. The formula is as follows:

[0150] In the formula, δ (t) These are the periodic residual, transient residual, and global residual at time point t, respectively. y (t) These are the actual observed values ​​of the periodically stable portion of the sequence, the transient portion of the sequence, and the log index sequence at time point t, respectively. The predicted outputs of the periodic modeling mechanism and the transient modeling mechanism at time point t are respectively. This represents the predicted value output by the joint prediction model at time point t.

[0151] S6A4: Calculates the absolute value of the residuals within a fixed-length sliding window for the periodic residuals, transient residuals, and global residuals, respectively. |δ (t) | Moving average sliding standard deviation and the slope of the residual change And construct multidimensional residual feature vectors

[0152] S6A5: Construct a direction labeling function D based on the positive and negative directions at each time point of the periodic residual, transient residual, and global residual. (t) D (t)∈{-1,0,1}, where -1 represents negative direction, 0 represents unbiased direction, and 1 represents positive direction;

[0153] S6A6: Define the residual directionality enhancement anomaly scoring function A (t) , is represented as:

[0154]

[0155] In the formula, λ j , λ 13 These are the preset weighting coefficients; R is the multidimensional residual eigenvector. (t) The j-th dimension component;

[0156] S6A7: The abnormal rating value A (t) The error is compared with a preset anomaly threshold. If the anomaly threshold is exceeded, an anomaly alarm is triggered, and the corresponding residual dimension and directionality are recorded.

[0157] Furthermore, the steps for performing residual propagation analysis include:

[0158] S6B1: Identify abnormal time points where the abnormal score value exceeds the preset abnormal threshold, and form an abnormal residual trigger event sequence;

[0159] S6B2: Using the abnormal time point as the anchor window, extract the segments in all log indicator sequences that overlap with or lag behind the abnormal time point, and construct a candidate set of residual variables.

[0160] S6B3: For candidate residual variables (X,Y) in the candidate residual variable candidate set, the time-delay-sensitive transfer entropy calculation method is used within a set lag time window τ∈[τ min ,τ max Within [the specified area], search for the maximum transition entropy value:

[0161] In the formula, This represents the transfer entropy of X to Y with a time lag of τ units; τ min τ max These are the minimum and maximum values ​​of the lag time window, respectively; the maximum transfer entropy value... Used for causal strength, a residual causal graph model is constructed, where nodes in the graph are log metrics, edge weights are maximum transition entropy values, and edge attributes record propagation lag.

[0162] S6B4: Limit the maximum allowed depth of propagation paths in the causal graph and calculate the link credibility score C for each causal edge. X→Y :

[0163] In the formula, γ1 and γ2 are the residual values ​​of the source index X at abnormal time points, and γ1 and γ2 are the weighting coefficients.

[0164] S6B5: Calculate the root cause priority scoring function for each node, using the following formula:

[0165]

[0166] In the formula, Q m D is the root cause score for node m. m Let m be the out-degree of node m; τ is the maximum transition entropy value from node m to node n; m→n For propagation delay; C m→n Link credibility score; η1, η2, η3, and η4 are weighting coefficients;

[0167] S6B6: Root cause score Q m Exceeding the preset threshold Q th The nodes were identified as potential root causes;

[0168] S6B7: Outputs the node identifier of potential root causes, propagation path structure, and key transition entropy values ​​as part of the anomaly alarm results to support operation and maintenance response and causal analysis decisions.

[0169] In a preferred embodiment, the method further includes: after generating the prediction output, receiving incremental data and performing incremental training operations based on replay and distillation, updating the model parameters of the periodic modeling mechanism and the transient modeling mechanism, and adjusting the structural configuration, specifically including:

[0170] The training dataset is composed of log samples already stored in the memory bank and incremental data. The time decay factor is calculated based on the sample generation time, and weights are assigned to the memory samples.

[0171] When performing incremental training, knowledge-based constraints are applied, specifically: recording the predicted output or intermediate representation of the previous version model for the memorized samples, minimizing the difference between the current model's output for the corresponding samples and the recorded values; identifying important parameters in the periodic modeling mechanism and the transient modeling mechanism, and applying weighted regularization terms to these parameters or keeping their values ​​unchanged.

[0172] The calculation periodic modeling mechanism assesses the modeling confidence of each frequency component in the periodic set and adjusts the number of frequency components retained in the periodic set.

[0173] The dimensions of the Koopman state space are dynamically adjusted based on the magnitude of the prediction error change within the sliding window in the transient modeling mechanism.

[0174] Evaluate the convergence performance of the regularization term within the current window and dynamically update the regularization strength;

[0175] Whether to perform the above structural adjustment operations is determined based on whether the change in verification error or the variance of prediction uncertainty exceeds a preset threshold.

[0176] like Figure 2 As shown, another embodiment of the present invention provides a log anomaly behavior detection system based on periodic pattern mining and incremental learning, applied to the log anomaly behavior detection method based on periodic pattern mining and incremental learning as described above, including:

[0177] The log collection and preprocessing module is used to acquire raw log data from the market supervision platform, perform preprocessing operations such as timestamp standardization, log template normalization, and key field extraction, and generate a log indicator sequence arranged in chronological order as the input sequence.

[0178] The periodic pattern mining module is used to perform a fast Fourier transform on the input sequence, extract the frequency components of the first N energy spectrum peaks to form a normal behavior periodic set, and divide the input sequence into a periodically stable part sequence and a transient change part sequence.

[0179] The periodic modeling module is used to model the periodically stable parts of the sequence. It includes a linear encoder driven by the periodic stability score to generate the predictive output of the periodic modeling mechanism.

[0180] The transient modeling module is used to model the transient variation part of the sequence. It includes a variational mode decomposition module and a Koopman encoder-decoder structure, and generates the prediction output of the transient modeling mechanism.

[0181] The joint prediction module is used to estimate the uncertainty of the prediction outputs of the periodic modeling module and the transient modeling module through Monte Carlo Dropout, and then fuse them based on the Bayesian weighting mechanism to generate the final prediction value.

[0182] The anomaly detection module is used to calculate the periodic residual, transient residual and global residual based on the error between the predicted value and the actual observed value, extract multidimensional residual feature vectors, generate anomaly scores, and determine whether to trigger an anomaly alarm.

[0183] The causal identification module is used to perform residual propagation analysis after an abnormal event occurs, construct a residual causal graph based on transfer entropy, and identify potential root cause paths and nodes.

[0184] The memory management module is used to build a memory bank based on prediction entropy or reconstruction error, select samples with high uncertainty and high diversity, and update the memory content.

[0185] The incremental training module is used to perform incremental training that combines replay and distillation based on memory samples and incremental data. It updates the model parameters of the periodic modeling mechanism and the transient modeling mechanism, and dynamically adjusts the number of frequency components, the dimension of the Koopman state space, and the configuration of regularization terms according to the structural confidence and error fluctuation.

[0186] In summary, this invention realizes a log anomaly detection method with the ability to model periodic structures, express transient dynamics, predict uncertainties, and reason about causality, providing effective technical support for building intelligent system operation and maintenance with high precision, high reliability, and high interpretability.

[0187] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A log anomaly behavior detection method based on periodic pattern mining and incremental learning, characterized in that, The method includes: Obtain raw log data from the market supervision platform and perform preprocessing operations such as timestamp standardization, log template normalization, and key field extraction to construct a log indicator sequence arranged in chronological order as the input sequence; Periodic pattern mining is performed on the input sequence. The frequency components of the first N energy spectrum peaks are extracted using time-frequency analysis to form a periodic set of normal behavior. Based on the periodic set, the input sequence is divided into a periodically stable part and a transient part. The periodically stable part of the sequence is encoded using a periodic modeling mechanism, which employs a linear encoder and introduces an adaptive linear constraint driven by a periodic stability score to enhance consistency across time periods. The transient change sequence is modeled using a transient modeling mechanism, which includes performing variational mode decomposition on the transient change sequence to extract several intrinsic mode function components, selecting high-frequency unsteady components as inputs to a fixed-dimensional Koopman dynamic encoder-decoder structure, introducing Koopman linear operators to model state evolution in the state space output by the encoder, and performing online estimation through dynamic mode decomposition. The periodic modeling mechanism and the transient modeling mechanism are combined into a joint prediction model. Monte Carlo Dropout is introduced to estimate the prediction uncertainty, and the final prediction value is generated based on Bayesian weighting. The joint prediction model of the periodic modeling mechanism and the transient modeling mechanism includes: The periodically stable part of the sequence generates corresponding prediction outputs based on the periodic modeling mechanism, while the transient part of the sequence generates corresponding prediction outputs based on the transient modeling mechanism. Monte Carlo Dropout is introduced into the prediction process of both the periodic modeling mechanism and the transient modeling mechanism to perform multiple forward propagations, generating multiple prediction output samples. Based on the multiple prediction output samples, the prediction mean and prediction variance of the periodic modeling mechanism and the transient modeling mechanism are calculated for uncertainty estimation. A Bayesian weighted strategy is constructed based on the prediction variance using both periodic and transient modeling mechanisms. The weights are inversely proportional to the prediction variance to generate the final predicted value. The fusion formula is as follows: ； in, and These are the prediction outputs of the periodic modeling mechanism and the transient modeling mechanism, respectively. and The prediction variances are for the periodic modeling mechanism and the transient modeling mechanism, respectively. A confidence gating mechanism is introduced in the prediction fusion process. When the difference between the prediction variances of the periodic modeling mechanism and the transient modeling mechanism exceeds the set difference threshold, Bayesian weighting is skipped, and the prediction output of the modeling mechanism with the smaller prediction variance is directly selected as the final prediction value. The joint prediction model continuously executes dynamic prediction output, uncertainty estimation and Bayesian weighted fusion during operation to support the calculation process of periodic residuals, transient residuals and global residuals; The real-time log indicator sequence is input into the joint prediction model to generate predicted values. The periodic residual, transient residual and overall residual are calculated and compared with the preset anomaly threshold. If the anomaly threshold is exceeded, an anomaly alarm is output and residual propagation analysis is performed. The transfer entropy is used to identify the anomaly causal path and potential root cause.

2. The log abnormal behavior detection method based on periodic pattern mining and incremental learning according to claim 1, characterized in that, The method of extracting the frequency components of the first N energy spectral peaks using time-frequency analysis to form a periodic set of normal behavior includes: The input sequence is divided into slices using a sliding window of fixed length, and a fast Fourier transform is performed on the log subsequence within each window to obtain the corresponding frequency domain amplitude spectrum. In the frequency domain amplitude spectrum corresponding to each window, the top N frequency components are selected according to their amplitude to form a local candidate frequency set. Frequency alignment is performed on the local candidate frequency sets extracted within all windows. The frequency of each frequency value in all sliding windows and the standard deviation of the amplitude change of the frequency value in different windows are calculated to measure the frequency energy stability. A comprehensive significance scoring function is constructed based on the occurrence frequency of each frequency and its corresponding energy stability, expressed as: ； In the formula, For the first in the local candidate frequency set One frequency, For frequency The overall significance score; For frequency Frequency of occurrence in all sliding windows; For frequency Standard deviation of spectral energy in different windows; This indicates a normalization operation; These are adjustable weighting coefficients, and ; Sort the scores in descending order and select the top M frequency values ​​based on the overall significance score to form the final period set.

3. The log abnormal behavior detection method based on periodic pattern mining and incremental learning according to claim 2, characterized in that, Based on the aforementioned period set, the input sequence is divided into a periodically stable portion and a transient portion, specifically including: For each frequency value in the period set, based on the drift range of the frequency value in each sliding window or the entropy value of the spectral energy distribution, a bandwidth interval centered on the frequency value is set as a frequency matching window to divide the frequency domain components. Perform a global fast Fourier transform on the complete input sequence to obtain all frequency components of the input sequence in the frequency domain, and determine whether the frequency value of each frequency component falls within any frequency matching window: If the frequency value of a frequency component belongs to any frequency matching window, then the frequency component is assigned to the frequency set corresponding to the periodically stable part of the sequence. If the frequency value of a frequency component does not belong to all frequency matching windows, then the frequency component is assigned to the frequency set corresponding to the transient change sequence. Frequency domain filtering and inverse transform operations are performed on the frequency sets corresponding to the periodically stable part of the sequence and the frequency sets corresponding to the transient part of the sequence, respectively, to obtain the reconstructed periodically stable part and transient part of the sequence.

4. The log abnormal behavior detection method based on periodic pattern mining and incremental learning according to claim 2, characterized in that, The periodically stable portion of the sequence is encoded using a periodic modeling mechanism, specifically including: The periodically stable partial sequence is input into a linear encoder with time-series modeling capability. The linear encoder performs a linear transformation on the current input time slice to generate a representation vector, and constructs a recursive encoding result by additively combining it with the historical representation vector. The comprehensive significance score calculated from each frequency value in the period set is used as the period stability score. The regularization strength or residual retention ratio in the linear coding process is adjusted based on the period stability score. If the period stability score is greater than the set stability threshold, the regularization strength of the linear term is increased; if the period stability score is lower than the stability threshold, the linear constraint is reduced. A mapping relationship between key fields and periodic frequency values ​​is constructed, and a symbolic periodic label is generated based on the mapping result. The symbolic periodic label is then fused with the numerical representation of the periodically stable part of the sequence after vector encoding. The original time index corresponding to each frequency value is recorded in the period set as the period start reference point, and the original time index corresponding to each time step is retained during the reconstruction of the period stable part of the sequence. The relative displacement is calculated based on the difference between the current time step index and the period start reference point. The relative displacement is used to generate a phase displacement vector to input the period position encoding process.

5. The log abnormal behavior detection method based on periodic pattern mining and incremental learning according to claim 1, characterized in that, The transient change portion of the sequence is modeled using a transient modeling mechanism, specifically including the following methods: Variational mode decomposition is performed on the transient change sequence to obtain multiple intrinsic mode function components; For each intrinsic mode function component, the dominant frequency is calculated, and all intrinsic mode function components with dominant frequencies higher than a preset minimum frequency threshold are screened out by frequency spectrum analysis to construct a set of high-frequency unsteady components; The set of high-frequency unsteady components is input into a fixed-dimensional Koopman dynamic encoder-decoder structure. The set of high-frequency unsteady components is mapped to the Koopman observation space through encoding operations. A fixed-dimensional Koopman linear operator is constructed in the Koopman observation space to model the temporal evolution path of the state in the Koopman observation space. The state transition structure of the Koopman linear operator is estimated online using the dynamic mode decomposition method, and the parameters of the Koopman linear operator are updated in real time according to the temporal change law of the observed state within the sliding time window. In the modeling process of the Koopman linear operator, Lyapunov stability constraints are introduced to enhance the stability and convergence of the state evolution path by controlling the spectral radius of the state transition structure or constructing a Lyapunov function that is monotonically decreasing on the state trajectory. The state modeling results generated in the Koopman observation space are output as the modeling representation of the transient modeling mechanism, and used for fusion and anomaly detection in the subsequent joint prediction model.

6. The log abnormal behavior detection method based on periodic pattern mining and incremental learning according to claim 1, characterized in that, The method for calculating the periodic residual, transient residual, and overall residual and comparing them with a preset anomaly threshold includes: Align the periodically stable sequence with the prediction output of the periodic modeling mechanism by time, and calculate the periodic residual at each time point, using the following formula: ; Align the transient variation sequence with the prediction output of the transient modeling mechanism by time, and calculate the transient residuals at the corresponding time points, using the following formula: ; Align the complete input sequence with the final predictions of the joint prediction model by time, and calculate the overall residuals at the corresponding time points, using the following formula: ; In the formula, , , The first Periodic residuals, transient residuals, and global residuals at each time point; , , These are the periodically stable portion of the sequence, the transient portion of the sequence, and the log indicator sequence at specific time points. The actual observed value; , These represent the periodic modeling mechanism and the transient modeling mechanism at different time points. The predicted output; For the joint prediction model at time points The output predicted value; For the periodic residual, transient residual, and global residual, calculate the absolute value of the residual within a sliding window of fixed length. Moving average Sliding standard deviation and the slope of the residual change And construct multidimensional residual feature vectors ; A direction labeling function is constructed based on the positive and negative directions at each time point of the periodic residual, transient residual, and global residual. ; Define the residual directionality enhancement anomaly scoring function as follows: ； In the formula, , These are the preset weighting coefficients; For multidimensional residual eigenvectors The first in Each dimension component; Abnormal rating values If the error is compared with a preset anomaly threshold, an anomaly alarm is triggered and the corresponding residual dimension and directionality are recorded.

7. The log abnormal behavior detection method based on periodic pattern mining and incremental learning according to claim 6, characterized in that, The residual propagation analysis performed uses transfer entropy to identify anomalous causal paths and potential root causes. The method includes: Identify abnormal time points where the abnormal score value exceeds the preset abnormal threshold, and form an abnormal residual trigger event sequence; Using the abnormal time point as the anchor window, extract segments from all log indicator sequences that overlap with or lag behind the abnormal time point to construct a candidate set of residual variables; For candidate residual variables in the candidate set of residual variables The time-delay-sensitive transfer entropy calculation method is adopted, within a set lag time window. Within, search for the maximum transition entropy value: ; In the formula, express Lag Under the condition of time unit The transfer entropy value; , These are the minimum and maximum values ​​of the lag time window, respectively; the maximum transfer entropy value... Used for causal strength, a residual causal graph model is constructed, where nodes in the graph are log metrics, edge weights are maximum transition entropy values, and edge attributes record propagation lag. The maximum allowed depth of the propagation path is limited in the causal graph, and a link credibility score is calculated for each causal edge. : ； In the formula, Source Indicators The residual value at the abnormal time point, , These are the weighting coefficients; Calculate the root cause priority scoring function for each node, using the following formula: ； In the formula, For nodes Root cause score; For nodes The degree of departure; For the node Pointing to node The maximum transfer entropy value; For transmission delay; Link credibility score; , , , These are weighting coefficients; Root cause score Exceeding the preset threshold The nodes were identified as potential root causes; The node identifiers of potential root causes, the propagation path structure, and the key transition entropy values ​​are output as part of the anomaly alarm results to support operation and maintenance response and causal analysis decisions.

8. The log abnormal behavior detection method based on periodic pattern mining and incremental learning according to claim 1, characterized in that, After generating the prediction output, incremental data is received and incremental training based on replay and distillation is performed to update the model parameters of the periodic modeling mechanism and the transient modeling mechanism, and to adjust the structure configuration, specifically including: The training dataset is composed of log samples already stored in the memory bank and incremental data. The time decay factor is calculated based on the sample generation time, and weights are assigned to the memory samples. When performing incremental training, knowledge-based constraints are applied, specifically: recording the predicted output or intermediate representation of the previous version model for the memorized samples, minimizing the difference between the current model's output for the corresponding samples and the recorded values; identifying important parameters in the periodic modeling mechanism and the transient modeling mechanism, and applying weighted regularization terms to these parameters or keeping their values ​​unchanged. The calculation periodic modeling mechanism assesses the modeling confidence of each frequency component in the periodic set and adjusts the number of frequency components retained in the periodic set. The dimensions of the Koopman state space are dynamically adjusted based on the magnitude of the prediction error change within the sliding window in the transient modeling mechanism. Evaluate the convergence performance of the regularization term within the current window and dynamically update the regularization strength; Whether to perform the above structural adjustment operations is determined based on whether the change in verification error or the variance of prediction uncertainty exceeds a preset threshold.

9. A log anomaly behavior detection system based on periodic pattern mining and incremental learning, applied to the log anomaly behavior detection method based on periodic pattern mining and incremental learning as described in any one of claims 1-8, characterized in that, The system includes: The log collection and preprocessing module is used to acquire raw log data from the market supervision platform, perform preprocessing operations such as timestamp standardization, log template normalization, and key field extraction, and generate a log indicator sequence arranged in chronological order as the input sequence. The periodic pattern mining module is used to perform a fast Fourier transform on the input sequence, extract the frequency components of the first N energy spectrum peaks to form a normal behavior periodic set, and divide the input sequence into a periodically stable part sequence and a transient change part sequence. The periodic modeling module is used to model the periodically stable parts of the sequence. It includes a linear encoder driven by the periodic stability score to generate the predictive output of the periodic modeling mechanism. The transient modeling module is used to model the transient variation part of the sequence. It includes a variational mode decomposition module and a Koopman encoder-decoder structure, and generates the prediction output of the transient modeling mechanism. The joint prediction module is used to estimate the uncertainty of the prediction outputs of the periodic modeling module and the transient modeling module through Monte Carlo Dropout, and then fuse them based on the Bayesian weighting mechanism to generate the final prediction value. The anomaly detection module is used to calculate the periodic residual, transient residual and global residual based on the error between the predicted value and the actual observed value, extract multidimensional residual feature vectors, generate anomaly scores, and determine whether to trigger an anomaly alarm. The causal identification module is used to perform residual propagation analysis after an abnormal event occurs, construct a residual causal graph based on transfer entropy, and identify potential root cause paths and nodes. The memory management module is used to build a memory bank based on prediction entropy or reconstruction error, select samples with high uncertainty and high diversity, and update the memory content. The incremental training module is used to perform incremental training that combines replay and distillation based on memory samples and incremental data. It updates the model parameters of the periodic modeling mechanism and the transient modeling mechanism, and dynamically adjusts the number of frequency components, the dimension of the Koopman state space, and the configuration of regularization terms according to the structural confidence and error fluctuation.

Images