Log abnormal behavior detection method and system based on periodic pattern mining and incremental learning

Abstract

The invention discloses a log abnormal behavior detection method and system based on periodic pattern mining and incremental learning, and relates to the technical field of system abnormality monitoring. The method comprises the steps of obtaining original log data and executing preprocessing operation; main periodic frequency components are extracted through time frequency analysis to form a periodic set, a periodic stable part sequence and a transient change part sequence are divided, and a periodic modeling mechanism and a transient modeling mechanism are used for modeling; a joint prediction model is constructed, Monte Carlo Dropout is introduced to estimate uncertainty, and Bayesian weighting is adopted to generate a prediction value; abnormity is judged through a periodic residual error, a transient residual error and an overall residual error, and an abnormal causal path is analyzed and identified in combination with transfer entropy; a memory sample driven playback and distillation mechanism is adopted to execute incremental training, and modeling structure parameters are dynamically adjusted. The method has the capabilities of periodic rule modeling, unsteady behavior expression, prediction fusion, abnormal causal identification and continuous model learning.

Description

Technical Field

[0001] The present invention relates to the technical field of system anomaly monitoring, and particularly to a method and system for detecting log abnormal behaviors based on periodic pattern mining and incremental learning. Background Art

[0002] As an important behavior record during the operation of a system, logs can reflect key information such as system operations, abnormal states, and fault evolutions, and have become the core data source for ensuring the stable operation of the system and supporting intelligent operation and maintenance decisions. The actual log data presents characteristics such as high frequency, large scale, and strong heterogeneity, and abnormal behaviors often manifest as the superposition of periodic patterns and non-periodic sudden changes, resulting in the difficulty of traditional static rule matching, single-model detection, or general reconstruction methods in meeting multiple requirements such as periodic dynamics, abnormal drift, behavior diversity, and semantic interpretation in complex system logs.

[0003] On the one hand, there are a large number of periodic operation patterns in logs (such as scheduled tasks, batch processing processes), and the stability and interference of these periodic behaviors have a significant impact on the abnormal recognition results. If the periodic patterns cannot be accurately extracted and modeled, false alarms or missed alarms are likely to occur. On the other hand, sudden abnormal events are often accompanied by non-linear transient evolutions, with diverse behavior forms, complex influence paths, and being easily affected by concept drift and changes in context conditions. Traditional static models are difficult to adapt to new abnormal patterns in a timely manner and lack support for model updates, parameter adjustments, and structure reconstructions.

[0004] In addition, existing anomaly detection schemes generally have the problem of insufficient interpretability and cannot provide a clear causal traceability path for the identified anomalies, restricting the response efficiency of the system to complex faults. Summary of the Invention

[0005] Aiming at problems such as inaccurate log behavior modeling, untimely anomaly detection, unclear alarm positioning, and the difficulty of the model in adapting to a continuously changing environment, the present invention proposes a method and system for detecting log abnormal behaviors based on periodic pattern mining and incremental learning. For the massive log data generated by the market supervision platform, by extracting periodic patterns and applying incremental learning strategies, accurate detection of abnormal behaviors and adaptive evolution of the model are achieved.

[0006] The present invention realizes the above object through the following technical solutions:

[0007] A method for detecting log abnormal behaviors based on periodic pattern mining and incremental learning, the method comprising:

[0008] Obtain the original log data of the market supervision platform, and perform preprocessing operations of timestamp standardization, log template normalization, and keyword field extraction to construct a log metric sequence arranged in chronological order as an input sequence;

[0009] Perform periodic pattern mining on the input sequence, extract the frequency components of the top N energy spectral peaks using time-frequency analysis method to form a set of periods of normal behavior, and divide the input sequence into a periodically stable partial sequence and a transient change partial sequence according to the set of periods;

[0010] Encode the periodically stable partial sequence using a periodic modeling mechanism, where the periodic modeling mechanism uses a linear encoder and introduces an adaptive linear constraint driven by a periodic stability score to enhance cross-time consistency;

[0011] Model the transient change partial sequence using a transient modeling mechanism, where the transient modeling mechanism includes performing variational mode decomposition on the transient change partial sequence to extract several intrinsic mode function components, and selecting high-frequency non-steady components to input a Koopman dynamic encoding-decoder structure with a fixed dimension. Introduce a Koopman linear operator in the state space output by the encoder to model state evolution, and perform online estimation through dynamic mode decomposition;

[0012] Combine the periodic modeling mechanism and the transient modeling mechanism to form a joint prediction model, introduce Monte Carlo Dropout to estimate prediction uncertainty, and generate the final prediction value based on Bayesian weighting;

[0013] Input the real-time log metric sequence into the joint prediction model to generate a prediction value, calculate the periodic residual, transient residual, and overall residual, and compare them with a preset anomaly threshold. If it exceeds the anomaly threshold, output an anomaly warning, and perform residual propagation analysis, and use transfer entropy to identify the anomaly causal path and potential root cause.

[0014] Preferably, the method of extracting the frequency components of the top N energy spectral peaks using time-frequency analysis method to form a set of periods of normal behavior includes:

[0015] Slice the input sequence in a sliding window manner with a fixed length, perform fast Fourier transform on the log subsequence in each window to obtain the corresponding frequency-domain amplitude spectrum;

[0016] In the frequency-domain amplitude spectrum corresponding to each window, select the top N frequency components according to the amplitude size to form a local candidate frequency set;

[0017] Align the local candidate frequency sets extracted from all windows, calculate the occurrence frequency of each frequency value in all sliding windows respectively, and the standard deviation of the corresponding amplitude change of this frequency value in different windows, which is used to measure the frequency energy stability;

[0018] Construct a comprehensive significance scoring function according to the occurrence frequency of each frequency and the corresponding energy stability, expressed as:

[0019] S(f i ) = α·Norm(c i ) + β·(1 - Norm(σ i ));

[0020] Wherein, f i is the i-th frequency in the local candidate frequency set, and S(f i ) is the comprehensive significance score of the frequency f i ; c i is the occurrence frequency of the frequency f i in all sliding windows; σ i is the standard deviation of the spectral energy of the frequency f i in different windows; Norm(·) represents the normalization operation; α, β ∈ [0, 1] are adjustable weight coefficients, and α + β = 1;

[0021] Sort in descending order according to the scoring results, and select the top M frequency values with the comprehensive significance score to form the final period set.

[0022] Preferably, according to the period set, the input sequence is divided into a periodic stable part sequence and a transient change part sequence, specifically including:

[0023] For each frequency value in the period set, according to the drift range or the entropy value of the spectral energy distribution of the frequency value in each sliding window, set a bandwidth interval centered on the frequency value as the frequency matching window for dividing the frequency domain components;

[0024] Perform a global fast Fourier transform on the complete input sequence once to obtain all frequency components of the input sequence in the frequency domain, and judge whether the frequency value of each frequency component falls into any frequency matching window:

[0025] If the frequency value of the frequency component belongs to any frequency matching window, then classify the frequency component into the frequency set corresponding to the periodic stable part sequence;

[0026] If the frequency value of the frequency component does not belong to all frequency matching windows, then classify the frequency component into the frequency set corresponding to the transient change part sequence;

[0027] Perform frequency domain filtering and inverse transformation operations on the frequency set corresponding to the periodic stable part sequence and the frequency set corresponding to the transient change part sequence respectively to obtain the reconstructed periodic stable part sequence and transient change part sequence.

[0028] Preferably, a periodic modeling mechanism is used to encode the periodic stable part sequence, and the method specifically includes:

[0029] Input the periodic stable partial sequence into a linear encoder with the ability of timing modeling. The linear encoder performs a linear transformation on the current input time slice to generate a representation vector, and constructs a recursive encoding result by means of additive combination with the historical representation vectors;

[0030] Use the comprehensive significance scores calculated for each frequency value in the period set as the period stability scores, and regulate the regularization strength or the residual retention ratio in the linear encoding process based on the period stability scores; if the period stability score is greater than the set stability threshold, increase the regularization strength of the linear term, and if the period stability score is lower than the stability threshold, reduce the linear constraint;

[0031] Construct the mapping relationship between the keyword fields and the period frequency values, and generate symbol period labels according to the mapping results. The symbol period labels are fused with the numerical representations of the periodic stable partial sequences after vector encoding;

[0032] Record the original time index corresponding to each frequency value in the period set as the period start reference point, and retain the original time index corresponding to each time step during the reconstruction of the periodic stable partial sequence. Calculate the relative displacement amount according to the difference between the current time step index and the period start reference point. The relative displacement amount is used to generate a phase displacement vector for input into the period position encoding process.

[0033] Preferably, use a transient modeling mechanism to model the transient change partial sequence. The method specifically includes:

[0034] Perform variational mode decomposition on the transient change partial sequence to obtain multiple intrinsic mode function components;

[0035] Calculate the dominant frequency for each intrinsic mode function component, and filter out all intrinsic mode function components with dominant frequencies higher than the preset minimum frequency threshold through frequency spectrum analysis to construct a high-frequency non-steady component set;

[0036] Input the high-frequency non-steady component set into a Koopman dynamic encoding-decoding structure with a fixed dimension. Map the high-frequency non-steady component set to the Koopman observation space through the encoding operation, and construct a Koopman linear operator with a fixed dimension in the Koopman observation space to model the time evolution path of the states in the Koopman observation space;

[0037] Adopt the dynamic mode decomposition method to perform online estimation on the state transition structure of the Koopman linear operator, and update the parameters of the Koopman linear operator in real time according to the temporal change law of the observed states within the sliding time window;

[0038] Introduce Lyapunov stability constraints in the modeling process of the Koopman linear operator, and enhance the stability and convergence of the state evolution path by controlling the spectral radius of the state transition structure or constructing a Lyapunov function that monotonically decreases on the state trajectory;

[0039] Output the state modeling results generated in the Koopman observation space as the modeling representation of the transient modeling mechanism for fusion and anomaly detection in the subsequent joint prediction model.

[0040] Preferably, the periodic modeling mechanism and the transient modeling mechanism are combined to form a joint prediction model, including:

[0041] The periodic stable part sequence generates corresponding prediction outputs based on the periodic modeling mechanism, and the transient change part sequence generates corresponding prediction outputs based on the transient modeling mechanism;

[0042] Introduce Monte Carlo Dropout to perform multiple forward propagations during the prediction processes of the periodic modeling mechanism and the transient modeling mechanism, generate multiple prediction output samples, and calculate the prediction mean and prediction variance of the periodic modeling mechanism and the transient modeling mechanism based on the multiple prediction output samples for uncertainty estimation;

[0043] Construct a Bayesian weighting strategy based on the prediction variances of the periodic modeling mechanism and the transient modeling mechanism, and allocate the weights inversely proportional to the prediction variances to generate the final prediction value The fusion formula is as follows:

[0044]

[0045] Where, and are the prediction outputs of the periodic modeling mechanism and the transient modeling mechanism respectively, and are the prediction variances of the periodic modeling mechanism and the transient modeling mechanism respectively;

[0046] Introduce a confidence gating mechanism during the prediction fusion process. When the difference between the prediction variances of the periodic modeling mechanism and the transient modeling mechanism exceeds the set difference threshold, skip the Bayesian weighting and directly select the prediction output of the modeling mechanism with a smaller prediction variance as the final prediction value;

[0047] The joint prediction model continuously performs dynamic prediction output, uncertainty estimation, and Bayesian weighted fusion during operation to support the calculation processes of periodic residuals, transient residuals, and overall residuals.

[0048] Preferably, the method for calculating the periodic residuals, transient residuals, and overall residuals and comparing them with the preset anomaly thresholds includes:

[0049] Align the periodic stable partial sequence with the predicted output of the periodic modeling mechanism in time, and calculate the periodic residuals at each time point respectively. The formula is:

[0050] Align the transient change partial sequence with the predicted output of the transient modeling mechanism in time, and calculate the transient residuals at the corresponding time points respectively. The formula is:

[0051] Align the complete input sequence with the final predicted value of the joint prediction model in time, and calculate the overall residuals at the corresponding time points respectively. The formula is:

[0052] In the formula, δ (t) are the periodic residual, transient residual, and overall residual at the t-th time point respectively; y (t) are the true observed values of the periodic stable partial sequence, transient change partial sequence, and log metric sequence at time point t respectively; are the predicted outputs of the periodic modeling mechanism and transient modeling mechanism at time point t respectively; is the predicted value output by the joint prediction model at time point t;

[0053] Calculate the absolute value of the residual, sliding mean |δ (t) |, sliding standard deviation sliding standard deviation and residual change slope for the periodic residual, transient residual, and overall residual respectively within a sliding window of fixed length, and construct a multi-dimensional residual feature vector R (t) ;

[0054] Construct a direction flag function D according to the positive and negative directions of each time point in the periodic residual, transient residual, and overall residual (t) ;

[0055] Define a residual directionality enhanced anomaly scoring function, expressed as:

[0056]

[0057] In the formula, λ j , λ 13 are preset weighting coefficients; is the j-th dimensional component in the multi-dimensional residual feature vector R (t) ;

[0058] Compare the anomaly score value A (t) with a preset anomaly threshold. If it exceeds the anomaly threshold, trigger an anomaly alarm and record the corresponding residual dimension and directionality.

[0059] Preferably, when performing residual propagation analysis, transfer entropy is used to identify abnormal causal paths and potential root causes. The method includes:

[0060] Identifying abnormal time points with abnormal score values exceeding a preset abnormal threshold to form an abnormal residual trigger event sequence;

[0061] Using the abnormal time points as anchor windows, extracting the overlapping or lagging segments of all log metric sequences with respect to the abnormal time points to construct a candidate set of residual variables;

[0062] For the candidate residual variables (X, Y) in the candidate set of residual variables, using the time-delay sensitive transfer entropy calculation method, within the set lag time window τ ∈ [τ min , τ max , searching for the maximum transfer entropy value:

[0063] In the formula, represents the transfer entropy value of X to Y under the condition of a lag of τ time units; τ min , τ max are respectively the minimum and maximum values of the lag time window; using the maximum transfer entropy value as the causal strength to construct a residual causal graph model, where the nodes in the graph are log metrics, the edge weights are the maximum transfer entropy values, and the edge attributes record the propagation lag;

[0064] Limiting the maximum allowable depth of the propagation path in the causal graph and calculating the link credibility score C X→Y for each causal edge:

[0065]

[0066] In the formula, is the residual value of the source metric X at the abnormal time point, and γ1, γ2 are weight coefficients;

[0067] Calculating the root cause priority scoring function for each node respectively, and the formula is:

[0068]

[0069] In the formula, Q m is the root cause score value of node m; D m is the out-degree of node m; is the maximum transfer entropy value from node m to node n; τ m→n is the propagation time delay; C m→n is the link credibility score; η1, η2, η3, η4 are weighting coefficients;

[0070] Taking the root cause score value Q m exceeding the preset threshold Q thThe nodes are identified as potential root causes;

[0071] Output the node identification of the potential root cause, the propagation path structure, and the key transfer entropy value as part of the abnormal alarm result to support the operation and maintenance response and the causal analysis decision.

[0072] Preferably, after generating the prediction output, receive incremental data and perform incremental training operations based on replay and distillation, update the model parameters of the periodic modeling mechanism and the transient modeling mechanism, and adjust the structure configuration, specifically including:

[0073] Combine the log samples stored in the memory bank with the incremental data to form a training data set, calculate the time decay factor based on the sample generation time, and assign weights to the memory samples;

[0074] When performing incremental training, apply knowledge retention constraints, specifically: record the prediction output or intermediate representation of the previous version of the model for the memory samples, minimize the difference between the output of the current model for the corresponding samples and the recorded values; identify the important parameters in the periodic modeling mechanism and the transient modeling mechanism, and apply weighted regularization terms or keep the values unchanged to these parameters;

[0075] Calculate the modeling confidence of the periodic modeling mechanism for each frequency component in the period set, and adjust the number of frequency components retained in the period set;

[0076] According to the change amplitude of the prediction error of the transient modeling mechanism within the sliding window, dynamically adjust the Koopman state space dimension;

[0077] Evaluate the convergence performance of the regularization term within the current window, and dynamically update the regularization strength;

[0078] Decide whether to perform the above structure adjustment operations based on whether the validation error change or the prediction uncertainty variance exceeds the preset threshold.

[0079] A log abnormal behavior detection system based on periodic pattern mining and incremental learning is applied to the log abnormal behavior detection method based on periodic pattern mining and incremental learning as described above. The system includes:

[0080] A log collection and preprocessing module for obtaining the original log data of the market supervision platform, performing preprocessing operations such as timestamp standardization, log template normalization, and keyword field extraction, and generating a log metric sequence arranged in chronological order as the input sequence;

[0081] A periodic pattern mining module for performing a fast Fourier transform on the input sequence, extracting the frequency components of the first N energy spectrum peaks to form a normal behavior period set, and dividing the input sequence into a periodic stable part sequence and a transient change part sequence;

[0082] A periodic modeling module for modeling the periodic stable part of a sequence, including a linear encoder driven by a periodic stability score to generate a prediction output of the periodic modeling mechanism;

[0083] A transient modeling module for modeling the transient change part of a sequence, including a variational mode decomposition module and a Koopman encoder-decoder structure, and generating a prediction output of the transient modeling mechanism;

[0084] A joint prediction module for estimating the uncertainty of the prediction outputs of the periodic modeling module and the transient modeling module through Monte Carlo Dropout, and fusing them based on a Bayesian weighting mechanism to generate a final prediction value;

[0085] An anomaly detection module for calculating the periodic residual, transient residual, and overall residual based on the error between the prediction value and the actual observation value, extracting a multi-dimensional residual feature vector, generating an anomaly score, and determining whether to trigger an anomaly alarm;

[0086] A causal identification module for performing residual propagation analysis after an anomaly event occurs, constructing a residual causal graph based on transfer entropy, and identifying potential root cause paths and nodes;

[0087] A memory management module for constructing a memory bank based on prediction entropy or reconstruction error, selecting samples with high uncertainty and high diversity, and updating the memory content;

[0088] An incremental training module for performing incremental training that combines replay and distillation based on memory samples and incremental data, updating the model parameters of the periodic modeling mechanism and the transient modeling mechanism, and dynamically adjusting the number of frequency components, the Koopman state space dimension, and the regularization term configuration according to the structural confidence and error fluctuation.

[0089] The beneficial effects of the present invention are as follows: By introducing a periodic pattern mining mechanism based on frequency analysis, the periodic patterns and aperiodic variations in the log metric sequence are separated, effectively reducing the interference of periodic noise on anomaly recognition; By constructing a dual-channel structure of a periodic modeling mechanism and a transient modeling mechanism, the perception ability of sudden anomalies is enhanced while maintaining the expression ability of periodic patterns; In the periodic modeling mechanism, an adaptive linear constraint driven by a periodic stability score is introduced, enabling the model to flexibly adjust the modeling intensity according to the confidence of periodic behavior, and enhancing the consistent modeling ability of the periodic structure across time periods; In the transient modeling mechanism, variational mode decomposition and a Koopman encoding-decoding structure are introduced, and the dynamic mode decomposition method is combined to online estimate the state evolution operator, improving the accuracy and scalability of modeling non-steady log anomaly behaviors; By introducing Monte Carlo Dropout and a Bayesian weighting mechanism into the joint prediction model, a dynamic fusion strategy is realized based on the model prediction uncertainty, and the most credible prediction path can be adaptively selected under periodic interference or transient mutation scenarios, enhancing the robustness of the model under abnormal pattern drift and structural uncertainty conditions; By decomposing the prediction error into periodic residuals, transient residuals, and overall residuals, and analyzing and determining them respectively, the anomaly recognition granularity and sensitivity are improved; Further, by combining residual propagation analysis and transfer entropy methods, the causal path and potential root cause between log metrics are identified, enhancing the interpretability and traceability of anomaly alarms. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for description in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0091] Wherein:

[0092] Figure 1 is the method flow chart in the embodiments of the present invention;

[0093] Figure 2 is the system structure block diagram in the embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0094] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the described embodiments of the present invention fall within the scope of protection of the present invention.

[0095] Such asFigure 1 As shown in the figure, this is an embodiment of the present invention. This embodiment provides a method for detecting abnormal behavior in logs based on periodic pattern mining and incremental learning, including the following steps:

[0096] S1: Obtain the original log data of the market supervision platform, and perform preprocessing operations such as timestamp standardization, log template normalization, and keyword field extraction to construct a log metric sequence arranged in chronological order as the input sequence.

[0097] S2: Perform periodic pattern mining on the input sequence, use the time-frequency analysis method to extract the frequency components of the first N energy spectrum peaks to form a set of periods of normal behavior, and divide the input sequence into a periodically stable partial sequence and a transient change partial sequence according to the set of periods.

[0098] In one embodiment, the implementation method of step S2 includes:

[0099] S2A1: Slice the input sequence in a sliding window manner with a fixed length. Assume the window length is L and the step size is S to obtain several overlapping or non-overlapping log subsequences. Perform a fast Fourier transform (FFT) on the log subsequence within each window to obtain the corresponding frequency domain amplitude spectrum;

[0100] S2A2: In the frequency domain amplitude spectrum corresponding to each window, select the top N frequency components according to the amplitude size to form a local candidate frequency set;

[0101] S2A3: Perform frequency alignment on the local candidate frequency sets extracted from all windows (classify based on frequency resolution), calculate the occurrence frequency of each frequency value in all sliding windows, and the standard deviation of the amplitude change corresponding to the frequency value in different windows to measure the frequency energy stability;

[0102] S2A4: Construct a comprehensive significance scoring function according to the occurrence frequency of each frequency and the corresponding energy stability, expressed as:

[0103] S(f i ) = α·Norm(c i ) + β·(1 - Norm(σ i ));

[0104] In the formula, f i is the i-th frequency in the local candidate frequency set, S(f i ) is the comprehensive significance score of the frequency f i ; c i is the occurrence frequency of the frequency f i in all sliding windows; σ i is the standard deviation of the amplitude change corresponding to the frequency f iStandard deviation of spectral energy in different windows; Norm(·) represents the normalization operation; α, β ∈ [0, 1] are adjustable weight coefficients used to balance the importance of frequency occurrence frequency and energy stability, and α + β = 1;

[0105] S2A5: Sort in descending order according to the scoring results, and select the top M frequency values with the highest comprehensive significance scores to form the final period set.

[0106] After obtaining the period set, in this embodiment, the log metric sequence is further divided and reconstructed into periodic and aperiodic components in the frequency domain, specifically including:

[0107] S2B1: For each frequency value in the period set Calculate the bandwidth interval of this frequency according to the drift range (such as the maximum and minimum difference) or the entropy value of the spectral energy distribution in each sliding window where δ k is the bandwidth factor, which can be set to a fixed value or determined adaptively according to the drift. The bandwidth interval centered on this frequency value is used as the frequency matching window for dividing the frequency domain components;

[0108] S2B2: Perform a global fast Fourier transform on the complete input sequence once to obtain its frequency domain representation F(ω), where ω is the discrete frequency coordinate;

[0109] S2B3: For each frequency component ω i ∈ F(ω), determine whether it falls into any bandwidth interval W k inside (k represents the kth frequency). If it satisfies then this frequency component is classified as a periodic stable component, otherwise it is classified as a transient change component; represents the union of the bandwidth intervals W1, W2,..., W M corresponding to all M periodic frequencies, that is, the range of frequency intervals considered to belong to periodic frequencies;

[0110] Suppose there are three frequency components in the period set:

[0111] f1 * = 0.1 Hz, corresponding to the bandwidth W1 = [0.08, 0.12];

[0112] corresponding to the bandwidth W2 = [0.22, 0.28];

[0113] corresponding to the bandwidth W3 = [0.48, 0.52];

[0114] Then:

[0115] Now there is a frequency component ω i = 0.26. Since 0.26 ∈ [0.22, 0.28] = W2, so there is That is, this frequency component belongs to the periodic stable component;

[0116] S2B4: Perform frequency-domain filtering on the divided periodic stable component and transient change component respectively, only retain the corresponding frequency components, set the rest to zero, and perform inverse FFT operations on the processed spectra respectively to reconstruct the periodic stable part sequence and transient change part sequence respectively.

[0117] S3: Encode the periodic stable part sequence using a periodic modeling mechanism. The periodic modeling mechanism uses a linear encoder and introduces an adaptive linear constraint driven by a periodic stability score to enhance cross-time consistency.

[0118] The periodic modeling mechanism aims to perform modeling encoding on the periodic stable part sequence for structural consistency and semantic interpretability, so as to enhance the model's generalization ability for periodic behavior and the robustness of anomaly detection. In one embodiment, the implementation method of step S3 includes:

[0119] S31: Input the periodic stable part sequence into a linear encoder with the ability of temporal modeling. The linear encoder adopts a recursive encoding strategy, generates a representation vector by performing a linear transformation on the current input time slice, and performs an additive combination with the encoding results of historical time slices to form a state representation sequence for capturing the evolution characteristics of periodic behavior;

[0120] S32: Introduce a periodic stability score to regulate the regularization strength or residual retention ratio in the linear encoding process. The periodic stability score comes from the comprehensive significance score calculated in the previous periodic pattern mining process; in the linear encoding stage, set a stability threshold to distinguish periodic segments with different degrees of stability. If the periodic stability score is greater than the set stability threshold, increase the regularization strength of the linear term to limit the degrees of freedom of the model, so as to maintain the consistency and reconstructability of the representation; if the periodic stability score is lower than the stability threshold, reduce the linear constraint and allow slight changes in the local structure to be retained;

[0121] S33: To enhance the model's semantic understanding ability of periodic patterns, a symbolic periodic pattern is introduced into the periodic modeling mechanism to assist the modeling path. Specifically, in the log preprocessing stage, keywords related to periods (such as scheduled tasks, batch processing scripts, etc.) in the log template are identified through keyword extraction operations, and the mapping relationship between the keywords and the periodic frequency values is constructed in combination with the frequency analysis results. According to this mapping, corresponding symbolic periodic tags are generated for the periodic stable partial sequences. After numerical encoding in the form of embedding vectors, they are fused with the numerical representation of the original periodic stable partial sequences at the input layer or the intermediate representation layer to enhance the semantic interpretability of the model for the periodic structure.

[0122] S34: To enable the periodic modeling mechanism to have the ability to perceive periodic phases, the original time index corresponding to each frequency value is recorded as the periodic starting reference point during the periodic set generation stage, and the original time index corresponding to each time step is retained during the reconstruction process of the periodic stable partial sequences. The relative displacement is calculated based on the difference between the current time step index and the periodic starting reference point. This relative displacement is used to generate a phase displacement vector as an additional input and introduced into the encoder to model the periodic internal phase characteristics.

[0123] Through the coordinated action of the modeling path, symbolic periodic semantic fusion, and periodic displacement perception mechanism regulated by the periodic stability score, the embodiments of the present invention achieve the structural consistency maintenance of periodic behaviors, cross-time robust modeling, and enhanced interpretability, effectively improving the sensitivity and accuracy of periodic behavior anomaly recognition.

[0124] S4: The transient change partial sequences are modeled using the transient modeling mechanism, including performing variational mode decomposition on the transient change partial sequences to extract several intrinsic mode function components, and selecting the high-frequency non-steady components to input into a fixed-dimensional Koopman dynamic encoding-decoding structure. The Koopman linear operator is introduced in the state space output by the encoder to model the state evolution, and online estimation is performed through dynamic mode decomposition.

[0125] In this embodiment, a specific implementation process for modeling transient behaviors in the log data of the market supervision platform is provided, which is used to construct the transient modeling mechanism in the log anomaly behavior detection system based on periodic pattern mining and incremental learning, including the following steps:

[0126] S41: The transient change partial sequences extracted from the preprocessed log metric sequences are used as input data. This sequence has removed the periodic stable components through frequency domain analysis and mainly contains short-term non-stable fluctuations during system operation, such as sudden failures, configuration changes, or aperiodic load disturbances. Variational mode decomposition (VMD) is performed on the transient change partial sequences to obtain multiple intrinsic mode function components.

[0127] In this embodiment, the decomposition order of VMD is set to K = 6, the penalty parameter α = 2000, and 6 intrinsic mode function (IMF) components are retained, which respectively represent the oscillation components in different frequency bands of the transient sequence;

[0128] S42: Calculate the dominant frequency for each intrinsic mode function component. The extraction method of the dominant frequency is based on the Fourier transform result of each component. Calculate the power spectral density function, and select the frequency corresponding to the maximum energy as the dominant frequency value. In this embodiment, the minimum threshold of the dominant frequency is set to f min = 0.1 Hz, and filter out all IMF components with dominant frequencies higher than f min to construct a high-frequency non-steady component set for characterizing the mutation behavior or non-steady perturbation in the log system;

[0129] S43: Input the high-frequency non-steady component set into a Koopman dynamic encoding-decoder structure with a fixed dimension. This structure includes a two-layer LSTM encoder with a hidden dimension of 64, which is responsible for encoding the input sequence into a state vector of length 16 to form the Koopman observation space;

[0130] S44: Construct a Koopman linear operator with a fixed dimension in the Koopman observation space for modeling the time evolution path of the states in the Koopman observation space; the form of this Koopman linear operator is a 16×16 state transition matrix, and the initial value is obtained through least squares fitting;

[0131] S45: In subsequent operations, the dynamic mode decomposition (DMD) method is used to update the state transition structure of the Koopman linear operator online. The DMD method uses the encoded state sequence within a sliding time window T = 30 steps, calculates the state evolution relationship in batches, and adjusts the value of the state transition matrix accordingly in real time to improve the adaptability to non-stationary dynamic changes;

[0132] S45: During the construction of the state transition structure, a Lyapunov stability constraint is introduced, which is realized in a spectral radius control manner. That is, after each DMD update, the eigenvalues of the state transition matrix are truncated in terms of spectral radius, forcing its maximum modulus to be less than 1, thereby ensuring that the state sequence does not diverge. This operation effectively suppresses the system instability caused by high-frequency perturbations;

[0133] Finally, the state evolution result of the constructed Koopman observation space is output as the modeling representation of the transient modeling mechanism, and is handed over to the subsequent joint prediction model for fusion with the output of the periodic modeling mechanism to participate in the generation of prediction values and the judgment of anomaly detection.

[0134] Through the transient modeling mechanism in this embodiment, the system can accurately model short-term high-frequency perturbations in the log, and has structural stability and online adaptation ability, significantly improving the detection accuracy of aperiodic abnormal behaviors.

[0135] S5: Combine the periodic modeling mechanism and the transient modeling mechanism to form a joint prediction model, introduce Monte Carlo Dropout to estimate the prediction uncertainty, and generate the final prediction value based on Bayesian weighting;

[0136] In one specific embodiment, the implementation method of step S5 for constructing a joint prediction model to generate the final prediction value includes:

[0137] The periodic stable part sequence generates corresponding prediction outputs based on the periodic modeling mechanism, and the transient change part sequence generates corresponding prediction outputs based on the transient modeling mechanism;

[0138] Introduce Monte Carlo Dropout respectively in the prediction processes of the periodic modeling mechanism and the transient modeling mechanism. Execute multiple forward propagations in the inference stage to obtain multiple prediction output samples, and calculate the prediction mean and prediction variance of the periodic modeling mechanism and the transient modeling mechanism based on the multiple prediction output samples for uncertainty estimation;

[0139] Construct a Bayesian weighting strategy based on the prediction variances of the periodic modeling mechanism and the transient modeling mechanism. Let the weights be inversely proportional to the prediction variances. Those with smaller prediction variances occupy higher weights in the final prediction, and generate the final prediction value The fusion formula is as follows:

[0140]

[0141] Where, and are the prediction outputs of the periodic modeling mechanism and the transient modeling mechanism respectively, and are the prediction variances of the periodic modeling mechanism and the transient modeling mechanism respectively;

[0142] Introduce a confidence gating mechanism in the prediction fusion process. When the difference between the prediction variances of the periodic modeling mechanism and the transient modeling mechanism exceeds the set difference threshold, skip the Bayesian weighting and directly select the prediction output of the modeling mechanism with a smaller prediction variance as the final prediction value;

[0143] Furthermore, a covariance adjustment term can be introduced to correct the correlation between the historical prediction errors of the periodic modeling mechanism and the transient modeling mechanism, thereby improving the stability of the fusion result and the overall prediction accuracy;

[0144] During the operation, the joint prediction model continuously performs dynamic prediction output, uncertainty estimation, and Bayesian weighted fusion to support the calculation processes of periodic residuals, transient residuals, and overall residuals.

[0145] S6: Input the real-time log metric sequence into the joint prediction model to generate prediction values, calculate the periodic residuals, transient residuals, and overall residuals, and compare them with the preset anomaly thresholds. If the thresholds are exceeded, output anomaly alarms and perform residual propagation analysis to identify the anomaly causal paths and potential root causes using transfer entropy.

[0146] In one embodiment, step S6 includes:

[0147] S6A1: Align the periodic stable part sequence and the prediction output of the periodic modeling mechanism in time, and calculate the periodic residuals at each time point respectively. The formula is:

[0148] S6A2: Align the transient change part sequence and the prediction output of the transient modeling mechanism in time, and calculate the transient residuals at the corresponding time points respectively. The formula is:

[0149] S6A3: Align the complete input sequence and the final prediction value of the joint prediction model in time, and calculate the overall residuals at the corresponding time points respectively. The formula is:

[0150] In the formula, δ (t) are the periodic residual, transient residual, and overall residual at the t-th time point respectively; y (t) are the true observed values of the periodic stable part sequence, transient change part sequence, and log metric sequence at time point t respectively; are the prediction outputs of the periodic modeling mechanism and the transient modeling mechanism at time point t respectively; is the prediction value output by the joint prediction model at time point t;

[0151] S6A4: Calculate the absolute value of the residual |δ (t) |, the moving average the moving standard deviation and the residual change slope for the periodic residuals, transient residuals, and overall residuals respectively within a sliding window of fixed length, and construct a multi-dimensional residual feature vector

[0152] S6A5: Construct a direction flag function D (t) , D (t)∈{-1, 0, 1}, where -1 represents negative direction, 0 represents unbiased, and 1 represents positive direction;

[0153] S6A6: Define the residual directionality enhanced anomaly scoring function A (t) , which is expressed as:

[0154]

[0155] In the formula, λ j , λ 13 are preset weighting coefficients; is the j-th dimensional component in the multi-dimensional residual feature vector R (t) ;

[0156] S6A7: Compare the anomaly score value A (t) with a preset anomaly threshold. If it exceeds the anomaly threshold, trigger an anomaly alarm and record the corresponding residual dimension and directionality.

[0157] Furthermore, the steps of performing residual propagation analysis include:

[0158] S6B1: Identify the anomaly time points where the anomaly score value exceeds the preset anomaly threshold, and form an anomaly residual trigger event sequence;

[0159] S6B2: Using the anomaly time points as anchor windows, extract the segments in all log metric sequences that overlap or lag behind the anomaly time points, and construct a candidate set of residual variables;

[0160] S6B3: For the candidate residual variables (X, Y) in the candidate set of residual variables, use the time-delay sensitive transfer entropy calculation method to search for the maximum transfer entropy value within the set lag time window τ ∈ [τ min , τ max :

[0161] In the formula, represents the transfer entropy value of X to Y under the condition of a lag of τ time units; τ min , τ max are respectively the minimum and maximum values of the lag time window; Use the maximum transfer entropy value as the causal strength to construct a residual causal graph model, where the nodes in the graph are log metrics, the edge weights are the maximum transfer entropy values, and the edge attributes record the propagation lag;

[0162] S6B4: Limit the maximum allowable depth of the propagation path in the causal graph, and calculate the link credibility score C X→Y for each causal edge:

[0163] In the formula, is the residual value of the source metric X at the abnormal time point, and γ1, γ2 are weight coefficients;

[0164] S6B5: Calculate the root cause priority scoring function for each node respectively. The formula is:

[0165]

[0166] In the formula, Q m is the root cause scoring value of node m; D m is the out-degree of node m; is the maximum transfer entropy value from node m to node n; τ m→n is the propagation delay; C m→n is the link credibility score; η1, η2, η3, η4 are weighting coefficients;

[0167] S6B6: Identify the nodes with the root cause scoring value Q m exceeding the preset threshold Q th as potential root causes;

[0168] S6B7: Output the node identification, propagation path structure, and key transfer entropy values of the potential root causes as part of the abnormal alarm result to support operation and maintenance response and causal analysis decision-making.

[0169] In a preferred embodiment, the method of this embodiment further includes: after generating the prediction output, receiving incremental data and performing incremental training operations based on replay and distillation, updating the model parameters of the periodic modeling mechanism and the transient modeling mechanism, and adjusting the structure configuration, specifically including:

[0170] Form a training data set by combining the log samples stored in the memory bank with the incremental data, calculate the time decay factor based on the sample generation time, and assign weights to the memory samples;

[0171] When performing incremental training, apply knowledge retention constraints, specifically: record the prediction output or intermediate representation of the previous version of the model for the memory samples, minimize the difference between the output of the current model for the corresponding samples and the recorded values; identify the important parameters in the periodic modeling mechanism and the transient modeling mechanism, and apply weighted regularization terms or keep the values unchanged to these parameters;

[0172] Calculate the modeling confidence of the periodic modeling mechanism for each frequency component in the frequency component set, and adjust the number of frequency components retained in the frequency component set;

[0173] Dynamically adjust the Koopman state space dimension according to the change amplitude of the prediction error of the transient modeling mechanism within the sliding window;

[0174] Evaluate the convergence performance of the regularization term within the current window, and dynamically update the regularization strength;

[0175] Determine whether to perform the above structural adjustment operation based on whether the validation error change or the variance of the prediction uncertainty exceeds a preset threshold.

[0176] As Figure 2 shown, this is another embodiment of the present invention. This embodiment provides a log anomaly behavior detection system based on periodic pattern mining and incremental learning, which is applied to the log anomaly behavior detection method based on periodic pattern mining and incremental learning as described above, and includes:

[0177] A log collection and preprocessing module, which is used to obtain the original log data of the market supervision platform, perform preprocessing operations such as timestamp standardization, log template normalization, and keyword field extraction, and generate a log metric sequence arranged in chronological order as an input sequence;

[0178] A periodic pattern mining module, which is used to perform a fast Fourier transform on the input sequence, extract the frequency components of the first N energy spectrum peaks to form a set of normal behavior periods, and divide the input sequence into a periodic stable part sequence and a transient change part sequence;

[0179] A periodic modeling module, which is used to model the periodic stable part sequence, including a linear encoder driven by a periodic stability score, and is used to generate the prediction output of the periodic modeling mechanism;

[0180] A transient modeling module, which is used to model the transient change part sequence, including a variational mode decomposition module and a Koopman encoder-decoder structure, and generate the prediction output of the transient modeling mechanism;

[0181] A joint prediction module, which is used to estimate the uncertainty of the prediction outputs of the periodic modeling module and the transient modeling module through Monte Carlo Dropout, and fuse them based on a Bayesian weighting mechanism to generate a final prediction value;

[0182] An anomaly detection module, which is used to calculate the periodic residual, transient residual, and overall residual according to the error between the prediction value and the actual observation value, extract a multi-dimensional residual feature vector, generate an anomaly score, and determine whether to trigger an anomaly alarm;

[0183] A causal identification module, which is used to perform residual propagation analysis after an anomaly event occurs, construct a residual causal graph based on transfer entropy, and identify potential root cause paths and nodes;

[0184] A memory management module, which is used to construct a memory bank according to the prediction entropy or reconstruction error, select samples with high uncertainty and high diversity, and update the memory content;

[0185] An incremental training module is used to perform incremental training that combines replay and distillation based on memory samples and incremental data, update the model parameters of the periodic modeling mechanism and the transient modeling mechanism, and dynamically adjust the number of frequency components, the Koopman state space dimension, and the regularization term configuration according to the structural confidence and error fluctuation.

[0186] In summary, the present invention realizes a log anomaly detection method with the capabilities of periodic structure modeling, transient dynamic expression, uncertainty fusion prediction, and causal reasoning, providing effective technical support for building intelligent system operation and maintenance with high precision, high reliability, and high interpretability.

[0187] As mentioned above, the above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various changes or substitutions, and these should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for detecting abnormal behavior in logs based on periodic pattern mining and incremental learning, characterized in that The method includes: obtaining the original log data of the market supervision platform, and performing preprocessing operations of timestamp standardization, log template normalization, and keyword field extraction to construct a log metric sequence arranged in chronological order as the input sequence; Performing periodic pattern mining on the input sequence, using the time-frequency analysis method to extract the frequency components of the first N energy spectral peaks to form a set of periods of normal behavior, and dividing the input sequence into a periodically stable partial sequence and a transient change partial sequence according to the set of periods; Encoding the periodically stable partial sequence using a periodic modeling mechanism, which uses a linear encoder and introduces an adaptive linear constraint driven by a periodic stability score to enhance cross-period consistency; Modeling the transient change partial sequence using a transient modeling mechanism, which includes performing variational mode decomposition on the transient change partial sequence to extract several intrinsic mode function components, and selecting the high-frequency non-steady components to input into a Koopman dynamic encoding-decoder structure with a fixed dimension, introducing a Koopman linear operator in the state space output by the encoder to model state evolution, and performing online estimation through dynamic mode decomposition; Combining the periodic modeling mechanism and the transient modeling mechanism to form a joint prediction model, introducing Monte Carlo Dropout to estimate prediction uncertainty, and generating the final prediction value based on Bayesian weighting; Inputting the real-time log metric sequence into the joint prediction model to generate a prediction value, calculating the periodic residual, transient residual, and overall residual and comparing them with a preset anomaly threshold. If it exceeds the anomaly threshold, an anomaly alarm is output, and residual propagation analysis is performed to identify the anomaly causal path and potential root cause using transfer entropy.

2. The log abnormal behavior detection method based on periodic pattern mining and incremental learning according to claim 1, characterized in that The method of using the time-frequency analysis method to extract the frequency components of the first N energy spectral peaks to form a set of periods of normal behavior includes: Slicing the input sequence in a sliding window manner with a fixed length, performing a fast Fourier transform on the log subsequence in each window to obtain the corresponding frequency-domain amplitude spectrum; In the frequency-domain amplitude spectrum corresponding to each window, selecting the top N frequency components according to the amplitude magnitude to form a local candidate frequency set; Aligning the frequencies of the local candidate frequency sets extracted from all windows, calculating the occurrence frequency of each frequency value in all sliding windows respectively, and the standard deviation of the corresponding amplitude change of the frequency value in different windows to measure the frequency energy stability; Constructing a comprehensive significance scoring function according to the occurrence frequency of each frequency and the corresponding energy stability, expressed as: S(f i ) = α·Norm(c i ) + β·(1 - Norm(σ i )); where f i is the i-th frequency in the local candidate frequency set, and S(f i ) is the comprehensive significance score of the frequency f i ; c i is the occurrence frequency of the frequency f i in all sliding windows; σ i is the standard deviation of the spectral energy of the frequency f i in different windows; Norm(·) represents the normalization operation; α, β ∈ [0, 1] are adjustable weight coefficients, and α + β = 1; Sorting the scoring results in descending order, and selecting the top M frequency values with the highest comprehensive significance score to form the final set of periods.

3. The method for detecting abnormal behavior of logs based on periodic pattern mining and incremental learning according to claim 2, wherein Dividing the input sequence into a periodically stable partial sequence and a transient change partial sequence according to the set of periods, specifically including: For each frequency value in the set of periods, setting a bandwidth interval centered on the frequency value as the frequency matching window according to the drift range of the frequency value in each sliding window or the entropy value of the spectral energy distribution for dividing the frequency domain components; Perform a global fast Fourier transform on the complete input sequence to obtain all frequency components of the input sequence in the frequency domain, and determine whether the frequency value of each frequency component falls into any frequency matching window: If the frequency value of the frequency component belongs to any frequency matching window, classify the frequency component into the frequency set corresponding to the periodic stable part sequence; If the frequency value of the frequency component does not belong to all frequency matching windows, classify the frequency component into the frequency set corresponding to the transient change part sequence; Perform frequency domain filtering and inverse transformation operations on the frequency set corresponding to the periodic stable part sequence and the frequency set corresponding to the transient change part sequence respectively to obtain the reconstructed periodic stable part sequence and transient change part sequence.

4. The method for detecting abnormal behavior of logs based on periodic pattern mining and incremental learning according to claim 2, wherein Encode the periodic stable part sequence using a periodic modeling mechanism. The method specifically includes: Input the periodic stable part sequence into a linear encoder with the ability of temporal modeling. The linear encoder performs a linear transformation on the current input time slice to generate a representation vector, and constructs a recursive encoding result by additive combination with the historical representation vector; Use the comprehensive significance score calculated for each frequency value in the period set as the period stability score, and regulate the regularization strength or residual retention ratio in the linear encoding process based on the period stability score; if the period stability score is greater than the set stability threshold, increase the linear term regularization strength, and if the period stability score is lower than the stability threshold, reduce the linear constraint; construct the mapping relationship between the keyword segment and the period frequency value, and generate a symbolic period label according to the mapping result. The symbolic period label is fused with the numerical representation of the periodic stable part sequence after vector encoding; Record the original time index corresponding to each frequency value in the period set as the period start reference point, and retain the original time index corresponding to each time step during the reconstruction of the periodic stable part sequence. Calculate the relative displacement amount according to the difference between the current time step index and the period start reference point. The relative displacement amount is used to generate a phase displacement vector for input into the period position encoding process.

5. The method for detecting abnormal behavior of logs based on periodic pattern mining and incremental learning according to claim 1, wherein Model the transient change part sequence using a transient modeling mechanism. The method specifically includes: Perform variational mode decomposition on the transient change part sequence to obtain multiple intrinsic mode function components; Calculate the dominant frequency for each intrinsic mode function component, and screen out all intrinsic mode function components with dominant frequencies higher than the preset minimum frequency threshold through frequency spectrum analysis to construct a high-frequency non-steady component set; Input the high-frequency non-steady component set into a Koopman dynamic encoding-decoding structure with a fixed dimension. Map the high-frequency non-steady component set to the Koopman observation space through the encoding operation, and construct a Koopman linear operator with a fixed dimension in the Koopman observation space to model the time evolution path of the state in the Koopman observation space; The state transition structure of the Koopman linear operator is estimated online using the dynamic mode decomposition method, and the parameters of the Koopman linear operator are updated in real time according to the temporal variation law of the observed states within the sliding time window; The Lyapunov stability constraint is introduced in the modeling process of the Koopman linear operator, and the stability and convergence of the state evolution path are enhanced by controlling the spectral radius of the state transition structure or constructing a Lyapunov function that monotonically decreases along the state trajectory; The state modeling results generated in the Koopman observation space are output as the modeling representation of the transient modeling mechanism for subsequent fusion and anomaly detection in the joint prediction model.

6. The log abnormal behavior detection method based on periodic pattern mining and incremental learning according to claim 1, characterized in that The periodic modeling mechanism and the transient modeling mechanism are combined to form a joint prediction model, including: The periodic stable partial sequence generates corresponding prediction outputs based on the periodic modeling mechanism, and the transient change partial sequence generates corresponding prediction outputs based on the transient modeling mechanism; During the prediction processes of the periodic modeling mechanism and the transient modeling mechanism, Monte Carlo Dropout is introduced to perform multiple forward propagations to generate multiple prediction output samples, and the prediction means and prediction variances of the periodic modeling mechanism and the transient modeling mechanism are calculated based on the multiple prediction output samples for uncertainty estimation; Construct a Bayesian weighting strategy based on the prediction variance of the periodic modeling mechanism and the transient modeling mechanism, and let the weights be inversely proportional to the prediction variance to generate the final prediction value. The fusion formula is as follows: Among them, and are the predicted outputs of the periodic modeling mechanism and the transient modeling mechanism respectively, and are the predicted variances of the periodic modeling mechanism and the transient modeling mechanism respectively; During the prediction fusion process, a confidence gating mechanism is introduced. When the difference between the prediction variances of the periodic modeling mechanism and the transient modeling mechanism exceeds the set difference threshold, Bayesian weighting is skipped, and the prediction output of the modeling mechanism with a smaller prediction variance is directly selected as the final prediction value; During the operation of the joint prediction model, dynamic prediction output, uncertainty estimation, and Bayesian weighted fusion are continuously performed to support the calculation processes of the periodic residual, transient residual, and overall residual.

7. The method for detecting abnormal behavior of logs based on periodic pattern mining and incremental learning according to claim 6, wherein The method for calculating the periodic residual, transient residual, and overall residual and comparing them with a preset anomaly threshold includes: Align the periodic stable partial sequence with the predicted output of the periodic modeling mechanism in time, and calculate the periodic residual at each time point respectively. The formula is as follows: Align the transient change part sequence with the predicted output of the transient modeling mechanism in time, and calculate the transient residuals at the corresponding time points respectively. The formula is as follows: Align the complete input sequence with the final prediction value of the joint prediction model in time, and calculate the overall residuals at the corresponding time points respectively. The formula is as follows: In the formula, δ (t) are the periodic residual, transient residual, and overall residual at the t-th time point, respectively; y (t) are the true observed values of the periodic stable part sequence, transient change part sequence, and log metric sequence at time point t, respectively; are the predicted outputs of the periodic modeling mechanism and the transient modeling mechanism at time point t, respectively; is the predicted value output by the joint prediction model at time point t; Calculate the absolute value of the residual, the moving average, the moving standard deviation, and the slope of the residual change of the periodic residual, transient residual, and overall residual respectively within a sliding window of a fixed length |δ (t) |, the moving average the moving standard deviation and the slope of the residual change and construct a multi-dimensional residual feature vector R (t) ; Construct a direction flag function D according to the positive and negative directions of each time point in the periodic residual, transient residual, and overall residual (t) ; Define a residual directionality enhanced anomaly scoring function, expressed as: where λ j , λ 13 are preset weighting coefficients; is the j-th dimensional component of the multi-dimensional residual feature vector R (t) ; Compare the abnormal score value A (t) with a preset abnormal threshold. If it exceeds the abnormal threshold, trigger an abnormal alarm and record the corresponding residual dimension and directionality.

8. The method for detecting abnormal behavior of logs based on periodic pattern mining and incremental learning according to claim 7, wherein The method for performing residual propagation analysis and identifying abnormal causal paths and potential root causes using transfer entropy includes: Identify the abnormal time points with abnormal scoring values exceeding the preset anomaly threshold to form an abnormal residual trigger event sequence; Using the abnormal time points as the anchor windows, extract the segments that overlap or lag with the abnormal time points in all log metric sequences to construct a candidate set of residual variables; For the candidate residual variables (X, Y) in the candidate set of residual variables, the time-delay sensitive transfer entropy calculation method is used to search for the maximum transfer entropy value within the set lag time window τ ∈ [τ min , τ max : In the formula, represents the transfer entropy value of X to Y under the condition of a lag of τ time units; τ min , τ max are the minimum and maximum values of the lag time window respectively; the maximum transfer entropy value is used as the causal strength to construct a residual causal graph model, where the nodes in the graph are log metrics, the edge weights are the maximum transfer entropy values, and the edge attributes record the propagation lag; Limit the maximum allowable depth of the propagation path in the causal graph and calculate the link credibility score C for each causal edge X→Y : In the formula, is the residual value of the source index X at the abnormal time point, and γ1 and γ2 are weight coefficients; Calculate the root cause priority scoring function for each node respectively, and the formula is: where Q m is the root cause score value of node m; D m is the out-degree of node m; is the maximum transfer entropy value from node m to node n; τ m→n is the propagation delay; C m→n is the link credibility score; η1, η2, η3, η4 are weighting coefficients; The root cause scoring value Q m exceeding the preset threshold Q th is identified as a potential root cause; Output the node identifier of the potential root cause, the propagation path structure, and the key transfer entropy value as part of the abnormal alarm result to support the operation and maintenance response and causal analysis decision-making.

9. The log abnormal behavior detection method based on periodic pattern mining and incremental learning according to claim 1, characterized in that After generating the prediction output, receive incremental data and perform incremental training operations based on replay and distillation to update the model parameters of the periodic modeling mechanism and the transient modeling mechanism, and adjust the structure configuration, specifically including: Combine the log samples stored in the memory bank with the incremental data to form a training data set, calculate the time decay factor according to the sample generation time, and assign weights to the memory samples; When performing incremental training, apply knowledge retention constraints, specifically: record the predicted output or intermediate representation of the previous version of the model for memory samples, and minimize the difference between the output of the current model for the corresponding samples and the recorded values; identify important parameters in the periodic modeling mechanism and the transient modeling mechanism, and apply weighted regularization terms or keep the values unchanged to these parameters; Calculate the modeling confidence of the periodic modeling mechanism for each frequency component in the periodic set, and adjust the number of frequency components retained in the periodic set; According to the change amplitude of the prediction error of the transient modeling mechanism within the sliding window, dynamically adjust the Koopman state space dimension; evaluate the convergence performance of the regularization term within the current window, and dynamically update the regularization strength; Decide whether to perform the above structural adjustment operations based on whether the validation error change or the prediction uncertainty variance exceeds a preset threshold.

10. A log anomaly behavior detection system based on periodic pattern mining and incremental learning, which is applied to the log anomaly behavior detection method based on periodic pattern mining and incremental learning according to any one of claims 1-9, and is characterized in that, The system includes: a log collection and preprocessing module, which is used to obtain the original log data of the market supervision platform, perform preprocessing operations such as timestamp standardization, log template normalization, and keyword field extraction, and generate a log metric sequence arranged in chronological order as the input sequence; A periodic pattern mining module, which is used to perform a fast Fourier transform on the input sequence, extract the frequency components of the first N energy spectral peaks to form a normal behavior periodic set, and divide the input sequence into a periodic stable part sequence and a transient change part sequence; a periodic modeling module, which is used to model the periodic stable part sequence, including a linear encoder driven by a periodic stability score, and is used to generate the predicted output of the periodic modeling mechanism; A transient modeling module, which is used to perform modeling on the transient change part sequence, including a variational mode decomposition module and a Koopman encoder-decoder structure, and generate the predicted output of the transient modeling mechanism; A joint prediction module, which is used to estimate the uncertainty of the predicted outputs of the periodic modeling module and the transient modeling module through Monte Carlo Dropout, and fuse them based on the Bayesian weighting mechanism to generate the final predicted value; An anomaly detection module, which is used to calculate the periodic residual, transient residual, and overall residual according to the error between the predicted value and the actual observed value, extract the multi-dimensional residual feature vector, generate an anomaly score, and determine whether to trigger an anomaly alarm; A causal identification module, which is used to perform residual propagation analysis after an abnormal event occurs, construct a residual causal graph based on transfer entropy, and identify potential root cause paths and nodes; A memory management module, which is used to construct a memory bank according to the prediction entropy or reconstruction error, select samples with high uncertainty and high diversity, and update the memory content; An incremental training module, which is used to perform incremental training combining replay and distillation based on memory samples and incremental data, update the model parameters of the periodic modeling mechanism and the transient modeling mechanism, and dynamically adjust the number of frequency components, the Koopman state space dimension, and the regularization term configuration according to the structural confidence and error fluctuation.

Images