An adversarial log generation method and system based on large language model reinforcement learning

Abstract

The application discloses an adversarial log generation method and system based on a large language model reinforcement learning, and belongs to the technical field of log anomaly detection and artificial intelligence. In order to solve the technical problem that the existing adversarial log generation is difficult to balance attack effectiveness, anomaly effectiveness and rewriting novelty, the application performs template analysis on original logs and determines a key anomaly mode, performs anomaly perception supervision fine-tuning on a large language model based on an anomaly event ID sequence and the key anomaly mode, and obtains a log rewriter; joint access judgment is performed according to the detection results of a frozen heterogeneous detector integration, an anomaly evidence reservation degree, structural rationality and a rewriting range, and a reward is determined, reinforcement learning fine-tuning is performed on the log rewriter based on the reward, an adversarial log generator is obtained, and an adversarial log sequence is generated. The application can balance anomaly evidence reservation, rewriting novelty and detection evasion capability, and is used for robustness evaluation and incremental reinforcement of a deep log anomaly detector.

Description

Technical Field

[0001] This invention proposes an adversarial log generation method and system based on reinforcement learning of a large language model, belonging to the interdisciplinary field of artificial intelligence and information security. Background Technology

[0002] With the widespread adoption of distributed systems, cloud platforms, and large-scale online services, log data has become a core data source for operations monitoring and security auditing. In actual deployments, operations personnel rely on anomaly detectors to first screen out suspicious execution traces before proceeding to manual diagnosis and root cause analysis. Therefore, log anomaly detection has become a key technology for ensuring system availability and security.

[0003] Methods for log anomaly detection have evolved from traditional rule matching and statistical learning to deep learning. In recent years, detectors based on deep neural networks have made significant progress. DeepLog uses recurrent neural networks to predict the next event, capturing short-range temporal dependencies; LogCNN uses convolutional filters to capture local n-gram patterns; and LogTransformer uses a self-attention mechanism to capture longer-range event dependencies. Recent research further leverages the knowledge from Large Language Models (LLMs) to enhance log representations or transfer cross-system semantics extracted from LLMs. These methods have achieved high detection accuracy on public benchmarks such as HDFS and Hadoop.

[0004] However, empirical studies show that baseline accuracy alone cannot guarantee robust performance of deep log detectors in real-world deployment environments. The same underlying anomaly does not always manifest in the same log format in a real system: machine crashes may be accompanied by different recovery operations, network outages may trigger different retry patterns, and disk-related failures may exhibit different local event sequences under different loads. Attackers, or naturally evolving system behavior, can bypass the first anomaly screening process if they can alter the surface pattern facing the detector, even when anomalous behavior still exists. Detectors trained based on historical event patterns are prone to overfitting to previously seen surface anomaly patterns, thus misjudging sequences where the surface has shifted but anomaly evidence still exists.

[0005] Existing adversarial generation or data augmentation methods struggle to simultaneously satisfy the requirements of attack effectiveness and anomaly effectiveness. A few oversampling and interpolation methods primarily address class imbalance problems and are not naturally suited to discrete event ID sequences. Rule-based perturbations can modify event order or frequency, but lack detector feedback, making it difficult to expose weaknesses in the decision boundary. General adversarial text attacks are also insufficient for direct transfer to log scenarios: optimizing only the evasion effect often destroys the anomaly itself, preventing the generated samples from reflecting the true detector blind spots; focusing solely on similarity or semantic preservation easily degenerates into identical or approximate copies of the original sequence, failing to expose detector robustness gaps.

[0006] Therefore, how to stably generate adversarial log sequences that can deceive multiple heterogeneous deep detectors while preserving key anomaly evidence and having non-trivial differences from the original sequence is a technical problem that urgently needs to be solved in the field of deep log anomaly detection. Summary of the Invention

[0007] The purpose of this invention is to propose an adversarial log generation method and system based on reinforcement learning of a large language model, so as to solve the technical problem that existing adversarial log generation methods cannot simultaneously take into account the effectiveness of attacks, the effectiveness of anomalies, and the novelty of rewriting. It achieves the effect of improving the ability to avoid heterogeneous deep anomaly detectors while retaining key anomaly evidence and forming nontrivial rewritings, and supports the robustness evaluation and incremental reinforcement of detectors.

[0008] To achieve the above objectives, the present invention adopts the following technical solution.

[0009] An adversarial log generation method based on reinforcement learning of a large language model includes the following steps: The original log is parsed using a log template to obtain an event ID sequence, and key anomaly patterns are determined based on the abnormal event ID sequence in the event ID sequence. Using supervised fine-tuning samples constructed based on the abnormal event ID sequence and the key abnormal patterns, anomaly-aware supervised fine-tuning is performed on the large language model to obtain a log rewriter; Based on the detection results of the candidate event ID sequence generated by the log rewriter by the frozen heterogeneous detector ensemble, and the degree of retention of anomalous evidence, structural rationality and rewriting range determined based on the key anomaly pattern, the candidate event ID sequence is jointly admitted and a reward is determined. Based on the reward, the log rewriter is fine-tuned by reinforcement learning to obtain an adversarial log generator. The sequence of exception event IDs to be rewritten is input into the adversarial log generator to obtain the adversarial log sequence.

[0010] Furthermore, the original logs are parsed using log templates to obtain an event ID sequence, and key anomaly patterns are determined based on the abnormal event ID sequence within the event ID sequence, including: The original logs are parsed using log templates, and each log is mapped to a corresponding event ID to obtain an event ID sequence; Determine the normal event ID sequence and the abnormal event ID sequence from the event ID sequence based on the log anomaly labels; Based on the event distribution and local temporal relationship of the abnormal event ID sequence and the normal event ID sequence, the key abnormal patterns are determined.

[0011] Furthermore, based on the event distribution and local temporal relationship of the abnormal event ID sequence and the normal event ID sequence, key abnormal patterns are determined, including: The key abnormal events are determined based on the occurrence of the event ID in the abnormal event ID sequence and the normal event ID sequence; Based on the occurrence of adjacent event ID combinations in the abnormal event ID sequence, key abnormal transitions are determined, and key abnormal patterns are formed based on the key abnormal events and the key abnormal transitions.

[0012] Furthermore, using supervised fine-tuning samples constructed based on the abnormal event ID sequence and the key abnormal pattern, anomaly-aware supervised fine-tuning is performed on the large language model to obtain a log rewriter, including: Based on the abnormal event ID sequence, the key abnormal pattern, and the rewriting requirements, construct instructions and the corresponding target rewritten event ID sequence to form a supervised fine-tuning sample. The large language model is subjected to anomaly-aware supervised fine-tuning using the supervised fine-tuning samples to obtain a log rewriter.

[0013] Further, based on the abnormal event ID sequence, the key abnormal pattern, and the rewriting requirements, instructions and corresponding target rewritten event ID sequences are constructed to form a supervised fine-tuning sample, including: Based on the event context, non-critical events in the abnormal event ID sequence are replaced with event IDs with similar contexts to obtain replacement and rewriting samples; Insert normal operation event IDs between the abnormal events in the abnormal event ID sequence to obtain a distributed rewrite sample; By placing the key abnormal events in the key abnormal pattern into a context formed by the normal operation event ID, an obfuscated rewrite sample is obtained. For the same abnormal event ID sequence combination, perform non-critical event replacement, normal operation event insertion, and critical abnormal event context adjustment to obtain a composite rewrite sample, and form a supervised fine-tuning sample based on the replacement rewrite sample, the dispersed rewrite sample, the obfuscated rewrite sample, and the composite rewrite sample.

[0014] Furthermore, the large language model is subjected to anomaly-aware supervised fine-tuning using the supervised fine-tuning samples to obtain a log rewriter, including: Freeze the original weights of the linear layer to be adapted in the large language model, and set a trainable low-rank matrix for the linear layer to be adapted; Using the rewritten event ID sequence in the supervised fine-tuning sample as the supervision target, the low-rank matrix is ​​trained using the supervised fine-tuning sample to obtain the log rewriter.

[0015] Further, based on the detection results of the candidate event ID sequence generated by the log rewriter through the frozen heterogeneous detector ensemble, and the degree of preservation of anomalous evidence, structural rationality, and rewriting magnitude determined based on the key anomaly pattern, a joint admission judgment is made on the candidate event ID sequence and a reward is determined. Based on the reward, the log rewriter is fine-tuned using reinforcement learning to obtain an adversarial log generator, including: Based on the event ID sequence, a normal event ID sequence is determined. Multiple deep anomaly detectors are trained using the normal event ID sequence, and the trained model parameters are frozen to obtain a frozen heterogeneous detector ensemble. The log rewriter generates a candidate event ID sequence, the frozen heterogeneous detector is integrated to detect the candidate event ID sequence, the detection result is obtained, the degree of abnormal evidence retention of the candidate event ID sequence is determined according to the key abnormal pattern and the abnormal event ID sequence, the structural rationality is determined according to the candidate event ID sequence, and the rewriting range is determined according to the candidate event ID sequence and the abnormal event ID sequence. The candidate event ID sequence is jointly admitted based on the degree of preservation of abnormal evidence, the rationality of the structure, and the extent of rewriting, and the reward is determined based on the joint admission judgment result and the detection result. The log rewriter is fine-tuned using reinforcement learning based on the rewards to obtain an adversarial log generator.

[0016] Furthermore, multiple deep anomaly detectors are trained based on the normal event ID sequence in the event ID sequence, and the trained model parameters are frozen to obtain a frozen heterogeneous detector ensemble, including: DeepLog, LogCNN, and LogTransformer are trained using the normal event ID sequence, respectively. DeepLog uses recurrent next event prediction, LogCNN uses one-dimensional convolution to extract local event patterns, and LogTransformer uses self-attention to extract event dependencies. The model parameters of DeepLog, LogCNN, and LogTransformer after training are frozen, and the frozen DeepLog, LogCNN, and LogTransformer are integrated into a heterogeneous detector.

[0017] Further, determining the degree of anomalous evidence retention of the candidate event ID sequence based on the key anomaly pattern and the anomaly event ID sequence includes: From the critical anomaly patterns, critical anomaly events and critical anomaly transitions are identified. The retention degree of critical events is determined based on the retention of critical anomaly events in the anomaly event ID sequence and the candidate event ID sequence. The retention degree of critical transitions is determined based on the retention of critical anomaly transitions in the candidate event ID sequence. The degree of preservation of anomalous evidence is determined based on the degree of preservation of key events, the degree of preservation of key transitions, the consistency of anomalous patterns between the anomalous event ID sequence and the candidate event ID sequence, and the degree of statistical anomalousness of the candidate event ID sequence relative to the distribution of normal events.

[0018] Furthermore, a joint admission judgment is performed on the candidate event ID sequence based on the degree of preservation of anomalous evidence, the rationality of the structure, and the extent of rewriting, and a reward is determined based on the joint admission judgment result and the detection result, including: The degree of preservation of the abnormal evidence, the rationality of the structure, and the extent of rewriting are compared with the corresponding admission thresholds, and it is determined whether the candidate event ID sequence is the same as the abnormal event ID sequence. A replicative penalty is applied to candidate event ID sequences that are identical to the anomalous event ID sequence. An invalid sample penalty is applied to candidate event ID sequences that are different from the anomalous event ID sequence and fail the joint admission judgment. An attack feedback term is determined based on the detection results for candidate event ID sequences that are different from the anomalous event ID sequence and pass the joint admission judgment. The final reward is determined in combination with the auxiliary reward term. The penalty intensity of the replicative penalty is greater than that of the invalid sample penalty.

[0019] Furthermore, for candidate event ID sequences that differ from the aforementioned abnormal event ID sequence and are determined through joint admission criteria, an attack feedback item is determined based on the detection results, and a final reward is determined in conjunction with an auxiliary reward item, including: Based on the normal class probabilities of each deep anomaly detector for the candidate event ID sequence in the detection results, determine the minimum and weighted average of each normal class probability, and determine the cross-detector variance penalty based on the difference between each normal class probability. Then, determine the attack feedback term based on the minimum, the weighted average and the cross-detector variance penalty. Auxiliary reward items are determined based on the structural rationality of the candidate event ID sequence, the degree of preservation of key anomalies, the number of deceived deep anomaly detectors, and the indication that all deep anomaly detectors are avoided. The attack feedback item and the auxiliary reward item are combined according to the principle that the weight of the attack feedback item is greater than the weight of the auxiliary reward item to obtain the final reward.

[0020] Furthermore, the log rewriter is fine-tuned using reinforcement learning based on the reward to obtain an adversarial log generator, including: Each decoding token in the process of generating the candidate event ID sequence by the log rewriter is taken as an action, the complete candidate event ID sequence is taken as a trajectory, and the sequence-level reward determined after generation is allocated to each decoding token. An adversarial log generator is obtained by optimizing the objective by employing a pruning strategy with relative entropy penalty for near-ends and updating the strategy parameters of the log rewriter according to the allocated sequence-level rewards.

[0021] Further, the sequence of exception event IDs to be rewritten is input into the adversarial log generator to obtain an adversarial log sequence, including: Construct rewriting prompt words based on the sequence of abnormal event IDs to be rewritten; The rewrite prompt is input into the adversarial log generator to rewrite the event ID sequence, thus obtaining the adversarial log sequence.

[0022] An adversarial log generation system based on reinforcement learning of a large language model includes: The log parsing module is used to parse the original logs using log templates to obtain an event ID sequence, and to determine key anomaly patterns based on the abnormal event ID sequence in the event ID sequence. The supervised fine-tuning module is used to perform anomaly-aware supervised fine-tuning on the large language model using supervised fine-tuning samples constructed based on the anomaly event ID sequence and the key anomaly pattern, to obtain a log rewriter. The reinforcement learning module is used to perform joint admission judgment and determine the reward for the candidate event ID sequence generated by the log rewriter based on the detection results of the frozen heterogeneous detector integration, as well as the degree of retention of abnormal evidence, structural rationality and rewriting range determined based on the key anomaly pattern. Based on the reward, the log rewriter is fine-tuned by reinforcement learning to obtain an adversarial log generator. The adversarial log generation module is used to input the sequence of abnormal event IDs to be rewritten into the adversarial log generator to obtain an adversarial log sequence.

[0023] The present invention has achieved the following beneficial effects.

[0024] 1. This invention employs a large language model to adversarially rewrite event ID sequences. It utilizes the sequence modeling and instruction following capabilities of the large language model to generate event ID sequences with valid structure and reasonable context. Furthermore, it uses low-rank adaptation to perform anomaly detection supervision and fine-tuning on the large language model. By injecting log rewriting priors into the large language model while freezing the original weights, it reduces training resource overhead and catastrophic forgetting, while also improving the problem of generating reasonable rewrites based on rule perturbation.

[0025] 2. This invention utilizes frozen heterogeneous detector integration to provide detection feedback on candidate event ID sequences, and fine-tunes the log rewriter through reinforcement learning by optimizing the near-end policy, enabling the log rewriter to update its policy based on the responses of anomaly detectors with different architectures; combined with the modeling capabilities of DeepLog, LogCNN, and LogTransformer for different event dependencies, it can reduce the tendency to perform speculative rewriting against a single detector and generate adversarial log sequences to expose the detection blind spots of anomaly detectors at different depths.

[0026] 3. This invention employs an attack-first additive reward mechanism, using the degree of preservation of anomalous evidence, structural rationality, and rewriting magnitude as joint admission criteria. Penalties are imposed on identity-replicated samples and invalid samples that fail the joint admission judgment. Rewards are determined only for candidate event ID sequences that pass the joint admission judgment, based on attack feedback terms and auxiliary reward terms. The attack feedback term integrates the minimum, weighted average, and cross-detector variance penalties of the normal class probabilities of each deep anomaly detector, and makes the attack feedback term dominant in the final reward. This coordinates attack effectiveness, anomaly effectiveness, and rewriting novelty, reduces the degradation of anomaly labels and identity replication, and improves the cross-architecture applicability of adversarial examples.

[0027] 4. The adversarial log sequences generated by this invention achieve high levels of success rate in operational attacks, preservation of anomalous evidence, and novelty in rewriting. They exhibit better overall performance compared to random replacement, word vector nearest neighbor replacement, masked language model rewriting, and genetic algorithm generation methods. The generated adversarial log sequences can be used to detect robustness gaps in DeepLog, LogCNN, and LogTransformer, and can be used as incremental reinforcement training data to improve the recall and F1 score of the detector on the mixed evaluation set, thereby supporting the robustness evaluation and incremental reinforcement of deep log anomaly detectors. Attached Figure Description

[0028] Figure 1 A schematic diagram defining the adversarial log sequence generation problem.

[0029] Figure 2 This is an overall framework diagram of an adversarial log generation method based on reinforcement learning of a large language model according to the present invention.

[0030] Figure 3 This is a comparison diagram of the automated production line and the reward ablation experiment of the present invention.

[0031] Figure 4 This is a comparison chart of the detector performance before and after incremental hardening according to the present invention.

[0032] Figure 5 This is a graph showing the changes in runtime and peak GPU memory usage of the present invention at different generation scales. Detailed Implementation

[0033] To make the various technical features, advantages, or effects of the present invention more apparent and understandable, detailed descriptions are provided below through embodiments.

[0034] This invention provides an adversarial log generation method based on reinforcement learning of a large language model. The method targets anomaly event ID sequences, obtains a log rewriter through anomaly perception-supervised fine-tuning, and fine-tunes the log rewriter through reinforcement learning based on the detection results of candidate event ID sequences by frozen heterogeneous detector integration, as well as the degree of anomalous evidence retention, structural rationality, and rewriting magnitude of the candidate event ID sequences, to obtain an adversarial log generator. The adversarial log generator is then used to generate adversarial log sequences.

[0035] The problem of this invention is defined as follows: Figure 1 As shown. For the same underlying exception execution, the event sequence surface can present different exception patterns. Historical exception log sequence The system possesses anomaly patterns learned during the anomaly detector's training process, which can be identified as anomalies by the detector. While preserving the underlying anomalous behavior and the key anomalous events and transitions supporting the anomaly labels, the non-critical surface structure of the event sequence is rewritten to obtain the anomaly log sequence where surface pattern shifts occur. Due to the abnormal log sequence Compared to historical anomaly log sequences Surface pattern changes occur, abnormal log sequences It can deviate from historical anomaly patterns learned by the anomaly detector. The adversarial log sequence generation task defined in this invention is: given a historical anomaly log sequence... Generate a sequence of historical anomaly logs. Anomaly log sequences with surface differences and preservation of key evidence of anomalies and utilize the abnormal log sequence As an adversarial log sequence.

[0036] The overall framework of the present invention is as follows Figure 2 As shown. Figure 2 The training process demonstrated consists of three stages: Stage 1 prepares the event ID sequence representation and the frozen heterogeneous detector integration, and mines key anomaly patterns; Stage 2 obtains the log rewriter through supervised fine-tuning based on low-rank adaptation anomaly perception; Stage 3 uses AFAR sequence-level rewards for proximal policy optimization and performs reinforcement learning fine-tuning on the log rewriter. After training, the obtained adversarial log generator is used to rewrite the sequence of anomaly event IDs to be rewritten, resulting in an adversarial log sequence.

[0037] In one embodiment of the present invention, the verification dataset includes an HDFS dataset and a Hadoop dataset. The HDFS dataset contains approximately 11 million log messages and block-level anomaly labels; the Hadoop dataset contains MapReduce execution traces, covering machine-down, network-disconnection, and disk-full failures.

[0038] Step S1: Parse the log template of the original log to obtain the event ID sequence, and determine the key exception mode based on the abnormal event ID sequence in the event ID sequence.

[0039] In an optional embodiment of the present invention, step S1 may include: Step S11: Parse the log template of the original log and map each log to the corresponding event ID to obtain the event ID sequence.

[0040] Specifically, a log template parsing algorithm is used to parse the original log text into log templates, and each log template is mapped to a corresponding event ID, thereby converting the original log into an event ID sequence. The log template parsing algorithm can be Drain. Taking the HDFS dataset as an example, after parsing, a total of 48 event templates are obtained. Each original log entry is mapped to a corresponding event ID, and an event ID sequence is formed according to the block-level log organization relationship, serving as input for subsequent anomaly detectors and log rewriters.

[0041] Step S12: Determine the normal event ID sequence and the abnormal event ID sequence from the event ID sequence based on the log exception label.

[0042] Specifically, the event ID sequence is divided according to the log anomaly label corresponding to the log data. The event ID sequence corresponding to the normal label is determined as the normal event ID sequence, and the event ID sequence corresponding to the anomaly label is determined as the anomaly event ID sequence.

[0043] Step S13: Determine the key anomaly pattern based on the event distribution and local temporal relationship of the abnormal event ID sequence and the normal event ID sequence.

[0044] Specifically, key anomalous events are identified based on the occurrence of event IDs in the anomalous event ID sequence and the normal event ID sequence. Key anomalous transitions are then identified based on the occurrence of adjacent event ID combinations within the anomalous event ID sequence. These key anomalous events and transitions form key anomalous patterns. Key anomalous events are event IDs that frequently appear in the anomalous log but less frequently in the normal log. Key anomalous transitions are high-frequency anomalous event pairs within the anomalous event ID sequence, used to characterize local temporal patterns between anomalous events. Key anomalous events and transitions constitute crucial anomalous evidence supporting anomalous labeling.

[0045] Step S2: Using supervised fine-tuning samples constructed based on the abnormal event ID sequence and key abnormal patterns, perform anomaly-aware supervised fine-tuning on the large language model to obtain the log rewriter.

[0046] In an optional embodiment of the present invention, step S2 may include: Step S21: Construct instructions and corresponding target rewritten event ID sequences based on the abnormal event ID sequence, key abnormal patterns, and rewriting requirements to form a supervised fine-tuning sample.

[0047] Specifically, the instructions include the original anomaly sequence, a key event template description, and rewriting requirements. The target output is the rewritten event ID sequence. For the anomaly event ID sequences in the training set, pairs of supervised fine-tuning samples are constructed using four automatic rewriting methods: replacement, dispersion, obfuscation, and compounding.

[0048] (1) The replacement method replaces non-critical events in the abnormal event ID sequence with event IDs with similar contexts to obtain replacement rewrite samples.

[0049] (2) The distributed method inserts normal operation event IDs between error bursts in the abnormal event ID sequence to obtain distributed rewrite samples.

[0050] (3) Obfuscation method places key abnormal events in a context that is closer to the normal log and formed by normal operation event IDs to obtain obfuscated rewrite samples.

[0051] (4) The composite method combines the non-critical event replacement, normal operation event insertion and critical abnormal event context adjustment on the same abnormal event ID sequence to obtain a composite rewrite sample.

[0052] A supervised fine-tuning sample is formed based on the replacement rewrite sample, the dispersed rewrite sample, the obfuscated rewrite sample, and the composite rewrite sample. The supervised fine-tuning sample includes the sequence of abnormal event IDs, the description of the key abnormal pattern, the rewrite requirements, and the corresponding target rewritten event ID sequence.

[0053] Step S22: Use supervised fine-tuning samples to perform anomaly-aware supervised fine-tuning on the large language model to obtain the log rewriter.

[0054] Specifically, general-purpose large language models are not naturally suited for event ID sequence generation tasks. When directly used for event ID sequence rewriting, they may generate free text, invalid event IDs, or empty responses, or generate rewritten results that fail to retain key anomaly evidence. This embodiment uses low-rank adaptation (LoRA) to perform anomaly-aware supervised fine-tuning of the large language model.

[0055] For the original weight matrix of the linear layer to be adapted in the large language model Freeze the original weight matrix And set a trainable low-rank matrix. and , by low-rank matrix and This forms a low-rank increment. The adapted equivalent weight is... The equivalent weights are derived from the frozen original weight matrix. Compared with the low-rank weight increments obtained during training Formed by superposition.

[0056] The low-rank matrix is ​​trained using the rewritten event ID sequence from the supervised fine-tuning samples as the supervision target. The objective function of anomaly-aware supervised fine-tuning is: in, The loss function representing anomaly-aware supervised fine-tuning; This represents the large language model strategy parameters to be fine-tuned. This represents the set of samples for supervised fine-tuning. This indicates an instruction that includes the original exception sequence, a key event template description, and rewrite requirements; This represents the sequence of event IDs after the target has been rewritten; This indicates the first event in the rewritten event ID sequence. One token; Indicates the first The sequence of tokens preceding each token; This represents the large language model strategy that needs to be fine-tuned.

[0057] Through anomaly-aware supervised fine-tuning, the large language model generates a structurally valid sequence of event IDs according to the event ID rewriting instructions. During the rewriting process, the anomaly structure supporting the anomaly labels is preserved, and identity copying is reduced, resulting in a log rewriter. This log rewriter serves as the initialization strategy for subsequent reinforcement learning fine-tuning.

[0058] Step S3: Based on the detection results of the candidate event ID sequence generated by the log rewriter through the frozen heterogeneous detector integration, as well as the degree of preservation of abnormal evidence, structural rationality and rewriting range determined based on the key anomaly pattern, the candidate event ID sequence is jointly admitted and a reward is determined. Based on the reward, the log rewriter is fine-tuned through reinforcement learning to obtain an adversarial log generator.

[0059] In an optional embodiment of the present invention, step S3 may include: Step S31: Determine the normal event ID sequence based on the event ID sequence, train multiple deep anomaly detectors using the normal event ID sequence, and freeze the trained model parameters to obtain a frozen heterogeneous detector ensemble.

[0060] Specifically, DeepLog, LogCNN, and LogTransformer are trained independently using normal event ID sequences. DeepLog employs a recurrent next-event prediction mechanism to predict the next event in the event sequence; LogCNN uses a one-dimensional convolutional filter to extract local n-gram patterns within a window; and LogTransformer uses a self-attention mechanism to extract long-range dependencies between events. The three deep anomaly detectors employ complementary event sequence modeling methods to form a heterogeneous detector ensemble.

[0061] DeepLog, LogCNN, and LogTransformer freeze their model parameters after training. During subsequent candidate event ID sequence evaluation and reinforcement learning fine-tuning, the model parameters are not updated; only forward prediction is performed, and the probability of a candidate event ID sequence being judged as normal is output. This is relevant to the first step in heterogeneous detector ensembles. A depth anomaly detector The forward prediction score is represented as Specifically, a deep anomaly detector Output the normal class probability for the input event ID sequence.

[0062] Step S32: Generate candidate event ID sequences using a log rewriter, detect the candidate event ID sequences using a frozen heterogeneous detector integration, obtain detection results, determine the degree of abnormal evidence retention of the candidate event ID sequences based on key abnormal patterns and abnormal event ID sequences, determine the structural rationality based on the candidate event ID sequences, and determine the rewriting extent based on the candidate event ID sequences and abnormal event ID sequences.

[0063] Specifically, in each rollout of the reinforcement learning fine-tuning, the log rewriter receives the rewrite prompt and samples it to generate a sequence of candidate event IDs. The frozen heterogeneous detector ensemble is used for candidate event ID sequences respectively. Perform forward prediction to obtain the normal class probability corresponding to each depth anomaly detector. .

[0064] The degree of anomalous evidence retention in the candidate event ID sequence is further determined based on key anomaly patterns. The degree of anomalous evidence retention comprehensively considers key event preservation, key transition preservation, anomaly pattern consistency, and the presence of a given source anomalous event ID sequence. Candidate event ID sequence under the condition The degree of statistical anomaly relative to the distribution of normal events is determined and characterized by the anomaly confidence level: in, Represents the sequence of candidate event IDs Relative to the source exception event ID sequence The anomaly confidence level is used to characterize the degree to which the candidate event ID sequence retains key anomalous evidence from the source anomalous event ID sequence; Indicates the sequence of source exception event IDs; This represents the sequence of candidate event IDs.

[0065] When determining the degree of preservation of anomalous evidence, the degree of preservation of key anomalous events is determined based on the preservation of key anomalous events in the source anomalous event ID sequence and the candidate event ID sequence, the degree of preservation of key anomalous transfers is determined based on the preservation of key anomalous transfers in the candidate event ID sequence, and the degree of preservation of anomalous evidence is determined by combining the consistency of anomalous patterns between the source anomalous event ID sequence and the candidate event ID sequence and the statistical anomalousness of the candidate event ID sequence relative to the distribution of normal events.

[0066] Structural rationality is used to evaluate the structural rationality of the candidate event ID sequence; the rewriting magnitude is determined based on the difference between the candidate event ID sequence and the source abnormal event ID sequence, and is used to characterize the degree of rewriting of the candidate event ID sequence relative to the source abnormal event ID sequence.

[0067] Step S33: Perform a joint admission judgment on the candidate event ID sequence based on the degree of preservation of abnormal evidence, the rationality of the structure, and the extent of rewriting, and determine the reward based on the joint admission judgment result and the detection result.

[0068] Specifically, the AFAR sequence-level reward rule is used to score candidate event ID sequences. AFAR first performs a joint admission check on the candidate event ID sequences, considering anomaly validity, structural rationality, and rewrite magnitude, and determines whether the candidate event ID sequences are identical to the source anomalous event ID sequences. Fixed penalties are applied to identically replicated samples and invalid samples that fail the joint admission check. Attack feedback and auxiliary reward terms are calculated only for candidate event ID sequences that pass the joint admission check and are not identically replicated. The AFAR sequence-level reward rule is expressed as follows: in, This represents the sequence-level reward corresponding to the candidate event ID sequence; This represents the fixed penalty for identically replicated samples; This indicates the fixed penalty for invalid samples that fail the joint access inspection; This indicates the results of the joint access inspection; Indicates an attack feedback item; Indicates supplementary reward items; This indicates the weight corresponding to the attack feedback item; This indicates the weight of the auxiliary reward item.

[0069] The joint access inspection also requires: in, The validity of evidence for anomalies is determined based on the retention of key anomaly evidence from the source anomaly event ID sequence within the candidate event ID sequence. This indicates the structural rationality of the candidate event ID sequence; This represents the normalized rewrite magnitude of the candidate event ID sequence relative to the source abnormal event ID sequence; Indicates the admission threshold corresponding to abnormal validity evidence; The threshold value corresponding to the rationality of the structure; This represents the admission threshold corresponding to the normalized rewrite magnitude.

[0070] The fixed penalty for identically replicated samples and the fixed penalty for invalid samples that fail the joint access check satisfy the following conditions: This relationship indicates that the penalty for identically replicated samples is stronger than the penalty for invalid samples that fail the joint access check.

[0071] For candidate event ID sequences that differ from the source anomalous event ID sequence and pass the joint admission check, the attack feedback item... Based on the normal class probability output by each depth anomaly detector The integration feedback is defined as including three types: (1) the minimum normal class probability of each depth anomaly detector; (2) the weighted average of the normal class probability of each depth anomaly detector; and (3) the cross-detector variance penalty of the normal class probability of each depth anomaly detector.

[0072] Auxiliary Rewards The reward is determined based on the structural rationality of the candidate event ID sequence, the degree of preservation of key anomalies, the number of deceived deep anomaly detectors, and the indication that all deep anomaly detectors are evaded. Attack feedback and auxiliary reward terms are added together to form the final reward corresponding to the candidate event ID sequence that passes the joint admission check, and satisfy the following conditions: This weighting relationship makes the attack feedback item the dominant item in the final reward, while the auxiliary reward item is a subordinate item. The retention and rewriting magnitude of anomalous evidence are used as admission criteria for joint admission checks, rather than as positive reward items after passing the joint admission checks.

[0073] Step S34: Fine-tune the log rewriter using reinforcement learning based on the reward to obtain an adversarial log generator.

[0074] Specifically, a near-end policy optimization (PPO) approach is used to fine-tune the log rewriter using reinforcement learning. In each rollout, each decoded token during the log rewriter's generation of a candidate event ID sequence is treated as an action, and the complete candidate event ID sequence is considered a trajectory. After the candidate event ID sequence is generated, a sequence-level reward is calculated and distributed to the token-level updates according to the PPO update method.

[0075] A standard pruning strategy with relative entropy (KL) penalty is adopted to update the log rewriter for the PPO objective, ensuring that the updated policy does not deviate too far from the reference model. During reinforcement learning fine-tuning, the LoRA-adapted policy parameters are updated based on the sequence-level reward determined by AFAR, resulting in the adversarial log generator.

[0076] Step S4: Input the sequence of exception event IDs to be rewritten into the adversarial log generator to obtain the adversarial log sequence.

[0077] In an optional embodiment of the present invention, step S4 may include: Step S41: Construct rewrite prompt words based on the sequence of exception event IDs to be rewritten.

[0078] Specifically, a rewrite prompt word is formed based on the sequence of exception event IDs to be rewritten to indicate the rewriting of the event ID sequence, and the rewrite prompt word is used as input to the adversarial log generator.

[0079] Step S42: Input the rewrite prompt into the adversarial log generator to rewrite the event ID sequence, and obtain the adversarial log sequence.

[0080] Specifically, the adversarial log generator generates a rewritten sequence of event IDs based on rewrite prompts, forming an adversarial log sequence. The generated adversarial log sequence retains key anomaly events and key anomaly transitions that support the anomaly labels, while also being consistent with the source anomaly event ID sequence. It forms surface pattern differences and can evaluate detection avoidance using the detection results of frozen heterogeneous detector integration.

[0081] In one embodiment of the present invention, a trained adversarial log generator is used to rewrite a known sequence of anomalous event IDs to obtain an adversarial log set, and the generation result is evaluated by ensemble attack success rate, anomaly retention rate, rewriting rate and operation attack success rate.

[0082] Order No. The sequence of individual source anomaly event IDs is represented as follows: The corresponding generated sequence is represented as , This represents the number of sample indices after filtering by the basic vocabulary and length; [Note: The original text contains some formatting errors and inconsistencies. A more accurate translation would require the full context.] Source anomaly event ID sequence The set of sample indices that have been identified as anomalous by at least one deep anomaly detector; denoted as To generate sequences Originally capable of identifying the corresponding source of abnormal event ID sequence The set of sample indexes for integrated avoidance in a deep anomaly detector; [denotes...] To generate sequences The sample index set examined through anomalous evidence; [Note: The original text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] Source anomaly event ID sequence With generated sequence The set of sample indexes with a normalized edit distance of not less than 0.15.

[0083] The success rate of an operational attack is expressed as: in, Indicates the success rate of the attack. This represents the number of evaluation samples used to calculate the success rate of operational attacks. This indicates the number of sample indices in the set. The success rate statistics for operational attacks represent the proportion of samples whose source anomaly event ID sequences can be identified by at least one deep anomaly detector, and whose corresponding generated sequences simultaneously satisfy the conditions of ensemble avoidance, anomaly evidence preservation, and nontrivial rewriting.

[0084] The success rate of an integrated attack is expressed as: in, This represents the success rate of the ensemble attack, used to characterize how well the generated sequence evades the ensemble of heterogeneous detectors, without limiting the effectiveness of the generated sequence's abnormality.

[0085] The abnormal retention rate is expressed as: in, The anomaly retention rate is used to characterize the proportion of generated sequences that retain key anomaly evidence from the source anomaly event ID sequence and pass anomaly evidence checks.

[0086] The rewrite rate is expressed as: in, Represents the rewrite rate, which characterizes the proportion of generated sequences that meet the non-trivial rewrite condition relative to the source anomalous event ID sequence.

[0087] In one embodiment of the present invention, adversarial log sequences with validity gating are used for incremental reinforcement training of the original deep anomaly detector.

[0088] Comparative Test: This invention was tested on the HDFS and Hadoop datasets. The proposed method was compared with four representative generation baselines, including: Random (replacing 30% of the event positions with uniformly sampled event IDs as a noise-free baseline), Embed-NN (using nearest neighbors in the Word2Vec space to replace events, representing local context rewriting), MLM (using a BERT-style masked language model, representing distribution-aware but detector-independent rewriting), and GA (black-box discrete optimization based on genetic algorithm to directly optimize the ensemble attack score, representing a baseline without anomaly validity constraints).

[0089] 1. Cross-dataset generation validity comparison This experiment compares the ensemble ASR, APR, RR, and operational ASR of the five methods on the HDFS dataset and the Hadoop dataset respectively. The evaluation results are shown in Tables 2 and 3.

[0090] Table 2 Comparison of generation effectiveness on HDFS datasets Table 3 Comparison of generation validity on Hadoop datasets As shown in Tables 2 and 3, the two datasets exhibit a consistent method hierarchy. Detector-independent rewriting methods (Random, Embed-NN, MLM) offer only limited evasion capabilities, while the unconstrained genetic algorithm, although achieving a high ensemble ASR (85.0%) on HDFS, suffers from a reduced APR (Average Perception) of only 14.7% due to its sacrifice of anomalous evidence preservation, resulting in an operational ASR of only 4.1%. On Hadoop, the genetic algorithm's RR drops to 61.7%, similarly limiting its usable output. In contrast, the proposed method achieves the highest operational ASR (57.2% and 30.8%, respectively) on both HDFS and Hadoop, while maintaining high levels in both APR and RR. Although the absolute values ​​vary with the corpus, the trade-off between attack and effectiveness remains consistent across different event vocabularies and fault distributions, indicating that the proposed method can stably generate adversarial logs that simultaneously satisfy attack effectiveness, anomalous effectiveness, and rewriting novelty across datasets.

[0091] 2. Detector-level operational ASR comparison To further verify whether the improvement of operation ASR holds true for all deep detectors, this experiment reports detector-level operation ASR for DeepLog, LogCNN and LogTransformer on the HDFS dataset, and the results are shown in Table 4.

[0092] Table 4. Detector-level operations ASR (%) on the HDFS dataset As shown in Table 4, the proposed method achieves the highest operational ASR on all three heterogeneous detectors, with advantages of 41.4, 31.0, and 41.3 percentage points respectively compared to the strongest non-inventive baseline. After effectiveness gating, the operational ASR of the genetic algorithm on all three detectors is lower than the noise baseline, indicating that its high ensemble ASR is not a robust detector-level advantage. This result shows that the overall performance gain of the proposed method is not dominated by a single particularly vulnerable detector, but rather exhibits cross-architecture transferable adversarial capabilities on three different detector architectures.

[0093] 3. Automated Ablation Experiment To differentiate the contributions of task-supervised adaptation and detector feedback optimization to the final performance, this experiment compares the performance of a Base LLM without task adaptation, an SFT-only pipeline with only supervised fine-tuning, and a complete SFT+PPO pipeline under the same data, decoding budget, and evaluation protocol. The results are as follows: Figure 3 Figure (a) shows the comparison results of Base LLM, SFT-only and the complete pipeline of the present invention in integrating ASR, APR, RR and operational ASR.

[0094] from Figure 3 As shown in Figure (a), although Base LLM can change the input surface, its APR is only 63.4%, indicating that a general large language model that has not been task-adapted is difficult to stably retain anomalous evidence. The SFT-only stage improves the APR to 100.0% and basically eliminates identity copying, but its ensemble ASR is only 5.9%, indicating that learning reasonable rewrites alone is not enough to expose the robustness gaps of the detector. After adding detector feedback PPO, both the ensemble ASR and the operational ASR are improved to 57.2%, while the APR and RR remain above 97%. This comparison shows that supervised fine-tuning mainly establishes effective rewrite priors, while PPO provides adversarial capabilities against the decision boundaries of the detector. The two are complementary and indispensable.

[0095] 4. Reward Ablation Experiment To verify the advantages of AFAR compared to other reward designs, this experiment started from the same SFT checkpoint, keeping the PPO optimization and decoding settings completely consistent, and only changing the reward function for comparison. Product gating multiplies attack, validity, and rewrite-related scores; a low score in any one of these factors lowers the overall reward. Strict gating requires candidates to simultaneously meet stricter validity conditions; otherwise, a penalty is imposed. AFAR, on the other hand, uses validity as an admission condition and performs additive optimization primarily based on attack feedback after a candidate passes the gating. The results are as follows: Figure 3 Figure (b) shows the comparison results of product gating, strict gating and the AFAR reward proposed in this invention under the same SFT initialization and PPO configuration.

[0096] from Figure 3 As shown in Figure (b), while the product reward can preserve anomalous evidence, the RR is only 54.2%, indicating that the policy is prone to convergence to a degenerate solution of near-replication. Strict gating improves the RR to 76.2%, but reduces the integrated ASR to 1.8%, indicating that excessively strong admission conditions will significantly inhibit the effective exploration of the policy. The AFAR proposed in this invention avoids the above two failure modes. The attack-priority additive structure restores stronger detection avoidance ability while maintaining high effectiveness and rewriting rate. This result shows that the performance improvement of this invention comes from the priority optimization of attack feedback after the sample passes through the effectiveness gating, rather than simple multiplication or tightening of all targets at the same time.

[0097] 5. Detector hardening effect To further verify the downstream value of the adversarial examples generated in this invention, this experiment examines the effect of the generated logs on incremental hardening of the detector on the HDFS dataset. The evaluation employs a three-stage protocol: M1 evaluates the original detector on a clean reference set; M2 evaluates the same original detector on a mixed test set with adversarial examples added; M3 starts from the original detector weights, performs incremental training using only an adversarial training subset, and evaluates on the same hold-out test set as M2. The recall, F1 score, and adversarial recall of the three detectors on the mixed set before and after hardening are shown below. Figure 4 As shown, the three sub-figures respectively illustrate the recall, F1 score, and adversarial recall of the DeepLog, LogCNN, and LogTransformer detectors.

[0098] from Figure 4As can be seen, after adding adversarial examples, the evaluation shifts from the clean reference M1 setting to the more difficult M2 mixture setting. The recall and F1 scores of all three detectors decrease significantly, with the overall adversarial recall being low. This indicates that the adversarial examples generated in this invention expose robustness gaps across detector architectures, rather than accidental failures of a single detector. Comparing M2 and M3 on the exact same mixture set shows that incremental reinforcement using the adversarial training subset generated in this invention significantly restores the robustness of all three detectors. DeepLog and LogTransformer show more significant improvements, with adversarial recall increasing from 50.4% and 54.4% to 88.7% and 79.0%, respectively. LogCNN's gains are relatively limited. One possible reason is that DeepLog and LogTransformer model longer event dependencies through recurrent states and self-attention, respectively, making it easier for the transfer signals contained in the boundary samples generated in this invention to be absorbed into their sequence-level discrimination rules. LogCNN, on the other hand, relies more on short-range patterns captured by local convolutional windows, and the same sample budget has a weaker effect on adjusting its representation space. Overall, the adversarial examples generated by this invention can serve as stress test samples to expose the detector's missed detection problems on boundary anomalies, and can also serve as targeted hardening data to significantly improve the robustness of the detector.

[0099] 6. Computational efficiency To verify the practical deployment feasibility of this invention, this experiment evaluated the end-to-end inference overhead at different generation scales on a single RTX 4090 (24 GB) graphics card, including total generation time, peak VRAM allocated by Torch, and peak VRAM of the entire NVIDIA-SMI card. The results are as follows: Figure 5 As shown, where Figure 5 Figure (a) shows the curve of total generation time as a function of the number of generated samples, with a linear fit attached. Figure 5 Figure (b) shows the curves of the peak video memory allocated by Torch and the peak video memory of the entire NVIDIA-SMI card as a function of the number of generated samples.

[0100] from Figure 5As can be seen, the runtime increases approximately proportionally with the number of output samples, requiring about 0.20 seconds per sample within the measured range. This trend applies to the offline workflow targeted by this invention: adversarial samples are pre-generated in batches and subsequently used for detector evaluation or hardening, rather than being part of the online detection path. Regarding memory overhead, the tensor memory allocated in this invention is consistently below 9 GB; even including CUDA context, cache memory, and runtime workspace, the peak total GPU size is approximately 19 GB. Therefore, the complete generation process can run on a single, common 24 GB GPU, without requiring multiple GPUs. Combined with the approximately linear runtime increase, this resource characteristic demonstrates the feasibility of deploying this invention as an offline robustness evaluation and detector hardening component in a real-world production environment.

[0101] In summary, the adversarial log sequence generation method based on large language models and reinforcement learning proposed in this invention achieved the highest operational attack success rate on both HDFS and Hadoop real system log datasets, while maintaining a high anomaly evidence retention rate and rewriting novelty. The adversarial examples generated by this invention can expose the robustness gaps of various heterogeneous deep detectors, and significantly improve the detector's recall, F1 score, and adversarial recall rate after being used for incremental reinforcement training. The running time and memory overhead of this invention are within the range that a single common GPU can bear, making it feasible for practical applications.

[0102] Although the present invention has been disclosed above with reference to embodiments, it is not intended to limit the present invention. Appropriate modifications or equivalent substitutions made by those skilled in the art to the technical solutions of the present invention should be covered within the protection scope of the present invention, which is defined by the claims.

Claims

1. An adversarial log generation method based on reinforcement learning of a large language model, characterized in that, Includes the following steps: The original log is parsed using a log template to obtain an event ID sequence, and key anomaly patterns are determined based on the abnormal event ID sequence in the event ID sequence. Using supervised fine-tuning samples constructed based on the abnormal event ID sequence and the key abnormal patterns, anomaly-aware supervised fine-tuning is performed on the large language model to obtain a log rewriter; Based on the detection results of the candidate event ID sequence generated by the log rewriter by the frozen heterogeneous detector ensemble, and the degree of retention of anomalous evidence, structural rationality and rewriting range determined based on the key anomaly pattern, the candidate event ID sequence is jointly admitted and a reward is determined. Based on the reward, the log rewriter is fine-tuned by reinforcement learning to obtain an adversarial log generator. The sequence of exception event IDs to be rewritten is input into the adversarial log generator to obtain the adversarial log sequence.

2. The method as described in claim 1, characterized in that, Using supervised fine-tuning samples constructed based on the abnormal event ID sequence and the key abnormal patterns, anomaly-aware supervised fine-tuning is performed on a large language model to obtain a log rewriter, including: Based on the abnormal event ID sequence, the key abnormal pattern, and the rewriting requirements, construct instructions and the corresponding target rewritten event ID sequence to form a supervised fine-tuning sample. The large language model is subjected to anomaly-aware supervised fine-tuning using the supervised fine-tuning samples to obtain a log rewriter.

3. The method as described in claim 2, characterized in that, Based on the abnormal event ID sequence, the key abnormal patterns, and the rewriting requirements, instructions and corresponding target rewritten event ID sequences are constructed to form a supervised fine-tuning sample, including: Based on the event context, non-critical events in the abnormal event ID sequence are replaced with event IDs with similar contexts to obtain replacement and rewriting samples; Insert normal operation event IDs between the abnormal events in the abnormal event ID sequence to obtain a distributed rewrite sample; By placing the key abnormal events in the key abnormal pattern into a context formed by the normal operation event ID, an obfuscated rewrite sample is obtained. For the same abnormal event ID sequence combination, perform non-critical event replacement, normal operation event insertion, and critical abnormal event context adjustment to obtain a composite rewrite sample, and form a supervised fine-tuning sample based on the replacement rewrite sample, the dispersed rewrite sample, the obfuscated rewrite sample, and the composite rewrite sample.

4. The method as described in claim 2, characterized in that, The large language model is subjected to anomaly-aware supervised fine-tuning using the supervised fine-tuning samples to obtain a log rewriter, including: Freeze the original weights of the linear layer to be adapted in the large language model, and set a trainable low-rank matrix for the linear layer to be adapted; Using the rewritten event ID sequence in the supervised fine-tuning sample as the supervision target, the low-rank matrix is ​​trained using the supervised fine-tuning sample to obtain the log rewriter.

5. The method as described in claim 1, characterized in that, Based on the detection results of the candidate event ID sequence generated by the log rewriter using the frozen heterogeneous detector ensemble, and the degree of preservation of anomalous evidence, structural rationality, and rewriting magnitude determined based on the key anomaly pattern, a joint admission judgment is made on the candidate event ID sequence and a reward is determined. Based on the reward, the log rewriter is fine-tuned using reinforcement learning to obtain an adversarial log generator, including: Based on the event ID sequence, a normal event ID sequence is determined. Multiple deep anomaly detectors are trained using the normal event ID sequence, and the trained model parameters are frozen to obtain a frozen heterogeneous detector ensemble. The log rewriter generates a candidate event ID sequence, the frozen heterogeneous detector is integrated to detect the candidate event ID sequence, the detection result is obtained, the degree of abnormal evidence retention of the candidate event ID sequence is determined according to the key abnormal pattern and the abnormal event ID sequence, the structural rationality is determined according to the candidate event ID sequence, and the rewriting range is determined according to the candidate event ID sequence and the abnormal event ID sequence. The candidate event ID sequence is jointly admitted based on the degree of preservation of abnormal evidence, the rationality of the structure, and the extent of rewriting, and the reward is determined based on the joint admission judgment result and the detection result. The log rewriter is fine-tuned using reinforcement learning based on the rewards to obtain an adversarial log generator.

6. The method as described in claim 5, characterized in that, Determining the degree of anomalous evidence retention of the candidate event ID sequence based on the key anomaly pattern and the anomaly event ID sequence includes: From the critical anomaly patterns, critical anomaly events and critical anomaly transitions are identified. The retention degree of critical events is determined based on the retention of critical anomaly events in the anomaly event ID sequence and the candidate event ID sequence. The retention degree of critical transitions is determined based on the retention of critical anomaly transitions in the candidate event ID sequence. The degree of preservation of anomalous evidence is determined based on the degree of preservation of key events, the degree of preservation of key transitions, the consistency of anomalous patterns between the anomalous event ID sequence and the candidate event ID sequence, and the degree of statistical anomalousness of the candidate event ID sequence relative to the distribution of normal events.

7. The method as described in claim 5, characterized in that, A joint admission judgment is performed on the candidate event ID sequence based on the degree of preservation of anomalous evidence, the rationality of the structure, and the extent of rewriting, and a reward is determined based on the joint admission judgment result and the detection result, including: The degree of preservation of the abnormal evidence, the rationality of the structure, and the extent of rewriting are compared with the corresponding admission thresholds, and it is determined whether the candidate event ID sequence is the same as the abnormal event ID sequence. A replicative penalty is applied to candidate event ID sequences that are identical to the anomalous event ID sequence. An invalid sample penalty is applied to candidate event ID sequences that are different from the anomalous event ID sequence and fail the joint admission judgment. An attack feedback term is determined based on the detection results for candidate event ID sequences that are different from the anomalous event ID sequence and pass the joint admission judgment. The final reward is determined in combination with the auxiliary reward term. The penalty intensity of the replicative penalty is greater than that of the invalid sample penalty.

8. The method as described in claim 7, characterized in that, For candidate event ID sequences that differ from the aforementioned abnormal event ID sequence and are determined through joint admission criteria, an attack feedback item is determined based on the detection results, and a final reward is determined in conjunction with an auxiliary reward item, including: Based on the normal class probabilities of each deep anomaly detector for the candidate event ID sequence in the detection results, determine the minimum and weighted average values ​​of each normal class probability, and determine the cross-detector variance penalty based on the differences between each normal class probability. Then, determine the attack feedback term based on the minimum value, the weighted average value, and the cross-detector variance penalty. Auxiliary reward items are determined based on the structural rationality of the candidate event ID sequence, the degree of preservation of key anomalies, the number of deceived deep anomaly detectors, and the indication that all deep anomaly detectors are avoided. The attack feedback item and the auxiliary reward item are combined according to the principle that the weight of the attack feedback item is greater than the weight of the auxiliary reward item to obtain the final reward.

9. The method as described in claim 5, characterized in that, The log rewriter is fine-tuned using reinforcement learning based on the rewards to obtain an adversarial log generator, including: Each decoded token in the process of generating the candidate event ID sequence by the log rewriter is taken as an action, the complete candidate event ID sequence is taken as a trajectory, and the sequence-level reward determined after generation is allocated to each decoded token. An adversarial log generator is obtained by optimizing the objective by employing a pruning strategy with relative entropy penalty for near-ends and updating the strategy parameters of the log rewriter according to the allocated sequence-level rewards.

10. An adversarial log generation system based on reinforcement learning of a large language model, characterized in that, include: The log parsing module is used to parse the original logs using log templates to obtain an event ID sequence, and to determine key anomaly patterns based on the abnormal event ID sequence in the event ID sequence. The supervised fine-tuning module is used to perform anomaly-aware supervised fine-tuning on the large language model using supervised fine-tuning samples constructed based on the anomaly event ID sequence and the key anomaly pattern, to obtain a log rewriter. The reinforcement learning module is used to perform joint admission judgment and determine the reward for the candidate event ID sequence generated by the log rewriter based on the detection results of the frozen heterogeneous detector integration, as well as the degree of retention of abnormal evidence, structural rationality and rewriting range determined based on the key anomaly pattern. Based on the reward, the log rewriter is fine-tuned by reinforcement learning to obtain an adversarial log generator. The adversarial log generation module is used to input the sequence of abnormal event IDs to be rewritten into the adversarial log generator to obtain an adversarial log sequence.

Images