Intrusion detection rule adaptive generation and optimization method based on large language model

Abstract

The application discloses an intrusion detection rule self-adaptive generation and optimization method based on a large language model, and belongs to the technical field of network security, comprising the following steps: collecting multi-source original data and performing pretreatment to obtain a high-quality data set; obtaining a key mode based on the high-quality data set and a fine-tuned large language model, and generating an initial rule template based on the key mode; generating new candidate rules by combining the large language model reasoning based on real-time traffic data and the initial rule template, and sorting the candidate rules; optimizing and screening the candidate rules by using a deep reinforcement learning algorithm and a genetic algorithm; and distributing and synchronously updating the candidate rules optimized and screened through a distributed architecture. The application realizes the automatic generation and dynamic optimization of intrusion detection rules, significantly improves the real-time detection capability for new unknown threats, and greatly reduces the false alarm rate and system maintenance cost.

Description

Technical Field

[0001] This invention belongs to the field of network security technology, and in particular relates to an adaptive generation and optimization method for intrusion detection rules based on a large language model. Background Technology

[0002] In recent years, cyberattacks have shown a trend of increasing complexity, enhanced concealment, expanded scale, and diversified forms. Attackers utilize advanced techniques such as zero-day vulnerabilities and APT attacks to bypass traditional defense mechanisms, and new attack behaviors are often hidden in normal traffic, making them difficult to detect through static rules or simple statistical models. In addition, threats such as large-scale distributed denial-of-service (DDoS) attacks and botnets have caused serious damage to network infrastructure.

[0003] Traditional intrusion detection systems (IDS / IPS) mainly rely on predefined static rule bases or statistical models, which have the following problems:

[0004] Delayed updates: Static rule bases require manual maintenance, have long update cycles, and are difficult to respond to new threats in a timely manner;

[0005] High false alarm rate: Rigid rules may cause a large amount of normal traffic to be misjudged as abnormal behavior;

[0006] Low coverage: Static rules struggle to cover all possible attack patterns, especially performing poorly in complex network environments;

[0007] High maintenance costs: Expanding and optimizing the rule base requires significant manual intervention;

[0008] Insufficient real-time performance: Traditional methods are difficult to meet the detection requirements in high-concurrency network environments.

[0009] Breakthroughs in large language models (such as GPT and BERT) in the field of natural language processing have opened up new possibilities for their application in cybersecurity. These models possess powerful semantic understanding, reasoning, and generalization capabilities, enabling them to extract attack patterns and key features from unstructured data and quickly adapt to new attack behaviors, significantly reducing the lag in rule updates. Meanwhile, intelligent optimization techniques such as reinforcement learning and genetic algorithms also provide strong support for dynamic rule generation and optimization. Summary of the Invention

[0010] To address the aforementioned technical problems, this invention proposes an adaptive generation and optimization method for intrusion detection rules based on a large language model, thereby resolving the issues present in the existing technologies.

[0011] To achieve the above objectives, this invention provides an adaptive generation and optimization method for intrusion detection rules based on a large language model, comprising:

[0012] Collect raw data from multiple sources and preprocess it to obtain a high-quality dataset;

[0013] Key patterns are obtained based on the high-quality dataset and the fine-tuned large language model, and initial rule templates are generated based on the key patterns.

[0014] Based on real-time traffic data and the initial rule template, new candidate rules are generated by combining large language model inference, and the candidate rules are sorted.

[0015] Deep reinforcement learning and genetic algorithms are used to optimize and screen candidate rules;

[0016] The optimized candidate rules are distributed and synchronously updated using a distributed architecture.

[0017] Optionally, the process of collecting and preprocessing raw data from multiple sources includes:

[0018] Distributed crawling technology is used to collect data from multiple network nodes in parallel, while a real-time stream processing framework is used for data filtering and aggregation. A sliding window algorithm is used to analyze the processed data and extract key features. Based on the extracted key features, a semi-supervised learning algorithm combined with an active learning algorithm is used for data annotation to obtain a high-quality dataset. The high-quality dataset includes historical normal behavior data and historical intrusion behavior data.

[0019] Optionally, the process of obtaining key patterns based on the high-quality dataset and the fine-tuned large language model includes: selecting a subset of historical intrusion behavior data from the high-quality dataset; inputting the subset of historical intrusion behavior data into the fine-tuned large language model for semantic analysis and association reasoning, and extracting key patterns representing attack behavior.

[0020] Optionally, the process of obtaining a fine-tuned large language model includes:

[0021] Using a pre-trained large language model as the base model, a classification head for intrusion detection tasks is added to the top of the network structure. The model with the added classification head is trained using the high-quality dataset. Dropout and L2 regularization techniques are introduced during the training process to obtain a fine-tuned large language model. The training objective is to minimize the cross-entropy loss function.

[0022] Optionally, the process of generating new candidate rules based on real-time traffic data and the initial rule template, combined with large language model reasoning, includes: using the real-time traffic data and the initial rule template as context information to input fine-tuning of the large language model, extracting behavioral semantics from the real-time traffic data; and reasoning and generating new candidate rules based on the behavioral semantics and the initial rule template.

[0023] Optionally, a priority scoring function is constructed to calculate the priority scores of candidate rules, and the candidate rules are sorted in descending order based on the calculation results. The priority scoring function is as follows:

[0024] ;

[0025] in, The score indicates the priority of the rule. The severity score indicates the rule's severity. A score indicating the urgency of the rule. Indicates the frequency at which the rule is triggered. For severity weighting, As an urgency weight, For frequency weights, and satisfying .

[0026] Optionally, the process of optimizing and screening candidate rules using deep reinforcement learning algorithms and genetic algorithms includes:

[0027] A multi-dimensional evaluation system for candidate rules is constructed and rule scores are calculated. Using the rule scores as reward signals, a deep reinforcement learning algorithm is used to interact with the environment to optimize the rule parameters and performance of the candidate rules. The optimized candidate rules are used as the initial population of a genetic algorithm to calculate the fitness value of each rule, which is obtained by mapping the rule score through the Sigmoid function. Selection, crossover, and mutation operations are iteratively performed on the population to select the subset of rules with the highest fitness, thus completing the optimization and selection of candidate rules.

[0028] Optionally, the evaluation system includes coverage, false alarm rate, and execution efficiency.

[0029] Optionally, the process of distributing and synchronously updating the optimized candidate rules using a distributed architecture includes:

[0030] The update process is triggered based on a preset periodic time window or real-time security event detection results; a consistent hashing algorithm is used to determine the target distribution location of rules in distributed nodes, and load balancing efficiency is improved by expanding virtual nodes; rule data is distributed to each node in parallel through a message queue with partition processing and replica redundancy mechanisms; each node receives and applies rule updates, and the reliability and efficiency of the distribution process are ensured by the partitioning and replica mechanisms of the message queue.

[0031] Optionally, the method also includes a quality assessment of the initial rule template, the process of which includes: constructing a comprehensive scoring function and quantitatively evaluating the initial rule template using the comprehensive scoring function; wherein the comprehensive scoring function is a weighted sum of coverage, precision and recall.

[0032] Compared with the prior art, the present invention has the following advantages and technical effects:

[0033] This invention integrates technologies such as Natural Language Processing (NLP), Time Series Analysis, Reinforcement Learning (RL), Genetic Algorithm (GA), and Distributed Computing to achieve automated generation and continuous optimization of intrusion detection rules. It can extract attack patterns from unstructured data, quickly respond to emerging threats, and dynamically adjust the rule base through intelligent optimization algorithms, significantly improving detection efficiency and accuracy. This addresses the problems of delayed rule updates, high false positive rates, and low coverage inherent in traditional intrusion detection systems. Applicable to various complex scenarios such as enterprise networks, cloud computing environments, the Internet of Things (IoT), and edge computing, this invention provides an efficient and intelligent security protection solution for critical infrastructure.

[0034] This invention introduces a large language model into the field of intrusion detection rule generation for the first time, realizing intelligent rule generation; it introduces reinforcement learning and genetic algorithms to continuously optimize the rules, improving the quality and efficiency of the rules; and it proposes a rule synchronization mechanism based on a distributed architecture to ensure efficient distribution of rules in large-scale networks. Attached Figure Description

[0035] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0036] Figure 1 This is a system architecture diagram according to an embodiment of the present invention;

[0037] Figure 2 This is a flowchart illustrating the rule generation and optimization process in an embodiment of the present invention.

[0038] Figure 3 This is a schematic diagram illustrating the interaction between the large language model and the intrusion detection system according to an embodiment of the present invention;

[0039] Figure 4 This is a schematic diagram illustrating the rule priority calculation in an embodiment of the present invention;

[0040] Figure 5 This is a schematic diagram of the rule optimization algorithm in an embodiment of the present invention. Detailed Implementation

[0041] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0042] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0043] Example 1.

[0044] (a) Data acquisition and preprocessing;

[0045] The process of collecting and preprocessing raw data from multiple sources includes:

[0046] Distributed crawling technology is used to collect data from multiple network nodes in parallel, while a real-time stream processing framework is used for data filtering and aggregation. A sliding window algorithm is used to analyze the processed data and extract key features. Based on the extracted key features, a semi-supervised learning algorithm combined with an active learning algorithm is used for data annotation to obtain a high-quality dataset. The high-quality dataset includes historical normal behavior data and historical intrusion behavior data.

[0047] Specifically, data acquisition is a fundamental step in intrusion detection systems, requiring the gathering of raw data from multiple sources. First, open-source tools such as Wireshark and Zeek, or commercial tools like Splunk, are used to capture and record network traffic in real time. The captured data includes source IP addresses, destination IP addresses, protocol types (such as TCP and UDP), port numbers, and traffic volume. Second, system logs are extracted from the operating system and critical services, such as Linux's syslog or Windows EventLog. These logs record user behavior and system state changes. Finally, alerts from IDS / IPS, firewalls, and other security devices are integrated to form a historical attack database. For example, SSH brute-force attacks can be labeled as LABEL: SSH bruteforce attack.

[0048] During the data cleaning phase, regular expressions are used to remove invalid fields or noisy data, missing values ​​are filled (e.g., using the mean to fill numerical data), and the timestamp format is standardized to the ISO 8601 standard (e.g., YYYY-MM-DDTHH:MM:SSZ). Subsequently, key features are extracted, including source IP address, destination IP address, source port, destination port, protocol type, traffic volume, and request frequency. To capture the time-dependent and periodic characteristics of network behavior, a sliding window algorithm is used to extract behavioral patterns within a time window, as shown in the following formula:

[0049] ;

[0050] in, Indicates the first The eigenvalues ​​at time t, Indicates the first The raw data at that moment This indicates the size of the sliding window.

[0051] The annotation process combines semi-supervised learning algorithms (such as self-training) and expert knowledge bases to label anomalous behaviors. For example, frequently accessing a specific port is labeled as LABEL: SSH brute forceattack. Furthermore, an active learning algorithm is introduced to select the most representative samples for manual annotation, thereby obtaining a high-quality dataset with minimal manual cost. Self-training is used as the semi-supervised learning algorithm, combined with uncertainty sampling as the active learning algorithm for data annotation.

[0052] The specific process for determining the most representative sample is as follows:

[0053] (1) Self-training the current model: using a small number of labeled samples ,in, Indicates sample features, This indicates the corresponding label. The total number of samples. The initial CNN sample classification model is used for training. ,in, These are the model parameters. The goal of the model is to minimize the loss function:

[0054] ;

[0055] in, This is the cross-entropy loss function.

[0056] (2) Calculate the uncertainty of the samples: For each unlabeled sample, predict its class probability distribution using the current model and calculate the entropy as a measure of uncertainty. The formula is as follows:

[0057] ;

[0058] in, Indicates sample Uncertainty Indicates sample Predicted as the first The probability of a class This indicates the total number of categories.

[0059] (3) Select high uncertainty samples: Select the samples with the highest entropy values ​​from all unlabeled samples as the most representative samples.

[0060] To further improve the comprehensiveness and accuracy of data collection, distributed crawling technology (such as Scrapy) is introduced to crawl data from multiple network nodes in parallel, and high-speed data streams are filtered and aggregated using real-time stream processing frameworks (such as Apache Flink). For example, IP addresses receiving more than 100 requests per second are marked as potential threats.

[0061] (ii) Rule template construction;

[0062] Based on the high-quality dataset and the fine-tuned large language model, key patterns are obtained, and initial rule templates are generated based on the key patterns. The process includes: selecting a subset of historical intrusion behavior data from the high-quality dataset; inputting the subset of historical intrusion behavior data into the fine-tuned large language model for semantic analysis and association reasoning, and extracting key patterns representing attack behavior.

[0063] The process of obtaining a fine-tuned large language model includes:

[0064] Using a pre-trained large language model as the base model, a classification head for intrusion detection tasks is added to the top of the network structure. The model with the added classification head is trained using the high-quality dataset. Dropout and L2 regularization techniques are introduced during the training process to obtain a fine-tuned large language model. The training objective is to minimize the cross-entropy loss function.

[0065] The process of quality evaluation of the initial rule template includes: constructing a comprehensive scoring function and using the comprehensive scoring function to quantitatively evaluate the initial rule template; wherein the comprehensive scoring function is a weighted sum of coverage, precision and recall.

[0066] Specifically, the rule template is constructed based on semantic analysis of historical intrusion behavior data using a large language model. First, historical intrusion behavior data, such as logs of SSH brute-force attacks, is input, and key patterns are extracted using a large language model (such as GPT). For example, the pattern "frequent access to a specific port" is extracted. An initial rule template is then generated based on the extracted patterns, in the following form:

[0067] RULE: IF source_ip IN [blacklist] AND destination_port = 22 THENalert("SSH brute force attack");

[0068] To improve the applicability of the rules, new conditions can be added dynamically. For example, time window limits or geographic location filtering can be added, expanding the rules to the following forms:

[0069] RULE: IF source_ip IN [blacklist] AND destination_port = 22 AND time_window > 5min THEN alert("SSH brute force attack");

[0070] To quantify the quality of rule templates, a scoring function for rule templates is defined, as follows:

[0071] ;

[0072] in, This indicates the score of the rule template. Indicates the coverage of the rule template. Indicates the precision of the rule template. This indicates the recall rate of the rule template. For coverage weight, For accuracy weighting, As the recall rate weight, and satisfying .

[0073] To further improve the performance of large language models in intrusion detection, domain-specific fine-tuning is performed. During fine-tuning, a pre-trained large language model (such as GPT) is used as the base model, and a classification head is added to it for rule generation. Training data includes labeled historical attack logs and normal traffic logs. The cross-entropy loss function is used, as shown in the following formula:

[0074] ;

[0075] in, Indicates the loss value. Indicates the number of samples. Indicates the number of categories. Indicates the first The true label of each sample Indicates the first The predicted value for the nth sample is... The probability of a class. To prevent overfitting, Dropout and L2 regularization are introduced.

[0076] (iii) Adaptive rule generation;

[0077] Based on real-time traffic data and the initial rule template, new candidate rules are generated by combining large language model inference, and the candidate rules are sorted.

[0078] The process of generating new candidate rules based on real-time traffic data and the initial rule template, combined with large language model reasoning, includes: using the real-time traffic data and the initial rule template as context information to input fine-tuning of the large language model, extracting behavioral semantics from the real-time traffic data; and reasoning and generating new candidate rules based on the behavioral semantics and the initial rule template.

[0079] A priority scoring function is constructed to calculate the priority scores of candidate rules, and the candidate rules are sorted in descending order based on the calculation results.

[0080] Specifically, the core of adaptive rule generation is to use a large language model to reason about real-time traffic data, infer possible attack patterns, and generate candidate rules. For example:

[0081] RULE: IF source_ip IN [new_blacklist] AND destination_port = 443 THENalert("HTTPS brute force attack");

[0082] Specifically, to determine the priority of a rule, a priority score is calculated using the following formula:

[0083] ;

[0084] in, Indicates the score based on rule priority. The severity score indicates the rule's severity. A score indicating the urgency of the rule. Indicates the frequency at which the rule is triggered. For severity weighting, As an urgency weight, For frequency weights, and satisfying .

[0085] (iv) Rule optimization and verification;

[0086] The candidate rules are optimized and screened using deep reinforcement learning and genetic algorithms. The process includes:

[0087] A multi-dimensional evaluation system for candidate rules is constructed and rule scores are calculated. Using the rule scores as reward signals, a deep reinforcement learning algorithm is used to interact with the environment to optimize the rule parameters and performance of the candidate rules. The optimized candidate rules are used as the initial population of a genetic algorithm to calculate the fitness value of each rule, which is obtained by mapping the rule score through the Sigmoid function. Selection, crossover, and mutation operations are iteratively performed on the population to select the subset of rules with the highest fitness, thus completing the optimization and selection of candidate rules.

[0088] Specifically, rule optimization consists of two parts: deep reinforcement learning and genetic algorithms. Deep reinforcement learning tests rule performance by constructing a simulated environment and updates rule parameters using a Q-value function, as shown in the following formula:

[0089] ;

[0090] in, Represents the state-action value function. Indicates the current state. This indicates the action taken (i.e., the generated rule). Indicates the reward value. Indicates the discount factor. This represents the learning rate.

[0091] Genetic algorithms select rules by calculating the fitness value of each rule, as shown in the following formula:

[0092] ;

[0093] in, This represents the fitness value of the rule. Let represent the rule score, and exp() be the exponential function. A new generation of rule populations is generated through selection, crossover, and mutation operations until convergence.

[0094] (v) Rule updates and deployment;

[0095] The optimized candidate rules are distributed and synchronously updated using a distributed architecture. The process includes:

[0096] The update process is triggered based on preset periodic time windows or real-time security event detection results; a consistent hashing algorithm is used to determine the target distribution location of rules in distributed nodes, and load balancing efficiency is improved by expanding virtual nodes; rule data is distributed to each node in parallel through a message queue with partition processing and replica redundancy mechanism; each node receives and applies rule updates, and the partitioning and replica mechanism of the message queue ensures the reliability and efficiency of the distribution process, ultimately achieving synchronous updates and consistent status of the entire network rule base.

[0097] Specifically, rule updates support two modes: periodic updates and event-driven dynamic adjustments. Periodic updates update the rule base regularly based on time windows, such as daily or weekly; event-driven updates trigger rule generation when a significant security event is detected. For example:

[0098] EVENT: High-frequency access to critical API detected;

[0099] ACTION: Generate new rule and update rule base;

[0100] To ensure efficient rule distribution across a large-scale network, a distributed architecture is used to synchronize rules to all nodes in the network. A consistent hashing algorithm is used to ensure the uniformity of rule distribution; the formula is as follows:

[0101] ;

[0102] in, Represents the hash value. Indicates input data, and For random parameters, It is a prime number. To further improve efficiency, virtual node consistent hashing is introduced, with the following formula:

[0103] ;

[0104] Message queues (such as Kafka) enable efficient rule distribution, while partitioning and replication mechanisms improve throughput and fault tolerance. The rule generation and optimization processes included in embodiments (II) to (V) are as follows: Figure 2 As shown in the figure. The interaction diagram between the large language model and the intrusion detection system in this embodiment is shown in the figure. Figure 3 As shown.

[0105] Example 2

[0106] like Figure 1 As shown, this embodiment provides an adaptive generation and optimization method for intrusion detection rules based on a large language model, including:

[0107] (a) Data acquisition and preprocessing;

[0108] To build high-quality intrusion detection rules, it is first necessary to collect raw data from multiple sources, including network traffic logs, system logs, and historical alarm records. Network traffic logs can be collected using open-source tools (such as Wireshark and Zeek) and contain information such as IP addresses, port numbers, protocol types, and traffic volume. System logs record user behavior and system status, such as Linux syslog and Windows Event Log. Historical alarm records integrate IDS / IPS alarms, firewall logs, etc., forming a rich database of historical attack behaviors.

[0109] During the data cleaning phase, regular expressions are used to remove noisy fields, fill in missing values, and standardize the timestamp format to the ISO 8601 standard (e.g., YYYY-MM-DDTHH:MM:SSZ). Subsequently, key features are extracted, including source IP address, destination IP address, source port, destination port, protocol type, traffic volume, and request frequency. To capture the time-dependent and periodic characteristics of network behavior, a sliding window algorithm is used to extract behavioral patterns within a time window, with the following formula:

[0110] ;

[0111] in, Indicates the first The eigenvalues ​​at time t, Indicates the first The raw data at that moment This indicates the size of the sliding window.

[0112] Finally, the data is labeled, and anomalous behaviors are marked using a semi-supervised learning algorithm (such as self-training) combined with an expert knowledge base. For example, SSH brute-force attack behavior is labeled as LABEL: SSH brute forceattack.

[0113] To further improve the comprehensiveness and accuracy of data collection, distributed web crawling technologies (such as Scrapy) and real-time stream processing frameworks (such as Apache Flink) are introduced for data collection and preprocessing. Distributed web crawling technologies can crawl data from multiple network nodes in parallel, while real-time stream processing frameworks can filter, aggregate, and store high-speed data streams in real time.

[0114] In the feature extraction stage, in addition to traditional statistical features (such as mean, variance, maximum, and minimum values), features based on time series analysis can be introduced, such as Fourier transform and wavelet transform to extract frequency domain features. The formula is as follows:

[0115] ;

[0116] in, Represents the frequency domain characteristics of a signal. Represents a time-domain signal. Indicates frequency, It is the imaginary unit.

[0117] Furthermore, to improve the accuracy and efficiency of annotation, an active learning algorithm is introduced to select the most representative samples for manual annotation. The core idea is to iteratively select the samples with the highest uncertainty for annotation, thereby obtaining high-quality labeled data with minimal manual cost.

[0118] (ii) Rule template construction;

[0119] During the rule template construction phase, a large language model is used to perform semantic analysis on historical intrusion behavior data, extract key patterns, and generate initial rule templates. For example, by analyzing historical SSH brute-force attack events, the pattern of "frequent access to specific ports" is extracted, and the following rule template is generated:

[0120] RULE: IF source_ip IN [blacklist] AND destination_port = 22 THENalert("SSH brute force attack");

[0121] To improve the applicability and scalability of the rules, it supports dynamically adding new features, such as adding time windows and geographical location conditions:

[0122] RULE: IF source_ip IN [blacklist] AND destination_port = 22 AND time_window > 5min THEN alert("SSH brute force attack");

[0123] To quantify the quality of generated rule templates, a scoring function for rule templates is defined as follows:

[0124] ;

[0125] in, This indicates the score of the rule template. Indicates the coverage of the rule template. Indicates the precision of the rule template. This indicates the recall rate of the rule template. For coverage weight, For accuracy weighting, As the recall rate weight, and satisfying .

[0126] (III) Fine-tuning and optimization of the large language model;

[0127] To further improve the performance of the large language model in intrusion detection, domain-specific fine-tuning is performed. During fine-tuning, a pre-trained large language model is used as the base model, and a classification head is added to it for rule generation. The cross-entropy loss function is used, as shown in the following formula:

[0128] ;

[0129] in, Indicates the loss value. Indicates the number of samples. Indicates the number of categories. Indicates the first The true label of each sample Indicates the first The predicted value for the nth sample is... The probability of a class.

[0130] In addition, to prevent overfitting, regularization techniques are introduced, such as Dropout and L2 regularization. The formula for Dropout is as follows:

[0131] ;

[0132] in, Represents the output of a neuron. This represents a random mask matrix, whose elements are set to 0 with a certain probability.

[0133] (iv) Adaptive rule generation;

[0134] In the adaptive rule generation phase, the large language model infers possible attack patterns based on real-time traffic data and generates new rules. Its core formula is as follows:

[0135] ;

[0136] in, Indicates the first New rules generated at any time Indicates the first Real-time context information at any given moment Represents the historical rule base. Represents the inference function of a large language model.

[0137] To determine the priority of a rule, a priority score is calculated using the following formula:

[0138] ;

[0139] in, Indicates the score based on rule priority. The severity score indicates the rule's severity. A score indicating the urgency of the rule. Indicates the frequency at which the rule is triggered. For severity weighting, As an urgency weight, For frequency weights, and satisfying A diagram illustrating the rule priority calculation process is shown below. Figure 4 As shown.

[0140] (v) Rule optimization and verification;

[0141] During the rule optimization and verification phase, the generated rules are evaluated from multiple dimensions, including metrics such as coverage, false positive rate, and execution efficiency. A schematic diagram of the rule optimization algorithm is shown below. Figure 5 As shown. The formula for calculating the rule score is as follows:

[0142] ;

[0143] in, Indicates the rule score. Indicates rule coverage. Indicates the false alarm rate. Indicates execution efficiency. These are the rule coverage weight, false positive rate weight, and execution efficiency weight, respectively, and they satisfy the following conditions: .

[0144] To further optimize the rules, a Deep Reinforcement Learning (DRL) algorithm is introduced, with the following formula:

[0145] ;

[0146] in, Represents the state-action value function. Indicates the current state. This indicates the action taken (i.e., the generated rule). Indicates the reward value. Indicates the discount factor. This represents the learning rate.

[0147] Furthermore, a genetic algorithm is used to filter the rules and calculate the fitness function for each individual:

[0148] ;

[0149] in, This represents the fitness value of the rule. This represents the rule score. A new generation of rule populations is generated through selection, crossover, and mutation operations until convergence.

[0150] (vi) Rule updates and deployment;

[0151] Rule updates support both periodic updates and event-driven dynamic adjustments. Periodic updates update the rule base periodically based on a time window, which can be adjusted as needed (e.g., daily, weekly). Event-driven updates are triggered when a significant security event is detected, for example:

[0152] EVENT: High-frequency access to critical API detected;

[0153] ACTION: Generate new rule and update rule base;

[0154] To ensure efficient rule distribution across a large-scale network, a distributed architecture is used to synchronize rules to all nodes in the network. Consistent hashing is used to ensure the uniformity of rule distribution, and its formula is as follows:

[0155] ;

[0156] in, Represents the hash value. Indicates input data, and For random parameters, It is a prime number. Message queues (such as Kafka) enable efficient distribution of rules.

[0157] In distributed architectures, to improve the efficiency and reliability of rule distribution, an improved version of the consistent hashing algorithm—Virtual Node Consistent Hashing—is introduced. The introduction of virtual nodes reduces the impact of changes in physical nodes on the hash ring. Its formula is as follows:

[0158] ;

[0159] in, The hash value representing the virtual node. Indicates the virtual node number.

[0160] In addition, to improve the throughput and latency performance of message queues, partitioning and replication mechanisms are introduced. Partitioning divides messages into multiple subsets, with each partition processed by an independent consumer; replication improves the system's fault tolerance through redundant storage.

[0161] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An adaptive generation and optimization method for intrusion detection rules based on a large language model, characterized in that, Includes the following steps: Collect raw data from multiple sources and preprocess it to obtain a high-quality dataset; The process of collecting and preprocessing raw data from multiple sources includes: Distributed crawling technology is used to collect data from multiple network nodes in parallel, while a real-time stream processing framework is used for data filtering and aggregation. A sliding window algorithm is used to analyze the processed data and extract key features. Based on the extracted key features, a semi-supervised learning algorithm combined with an active learning algorithm is used for data annotation to obtain a high-quality dataset. The high-quality dataset includes historical normal behavior data and historical intrusion behavior data. Key patterns are obtained based on the high-quality dataset and the fine-tuned large language model, and initial rule templates are generated based on the key patterns. Based on real-time traffic data and the initial rule template, new candidate rules are generated by combining large language model inference, and the candidate rules are sorted. Deep reinforcement learning and genetic algorithms are used to optimize and screen candidate rules; The optimized candidate rules are distributed and synchronously updated using a distributed architecture; The process of generating new candidate rules based on real-time traffic data and the initial rule template, combined with reasoning from a large language model, includes: inputting the real-time traffic data and the initial rule template as context information into a fine-tuned large language model to extract behavioral semantics from the real-time traffic data; and reasoning and generating new candidate rules based on the behavioral semantics and the initial rule template. A priority scoring function is constructed to calculate the priority scores of candidate rules. Based on the calculation results, the candidate rules are sorted in descending order. The priority scoring function is as follows: ； in, The score indicates the priority of the rule. The severity score indicates the rule's severity. A score indicating the urgency of the rule. Indicates the frequency at which the rule is triggered. For severity weighting, As an urgency weight, For frequency weights, and satisfying ; The process of optimizing and screening candidate rules using deep reinforcement learning algorithms and genetic algorithms includes: A multi-dimensional evaluation system for candidate rules is constructed and rule scores are calculated. Using the rule scores as reward signals, a deep reinforcement learning algorithm is used to interact with the environment to optimize the rule parameters and performance of the candidate rules. The optimized candidate rules are used as the initial population for a genetic algorithm, and the fitness value of each rule is calculated. The fitness value is obtained by mapping the rule score through the Sigmoid function. Selection, crossover, and mutation operations are iteratively performed on the population to select the subset of rules with the highest fitness, thus completing the optimization and selection of candidate rules. The process of distributing and synchronously updating the optimized candidate rules using a distributed architecture includes: The update process is triggered based on a preset periodic time window or real-time security event detection results; a consistent hashing algorithm is used to determine the target distribution location of rules in the distributed nodes, and load balancing efficiency is improved by expanding virtual nodes; rule data is distributed to each node in parallel through a message queue with partition processing and replica redundancy mechanism; each node receives and applies rule updates, and the partition and replica mechanism of the message queue ensures the reliability and efficiency of the distribution process, realizing synchronous updates and consistent states of candidate rules.

2. The adaptive generation and optimization method for intrusion detection rules based on a large language model according to claim 1, characterized in that, The process of obtaining key patterns based on the high-quality dataset and the fine-tuned large language model includes: selecting a subset of historical intrusion behavior data from the high-quality dataset; inputting the subset of historical intrusion behavior data into the fine-tuned large language model for semantic analysis and association reasoning, and extracting key patterns representing attack behavior.

3. The adaptive generation and optimization method for intrusion detection rules based on a large language model according to claim 1, characterized in that, The process of obtaining a fine-tuned large language model includes: Using a pre-trained large language model as the base model, a classification head for intrusion detection tasks is added to the top of the network structure. The model with the added classification head is trained using the high-quality dataset. Dropout and L2 regularization techniques are introduced during the training process to obtain a fine-tuned large language model. The training objective is to minimize the cross-entropy loss function.

4. The adaptive generation and optimization method for intrusion detection rules based on a large language model according to claim 1, characterized in that, The evaluation system includes coverage, false alarm rate, and execution efficiency.

5. The adaptive generation and optimization method for intrusion detection rules based on a large language model according to claim 1, characterized in that, It also includes a quality assessment of the initial rule template, the process of which includes: constructing a comprehensive scoring function and using the comprehensive scoring function to quantitatively evaluate the initial rule template; wherein the comprehensive scoring function is a weighted sum of coverage, precision and recall.

Images