Reasonable judgment method for large model-based intrusion detection scheme

Abstract

The application discloses a rationality judgment method of an intrusion detection scheme based on a large model, relates to the technical field of network security, and comprises the following steps: collecting multidimensional data, and forming a high-quality data set by cleaning, labeling and format conversion of the collected multidimensional data; selecting a security pre-training large model or a general model based on the high-quality data set to perform training and parameter optimization; and constructing an interpretable threat mode and rule association graph. The application upgrades static rule verification to a dynamic intelligent evaluation system through rationality judgment of the intrusion detection scheme based on the large model, and through comprehensive coverage of the data layer, field adaptation of the model layer, multidimensional quantification of the evaluation layer and closed-loop feedback of the optimization layer, the application can not only more accurately judge the rationality of the scheme, but also actively cope with unknown threats, simplify the rule optimization process, control the risk of false positives and false negatives, and provide more adaptive protection for the network security system.

Description

Technical Field

[0001] This invention belongs to the field of network security technology, specifically relating to a method for judging the rationality of intrusion detection schemes based on large models. Background Technology

[0002] Intrusion detection systems are a key component of cybersecurity architecture. They are a complete set of strategies, methods, and technologies that collect and analyze information such as behavior, security logs, and audit data in computer systems or networks to identify potential intrusions or violations of security policies. In order for a reasonable intrusion detection system to accurately identify potential threats, its rationality must be effectively judged. If the system is unreasonable, it may lead to a large number of false alarms or missed alarms, making it difficult for the security team to deal with real threats.

[0003] Currently, the rationality of intrusion detection solutions is usually judged by rule verification. This involves checking whether the intrusion detection rules are accurate and effective, analyzing whether the rules can correctly identify known threats, and assessing whether they generate too many false alarms. For example, in SIEM or EDR products, rules are tested and optimized to ensure that they match the specific threats faced by the organization.

[0004] However, rule verification is mainly based on known threats and has limited ability to detect unknown attacks or new attack patterns. Moreover, rule adjustments require continuous trial and optimization, which can be a complex process and makes it difficult to completely avoid false positives and false negatives. Summary of the Invention

[0005] The purpose of this invention is to provide a rationality judgment method for intrusion detection schemes based on large models, which can proactively respond to unknown threats, simplify the rule optimization process, and control the risk of false positives and false negatives, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] The rationality judgment method for intrusion detection schemes based on large models includes the following steps:

[0008] S1. Collect multi-dimensional data and clean, label, and convert the collected multi-dimensional data to form a high-quality dataset. The multi-dimensional data includes known threat data, unknown attack simulation data, historical false negative data, and business baseline data.

[0009] S2. Based on high-quality datasets, select a safe pre-trained large model or a general model for training and parameter optimization to obtain a prediction model;

[0010] S3. Extract explicit known threat features and implicit unknown attack association features through prediction models, and construct an interpretable threat pattern and rule association map;

[0011] S4. Conduct a reasonableness assessment from multiple dimensions, including threat coverage, false negative rate prediction, and business scenario adaptability. If the threat coverage, false negative rate prediction, and business scenario adaptability assessment all meet the standards, the assessment is reasonable; otherwise, the assessment is unreasonable.

[0012] S5. Collect operational feedback data, automatically optimize the prediction model and detection rules, and repeat the evaluation until the preset standards are met.

[0013] Preferably, when cleaning multidimensional data, a hash deduplication algorithm is used to delete duplicate logs, then regular expressions are used to match the log structure and filter out non-matching logs. For numerical data, the DBSCAN clustering algorithm is used to identify outliers and remove them. Finally, logs missing key fields are completed by associating with other logs.

[0014] When annotating the cleaned multi-dimensional data, a combination of manual and automatic annotation is used. During format conversion, unstructured text logs are converted into vectors that can be recognized by the prediction model. This includes text segmentation and encoding, and log vector concatenation. Text segmentation and encoding involves splitting keywords from the log text using a security domain segmentation tool, and then converting the keywords into word vectors through a conversion model.

[0015] Preferably, the secure pre-trained large model performs the following training and parameter optimization process:

[0016] A1. Load the pre-trained weights officially released by the secure pre-trained large model;

[0017] A2. Select core data for security scenarios from high-quality datasets and divide them into training and validation sets in an 8:2 ratio;

[0018] A3. Based on the training set data, a low-rank adaptation algorithm is used to insert low-rank matrices into the key layers of a safe pre-trained large model and train the inserted matrices.

[0019] A4. Traverse the key hyperparameter combinations of the low-rank adaptation algorithm through grid search, test the attack identification accuracy on the validation set, and select the parameter combination with the highest accuracy as the final model parameters.

[0020] Preferably, the general-purpose model performs the following training and parameter optimization process:

[0021] B1. Select known threat data and security document data from high-quality datasets to perform incremental pre-training on the general model, injecting security knowledge;

[0022] B2. Enhance security domain features on high-quality datasets to generate text pairs of logs and tactical descriptions;

[0023] If the amount of data is sufficient, perform full fine-tuning; if the amount of data is small, perform prefix fine-tuning.

[0024] B3. To address the issue of a limited number of unknown attack samples, a weighted cross-entropy loss method is used to increase the loss weight of unknown attack samples.

[0025] B4. Optimize the parameters of the general model based on the calculated loss value until it meets the prediction requirements.

[0026] Preferably, the prediction models that are predicted and optimized by using a security pre-trained large model or a general model all have the ability to automatically extract features, learn threat patterns, and adapt to different scenarios.

[0027] Preferably, explicit known threat feature extraction is based on the IOC library and ATT&CK knowledge built into the prediction model, directly matching known features from log vectors, while implicit unknown attack association feature extraction is achieved by mining implicit associations between features through an attention mechanism.

[0028] The preferred, interpretable threat pattern and rule association graph construction process is as follows:

[0029] C1. Define graph nodes and edges based on the extracted features. Graph nodes include feature nodes, threat pattern nodes, detection rule nodes, and business scenario nodes.

[0030] C2. Import the feature extraction results, threat patterns, detection rules and business scenario data into the graph database, and use the association rule mining algorithm to calculate the association strength between nodes. The association strength includes support and confidence.

[0031] C3. Use graph visualization tools to display the map and mark high-risk associations;

[0032] C4. Generate natural language interpretation documents for key associations in the graph.

[0033] The preferred procedure for assessing the reasonableness of threat coverage is as follows:

[0034] D1. Input the known threat data and unknown attack data from the high-quality dataset into the intrusion detection scheme to be evaluated;

[0035] D2. Calculate the detection rate of known threats and the detection rate of unknown attacks;

[0036] D3. When the detection rate of known threats is greater than 95% and the detection rate of unknown attacks is greater than 60%, the coverage meets the standard; otherwise, the missing threat type is identified.

[0037] The process for assessing the reasonableness of false negative rate predictions is as follows:

[0038] D4. Input normal business data and abnormal attack data from the high-quality dataset into the prediction model, and statistically analyze the number of correctly detected attacks, correctly excluded normal attacks, false positives, and false negatives predicted by the model.

[0039] D5. Calculate the false alarm rate and the false negative rate respectively. If the false alarm rate is less than 10% and the false negative rate is less than 8%, the accuracy meets the standard; otherwise, locate the problem according to the rules.

[0040] The process for assessing the suitability of business scenarios is as follows:

[0041] D6. Filter high-quality datasets to concentrate business baseline data;

[0042] D7. Check for conflicts and redundancies between each detection rule and the business baseline;

[0043] D8. Calculate the fit. If the fit is greater than 85%, the fit meets the standard; otherwise, output rule adjustment suggestions.

[0044] The preferred process for automatic optimization of the prediction model is as follows:

[0045] E1. When new attack data appears, the elastic weight consolidation algorithm is used for incremental learning;

[0046] E2. By training the prediction model through a deep Q-network, the prediction model can maximize the cumulative reward during classification, thereby reducing the false negative rate.

[0047] Preferably, when automatically optimizing the detection rules, Bayesian optimization is used to find the optimal threshold for rules with high false positive rates; and the Apriori algorithm is used to mine the features that need to be supplemented for rules with high false negative rates.

[0048] The rationality judgment method for intrusion detection schemes based on large models proposed in this invention has the following advantages compared with existing technologies:

[0049] 1. This invention upgrades static rule verification into a dynamic intelligent evaluation system by judging the rationality of intrusion detection schemes through a large-scale model. Through comprehensive coverage of the data layer, domain adaptation of the model layer, multi-dimensional quantification of the evaluation layer, and closed-loop feedback of the optimization layer, it can not only more accurately judge the rationality of the scheme, but also proactively respond to unknown threats, simplify the rule optimization process, and control the risk of false positives and false negatives, thus providing more adaptive protection for the network security system.

[0050] 2. This invention solves the black box problem of models by extracting explicit and implicit features from the trained model, and associates features, threat patterns, detection rules, and business scenarios through a graph database to generate a visual map. The association strength is calculated by the Apriori algorithm, and finally a natural language explanation document is output, making the relationship between threat, rule, and business intuitively understandable.

[0051] 3. This invention quantifies the rationality of an intrusion detection solution from three dimensions: threat coverage, false negative rate, and business adaptability, rather than relying on subjective judgment. This avoids the one-sidedness of traditional assessments that only consider threat coverage. The addition of false negative rate and business adaptability ensures that the solution can detect threats, has fewer false positives, and does not affect business operations. Attached Figure Description

[0052] Figure 1 A flowchart of the rationality judgment process according to an embodiment of the present invention is shown;

[0053] Figure 2 A flowchart illustrating the training and parameter optimization process of a secure pre-trained large model according to an embodiment of the present invention is shown.

[0054] Figure 3 A flowchart illustrating the training and parameter optimization process of a general-purpose model according to an embodiment of the present invention is shown.

[0055] Figure 4 A flowchart illustrating the construction process of an interpretable threat pattern and rule association graph according to an embodiment of the present invention is shown. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The specific embodiments described herein are merely used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] This invention provides, for example Figure 1-4 The rationality judgment method for the intrusion detection scheme based on a large model, as shown, includes the following steps:

[0058] S1. Collect multi-dimensional data and clean, label, and convert the collected multi-dimensional data to form a high-quality dataset. The multi-dimensional data includes known threat data, unknown attack simulation data, historical false negative data, and business baseline data.

[0059] Known threat data is collected through public threat databases, commercial threat intelligence, and internal accumulation. Public threat databases include the MITRE Attack Tactics / Techniques and Common Sense Framework, the General Vulnerability Disclosure Database, and the Intrusion Indicators Database. The MITRE Attack Tactics / Techniques and Common Sense Framework is used to obtain attack tactics and techniques, the General Vulnerability Disclosure Database is used to obtain vulnerability details, and the Intrusion Indicators Database is used to obtain malicious IPs, hashes, and domain names. Commercial threat intelligence includes QiAnXin Threat Intelligence Center and 360 Threat Intelligence Platform. Internal accumulation is used to obtain historical blocked attack logs, such as malicious traffic recorded by IDS / EDR.

[0060] Unknown attack simulation data is generated by using fuzz testing tools to produce malformed network packets or simulate abnormal process links for batch data transmission during non-working hours.

[0061] Historical false alarm and missed alarm data are based on false alarm and missed alarm events marked in the logs of existing security devices, including Security Information and Event Management (SIEM) and Endpoint Detection and Response (EDR).

[0062] Business baseline data is collected from business system logs, including web server access logs, database operation logs, and OA system login logs. The NetFlow collection tool is used to record port access frequency, bandwidth usage, and IP communication range during normal periods.

[0063] When cleaning multidimensional data, a hash deduplication algorithm is used to remove duplicate logs. Then, regular expressions are used to match the log structure and filter out non-matching logs. For numerical data, the DBSCAN clustering algorithm is used to identify and remove outliers. Finally, logs missing key fields are completed by correlating with other logs. The hash deduplication algorithm generates a unique hash value for each log and deletes entries with duplicate hash values. The hash deduplication algorithm formula is: h = SHA256(log_content), where h is the log hash value and log_content is the original log text.

[0064] When annotating the cleaned multi-dimensional data, a combination of manual and automatic annotation is used. In manual annotation, the core information to be annotated is first identified. The core information includes behavior type, attack association, and false positive / false negative flags. When annotating behavior type, normal behavior is marked as O, known attack is marked as A_known, and unknown attack is marked as A_unknown. When annotating attack association, known attack is marked as ATT&CK or IOC. When annotating false positives / false negatives, historical false positives are marked as FP, false negatives as FN, correct errors as TP, and correct exclusions as TN.

[0065] During automatic labeling, use rule matching tools such as LabelStudio to batch label known attacks. Then, manually verify the automatic labeling results and correct labeling errors, such as changing normal business SQL queries from SQL injection to normal behavior.

[0066] During format conversion, unstructured text logs are transformed into vectors recognizable by the prediction model. This process includes text segmentation and encoding, and log vector concatenation. Text segmentation and encoding involves splitting the log text into keywords using a security-domain word segmentation tool, and then converting the keywords into word vectors using the conversion model. The formula for the conversion model is as follows:

[0067] ,

[0068] in, For target words, For the set of context words, This represents the probability value of the target word occurring in the context word set. target words The corresponding word vectors, for The transpose operation, Let V be the mean vector of the context word vectors, and V be the vocabulary. Let w be the word vector corresponding to any word w in vocabulary V;

[0069] When concatenating log vectors, all word vectors of a single log entry are weighted according to the priority of key fields and concatenated to generate the final log vector. When concatenating by weight, the field vector of a single field is calculated first, then the field vectors are weighted according to their priority, and finally all weighted field vectors are concatenated to generate the final log vector.

[0070] For example, suppose a single log entry contains m key fields, such as behavior type, source IP, destination port, and timestamp, and word vectors for all keywords under each field have been obtained through a transformation model. For the i-th key field... For example, behavior types, if they include Each keyword has a corresponding word vector. , , ..., each Then the field vector of that field The average of all word vectors within a field; weights are assigned to the i-th field based on field priority. Weight of high-priority fields Larger and meets To ensure weight normalization, the m weighted field vectors are concatenated in descending order of field priority to obtain the final log vector X, i.e., X = ,in, These are the weighted field vectors for the 1st, 2nd, ..., mth fields, respectively. X is the transpose of the horizontal concatenation of vectors, where X is the final log vector;

[0071] S2. Based on high-quality datasets, select a safe pre-trained large model or a general model for training and parameter optimization to obtain a prediction model;

[0072] The secure pre-trained large model performs the following training and parameter optimization process:

[0073] A1. Load the pre-training weights officially released by the security pre-training large model, such as the 360 ​​Security Large Model and the Qi An Xin Tian Qing Large Model. The pre-training weights released by the official security pre-training large model contain security terminology and attack pattern knowledge.

[0074] A2. Select core data for security scenarios from high-quality datasets and divide them into training and validation sets in an 8:2 ratio. Core data for security scenarios include attack logs and rule documents.

[0075] A3. Based on the training set data, the low-rank adaptation algorithm is used to insert low-rank matrices into the key layers of the safe pre-trained large model and train the inserted matrices instead of training all parameters of the safe pre-trained large model, which greatly reduces the computational cost.

[0076] For example, the original weight matrix of the secure pre-trained model is , To optimize the hidden layer dimension of the model, the low-rank adaptation algorithm inserts a low-rank matrix. and , where r is a hyperparameter of the low-rank adaptation algorithm, and during fine-tuning, the actual weights W of the safe pre-trained model are:

[0077] W 实 =W0+B*A, during training, W0 is fixed and not updated, only the parameters of A and B are updated, and the result of B*A is merged with W0, without the need for additional calculation;

[0078] A4. Traverse the key hyperparameter combinations of the low-rank adaptation algorithm through grid search, test the attack identification accuracy on the validation set, and select the parameter combination with the highest accuracy as the final model parameters.

[0079] The general-purpose model performs the following training and parameter optimization process:

[0080] B1. Select known threat data and security document data from high-quality datasets to perform incremental pre-training on the general model, injecting security knowledge;

[0081] B2. Enhance security domain features on high-quality datasets, such as associating logs with ATT&CK to generate text pairs of logs and tactical descriptions;

[0082] If the amount of data is sufficient, a full fine-tuning is performed; if the amount of data is small, a prefix fine-tuning is performed. For example, if the amount of data is greater than 100,000, it is considered sufficient; if the amount of data is less than 10,000, it is considered small.

[0083] B3. For cases with few unknown attack samples, a weighted cross-entropy loss is used to increase the loss weight of unknown attack samples. The cross-entropy loss formula is as follows:

[0084] ,

[0085] in, This represents the cross-entropy loss value. Where is the total number of samples in a single training run, and C is the total number of intrusion detection categories. The weight value for category c. Let i be the true label of the i-th sample. Predict probabilities for the model;

[0086] Predictive models that have undergone prediction and parameter optimization using large-scale or general-purpose pre-trained models all possess automatic feature extraction capabilities, threat pattern learning capabilities, and scenario adaptation capabilities. Specifically, the automatic feature extraction capability means that it can automatically extract threat features from logs without the need for manual rule definition. For example, if a port fails 5 times in a row, it can automatically extract brute-force attacks.

[0087] Threat pattern learning capability is specifically manifested in: the ability to correlate multi-dimensional data to identify attack chains; scenario adaptation capability is manifested in: the ability to integrate business baseline data and distinguish between attack behavior and normal business fluctuations.

[0088] B4. Optimize the parameters of the general model based on the calculated loss value until it meets the prediction requirements;

[0089] Based on business resources (data volume, computing cost), a security pre-trained model or a general model is selected. Intrusion detection scenario knowledge is injected through fine-tuning (LoRA / PrefixTuning) to train a predictive model with attack classification and rule rationality reasoning capabilities. Key parameters (LoRA rank, learning rate, batch size) are adjusted through grid search. Combined with cross-entropy / weighted cross-entropy loss functions, the model bias problem caused by the small number of unknown attack samples is solved to ensure that the model performance meets the standards.

[0090] S3. Extract explicit known threat features and implicit unknown attack association features through prediction models, and construct an interpretable threat pattern and rule association map;

[0091] The extraction of explicit known threat features is based on the prediction model's built-in IOC library and ATT&CK knowledge. It directly matches known features from the log vector. For example, if the log vector contains malicious hash = A1B2C3, it is extracted as explicit feature 1 and associated with malicious file IOC; if the log behavior matches T1059.001 and the command line is executed, it is extracted as explicit feature 2 and associated with ATT&CK technology.

[0092] Implicit unknown attack correlation feature extraction utilizes an attention mechanism to uncover implicit correlations between features. The formula for the attention mechanism is:

[0093] ,

[0094] in, For the current log characteristics, For historical attack signature database, For feature association weights, The dimension is K;

[0095] For example, when the prediction model finds that the feature combination of A-start, no file landing and connection to overseas IP is greater than 0.8 with the feature combination of historical unknown attacks, it is extracted as a latent feature. The latent feature is the combination of no file and overseas communication.

[0096] The process for constructing an interpretable threat pattern and rule association graph is as follows:

[0097] C1. Define graph nodes and edges based on the extracted features. Graph nodes include feature nodes, threat pattern nodes, detection rule nodes, and business scenario nodes.

[0098] C2. Import the feature extraction results, threat patterns, detection rules, and business scenario data into the graph database, and use the association rule mining algorithm to calculate the association strength between nodes. The association strength includes support and confidence.

[0099] Support: Where M is the total number of nodes, and Y and Z are two nodes out of the M nodes. This represents the probability that nodes Y and Z appear simultaneously.

[0100] Confidence level: ,in, Let Z be the probability of Z occurring when Y occurs.

[0101] C3. Use graph visualization tools to display the map and mark high-risk associations;

[0102] C4. Generate natural language explanation documents for key associations in the graph;

[0103] If detection rule R3 does not cover threat T2, R3 is based on malicious file hash detection, while T2 is fileless ransomware. The reason is that T2 has no files written to the disk, and R3's file hash cannot be matched. It is recommended to add memory behavior detection features to R3, such as abnormal process memory injection.

[0104] This approach extracts explicit features (IOC / ATT&CK techniques for known threats) and implicit features (behavioral associations of unknown attacks, such as login failure → batch download) from the trained model to address the model black box problem. It then links features, threat patterns, detection rules, and business scenarios through a graph database (such as Neo4j) to generate a visual graph. The association strength (support / confidence) is calculated using the Apriori algorithm, and finally, a natural language explanation document is output, making the relationship between threats, rules, and business intuitively understandable.

[0105] S4. Conduct a reasonableness assessment from multiple dimensions, including threat coverage, false negative rate prediction, and business scenario adaptability. If the threat coverage, false negative rate prediction, and business scenario adaptability assessment all meet the standards, the assessment is reasonable; otherwise, the assessment is unreasonable.

[0106] The process for assessing the reasonableness of threat coverage is as follows:

[0107] D1. Input the known threat data and unknown attack data from the high-quality dataset into the intrusion detection scheme to be evaluated;

[0108] D2. Calculate the detection rate of known threats and the detection rate of unknown attacks;

[0109] The known formula for calculating the threat detection rate is:

[0110] ,

[0111] in, Given the known threat detection rate, The number of known attacks that were correctly detected. This represents the number of known attacks that were missed.

[0112] The formula for calculating the detection rate of unknown attacks is:

[0113] ,

[0114] in, For the detection rate of unknown attacks The number of unknown attacks that were correctly detected. This represents the number of unknown attacks that were missed.

[0115] D3. When the detection rate of known threats is greater than 95% and the detection rate of unknown attacks is greater than 60%, the coverage meets the standard; otherwise, locate the missing threat type, such as the LESS attack that does not cover the file.

[0116] The process for assessing the reasonableness of false negative rate predictions is as follows:

[0117] D4. Input normal business data and abnormal attack data from the high-quality dataset into the prediction model, and statistically analyze the number of correctly detected attacks (TP1), the number of correctly excluded normal attacks (TN1), the number of false alarms (FP1), and the number of missed attacks (FN1) predicted by the model.

[0118] D5. Calculate the false alarm rate and the false negative rate respectively. If the false alarm rate is less than 10% and the false negative rate is less than 8%, the accuracy meets the standard; otherwise, locate the problem according to the rules.

[0119] The formula for calculating the false alarm rate is: , R represents the probability that normal data is mistakenly identified as an attack.

[0120] The formula for calculating the false negative rate is: , R represents the probability that attack data is misjudged as normal.

[0121] The process for assessing the suitability of business scenarios is as follows:

[0122] D6. Filter high-quality datasets to concentrate business baseline data;

[0123] D7. Check for conflicts and redundancies between each detection rule and the business baseline. Conflicting rules include: rule R4, which determines that QPS > 3000 = DDoS, conflicts with the peak sales season baseline (5000 QPS); redundant rules include: rule R5, which detects Linux rootkits, but the enterprise does not have a Linux server.

[0124] D8. Calculate the fit. If the fit is greater than 85%, the fit meets the standard; otherwise, output rule adjustment suggestions.

[0125] The rationality of an intrusion detection solution is quantitatively evaluated from three dimensions: threat coverage, false negative rate, and business adaptability, rather than relying on subjective judgment. This avoids the one-sidedness of traditional evaluations that only consider threat coverage. The addition of false negative rate (accuracy) and business adaptability (practicability) ensures that the solution can detect threats, minimizes false alarms, and does not affect business operations.

[0126] S5. Collect operational feedback data, automatically optimize the prediction model and detection rules, and repeatedly evaluate until the preset standards are met;

[0127] The process of automatic optimization of the prediction model is as follows:

[0128] E1. When new attack data emerges, such as new types of DDoS attacks, the Elastic Weight Consolidation (EWC) algorithm is used for incremental learning to avoid catastrophic forgetting and forgetting old knowledge.

[0129] The formula for the Elastic Weight Consolidation (EWC) algorithm is as follows:

[0130] ,

[0131] in, For cross-entropy loss, This is a weighting parameter, with a value ranging from 1 to 10. For parameters Importance weight, These are the parameter values ​​for the old prediction model;

[0132] E2. The prediction model is trained by Deep Q-Network (DQN) to maximize the cumulative reward during classification, thereby reducing the false negative rate.

[0133] Incremental learning can quickly absorb new threat data, solving the problem that models cannot cope with new types of attacks; reinforcement learning can adjust classification strategies in real time to reduce false positives and false negatives.

[0134] When automatically optimizing detection rules, Bayesian optimization is used to find the optimal threshold for rules with high false positive rates. For example, three failed login attempts on port 22 are falsely reported as brute-force attacks. For rules with high false negative rates, the Apriori algorithm is used to mine features that need to be added. For example, LESS attacks on uncovered files are false negatives. Bayesian optimization does not require traversing all thresholds, and the Apriori algorithm automatically mines high-confidence features, which is more than 10 times more efficient than manual optimization and reduces the workload of the security team.

[0135] By assessing the rationality of intrusion detection schemes using large-scale models, static rule verification is upgraded to a dynamic intelligent evaluation system. Through comprehensive coverage of the data layer, domain adaptation of the model layer, multi-dimensional quantification of the evaluation layer, and closed-loop feedback of the optimization layer, the system can not only more accurately determine the rationality of the scheme, but also proactively respond to unknown threats, simplify the rule optimization process, and control the risks of false positives and false negatives, thus providing more adaptive protection for the network security system.

[0136] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for judging the rationality of intrusion detection schemes based on large models, characterized in that: Includes the following steps: S1. Collect multi-dimensional data and clean, label, and convert the collected multi-dimensional data to form a high-quality dataset. The multi-dimensional data includes known threat data, unknown attack simulation data, historical false negative data, and business baseline data. S2. Based on high-quality datasets, select a safe pre-trained large model or a general model for training and parameter optimization to obtain a prediction model; S3. Extract explicit known threat features and implicit unknown attack association features through prediction models, and construct an interpretable threat pattern and rule association map; S4. Conduct a reasonableness assessment from multiple dimensions, including threat coverage, false negative rate prediction, and business scenario adaptability. If the threat coverage, false negative rate prediction, and business scenario adaptability assessment all meet the standards, the assessment is reasonable; otherwise, the assessment is unreasonable. The process for assessing the reasonableness of threat coverage is as follows: D1. Input the known threat data and unknown attack data from the high-quality dataset into the intrusion detection scheme to be evaluated; D2. Calculate the detection rate of known threats and the detection rate of unknown attacks; D3. When the detection rate of known threats is greater than 95% and the detection rate of unknown attacks is greater than 60%, the coverage meets the standard; otherwise, the missing threat type is identified. The process for assessing the reasonableness of false negative rate predictions is as follows: D4. Input normal business data and abnormal attack data from the high-quality dataset into the prediction model, and statistically analyze the number of correctly detected attacks, correctly excluded normal attacks, false positives, and false negatives predicted by the model. D5. Calculate the false alarm rate and the false negative rate respectively. If the false alarm rate is less than 10% and the false negative rate is less than 8%, the accuracy meets the standard; otherwise, locate the problem according to the rules. The process for assessing the suitability of business scenarios is as follows: D6. Filter high-quality datasets to concentrate business baseline data; D7. Check for conflicts and redundancies between each detection rule and the business baseline; D8. Calculate the fit. If the fit is greater than 85%, the fit meets the standard; otherwise, output rule adjustment suggestions. S5. Collect operational feedback data, automatically optimize the prediction model and detection rules, and repeat the evaluation until the preset standards are met.

2. The method for judging the rationality of an intrusion detection scheme based on a large model according to claim 1, characterized in that: When cleaning multidimensional data, a hash deduplication algorithm is used to delete duplicate logs, then regular expressions are used to match the log structure and filter out non-matching logs. For numerical data, the DBSCAN clustering algorithm is used to identify outliers and remove them. Finally, logs with missing key fields are completed by correlating with other logs. When annotating the cleaned multi-dimensional data, a combination of manual and automatic annotation is used. During format conversion, unstructured text logs are converted into vectors that can be recognized by the prediction model. This includes text segmentation and encoding, and log vector concatenation. Text segmentation and encoding involves splitting keywords from the log text using a security domain segmentation tool, and then converting the keywords into word vectors through a conversion model.

3. The method for judging the rationality of an intrusion detection scheme based on a large model according to claim 2, characterized in that: The secure pre-trained large model performs the following training and parameter optimization process: A1. Load the pre-trained weights officially released by the secure pre-trained large model; A2. Select core data for security scenarios from high-quality datasets and divide them into training and validation sets in an 8:2 ratio; A3. Based on the training set data, a low-rank adaptation algorithm is used to insert low-rank matrices into the key layers of a safe pre-trained large model and train the inserted matrices. A4. Traverse the key hyperparameter combinations of the low-rank adaptation algorithm through grid search, test the attack identification accuracy on the validation set, and select the parameter combination with the highest accuracy as the final model parameters.

4. The method for judging the rationality of an intrusion detection scheme based on a large model according to claim 3, characterized in that: The general-purpose model performs the following training and parameter optimization process: B1. Select known threat data and security document data from high-quality datasets to perform incremental pre-training on the general model, injecting security knowledge; B2. Enhance security domain features on high-quality datasets to generate text pairs of logs and tactical descriptions; If the amount of data is sufficient, perform full fine-tuning; if the amount of data is small, perform prefix fine-tuning. B3. To address the issue of a limited number of unknown attack samples, a weighted cross-entropy loss method is used to increase the loss weight of unknown attack samples. B4. Optimize the parameters of the general model based on the calculated loss value until it meets the prediction requirements.

5. The method for judging the rationality of an intrusion detection scheme based on a large model according to claim 4, characterized in that: Predictive models that have undergone prediction and parameter optimization using large-scale or general-purpose pre-trained security models all possess the ability to automatically extract features, learn threat patterns, and adapt to different scenarios.

6. The method for judging the rationality of an intrusion detection scheme based on a large model according to claim 5, characterized in that: Explicit known threat feature extraction is based on the IOC library and ATT&CK knowledge built into the prediction model, directly matching known features from log vectors. Implicit unknown attack association feature extraction is achieved by mining implicit associations between features through an attention mechanism.

7. The method for judging the rationality of an intrusion detection scheme based on a large model according to claim 6, characterized in that: The process for constructing an interpretable threat pattern and rule association graph is as follows: C1. Define graph nodes and edges based on the extracted features. Graph nodes include feature nodes, threat pattern nodes, detection rule nodes, and business scenario nodes. C2. Import the feature extraction results, threat patterns, detection rules and business scenario data into the graph database, and use the association rule mining algorithm to calculate the association strength between nodes. The association strength includes support and confidence. C3. Use graph visualization tools to display the map and mark high-risk associations; C4. Generate natural language interpretation documents for key associations in the graph.

8. The method for judging the rationality of an intrusion detection scheme based on a large model according to claim 7, characterized in that: The process of automatic optimization of the prediction model is as follows: E1. When new attack data appears, the elastic weight consolidation algorithm is used for incremental learning; E2. By training the prediction model through a deep Q-network, the prediction model can maximize the cumulative reward during classification, thereby reducing the false negative rate.

9. The method for judging the rationality of an intrusion detection scheme based on a large model according to claim 8, characterized in that: When automatically optimizing detection rules, Bayesian optimization is used to find the optimal threshold for rules with high false positive rates; and Apriori algorithm is used to mine the features that need to be supplemented for rules with high false negative rates.

Images