A rule and large model-based security alarm noise reduction method

Abstract

The application relates to a rule and large model-based security alarm noise reduction method, belonging to the network security field, which comprises the following steps: collecting security alarm data and security logs and performing pretreatment; constructing a dynamic comprehensive rule knowledge base containing a screening rule base and an external knowledge base, and periodically updating the dynamic comprehensive rule knowledge base; selecting an open source large model in the security field, using a security field open source data set and historical security alarm data, and fine-tuning a noise reduction small model by using a distillation technology; adopting a double-path coordination mechanism of periodic fine-tuning driven by the knowledge base and dynamic knowledge injection to improve the capability of the noise reduction small model; and finally, realizing efficient and accurate alarm noise reduction based on the screening rule base and the noise reduction small model. The application effectively improves the efficiency and accuracy of security alarm noise reduction, and provides an efficient, accurate and intelligent alarm noise reduction method for the network security field.

Description

TECHNICAL FIELD

[0001] The present application relates to a rule and large model based security alarm noise reduction method, belonging to the technical field of network security. BACKGROUND

[0002] With the rapid development of information technology, network security threats are increasingly complex and diverse. Network attack means is constantly evolving, from simple brute force, port scanning to complex advanced persistent threats, zero-day vulnerability attacks, etc., bringing great challenges to network security protection.

[0003] In order to cope with these threats, enterprises, institutions and organizations usually deploy a variety of security devices, such as firewalls, intrusion detection systems, intrusion prevention systems, security information and event management systems, etc. These devices can monitor network traffic, system logs and other information in real time, and generate a large number of security alarms. These alarms have the characteristics of large data volume, high noise data, high complexity, strong dynamics, etc.

[0004] In the field of security alarm processing today, common methods include rule-based alarm processing, statistical analysis-based alarm processing, manual analysis processing, machine learning-based alarm processing, and large model-based alarm processing methods. The rule-based method is widely used because of its clear rules, easy to understand and implement, but this method has many limitations, such as high rule maintenance cost, inability to cope with complex scenarios, and high false alarm rate. The statistical analysis-based method can identify abnormal behavior hidden in normal data, but this method relies heavily on historical data and needs a large amount of historical data as a reference, which may not be able to detect new threats in time. Manual analysis processing is excellent in handling complex and ambiguous situations and identifying unknown threats, but when faced with massive alarm data, its efficiency is extremely low, and security personnel have to spend a lot of time on low-value alarms, which is easy to miss important information due to fatigue. The machine learning-based method can automatically learn patterns and features in data and has certain processing capability for complex data, but it needs a large amount of high-quality labeled data for training, which not only consumes time and effort, but also the model may not be able to effectively identify new attack patterns and unknown threats. In addition, large models can utilize large-scale pre-training data to perform well in various tasks and can be adapted to specific security alarm tasks through fine-tuning, etc. However, this method also has some limitations. On the one hand, the training and deployment of large models require a large amount of computing resources, resulting in high cost. On the other hand, large models may face efficiency problems when processing massive alarm data, especially in scenarios with high real-time requirements.

[0005] Therefore, in view of the deficiencies of the prior art, the present application proposes a rule and large model based security alarm noise reduction method. SUMMARY

[0006] The technical problem solved by the present application is that the present application provides a security alarm noise reduction method based on rules and large models, which effectively addresses the challenges of excessive alarm data volume, high false positive and false negative rates, low efficiency of manual analysis, and low adaptability and mass data processing efficiency of models. The present application provides an efficient, accurate and intelligent alarm noise reduction method for the field of network security.

[0007] The technical solution of the present application is a security alarm noise reduction method based on rules and large models, which comprises:

[0008] S1. Collection and preprocessing of security alarm data: collect security alarm data and security logs from various security devices and perform preprocessing;

[0009] S2. Building a dynamic comprehensive rule knowledge base: building a dynamic comprehensive rule knowledge base containing a filtering rule base and an external knowledge base, and periodically updating the dynamic comprehensive rule knowledge base;

[0010] S3. Building a noise reduction small model based on a security domain open source large model: selecting a security domain open source large model, using security domain open source datasets and historical security alarm data, and fine-tuning a noise reduction small model using distillation technology;

[0011] S4. Continuously enhancing and driving the ability of the noise reduction small model through the external knowledge base: adopting a dual-path coordination mechanism of periodic fine-tuning and dynamic knowledge injection driven by the knowledge base to improve the ability of the noise reduction small model;

[0012] S5. Full-process automatic alarm noise reduction: after collecting and preprocessing the security alarm data, parallel or serially enter the dynamic comprehensive rule knowledge base built in S2 and the enhanced noise reduction small model in S4 to perform multi-dimensional analysis and cross-validation on the security alarm data.

[0013] Through the above steps, the present application realizes full-process optimization from data collection, preprocessing, preliminary noise reduction, deep noise reduction to accurate analysis, effectively improving the efficiency and accuracy of security alarm processing.

[0014] Further, the S1 comprises:

[0015] S11. Connect various security devices to the data collection system and collect security alarm data and security logs generated by the devices in real time; configure data acquisition parameters for each security device to build a comprehensive and continuous data stream; the data acquisition parameters include data format, acquisition frequency and data transmission protocol;

[0016] S12. The data collection system receives raw security alarm data from different security devices, and performs preliminary analysis and multi-level preprocessing; the preprocessing includes data cleaning, format conversion and data standardization;

[0017] The data cleaning includes:

[0018] Repeated alarm records are identified and deleted using a hash algorithm to avoid redundant information;

[0019] The linear interpolation method is used to scientifically fill in the missing key field values;

[0020] Data format errors are checked and corrected to ensure that the core information conforms to the standard format definition; the core information includes IP address, timestamp, port number;

[0021] The format conversion includes:

[0022] A field mapping table is established to unify the field names of different security devices;

[0023] The timestamp is converted to a unified format, and the numerical data is also converted to a unified unit;

[0024] The data standardization includes:

[0025] The text format security alarm data is standardized and preprocessed, including: unifying character encoding to UTF-8, standardizing the newline character to the expression of backslash plus lowercase n, and standardizing the indentation style;

[0026] Regular expression technology is used to extract key information from the text and store it as structured fields;

[0027] S13. A distributed database is selected as the storage system for security alarm data, and an index is established for the stored security alarm data to support fast retrieval and query based on key fields; the key fields include timestamp, event type, and source address; at the same time, the stored security alarm data is backed up regularly, and a data recovery strategy is developed.

[0028] Further, the S2 includes the following steps:

[0029] S21. A filtering rule library is constructed, which is clearly divided into multiple types and determines the standardized representation and storage method, for fast matching and preliminary noise reduction; the filtering rule library includes blacklist and whitelist rules, feature matching rules, statistical analysis rules, semantic analysis rules, and intelligence-driven rules;

[0030] S22. An external knowledge base is constructed, which integrates security field professional knowledge, threat intelligence and attack patterns, disposal process and emergency response methods, and personalized environment information in multiple dimensions;

[0031] S23. Integrate knowledge base and establish dynamic updating mechanism.

[0032] Further, the S21 comprises:

[0033] The white list rule is used to identify known safe network entities, and the alarms generated by these entities are directly ignored; the black list rule is used to identify known malicious network entities, and the alarms generated by these entities are paid attention to and retained;

[0034] Specifically, the white list rule is represented as a set , wherein represents a network entity in the white list; the black list rule is represented as a set , wherein represents a network entity in the black list; for a network entity in the given alarm data , if , the alarm is marked as an invalid alarm; if , the alarm is marked as a valid alarm, is paid attention to and retained.

[0035] The feature matching rule is used to quickly identify alarms matching known attack patterns; the feature matching rule is based on known attack features or patterns, including malware signatures, abnormal network traffic features, and known exploit behaviors;

[0036] The feature matching rule is represented as a regular expression or a string pattern , wherein represents an attack feature pattern; for given alarm information , if matches any one of the patterns , the alarm is considered to match a known attack pattern and is marked as a valid alarm and retained;

[0037] The statistical analysis rule is used to identify abnormal behaviors based on time, frequency or other statistical features;

[0038] A frequency threshold of alarms triggered by an IP address in a short time is set, and alarms exceeding the threshold are considered as abnormal behaviors; a time window and a frequency threshold are set, for an IP address , the number of alarms triggered in the time window is , if , the behavior of the IP address is considered as an abnormal behavior, is marked as a valid alarm and retained; if If the alarm is not directly considered as a valid alarm, it is regarded as a low-frequency candidate alarm and is handed over to a subsequent alarm judgment module for further research and judgment.

[0039] The semantic analysis rule is to analyze the alarm text content by keyword and natural language processing technology, and identify the semantic pattern related to the known threat;

[0040] The intelligence-driven rule is to quickly identify alarms related to known threats by using the compromise indicators IOCs in the latest threat intelligence TI.

[0041] Further, the S22 comprises:

[0042] The security field professional knowledge covers the knowledge system refined from authoritative research literature, industry standards and expert experience, including attack techniques and vulnerability information in the MITRE ATT&CK framework;

[0043] The threat intelligence and attack pattern refers to the continuous collection and update of typical attack patterns and threat intelligence from public vulnerability databases, global threat intelligence platforms and security communities, including the life cycle steps of various attacks, commonly used technical tools of attackers and the latest compromise indicators IOCs;

[0044] The disposal process and emergency response method refers to the systematic arrangement and archiving of standard disposal processes and emergency response methods for various security events, including emergency response steps, vulnerability repair measures and traceability guidelines;

[0045] The personalized environment information collects and integrates the enterprise's internal network topology, asset list, business system correlation and labeled historical alarm data, including real and valid alarms and invalid alarms.

[0046] Further, the S23 comprises:

[0047] The rules collected in the screening rule library, the security field professional knowledge in the external plug-in knowledge base, the threat intelligence and attack pattern, the disposal process and emergency response method, and the personalized environment information are integrated into a unified knowledge base to construct a structured knowledge system. At the same time, an update strategy is developed to regularly obtain new knowledge and patterns from security research, actual attack event analysis and industry intelligence sharing platform to ensure the timeliness and integrity of the knowledge base. An audit process is set up to ensure the accuracy and reliability of the updated content. The effectiveness of the dynamic comprehensive rule knowledge base is verified by simulation testing and actual application, and the verification results are used for adjustment and optimization.

[0048] Further, the S3 further comprises:

[0049] S31. Determine the teacher model and prepare the data set for distillation training;

[0050] A large open-source model pre-trained in the security field was selected as the teacher model. Meanwhile, prepare the dataset. ,in, This represents the i-th security alarm data sample after preprocessing in step S1. This represents the true category label corresponding to the i-th security alarm data sample. This indicates the total number of samples in the dataset; the dataset includes open-source datasets in the security field and historical security alert data from within the enterprise that has been preprocessed and labeled using the S1 step.

[0051] S32. In the teacher model and dataset Once everything is ready, execute the knowledge distillation training process to train the student model. Teacher Model Process each security alarm data sample It then outputs its prediction for that sample. , By applying temperature parameters to the output layer of the teacher model The Softmax function yields:

[0052]

[0053] in, Teacher model For security alarm data samples Category The original output without Softmax normalization, where j is the index variable in the summation operation, used to iterate through all possible output categories; it is the category index. Teacher model Sample of input security alarm data The raw output belonging to the j-th category without Softmax normalization; this summation operation is performed on all possible categories to compute the denominator, i.e., the normalization factor, to ensure that the Softmax output is an effective probability distribution, where j indicates the category corresponding to each neuron in the teacher model output layer in order to compute the denominator of the Softmax function;

[0054] Student Model It then processes the same security alarm data samples through its own network structure. And attempt to mimic the output of soft targets from the teacher model. :

[0055]

[0056] in, student model for safety alert data samples belongs to the class unnormalized raw output;

[0057] loss function of knowledge distillation is a weighted combination of two parts of loss:

[0058]

[0059] where, is a weight parameter, used to balance the importance of soft target loss and hard label loss ;

[0060] is the KL divergence loss between student model prediction and teacher model soft target, which encourages the student model to imitate the judgment logic of the teacher model:

[0061]

[0062] is the cross-entropy loss between student model prediction and original hard label, which is used to ensure that the student model learns the correct classification:

[0063]

[0064] where, is the natural logarithm, is an indicator function, which is equal to 1 when the true class of safety alert data samples is equal to the class ; otherwise, it is 0;

[0065] S33. Further fine-tuning and performance evaluation of the distilled denoising small model, i.e. the student model , using safety alert data and the latest representative samples extracted from the external knowledge base to enhance the performance of the small model in the safety alert denoising task and its generalization ability to new threats. After training, the denoising small model is subjected to multi-dimensional performance evaluation.

[0066] The external knowledge base in the application continuously enhances the small model by periodically extracting the latest, high-quality effective alarm samples, false alarm samples and emerging threat intelligence from the external knowledge base. These samples are artificially confirmed or automatically verified, and have high representativeness and high value. The distilled small model is retrained in a lightweight and incremental manner using the extracted new samples, only part of the layers or parameters of the model are updated, a small learning rate is used, and the computational resource consumption and training time are significantly reduced. The internal parameters of the small model are continuously optimized, and strict model version management and rollback mechanism are implemented to ensure that the new model version is verified by offline testing after each incremental fine-tuning, and can be rolled back to the old version when necessary.

[0067] Further, the S4 comprises:

[0068] S41. Periodically fine-tune the distilled noise reduction small model in a lightweight and incremental manner using the latest data from the external knowledge base;

[0069] Periodically extract the latest effective alarm samples, false alarm samples and emerging threat intelligence from the external knowledge base;

[0070] The distilled noise reduction small model is retrained in a lightweight and incremental manner using the extracted samples, only part of the layers or parameters of the model are updated, a small learning rate is used, and the computational resource consumption and training time are significantly reduced;

[0071] When the security alarm data is input, the associated knowledge in the external knowledge base is dynamically retrieved and fused with the security alarm data to input the distilled noise reduction small model, which is used to enhance the semantic analysis capability of the model; through artificial feedback and system automatic learning, the external knowledge base is continuously updated and optimized, forming a closed-loop knowledge enhancement and model iteration mechanism;

[0072] The lightweight fine-tuning refers to the parameters of the current model The new sample set extracted from the external knowledge base is , wherein is the new security alarm data sample, is the corresponding true label, is the number of new samples; a loss function is defined to measure the prediction error of the model on the new sample set;

[0073] In the lightweight and incremental fine-tuning process, only part of the parameters of the distilled noise reduction small model are updated, including the parameters of the last layer or the intermediate layer, and these updatable parameters are denoted as ; the updatable parameters are updated by the Adam optimization algorithm; a small learning rate , to ensure that the step of updating is small, to avoid causing too much disturbance to the overall parameters of the model, and the learning rate takes a value between 0 and 1; the update formula is:

[0074]

[0075] wherein, represents the loss function of the model; the gradient of the updateable parameter ;

[0076] Finally, the new value of the updated updateable parameter is replaced into the original distilled small denoising model to obtain new small denoising model parameters :

[0077]

[0078] The incremental fine-tuning process will continuously optimize the internal parameters of the small denoising model to make it have better fitting ability for new data.

[0079] S42. In the real-time inference stage of alarm processing, the safety alarm data is dynamically fused with the associated knowledge in the external knowledge base.

[0080] When the system receives new safety alarm data input, the associated knowledge in the external knowledge base is dynamically retrieved and fused with the safety alarm data and input into the small model, specifically:

[0081] When the system receives new safety alarm data , the key fields are used as query conditions to retrieve associated knowledge in the external knowledge base , and the retrieval process is represented as:

[0082]

[0083] wherein, is the retrieved associated knowledge, represents the retrieval operation;

[0084] The retrieved associated knowledge is fused with the new safety alarm data to obtain the fused enhanced alarm input; the fusion methods include feature splicing, text embedding enhancement, or attention mechanism injection.

[0085] The enhanced alarm input is input into the small denoising model for semantic analysis, and the alarm classification result and risk level assessment are output and treatment suggestions :

[0086] .

[0087] Further, the S5 comprises:

[0088] The collected and preprocessed security alarm data is input into the screening rule base and the small noise reduction model of the dynamic comprehensive rule knowledge base for multi-dimensional analysis and cross verification;

[0089] In the serial mode, the security alarm data is first screened by the screening rule base and then input into the small noise reduction model for analysis;

[0090] In the parallel mode, the security alarm data is simultaneously input into the screening rule base and the small noise reduction model for independent processing; the matching result of the dynamic comprehensive rule knowledge base and the analysis output of the small noise reduction model are used to generate the alarm result and the treatment suggestion through weighted calculation and decision logic; the specific calculation formula is:

[0091]

[0092] wherein, represents the maximum weight score, represents the judgment result of the dynamic comprehensive rule knowledge base, represents the judgment result of the small noise reduction model, and are the weight coefficients of the dynamic comprehensive rule knowledge base and the small noise reduction model respectively, the value range is [0, 1], and .

[0093] The application also provides a security alarm noise reduction system based on rules and large models, which comprises a module for executing the security alarm noise reduction method based on rules and large models.

[0094] The application has the advantages that: the application innovatively combines the advantages of rule-based security alarm noise reduction and large model-based security alarm noise reduction; firstly, the rule base is used to preliminarily reduce the massive security alarm data, quickly filter low-value alarms, and reduce the data volume; then, the small model distilled from the large model in the security field by using the knowledge distillation technology is input, the small model is fine-tuned by using the external knowledge base and dynamically injected with knowledge, and the small model is used for analysis and high-precision alarm recognition; the method not only has the efficiency and pertinence of the rule base, but also improves the accuracy and adaptability of alarm processing by means of the strong learning ability of the large model, and effectively solves the problems in the prior art.

[0095] The application utilizes artificial intelligence and large model technology to efficiently denoise, accurately screen and intelligently process massive network security alarm data, improves the response efficiency and accuracy of network security events, reduces the false positive and false negative rates, and simultaneously assists in reducing the work burden of network security operation personnel. BRIEF DESCRIPTION OF DRAWINGS

[0096] Figure 1 A flowchart of a security alarm denoising method based on a large model provided by the application;

[0097] Figure 2 A module connection diagram of a security alarm denoising method based on a large model provided by the application;

[0098] Figure 3 A model architecture diagram of a security alarm denoising method based on a large model provided by the application;

[0099] Figure 4 A structural diagram of an electronic device provided by the application. DETAILED DESCRIPTION

[0100] Embodiment 1: As shown in the following, a security alarm denoising method based on rules and a large model, the method comprising: Figures 1-4

[0101] S1. Collection and preprocessing of security alarm data: collecting massive multi-source heterogeneous security alarm data and security logs from various security devices and performing preprocessing; providing a comprehensive and high-quality data basis for subsequent processing;

[0102] Further, the S1 comprises:

[0103] S11. Connecting various security devices (such as firewalls, intrusion detection systems, intrusion prevention systems, security information and event management systems, etc.) to the data collection system, and collecting security alarm data and security logs generated by the devices in real time; configuring data acquisition parameters for each security device to build a comprehensive and continuous data stream; the data acquisition parameters include data format, acquisition frequency, data transmission protocol, to ensure that the alarm data generated by all devices can be stably and efficiently transmitted to the data collection system;

[0104] S12. The data collection system receives raw security alarm data from different security devices and performs preliminary analysis and multi-level preprocessing; preprocessing includes three key steps of data cleaning, format conversion and data standardization to ensure the high quality and consistency of the data;

[0105] The preliminary analysis refers to the structured analysis of the raw alarm data collected from different security devices (such as firewalls, IDS / IPS, EDR, SIEM, etc.) for subsequent processing. The specific technical means include:​

[0106] Protocol parsing: According to the log transmission protocol used by each security device, call the corresponding parser to convert the original byte stream or text into structured fields. For example: for EDR alerts in JSON format, verify and extract key attributes such as process path, hash value, parent process ID, and behavior type through JSON Schema.

[0107] Log template matching and normalization: Use log parsing tools (Logstash, Fluentd, Apache NiFi, etc.) based on clustered log pattern recognition (IPLoM algorithm) to map unstructured or semi-structured logs to a unified event model.

[0108] Metadata enhancement: Query the internal asset database based on IP addresses to supplement hostnames, business system ownership, and responsible persons, etc. Context information; or enrich URLs, domain names, and file hashes through threat intelligence platforms.

[0109] The data cleaning includes:

[0110] Using a hash algorithm to identify and delete duplicate alert records to avoid redundant information;

[0111] Using linear interpolation to scientifically fill in missing key field values;

[0112] Checking and correcting data format errors to ensure that core information meets standard format definitions; core information includes IP addresses, timestamps, and port numbers;

[0113] The format conversion includes:

[0114] Establishing a field mapping table to unify field names from different security devices; for example, "source_ip" is mapped to "source address"; timestamps are converted to ISO 8601 format; numerical data is converted to a unified unit;

[0115] Converting timestamps to a unified format and numerical data to a unified unit to ensure data consistency;

[0116] The data standardization includes:

[0117] Standardizing the preprocessing of text format security alert data, including: unifying character encoding to UTF-8, standardizing line breaks to backslash plus lowercase n expression, and standardizing indentation style;

[0118] Specifically, the text format alert data can be uniformly processed as UTF-8 encoding; regular expressions are used to extract key information such as IP addresses, port numbers, and event types, and store them as structured fields;

[0119] Key information in the text is extracted using regular expression technology and stored as structured fields, improving data readability and usability;

[0120] S13. Select a distributed database (Apache Cassandra) as the storage system for security alert data, which has significant advantages such as high throughput, low latency, and high availability; index the stored security alert data to support fast retrieval and query based on key fields; key fields include timestamp, event type, and source address; at the same time, regularly back up the stored security alert data and develop data recovery strategies to comprehensively ensure the safety and reliability of the data.

[0121] After the processing of the above step S1, the text format data of different sources is converted into structured data of a unified format, which effectively ensures the consistency and readability of the data. This lays a solid foundation for subsequent rule base noise reduction processing, enabling the rule base to efficiently perform preliminary noise reduction tasks and significantly improve overall processing performance.

[0122] S2. Construct a dynamic comprehensive rule knowledge base: construct a dynamic comprehensive rule knowledge base containing a filtering rule base and an external knowledge base, and periodically update the dynamic comprehensive rule knowledge base;

[0123] The present application enters the stage of constructing a dynamic comprehensive rule knowledge base after completing the collection of massive alarm data. The construction of the dynamic comprehensive rule knowledge base aims to provide strong support for subsequent alarm processing, and by integrating traditional rules and multi-dimensional security information, it endows the large model with deep background understanding ability, thereby enhancing its accuracy and efficiency in analyzing alarm data;

[0124] Further, the S2 includes the following steps:

[0125] S21. Construct a filtering rule base, clearly divided into multiple types and determine standardized representation and storage method, used for fast matching and preliminary noise reduction; the filtering rule base includes blacklist and whitelist rules, feature matching rules, statistical analysis rules, semantic analysis rules, intelligence-driven rules, etc.

[0126] S22. Construct an external knowledge base, integrating multi-dimensional security field professional knowledge, threat intelligence and attack patterns, disposal process and emergency response methods, and personalized environment information; the external knowledge base provides the large model with rich and high-quality background knowledge required for analysis, enhancing its understanding ability and analysis and processing ability for alarm data;

[0127] S23. Integrate the knowledge base and establish a dynamic updating mechanism to ensure the timeliness and accuracy of the knowledge.

[0128] Further, the S21 includes:

[0129] The white list rules are used to identify known safe network entities (such as IP addresses, port numbers, user accounts, etc.), and the alarms generated by these entities are directly ignored; the black list rules are used to identify known malicious network entities, and the alarms generated by these entities are paid attention to and retained;

[0130] Specifically, the white list rules are represented as a set , wherein represents a network entity in the white list; the black list rules are represented as a set , wherein represents a network entity in the black list; for a given network entity in the alarm data , if , the alarm is marked as an invalid alarm; if , the alarm is marked as a valid alarm, is paid attention to, and is retained.

[0131] The feature matching rules are used to quickly identify alarms matching known attack patterns; the feature matching rules are based on known attack features or patterns, including malware signatures, abnormal network traffic features, known exploit behaviors, etc.

[0132] The feature matching rules are represented as regular expressions or string patterns , wherein represents an attack feature pattern; for a given alarm information , if matches any one of the patterns (i.e. is a substring of or matches the regular expression of ), the alarm is considered to match a known attack pattern and is marked as a valid alarm and is retained.

[0133] The statistical analysis rules are used to identify abnormal behaviors based on time, frequency, or other statistical features.

[0134] A frequency threshold of alarms triggered by an IP address in a short period of time is set, and alarms exceeding the threshold are considered abnormal behaviors; a time window and a frequency threshold are set, for an IP address , the number of alarms triggered in the time window is , if , the behavior of the IP address is considered abnormal, is marked as a valid alarm, and is retained; if If so, the alert is not directly considered as a valid alert, but as a low-frequency candidate alert, which is handed over to the subsequent alert judgment module for further research and judgment.

[0135] The semantic analysis rule is to analyze the content of the alert text through keywords and natural language processing technology, and identify the semantic pattern related to known threats.

[0136] The intelligence-driven rule is to directly use the compromised indicators IOCs (Indicators of Compromise, IOCs) in the latest threat intelligence TI, such as malicious IP addresses, domain names, hash values, etc., to quickly identify alerts related to known threats.

[0137] Further, the S22 comprises:

[0138] The security field professional knowledge covers the knowledge system refined from authoritative research literature, industry standards and expert experience, including attack techniques and vulnerability information in the MITRE ATT&CK framework.

[0139] The threat intelligence and attack pattern refers to the continuous collection and update of typical attack patterns and threat intelligence from public vulnerability databases, global threat intelligence platforms and security communities, including the life cycle steps of various attacks, the commonly used technical tools of attackers and the latest compromised indicators IOCs.

[0140] The disposal process and emergency response method refers to the systematic arrangement and archiving of standard disposal processes and emergency response methods for various security incidents, including emergency response steps, vulnerability repair measures and traceability guidelines.

[0141] The personalized environment information collects and integrates the enterprise's internal network topology, asset list, business system correlation and labeled historical alert data, including real and valid alerts and invalid alerts, to enhance the model's understanding and analysis ability for specific environments.

[0142] Further, the S23 comprises:

[0143] The rules collected in the screening rule library, the security field professional knowledge in the external plug-in knowledge base, the threat intelligence and attack pattern, the disposal process and emergency response method, and the personalized environment information are integrated into a unified knowledge base to construct a structured knowledge system. At the same time, an update strategy is developed to regularly obtain new knowledge and patterns from security research, actual attack event analysis and industry intelligence sharing platform to ensure the timeliness and integrity of the knowledge base. An audit process is set up to ensure the accuracy and reliability of the updated content. The effectiveness of the dynamic comprehensive rule knowledge base is verified through simulation testing and actual application, and adjustments and optimizations are made based on the verification results.

[0144] Through the above step S2, the constructed comprehensive rule knowledge base can not only support the preliminary noise reduction based on rules, but also provide rich background knowledge for the large model, thereby enhancing the understanding and analysis ability of the large model on the alarm data.

[0145] S3. Constructing a noise reduction small model based on a security field open source large model: selecting a security field open source large model, using a security field open source data set and historical security alarm data, and using a distillation technology to fine-tune a noise reduction small model to strengthen the noise reduction performance of the small model;

[0146] The present application uses the knowledge distillation technology to extract an efficient and lightweight small model specially for security alarm noise reduction from a pre-trained advanced security field large model. This process enables the small model to inherit the deep professional knowledge of the large model while achieving significant optimization of reasoning efficiency and resource consumption, thereby being able to efficiently process massive alarm data and achieve accurate noise reduction.

[0147] The present application selects a pre-trained advanced open source large model in the security field (such as Foundation-Sec-8B) as a teacher model, transfers the knowledge of the teacher model to the small model by means of the knowledge distillation technology, and distills a small model specially for security alarm noise reduction. In this process, based on the open source large model, the security field open source data set and the historical security alarm information are used to distill the noise reduction small model to strengthen the noise reduction performance of the noise reduction small model. The distilled noise reduction small model not only has the deep professional knowledge of the large model in the network security field, but also has efficient reasoning ability and low resource consumption when processing security alarms, so that it can process massive alarm data and achieve efficient and accurate alarm noise reduction.

[0148] Specifically, the S3 includes;

[0149] S31. Determine the teacher model and prepare the data set for distillation training;

[0150] Select an open source large model pre-trained in the security field as the teacher model ; at the same time, prepare the data set , wherein, represents the i-th security alarm data sample preprocessed by the S1 step, represents the true class label corresponding to the i-th security alarm data sample, represents the total number of samples in the data set; the data set includes a security field open source data set and historical security alarm data preprocessed and labeled by the S1 step in the enterprise;

[0151] S32. After the teacher model and the data set are prepared, execute the knowledge distillation training process for training the student model ; teacher model process each safety alert data sample and output its prediction for the sample , obtained by applying a Softmax function with a temperature parameter at the output layer of the teacher model:

[0152]

[0153] wherein denotes the raw, unnormalized output of the teacher model for a safety alert data sample belonging to class j is an index variable in the summation operation, used to iterate over all possible output classes, i.e. class indices, denotes the raw, unnormalized output of the teacher model for an input safety alert data sample belonging to the j-th class; this summation operation is over all possible classes, in order to compute the denominator part, i.e. the normalization factor, to ensure that the Softmax output is a valid probability distribution, j indicates the class corresponding to each neuron in the output layer of the teacher model, in order to compute the denominator of the Softmax function;

[0154] student model then processes the same safety alert data sample through its own network structure and tries to mimic the soft target output by the teacher model :

[0155]

[0156] wherein denotes the raw, unnormalized output of the student model for a safety alert data sample belonging to class ;

[0157] loss function for knowledge distillation is a weighted combination of two loss components:

[0158]

[0159] wherein is a weight parameter, used to balance the importance of the soft target loss and the hard label loss ;

[0160] is the KL divergence loss between student model prediction and teacher model soft target, which encourages student model to imitate the judgment logic of teacher model:

[0161]

[0162] is the cross-entropy loss between student model prediction and original hard label, which is used to ensure that student model learns the correct classification:

[0163]

[0164] where, is the natural logarithm, is an indicator function, which is 1 when the true class of the security alert data sample is equal to the class ; otherwise, it is 0;

[0165] S33. Further fine-tuning and performance evaluation of the distilled noise reduction small model, i.e., student model , is performed using security alert data and the latest representative samples extracted from the external knowledge base to enhance the performance of the small model in the security alert noise reduction task and its generalization ability to new threats. After training, the noise reduction small model is evaluated in multiple dimensions, including accuracy, recall rate, and F1 score. Further:

[0166] The accuracy is the proportion of the total number of samples correctly classified by the model to the total number of samples:

[0167]

[0168] where is the number of effective alerts (positive examples) correctly classified by the model; is the number of invalid alerts (negative examples) correctly classified by the model; is the number of invalid alerts (negative examples) incorrectly classified by the model as valid alerts; is the number of valid alerts (positive examples) incorrectly classified by the model as invalid alerts;

[0169] The recall rate is the proportion of the number of valid alerts correctly identified by the model to the total number of valid alerts, which measures the ability of the model to discover real threats:

[0170]

[0171] The F1 score is the harmonic mean of the accuracy and recall, which comprehensively measures the classification performance of the model;

[0172]

[0173] wherein the accuracy represents how many of the samples identified by the model as effective alarms are true effective alarms;

[0174] At the same time, this step also evaluates the inference speed and resource consumption of the model, such as memory occupation, CPU / GPU usage, and inference delay, to ensure that the model meets the strict requirements of performance and efficiency for actual safe operation. Through these comprehensive evaluations, the present application ensures that the student not only inherits the deep professional knowledge and complex scene recognition ability of the teacher model in the field of network security, but also truly possesses efficient inference ability and low resource consumption, enabling it to process massive alarm data and achieve efficient and accurate alarm noise reduction;

[0175] In the above step S3, the noise reduction small model not only inherits the deep professional knowledge of the large model, but also significantly optimizes the inference efficiency and resource consumption, thereby enabling it to efficiently process massive alarm data and achieve accurate noise reduction.

[0176] S4. Continuously enhance and drive the ability of the noise reduction small model through the external knowledge base: adopt a dual-path coordination mechanism of periodic fine-tuning and dynamic knowledge injection driven by the knowledge base to improve the ability of the noise reduction small model; This strategy ensures that the noise reduction small model not only acquires knowledge from offline training, but also responds to changing network security situations through real-time knowledge injection and continuous learning;

[0177] The present application improves the ability of the noise reduction small model through the dual-path coordination of periodic fine-tuning and dynamic knowledge injection driven by the knowledge base, continuously improves the adaptability and analysis ability of the small model, regularly extracts high-quality effective samples from the external knowledge base, continuously optimizes the parameters of the small model through lightweight incremental fine-tuning, and strengthens its generalization ability to unknown threats; The dual-path coordination mechanism of dynamic knowledge injection automatically retrieves associated knowledge from the external knowledge base when security alarm data is input, and inputs the key information and alarm data into the noise reduction small model after fusion, thereby enhancing the semantic analysis ability of the model. The noise reduction small model combines the knowledge in the external knowledge base for semantic analysis and outputs accurate risk assessment and disposal suggestions. At the same time, the system only retrieves knowledge for effective alarm triggering solutions to optimize resource consumption, and through feedback and system automatic learning, continuously and periodically updates and optimizes the external knowledge base, forming a closed-loop knowledge enhancement and model iteration mechanism.

[0178] Further, the S4 comprises:

[0179] S41. Periodically use the latest data of the external knowledge base to fine-tune the distilled noise-reducing small model in a lightweight and incremental manner, thereby continuously optimizing its parameters and enhancing its generalization ability to unknown threats;

[0180] Periodically extract the latest effective alarm samples, false alarm samples, and emerging threat intelligence from the external knowledge base. These samples are confirmed by humans or verified automatically, have high representativeness and high value, and can reflect the latest attack patterns, defense evasion techniques, or false alarm situations in specific environments. For example, when a new vulnerability is disclosed and actual attack samples appear, the related alarm information and its context will be extracted first;

[0181] Use the extracted samples to retrain the distilled noise-reducing small model in a lightweight and incremental manner, updating only part of the model's layers or parameters, using a small learning rate to significantly reduce computational resource consumption and training time. This approach enables the small model to quickly absorb new knowledge, such as identifying alarm features associated with the latest threat intelligence or correcting previously identified false alarms that are now confirmed as real alarms, effectively enhancing its generalization ability and robustness to unknown threats;

[0182] When security alarm data is input, dynamically retrieve associated knowledge from the external knowledge base and input it into the distilled noise-reducing small model after fusion with the security alarm data, to enhance the model's semantic analysis ability. Through human feedback and system automatic learning, continuously update and optimize the external knowledge base, forming a closed-loop knowledge enhancement and model iteration mechanism;

[0183] The lightweight fine-tuning refers to updating the parameters of the current model , and the new sample set extracted from the external knowledge base is , where is the new security alarm data sample, is its corresponding true label, is the number of new samples; define a loss function to measure the prediction error of the model on the new sample set;

[0184] During the lightweight and incremental fine-tuning process, only part of the parameters of the distilled noise-reducing small model are updated, including the parameters of the last layer or intermediate layers, denoted as . The updateable parameters are updated using the Adam optimization algorithm; a small learning rate is used to ensure that the step size of the update is small, avoiding excessive disturbance to the overall parameters of the model. The learning rate takes a value between 0 and 1; the update formula is:

[0185]

[0186] wherein, represents a loss function the gradient of the updatable parameters ; represents the new value of the updatable parameters after updating;

[0187] Finally, the new value of the updated updatable parameters is replaced into the original distilled small denoising model to obtain new small denoising model parameters :

[0188]

[0189] The small denoising model can quickly absorb the knowledge of field denoising, enhance the generalization ability and robustness to unknown security threats, while keeping most of the parameters unchanged, and realize lightweight update;

[0190] The incremental fine-tuning process will continuously optimize the internal parameters of the small denoising model to make it have better fitting ability to new data; at the same time, strict model version management and rollback mechanism are implemented to ensure that after each incremental fine-tuning, the new model version can pass the offline test to verify its performance improvement, and can be rolled back to the old version when necessary, to ensure the stability of the system;

[0191] S42. In the real-time inference stage of alarm processing, the security alarm data is dynamically fused with the associated knowledge in the external knowledge base, thereby greatly enhancing the context understanding and semantic analysis ability of the small model.

[0192] When the system receives new security alarm data input, the associated knowledge in the external knowledge base is dynamically retrieved and fused with the security alarm data before inputting into the small model to enhance the analysis ability of the small model. Specifically:

[0193] When the system receives new alarm data input, it will first perform preliminary evaluation. In order to optimize resource consumption, only those alarms that are preliminarily screened and judged as "possibly effective" by the rule base constructed in S2 or "exist potential risks" by the model trigger the subsequent knowledge retrieval process. The system will use the key fields of the alarm (such as source / destination IP, event type, involved asset ID, alarm description keywords, etc.) as query conditions to automatically retrieve the strongly associated context information, threat intelligence and expert disposal suggestions from the external knowledge base in real time. For example, retrieve the historical attack records related to the alarm IP, the MITRE ATT&CK technical description related to the event type, or the business importance information related to the attacked asset.

[0194] When the system receives new security alarm data At this time, the key fields (such as source / destination IP, event type, etc.) are used as query conditions to retrieve associated knowledge in the external knowledge base The retrieval process is represented as:

[0195]

[0196] wherein, is the retrieved associated knowledge, represents the retrieval operation;

[0197] The retrieved key knowledge information will be structured and fused with the original alarm data. This fusion can be feature splicing, text embedding enhancement, or context injection through an attention mechanism. The fused data forms a "knowledge-enhanced alarm input" that contains detailed content of the original alarm and relevant background knowledge obtained from the knowledge base, providing a more rich and semantic input for the small model. The enhanced alarm input is then sent to the distilled small model for semantic analysis and judgment. The small model, combined with the real-time background knowledge and context information provided by the knowledge base, can make more accurate risk assessment and identify complex associated threats that traditional models cannot detect. Finally, the model will output accurate alarm classification results, detailed risk level assessment, and refined handling suggestions, such as recommended response steps, associated threat intelligence, affected asset range, and preventive measures.

[0198] The retrieved associated knowledge is fused with the new security alarm data to form a "knowledge-enhanced alarm input", and the fused enhanced alarm input is obtained; the fusion methods include feature splicing, text embedding enhancement, or attention mechanism injection;

[0199] Taking feature splicing as an example:

[0200]

[0201] wherein, represents the feature splicing operation, is the fused enhanced alarm input;

[0202] The enhanced alarm input is input into the noise reduction small model for semantic analysis, and outputs alarm classification results , risk level assessment , and handling suggestions :

[0203] .

[0204] The dynamic knowledge injection mechanism is also a source of continuous optimization of the knowledge base. Through manual feedback on the output results of the model (such as confirming that a certain alarm is a valid alarm or a false alarm, or adopting a disposal suggestion), and automatic learning of the system (such as identifying repeated false alarm patterns and discovering new attack indicators), these feedback information will be collected regularly and used to update and optimize the content of the external knowledge base. This forms a closed-loop knowledge enhancement and model iteration mechanism, ensuring the freshness of the knowledge base and the continuous learning ability of the model, thereby continuously improving the efficiency and accuracy of the overall noise reduction method.

[0205] Through the detailed steps of S4 above, the present application realizes the continuous adaptability and intelligent analysis capability of the noise reduction small model in dealing with complex and variable network security threats.

[0206] S5. Full-process automatic alarm noise reduction: after collecting and preprocessing the security alarm data, the dynamic comprehensive rule knowledge base constructed in S2 and the enhanced noise reduction small model in S4 are entered in parallel or series to perform multi-dimensional analysis and cross-validation on the security alarm data.

[0207] The present application aims to build an efficient and automated full-process alarm processing method to realize seamless flow, intelligent noise reduction and accurate analysis of alarm data. Through the collaborative work of the rule base and the knowledge-enhanced distillation small model, this process can ensure that key threats are not missed and can be flexibly adjusted according to different security scene requirements.

[0208] Further, the S5 comprises:

[0209] The collected and preprocessed security alarm data is input into the screening rule base of the dynamic comprehensive rule knowledge base and the noise reduction small model for multi-dimensional analysis and cross-validation, and the final output of high-precision alarm results and reasonable disposal suggestions is obtained through weighted calculation and comprehensive judgment, ensuring that key threats are not missed;

[0210] In the serial mode, the security alarm data is first screened by the screening rule base and then input into the noise reduction small model for analysis;

[0211] Specifically: in the serial mode, the security alarm data is first filtered and denoised in the first stage by the screening rule base constructed in S2. The screening rule base, with its pre-defined black and white list rules, feature matching rules, statistical analysis rules, semantic analysis rules, and intelligence-driven rules, can quickly identify and eliminate a large number of known false alarms or low-priority alarms, while preferentially marking known valid alarms. Only the alarms that pass the preliminary screening by the screening rule base will be further input into the enhanced noise reduction small model in S4 for semantic analysis and complex pattern recognition. This mode is suitable for scenarios that require rapid filtering of a large amount of noise.

[0212] In parallel mode, the preprocessed security alert data is simultaneously input into the screening rule library and the noise reduction small model for independent processing; the matching results of the dynamic comprehensive rule knowledge base and the analysis output of the noise reduction small model are dynamically integrated, and high-precision alarm results and reasonable disposal suggestions are generated through weighted calculation and decision logic. This process not only ensures that key threats are not missed, but also adapts to different security scene requirements through flexible rule combination and weight adjustment, thereby realizing efficient and accurate alarm noise reduction. This mode can complement each other through the determinacy of rules and the generality of models, speed up the overall processing speed, and provide more comprehensive information for subsequent comprehensive judgment.

[0213] Regardless of the mode adopted, the noise reduction small model will benefit from the dynamic knowledge injection (such as step S42) mechanism during analysis, that is, the system will retrieve associated knowledge from the external knowledge base according to the alarm content, and input the key information and alarm data into the noise reduction small model after fusion, thereby enhancing the understanding and judgment ability of the noise reduction small model for the alarm context.

[0214] Combined with the parallel or serial analysis results, high-precision final alarms are generated and executable disposal suggestions are provided. The model will comprehensively consider the matching results from the rule library (such as the severity of the hit rule, the type of the rule), the analysis output of the distilled noise reduction small model (such as the risk score of the alarm, the threat type, the confidence), and the context information provided by the dynamic knowledge injection. Through weighted calculation and decision logic, the system makes a comprehensive judgment on the alarm, for example, the final weight score The specific calculation formula is:

[0215]

[0216] Among them, represents the final weight score, represents the judgment result of the dynamic comprehensive rule knowledge base, represents the judgment result of the noise reduction small model, and are the weight coefficients of the dynamic comprehensive rule knowledge base and the noise reduction small model, respectively, with a value range of [0, 1], and , the model outputs high-precision alarm results and provides reasonable disposal suggestions and associated threat intelligence.

[0217] Based on , the system will output high-precision alarm results. For alarms identified as real and high-quality, the system will further provide reasonable disposal suggestions, associated threat intelligence, and detailed context explanations to guide security analysts for rapid response. This intelligent decision-making process not only ensures that key threats are not missed, but also through flexible rule combination and weight adjustment, it can adapt to different security scenarios and organization-specific needs, thus achieving efficient and accurate alarm noise reduction and intelligent response.

[0218] The application also provides a security alarm noise reduction system based on rules and large models, comprising:

[0219] A mass alarm data collection and processing module is used to collect security alarm data and security logs from various security devices and perform preprocessing;

[0220] A dynamic comprehensive rule knowledge base construction module is used to construct a dynamic comprehensive rule knowledge base containing a screening rule base and an external knowledge base, and periodically update the dynamic comprehensive rule knowledge base;

[0221] A large model distillation-based noise reduction small model module is used to select a security domain open source large model, use a security domain open source dataset and historical security alarm data, and fine-tune a noise reduction small model using distillation technology;

[0222] An external knowledge base fine-tuning noise reduction small model and dynamic knowledge injection module is used to adopt a knowledge base-driven periodic fine-tuning and dynamic knowledge injection dual-path coordination mechanism to improve the capability of the noise reduction small model;

[0223] A full-process security alarm noise reduction module is used to collect and preprocess security alarm data, and then enter the dynamic comprehensive rule knowledge base constructed by S2 and the enhanced noise reduction small model in S4 for multi-dimensional analysis and cross-validation of the security alarm data in parallel or series.

[0224] Through the above embodiment specific steps, the application realizes full-process optimization from data collection, preprocessing, preliminary noise reduction, deep noise reduction to accurate analysis, effectively improving the efficiency and accuracy of security alarm processing.

[0225] In order to illustrate the effect of the application, the application is compared and illustrated from the following aspects.

[0226] Table 1 is a comparison of security alarm noise reduction methods

[0227] Alarm noise reduction method Advantages Disadvantages Rule-based security alarm noise reduction Rules are clear and easy to understand, easy to implement and maintain; suitable for processing structured data and clear threat patterns. High labor cost, need regular maintenance. New alarms cannot be effectively identified. Large model-based security alarm noise reduction Strong learning and generalization ability, can handle complex and diverse data; automatically discover hidden patterns in data. High demand for computing resources, high cost of training and inference; strong dependence on data quality and quantity, mass alarms cannot support actual use scenarios. The method of the present application (rule-based and large model-based security alarm noise reduction) Combine the clarity of rules and the learning ability of large models to improve alarm processing accuracy; rule base preliminary screening, reduce the burden of large models, achieve efficient and accurate noise reduction. Increased system complexity, need to manage and maintain rule knowledge base and small model

[0228] The application embodiment simultaneously provides an electronic device. Figure 4 An example shows the entity structure diagram of the electronic device, as shown in Figure 4As shown, the electronic device mainly includes a processor 310, a communications interface 320, a memory 330, and a communications bus 340. Among them, the processor 310, the communications interface 320, and the memory 330 are communicatively connected to each other through the communications bus 340. The processor 310 can invoke the logic instructions in the memory 330 to execute the foregoing rule-based and large model-based security alarm noise reduction method.

[0229] In addition, when the logic instructions in the memory 330 are implemented in the form of a software function unit and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on this understanding, the core contribution part or the whole technical solution of the technical solution of the present application can be presented in the form of a software product. The computer software product stored in a specific storage medium contains a plurality of instructions, which can make a computer device (such as a personal computer, a server, a network device, etc.) execute all or part of the operation steps of the method described in each embodiment of the present application. The above-mentioned storage medium covers various types of storage media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, etc.

[0230] The embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, and the computer program can implement the rule-based and large model-based security alarm noise reduction method when executed by a processor.

[0231] Meanwhile, the embodiment of the present application provides a computer program product containing a computer program, which can implement the rule-based and large model-based security alarm noise reduction method when executed by a processor.

[0232] The specific embodiments of the present application are described in detail above in combination with the accompanying drawings, but the present application is not limited to the above-mentioned embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the purpose of the present application.

Claims

1. A rule and large model based security alert noise reduction method, characterized in that: The method comprises: S1. Collection and preprocessing of security alarm data: collecting security alarm data and security logs from various security devices and preprocessing; S2. Building a dynamic comprehensive rule knowledge base: building a dynamic comprehensive rule knowledge base containing a screening rule base and an external knowledge base, and periodically updating the dynamic comprehensive rule knowledge base; S3. Building a noise reduction small model based on a security field open source large model: selecting a security field open source large model, using a security field open source dataset and historical security alarm data, and fine-tuning a noise reduction small model using distillation technology; S4. Continuously enhancing and driving the ability of the noise reduction small model through the external knowledge base: adopting a dual-path coordination mechanism of periodic fine-tuning and dynamic knowledge injection driven by the knowledge base to improve the ability of the noise reduction small model; S5. Full-process automatic alarm noise reduction: after collecting and preprocessing the security alarm data, entering the dynamic comprehensive rule knowledge base built in S2 and the enhanced noise reduction small model in S4 for multi-dimensional analysis and cross-validation of the security alarm data in parallel or series; The S3 further comprises: S31. Determine the teacher model and prepare the dataset for distillation training; An open-source large model pre-trained in the security field is selected as a teacher model ; meanwhile, a data set is prepared , wherein, represents the i-th security alarm data sample pre-processed through the S1 step, represents the true class label corresponding to the i-th security alarm data sample, indicates the total number of samples in the data set; the data set includes an open-source data set in the security field and historical security alarm data pre-processed and labeled through the S1 step in the enterprise. S32. In the teacher model and the dataset After being prepared, a training procedure of knowledge distillation is performed for training the student model ; the teacher model processes each security alert data sample and outputs its prediction for the sample , by applying a Softmax function with a temperature parameter on the output layer of the teacher model. ； wherein, denotes the teacher model for a security alert data sample unnormalized raw output belonging to the class j is an index variable in the summation operation for iterating through all possible output classes, i.e. class indices, denotes the teacher model for an input security alert data sample unnormalized raw output belonging to the j-th class; Student model then processes the same security alert data sample through its own network architecture and attempts to mimic the teacher model output soft target : ； wherein, representing a student model for a safety alert data sample belongs to a class raw output without softmax normalization; The loss function for knowledge distillation is a weighted combination of two-part loss: ； wherein, is a weight parameter balancing the importance of the soft target loss and the hard label loss ; is the KL divergence loss between student model predictions and teacher model soft targets, encouraging the student model to mimic the judge- ment logic of the teacher model: ； is the cross-entropy loss between the student model predictions and the original hard labels, used to ensure that the student model learns the correct classification: ； wherein, is the natural logarithm, is an indicator function that takes the value 1 when the true class of a security alert data sample is equal to the class ; otherwise it takes the value 0. ; otherwise it takes the value 0. S33. Distilling the noise-reduced small model, i.e., the student model Further fine-tuning and performance evaluation are performed to strengthen the performance of the small model in the security alert noise reduction task and the generalization ability to new threats using security alert data and the latest representative samples extracted from the external knowledge base. After training, the noise-reduced small model is evaluated in multiple dimensions.

2. The rule and large model-based security alarm noise reduction method of claim 1, wherein: The S1 comprises: S11. Connect various security devices to the data collection system and collect real-time security alarm data and security logs generated by the devices; configure data acquisition parameters for each security device; the data acquisition parameters include data format, acquisition frequency, and data transmission protocol; S12. The data collection system receives raw security alarm data from different security devices and performs preliminary analysis and multi-level preprocessing; preprocessing includes data cleaning, format conversion, and data standardization; The data cleaning comprises: Use the hash algorithm to identify and delete duplicate alarm records; Use the linear interpolation method to scientifically fill in the missing key field values; Check and correct data format errors to ensure that the core information meets the standard format definition; the core information includes IP address, timestamp, and port number; The format conversion comprises: Establish a field mapping table to unify the field names of different security devices; Convert the timestamp to a unified format and convert the numerical data to a unified unit; The data standardization comprises: Standardize the preprocessing of text format security alarm data, including: unify the character encoding to UTF-8, standardize the newline character to the expression of backslash plus lowercase n, and standardize the indentation style; Use regular expression technology to extract key information from the text and store it as structured fields; S13. Select a distributed database as the storage system for security alarm data, index the stored security alarm data, support fast retrieval and query based on key fields; the key fields include timestamp, event type, and source address; at the same time, regularly back up the stored security alarm data and develop a data recovery strategy.

3. The rule and large model-based security alert noise reduction method of claim 1, wherein: The S2 comprises the following steps: S21. Build a screening rule base, clearly divide it into multiple types and determine the standardized representation and storage method, for fast matching and preliminary noise reduction; the screening rule base includes blacklist and whitelist rules, feature matching rules, statistical analysis rules, semantic analysis rules, and intelligence-driven rules; S22. Constructing an external knowledge base, integrating multi-dimensional security field professional knowledge, threat intelligence and attack patterns, disposal process and emergency response methods, and personalized environment information; S23. Integrating the knowledge base and establishing a dynamic updating mechanism.

4. The rule and large model-based security alert noise reduction method of claim 3, wherein: The S21 includes: The white list rule is used to identify known safe network entities, and the alarms generated by these entities are directly ignored; the black list rule is used to identify known malicious network entities, and the alarms generated by these entities are paid attention to and retained; In particular, a whitelist rule is represented as a set wherein, denotes a network entity in the whitelist; a blacklist rule is represented as a set wherein, denotes a network entity in the blacklist; for a given network entity in the alarm data if then the alarm is marked as an invalid alarm; if then the alarm is marked as a valid alarm, is attended and is kept. The feature matching rule is used to quickly identify alarms matching known attack patterns; the feature matching rule is based on known attack features or pattern design rules, including malware signatures, abnormal network traffic features, and known exploit behaviors; The feature matching rule is expressed as a regular expression or a string pattern wherein, represents an attack feature pattern; for a given alert information , if matches any one of the patterns , the alert is considered to be matched with the known attack pattern, and is marked as a valid alert and is preserved; The statistical analysis rule is used to identify abnormal behaviors based on time and frequency statistical features; Set a frequency threshold of alarms triggered by a certain IP address in a short time, and alarms exceeding the threshold will be considered as abnormal behavior; set a time window and a frequency threshold , for an IP address , the number of alarms triggered in the time window is If , the behavior of the IP address is considered as abnormal behavior, marked as a valid alarm and retained; if , the alarm is not directly considered as a valid alarm, but as a low-frequency candidate alarm, handed over to the subsequent alarm judgment module for further research and judgment; The semantic analysis rule is to analyze the alarm text content through keyword and natural language processing technology to identify semantic patterns related to known threats; The intelligence-driven rule directly uses the compromise indicators IOCs in the latest threat intelligence TI to identify alarms related to known threats.

5. The rule and large model based security alert noise reduction method of claim 3, wherein: The S22 includes: The security field professional knowledge covers authoritative research literature, industry standards, and expert experience refined knowledge system, including attack techniques and vulnerability information in the MITRE ATT&CK framework; The threat intelligence and attack pattern refers to the continuous collection and updating of typical attack patterns and threat intelligence from public vulnerability databases, global threat intelligence platforms, and security communities, including the life cycle steps of various attacks, commonly used technical tools of attackers, and the latest compromise indicators IOCs; The disposal process and emergency response method refers to the systematic arrangement and archiving of standard disposal processes and emergency response methods for various security incidents, including emergency response steps, vulnerability repair measures, and traceability guidelines; The personalized environment information collects and integrates the enterprise's internal network topology, asset list, business system correlation, and labeled historical alarm data, including real and valid alarms and invalid alarms.

6. The rule and large model based security alert noise reduction method of claim 3, wherein: The S23 includes: Integrate the collected rules in the screening rule base, the security field professional knowledge in the external knowledge base, the threat intelligence and attack pattern, the disposal process and emergency response method, and the personalized environment information into a unified knowledge base to build a structured knowledge system; at the same time, develop an updating strategy to regularly obtain new knowledge and patterns from security research, actual attack event analysis, and industry intelligence sharing platforms to ensure the timeliness and completeness of the knowledge base; set up an audit process to ensure the accuracy and reliability of the updated content; use simulation testing and actual application to verify the effectiveness of the dynamic comprehensive rule knowledge base, and adjust and optimize it according to the verification results.

7. The rule and large model based security alert noise reduction method of claim 1, wherein: The S4 includes: S41. Periodically fine-tune the distilled noise reduction small model using the latest data from the external knowledge base; Periodically extract the latest valid alarm samples, false alarm samples, and emerging threat intelligence from the external knowledge base; The distilled noise reduction small model is retrained in a light and incremental manner using the extracted samples, only part of the layers or parameters of the model are updated, and a small learning rate is used to significantly reduce the consumption of computing resources and training time; When the safety alarm data is input, the associated knowledge in the external knowledge base is dynamically retrieved and fused with the safety alarm data, and then input into the distilled noise reduction small model to enhance the semantic analysis capability of the model; through artificial feedback and system automatic learning, the external knowledge base is continuously updated and optimized, forming a closed-loop knowledge enhancement and model iteration mechanism; The light-weight fine-tuning refers to that parameters of a current model are A new sample set extracted from the external knowledge base is Wherein is a new security alarm data sample, is a corresponding true label thereof, is a number of new samples; a loss function is defined to measure a prediction error of the model on the new sample set. In the lightweight, incremental fine-tuning process, only part of the parameters of the distilled noise reduction small model are updated, including the parameters of the last layer or the intermediate layer, and these updatable parameters are denoted as ; the updatable parameters are updated by the Adam optimization algorithm; the update formula is: ； wherein, represents a loss function the gradient of the updatable parameters ; represents the new value of the updatable parameters after the update, is the learning rate; Finally, the new values of the updatable parameters are updated The new denoising small model parameters are obtained by replacing the denoising small model after the original distillation : ； The incremental fine-tuning process continuously optimizes the internal parameters of the noise reduction small model to improve its fitting capability for new data; S42. In the real-time reasoning stage of alarm processing, the safety alarm data is dynamically fused with the associated knowledge in the external knowledge base; When the system receives new safety alarm data input, the associated knowledge in the external knowledge base is dynamically retrieved and fused with the safety alarm data, and then input into the small model, specifically: When the system receives new security alarm data , it uses the key fields as query conditions to search for related knowledge in the external knowledge base . The search process is represented as follows: ； wherein, retrieved associated knowledge, denotes a retrieval operation; retrieved associated knowledge with the new security alert data fusion, obtaining a fused enhanced alert input; the fusion mode includes feature splicing, text embedding enhancement, or attention mechanism injection; Enhanced alarm input Input noise reduction small model , semantic analysis, output alarm classification result , risk level assessment And disposal suggestion : 。 8. The rule and large model based security alert noise reduction method of claim 1, wherein: The S5 includes: The collected and preprocessed safety alarm data is input into the screening rule base of the dynamic comprehensive rule knowledge base and the noise reduction small model for multi-dimensional analysis and cross-validation; In the serial mode, the safety alarm data is first screened by the screening rule base and then input into the noise reduction small model for analysis; In the parallel mode, the safety alarm data is simultaneously input into the screening rule base and the noise reduction small model for independent processing; the matching results of the dynamic comprehensive rule knowledge base and the analysis output of the noise reduction small model are used to generate alarm results and disposal suggestions through weighted calculation and decision logic; the specific calculation formula is: ； wherein, denotes the maximum weight score, denotes the judgment result of the dynamic comprehensive rule knowledge base, denotes the judgment result of the denoising small model, and are weight coefficients of the dynamic comprehensive rule knowledge base and the denoising small model respectively, the value range is [0, 1], and .

9. A rule and large model based security alert noise reduction system, characterized in that, The system includes a module for executing a safety alarm noise reduction method based on rules and large models as claimed in any one of claims 1 to 8.

Images