System and method for combining intelligent alarm study and judgment with safe operation system

Abstract

The invention discloses an intelligent alarm research and judgment and safety operation system combined system and method, and the system comprises an alarm classification device which is used for receiving alarm information and original logs from a safety information and event management system, and classifying alarms into a class A, a class B and a class C; the small model research and judgment module is used for processing the A-type alarms and calculating a false alarm probability through local endogenous information matching and a naive Bayes algorithm; the large model research and judgment module is used for processing the B-type alarms and generating a research and judgment result based on the extracted alarm vulnerability information and a preset question template; the manual research and judgment module is used for processing the C-type alarms and alarms which cannot pass through small model research and judgment or large model research and judgment; and the alarm memory is used for storing research and judgment results and mark information of all alarms. According to the method, the workload of a security analyst is reduced, the alarm average response time is prolonged, and the number of alarms processed manually is reduced on the premise that the missing report rate is not obviously increased.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and particularly relates to a system and method combining intelligent alarm judgment and security operation system. Background Art

[0002] In the network security operation system, daily alarms are an important basis for discovering and handling network security threats. However, with the continuous expansion of the network scale and the increasing complexity of business, the number of alarms has increased explosively, and problems such as untimely alarm processing, excessive false alarms, and low processing efficiency have emerged. These problems not only increase the workload of operation and maintenance personnel, but also may cause real security threats to be ignored, seriously affecting the quality and efficiency of network security operation.

[0003] In recent years, with the emergence of large models (large language models), some security operation platforms have also used the intelligent judgment solution of large models. Since the judgment ability of current large models is not sufficient to completely replace human analysts, it only plays an auxiliary role in many intelligent judgment solutions, and the results of intelligent judgment are only used as reference information for manual judgment, with limited improvement in operation quality.

[0004] Therefore, a solution is needed to maximize the role of large models in the security operation system, minimize false alarms without increasing the missed alarm rate, and reduce the workload of analysts. Summary of the Invention

[0005] In view of this, on the one hand, the present invention provides a system combining intelligent alarm judgment and security operation system, including, An alarm classification device, configured to receive alarm information and original logs from a security information and event management system, and classify the alarms into category A, category B, and category C; A small model judgment module, configured to process the category A alarms, and calculate the false alarm probability through local endogenous intelligence matching and the Naive Bayes algorithm; A large model judgment module, configured to process the category B alarms, and generate a judgment result based on the extracted alarm vulnerability information and a preset question template; An artificial judgment module, configured to process the category C alarms and the alarms that cannot be judged by the small model or the large model; An alarm memory, configured to store the judgment results and marking information of all alarms.

[0006] Further, the classification of alarms by the alarm classification device is: Category A alarms are alarm types with simple logic, including denial of service, weak password, brute force attack, port scan, intelligence-based alarms, and alarms that large models are not good at judging; Type B alarms are alarm types suitable for large model research and judgment and with a large model test accuracy rate higher than the preset threshold, including Web attack type alarms; Type C alarms are important alarm types, including C2, WebShell high-risk alarms, honeypot type alarms, illegal software use, and user-defined key business alarms.

[0007] Furthermore, the processing method of the small model research and judgment module specifically includes: If the alarm matches the local endogenous intelligence successfully, it is directly marked as a positive report; If there is no match, the false alarm probability is calculated based on the Naive Bayes algorithm. When the false alarm probability exceeds the preset threshold H, it is marked as a false alarm; otherwise, it is marked as a positive report; If the data volume in the alarm memory is insufficient or the false alarm probability cannot be calculated, the alarm is marked as unable to be judged.

[0008] Furthermore, the formula for calculating the false alarm probability by the Naive Bayes algorithm is: ; Among them, P(Y|X) is the probability of a false alarm on the premise that the current alarm has occurred; P(Y) is the probability of the current alarm occurring; P(X) is the probability of a false alarm occurring; P(X|Y) is the probability of the current alarm occurring among the alarms that have been marked as false alarms.

[0009] Furthermore, the value range of the preset threshold H is from 0.5 to 1.0.

[0010] Furthermore, the processing method of the large model research and judgment module specifically includes: Extract the vulnerability information of the alarm, and the vulnerability information includes the occurrence time and the original log content; Combine the vulnerability information into a structured question template and input it into the large model, and require it to output the research and judgment result in a specified format; Analyze the output of the large model through keywords to determine whether the alarm is a positive report or a false alarm.

[0011] Furthermore, the processing method of the manual research and judgment module specifically includes: Push the type C alarms and the alarms that cannot be judged to the analyst workbench for manual research and judgment by the analyst; The research and judgment result of the analyst is fed back to the alarm memory for updating the training data of the small model and the large model.

[0012] Furthermore, the data update rule of the alarm memory is: All judgment results are recorded in the alarm memory; the judgment results include positive reports, false alarms, and manual markings. The stored data is used to optimize the Naive Bayes algorithm parameters of the small model and the question templates of the large model.

[0013] On the other hand, the present invention also provides a method combining intelligent alarm judgment and security operation system, including the following steps: The system generates an alarm and transmits it to the alarm classification device. The alarm classification device classifies the alarms into Class A, Class B, and Class C, and hands them over to the small model judgment module, the large model judgment module, and the manual judgment module for processing respectively. The judgment results of each module are pushed to the alarm memory, and the subsequent alarm processing priorities are adjusted based on the results.

[0014] Furthermore, the method also includes a false alarm rate control mechanism: By adjusting the threshold H in the small model judgment module and the question templates of the large model, the miss rate and the false alarm rate are balanced. Regularly count the data in the alarm memory to optimize the diversion strategy of the classification device and the model parameters.

[0015] Advantages of the present invention: The present invention combines the large model and the small model to judge alarms. During preprocessing, the alarms are divided into three categories: alarms suitable for large model judgment are judged by the large model, alarms suitable for small model judgment are judged by the small model, and alarms not suitable for automatic judgment are processed manually. Compared with the traditional technology, it reduces the workload of security analysts, improves the average alarm response time, and reduces the number of alarms processed manually without significantly increasing the miss rate. Brief Description of the Drawings

[0016] Figure 1 System structure diagram of the present invention; Figure 2 Classification schematic diagram of the alarms of the present invention; Figure 3 Flow chart of the large model processing alarms of the present invention. Detailed Description of the Invention

[0017] To make the objectives, advantages, and features of the present invention more obvious, the following detailed description further elaborates on the present invention.

[0018] The steps and structural components of the present invention are as Figure 1 shown: 1. SIEM generates an alarm and delivers the alarm information and its corresponding original log to the alarm classification device.

[0019] 2. As Figure 2As shown, the alarm classification device classifies the alarms transmitted by SIEM into three categories: Type A alarms, alarms that are not suitable for large model analysis, and other alarms with simple logic, such as denial of service, weak passwords, port detection, and intelligence alarms.

[0020] Type B alarms, alarms that are suitable for large model analysis. Before classifying Type B alarms, the large model should be tested to distinguish which types of alarms have high accuracy in the large model. Alarms with high accuracy are classified into Type B alarms, and those with low accuracy are classified into Type A. For this model, it is good at handling Web attack alarms, so Web attacks belong to Type B.

[0021] Type C alarms, important alarms. Once such alarms are accurate, the consequences are very serious. It includes C2, WebShell, honeypot alarms. Because for C2 and WebShell, the gap between the test environment and the actual combat environment is large. In the actual combat environment, hackers will try to confuse and encrypt communication streams and try their best to disguise as traffic, causing certain difficulties for the detection of the large model. It also includes alarms customized by customers according to their own business scenarios, such as using illegal software, access control, availability alarms, etc.

[0022] 3. The small model analysis module is used to process Type A alarms. First, use local endogenous intelligence for matching. If a match is found, the alarm is regarded as a positive alarm; if no match is found, use the Naive Bayes model to classify the alarm. According to the Naive Bayes formula, the probability that the current alarm is a false alarm is: ; P(Y|X): Given that the current alarm has occurred, what is the probability that it is a false alarm. This is the probability that this algorithm needs to calculate.

[0023] P(Y): The probability that the current alarm appears. Take the name of the current alarm, or the alarm name and the victim IP as conditions, retrieve the alarm memory, and the number of retrieved entries is used as the numerator. The denominator is the total number of alarms, and this value can be calculated.

[0024] P(X): The probability that a false alarm appears. Retrieve the alarm memory, the numerator is the number of false alarms, and the denominator is the total number of alarms.

[0025] P(X|Y): Among the alarms that have been marked as false alarms, what is the probability that the current alarm appears. Take the name of the current alarm, or the alarm name and the victim IP as condition one, and being marked as a false alarm as condition two. Condition one and condition two are in an AND relationship to retrieve the alarm memory, and the number of retrieved entries is used as the numerator, and the denominator is the number of false alarms, and this value can be calculated.

[0026] The threshold value H ranges from 0.5 to 1.0, and it is recommended to take a value above 0.9. When P(Y|X) exceeds the threshold value H, the current alarm is a false alarm; otherwise, it is a true alarm. Therefore, the higher the value of H, the lower the false negative rate of the small model and the higher the false positive rate.

[0027] Before using the Naive Bayes algorithm, two unexpected situations need to be checked: the first is that the value of P(X) is 0, and the second is that the number of marked alarms in the alarm memory is too small. In these two cases, the amount of data in the alarm memory is not sufficient to form statistical value, and the Bayesian algorithm should not be used. All input alarms should be classified into the category of "unable to judge".

[0028] 4. As Figure 3 shown, for the large model judgment module, Class B alarms are judged by the large model. First, vulnerability information will be extracted, including the alarm occurrence time, the content of the original log, etc. It will be combined into a question template to ask the large model whether this alarm is a false alarm and require the large model to output in the format you specify.

[0029] Analyze the content of the large model's reply. If the large model outputs according to the format requirements, then the output result can be analyzed by using keywords, and this alarm can be classified as a true alarm or a false alarm.

[0030] 5. Class C alarms are directly classified into the category of "unable to judge".

[0031] 6. For true alarm class alarms, as high-precision alarms, they are pushed to the analyst's workbench, and analysts are required to process them first. Analysts will conduct a secondary judgment on the alarms. If the judgment result is still a true alarm, then an emergency response will be carried out and the true alarm will be marked and written into the alarm memory. If it is a false alarm, it will be marked as a false alarm and written into the alarm memory.

[0032] 7. For alarms in the category of "unable to judge", they are pushed to the analyst's workbench and are manually handled by analysts in the normal order. Analysts will conduct a judgment on the alarms. If the result is a true alarm, then an emergency response will be carried out and the true alarm will be marked and written into the alarm memory. If it is a false alarm, it will be marked as a false alarm and written into the alarm memory.

[0033] 8. For false alarm class alarms, they are marked as false alarms and written into the alarm memory.

[0034] The above embodiments have described the technical solutions of the present invention in detail. Obviously, the present invention is not limited to the described embodiments. Based on the embodiments of the present invention, those skilled in the art can also make various changes accordingly, but any changes equivalent or similar to the present invention fall within the protection scope of the present invention.

[0035] The content not described in detail in this specification belongs to the well-known prior art of those skilled in the art.

Claims

1. A system that combines intelligent alarm judgment and security operation system, characterized in that, including, an alarm classification device, configured to receive alarm information and original logs from a security information and event management system, and classify the alarms into Class A, Class B, and Class C; a small model research and judgment module, configured to process the Class A alarms, and calculate the false alarm probability through local endogenous intelligence matching and the Naive Bayes algorithm; a large model research and judgment module, configured to process the Class B alarms, and generate a research and judgment result based on the extracted alarm vulnerability information and a preset question template; an artificial research and judgment module, configured to process the Class C alarms and alarms that cannot be researched and judged by the small model or the large model; an alarm memory, configured to store the research and judgment results and marking information of all alarms.

2. The system combining intelligent alarm judgment and security operation system according to claim 1, characterized in that, The classification of alarms by the alarm classification device is as follows: Class A alarms are alarm types with simple logic, including denial of service, weak password, brute force attack, port scanning, intelligence-based alarms, and alarms that are not good at being researched and judged by the large model; Class B alarms are alarm types suitable for large model research and judgment and with a large model test accuracy rate higher than a preset threshold, including Web attack alarms; Class C alarms are important alarm types, including C2, WebShell high-risk alarms, honeypot alarms, illegal software use, and user-defined critical business alarms.

3. The system combining intelligent alarm research and judgment with a security operation system according to claim 2, characterized in that, The processing method of the small model research and judgment module specifically includes: If the alarm matches the local endogenous intelligence successfully, it is directly marked as a positive alarm; If there is no match, the false alarm probability is calculated based on the Naive Bayes algorithm. When the false alarm probability exceeds the preset threshold H, it is marked as a false alarm, otherwise it is marked as a positive alarm; If the data volume in the alarm memory is insufficient or the false alarm probability cannot be calculated, the alarm is marked as unable to be researched and judged.

4. The system combining intelligent alarm research and judgment with a security operation system according to claim 3, characterized in that, The formula for calculating the false alarm probability by the Naive Bayes algorithm is: ； where, P(Y|X) is the probability of a false alarm on the premise that the current alarm has occurred; P(Y) is the probability of the current alarm occurring; P(X) is the probability of a false alarm occurring; P(X|Y) is the probability of the current alarm occurring among the alarms that have been marked as false alarms.

5. The system combining intelligent alarm judgment and security operation system according to claim 4, characterized in that, The value range of the preset threshold H is from 0.5 to 1.

0.

6. The system combining intelligent alarm judgment and security operation system according to claim 5, characterized in that, The processing method of the large model research and judgment module specifically includes: extracting the vulnerability information of the alarm, where the vulnerability information includes the occurrence time and the original log content; combining the vulnerability information into a structured question template and inputting it into the large model, and requiring it to output a research and judgment result in a specified format; analyzing the output of the large model through keywords to determine whether the alarm is a positive alarm or a false alarm.

7. The system combining intelligent alarm judgment and security operation system according to claim 6, characterized in that, The processing method of the artificial research and judgment module specifically includes: pushing the Class C alarms and alarms that cannot be researched and judged to the analyst workbench for manual research and judgment by the analyst; the research and judgment result of the analyst is fed back to the alarm memory for updating the training data of the small model and the large model.

8. The system combining intelligent alarm judgment and security operation system according to claim 7, wherein The data update rule of the alarm memory is: all research and judgment results are recorded in the alarm memory; the research and judgment results include positive alarms, false alarms, and manual markings; the stored data is used to optimize the Naive Bayes algorithm parameters of the small model and the question template of the large model.

9. A method combining intelligent alarm judgment and security operation system, characterized in that, including the following steps: the system generates an alarm and transmits it to the alarm classification device; the alarm classification device classifies the alarm into Class A, Class B, and Class C, and hands them over to the small model research and judgment module, the large model research and judgment module, and the artificial research and judgment module for processing respectively; The judgment results of each module are pushed to the alarm memory, and the subsequent alarm handling priority is adjusted based on the results.

10. The method of combining intelligent alarm judgment and security operation system according to claim 9, characterized in that, The method further includes a false alarm rate control mechanism: By adjusting the threshold H in the small model judgment module and the question template of the large model, the false negative rate and the false alarm rate are balanced; Regularly count the data in the alarm memory, and optimize the diversion strategy and model parameters of the classification device.

Images