A method to reduce the false alarm rate of IDPS

Abstract

This application provides a method for reducing the false positive rate of IDPS, comprising the following steps: setting alarm filtering rules corresponding to attack detection rules, wherein the alarm filtering rules include filter instrumentation keywords, the filter instrumentation keywords are used to set filter instrumentation on the flow management handle, and the filter instrumentation records the filtering object and lifecycle of the filter instrumentation; when a data flow matches the alarm filtering rule, setting filter instrumentation on the flow management handle of the data flow, recording the attack detection rule corresponding to the alarm filtering rule and the lifecycle of the filter instrumentation in the filter instrumentation, monitoring the lifecycle of the filter instrumentation, and destroying filter instrumentation that is not within its lifecycle; matching the data flow with the attack detection rule, if the match is successful and a filter instrumentation exists, determining it as non-attack behavior; if no filter instrumentation exists, determining it as attack behavior. This application effectively reduces the false positive rate and improves flexibility.

Description

Technical Field

[0001] This application relates to the field of information security technology, specifically to a method for reducing the false alarm rate of intrusion detection and prevention systems. Background Technology

[0002] IDPS (Intrusion Detection and Prevention Service) comprises IDS (Intrusion Detection System) and IPS (Intrusion Prevention System). IDS, based on specific security policies, monitors the operation of networks and systems through software and hardware, detecting various attack attempts, behaviors, or results to ensure the confidentiality, integrity, and availability of network resources. IDS is typically deployed in a bypass (parallel) mode, replicating data to the IDS via port mirroring for packet sniffing and detection. IDS primarily functions as an observer and monitor, issuing alerts. Network administrators can use IDS warnings to address network security issues, facilitating security statistics and post-incident tracing, and providing valuable clues for tracing and preventing denial-of-service attacks. IPS, located between firewalls and network devices, inspects incoming data packets to determine their true purpose. If an attack is detected, IPS will block malicious communication before it spreads to other parts of the network. An IPS (Intrusion Prevention System) is directly embedded in network traffic to proactively intercept intrusion activities and malicious network traffic, preventing them from causing damage. It receives traffic from external systems through one network port, checks it to confirm it contains no abnormal activity or suspicious content, and then forwards it to the internal system through another port. In this way, problematic packets, as well as all subsequent packets from the same data stream, can be removed by the IPS.

[0003] Based on the working principles of IDS and IPS, it's clear that the false positive rate (PPR) for attack detection alerts is a key indicator of IDS detection capabilities. A large number of false positive alerts significantly impairs the efficiency of IDS and IPS; therefore, PPR control has always been a crucial research direction for IDS. Traditional PPR control methods primarily rely on the content of network packets corresponding to attack behaviors. By optimizing attack detection rules, they aim to improve the accuracy of attack behavior judgment, thereby achieving precise detection and reducing the false positive rate. However, this method is limited by the characterization capabilities of rule definition languages ​​(such as Snort rules and Suricata rules). With the emergence of various evasion detection methods, simply relying on precisely defined attack detection rules is no longer sufficient to meet the demands of accurate detection. Therefore, various PPR control methods have emerged. For example, Suricata provides a method for filtering alarm events based on IP and port, but this method only supports IP and port as filtering conditions and does not support other specified conditions, resulting in a limited scope of operation. Additionally, patent application CN201710968803.X proposes a method to control false alarms by analyzing the IPs after an alarm. This involves manually or automatically analyzing alarm data to determine which IPs are false alarms, but this filtering method also only supports filtering specific IPs and does not support other specified conditions. The aforementioned prior art methods for controlling the false alarm rate define rules outside of attack detection rules. A set of filtering rules, limited by their common attributes, cannot be adjusted using special keywords in the definition of attack detection rules, resulting in a narrow application scope, low flexibility, and insufficient operability. Furthermore, patent application CN202111513344.9 proposes a method for reverse optimization and pruning of detection rules after an attack alert is generated. This method involves defining attack methods and then modifying them in reverse. However, this method relies on modifying the original model, requiring modification and re-issuance, which is complex and can affect system stability. Summary of the Invention

[0004] To address the aforementioned problems, this application provides a method for reducing the false alarm rate of IDPS, comprising the following steps:

[0005] S1, set alarm filtering rules corresponding to attack detection rules. The alarm filtering rules can filter false alarms that trigger the attack detection rules according to filtering conditions. The alarm filtering rules include filter instrumentation keywords. The filter instrumentation keywords are used to set filter instrumentation on the flow management handle. The filter instrumentation records the filtering object and the lifecycle of the filter instrumentation. The filtering object of the filter instrumentation is the corresponding attack detection rule, and the lifecycle of the filter instrumentation is the data unit of the detection content of the attack detection rule.

[0006] S2, when the data stream matches the alarm filtering rule, proceed to step S3;

[0007] If the data stream does not match the alarm filtering rule, proceed to step S4;

[0008] S3. Based on the filter instrumentation keyword, set a filter instrumentation on the flow management handle of the data stream, record the filter object and the life cycle of the filter instrumentation in the filter instrumentation, monitor the life cycle of the filter instrumentation, and destroy the filter instrumentation that is not in the life cycle.

[0009] The data stream is matched against the attack detection rule. If a match is found, proceed to step S31.

[0010] When a match fails, it is determined to be a non-attack behavior, and the warning generated by the attack detection rules is filtered out.

[0011] S4, match the data stream with the attack detection rule;

[0012] When a match is successful, it is determined to be an attack, and the warning generated by the attack detection rules is reported.

[0013] When a match fails, it is determined to be a non-attack behavior, and the warning generated by the attack detection rules is filtered out.

[0014] S31, When there is a filter instrumentation on the flow management handle that matches the attack detection rule, it is determined to be a non-attack behavior, and the warning generated by the attack detection rule is filtered out.

[0015] If no filter instrument matching the attack detection rule exists on the flow management handle, it is determined to be an attack, and the warning generated by the attack detection rule is reported.

[0016] In step S1, the data stream is TCP / UDP network traffic data, and the stream management handle refers to the handle used by the detection engine to manage resources between the TCP three-way handshake and the four-way handshake, or between the two parties exchanging information and disconnecting in UDP.

[0017] The attack detection rules and the alarm filtering rules are a set of configuration files. When performing traffic security detection on IDPS, the flow management handle reads the configuration files.

[0018] The lifecycle of the filter instrument includes single packet, session, and rectification. When the lifecycle of the filter instrument is single packet, it represents filtering a single traffic packet data frame. When the lifecycle of the filter instrument is session, it represents filtering request messages and corresponding response messages in the application layer session. When the lifecycle of the filter instrument is rectification, it represents filtering alarms generated by the entire data stream.

[0019] The alarm filtering rules are set by adding the keyword "eventfilter" to the intrusion detection engine Suricata.

[0020] The syntax for the filter instrumentation keyword `eventfilter` is set as follows:

[0021] eventfilter:type[packet|tx|flow],gen_id <value>,say_id <value>,tenant_id <value>;

[0022] The `type` option indicates the lifecycle of the filter instrumentation. `packet` indicates that the filter instrumentation's lifecycle is a single packet, `tx` indicates that the filter instrumentation's lifecycle is a session, and `flow` indicates that the filter instrumentation's lifecycle is a rectified flow. `gen_id`, `sig_id`, and `tenant` represent the group ID, attack detection rule ID, and tenant ID of the attack detection rule that the filter instrumentation needs to record.

[0023] Among them, the priority of setting alarm filtering rules is higher than the priority of attack detection rules.

[0024] In the case where the lifecycle of the filter instrumentation is a session, the filtering conditions of the alarm filtering rule are defined in the response message, and the alarm conditions of the attack detection rule are defined in the request message, the method to set the priority of the alarm filtering rule to be higher than the priority of the attack detection rule is to set the engine to execute the detection of the alarm filtering rule in advance or the detection of the attack detection rule in a delayed manner.

[0025] The filtering conditions of the alarm filtering rule can be set to all conditions supported by the attack detection rule.

[0026] The alarm filtering rules can be used to filter false alarms from a single attack detection rule or to filter false alarms from all attack detection rules in the same group.

[0027] The beneficial effects achieved by this application are as follows:

[0028] The false positive control method is simple to implement, requiring only the addition of a corresponding alarm filtering rule to the attack detection rules. It supports all conditions supported by the open-source rules. It is easy to implement, as the proposed alarm filtering rule is completely decoupled from the attack detection rule, requiring no changes to the detection engine's implementation logic. It is flexible, with proposed instrumentation lifecycle options allowing rule definers to flexibly specify the scope of influence of alarm filtering rules based on different actual attack scenarios, improving accuracy. It is highly adaptable, applicable not only to false positive control of a single attack detection rule but also to recording when an alarm filtering rule is hit during instrumentation. It allows for multi-dimensional and multi-faceted filtering of various alarms, making it suitable for multiple user scenarios and highly scalable. Attached Figure Description

[0029] To more clearly illustrate the present invention, the accompanying drawings will be described and explained below. Obviously, the drawings described below only illustrate certain aspects of some exemplary embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0030] Figure 1 This is a flowchart illustrating the method for filtering and judging warnings from attack detection rules in this application. Detailed Implementation

[0031] The following is for reference only. Figure 1 Various exemplary embodiments of this disclosure are described in detail. The descriptions of exemplary embodiments are merely illustrative and are in no way intended to limit this disclosure or its application or use. This disclosure may be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided so that this disclosure will be thorough and complete, and will fully express the scope of this disclosure to those skilled in the art. It should be noted that, unless otherwise stated, the relative arrangement of components and steps, numerical expressions, and values ​​set forth in these embodiments should be interpreted as merely exemplary and not as limiting.

[0032] like Figure 1 As shown, this application provides a method to reduce the false alarm rate of IDPS. The design idea is that, considering the ability to control false alarms of events by combining traffic context content characteristics, we pre-establish warning filtering rules corresponding to attack detection rules to filter false alarms of attack detection rules. The filtering conditions are set by setting instrumentation keywords in the warning filtering rules so that when network traffic matches the warning filtering rules, the warning of attack detection rules is filtered and judged by instrumentation in the flow management handle.

[0033] Specifically, the network traffic refers to TCP / UDP data streams. The flow management handle is the handle used by the detection engine to manage resources between the TCP three-way handshake and the four-way handshake, or between the two parties exchanging information and disconnecting in UDP. Attack rules and filtering rules are a set of configuration files. When the flow management handle manages the network traffic data stream, it reads the attack detection rules and alarm filtering rules.

[0034] The specific implementation method is as follows: set alarm filtering rules corresponding to attack detection rules. The alarm filtering rules can filter alarms generated by attack detection rules. The alarm filtering rules include instrumentation keywords. The filtering instrumentation and the content of the filtering instrumentation record are set through the instrumentation keywords. The content of the filtering instrumentation record is the object (attack detection rule) filtered by the alarm filtering rules and the life cycle of the filtering instrumentation. The data unit of the detection content of the alarm conditions in the attack detection rules is used as the instrumentation life cycle.

[0035] When a data stream matches an alarm filtering rule, an instrument is inserted into the stream management handle of that data stream. The instrument records information about the attack detection rule filtered by that alarm filtering rule and its lifecycle. The lifecycle of the instrument is monitored in the stream management handle, and instrumentations whose lifecycles have ended are destroyed. The data stream is then matched against the attack detection rule again. If the match fails, it is determined to be a non-attack behavior, and the warning generated by the attack detection rule is filtered. If the match succeeds, the stream management handle is checked to see if an instrument for that attack detection rule exists. If an instrument exists, the warning is filtered; otherwise, the warning is reported.

[0036] When network traffic does not match an alarm filtering rule, the data stream is matched with the attack detection rule filtered by that alarm filtering rule. If the match is successful, it is determined to be an attack, and the warning generated by the attack detection rule is reported. If the match fails, it is determined to be a non-attack, and the warning is filtered.

[0037] In the instrumentation keyword settings, the data unit of the detection content of the alarm conditions in the attack detection rules is used to filter the instrumentation lifecycle. Since instrumentation is based on the flow management resource handle, the basic data unit of the detection content managed by the flow management handle is a single packet, a session, and the entire flow. According to the attack detection rules targeted by the instrumentation, the data unit of the existing detection content of the alarm conditions in the attack detection rules is used as the lifecycle of the instrumentation. When the data unit of this detection content ends, the lifecycle of the instrumentation also ends, and the instrumentation is destroyed.

[0038] If the lifecycle of a filter instrument is a single packet, it means that a single traffic packet data frame is filtered. This is often used for filtering TCP, UDP, and other network layer alarms.

[0039] If the lifecycle of a filter instrument is a session, it represents the filtering of application layer sessions such as HTTP request messages and corresponding response messages. This relies on the application layer parsing of the engine and is also the lifecycle with the most application scenarios.

[0040] If the lifecycle of a filter instrument is rectified, it means that the specified alarms generated by the entire data stream are filtered, which has the widest impact and the longest lifecycle.

[0041] In addition, this application requires that instrumentation occur before alarms are triggered. That is, for network traffic, alarm filtering rules must be matched before attack detection rules are matched. Therefore, priority adjustment of alarm filtering rules is also required. The method is as follows:

[0042] First, the priority of alarm filtering rules needs to be set higher than that of attack detection rules. This ensures that, in the same network and detection environment, alarm filtering rules are detected before attack detection rules, and instrumentation precedes alarms.

[0043] Secondly, in scenarios where the filtering conditions for alarm filtering rules are defined in response messages, while the alarm conditions for attack detection rules are defined in request messages, the detection order may not meet expectations due to differences in message capture time. To ensure that instrumentation precedes attack detection, the detection of alarm filtering rules needs to be performed earlier or the detection of attack features needs to be delayed. This situation heavily relies on the session maintenance of the detection engine. Since this only occurs when the data unit of the attack detection rule's detection content is a session, i.e., the instrumentation's lifecycle is a session, we can only consider setting up detection in advance for alarm filtering rules with a session lifecycle in the system. Other lifecycle types do not need to be considered. By handling the difference in the direction of message requests and responses using the above method, and based on the priority of each rule, we can ensure that instrumentation precedes the detection of attack features.

[0044] Specifically, in one embodiment, the keyword "eventfilter" is added to the detection engine Suricata, with the following syntax:

[0045] eventfilter:type[packet|tx|flow],gen_id <value>,say_id <value>,tenant_id <value>

[0046] The `type` option is used to indicate the lifecycle of the filter instrumentation. `packet` indicates that the filter instrumentation lifecycle is a single traffic packet data frame, `tx` indicates that the filter instrumentation lifecycle is an application layer session, and `flow` indicates that the filter instrumentation lifecycle is the entire flow.

[0047] The `gen_id`, `sig_id`, and `tenant` options are used to indicate the attack detection rule information that needs to be recorded for the filtering instrumentation. `gen_id` corresponds to the group ID of the attack detection rule, `sig_id` corresponds to the ID of the attack detection rule, and `tenant` corresponds to the tenant ID of the attack detection rule. More dimensions of information can also be added and recorded depending on the use case.

[0048] Examples are given below:

[0049] There is an attack detection rule that uses directory traversal to access Linux system files, defined as follows:

[0050] alert http any any->any any(msg:"test";http.uri;content:".. / .. / etc / passwd";sid:100;gid:1;)

[0051] In this rule, the `alert` at the beginning of the rule header indicates that the rule type is an alert. `http any any->any any` indicates that it is from the IP port to the destination port. The `msg` keyword in the rule body indicates that the rule name is "test". The `http.` keyword specifies the URI of the detection point of the rule. The detection point has different settings depending on the protocol. The `content:` keyword sets the detection content. `sid:100; gid:1` means that the attack detection rule ID is 100. The group of the detection rule is 1. As shown in the above syntax example, this means: an attack detection rule with group 1 and ID 100. The specific policy content is to issue an alert when an HTTP request attempts to access sensitive files of the Linux system through directory traversal.

[0052] The above-mentioned rule, which can detect HTTP requests attempting to access sensitive files on a Linux system through directory traversal, will be directly identified by the web service in relatively well-protected web services because the attack method it corresponds to is quite rudimentary. The web service will then directly return a "404 Not Found" error to resolve this low-level security issue. If the attack detection rule still issues an alert for it, it will generate a large number of false alarms.

[0053] Therefore, based on the characteristics of the response codes returned by false alarm HTTP requests, the 404 status code can be used as a filtering condition to control false alarms.

[0054] Specifically, use the `eventfilter` keyword to add a filtering rule, with the following content:

[0055] alert http any any->any any(msg:"filter";http.method;content:"404";noalert;gid:1;sid:200;

[0056] eventfilter:tx,sig_id 100,gen_id 1;)

[0057] Similar to attack detection rules, the rule header information includes "alert" indicating that the rule type is alarm, "http anyany->any any" indicating that the rule is from the IP port to the destination port, and the "msg" keyword in the rule body information indicating that the alarm filtering rule name is "filter".

[0058] The keyword `http.` specifies the detection method, which is the response code. The keyword `content:` sets the detection content to "404". `noalert` means to cancel the alarm, i.e., to filter it. `gid:1; sid:200` means that the alarm filtering rule ID is 200 and the group ID is 1.

[0059] Therefore, the above statement means: when a 404 response code is detected, the attack detection rules are filtered.

[0060] In this embodiment, the native Suricata keyword is used to define the filtering conditions: http.method; content:"404"; specifies that the filtering condition is that the HTTP response code contains the character content "404". eventfilter:tx,sig_id 100,gen_id 1; indicates that if the feature is matched, instrumentation will be performed on the stream management handle. The lifespan of the instrumentation is the application layer session. If the lifespan is too short, it will not be able to associate the request message and response message. If the lifespan is too long, the detected response content will not be the expected response data of the request message.

[0061] Instrumentation records information about attack detection rules for group ID 1 and rule ID 100. When a directory traversal rule is hit, the information of the filter instrumentation on the flow management handle is checked. If a filter instrumentation already exists for the attack detection rule for group ID 1 and rule ID 100, no directory traversal log alarm is generated, thereby achieving the purpose of false alarm control.

[0062] The alarm filtering rules can be applied to a single attack rule or to a batch of attack categories within the same group.

[0063] As used in this disclosure, the words "including" or "contains" or similar terms mean that the element preceding the word covers the element listed after the word, and do not exclude the possibility that it may also cover other elements.

[0064] All terms used in this disclosure (including technical or scientific terms) have the same meaning as understood by one of ordinary skill in the art to which this disclosure pertains, unless otherwise specifically defined. It should also be understood that terms defined in general dictionaries should be understood to have meanings consistent with their meanings in the context of the relevant art, and not to be interpreted with idealized or highly formalized meanings, unless expressly defined herein.

[0065] For components, specific model numbers and other parameters of components not described in detail in this section, the interrelationships between components and control circuits, these may be considered as techniques, methods and devices known to those skilled in the art, but where appropriate, such techniques, methods and devices should be considered part of the specification.

[0066] It should be understood that the specific embodiments described above are only used to explain the present invention, and the scope of protection of the present invention is not limited thereto. Any modifications, substitutions, or combinations made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.< / value> < / value> < / value> < / value> < / value> < / value>

Claims

1. A method for reducing the false alarm rate of IDPS, characterized in that, Includes the following steps: S1, set alarm filtering rules corresponding to attack detection rules. The alarm filtering rules can filter false alarms that trigger the attack detection rules according to filtering conditions. The alarm filtering rules include filter instrumentation keywords. The filter instrumentation keywords are used to set filter instrumentation on the flow management handle. The filter instrumentation records the filtering object and the lifecycle of the filter instrumentation. The filtering object of the filter instrumentation is the corresponding attack detection rule, and the lifecycle of the filter instrumentation is the data unit of the detection content of the attack detection rule. S2, when the data stream matches the alarm filtering rule, proceed to step S3; If the data stream does not match the alarm filtering rule, proceed to step S4; S3. Based on the filter instrumentation keyword, set a filter instrumentation on the flow management handle of the data stream, record the filter object and the life cycle of the filter instrumentation in the filter instrumentation, monitor the life cycle of the filter instrumentation, and destroy the filter instrumentation that is not in the life cycle. The data stream is matched against the attack detection rule. If a match is found, proceed to step S31. When a match fails, it is determined to be a non-attack behavior, and the warning generated by the attack detection rules is filtered out. S4, match the data stream with the attack detection rule; When a match is successful, it is determined to be an attack, and the warning generated by the attack detection rules is reported. When a match fails, it is determined to be a non-attack behavior, and the warning generated by the attack detection rules is filtered out. S31, When there is a filter instrumentation on the flow management handle that matches the attack detection rule, it is determined to be a non-attack behavior, and the warning generated by the attack detection rule is filtered out. If no filter instrument matching the attack detection rule exists on the flow management handle, it is determined to be an attack, and the warning generated by the attack detection rule is reported.

2. The method for reducing the false alarm rate of IDPS as described in claim 1, characterized in that, In step S1, the data stream is TCP / UDP network traffic data, and the stream management handle refers to the handle used by the detection engine to manage resources between the TCP three-way handshake and the four-way handshake or between the two parties in UDP exchanging information and disconnecting.

3. The method for reducing the false alarm rate of IDPS as described in claim 1, characterized in that, The attack detection rules and the alarm filtering rules are a set of configuration files. When performing traffic security detection on IDPS, the flow management handle reads the configuration files.

4. The method for reducing the false alarm rate of IDPS as described in claim 1, characterized in that, The lifecycle of the filter instrument includes single packet, session, and rectification. When the lifecycle of the filter instrument is single packet, it represents filtering a single traffic packet data frame. When the lifecycle of the filter instrument is session, it represents filtering request messages and corresponding response messages in the application layer session. When the lifecycle of the filter instrument is rectification, it represents filtering alarms generated by the entire data stream.

5. The method for reducing the false alarm rate of IDPS as described in claim 4, characterized in that, The alarm filtering rules are set by adding the keyword "eventfilter" to the intrusion detection engine Suricata.

6. The method for reducing the false alarm rate of IDPS as described in claim 5, characterized in that, The syntax for the filter instrumentation keyword `eventfilter` is set as follows: eventfilter:type[packet|tx|flow],gen_id <value>,say_id <value>,tenant_id <value> ；< / value> < / value> < / value> The `type` option indicates the lifecycle of the filter instrumentation. `packet` indicates that the filter instrumentation's lifecycle is a single packet, `tx` indicates that the filter instrumentation's lifecycle is a session, and `flow` indicates that the filter instrumentation's lifecycle is a rectified flow. `gen_id`, `sig_id`, and `tenant` represent the group ID, attack detection rule ID, and tenant ID of the attack detection rule that the filter instrumentation needs to record.

7. The method for reducing the false alarm rate of IDPS as described in claim 1, characterized in that, Alarm filtering rules have a higher priority than attack detection rules.

8. The method for reducing the false alarm rate of IDPS as described in claim 7, characterized in that, When the lifecycle of a filter instrumentation is a session, the filtering conditions of an alarm filtering rule are defined in the response message, while the alarm conditions of an attack detection rule are defined in the request message, the method to set the priority of the alarm filtering rule to be higher than that of the attack detection rule is to configure the engine to execute the detection of the alarm filtering rule earlier or the detection of the attack detection rule later.

9. The method for reducing the false alarm rate of IDPS as described in claim 1, characterized in that, The filtering conditions of the alarm filtering rules can be set to all conditions supported by the attack detection rules.

10. The method for reducing the false alarm rate of IDPS as described in claim 1, characterized in that, The alarm filtering rules can be used to filter false alarms from a single attack detection rule or to filter false alarms from all attack detection rules in the same group.

Images