A method and system for matching ten million rules based on classification preprocessing

Abstract

This invention proposes a method and system for matching tens of millions of rules based on classification preprocessing. It belongs to the field of packet filtering in data stream processing. The method includes: firstly, preprocessing the rule set by classifying and statistically analyzing the protocol information carried by the rules; introducing the concept of "discrimination" and determining the main protocol category of each rule based on protocol layer priority (application layer > transport layer > network layer) and the number of associated rules, and constructing rule subsets with multiple protocol dimensions accordingly; during real-time traffic matching, parsing and tagging the packet protocol, dynamically selecting and matching only the rule subset corresponding to the protocol category based on the packet tags, and finally matching a general rule set containing only five-tuples. This invention combines static rule statistics with dynamic packet features to establish a heuristic decision-making mechanism, effectively reducing the number of rules that need to be traversed in a single match, thus achieving efficient multi-rule hit matching with a rule capacity of tens of millions, and optimizing overall performance.

Description

Technical Field

[0001] This invention relates to the fields of network security and data stream processing technology, and in particular to a method and system for matching tens of millions of rules based on classification preprocessing. Background Technology

[0002] With the rapid development of internet technology, network traffic is growing exponentially, and network attack methods are becoming increasingly complex. In the field of network security, packet filtering and access control lists (ACLs) are core foundational technologies. However, traditional technologies are gradually revealing their limitations when facing the demands of modern large-scale, in-depth data stream processing:

[0003] 1. Limited Dimensions: Traditional switch and router ACLs typically only support filtering based on the five-tuple (source IP, destination IP, source port, destination port, protocol number) of IP or IP network segments, and cannot deeply support filtering of TCP state and application layer (such as HTTP, DNS, etc.) information.

[0004] 2. The conflict between performance and capacity:

[0005] While dimensional matching-based methods (such as decision tree algorithms) are fast, they typically require finding the intersection of results for each dimension. Memory consumption increases dramatically with the number of rule dimensions, and they can usually only support a capacity of up to 100,000 rules.

[0006] While spatial partitioning-based methods have no limit on rule capacity, they require extremely long pre-compilation time and consume a large amount of memory during runtime, making them unsuitable for scenarios with frequent rule updates.

[0007] 3. Limited output results: Common packet filtering algorithms usually stop after finding the first match (FirstMatch), while data stream processing products (such as auditing systems and advanced threat detection) usually need to find all rules or the top N rules that meet the conditions (Multi-Match / Top-N).

[0008] 4. Challenges in rule scale: Modern security gateways often need to support rule bases of millions or even tens of millions. Existing matching algorithms struggle to simultaneously meet the requirements of low memory consumption, low matching latency, and full matching under such a massive rule volume.

[0009] Therefore, there is an urgent need for a new rule matching mechanism that can optimize the matching path through preprocessing to achieve efficient multi-rule hit matching, while supporting tens of millions of rules. Summary of the Invention

[0010] To address the problems of limited rule capacity, low efficiency of multi-dimensional matching, and difficulty in supporting full matching results in the existing technologies, this invention proposes a matching method for tens of millions of rules based on classification preprocessing.

[0011] The basic idea of ​​this invention is to abandon the traditional linear or tree-based search based solely on 5-tuples, and instead combine the static statistical characteristics of rule sets with the dynamic protocol features of packets. By selecting the matching dimension with the optimal "discrimination" to establish a rule subset, during traffic matching, the protocol tags parsed from the packets are used as indexes to quickly locate and match only the relevant rule subsets, thereby significantly reducing the search space for a single match.

[0012] The first aspect of this invention discloses a method for matching tens of millions of rules based on classification preprocessing; the method includes:

[0013] Step S1: Rule Preprocessing and Statistics

[0014] Traverse each rule in the initial rule set and extract the protocol element information contained in the rule body; determine the protocol category to which the protocol element information belongs based on the preset protocol information database, and count the number of rules associated with each protocol category;

[0015] Step S2: Calculate rule identification degree and determine the main protocol category

[0016] Based on the priority of the protocol hierarchy and the number of rules associated with the protocol category, the distinctiveness of each rule in its respective protocol category is determined; the protocol category with the highest distinctiveness is selected as the main protocol category of the rule.

[0017] Step S3: Construct a subset of rules

[0018] Based on the determined main protocol category, rules are assigned to the corresponding protocol category rule subsets, forming multiple rule subsets based on protocol dimensions; rules that do not contain specific protocol element information are classified into a general rule subset containing only quintuple information.

[0019] Step S4: Message parsing and tagging

[0020] Receive network data packets, parse them layer by layer according to the protocol, and tag the packets with corresponding protocol labels based on the parsing results. The protocol labels identify one or more protocol categories to which the packets belong.

[0021] Step S5: Selective Matching

[0022] Based on the protocol tag carried in the message, the rule subset corresponding to the protocol category is dynamically selected for matching; if the message does not carry a tag for a certain protocol category, the rule subset corresponding to that protocol category is skipped directly; finally, the general rule subset is matched, and all matching rule information is output.

[0023] Preferably, in step S2, the criterion for determining the level of recognition is:

[0024] First, it is determined based on the priority of the protocol layer. The higher the protocol layer, the more easily it can be identified.

[0025] When the protocol levels are the same, the fewer the number of associated rules a protocol category has, the higher its recognizability.

[0026] Preferably, the priority order of the protocol layers from high to low is defined as: application layer, transport layer, network layer, and network interface layer.

[0027] Preferably, the specific process for determining the main protocol category in step S2 is as follows:

[0028] For a rule that belongs to multiple protocol categories, first filter out all protocol categories at the highest protocol level;

[0029] If there are multiple protocol categories under the highest protocol level, then compare the number of global rules associated with these protocol categories and select the protocol category with the fewest associated rules as the main protocol category of that rule.

[0030] Preferably, in step S5, the specific execution order of the matching is as follows:

[0031] Prioritize matching the corresponding subset of application layer protocol rules based on the application layer protocol tag of the message;

[0032] Next, based on the transport layer protocol tag of the message, match the corresponding transport layer protocol rule subset;

[0033] Next, match the corresponding subset of network layer protocol rules based on the network layer protocol tag of the message;

[0034] Finally, based on the network interface layer protocol tag of the message, match the corresponding subset of network interface layer protocol rules;

[0035] After completing the above matching, the general rule subset containing only quintuple information is then matched.

[0036] Preferably, the protocol information database contains a mapping relationship between protocol element aliases and standard protocol categories; in step S1, by searching the protocol information database, different forms of protocol element aliases in the rule body are uniformly mapped to their respective standard protocol categories.

[0037] Preferably, the rule header is fixed as IP 5-tuple information as the basic matching dimension; when constructing rule subsets, the accuracy of the 5-tuple information is not used as the basis for division, but the protocol elements in the rule body are used as the basis for dimension division.

[0038] A second aspect of this invention discloses a matching system for tens of millions of rules based on classification preprocessing; the system employs the method described in any one of the first aspects, and the system includes:

[0039] The preprocessing module is used to traverse the rule set, count the number of rules for each protocol dimension, calculate the rule identification degree based on the protocol level and the number of rules, and divide the rules into the rule subset corresponding to the main protocol category with the highest identification degree.

[0040] The message parsing module is used to parse network messages in real time, identify the protocol level to which the message belongs, and tag the message with the corresponding protocol.

[0041] The matching engine module is used to filter out a subset of rules to be matched based on the protocol tags of the message, and perform matching operations in sequence according to the protocol hierarchy priority, and output the hit results.

[0042] A third aspect of this invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of a method for matching tens of millions of rules based on classification preprocessing, as described in any of the first aspects of this disclosure.

[0043] A fourth aspect of this invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for matching tens of millions of rules based on classification preprocessing, as described in any of the first aspects of this disclosure.

[0044] The beneficial effects of this invention are as follows:

[0045] 1. Macro-level performance optimization: By establishing a heuristic decision-making mechanism based on "discrimination," the large-scale matching problem involving tens of millions of messages is decomposed into several smaller subset matching problems. For a specific message, only a subset of rules related to its protocol needs to be matched, greatly reducing invalid matching calculations.

[0046] 2. Balancing capacity and efficiency: By preprocessing the classification, the algorithm keeps memory consumption and matching latency within a controllable range when dealing with tens of millions of rules, solving the pain point of traditional algorithms that struggle to balance "large capacity" and "high performance".

[0047] 3. Supports multiple result matching: The architecture of this method naturally supports traversing all relevant subsets, thereby outputting all rules or the top N rules that meet the conditions, which meets the needs of deep data stream processing products.

[0048] 4. Dynamic adaptability: By combining static rule statistics (number of rules) with dynamic message characteristics (protocol type), the matching path is optimal for each message type. Attached Figure Description

[0049] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0050] Figure 1 This is a flowchart of a method for matching tens of millions of rules based on classification preprocessing according to an embodiment of the present invention;

[0051] Figure 2 This is a diagram illustrating the organizational structure of a rule set of tens of millions of levels based on classification preprocessing and the rule subsets after rule classification, according to the present invention.

[0052] Figure 3 This is a flowchart of the real-time message matching process in a method for matching tens of millions of rules based on classification preprocessing according to the present invention.

[0053] Figure 4 This is a structural diagram of a rule matching system with tens of millions of rules based on classification preprocessing according to an embodiment of the present invention;

[0054] Figure 5 This is a structural diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] The first aspect of this invention discloses a method for matching rules with tens of millions of records based on classification preprocessing. Figure 1 Here is a flowchart of a method for matching tens of millions of rules based on classification preprocessing according to an embodiment of the present invention, as shown below. Figure 1 and Figure 3 As shown, the method includes:

[0057] Step S1: Rule Preprocessing and Statistics

[0058] Traverse each rule in the initial rule set and extract the protocol element information contained in the rule body; determine the protocol category to which the protocol element information belongs based on the preset protocol information database, and count the number of rules associated with each protocol category;

[0059] In addition, the protocol information database contains a mapping relationship between protocol element aliases and standard protocol categories; in step S1, by searching the protocol information database, different forms of protocol element aliases in the rule body are uniformly mapped to their respective standard protocol categories.

[0060] The rule header is fixed as IP 5-tuple information, which serves as the basic matching dimension. When constructing rule subsets, the accuracy of the 5-tuple information is not used as the basis for division, but rather the protocol elements in the rule body are used as the basis for dimension division.

[0061] In some specific embodiments, such as Figure 2 As shown, this method first preprocesses the rule base provided by the user, either offline or online. Assume the rule base contains 10 million rules.

[0062] We divide the rules into "rule header" and "rule body".

[0063] Rule header: IP 5-tuple (source IP, destination IP, protocol, source port, destination port). This part is not used as the primary basis for classification, but rather as a basic attribute of the rules in each subset.

[0064] Rule body: Matching elements involving specific protocol content (such as http.uri, tls.sni, mysql.user, etc.).

[0065] Protocol information database: The system maintains a global table that records the correspondence between protocol element aliases and standard protocols. For example, http.host and http.uri both belong to the HTTP protocol; tcp.flags belongs to the TCP protocol.

[0066] Step S2: Calculate rule identification degree and determine the main protocol category

[0067] Based on the priority of the protocol hierarchy and the number of rules associated with the protocol category, the distinctiveness of each rule in its respective protocol category is determined; the protocol category with the highest distinctiveness is selected as the main protocol category of the rule.

[0068] In step S2, the criteria for determining the level of recognition are as follows:

[0069] First, it is determined based on the priority of the protocol layer. The higher the protocol layer, the more easily it can be identified.

[0070] When the protocol levels are the same, the fewer the number of associated rules a protocol category has, the higher its recognizability.

[0071] The priority order of the protocol layers, from highest to lowest, is defined as: application layer, transport layer, network layer, and network interface layer.

[0072] In some specific implementations, the system iterates through all rules. Suppose there is a rule R1, which states: "Allow 192.168.1.0 / 24 to access any destination IP, provided that tcp.port=80 and http.method=GET."

[0073] The protocols involved in rule R1 are TCP (transport layer) and HTTP (application layer).

[0074] The protocol layer priority is defined as: Application Layer > Transport Layer > Network Layer > Network Interface Layer.

[0075] Quantity statistics (hypothetical values): In the global rule base, there are 5 million rules involving TCP and 2 million rules involving HTTP.

[0076] The specific process for determining the main protocol category in step S2 is as follows:

[0077] For a rule that belongs to multiple protocol categories, first filter out all protocol categories at the highest protocol level;

[0078] If there are multiple protocol categories under the highest protocol level, then compare the number of global rules associated with these protocol categories and select the protocol category with the fewest associated rules as the main protocol category of that rule.

[0079] For rule R1:

[0080] First, consider the layer hierarchy: HTTP (application layer) has higher priority than TCP (transport layer). Therefore, HTTP should be considered first.

[0081] If the rules also involve another application layer protocol (such as a specific type of App protocol), and that App protocol only has 10,000 rules, then, in the case of both being application layers, the protocol with fewer rules (10,000 < 2 million) should be selected as the main protocol category.

[0082] In this example, R1's main protocol category is determined to be HTTP.

[0083] Step S3: Construct a subset of rules

[0084] Based on the determined main protocol category, rules are assigned to the corresponding protocol category rule subsets, forming multiple rule subsets based on protocol dimensions; rules that do not contain specific protocol element information are classified into a general rule subset containing only quintuple information.

[0085] Specifically, place rule R1 into the "HTTP Protocol Rules Subset". Similarly, assign all rules.

[0086] If a rule R2 contains only ip src 1.1.1.1 and has no rule body, it is placed into the "general quintuple rule subset".

[0087] If a rule R3 contains only tcp.flags=SYN, then its main protocol is TCP, and it is placed in the "TCP protocol rule subset".

[0088] In this way, 10 million rules are broken down into dozens or even hundreds of subsets of different protocol dimensions (such as HTTP subset, DNS subset, TCP subset, etc.), as well as a general subset.

[0089] Step S4: Message parsing and tagging

[0090] Receive network data packets, parse them layer by layer according to the protocol, and tag the packets with corresponding protocol labels based on the parsing results. The protocol labels identify one or more protocol categories to which the packets belong.

[0091] In some specific embodiments, the system receives a message, which is then parsed by a protocol identification engine (DPI).

[0092] Suppose we receive an HTTP GET request message.

[0093] The parsing results include:

[0094] Network layer: IP

[0095] Transport layer: TCP

[0096] Application layer: HTTP

[0097] The system labels the message as: [IP, TCP, HTTP].

[0098] Step S5: Selective Matching

[0099] Based on the protocol tag carried in the message, the rule subset corresponding to the protocol category is dynamically selected for matching; if the message does not carry a tag for a certain protocol category, the rule subset corresponding to that protocol category is skipped directly; finally, the general rule subset is matched, and all matching rule information is output.

[0100] In step S5, the specific execution order of the matching is as follows:

[0101] Prioritize matching the corresponding subset of application layer protocol rules based on the application layer protocol tag of the message;

[0102] Next, based on the transport layer protocol tag of the message, match the corresponding transport layer protocol rule subset;

[0103] Next, match the corresponding subset of network layer protocol rules based on the network layer protocol tag of the message;

[0104] Finally, based on the network interface layer protocol tag of the message, match the corresponding subset of network interface layer protocol rules;

[0105] After completing the above matching, the general rule subset containing only quintuple information is then matched.

[0106] In some specific embodiments, the matching engine schedules subset matching according to the tags and in order of priority (application layer > transport layer > network layer > ...):

[0107] Check the application layer: the message contains an HTTP tag.

[0108] Action: Load the "HTTP protocol rule subset" for matching.

[0109] Result: In this subset, rule R1 mentioned above may be matched. Simultaneously, the system automatically skips matching other application-layer subsets such as DNS, FTP, and SMTP, saving significant computing resources.

[0110] Check the transport layer: the message has a TCP tag.

[0111] Action: Load the "TCP protocol rule subset" for matching.

[0112] Effect: Matches rules that only care about TCP flags or ports but not the specific HTTP content (such as rule R3).

[0113] Check the network layer: the packet has an IP label.

[0114] Action: Load the "Network Layer Protocol Rule Subset" (if there are specific IP protocol option matching rules).

[0115] General matching: Finally, always match the "general quintuple rule subset".

[0116] Effect: Matches pure IP rules like rule R2.

[0117] The system summarizes all the rules matched in the above steps. Since data stream processing products may require "first N" or "all" matching results, this method naturally supports merging the matching results of each subset in the output, rather than stopping after the first rule is matched.

[0118] Advantages Analysis

[0119] Assume it is a non-HTTP UDP message (such as a DNS query).

[0120] Resolve tags: [IP, UDP, DNS].

[0121] Matching path:

[0122] Match a subset of DNS rules (skip millions of rules such as HTTP and SMTP).

[0123] Match the UDP rule subset (skip the TCP rule subset).

[0124] Matches a general subset.

[0125] This "on-demand matching" strategy ensures that the actual number of rules to be scanned for any given message type is far less than the total rule base of 10 million, resulting in extremely high matching efficiency on a macro level. Even in extreme cases (such as when a large number of rules are concentrated in HTTP), the system tends to utilize finer-grained features (if they exist) to further segment the traffic due to the introduction of "number of associated rules" as a distinguishing factor, or it may degrade to normal efficiency, but it will not be lower than traditional methods.

[0126] In summary, the method of this invention, based on a matching method for tens of millions of rules after classification preprocessing, transforms the existing method of first filtering the rule set based on five-tuple information and then performing a secondary filtering. It combines the static statistical characteristics of the rule set (the number of rules in each protocol dimension) with the dynamic characteristics of the message (the protocol parsed from the message), establishing a heuristic decision-making mechanism based on "discrimination" to dynamically select the most efficient matching path at runtime, thereby macroscopically optimizing the overall matching performance of tens of millions of rules. Specifically, it includes:

[0127] Categorization and Statistics: Categorization and statistics are performed based on the protocol information carried in the rule information. Matching dimensions are defined to establish sub-rule sets. The rule header is fixed as the IP 5-tuple information, while the rule body focuses on the portion containing protocol element information, treating each protocol as a separate matching dimension.

[0128] Constructing a highly identifiable subset: Introducing the concept of rule "identifiability". Identifiability depends on two dimensions: first, the priority of the protocol layer (application layer > transport layer > network layer > network interface layer); and second, the number of rules associated with that protocol layer. Identify the protocol with the fewest associated rules at the highest protocol layer in the protocol information carried by the rule, and define this as the rule's "main protocol category".

[0129] Rule distribution: Based on the determined main protocol category, rules are placed into the rule subset corresponding to that protocol category. Rules without any protocol element information are placed into the general rule subset containing only 5-tuples.

[0130] Optimal path matching: During real-time traffic processing, packets are parsed and tagged using multiple protocols. The matching engine matches the corresponding rule subsets according to the protocol level, from highest to lowest, based on the tags carried in the packets. If a packet does not have a tag for a certain protocol, the rule subset corresponding to that protocol is skipped. Finally, a general rule subset matching is performed.

[0131] The second aspect of the present invention discloses a matching system for tens of millions of rules based on classification preprocessing, wherein the system employs the matching method for tens of millions of rules based on classification preprocessing as described in any one of the first aspects. Figure 4 This is a structural diagram of a rule matching system with tens of millions of rules based on classification preprocessing according to an embodiment of the present invention; as shown below. Figure 4 As shown, the system 100 includes:

[0132] The preprocessing module 101 is used to traverse the rule set, count the number of rules in each protocol dimension, calculate the rule identification degree based on the protocol level and the number of rules, and divide the rules into the rule subset corresponding to the main protocol category with the highest identification degree.

[0133] The message parsing module 102 is used to parse network messages in real time, identify the protocol level to which the message belongs, and tag the message with the corresponding protocol tag.

[0134] The matching engine module 103 is used to filter out the subset of rules to be matched based on the protocol tags of the message, and to perform the matching operation in sequence according to the protocol hierarchy priority, and output the hit result.

[0135] A third aspect of this invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of a method for matching tens of millions of rules based on classification preprocessing, as described in any of the first aspects of this invention.

[0136] Figure 5 This is a structural diagram of an electronic device according to an embodiment of the present invention, such as... Figure 5As shown, the electronic device includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, Near Field Communication (NFC), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.

[0137] Those skilled in the art will understand that Figure 5 The structure shown is merely a structural diagram of the part related to the technical solution of this disclosure and does not constitute a limitation on the electronic device to which the solution of this application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0138] A fourth aspect of this invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for matching tens of millions of rules based on classification preprocessing, as described in any of the first aspects of this invention.

[0139] Please note that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. The above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be pointed out that for those skilled in the art, several modifications and improvements can be made without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for matching rules with tens of millions of records based on classification preprocessing, characterized in that, Includes the following steps: Step S1: Rule Preprocessing and Statistics Traverse each rule in the initial rule set and extract the protocol element information contained in the rule body; determine the protocol category to which the protocol element information belongs based on the preset protocol information database, and count the number of rules associated with each protocol category; Step S2: Calculate rule identification degree and determine the main protocol category Based on the priority of the protocol hierarchy and the number of rules associated with the protocol category, determine the identifiability of each rule in its respective protocol category; The protocol category with the highest recognizability is selected as the main protocol category for this rule; Step S3: Construct a subset of rules Based on the determined main protocol category, rules are assigned to the corresponding protocol category rule subsets, forming multiple rule subsets based on protocol dimensions; rules that do not contain specific protocol element information are classified into a general rule subset containing only quintuple information. Step S4: Message parsing and tagging Receive network data packets, parse them layer by layer according to the protocol, and tag the packets with corresponding protocol labels based on the parsing results. The protocol labels identify one or more protocol categories to which the packets belong. Step S5: Selective Matching Based on the protocol tag carried in the message, the rule subset corresponding to the protocol category is dynamically selected for matching; if the message does not carry a tag for a certain protocol category, the rule subset corresponding to that protocol category is skipped directly; finally, the general rule subset is matched, and all matching rule information is output. In step S2, the criteria for determining the level of recognition are as follows: First, it is determined based on the priority of the protocol layer. The higher the protocol layer, the more easily it can be identified. When the protocol levels are the same, the fewer the number of associated rules a protocol category has, the higher its recognizability.

2. The method according to claim 1, characterized in that, The priority order of the protocol layers, from highest to lowest, is defined as: application layer, transport layer, network layer, and network interface layer.

3. The method according to claim 1, characterized in that, The specific process for determining the main protocol category in step S2 is as follows: For a rule that belongs to multiple protocol categories, first filter out all protocol categories at the highest protocol level; If there are multiple protocol categories under the highest protocol level, then compare the number of global rules associated with these protocol categories and select the protocol category with the fewest associated rules as the main protocol category of that rule.

4. The method according to claim 1, characterized in that, In step S5, the specific execution order of the matching is as follows: Prioritize matching the corresponding subset of application layer protocol rules based on the application layer protocol tag of the message; Next, based on the transport layer protocol tag of the message, match the corresponding transport layer protocol rule subset; Next, match the corresponding subset of network layer protocol rules based on the network layer protocol tag of the message; Finally, based on the network interface layer protocol tag of the message, match the corresponding subset of network interface layer protocol rules; After completing the above matching, the general rule subset containing only quintuple information is then matched.

5. The method according to claim 1, characterized in that, The protocol information database contains a mapping relationship between protocol element aliases and standard protocol categories; in step S1, by searching the protocol information database, different forms of protocol element aliases in the rule body are uniformly mapped to their respective standard protocol categories.

6. The method according to claim 1, characterized in that, The rule header is fixed as IP 5-tuple information, which serves as the basic matching dimension. When constructing rule subsets, the accuracy of the 5-tuple information is not used as the basis for division, but rather the protocol elements in the rule body are used as the basis for dimension division.

7. A rule matching system for tens of millions of rules based on classification preprocessing, characterized in that, The system employs the method described in any one of claims 1-6, and the system comprises: The preprocessing module is used to traverse the rule set, count the number of rules for each protocol dimension, calculate the rule identification degree based on the protocol level and the number of rules, and divide the rules into the rule subset corresponding to the main protocol category with the highest identification degree. The message parsing module is used to parse network messages in real time, identify the protocol level to which the message belongs, and tag the message with the corresponding protocol. The matching engine module is used to filter out a subset of rules to be matched based on the protocol tags of the message, and perform matching operations in sequence according to the protocol hierarchy priority, and output the hit results.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements the steps of the method for matching tens of millions of rules based on classification preprocessing as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for matching tens of millions of rules based on classification preprocessing as described in any one of claims 1 to 6.

Images