Data detection method and device based on multi-mode matching, equipment and medium

Abstract

The application relates to a data detection method and device based on multi-mode matching, equipment and a medium. The method comprises the following steps: obtaining a to-be-detected data packet, extracting quintuple information of the to-be-detected data packet, determining a corresponding signature group from a preset signature group table based on the quintuple information, wherein the signature group is associated with a plurality of detection rules and a pre-filtering engine, performing fast multi-mode scanning on data content of the to-be-detected data packet based on the pre-filtering engine to obtain a preliminary screening result, if the preliminary screening result indicates that the data content comprises at least one mode feature, calling a corresponding multi-mode matching algorithm to perform accurate mode matching on the data content to obtain at least one matched mode identifier, obtaining a target detection rule associated with the at least one matched mode identifier from the plurality of detection rules, and performing rule matching based on the target detection rule to generate a data detection result.

Description

Technical Field

[0001] This application relates to the field of network security technology, and in particular to a data detection method, apparatus, device and medium based on multi-pattern matching. Background Technology

[0002] In the current field of network attack detection, feature-based malware identification methods are widely used. However, existing detection methods have several technical limitations. First, most detection systems lack a rule pre-grouping mechanism, requiring each data packet to be detected to be compared with all detection rules one by one. A large number of rules completely irrelevant to the current data packet participate in invalid matching, severely reducing detection efficiency. Second, existing solutions typically couple pattern matching with rule matching, lacking a phased pre-filtering step. That is, there is no screening process to quickly exclude data packets that do not contain any pattern features before precise matching, causing a large number of normal data packets to still bear the full cost of multi-pattern matching. Furthermore, existing methods often use a fixed multi-pattern matching algorithm and cannot dynamically adjust the matching strategy according to the characteristics of the current data packet or system load, resulting in insufficient adaptability to different network environments. Summary of the Invention

[0003] In view of the above, this application provides a data detection method, apparatus, device and storage medium based on multi-pattern matching, the purpose of which is to solve the above-mentioned technical problems.

[0004] Firstly, this application provides a data detection method based on multi-pattern matching, the method comprising: Obtain the data packet to be detected; Extract the five-tuple information of the data packet to be detected. The five-tuple information includes the source address, destination address, source port, destination port, and protocol type. Based on the quintuple information, the corresponding signature group is determined from the preset signature group table, wherein the signature group is associated with multiple detection rules and a pre-filtering engine; Based on the pre-filtering engine, the data content of the data packet to be detected is quickly scanned in multiple modes to obtain preliminary screening results; If the preliminary screening result indicates that the data content includes at least one pattern feature, then the corresponding multi-pattern matching algorithm is invoked to perform precise pattern matching on the data content to obtain at least one matching pattern identifier. Obtain the target detection rule associated with the at least one matching pattern identifier from the plurality of detection rules, and perform rule matching based on the target detection rule to generate data detection results; The preset signature grouping table is obtained in the following way: Obtain the pre-configured full set of detection rules and parse the quintuple constraints of each detection rule; Detection rules with the same quintuple constraints are grouped into the same signature group; A corresponding pre-filtering engine is established for each signature group. The pre-filtering engine is used to perform a fast multi-mode scan on the data content of the data packet to exclude data packets that do not contain any pattern features. The completed signature groups are stored in the preset signature group table.

[0005] In some embodiments, the corresponding multi-pattern matching algorithm is determined in the following manner: Detect the hardware characteristics of the current deployment environment, including CPU architecture type and compilation configuration parameters; Based on the hardware characteristics, a multi-pattern matching algorithm matching the hardware characteristics is determined from a preset algorithm registry.

[0006] In some embodiments, determining a multi-pattern matching algorithm that matches the hardware feature from a preset algorithm registry based on the hardware feature includes: Based on the hardware characteristics, one or more preset multi-mode matching algorithms that match the hardware characteristics are determined from a preset algorithm registry. For each preset multi-pattern matching algorithm: Obtain the performance metrics of the detection threads after the one or more preset multi-pattern matching algorithms have been put into operation in history. The performance metrics include the average detection latency and throughput of data packets per unit time. The performance metrics are compared with a preset algorithm performance baseline; Select one of the preset multi-pattern matching algorithms as the matching algorithm from the preset multi-pattern matching algorithms where the average detection latency is lower than a first threshold and / or the throughput is higher than a second threshold.

[0007] In some embodiments, the step of performing rule matching based on the target detection rule to generate data detection results includes: The rule fields of the target detection rules are traversed in priority order. Perform a full rule match on each of the aforementioned rule fields; If a match is successful, a data detection result is generated according to the action type configured in the target detection rule. The action type includes alarm, discard, or pass.

[0008] In some embodiments, traversing the target detection rules in priority order includes: Obtain a historical hit counter for each rule field within the signature group; wherein, after each detection is completed, if the rule field matches successfully, the value of the historical hit counter is incremented; The rule fields within the signature group are reordered based on the value of the historical hit counter, and the rule fields with higher historical hit counts are moved to the front of the sorting. The rule fields of the target detection rule are traversed based on the reordered priority.

[0009] In some embodiments, traversing the target detection rules in priority order includes: Obtain the historical hit time series for each rule field within the signature group, wherein the historical hit time series records the timestamps of the most recent N successful matches of the rule field; The weighted score is determined by summing the weight values ​​corresponding to each successful match of each rule field within a preset time. The weight value of each successful match is determined based on the time difference between the time of the match and the current time. The smaller the time difference, the larger the weight value, and vice versa. The rule fields within the signature group are reordered according to the weighted score. The rule fields of the target detection rule are traversed based on the reordered priority.

[0010] Secondly, this application provides a data detection device based on multi-pattern matching, the data detection device based on multi-pattern matching comprising: The acquisition module is used to acquire the data packets to be detected; The extraction module is used to extract the five-tuple information of the data packet to be detected. The five-tuple information includes the source address, destination address, source port, destination port, and protocol type. The determination module is used to determine the corresponding signature group from a preset signature group table based on the five-tuple information, wherein the signature group is associated with multiple detection rules and a pre-filtering engine; The scanning module is used to perform a fast multi-mode scan of the data content of the data packet to be detected based on the pre-filtering engine to obtain preliminary screening results; The matching module is used to call the corresponding multi-pattern matching algorithm to perform precise pattern matching on the data content if the preliminary screening result indicates that the data content includes at least one pattern feature, so as to obtain at least one matching pattern identifier. The generation module is used to obtain the target detection rule associated with the at least one matching pattern identifier from the plurality of detection rules, and perform rule matching based on the target detection rule to generate data detection results; The preset signature grouping table is obtained in the following way: Obtain the pre-configured full set of detection rules and parse the quintuple constraints of each detection rule; Detection rules with the same quintuple constraints are grouped into the same signature group; A corresponding pre-filtering engine is established for each signature group. The pre-filtering engine is used to perform a fast multi-mode scan on the data content of the data packet to exclude data packets that do not contain any pattern features. The completed signature groups are stored in the preset signature group table.

[0011] Thirdly, this application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements the steps of the data detection method based on multi-pattern matching as described in any embodiment of the first aspect.

[0012] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the data detection method based on multi-pattern matching as described in any embodiment of the first aspect.

[0013] The technical solutions provided in this application have the following advantages compared with the prior art: (1) By first locating the signature group based on the five-tuple information, then using the pre-filtering engine within the group for fast multi-mode scanning, and finally calling precise multi-mode matching and executing rule matching only on the data packets that have passed the initial screening, the number of invalid rule matchings is significantly reduced. (2) This method reduces the average number of matching rules per data packet from the total number of rules to the number of groups that are only related to the current quintuple. Combined with the fast exclusion capability of pre-filtering, the overall detection throughput is effectively improved while maintaining the recall rate. (3) Since exact matching is only performed after the initial screening is passed, the overhead of multi-pattern matching algorithm to perform a complete scan of each data packet is avoided, effectively reducing CPU usage and achieving high-performance adaptive attack detection. Attached Figure Description

[0014] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating a preferred embodiment of the data detection method based on multi-pattern matching in this application; Figure 2 This is a schematic diagram of a preferred embodiment of the data detection device based on multi-pattern matching in this application; Figure 3 This is a schematic diagram of a preferred embodiment of the electronic device of this application; The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0018] It should be noted that the use of terms such as "first" and "second" in this application is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of those features. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed in this application.

[0019] Reference Figure 1 The diagram shown is a flowchart illustrating an embodiment of the data detection method based on multi-pattern matching according to this application. The method is executed by an electronic device, which can be implemented by a software system and / or a hardware system. The data detection method based on multi-pattern matching includes: Step 101: Obtain the data packet to be detected.

[0020] A data packet to be inspected refers to a raw network transmission unit captured through a network interface. A data packet to be inspected includes complete data from the link layer, network layer, transport layer, and application layer. For example, a data packet to be inspected can be a TCP segment carried by an Ethernet frame, or it can be a UDP datagram or an ICMP message.

[0021] In some embodiments, acquiring the data packet to be detected includes: reading the raw Ethernet frame from the network interface driver using a packet capture tool, storing the frame in a pre-allocated buffer, and recording the arrival timestamp and original length of the data packet.

[0022] Step 102: Extract the 5-tuple information of the data packet to be detected. The 5-tuple information includes the source address, destination address, source port, destination port, and protocol type.

[0023] The quintuple information is a combination of five key fields used to uniquely identify a network connection. The source and destination addresses are IP addresses, which can be IPv4 addresses, such as 192.168.1.100, or IPv6 addresses, such as 2001:db8::1; the source and destination ports are transport layer port numbers, ranging from 0 to 65535; and the protocol type indicates the transport layer protocol, such as TCP, UDP, or ICMP.

[0024] In some embodiments, extracting the five-tuple information of the data packet to be detected includes: first, parsing the type field in the Ethernet header to determine the network layer protocol, and then parsing the IP header to obtain the source IP address, destination IP address, and protocol field. Next, based on the protocol field value, if the value is 6 indicating TCP, the TCP header is parsed to obtain the source and destination ports; if the value is 17 indicating UDP, the UDP header is parsed to obtain the source and destination ports; and if the value is 1 indicating ICMP, the source and destination ports are filled with 0.

[0025] Step 103: Based on the quintuple information, determine the corresponding signature group from the preset signature group table, wherein the signature group is associated with multiple detection rules and pre-filtering engines.

[0026] The signature group table is a pre-built hash map data structure, where the key is a combination identifier of the five-tuple feature constraints, and the value is the corresponding signature group object.

[0027] A signature group is a logical unit that aggregates a set of detection rules with the same or similar 5-tuple constraints. Each signature group is associated with one or more detection rules and a pre-filtering engine.

[0028] For example, all rules targeting the HTTP protocol, with a destination port of 80 and a destination address belonging to the 192.168.1.0 network segment can be grouped into the same signature block.

[0029] Detection rules are logical expressions that define the behavioral characteristics of malware. Each rule includes at least one pattern feature, one quintuple constraint, and one action type.

[0030] The pre-filtering engine is a fast scanning component used to quickly exclude packets that obviously do not contain any pattern characteristics before deep detection.

[0031] In some embodiments, determining the corresponding signature packet from a preset signature packet table based on the 5-tuple information includes: combining the protocol type, source address subnet mask, destination address subnet mask, source port range, and destination port range from the extracted 5-tuple into a query key; calculating a hash value for the query key; and then using the hash value to search for the corresponding signature packet object in the signature packet table hash table. If the search is successful, the signature packet object is identified as the corresponding signature packet; if no match is found, a default empty packet is returned, indicating that no matching rule exists.

[0032] In some embodiments, before determining the corresponding signature group from the preset signature group table based on the five-tuple information, the preset signature group table can be obtained through the following operations: S11: Obtain the pre-configured full set of detection rules and parse the quintuple constraints of each detection rule.

[0033] Pre-configured full detection rules refer to the complete set of all attack detection rules pre-written and loaded into the detection engine by the security administrator. Each detection rule is a logical expression used to define the behavioral characteristics of malware or network attacks.

[0034] The five-tuple constraint refers to the network connection matching conditions specified in the detection rules, specifically including protocol type, source address range, destination address range, source port range, and destination port range.

[0035] In some embodiments, obtaining the pre-configured full set of detection rules and parsing the five-tuple constraints of each detection rule includes: during the system initialization phase, reading all rule entries from the rule base file; for each rule, using a rule parser to extract its five-tuple fields, specifically including parsing the protocol type field (e.g., TCP or UDP), the source address field (supporting a single IP or CIDR network segment), the destination address field, the source port field (supporting a single port or a port range), and the destination port field. The parsed five fields are then organized into a five-tuple constraint structure. For rules that do not explicitly specify a constraint, the constraint is set as a wildcard, indicating that any value is matched.

[0036] In some embodiments, obtaining rules may also include validating the rules, such as checking whether the IP address format is correct and whether the port number is in the range of 1 to 65535. Rules with incorrect formats are skipped and alarm logs are recorded.

[0037] S12 groups detection rules with the same quintuple constraints into the same signature group.

[0038] The same quintuple constraint means that two or more rules have the same values ​​in the five dimensions of protocol type, source address range, destination address range, source port range, and destination port range.

[0039] S13, establish a corresponding pre-filtering engine for each signature group. The pre-filtering engine is used to perform a fast multi-mode scan on the data content of the data packet to exclude data packets that do not contain any pattern features.

[0040] Data content refers to the application layer data content portion of a data packet after removing the headers of each protocol layer.

[0041] Fast multi-pattern scanning refers to the process of simultaneously searching for multiple predefined pattern features in a data buffer using a multi-pattern matching algorithm.

[0042] Pattern features refer to key strings, regular expressions, or byte sequences defined in the detection rules.

[0043] In some embodiments, for all detection rules within the current signature group, pattern features are extracted from each rule. Pattern features include the content specified by a fixed string pattern in the rule, as well as literal parts in the regular expression. All extracted pattern features are deduplicated and input into a multi-pattern matching compiler to build an Aho-Corasick automaton or Hyperscan database, which is then used as the pre-filtering engine for that signature group.

[0044] S14, the completed signature group is stored in the preset signature group table.

[0045] A completed signature group refers to a complete signature group object that already contains the rule set and the pre-filtering engine.

[0046] In this embodiment, by pre-grouping the full detection rules according to the five-tuple constraints before detection and establishing a dedicated pre-filtering engine for each group, runtime detection can quickly locate small-scale groups based on the five-tuple information of the data packets. Then, the pre-filtering engine can quickly exclude data packets that do not contain any pattern features. This preprocessing mechanism reduces the linear complexity of matching each data packet with the full rules one by one in the traditional scheme to a constant complexity related to the target group size, greatly reducing the number of invalid matches.

[0047] Step 104: Perform a fast multi-mode scan of the data content of the data packet to be detected based on the pre-filtering engine to obtain preliminary screening results.

[0048] The initial screening result is a Boolean value indicating whether the data content contains at least one pattern feature.

[0049] In some embodiments, the application layer data content of the data packet to be detected can be used as input text to drive the pre-filtering engine to perform scanning and obtain preliminary screening results.

[0050] Step 105: If the preliminary screening result indicates that the data content includes at least one pattern feature, then the corresponding multi-pattern matching algorithm is invoked to perform precise pattern matching on the data content to obtain at least one matching pattern identifier.

[0051] Pattern features refer to key strings, regular expressions, or byte sequences used in detection rules to match the content of data packets. For example, pattern features can be the fixed signature of malware or keywords in SQL injection attacks.

[0052] A multi-pattern matching algorithm is a string matching algorithm that can search for multiple pattern strings in a text simultaneously.

[0053] Precise pattern matching refers to the use of a selected multi-pattern matching algorithm to perform a complete and precise scan of the data packet content, provided that at least one pattern feature has been confirmed.

[0054] Pattern identifiers are unique codes for each predefined pattern feature, used to establish a mapping relationship between detection rules and matching results.

[0055] In some embodiments, a pre-selected target multi-pattern matching algorithm instance, such as a Hyperscan-compiled database, can be read from the global context of the detection engine. The data packet content is passed to the matching interface of the instance, which returns a list of matching events. Each event contains the hit pattern identifier and the hit offset. All unique pattern identifiers are extracted from the list and deduplicated to form the set of matching pattern identifiers.

[0056] In some embodiments, the corresponding multi-pattern matching algorithm is determined by: detecting the hardware characteristics of the current deployment environment, the hardware characteristics including CPU architecture type and compilation configuration parameters; and, based on the hardware characteristics, determining a multi-pattern matching algorithm that matches the hardware characteristics from a preset algorithm registry.

[0057] The current deployment environment refers to the overall operating system and hardware platform of the physical server, virtual machine, or embedded device that runs the data detection method.

[0058] Hardware characteristics refer to the set of attributes used to characterize processor capabilities and compilation environment.

[0059] CPU architecture type refers to the instruction set architecture of the central processing unit, such as x86_64 architecture, ARM architecture, Tile-Gx multi-core processor architecture, or CUDA architecture that supports GPU acceleration.

[0060] Compilation configuration parameters refer to options set through compiler macro definitions or build scripts when compiling the detection engine source code, such as whether the HAVE_HYPERSCAN macro is defined to enable Hyperscan support, or whether the HAVE_CUDA macro is defined to enable GPU acceleration support.

[0061] In some embodiments, detecting the hardware characteristics of the current deployment environment includes: directly probing the hardware characteristics of the current processing device using hardware instructions.

[0062] The default algorithm registry is a data structure used to store metadata for various multi-pattern matching algorithms and the hardware conditions applicable to each algorithm.

[0063] The algorithm registry predefines multiple sets of mappings, each associating a hardware feature condition with a multi-pattern matching algorithm instance. Multi-pattern matching algorithms are those capable of searching multiple pattern strings simultaneously, such as the standard Aho-Corasick algorithm, the Aho-Corasick bit-parallel algorithm, the Aho-Corasick Tile optimized algorithm, the Hyperscan algorithm, and the GPU-accelerated Aho-Corasick algorithm.

[0064] In some embodiments, each entry in the algorithm registry can be traversed, and the hardware characteristics required for each entry can be checked to see if they match the currently detected hardware characteristics. If the CPU architecture type required by an entry is consistent with the current architecture, and the required compilation configuration parameters are defined in the current environment, then the algorithm corresponding to that entry is determined to be the matching multi-pattern matching algorithm.

[0065] In this embodiment, by dynamically selecting the most suitable multi-pattern matching algorithm based on the CPU architecture type and compilation configuration parameters of the current deployment environment, the same detection method can adaptively run on different hardware platforms. This adaptive selection mechanism avoids the tediousness of manually configuring the algorithm while ensuring optimal matching performance in each deployment environment, thereby significantly improving the overall throughput of data detection.

[0066] In some embodiments, determining a multi-pattern matching algorithm that matches the hardware feature from a preset algorithm registry based on the hardware feature includes: S21, based on the hardware characteristics, determine one or more preset multi-mode matching algorithms that match the hardware characteristics from a preset algorithm registry.

[0067] In some embodiments, all entries in the algorithm registry can be traversed, and for each entry, it can be checked whether its declared CPU architecture requirements are consistent with or compatible with the currently detected CPU architecture type, and at the same time, it can be checked whether its declared compilation configuration requirements are met in the current compilation environment. All algorithms that meet the conditions are collected into the candidate algorithm list, and one or more preset multi-pattern matching algorithms are determined.

[0068] S22, for each preset multi-pattern matching algorithm: obtain the performance indicators of the detection threads after the one or more preset multi-pattern matching algorithms have been put into operation in history, the performance indicators include the average detection latency and throughput of data packets per unit time.

[0069] Historical deployment refers to the algorithm being actually used in previous detection sessions or test cycles.

[0070] A detection thread is a worker thread that performs packet detection tasks.

[0071] Performance metrics are quantitative values ​​that measure the efficiency of algorithm execution.

[0072] Average detection latency refers to the average time taken from when a data packet enters the detection engine to when multi-pattern matching is completed, which can be measured in microseconds or nanoseconds.

[0073] Throughput refers to the number of data packets that a detection thread can process per unit of time, usually expressed as data packets per second or bytes per second.

[0074] In some embodiments, obtaining historical performance metrics includes reading performance logs saved during the previous run from persistent storage at system startup. For new algorithms that have never been run, their historical performance metrics are initialized to preset default values ​​or data obtained through offline benchmarking.

[0075] S23, compare the performance indicators with a preset algorithm performance baseline.

[0076] The preset algorithm performance baseline refers to the reference threshold or expected performance range that is set in advance for each algorithm.

[0077] The algorithm performance baseline can include a first threshold for average detection latency and a second threshold for throughput.

[0078] The first threshold is the upper limit of the average detection latency, such as 100 microseconds. A latency below this upper limit indicates that the algorithm's latency performance is acceptable. The second threshold is the lower limit of the throughput, such as 10,000 packets per second. A throughput above this lower limit indicates that the algorithm's throughput performance is acceptable.

[0079] In some embodiments, the actual average detection latency can be compared with a first threshold to determine whether it is lower than the first threshold; the actual throughput can be compared with a second threshold to determine whether it is higher than the second threshold.

[0080] S24, select one of the preset multi-pattern matching algorithms from the preset multi-pattern matching algorithms where the average detection delay is lower than a first threshold and / or the throughput is higher than a second threshold as the multi-pattern matching algorithm for the matching.

[0081] In this embodiment, after selecting candidate algorithms based on hardware characteristics, the system further combines historical performance metrics such as average detection latency and throughput with a preset baseline to select the best algorithm. This ensures that the determination of the multi-pattern matching algorithm depends not only on static hardware compatibility but also on dynamic runtime performance. This allows the system to adapt to changes in traffic characteristics and fluctuations in system load. For example, if a certain algorithm experiences increased latency due to rule base updates, the system will automatically switch to another algorithm that meets the performance requirements.

[0082] Step 106: Obtain the target detection rule associated with the at least one matching pattern identifier from the plurality of detection rules, and perform rule matching based on the target detection rule to generate data detection results.

[0083] Multiple detection rules refer to a set of detection rules associated with a signature group. Target detection rules refer to detection rules associated with a pattern identifier that require further execution of complete conditional judgments.

[0084] Rule matching refers to verifying conditions other than pattern features in target detection rules, such as verifying quintuple constraints, flow direction, data content length, or protocol status.

[0085] The data detection result is the final output, including the rule identifier of a successful match, the type of action triggered such as alarm or drop, and related metadata such as timestamp and 5-tuple information.

[0086] In some embodiments, the step of performing rule matching based on the target detection rule to generate data detection results includes: S31, Traverse the rule fields of the target detection rule in priority order.

[0087] Rule fields refer to the various conditional attribute units that constitute a detection rule, such as the five-tuple constraint field, pattern feature field, flow direction field, data content length field, protocol status field, and action type field.

[0088] Priority order refers to the pre-defined order of rules based on their importance or probability of success; rules with higher priority are listed first and processed first. Traversal refers to visiting each rule field in this order.

[0089] S32, perform a full rule match on each of the rule fields.

[0090] Complete rule matching involves validating every condition field in the rule to determine if the state of the data packet or flow meets the requirements of that field. For the 5-tuple constraint field, matching checks if the 5-tuple of the data packet falls within the address and port range specified by the rule. For the pattern feature field, matching checks if the data packet content contains the pattern string or regular expression defined by the rule. For the flow direction field, matching checks whether the current data packet is in a client-to-server or server-to-client direction. For the data content length field, matching compares whether the number of bytes in the data packet content is within the range specified by the rule.

[0091] In some embodiments, each condition field of the target detection rule can be validated sequentially. For the protocol type field, check whether the transport layer protocol number in the packet parsing result matches the one specified in the rule. For the source address field, perform a subnet inclusion check between the source IP of the packet and the IP address or CIDR network segment defined in the rule. The same applies to the destination address field. For the source port and destination port fields, check whether the port numbers are within the port range specified in the rule, such as 1024 to 65535. For the pattern feature field, use the pattern identifier that has already been successfully matched as a fast pass condition. When all rule fields have been validated successfully, the rule is confirmed to have matched successfully.

[0092] S33, if the match is successful, a detection result is generated according to the action type configured in the target detection rule, wherein the action type includes alarm, discard, or pass.

[0093] A successful match means that all rule fields of the target detection rule meet the conditions after verification.

[0094] The action type is a predefined processing method in each detection rule. Alarm means that only the log is recorded and a warning message is reported, and the data packet is still forwarded normally; drop means that the data packet is intercepted and prevented from continuing to be transmitted; pass means that no processing is done and the data packet is allowed to pass normally.

[0095] In this embodiment, by traversing each rule field of the target detection rule according to priority and performing complete rule matching sequentially, the corresponding detection result can be generated promptly based on the action type after a successful match. The priority traversal mechanism ensures that high-priority rules, such as discard rules, are processed first, and the verification of subsequent rule fields can be terminated early once a match is found, reducing computational overhead. Furthermore, by pre-grouping detection rules using five-tuple information and combining it with two-stage matching (fast pre-filtering combined with precise pattern matching), the problem of excessive invalid matches and low efficiency in traditional solutions is effectively solved. In some embodiments, traversing the target detection rules in priority order includes: S41, obtain the historical hit counter established for each rule field in the signature group; wherein, after each detection is completed, if the rule field is successfully matched, the value of the historical hit counter is incremented.

[0096] The historical hit counter is an integer variable associated with each rule field, used to record the cumulative number of times that rule field has been successfully matched in the historical detection process.

[0097] In some embodiments, during system initialization, an independent hit counter is assigned to each rule field within each signature packet. This hit counter is stored in the metadata structure of the rule field. The initial value of the counter is 0. Each time a detection thread completes a full rule match on a data packet, the hit counter corresponding to that rule field is incremented by 1.

[0098] S42, reorder the rule fields in the signature group according to the value of the historical hit counter, and move the rule fields with high historical hit counts to the front of the sorting.

[0099] Reordering refers to changing the traversal order of rule fields within a signature group. A high historical hit count means a relatively large counter value, indicating that the rule field has been frequently matched successfully over a period of time. The front of the sort refers to the starting position of the rule field list; rule fields at this position will be traversed first.

[0100] S43, traverse the rule fields of the target detection rule based on the reordered priority.

[0101] In this embodiment, a historical hit counter is established for each rule field in the signature group, and the counter value is incremented according to the matching success after each detection. Then, the rule fields are reordered according to the counter value, and the fields with high historical hit counts are moved to the front of the sorting, so that the detection engine can adaptively adjust the traversal order of the rule fields.

[0102] In some embodiments, traversing the target detection rules in priority order further includes: S51, obtain the historical hit time series established for each rule field in the signature group, wherein the historical hit time series records the timestamps of the most recent N successful matches of the rule field.

[0103] Historical match time series is a list arranged chronologically, used to store the times when a specific rule field was successfully matched within a certain period of time. "Most recent N times" refers to retrieving the N most recent successful match events, where N is a preset positive integer, such as N equal to 10 or 20.

[0104] A timestamp refers to the specific moment when a successful match occurs, and can be represented by a system clock value.

[0105] In some embodiments, a fixed-length circular queue, with a queue length of N, is allocated for each rule field during system initialization. After each detection, if a rule field matches successfully, the current system's monotonic clock value is obtained, and the timestamp is written to the circular queue corresponding to that rule field. If the queue is full, the oldest timestamp record is overwritten. When historical hit time series are needed, all non-empty timestamp entries are read directly from the circular queue and assembled into a time series in ascending order.

[0106] S52, the weight values ​​corresponding to each successful match of each rule field within a preset time are accumulated to determine the weighted score; wherein, the weight value of each successful match is determined according to the time difference between the time of its occurrence and the current time, the smaller the time difference, the larger the weight value, and the larger the time difference, the smaller the weight value.

[0107] The preset time refers to a time window, such as the last 5 minutes or the last 30 minutes, and only successful matches occurring within this time window are considered. The weight value is a numerical coefficient used to reflect the importance of a single match event. The time difference is the difference between the moment a successful match occurred and the current moment, and can be measured in seconds or milliseconds.

[0108] The weighted score is the sum of the weights of all successfully matched events within a preset time period.

[0109] In some embodiments, to reduce computational complexity, a tiered decay strategy can be adopted: the time window is divided into multiple intervals, such as the most recent 1 minute, 1 to 3 minutes, and 3 to 5 minutes, and fixed weight values ​​such as 1.0, 0.5, and 0.2 are assigned to each interval. Then, the number of hits in each interval is counted, and the number of hits is multiplied by the interval weight and then summed to obtain a weighted score.

[0110] S53, reorder the rule fields within the signature group according to the weighted score.

[0111] The weighted score refers to the magnitude of the numerical value; a high score indicates that the rule field has been matched successfully in recent and frequent cases.

[0112] S54, traverse the rule fields of the target detection rule based on the reordered priority.

[0113] In this embodiment, the historical hit time series of each rule field is obtained, and a decreasing weight is assigned based on the time difference between the successful match and the current time. This weighted score is accumulated, and then the rule fields are reordered according to their scores. This prioritizes fields that have recently had frequent successful matches, placing them at the front of the traversal order. Compared to sorting methods that simply rely on a hit count counter, the time-decay-based weighted score better reflects the real-time popularity changes of rules and can quickly respond to dynamic shifts in traffic behavior.

[0114] Reference Figure 2 The diagram shown is a functional module schematic of the data detection device 100 based on multi-pattern matching in this application.

[0115] The data detection device 100 based on multi-pattern matching described in this application is installed in an electronic device. Depending on its functions, the data detection device 100 includes an acquisition module 110, an extraction module 120, a determination module 130, a scanning module 140, a matching module 150, and a generation module 160. These modules can also be referred to as units, which are a series of computer program segments that can be executed by the processor of an electronic device and perform a fixed function, and are stored in the memory of the electronic device.

[0116] In this embodiment, the functions of each module / unit are as follows: Module 110 is used to acquire the data packet to be detected; The extraction module 120 is used to extract the five-tuple information of the data packet to be detected, wherein the five-tuple information includes source address, destination address, source port, destination port and protocol type; The determining module 130 is used to determine the corresponding signature group from a preset signature group table based on the five-tuple information, wherein the signature group is associated with multiple detection rules and a pre-filtering engine; the preset signature group table is obtained by: acquiring a pre-configured full set of detection rules and parsing the five-tuple constraints of each detection rule; grouping detection rules with the same five-tuple constraints into the same signature group; establishing a corresponding pre-filtering engine for each signature group, wherein the pre-filtering engine is used to perform a fast multi-mode scan of the data content of the data packet to exclude data packets that do not contain any pattern features; and storing the established signature group in the preset signature group table. The scanning module 140 is used to perform a fast multi-mode scan of the data content of the data packet to be detected based on the pre-filtering engine to obtain preliminary screening results; The matching module 150 is used to call the corresponding multi-pattern matching algorithm to perform precise pattern matching on the data content if the preliminary screening result indicates that the data content includes at least one pattern feature, so as to obtain at least one matching pattern identifier. The generation module 160 is used to obtain the target detection rule associated with the at least one matching pattern identifier from the plurality of detection rules, and perform rule matching based on the target detection rule to generate data detection results.

[0117] The specific implementation of the data detection device based on multi-pattern matching in this application is largely the same as the specific implementation of the data detection method based on multi-pattern matching described above, and will not be repeated here.

[0118] Reference Figure 3 The diagram shown is a schematic representation of a preferred embodiment of the electronic device of this application.

[0119] The electronic device includes a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114. The memory 113 is used to store computer programs, such as a data detection program based on multi-pattern matching; In some embodiments, the processor 111 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 111 is typically used to control the overall operation of the electronic device, such as performing data interaction or communication-related control and processing. In this embodiment, the processor 111 is used to run program code stored in the memory 113 or process data.

[0120] The communication interface 112 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The communication interface 112 may also be used to establish a communication connection between the electronic device and other electronic devices.

[0121] The memory 113 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 113 may be an internal storage unit of the electronic device, such as the hard disk or memory of the electronic device. In some embodiments, the memory 113 may also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. of the electronic device. Of course, the memory 113 may include both internal storage units and external storage devices of the electronic device. In this embodiment, the memory 113 is typically used to store the operating system and various computer programs installed on the electronic device, such as program code for a data detection program based on multi-pattern matching. In addition, the memory 113 may also be used to temporarily store various types of data that have been output or will be output.

[0122] Figure 3 Only an electronic device having a processor 111, a communication interface 112, a memory 113 and a communication bus 114 is shown. However, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0123] In one embodiment of this application, when the processor 111 executes the program stored in the memory 113, it implements the data detection method based on multi-pattern matching provided in any of the foregoing method embodiments, including: Obtain the data packet to be detected; Extract the five-tuple information of the data packet to be detected. The five-tuple information includes the source address, destination address, source port, destination port, and protocol type. Based on the quintuple information, the corresponding signature group is determined from the preset signature group table, wherein the signature group is associated with multiple detection rules and a pre-filtering engine; Based on the pre-filtering engine, the data content of the data packet to be detected is quickly scanned in multiple modes to obtain preliminary screening results; If the preliminary screening result indicates that the data content includes at least one pattern feature, then the corresponding multi-pattern matching algorithm is invoked to perform precise pattern matching on the data content to obtain at least one matching pattern identifier. Obtain the target detection rule associated with the at least one matching pattern identifier from the plurality of detection rules, and perform rule matching based on the target detection rule to generate data detection results; The preset signature grouping table is obtained in the following way: Obtain the pre-configured full set of detection rules and parse the quintuple constraints of each detection rule; Detection rules with the same quintuple constraints are grouped into the same signature group; A corresponding pre-filtering engine is established for each signature group. The pre-filtering engine is used to perform a fast multi-mode scan on the data content of the data packet to exclude data packets that do not contain any pattern features. The completed signature groups are stored in the preset signature group table.

[0124] For a detailed explanation of the above steps, please refer to the above. Figure 1 A flowchart illustrating an embodiment of a data detection method based on multi-pattern matching.

[0125] Furthermore, this application also proposes a computer-readable storage medium that is both non-volatile and volatile. This computer-readable storage medium is any one or any combination of several of the following: hard disk, multimedia card, SD card, flash memory card, SMC, read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, etc. The computer-readable storage medium includes a data storage area and a program storage area. The program storage area stores a data detection program based on multi-pattern matching. When executed by a processor, the data detection program based on multi-pattern matching performs the following operations: Obtain the data packet to be detected; Extract the five-tuple information of the data packet to be detected. The five-tuple information includes the source address, destination address, source port, destination port, and protocol type. Based on the quintuple information, the corresponding signature group is determined from the preset signature group table, wherein the signature group is associated with multiple detection rules and a pre-filtering engine; Based on the pre-filtering engine, the data content of the data packet to be detected is quickly scanned in multiple modes to obtain preliminary screening results; If the preliminary screening result indicates that the data content includes at least one pattern feature, then the corresponding multi-pattern matching algorithm is invoked to perform precise pattern matching on the data content to obtain at least one matching pattern identifier. Obtain the target detection rule associated with the at least one matching pattern identifier from the plurality of detection rules, and perform rule matching based on the target detection rule to generate data detection results; The preset signature grouping table is obtained in the following way: Obtain the pre-configured full set of detection rules and parse the quintuple constraints of each detection rule; Detection rules with the same quintuple constraints are grouped into the same signature group; A corresponding pre-filtering engine is established for each signature group. The pre-filtering engine is used to perform a fast multi-mode scan on the data content of the data packet to exclude data packets that do not contain any pattern features. The completed signature groups are stored in the preset signature group table.

[0126] The specific implementation of the computer-readable storage medium in this application is largely the same as the specific implementation of the data detection method based on multi-pattern matching described above, and will not be repeated here.

[0127] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0128] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware simulation platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0129] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A data detection method based on multi-pattern matching, characterized in that, The method includes: Obtain the data packet to be detected; Extract the five-tuple information of the data packet to be detected. The five-tuple information includes the source address, destination address, source port, destination port, and protocol type. Based on the quintuple information, the corresponding signature group is determined from the preset signature group table, wherein the signature group is associated with multiple detection rules and a pre-filtering engine; Based on the pre-filtering engine, the data content of the data packet to be detected is quickly scanned in multiple modes to obtain preliminary screening results; If the preliminary screening result indicates that the data content includes at least one pattern feature, then the corresponding multi-pattern matching algorithm is invoked to perform precise pattern matching on the data content to obtain at least one matching pattern identifier. Obtain the target detection rule associated with the at least one matching pattern identifier from the plurality of detection rules, and perform rule matching based on the target detection rule to generate data detection results; The preset signature grouping table is obtained in the following way: Obtain the pre-configured full set of detection rules and parse the quintuple constraints of each detection rule; Detection rules with the same quintuple constraints are grouped into the same signature group; A corresponding pre-filtering engine is established for each signature group. The pre-filtering engine is used to perform a fast multi-mode scan on the data content of the data packet to exclude data packets that do not contain any pattern features. The completed signature groups are stored in the preset signature group table; The step of performing rule matching based on the target detection rule to generate data detection results includes: The rule fields of the target detection rules are traversed in priority order. Perform a full rule match on each of the aforementioned rule fields; If a match is successful, a data detection result is generated according to the action type configured in the target detection rule. The action type includes alarm, discard, or pass. The step of traversing the target detection rules in priority order includes: Obtain the historical hit time series for each rule field within the signature group, wherein the historical hit time series records the timestamps of the most recent N successful matches of the rule field; The weighted score is determined by summing the weight values ​​corresponding to each successful match of each rule field within a preset time. The weight value of each successful match is determined based on the time difference between the time of the match and the current time. The smaller the time difference, the larger the weight value, and vice versa. The rule fields within the signature group are reordered according to the weighted score. The rule fields of the target detection rule are traversed based on the reordered priority.

2. The data detection method based on multi-pattern matching as described in claim 1, characterized in that, The corresponding multi-pattern matching algorithm is determined in the following way: Detect the hardware characteristics of the current deployment environment, including CPU architecture type and compilation configuration parameters; Based on the hardware characteristics, a multi-pattern matching algorithm matching the hardware characteristics is determined from a preset algorithm registry.

3. The data detection method based on multi-pattern matching as described in claim 2, characterized in that, The step of determining a multi-pattern matching algorithm that matches the hardware features from a preset algorithm registry based on the hardware features includes: Based on the hardware characteristics, one or more preset multi-mode matching algorithms that match the hardware characteristics are determined from a preset algorithm registry. For each preset multi-pattern matching algorithm: Obtain the performance metrics of the detection threads after the one or more preset multi-pattern matching algorithms have been put into operation in history. The performance metrics include the average detection latency and throughput of data packets per unit time. The performance metrics are compared with a preset algorithm performance baseline; Select one of the preset multi-pattern matching algorithms as the matching algorithm from the preset multi-pattern matching algorithms where the average detection latency is lower than a first threshold and / or the throughput is higher than a second threshold.

4. A data detection device based on multi-pattern matching, characterized in that, The device includes: The acquisition module is used to acquire the data packets to be detected; The extraction module is used to extract the five-tuple information of the data packet to be detected. The five-tuple information includes the source address, destination address, source port, destination port, and protocol type. The determination module is used to determine the corresponding signature group from a preset signature group table based on the five-tuple information, wherein the signature group is associated with multiple detection rules and a pre-filtering engine; The scanning module is used to perform a fast multi-mode scan of the data content of the data packet to be detected based on the pre-filtering engine to obtain preliminary screening results; The matching module is used to call the corresponding multi-pattern matching algorithm to perform precise pattern matching on the data content if the preliminary screening result indicates that the data content includes at least one pattern feature, so as to obtain at least one matching pattern identifier. The generation module is used to obtain the target detection rule associated with the at least one matching pattern identifier from the plurality of detection rules, and perform rule matching based on the target detection rule to generate data detection results; The preset signature grouping table is obtained in the following way: Obtain the pre-configured full set of detection rules and parse the quintuple constraints of each detection rule; Detection rules with the same quintuple constraints are grouped into the same signature group; A corresponding pre-filtering engine is established for each signature group. The pre-filtering engine is used to perform a fast multi-mode scan on the data content of the data packet to exclude data packets that do not contain any pattern features. The completed signature groups are stored in the preset signature group table; The step of performing rule matching based on the target detection rule to generate data detection results includes: The rule fields of the target detection rules are traversed in priority order. Perform a full rule match on each of the aforementioned rule fields; If a match is successful, a data detection result is generated according to the action type configured in the target detection rule. The action type includes alarm, discard, or pass. The step of traversing the target detection rules in priority order includes: Obtain the historical hit time series for each rule field within the signature group, wherein the historical hit time series records the timestamps of the most recent N successful matches of the rule field; The weighted score is determined by summing the weight values ​​corresponding to each successful match of each rule field within a preset time. The weight value of each successful match is determined based on the time difference between the time of the match and the current time. The smaller the time difference, the larger the weight value, and vice versa. The rule fields within the signature group are reordered according to the weighted score. The rule fields of the target detection rule are traversed based on the reordered priority.

5. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the data detection method based on multi-pattern matching as described in any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the data detection method based on multi-pattern matching as described in any one of claims 1 to 3.

Images