High-speed mode matching device and method based on FPGA

Abstract

The invention provides a high-speed mode matching device and method based on an FPGA, and the device comprises an extraction module which is used for carrying out the multi-dimensional feature extraction of a rule set; the clustering module is used for generating rule categories; the Bloom filtering module is used for screening the input data; the Hash matching module is used for verifying whether the data flow is matched with a known mode or not; the probability verification module is used for calculating and verifying a hash matching passing probability; the regular matching module is used for matching complex modes; the feedback module is used for dynamically adjusting Bloom filter parameters, hash table distribution and probability formula coefficients; and the matching result reporting module is used for reporting the matching information. According to the method, the balance of high efficiency, accuracy and expandability can be realized in a multi-mode matching task, the resource utilization efficiency of an FPGA (Field Programmable Gate Array) is improved, and the matching requirement when a mode matching rule set is relatively large is met.

Description

Technical Field

[0001] The present invention relates to the field of network traffic processing and hardware-accelerated computing, and particularly relates to a high-speed pattern matching device and method based on FPGA. Background Art

[0002] With the continuous increase of Internet applications and the complexity of application layer protocols, the traditional state detection technology based on network packet headers can no longer meet the current network security processing requirements. Therefore, the deep detection technology based on network packet payloads has emerged and has become the key component and core technology of devices such as service-aware routers, deep detection firewalls, network intrusion detection systems, network intrusion prevention systems, and unified threat management. With the proposal of concepts such as software-defined networks and the increasing popularity of cloud computing, pattern matching faces requirements such as complex rule features, a large number of rules, rapid rule updates, high throughput, low latency, and low power consumption in programmable network transmission nodes and service nodes. Therefore, combining the latest progress of current software and hardware platforms to study new and high-performance pattern matching algorithms is of great significance for the development and promotion of high-performance network security detection and content awareness and other devices.

[0003] In the current technical context, pattern matching tasks are widely applied in fields such as network security, data compression, and bioinformatics. Traditional pattern matching methods are mainly implemented based on software. For example, through CPU-driven string matching algorithms (such as KMP, Boyer-Moore) or multi-pattern matching optimization schemes. Although these methods perform well in processing medium and small-scale data, when faced with massive data or high-throughput scenarios, problems such as high resource occupancy and weak parallel processing ability gradually emerge, resulting in the matching efficiency being difficult to meet real-time requirements. For example, the invention "A Multi-Pattern Matching Method, Device, Electronic Device, and Storage Medium (CN 113377917 B)" proposes to symbolize the text to be matched into a symbol sequence, convert the regular expression into a symbol sequence, and then sequentially match the symbol sequences on the trie tree, and adopt different matching strategies for different symbol types for matching. Another patent, "FPGA-Based Network Traffic String Automatic Matching Method and Matching Device (CN 117574178 B)", adopts a hardware acceleration scheme, parallelizes the data stream through FPGA, and significantly improves the throughput. However, its pattern rule update requires re-burning the hardware logic, with poor flexibility and difficulty in adapting to the requirements of a dynamically changing rule library. Although the prior art has improved the pattern matching efficiency in different dimensions, it still faces three challenges: First, the software solution is restricted by the CPU computing power and is difficult to break through the real-time bottleneck; second, the hardware acceleration solution has insufficient flexibility and is difficult to support dynamic rule updates; third, although the distributed architecture alleviates the single-point pressure, it introduces additional complexity and communication costs. Especially in scenarios such as encrypted traffic detection and genome sequence alignment, there is an urgent need for a pattern matching method with high throughput, low latency, and dynamic adaptability to provide underlying support for real-time data processing in complex environments. Summary of the Invention

[0004] Object of the Invention: The technical problem to be solved by the present invention is to provide a high-speed pattern matching method and device based on FPGA (Field Programmable Gate Array) in view of the deficiencies of the prior art. By utilizing the parallelism, programmability, and pipelining characteristics of FPGA, through a three-level hierarchical processing architecture of Bloom filter, hash pre-screening, and regular expression engine, high-efficiency and low-latency large-scale rule matching is achieved at the FPGA hardware level, taking into account high throughput, low false positive rate, and hardware resource conservation in scenarios such as network security and streaming data processing, and solving the performance bottleneck of traditional CPU / GPU solutions when processing massive rules.

[0005] The present invention first provides a high-speed pattern matching device based on FPGA, including an extraction module, a clustering module, a Bloom filter module, a hash matching module, a time-frequency weighted probability verification module, a regular matching module, a feedback module, and a matching result reporting module; The extraction module is used to perform multi-dimensional feature extraction on the rule set to generate a feature vector including rule length, type, historical matching frequency, and information entropy; The clustering module is used to perform real-time clustering analysis on the feature vector to generate multiple rule categories; The Bloom filter module is used to quickly screen the input data to determine whether it may match; The hash matching module is used to quickly verify whether the data stream screened by the Bloom filter matches a known pattern; The time-frequency weighted probability verification module is used to calculate and verify the probability of passing the hash match based on the time weight and historical matching frequency of the rule; The regular matching module is used to accurately match complex rules; The feedback module is used to analyze the regular matching results and dynamically adjust the parameters of the Bloom filter, the distribution of the hash table, and the coefficients of the probability formula; The matching result reporting module is used to summarize and analyze the matching information and report the final matching result.

[0006] The present invention also provides a high-speed pattern matching method based on FPGA implemented by using the above device, including the following steps: Step 1: Perform multi-dimensional feature extraction on the network rule set to generate a feature vector including rule length, type, historical matching frequency, and information entropy. Use the hardware-accelerated clustering module to perform real-time clustering analysis on the feature vector to generate rule categories. Dynamically map the rules to the three-level storage structure of the FPGA (Field Programmable Gate Array) according to the clustering results. Configure a hierarchical Bloom filter for each level of the structure. Dynamically trigger rule migration and adjustment of the Bloom filter parameters by real-time monitoring the matching metrics of each storage level; Step 2: The network traffic data is screened through the three-level Bloom filter in sequence to determine whether the input data segment may match the rule. If any level of the Bloom filter determines that it may match, trigger the hash matching module; otherwise, directly discard the data; Step 3: Use the hash matching module to match the data determined by the Bloom filter to be possibly matching to verify whether the screened data matches. If a match is determined, the time-frequency weighted probability verification module is used for verification. If no match occurs, it is marked as a false alarm and the false alarm rate is updated; Step 4: For the possibly matching data screened in Step 3, calculate the matching probability based on the time weight and historical matching frequency of the rule. At the same time, set a matching probability threshold (generally 0.5 to 0.7) according to the system resource load and real-time traffic characteristics, and send the traffic data higher than the matching probability threshold to the regular matching module for complete matching; Step 5, the regular matching module performs rule verification on traffic data with a matching probability higher than the threshold, verifies whether the data segment completely conforms to the preset rules, and simultaneously feedbacks and optimizes the module parameters according to the matching results.

[0007] In step 1, the hardware acceleration clustering module performs the following steps: Step 1-1, feature extraction: Through two or more parallel computing units, use hardware acceleration to preprocess the preset rule set to generate a multi-dimensional feature vector , where is the length of the i-th rule, is the type of the i-th rule, is the historical matching frequency of the i-th rule, is the information entropy of the i-th rule; Step 1-2, clustering calculation: Use the Manhattan distance formula to calculate the distance between the feature vector of each rule and the current cluster center: , where represents the Manhattan distance between the feature vector of the i-th rule and the cluster center, represents the feature value of the i-th rule in the d-th dimension, represents the cluster center coordinate of the k-th cluster in the d-th dimension; According to the Manhattan distance between the feature vector of each rule and the cluster center, the rules are assigned to the nearest cluster, and the new cluster center is calculated. The updated cluster center will be used for the distance calculation in the next cycle; Step 1-3, dynamic cluster adjustment, each cluster independently calculates the standard deviation of each dimension : , where represents the feature value of the i-th rule in the d-th dimension, is the cluster center coordinate of the current cluster in the d-th dimension, and N is the total number of rules in the current cluster; When the standard deviation between the rules in the cluster is greater than the threshold of 0.5, it is determined that splitting is required. The splitting operation selects the dimension with the largest variance in the cluster for splitting and forms two sub-clusters; When the centroid distance between two clusters is less than 0.5 times the average cluster spacing and the feature distribution similarity of the rules is greater than the threshold of 0.9, the two clusters are merged into a new cluster and the cluster center is updated; Step 1-4, rule classification and storage mapping: According to the results of clustering calculation and cluster update, each rule is dynamically mapped to a three-level storage structure to optimize storage and rule matching efficiency.

[0008] In Steps 1-4, the three-level storage structure includes L1-level storage, L2-level storage, and L3-level storage; among them, the L1-level storage is used to store high-frequency and high-entropy rules, the L2-level storage is used to store medium-frequency rules, and the L3-level storage is used to store low-frequency rules; The rules satisfied by the L1-level storage are any of the following: the rule length ≤ 16 bytes and the matching frequency > 80%; the average information entropy of the cluster ≥ 1.5 and the rule type consistency within the cluster is high (type standard deviation < 0.3); The rules satisfied by the L2-level storage are any of the following: the rule length is in the range of 16 to 64 bytes; the average matching frequency of the cluster is between 30% and 80%; The rules satisfied by the L3-level storage are all of the following conditions: the average length of the rules within the cluster > 64 bytes; the matching frequency < 30%; the average information entropy of the cluster ≤ 1.0; Dynamically adjust the storage level according to the matching frequency of the rules within the cluster. The specific rule migration conditions are: Downgrade of L1-level rules: When the historical matching frequency of a rule in the L1-level storage decreases by more than 20% for 3 consecutive cycles or the average feature distance between the cluster center and the current data stream is the rule feature variance, then it is downgraded to the L2-level storage, is the rule feature variance; Downgrade of L2-level rules: When the rule historical matching frequency is lower than the threshold of 30% for 5 consecutive cycles or the information entropy ; then it is downgraded to the L3-level storage; Upgrade of L2-level rules: When the similarity of the rule feature distribution between the L2 layer and the L1 layer is greater than 0.85, the matching frequency increases by more than 30% and the resource occupancy rate is less than 70%; then it is upgraded to the L1-level storage; Upgrade of L3-level rules: Triggered by burst traffic (the change in the rule historical matching frequency is greater than 500%), then it is upgraded to the L2-level storage.

[0009] In Step 1, the adjustment of the Bloom filter parameters includes: Calculating the length of the bit vector: where m is the length of the Bloom filter bit vector, n is the number of rules in the current storage level, p represents the target false positive rate, is the frequency change coefficient, is the frequency change rate of the current period; Adjusting the number of hash layers: where k represents the number of hash function layers, is the variance sensitivity coefficient. is the variance sensitivity coefficient.

[0010] In step 3, the time-frequency weighted probability verification module performs the following steps: Calculate the time weight: Based on the difference between the current time and the last matching time of the rule, calculate the time weight of the rule : , where λ is the time decay coefficient, T is the current time, is the last matching time of the rule, and e is the natural constant; Statistical historical matching frequency: Record the number of times the rule is matched and the total number of matches, and calculate the matching frequency F of the rule: , where K represents the number of times the rule is confirmed by the regular matching module, M is the total number of regular validations of all rules in the device, is the smoothing factor to prevent the frequency from being zero; Calculate the comprehensive probability: Calculate the matching probability through the time weight and the historical matching frequency : , where, is the time weight coefficient, is the historical frequency coefficient, is the time interval adjustment coefficient, and sigmoid is the activation function.

[0011] In step 5, the regular matching module disassembles the regular expression matching task into two or more independent hardware units by pre-constructing a functional area. The hardware units are responsible for different regular expression elements (characters, quantifiers, branches, etc.). During the matching process, the hardware units dynamically load the state transition relationship of the regular expression from the block random access memory BRAM and construct the matching combination in real time according to the state transition relationship. The specific steps are as follows: Step 5-1, cache the data stream confirmed by the time-frequency weighted probability verification module, and use the sliding window method to extract every two consecutive characters as a group. Subsequently, according to the preset regular expression acceptable character table, look up and match the extracted character pairs, filter out the set of regular expressions that can accept the character pairs at the same time, and determine the address of the regular expression to be activated according to the matching result; Step 5-2: According to the regular expression address obtained in Step 5-1, query the block random access memory (BRAM) storing the specific matching information of the regular expression, and allocate the data read from the BRAM to the character matching functional unit and the quantifier matching functional unit to construct a regular expression matching combination unit. The character matching functional unit is responsible for processing the direct matching of single characters or character sets in the regular expression, and the quantifier matching functional unit is responsible for processing the logic related to quantifiers in the regular expression, that is, controlling the repetition times of characters or sub-patterns. Step 5-3: The regular expression matching combination unit sequentially obtains the data to be matched from the cached data stream, and at the same time manages and controls the signals according to the regular expression state machine read from the block random access memory (BRAM), ensuring that the character matching functional unit and the quantifier matching functional unit execute in the correct order. The state machine will determine whether to continue the match, whether to roll back to the previous state, or whether to end the current matching process. Step 5-4: When the character matching functional unit and the quantifier matching functional unit complete the match in sequence according to the requirements of the regular expression state machine, the regular matching module outputs the matching result and feedbacks and optimizes the module parameters according to the matching result.

[0012] Step 5-1 includes: In the first clock cycle, cache and extract a batch of characters (such as two characters) from the input data stream, and then through a look-up table mechanism, query the pre-configured regular expression acceptable character table based on the extracted characters to determine the regular expression that can accept the characters, and activate the regular expression for matching. In each subsequent clock cycle, continue to extract the next batch of characters from the input data stream and use the look-up table mechanism to match the characters. For each pair of characters in each clock cycle, first check whether the look-up table result contains the regular expression activated in the previous cycle. If the look-up table result contains the activated regular expression, directly pass the data of the pair of characters to the activated regular expression matching combination unit to continue the matching operation; for the new regular expression contained in the look-up table result, read the data of the new regular expression from the block random access memory (BRAM) to construct a new regular expression matching combination unit.

[0013] Step 5-4 includes: Step 5-4-1: Real-time collect the output results of the regular matching module, including the rule identifier of successful matching, the matching timestamp and traffic feature data, and the false alarm sample information of unsuccessful matching. Step 5-4-2: Dynamically adjust the rule storage level according to the output result of Step 5-4-1. When a rule meets the rule upgrade condition in the low-level storage layer, migrate the rule to the high-level storage layer and update the Bloom filter of the corresponding layer; when a rule meets the downgrade condition in the high-level storage layer, downgrade the rule to the low-level storage layer and update the Bloom filter of the corresponding layer. Step 5-4-3: Based on the distribution characteristics of false positive samples, reversely correct the hash function parameters and bit array layout of the Bloom filter, calculate the hash collision points of the false positive samples, generate a bit correction mask, and perform local reconstruction on the bit vector of the Bloom filter that generates false matches. Step 5-4-4: Iteratively update the cluster center coordinates and cluster boundaries.

[0014] The present invention also provides an FPGA device, including: an FPGA chip, and one or more Ethernet interfaces; the Ethernet interfaces are used for data transmission with the outside, and one or more programs are stored in the FPGA chip, and when the programs are executed by a processor, the steps of the above method are implemented.

[0015] Compared with the prior art, the beneficial effects of the present invention are: (1) Existing software-based pattern matching systems are difficult to meet the requirements of dynamic rule update and real-time matching in a high-speed network environment, especially facing computational latency and storage bottlenecks in high-throughput scenarios. The present invention realizes real-time analysis of multi-dimensional feature vectors through the FPGA hardware-accelerated clustering module, and combines the dynamic migration mechanism of the three-level storage structure, so that the rule matching latency is reduced to the nanosecond level. By adopting a hierarchical Bloom filter and hash matching collaborative architecture, the false positive rate is greatly reduced compared with traditional methods. Through the dynamic weight calculation of the timing probability model, the computational load of regular expression matching is greatly reduced, and the processing efficiency of network traffic is significantly improved.

[0016] (2) Traditional solutions have problems of storage fragmentation and low resource utilization when processing dynamic rule sets. The matrix storage structure based on hardware acceleration proposed by the present invention reduces the BRAM resource occupancy rate, and at the same time, by dynamically adjusting the Bloom filter parameters according to rule characteristics, the storage space utilization rate is improved. In particular, the adaptive migration strategy of the L1-L3 storage levels can improve the burst traffic processing ability, and the feedback optimization module reduces the false positive rate fluctuation during continuous operation of the system by real-time correcting the hash function parameters, significantly enhancing the stability and adaptability of the pattern matching system.

[0017] (3) The regular part of this solution has significant dynamic processing advantages. Through the sliding window mechanism and the look-up table method, it can efficiently perform regular expression matching and support parallel processing. Through the matching functional units pre-built in hardware, the system can dynamically load the matching information of regular expressions according to real-time data and reconfigure the matching strategy at runtime as needed. In particular, by modifying the data in the BRAM, the reconfiguration of the regular matching module can be achieved, enabling the system to flexibly adapt to different matching requirements and traffic changes, thereby improving the matching efficiency and the scalability of the system. Description of the Drawings

[0018] Figure 1 It is a flowchart of the method of the present invention.

[0019] Figure 2 It is a schematic diagram of the data flow in the method of the present invention.

[0020] Figure 3 It is a flowchart of the operation of the regular expression matching module of the present invention.

[0021] Figure 4 It is a schematic diagram of the structure of the high-speed pattern matching device based on FPGA of the present invention.

[0022] Figure 5 It is a structural diagram of an electronic device provided by the present invention. Specific Embodiments

[0023] The following further specific descriptions of the present invention are made in conjunction with the drawings and specific embodiments, and the above and / or other advantages of the present invention will become clearer.

[0024] Refer to Figure 1, an embodiment of the present invention provides a high-speed pattern matching method based on FPGA. This method is implemented on FPGA, and its main steps are as follows: perform multi-dimensional feature extraction on the network rule set and generate feature vectors, use a hardware-accelerated clustering module to perform real-time clustering analysis on the feature vectors, generate multiple rule categories, dynamically map the rules to the three-level storage structure of FPGA according to the clustering results, configure a hierarchical Bloom filter for each level of structure, and the input data is screened through the three-level Bloom filter in turn to quickly determine whether the input data segment may match the rules. If any level of the filter determines "possible match", the hash matching module is triggered. The hash matching module matches the data determined to be possibly matched to quickly verify whether the screened data matches. If a match is determined, it is verified by the probability verification module. If no match occurs, it is marked as a false alarm and the false alarm rate is updated. For the data possibly matched screened by the hash matching, calculate the matching probability based on the time weight of the rule and the historical matching frequency, and at the same time, according to the set matching probability threshold, determine the probability of rule matching. Only the traffic data with a high probability is handed over to the regular matching module for complete matching. The regular matching module performs rule verification on the high-probability input data to verify whether the data segment completely conforms to the preset rules, and at the same time, feedback and optimize the parameters of each module according to the matching results.

[0025] As Figure 2 shown, in the FPGA implementation architecture of the present invention, after the data stream enters the system through the SFP optical fiber interface, it first passes through the L1-L3 level Bloom filter for feature pre-screening, then passes through the hash matching module for verification and matching, and finally passes through the regular expression module for precise matching. The data stream that does not match or is mis-matched during the process flows out through the optical fiber outlet, while the data that is verified to match flows out through the report channel.

[0026] In the embodiment of the present invention, the specific steps of the method are as follows: Step 1, perform multi-dimensional feature extraction on the network rule set and generate feature vectors including rule length, type, historical matching frequency, and information entropy. Use a hardware-accelerated clustering module to perform real-time clustering analysis on the feature vectors, generate multiple rule categories, dynamically map the rules to the three-level storage structure of FPGA according to the clustering results, configure a hierarchical Bloom filter for each level of structure, and dynamically trigger rule migration and Bloom filter parameter adjustment by real-time monitoring the matching metrics of each storage level; According to the embodiment of the present invention, step 1 includes the following steps: Step 1-1: Perform multi-dimensional feature extraction on the network rule set. Specifically, multiple key attributes are extracted from each rule, including the length of the rule, the rule type, the historical matching frequency, and the information entropy. By statistically calculating these attributes, each rule is converted into a multi-dimensional feature vector, which can comprehensively reflect the performance characteristics of the rule in the actual matching process and provide data support for subsequent clustering analysis; Step 1-2: Use the hardware-accelerated clustering module to perform real-time clustering on the previously generated feature vectors. This module quickly groups rules with similar features into one category, generating multiple rule categories; Step 1-3: According to the clustering results, the system dynamically maps rules of different categories to the three-level storage structure of the FPGA. The three-level storage structure includes L1-level storage (used to store high-frequency and high-entropy rule clusters), L2-level storage (used to store medium-frequency rule clusters), and L3-level storage (used to store low-frequency rule clusters). Among them, frequently used rules are preferentially mapped to the storage layer with faster speed, while infrequently used or low-frequency rules are stored in the storage layer with larger capacity; Step 1-4: Configure a hierarchical Bloom filter for each level of the storage structure according to the data in the storage, and customize the parameters of the Bloom filter according to the characteristics of each storage layer and the rule categories to minimize the false positive rate and improve the search speed; Step 1-5: Continuously track the matching metrics of each storage level, such as key performance parameters such as matching latency and hit rate. When it is detected that the performance metrics of a certain level deviate from the preset threshold, the system will automatically trigger rule migration, reallocate some rules between the storage layers to achieve load balancing, and dynamically adjust the parameters of each level of Bloom filter according to real-time data to ensure that the pre-filtering effect is always in the optimal state.

[0027] The feature extraction described in Step 1-1 refers to preprocessing the preset rule set through multiple parallel computing units using hardware acceleration to generate a multi-dimensional feature vector , where is the rule length, is the rule type, is the historical matching frequency, is the information entropy of the rule. The implementation method for extracting each feature is as follows: (1) Rule length is defined as the number of characters or bytes in the rule definition, and its extraction method is implemented through a hardware counter. Specifically, the rule string is input into the FPGA in parallel, and an adder is used to accumulate the number of characters bit by bit or byte by byte until the rule is processed. For example, for the rule , the system counts the number of its characters as 8, thus obtaining The implementation of this method relies on the parallel architecture of the FPGA, and the processing delay is optimized through pipeline design. Although its time complexity is O(n), where n is the rule length, with the support of hardware acceleration, the actual execution time is significantly shortened, ensuring high efficiency.

[0028] (2)Rule type It is used to classify according to the matching mechanism of the rule, such as exact string matching, regular expression, or other specific types. Its extraction method is based on the parsing of the rule syntax, and the structural characteristics of the rule are judged through predefined classification criteria. When specifically implemented, after the rule data is input into the FPGA, its pattern characteristics are analyzed through a lookup table or a parallel comparator. For example, the rule "abc" is classified as an exact match, while is classified as a regular expression. The classification mapping rules are pre-loaded in the LUT, enabling the type determination to be completed in constant time with a time complexity of O(1).

[0029] (3)Historical matching frequency It reflects the frequency of successful matches of the rule in the past and is an important indicator for evaluating the activity of the rule. Its extraction method is to maintain a counter for each rule to record the number of matches. When specifically implemented, the counter is stored in the register or block RAM of the FPGA, and each time the rule matches successfully, the count is incremented through an adder. The system can record the absolute number of matches. For example, if a certain rule is matched 100 times within an hour, then fi = 100; or the relative frequency can be calculated through the ratio of the number of matches to the total number of matches or the number of matches within a time window. To achieve real-time update, the counters of multiple rules support parallel operations, and the time complexity of a single update is O(1). If normalized frequency is required, it is calculated periodically in the background with a complexity of O(n), where n is the total number of rules.

[0030] (4)Information entropy It is calculated through the Shannon entropy formula , and is used to measure the randomness and complexity of the rule content, where represents the occurrence probability of the symbol in the rule. The extraction process is divided into the following steps: First, use a hardware counter to count the occurrence times of each unique symbol in the rule; then, divide the number of times of each symbol by the total length of the rule to calculate the probability ; finally, calculate for each item , and sum through an accumulator and take the negative value. Since the FPGA is not good at floating-point operations, a lookup table is used in the system to approximate the logarithmic function to accelerate the calculation. For example, the entropy of the rule "aaaaa" is 0 (all characters are the same), while the entropy of "abcde" is (unique for each character). In the FPGA, a parallel counter and accumulator are assigned to each rule, with a time complexity of O(n + k), where n is the rule length and k is the number of unique symbols.

[0031] In step 1-2, the clustering module uses the Manhattan distance formula to calculate the distance between the feature vector of each rule and the current cluster center, and assigns each rule to the nearest cluster according to the calculated distance of each rule. At the same time, a new cluster center is calculated for the distance calculation in the next cycle. The clustering module implements a dynamic cluster adjustment mechanism, and each cluster independently calculates the standard deviation of each dimension . When the standard deviation between the rules within a cluster is greater than the threshold, it is considered that the rule feature distribution within the cluster is relatively dispersed and needs to be split. The splitting operation selects the dimension with the largest variance within the cluster for splitting and forms two sub-clusters. When the centroid distance between two clusters is less than 0.5 times the average cluster spacing of the system and the similarity of the rule feature distribution is greater than 0.9, the two clusters are merged into a new cluster and its cluster center is updated.

[0032] According to the results of clustering calculation and cluster update, the system dynamically maps each rule to a three-level storage structure to optimize storage and rule matching efficiency. In the three-level storage structure, the L1-level storage is used to store high-frequency and high-entropy rule clusters, the L2-level storage is used to store medium-frequency rule clusters, and the L3-level storage is used to store low-frequency rule clusters.

[0033] In step 1-4, two core improvements are made to the parameter design of the traditional Bloom filter to solve the problem of insufficient adaptability of the static model in a dynamic network environment: The first improvement is aimed at the bit vector length calculation model, and a traffic frequency change correction term is introduced into the classical formula ( ), where represents the real-time change rate of the rule matching frequency, is the sensitivity coefficient fitted through historical data. This correction term enables the bit vector capacity to be adaptively adjusted by dynamically perceiving the change in traffic intensity. When the traffic surges, the bit vector is expanded to suppress the fluctuation of the false positive rate, and when the traffic is stable, it maintains the theoretical optimal value, thus balancing the storage efficiency and stability requirements; The second improvement focuses on the hash layer number adjustment model, and a feature variance compensation factor is added to the traditional formula, where is the variance of the rule feature distribution, is the variance sensitivity coefficient. This compensation factor dynamically adjusts the hash calculation complexity by quantifying the distribution characteristics of the rule set, increasing the number of hash layers for high-variance rule sets with dispersed features to reduce the collision probability, and reducing the number of hash layers for low-variance rule sets with concentrated features to optimize the calculation efficiency.

[0034] Two parameter improvements form a synergy mechanism. By responding in real time to the dynamics of traffic and the characteristics of rule distribution, it optimizes the technical defects of the static model, such as rigid resource allocation and unstable false positive rate, in a dynamic network environment. The specific formulas are as follows: (1)Bit vector length calculation formula: , where m is the length of the bit vector of the Bloom filter, n is the number of rules in the current storage level, p represents the target false positive rate, is the change rate of the matching frequency in the current period; (2)Hash layer adjustment formula: , where k represents the number of layers of hash functions.

[0035] Step 2: The input data is screened through a three-level Bloom filter in sequence to quickly determine whether the input data segment is likely to match the rules. If any level of the filter determines "likely to match", the hash matching module is triggered; otherwise, the data is directly discarded.

[0036] The specific implementation process of Step 2 is as follows: The input data first enters the first-level Bloom filter (L1-BF). The hash function is used to calculate the data key fields and query the bit array. If all bits are 1, it is determined as "likely to match", and the subsequent screening process is immediately terminated, and the data is directly sent to the hash matching module for precise verification. If any bit is 0, it is determined as "not matching", and the data enters the second-level Bloom filter (L2-BF) to repeat the same operation. If L2-BF determines "likely to match", the hash matching module is also immediately triggered. If there is no match, the data continues to be passed to the third-level Bloom filter (L3-BF). Finally, if L3-BF determines "likely to match", the data enters the hash matching module; if all three levels of filters determine "not matching", the data is directly discarded. In this design, each level of the filter makes independent judgments and has the ability to trigger immediately, avoiding the delay caused by multi-level cascading, and ensuring that any preliminary match at any level can quickly start precise verification. At the same time, the false positive rate statistics are recorded separately according to the triggering level (for example, the false positives triggered by L1 are only accumulated to the L1 statistics item), supporting subsequent dynamic parameter optimization for different levels. This process significantly improves the processing efficiency, especially suitable for quickly screening out irrelevant data and reducing the computational load in high-frequency traffic scenarios. By using the FPGA parallel pipeline structure, the data stream is screened through the Bloom filter level by level, effectively improving the efficiency and response speed of the system in processing input data. By pre-using the Bloom filter in each storage level for quick possibility judgment, the system can quickly filter out most of the irrelevant data in large-scale rule matching scenarios, thus saving computational resources and processing time.

[0037] Step 3: The hash matching module matches the data determined by the Bloom filter as possibly matching, quickly verifying whether the filtered data matches. If a match is determined, the probability verification module conducts verification. If no match occurs, it is marked as a false positive and the false positive rate is updated. As the core verification link of the system, the hash matching module performs strict matching of all data after the preliminary screening by the Bloom filter. Its process is as follows: Step 3-1: Through preprocessing and full-content chunking of the data to be matched, the input data is completely retained and converted into a standardized format, including the packet header, payload, and tail information. The data is dynamically segmented into multiple logical blocks according to the protocol semantics, and the hash value of each block is calculated independently to construct a hierarchical hash tree covering the overall data. Step 3-2: Perform hash matching on all data in the rule library. The rule library pre-stores the complete hash tree structure of the data to be matched. During matching, the system compares the hash tree nodes of the data with the target hash values in the rule library level by level. First, compare the root hash of the data with the root hash of the rule. If they are the same, it is directly determined as a complete match. If they are different but the rule allows partial matching, further verify the hash values of the specified sub-blocks. When the rule needs to verify a specific block, the system only needs to compare the corresponding block hash chain and perform bit-level verification in combination with the binary content of the original data to avoid misjudgment caused by hash conflicts. Step 3-3: If the hash values match but the content does not match (conflict scenario), the system starts a forced verification mechanism, traverses all entries in the rule library with the same hash value, and precisely compares them with the binary stream of the original data one by one. The data that fails to match will trigger a false positive feedback, update the corresponding false positive rate statistics according to the Bloom filter level that triggered this match, mark the rule features, and the Bloom filter level with high-frequency false positives will automatically expand the bit array or replace the hash function combination.

[0038] Step 4: For the possibly matching data screened by hash matching, calculate the matching probability based on the time weight and historical matching frequency of the rule. At the same time, according to the set matching probability threshold, determine the probability of rule matching. Only the traffic data with a high probability is handed over to the regular matching module for complete matching.

[0039] The timing probability model verification method dynamically calculates the threat matching probability by quantifying the time sensitivity and historical credibility of the rule. The specific implementation process is divided into three stages: time weight calculation, historical frequency statistics, and comprehensive probability verification: First, the time weight calculation takes the formula as the core, where the current timestamp and the time of the last successful match of the rule The difference directly affects the weight attenuation amplitude, and the time decay coefficient It can be flexibly adjusted according to threat scenarios. Secondly, the historical matching frequency statistics record the proportion of valid trigger of rules through a long-time window. The calculation formula is F = K / M, where K represents the number of legal threat matches confirmed by the regular expression engine, and M is the total number of regular validations of all rules in the system. To avoid the problem of too small denominator for low-frequency rules, a smoothing factor is introduced. , and the actual frequency calculation is ; Finally, the comprehensive probability calculation is performed, and the time weight and the historical frequency F are linearly combined, and the weights are adjusted through coefficients , . Finally, it is compressed to the probability interval through the Sigmoid function, and at the same time, the final probability value is obtained by superimposing the time interval adjustment factor .

[0040] Step 5, the regular matching module only performs rule verification on high-probability input data, verifies whether the data segment completely conforms to the preset rules, and at the same time optimizes the parameters of each module according to the matching results.

[0041] Figure 3 Figure 5 shows the specific process of Step 5, including the following steps: Step 5-1, cache the data stream confirmed by the time-frequency weighted probability verification module, and adopt a sliding window method to extract every two consecutive characters as a group. Subsequently, according to the character table acceptable by the preset regular expression, look up and match the extracted character pairs, filter out the set of regular expressions that can accept the character pairs at the same time, and determine the address of the regular expression to be activated according to the matching results; Step 5-2, query the block random access memory BRAM storing the specific matching information of the regular expression according to the regular expression address obtained in Step 5-1, and allocate the data read from the block random access memory BRAM to the matching functional unit, so as to construct the matching combination unit corresponding to the regular expression; Step 5-3, the constructed regular expression matching combination unit sequentially obtains the data to be matched from the data buffer, and at the same time manages the control signals according to the regular expression state machine read from the block random access memory BRAM to ensure that the matching functional unit executes in the correct order. The state machine will determine whether to continue matching, whether to return to the previous state, or whether to end the current matching process; Step 5-4, when the character matching unit and the quantifier processing unit complete the matching in sequence according to the requirements of the regular expression, the hardware system will output the matching results, and at the same time optimize the parameters of each module according to the matching results.

[0042] In step 5-1, the character set that the regular expression can receive refers to the set of all possible characters that a regular expression can match, which defines which characters conform to the regular rule. In this method, two consecutive characters in the buffer are extracted in each clock cycle through a sliding window, and these two consecutive characters are used to query the acceptable character set simultaneously to determine the regular expressions activated in that cycle. The specific operation process is as follows: In the first clock cycle, a batch of regular expressions are determined through the window extraction and table lookup mechanism, and these regular expressions are activated for matching. In each subsequent clock cycle, the subsequent character batches are continuously extracted, and the table lookup mechanism is used to query whether these characters match the activated regular expressions; if the matching results found in the current cycle include the regular expressions activated in the previous cycle, the data is directly transmitted to these activated regular matching modules for continuous matching operations; at the same time, if new regular expressions are found to match the current characters in the table lookup result, these new regular expressions are activated, and the matching data is transmitted to the corresponding matching modules; this process will be carried out in sequence. In each clock cycle, new regular expressions are continuously activated according to the table lookup result and the matching data is transmitted, ensuring parallel and continuous regular expression matching in hardware.

[0043] The matching functional units described in step 5-2 are independent matching functional units pre-built in the hardware design stage. These functional units are responsible for implementing different regular expression elements (characters, quantifiers, branches, etc.). The units themselves do not store any specific matching data, but preset the functions and behaviors, and only execute corresponding operations according to the input data and control signals. During the matching process, the hardware units dynamically load the specific content of the regular expressions from the BRAM and construct the matching combinations in real time according to the stored data. For example, when it is necessary to match the character 'a', the control unit reads the character from the BRAM and passes it to the character matching unit, and the latter can start executing the character matching operation after receiving the control signal.

[0044] Based on the above method, a specific embodiment of high-speed pattern matching based on FPGA in a network traffic environment is as follows: For the detection scenarios of malicious file download (malware.exe) and SQL injection attack (' OR 1=1 --), the rule library contains 5,000 rules. Among them, the high-frequency rule "malware.exe" (historical matching frequency 200 times / hour) is stored in the L1-level SRAM (delay 1 ns), and the low-frequency rule "OR 1=1" (initial frequency 5 times / day) is stored in the L3-level DDR4 (delay 50 ns); When the input data packet "GET / download?file=malware.exe" enters the device, it is first screened by the first-level Bloom filter. The keyword field malware.exe generates a bit vector index through hash function calculation. Using the index to query the bit vector, the result is all 1. It is verified by the Bloom filter and triggers the hash matching module for verification. The hash matching module constructs a hash tree for data chunking. The root hash completely matches the rule library, and the content verification is consistent bit by bit, which is determined as a valid match. Further, the time-frequency weighting module calculates the comprehensive probability P = 0.738 > the threshold 0.5 (time weight = 1, historical frequency F = 0.0398), and the data enters the regular matching module. The FPGA extracts the character pairs ma / al / lw through a sliding window, dynamically loads the regular state machine in the BRAM, and completes the exact match, and finally outputs a security alert.

[0045] Based on the same technical concept as the above method, an embodiment of the present invention also provides a high-speed pattern matching device based on FPGA, as Figure 4 shown, including the following modules: An extraction module, used to perform multi-dimensional feature extraction on the rule set to generate a feature vector including rule length, type, historical matching frequency, and information entropy; A clustering module, used to perform real-time clustering analysis on the feature vector to generate multiple rule categories; A Bloom filter module, used to quickly screen the input data to determine whether it may match; A hash matching module, used to quickly verify whether the data stream screened by the Bloom filter matches the known pattern; A probability verification module, calculating the correct matching probability of the hash matching module based on the time weight and historical matching frequency of the rule; A regular matching module, used to exactly match complex rules; A feedback module, used to analyze the regular matching result and dynamically adjust the Bloom filter parameters, hash table distribution, and probability formula coefficients; A matching result reporting module, used to summarize and analyze the matching information and report the final matching result.

[0046] It should be understood that a high-speed pattern matching device based on FPGA in an embodiment of the present invention can implement all the technical solutions in the above method embodiment. The functions of its respective functional modules can be specifically implemented according to the method in the above method embodiment. The specific implementation process can refer to the relevant descriptions in the above embodiments and will not be elaborated here.

[0047] The ultimate effect of a high-speed pattern matching device and method based on FPGA according to the present invention is as follows: collect data packets in the Ethernet, and through the method or device of the present invention, it is possible to perform matching discrimination on the patterns contained in the data packets, quickly identify matching rules, and report matching information.

[0048] The embodiment of the present invention further provides an FPGA device, which includes one or more 10G Ethernet interfaces; GT transceivers; and an FPGA chip. The FPGA device is configured to implement the steps of the above-mentioned high-speed pattern matching method based on FPGA when executed. The Ethernet interface is mainly responsible for the physical layer reception and transmission of network data. In the method, the functions of the Ethernet interface include continuously receiving network traffic and providing the original data of network data packets to subsequent processing modules. This is the primary step of traffic collection to ensure seamless data acquisition at the physical level. GT transceivers are usually used for high-speed data transmission. In the FPGA chip, the GT transceivers are responsible for processing high-speed serial data communication, including but not limited to Ethernet data. They can effectively receive data from the Ethernet interface, convert the data into a format that the FPGA can process, while maintaining data integrity and timing accuracy. The FPGA (Field Programmable Gate Array) chip is a highly flexible and configurable hardware device suitable for performing parallel data processing tasks. In the method, the functions of the FPGA chip include: buffering input data; clustering analysis rule sets; storing classified three-level rules; implementing multi-level Bloom filters; performing hash matching and probability verification on data; complete matching verification of regular expressions; and dynamically optimizing according to matching results.

[0049] Figure 5 It is a schematic structural diagram of an electronic device provided in an embodiment of the present application. The electronic device may specifically include: at least one processor, at least one memory, a power supply, a communication interface, an input / output interface, and a communication bus. Among them, the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the relevant steps in a high-speed pattern matching method based on FPGA disclosed in any of the foregoing embodiments. In addition, the electronic device in this embodiment may specifically be an electronic computer.

[0050] In this embodiment, the power supply is used to provide working voltage for each hardware device of the electronic device 2; the communication interface can create a data transmission channel between the electronic device and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of the present application, and no specific limitation is imposed here; the input / output interface is used to obtain external input data or output data to the outside, and its specific interface type can be selected according to specific application requirements, and no specific limitation is made here.

[0051] In addition, as a carrier for resource storage, the memory can be a read-only memory, a random access memory, a magnetic disk, an optical disc, etc. The resources stored thereon can include an operating system, computer programs, etc., and the storage method can be transient storage or permanent storage.

[0052] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a high-speed pattern matching method based on FPGA as described above are implemented.

[0053] The present invention provides a high-speed pattern matching device and method based on FPGA. There are many methods and ways to specifically implement this technical solution. The above description is only a preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention. Each component not clearly defined in this embodiment can be implemented by existing technologies.

Claims

1. A high-speed pattern matching device based on FPGA, characterized in that: It includes extraction module, clustering module, Bloom filter module, hash matching module, time-frequency weighted probability verification module, regular matching module, feedback module and matching result reporting module; The extraction module is used to perform multi-dimensional feature extraction on the rule set to generate a feature vector including rule length, type, historical matching frequency and information entropy; The clustering module is used to perform real-time clustering analysis on the feature vectors to generate multiple rule categories; The Bloom filter module is used to quickly screen the input data to determine whether a match is possible; The hash matching module is used to quickly verify whether the data stream filtered by the Bloom filter matches a known pattern; The time-frequency weighted probability verification module is used to calculate and verify the hash matching pass probability based on the time weight and historical matching frequency of the rule; The regular matching module is used to accurately match complex rules; The feedback module is used to analyze the regular matching results and dynamically adjust the Bloom filter parameters, hash table distribution and probability formula coefficients; The matching result reporting module is used to summarize and analyze the matching information and report the final matching result.

2. A high-speed pattern matching method based on FPGA implemented by the device as claimed in claim 1, characterized in that: The following steps are involved: Step 1: Perform multi-dimensional feature extraction on the network rule set to generate a feature vector containing rule length, type, historical matching frequency, and information entropy. Use a hardware-accelerated clustering module to perform real-time clustering analysis on the feature vector to generate rule categories. According to the clustering results, the rules are dynamically mapped to the three-level storage structure of the FPGA. A hierarchical Bloom filter is configured for each level of the structure. By real-time monitoring of the matching indicators of each storage level, rule migration and Bloom filter parameter adjustment are dynamically triggered. Step 2: The network traffic data is screened through three levels of Bloom filters in turn to determine whether the input data fragment may match the rule. If any level of Bloom filter determines that it may match, the hash matching module is triggered; otherwise, the data is directly discarded; Step 3, the hash matching module is used to match the data that the Bloom filter determines as a possible match, and the filtered data is verified to see if it matches. If a match is determined, it is verified by the time-frequency weighted probability verification module. If no match occurs, it is marked as a false alarm and the false alarm rate is updated. Step 4: For the possible matching data screened in step 3, the matching probability is calculated based on the time weight of the rule and the historical matching frequency. At the same time, the matching probability threshold is set according to the system resource load and real-time traffic characteristics, and the traffic data above the matching probability threshold is sent to the regular matching module for complete matching; Step 5: The regular matching module performs rule verification on the traffic data that is higher than the matching probability threshold to verify whether the data segment fully complies with the preset rules, and optimizes the module parameters based on the matching result feedback.

3. The method according to claim 2, characterized in that In step 1, the hardware accelerated clustering module performs the following steps: Step 1-1, feature extraction: Use two or more parallel computing units to preprocess the preset rule set using hardware acceleration to generate a multi-dimensional feature vector ,in is the length of the i-th rule, is the type of the i-th rule, is the historical matching frequency of the ith rule, is the information entropy of the i-th rule; Step 1-2, clustering calculation: Use the Manhattan distance formula to calculate the distance between the feature vector of each rule and the current cluster center: ， in represents the Manhattan distance between the feature vector of the i-th rule and the cluster center, represents the eigenvalue of the dth dimension of the ith rule, Represents the cluster center coordinates of the kth cluster on the dth dimension; according to the Manhattan distance between the feature vector of each rule and the cluster center, the rule is assigned to the nearest cluster and a new cluster center is calculated. The updated cluster center will be used for distance calculation in the next cycle; Steps 1-3: Dynamic cluster adjustment, each cluster independently calculates the standard deviation of each dimension : ， in represents the eigenvalue of the ith rule in the dth dimension, is the cluster center coordinate of the current cluster in the dth dimension, and N is the total number of rules contained in the current cluster; When the standard deviation between rules within a cluster is greater than the threshold, it is determined that splitting is required. The splitting operation selects the dimension with the largest variance within the cluster for segmentation and forms two sub-clusters. When the centroid distance between two clusters is less than 0.5 times the average cluster spacing and the feature distribution similarity of the rules is greater than the threshold, the two clusters are merged into a new cluster and the cluster center is updated. Step 1-4, rule classification and storage mapping: According to the results of clustering calculation and cluster update, each rule is dynamically mapped to a three-level storage structure to optimize storage and rule matching efficiency.

4. The method according to claim 3, characterized in that In steps 1-4, the three-level storage structure includes L1 storage, L2 storage and L3 storage; wherein L1 storage is used to store high-frequency high-entropy rules, L2 storage is used to store medium-frequency rules, and L3 storage is used to store low-frequency rules; L1 storage meets any of the following rules: rule length ≤ 16 bytes and matching frequency > 80%; average information entropy of the cluster ≥ 1.5 and high consistency of rule types within the cluster; L2 storage meets any of the following rules: the rule length is between 16 and 64 bytes; the average matching frequency of the cluster is between 30% and 80%; Level 3 stores rules that meet all of the following conditions: average length of rules within a cluster > 64 bytes; matching frequency < 30%; average information entropy of the cluster ≤ 1.0; The storage tier is dynamically adjusted based on the matching frequency of the cluster rules. The specific rule migration conditions are: L1 rule downgrade: When a rule has a historical matching frequency stored at the L1 level The average characteristic distance between the cluster center and the current data stream decreases by more than 20% for three consecutive cycles or , it is downgraded to L2 storage, is the regular feature variance; L2 rule downgrade: when the rule history matches the frequency The information entropy is below the threshold of 30% or below the threshold for 5 consecutive cycles ; then downgrade to L3 storage; L2 rule upgrade: When the similarity between the L2 rule and the L1 rule feature distribution is greater than 0.85, the matching frequency If the growth rate exceeds 30% and the resource utilization rate is less than 70%, it will be upgraded to L1 storage; L3 rule upgrade: When triggered by burst traffic, it is upgraded to L2 storage.

5. The method according to claim 4, characterized in that In step 1, the Bloom filter parameter adjustment includes: Calculate the bit vector length: ， Where m is the length of the Bloom filter bit vector, n is the number of rules in the current storage level, and p is the target false alarm rate. is the frequency variation coefficient, Match the frequency change rate for the current period; Adjust the number of hash layers: ， Where k represents the number of hash function layers, is the variance sensitivity coefficient.

6. The method according to claim 5, characterized in that In step 3, the time-frequency weighted probability verification module performs the following steps: Calculate time weight: Calculate the time weight of the rule based on the difference between the current time and the last matching time of the rule : ， Among them, λ is the time attenuation coefficient, T is the current time, is the last matching time of the rule, e is a natural constant; Statistics of historical matching frequency: record the number of rule matches and the total number of matches, and calculate the matching frequency F of the rule: ， Where K represents the number of rule matches confirmed by the regular matching module, and M is the total number of regular verifications of all rules in the device. Prevent frequencies from being zero for the smoothing factor; Calculate the comprehensive probability: Calculate the matching probability by time weight and historical matching frequency : ， in, is the time weight coefficient, is the historical frequency coefficient, is the time interval adjustment coefficient, and sigmoid is the activation function.

7. The method according to claim 6, characterized in that In step 5, the regular matching module decomposes the regular expression matching task into two or more independent hardware units by pre-building a functional area. The hardware units are responsible for different regular expression elements. During the matching process, the hardware units dynamically load the state transition relationship of the regular expression from the block random access memory BRAM, and build a matching combination in real time according to the state transition relationship. Specifically, the following steps are included: Step 5-1, cache the data stream confirmed by the time-frequency weighted probability verification module, and extract two consecutive characters as a group using a sliding window method, then, according to a preset regular expression acceptable character table, perform table lookup matching on the extracted character pairs, screen out a set of regular expressions that can accept character pairs at the same time, and determine the regular expression address to be activated based on the matching result; Step 5-2, according to the regular expression address obtained in step 5-1, query the block random access memory BRAM storing specific matching information of the regular expression, and assign the data read from the BRAM to the character matching function unit and the quantifier matching function unit to construct a regular expression matching combination unit, wherein the character matching function unit is responsible for processing the direct matching of a single character or a set of characters in the regular expression, and the quantifier matching function unit is responsible for processing the logic related to the quantifier in the regular expression, that is, controlling the number of repetitions of a character or a sub-pattern; Step 5-3, the regular expression matching combination unit sequentially obtains the to-be-matched data from the cached data stream, and manages the control signal according to the regular expression state machine read from the block random access memory BRAM, to ensure that the character matching function unit and the quantifier matching function unit are executed in the correct order, and the state machine determines whether to continue matching, whether to roll back to the previous state, or whether to end the current matching process; Step 5-4, when the character matching function unit and the quantifier matching function unit complete the matching in sequence according to the requirements of the regular expression state machine, the regular matching module outputs the matching result and optimizes the module parameters according to the matching result feedback.

8. The method according to claim 7, characterized in that Step 5-1 includes: In a first clock cycle, a batch of characters in an input data stream is cached and extracted, and then a pre-configured regular expression acceptable character table is queried based on the extracted character pairs through a table lookup mechanism to determine a regular expression that can accept the characters, and the regular expression is activated for matching; In each subsequent clock cycle, the next batch of characters in the input data stream is continuously extracted, and the characters are matched using a table lookup mechanism. For a character pair in each clock cycle, firstly, a check is made as to whether the table lookup result contains a regular expression that has been activated in the previous cycle. If the table lookup result contains an activated regular expression, the data of the character pair is directly passed to the activated regular expression matching combination unit to continue the matching operation. For a new regular expression contained in the table lookup result, the data of the new regular expression is read from a block random access memory (BRAM) to construct a new regular expression matching combination unit.

9. The method according to claim 8, characterized in that Step 5-4 includes: Step 5-4-1, real-time collection of the output results of the regular matching module, including the successfully matched rule identifier, matching timestamp and traffic feature data, as well as the false positive sample information of unsuccessfully matched; Step 5-4-2, dynamically adjust the rule storage level according to the output result of step 5-4-1. When the rule meets the rule upgrade condition in the low-level storage level, migrate the rule to the high-level storage level and update the Bloom filter of the corresponding level; when the rule meets the downgrade condition in the high-level storage level, downgrade the rule to the low-level storage level and update the Bloom filter of the corresponding level; Step 5-4-3, based on the distribution characteristics of the false positive samples, reversely correct the hash function parameters and bit array layout of the Bloom filter, calculate the hash collision points of the false positive samples, generate a bit correction mask, and locally reconstruct the Bloom filter bit vector that produces the false match; Step 5-4-4, iteratively update the cluster center coordinates and cluster boundaries.

10. An FPGA device, characterized in that: include: FPGA chip, one or more Ethernet interfaces; The Ethernet interface is used for external data transmission. One or more programs are stored in the FPGA chip. When the programs are executed by the processor, the steps of the method described in claim 2 are implemented.

Images