A High-Speed Pattern Matching Device and Method Based on FPGA

Abstract

The present invention provides a high-speed pattern matching device and method based on FPGA. The device includes: an extraction module for performing multi-dimensional feature extraction on a rule set; a clustering module for generating rule categories; a Bloom filter module for screening input data; a hash matching module for verifying whether a data stream matches a known pattern; a probability verification module for calculating the probability of passing the verified hash match; a regular matching module for matching complex patterns; a feedback module for dynamically adjusting the parameters of the Bloom filter, the distribution of the hash table, and the coefficients of the probability formula; and a matching result reporting module for reporting matching information. The present invention can achieve a balance among efficiency, accuracy, and scalability in multi-pattern matching tasks, improve the resource utilization efficiency of FPGA, and meet the matching requirements when the pattern matching rule set is relatively large.

Description

Technical Field

[0001] The present invention relates to the field of network traffic processing and hardware acceleration computing, and particularly relates to a high-speed pattern matching device and method based on FPGA. Background Art

[0002] With the continuous increase of Internet applications and the complexity of application layer protocols, the traditional state detection technology based on network packet headers can no longer meet the current network security processing requirements. Therefore, the deep detection technology based on network packet payloads has emerged and has become the key component and core technology of devices such as service-aware routers, deep detection firewalls, network intrusion detection systems, network intrusion prevention systems, and unified threat management. With the proposal of concepts such as software-defined networks and the increasing popularity of cloud computing, pattern matching faces requirements such as complex rule features, a large number of rules, rapid rule updates, high throughput, low latency, and low power consumption in programmable network transmission nodes and service nodes. Therefore, combining the latest progress of current software and hardware platforms to study new high-performance pattern matching algorithms is of great significance for the development and promotion of high-performance network security detection and content awareness and other devices.

[0003] In the current technical context, pattern matching tasks are widely used in fields such as network security, data compression, and bioinformatics. Traditional pattern matching methods are mainly implemented based on software, such as string matching algorithms driven by the CPU (e.g., KMP, Boyer-Moore) or multi-pattern matching optimization schemes. Although these methods perform well in processing medium and small-scale data, when faced with massive data or high-throughput scenarios, problems such as high resource occupancy and weak parallel processing capabilities gradually emerge, resulting in the matching efficiency being difficult to meet real-time requirements. For example, the invention "A Multi-Pattern Matching Method, Device, Electronic Device, and Storage Medium (CN 113377917 B)" proposes that by symbolizing the text to be matched and converting it into a symbol sequence, the regular expression can be converted into a symbol sequence, and then the symbol sequence is sequentially matched on the trie tree, and different matching strategies are adopted for different symbol types for matching. Another patent, "FPGA-Based Network Traffic String Automatic Matching Method and Matching Device (CN 117574178 B)", adopts a hardware acceleration scheme. By parallelizing the processing of data streams through FPGA, the throughput is significantly improved. However, its pattern rule update requires re-burning the hardware logic, with poor flexibility and difficulty in adapting to the requirements of a dynamically changing rule library. Although the prior art has improved the pattern matching efficiency in different dimensions, it still faces three challenges: First, the software solution is restricted by the CPU computing power and is difficult to break through the real-time bottleneck; second, the hardware acceleration solution lacks flexibility and is difficult to support dynamic rule updates; third, although the distributed architecture alleviates the single-point pressure, it introduces additional complexity and communication costs. Especially in scenarios such as encrypted traffic detection and genomic sequence alignment, there is an urgent need for a pattern matching method with high throughput, low latency, and dynamic adaptability to provide underlying support for real-time data processing in complex environments. Summary of the Invention

[0004] Object of the Invention: The technical problem to be solved by the present invention is to provide a high-speed pattern matching method and device based on FPGA (Field Programmable Gate Array) in view of the deficiencies of the prior art. By utilizing the parallelism, programmability, and pipeline characteristics of FPGA, through a three-level hierarchical processing architecture of Bloom filter, hash pre-screening, and regular expression engine, high-efficiency and low-latency large-scale rule matching is achieved at the FPGA hardware level, and in scenarios such as network security and stream data processing, high throughput, low false positive rate, and hardware resource conservation are taken into account, solving the performance bottleneck of traditional CPU / GPU solutions when processing massive rules.

[0005] The present invention first provides a high-speed pattern matching device based on FPGA, including an extraction module, a clustering module, a Bloom filter module, a hash matching module, a time-frequency weighted probability verification module, a regular matching module, a feedback module, and a matching result reporting module;

[0006] The extraction module is used to perform multi-dimensional feature extraction on the rule set to generate a feature vector containing rule length, type, historical matching frequency, and information entropy;

[0007] The clustering module is used to perform real-time clustering analysis on the feature vector to generate multiple rule categories;

[0008] The Bloom filter module is used to quickly screen the input data to determine whether it may match;

[0009] The hash matching module is used to quickly verify whether the data stream screened by the Bloom filter matches a known pattern;

[0010] The time-frequency weighted probability verification module is used to calculate and verify the probability of passing the hash match by calculating the time weight and historical matching frequency of the rule;

[0011] The regular matching module is used to accurately match complex rules;

[0012] The feedback module is used to analyze the regular matching results and dynamically adjust the parameters of the Bloom filter, the distribution of the hash table, and the coefficients of the probability formula;

[0013] The matching result reporting module is used to summarize and analyze the matching information and report the final matching result.

[0014] The present invention also provides a high-speed pattern matching method based on FPGA implemented by using the described device, including the following steps:

[0015] Step 1: Perform multi-dimensional feature extraction on the network rule set to generate a feature vector containing rule length, type, historical matching frequency, and information entropy. Use the hardware-accelerated clustering module to perform real-time clustering analysis on the feature vector to generate rule categories. Dynamically map the rules to the three-level storage structure of the FPGA (Field Programmable Gate Array) according to the clustering results. Configure a hierarchical Bloom filter for each level of the structure. Dynamically trigger rule migration and adjustment of Bloom filter parameters by monitoring the matching metrics of each storage level in real time;

[0016] Step 2: Screen the network traffic data through the three-level Bloom filter in sequence to determine whether the input data segment may match the rule. If any level of the Bloom filter determines that it may match, trigger the hash matching module; otherwise, directly discard the data;

[0017] Step 3: Match the data determined by the Bloom filter to be possibly matching through the hash matching module to verify whether the screened data matches. If a match is determined, verify it by the time-frequency weighted probability verification module. If no match occurs, mark it as a false alarm and update the false alarm rate;

[0018] Step 4: For the potentially matching data screened in Step 3, calculate the matching probability based on the time weight of the rule and the historical matching frequency. At the same time, set a matching probability threshold (usually 0.5 to 0.7) according to the system resource load and real-time traffic characteristics, and send the traffic data with a matching probability higher than the threshold to the regular matching module for exact matching;

[0019] Step 5: The regular matching module performs rule verification on the traffic data with a matching probability higher than the threshold, verifies whether the data segment completely conforms to the preset rules, and feeds back and optimizes the module parameters according to the matching results.

[0020] In Step 1, the hardware acceleration clustering module performs the following steps:

[0021] Step 1-1: Extract features: Through two or more parallel computing units, use hardware acceleration to preprocess the preset rule set to generate a multi-dimensional feature vector where L i is the length of the i-th rule, T i is the type of the i-th rule, f i is the historical matching frequency of the i-th rule, and E i is the information entropy of the i-th rule;

[0022] Step 1-2: Clustering calculation: Use the Manhattan distance formula to calculate the distance between the feature vector of each rule and the current cluster center:

[0023]

[0024] where D i represents the Manhattan distance between the feature vector of the i-th rule and the cluster center, V i,d represents the feature value of the i-th rule in the d-th dimension, and C k,d represents the cluster center coordinate in the d-th dimension of the k-th cluster; According to the Manhattan distance between the feature vector of each rule and the cluster center, allocate the rules to the nearest cluster, calculate the new cluster center, and the updated cluster center will be used for the distance calculation in the next cycle;

[0025] Step 1-3: Dynamic cluster adjustment, each cluster independently calculates the standard deviation σ of each dimension d :

[0026]

[0027] where V i,d represents the feature value of the i-th rule in the d-th dimension, C d is the cluster center coordinate of the current cluster in the d-th dimension, and N is the total number of rules contained in the current cluster;

[0028] When the standard deviation between rules within a cluster is greater than the threshold of 0.5, it is determined that splitting is required. The splitting operation selects the dimension with the largest variance within the cluster for splitting and forms two sub-clusters; when the centroid distance between two clusters is less than 0.5 times the average cluster spacing and the similarity of the feature distribution of the rules is greater than the threshold of 0.9, the two clusters are merged into a new cluster and the cluster center is updated.

[0029] Steps 1 - 4, rule classification and storage mapping: According to the results of clustering calculation and cluster update, each rule is dynamically mapped to a three - level storage structure to optimize storage and rule matching efficiency.

[0030] In steps 1 - 4, the three - level storage structure includes L1 - level storage, L2 - level storage, and L3 - level storage; among them, L1 - level storage is used to store high - frequency and high - entropy rules, L2 - level storage is used to store medium - frequency rules, and L3 - level storage is used to store low - frequency rules;

[0031] Rules that satisfy any of the following for L1 - level storage: rule length ≤ 16 bytes and matching frequency > 80%; average information entropy of the cluster ≥ 1.5 and high consistency of rule types within the cluster (type standard deviation < 0.3);

[0032] Rules that satisfy any of the following for L2 - level storage: rule length in the range of 16 to 64 bytes; average matching frequency of the cluster between 30% and 80%;

[0033] Rules that satisfy all of the following conditions for L3 - level storage: average length of rules within the cluster > 64 bytes; matching frequency < 30%; average information entropy of the cluster ≤ 1.0;

[0034] Dynamically adjust the storage level according to the matching frequency of rules within the cluster. The specific rule migration conditions are:

[0035] Demotion of L1 - level rules: When the historical matching frequency f of a rule in L1 - level storage i decreases by more than 20% for 3 consecutive cycles or the distance d between the cluster center and the average feature of the current data stream avg > 1.5σ, then it is demoted to L2 - level storage, where σ is the rule feature variance;

[0036] Demotion of L2 - level rules: When the historical matching frequency f of the rule i is lower than the threshold of 30% for 5 consecutive cycles or the information entropy E i < 1.0; then it is demoted to L3 - level storage;

[0037] Promotion of L2 - level rules: When the similarity of the feature distribution between the rules in L2 - level and the rules in L1 - level is greater than 0.85, the matching frequency f i increases by more than 30% and the resource occupancy rate is less than 70%; then it is promoted to L1 - level storage;

[0038] L3 rule upgrade: If sudden traffic is triggered (the change in the rule history matching frequency is greater than 500%), it will be upgraded to L2-level storage.

[0039] In step 1, the adjustment of the Bloom filter parameters includes:

[0040] Calculate the length of the bit vector:

[0041]

[0042] where m is the length of the Bloom filter bit vector, n is the number of rules in the current storage level, p represents the target false positive rate, τ is the frequency change coefficient, and △f is the change rate of the matching frequency in the current period;

[0043] Adjust the number of hash layers:

[0044]

[0045] where k represents the number of hash function layers and β is the variance sensitivity coefficient.

[0046] In step 3, the time-frequency weighted probability verification module performs the following steps:

[0047] Calculate the time weight: Based on the difference between the current time and the last matching time of the rule, calculate the time weight W of the rule t :

[0048]

[0049] where λ is the time decay coefficient, T is the current time, T last is the last matching time of the rule, and e is the natural constant;

[0050] Statistical historical matching frequency: Record the number of times the rule is matched and the total number of matches, and calculate the matching frequency F of the rule:

[0051]

[0052] where K represents the number of times the rule is confirmed by the regular matching module, M is the total number of regular validations of all rules in the device, and ε is a smoothing factor to prevent the frequency from being zero;

[0053] Calculate the comprehensive probability: Calculate the matching probability P through the time weight and the historical matching frequency:

[0054]

[0055] where, is the time weight coefficient, μ is the historical frequency coefficient, γ is the time interval adjustment coefficient, and sigmoid is the activation function.

[0056] In step 5, the regular matching module disassembles the regular expression matching task into more than two independent hardware units by pre - constructing a functional area. The hardware units are responsible for different regular expression elements (characters, quantifiers, branches, etc.). During the matching process, the hardware units dynamically load the state transition relationship of the regular expression from the block random access memory (BRAM), and construct the matching combination in real - time according to the state transition relationship. The specific steps are as follows:

[0057] Step 5 - 1: Cache the data stream confirmed by the time - frequency weighted probability verification module, and adopt a sliding window method to extract every two consecutive characters as a group. Subsequently, according to the preset regular expression acceptable character table, look up and match the extracted character pairs, filter out the set of regular expressions that can accept both character pairs, and determine the address of the regular expression to be activated based on the matching result;

[0058] Step 5 - 2: According to the regular expression address obtained in step 5 - 1, query the block random access memory (BRAM) that stores the specific matching information of the regular expression, and allocate the data read from the BRAM to the character matching functional unit and the quantifier matching functional unit to construct a regular expression matching combination unit. The character matching functional unit is responsible for handling the direct matching of single characters or character sets in the regular expression, and the quantifier matching functional unit is responsible for handling the logic related to quantifiers in the regular expression, that is, controlling the repetition times of characters or sub - patterns;

[0059] Step 5 - 3: The regular expression matching combination unit sequentially obtains the data to be matched from the cached data stream, and at the same time manages the control signals according to the regular expression state machine read from the block random access memory (BRAM), ensuring that the character matching functional unit and the quantifier matching functional unit execute in the correct order. The state machine determines whether to continue the matching, whether to roll back to the previous state, or whether to end the current matching process;

[0060] Step 5 - 4: When the character matching functional unit and the quantifier matching functional unit complete the matching in sequence according to the requirements of the regular expression state machine, the regular matching module outputs the matching result, and at the same time feeds back and optimizes the module parameters according to the matching result.

[0061] Step 5 - 1 includes:

[0062] In the first clock cycle, cache and extract a batch of characters (such as two characters) from the input data stream, and then through a look - up mechanism, query the pre - configured regular expression acceptable character table based on the extracted character pairs, determine the regular expressions that can accept the characters, and activate the regular expressions for matching;

[0063] In each subsequent clock cycle, continue to extract the next batch of characters from the input data stream, and use a look-up table mechanism to match the characters. For each pair of characters in a clock cycle, first check whether the look-up table result contains the regular expressions that have been activated in the previous cycle. If the look-up table result contains the activated regular expressions, directly pass the data of the character pair to the activated regular expression matching combination unit and continue to perform the matching operation; for the new regular expressions contained in the look-up table result, read the data of the new regular expressions from the block random access memory (BRAM), and construct a new regular expression matching combination unit.

[0064] Step 5-4 includes:

[0065] Step 5-4-1: Real-time collect the output results of the regular matching module, including the rule identifiers that have been successfully matched, the matching timestamps, and the traffic feature data, as well as the false alarm sample information that has not been successfully matched.

[0066] Step 5-4-2: Dynamically adjust the rule storage hierarchy according to the output results of Step 5-4-1. When a rule meets the rule upgrade condition in the low-level storage layer, migrate the rule to the high-level storage layer and update the Bloom filter of the corresponding layer; when a rule meets the downgrade condition in the high-level storage layer, downgrade the rule to the low-level storage layer and update the Bloom filter of the corresponding layer.

[0067] Step 5-4-3: Based on the distribution characteristics of the false alarm samples, inversely correct the hash function parameters and the bit array layout of the Bloom filter, calculate the hash collision points of the false alarm samples, generate a bit correction mask, and perform local reconstruction on the bit vector of the Bloom filter that generates false matches.

[0068] Step 5-4-4: Iteratively update the cluster center coordinates and the cluster boundaries.

[0069] The present invention also provides an FPGA device, including: an FPGA chip and one or more Ethernet interfaces; the Ethernet interfaces are used for data transmission with the outside, and one or more programs are stored in the FPGA chip, and when the programs are executed by a processor, the steps of the method are implemented.

[0070] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) Existing software-based pattern matching systems are difficult to meet the requirements of dynamic rule updates and real-time matching in high-speed network environments, especially facing computational latency and storage bottlenecks in high-throughput scenarios. The present invention realizes real-time analysis of multi-dimensional feature vectors through an FPGA hardware-accelerated clustering module, and combines the dynamic migration mechanism of a three-level storage structure, reducing the rule matching latency to the nanosecond level. By adopting a hierarchical Bloom filter and hash matching collaborative architecture, the false positive rate is significantly reduced compared with traditional methods. Through the dynamic weight calculation of the timing probability model, the computational load of regular expression matching is greatly reduced, significantly improving the processing efficiency of network traffic.

[0071] (2) Traditional solutions have problems of storage fragmentation and low resource utilization when dealing with dynamic rule sets. The matrix storage structure based on hardware acceleration proposed by the present invention reduces the BRAM resource occupancy rate, and at the same time, by dynamically adjusting the Bloom filter parameters according to rule characteristics, the storage space utilization rate is improved. In particular, the adaptive migration strategy of the L1-L3 storage levels can improve the burst traffic processing ability, and the feedback optimization module reduces the false positive rate fluctuation during continuous operation of the system by real-time correcting the hash function parameters, significantly enhancing the stability and adaptability of the pattern matching system.

[0072] (3) The regular part of this solution has significant dynamic processing advantages. Through the sliding window mechanism and the look-up table method, it can efficiently perform regular expression matching and support parallel processing. Through the matching functional units pre-constructed in hardware, the system can dynamically load the matching information of regular expressions according to real-time data and reconfigure the matching strategy at runtime according to requirements. In particular, by modifying the data in BRAM, the reconfiguration of the regular matching module can be achieved, enabling the system to flexibly adapt to different matching requirements and traffic changes, thereby improving the matching efficiency and the scalability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 is a flowchart of the method of the present invention.

[0074] Figure 2 is a schematic diagram of the data flow in the method of the present invention.

[0075] Figure 3 is a flowchart of the operation of the regular expression matching module of the present invention.

[0076] Figure 4 is a schematic diagram of the structure of the high-speed pattern matching device based on FPGA of the present invention.

[0077] Figure 5 is a structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0078] The following further specifically describes the present invention in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become clearer.

[0079] Referring to Figure 1 , an embodiment of the present invention provides a high-speed pattern matching method based on FPGA. This method is implemented on FPGA, and its main steps are as follows: perform multi-dimensional feature extraction on the network rule set and generate feature vectors, use a hardware-accelerated clustering module to perform real-time clustering analysis on the feature vectors to generate multiple rule categories, dynamically map the rules to the three-level storage structure of FPGA according to the clustering results, configure a hierarchical Bloom filter for each level of structure, and input data is screened through the three-level Bloom filter in sequence to quickly determine whether the input data segment may match the rule. If any level of the filter determines "possible match", the hash matching module is triggered. The hash matching module matches the data determined to be a possible match to quickly verify whether the screened data matches. If a match is determined, it is verified by the probability verification module. If no match occurs, it is marked as a false alarm and the false alarm rate is updated. For the possible match data screened by the hash matching, calculate the matching probability based on the time weight of the rule and the historical matching frequency, and at the same time, according to the set matching probability threshold, determine the probability of rule matching. Only the high-probability traffic data is handed over to the regular matching module for complete matching. The regular matching module performs rule verification on the high-probability input data to verify whether the data segment completely conforms to the preset rule, and at the same time, feedback and optimize the parameters of each module according to the matching result.

[0080] As Figure 2 shown, in the FPGA implementation architecture of the present invention, after the data stream enters the system through the SFP optical fiber interface, it first passes through the L1-L3 level Bloom filters for feature pre-screening, then passes through the hash matching module for verification and matching, and finally passes through the regular expression module for precise matching. The data stream that does not match or is mis-matched during the process flows out through the optical fiber outlet, while the data that is verified to match flows out through the report channel.

[0081] In the embodiment of the present invention, the specific steps of the method are as follows:

[0082] Step 1: Perform multi-dimensional feature extraction on the network rule set and generate feature vectors including rule length, type, historical matching frequency, and information entropy. Use a hardware-accelerated clustering module to perform real-time clustering analysis on the feature vectors to generate multiple rule categories, dynamically map the rules to the three-level storage structure of FPGA according to the clustering results, configure a hierarchical Bloom filter for each level of structure, and dynamically trigger rule migration and Bloom filter parameter adjustment by real-time monitoring the matching metrics of each storage level;

[0083] According to the embodiment of the present invention, Step 1 includes the following steps:

[0084] Step 1-1: Perform multi-dimensional feature extraction on the network rule set. Specifically, multiple key attributes are extracted from each rule, including the rule length, rule type, historical matching frequency, and information entropy. By statistically analyzing and calculating these attributes, each rule is converted into a multi-dimensional feature vector, which can comprehensively reflect the performance characteristics of the rule during the actual matching process and provide data support for subsequent clustering analysis;

[0085] Step 1-2: Use the hardware-accelerated clustering module to perform real-time clustering on the previously generated feature vectors. This module quickly groups rules with similar features into one category, generating multiple rule categories;

[0086] Step 1-3: According to the clustering results, the system dynamically maps rules of different categories to the three-level storage structure of the FPGA. The three-level storage structure includes L1-level storage (for storing high-frequency and high-entropy rule clusters), L2-level storage (for storing medium-frequency rule clusters), and L3-level storage (for storing low-frequency rule clusters). Among them, frequently used rules are preferentially mapped to the faster storage layer, while infrequently used or low-frequency rules are stored in the larger-capacity storage layer;

[0087] Step 1-4: Configure a hierarchical Bloom filter for each level of the storage structure according to the data in the storage, and customize the parameters of the Bloom filter according to the characteristics of each storage layer and rule categories to minimize the false positive rate and improve the search speed;

[0088] Step 1-5: Continuously track the matching metrics of each storage level, such as key performance parameters like matching latency and hit rate. When it is detected that the performance metrics of a certain level deviate from the preset threshold, the system will automatically trigger rule migration, redistribute some rules among the storage layers to achieve load balancing, and dynamically adjust the parameters of each level of Bloom filter according to real-time data to ensure that the pre-filtering effect is always in the optimal state.

[0089] The feature extraction described in Step 1-1 refers to preprocessing the preset rule set through multiple parallel computing units using hardware acceleration to generate multi-dimensional feature vectors where L i is the rule length, T i is the rule type, f i is the historical matching frequency, E i is the information entropy of the rule. The implementation method for extracting each feature is as follows:

[0090] (1) Rule length L iDefined as the number of characters or bytes in a rule definition, and its extraction method is implemented through a hardware counter. Specifically, the rule string is input into the FPGA in parallel, and an adder is used to accumulate the number of characters bit by bit or byte by byte until the rule is processed. For example, for the rule "abc.*def", the system counts its number of characters as 8, thus obtaining L i = 8. The implementation of this method depends on the parallel architecture of the FPGA, and the processing delay is optimized through pipeline design. Although its time complexity is O(n), where n is the rule length, with the support of hardware acceleration, the actual execution time is significantly shortened, ensuring high efficiency.

[0091] (2) Rule type f i Used to classify according to the matching mechanism of the rule, such as exact string matching, regular expression, or other specific types. Its extraction method is based on parsing the rule syntax and judging the structural characteristics of the rule through predefined classification criteria. In specific implementation, after the rule data is input into the FPGA, its pattern characteristics are analyzed through a lookup table or a parallel comparator. For example, the rule "abc" is classified as an exact match, while "a.*b" is classified as a regular expression. The classification mapping rules are pre-loaded in the LUT, enabling the type determination to be completed in constant time with a time complexity of O(1).

[0092] (3) Historical matching frequency T i Reflects the frequency at which the rule has been successfully matched in the past and is an important indicator for evaluating the activity of the rule. Its extraction method is to maintain a counter for each rule to record the number of matches. In specific implementation, the counter is stored in the register or block RAM of the FPGA, and each time the rule is successfully matched, the count is incremented by an adder. The system can record the absolute number of matches. For example, if a certain rule is matched 100 times in one hour, then fi = 100; or the relative frequency can be calculated through the ratio of the number of matches to the total number of matches or the number of matches within a time window. To achieve real-time update, the counters of multiple rules support parallel operations, and the time complexity of a single update is O(1). If the frequency needs to be normalized, it is calculated periodically in the background with a complexity of O(n), where n is the total number of rules.

[0093] (4) Information entropy E i Through the Shannon entropy formula E i = -∑ xThe calculation of p(x)log2p(x) is used to measure the randomness and complexity of the rule content, where p(x) represents the occurrence probability of symbol x in the rule. The extraction process is divided into the following steps: First, use the hardware counter to count the occurrence times of each unique symbol in the rule; then, divide the occurrence times of each symbol by the total length of the rule to calculate the probability p(x); finally, calculate p(x)log2p(x) for each item of p(x), and sum through the accumulator and take the negative value. Since FPGA is not good at floating-point operations, a lookup table is used in the system to approximate the logarithmic function to accelerate the calculation. For example, the entropy of the rule "aaaaa" is 0 (all characters are the same), while the entropy of "abcde" is log25≈2.32 (each character is unique). In the FPGA, parallel counters and accumulators are allocated for each rule, and the time complexity is O(n + k), where n is the rule length and k is the number of unique symbols.

[0094] In step 1-2, the clustering module uses the Manhattan distance formula Calculate the distance between the feature vector of each rule and the current cluster center, and assign each rule to the nearest cluster according to the calculated distance of each rule. At the same time, calculate the new cluster center for the distance calculation in the next cycle. The clustering module implements a dynamic cluster adjustment mechanism, and each cluster independently calculates the standard deviation of each dimension When the standard deviation between the rules within the cluster is greater than the threshold, it is considered that the rule feature distribution within the cluster is relatively dispersed and needs to be split. The splitting operation selects the dimension with the largest variance within the cluster for splitting and forms two sub-clusters. When the centroid distance between two clusters is less than 0.5 times the system average cluster spacing and the similarity of the rule feature distribution is greater than 0.9, merge the two clusters into a new cluster and update its cluster center.

[0095] According to the results of clustering calculation and cluster update, the system will dynamically map each rule to a three-level storage structure to optimize the storage and rule matching efficiency. In the three-level storage structure, the L1-level storage is used to store high-frequency and high-entropy rule clusters, the L2-level storage is used to store medium-frequency rule clusters, and the L3-level storage is used to store low-frequency rule clusters.

[0096] Two core improvements have been made to the parameter design of the traditional Bloom filter in Steps 1-4 to address the problem of insufficient adaptability of the static model in a dynamic network environment: The first improvement targets the calculation model of the bit vector length, introducing a traffic frequency change correction term (τ·△f) into the classical formula, where △f represents the change rate of the matching frequency in the current period, and τ is the frequency change coefficient. This correction term enables the bit vector capacity to adaptively adjust by dynamically perceiving changes in traffic intensity. When traffic surges, the bit vector is expanded to suppress fluctuations in the false positive rate, and when traffic is stable, it maintains the theoretical optimal value, thus balancing the requirements of storage efficiency and stability. The second improvement focuses on the hash layer adjustment model, adding a feature variance compensation factor 1+β·σ to the traditional formula, where σ is the variance of the regular feature distribution and β is the variance sensitivity coefficient. This compensation factor dynamically adjusts the hash calculation complexity by quantifying the distribution characteristics of the rule set, increasing the number of hash layers for high-variance rule sets with dispersed features to reduce the collision probability, and reducing the number of hash layers for low-variance rule sets with concentrated features to optimize the calculation efficiency.

[0097] The two parameter improvements form a collaborative mechanism, which optimizes the technical defects of rigid resource allocation and unstable false positive rate of the static model in a dynamic network environment by responding in real time to traffic dynamics and rule distribution characteristics. The specific formulas are as follows:

[0098] (1) Bit vector length calculation formula:

[0099]

[0100] where m is the length of the Bloom filter bit vector, n is the number of rules in the current storage level, p represents the target false positive rate, and △f is the change rate of the matching frequency in the current period;

[0101] (2) Hash layer adjustment formula:

[0102]

[0103] where k represents the number of hash function layers.

[0104] In Step 2, the input data is screened through a three-level Bloom filter in sequence to quickly determine whether the input data segment is likely to match the rules. If any level of the filter determines "possible match", the hash matching module is triggered; otherwise, the data is directly discarded.

[0105] The specific implementation process of Step 2 is as follows: The input data first enters the first-level Bloom filter (L1-BF). The hash function is used to calculate the data key field and query the bit array. If all bits are 1, it is determined as "possibly matching", and the subsequent screening process is immediately terminated, and the data is directly sent to the hash matching module for precise verification. If any bit is 0, it is determined as "not matching", and the same operation is repeated in the second-level Bloom filter (L2-BF). If L2-BF determines "possibly matching", the hash matching module is also immediately triggered. If there is no match, it continues to be passed to the third-level Bloom filter (L3-BF). Finally, if L3-BF determines "possibly matching", the data enters the hash matching module. If all three-level filters determine "not matching", the data is directly discarded. In this design, each-level filter makes independent judgments and has the ability to trigger immediately, avoiding the delay caused by multi-level cascading, and ensuring that the preliminary matching at any level can quickly start precise verification. At the same time, the false positive rate statistics are separately recorded according to the triggering level (for example, the false positives triggered by L1 are only accumulated to the L1 statistics item), supporting subsequent dynamic parameter optimization for different levels. This process significantly improves the processing efficiency, especially suitable for quickly screening out irrelevant data and reducing the computational load in high-frequency traffic scenarios. By using the FPGA parallel pipeline structure, the data stream is screened through the Bloom filter level by level, effectively improving the efficiency and response speed of the system in processing input data. By pre-using the Bloom filter in each storage level for quick possibility judgment, the system can quickly filter out most of the irrelevant data in large-scale rule matching scenarios, thus saving computational resources and processing time.

[0106] Step 3: The hash matching module matches the data determined as possibly matching by the Bloom filter, quickly verifying whether the screened data matches. If a match is determined, it is verified by the probability verification module. If no match occurs, it is marked as a false positive and the false positive rate is updated.

[0107] As the core verification link of the system, the hash matching module performs strict matching of all data after the preliminary screening by the Bloom filter. The process is as follows:

[0108] Step 3-1: By preprocessing the data to be matched and dividing the whole content into blocks, the input data is completely retained and converted into a standardized format, including the packet header, payload, and tail information. The data is dynamically segmented into multiple logical blocks according to the protocol semantics. Each block independently calculates the hash value, and a hierarchical hash tree covering the overall data is constructed.

[0109] Step 3-2: Perform hash matching on all the data in the rule library. The rule library pre-stores the complete hash tree structure of the data to be matched. During matching, the system compares the hash tree nodes of the data with the target hash values in the rule library level by level. First, it preferentially compares the root hash of the data with the root hash of the rule. If they are the same, it directly determines a complete match. If they are different but partial matching is allowed by the rule, it further verifies the specified sub-block hash values. When the rule needs to verify a specific block, the system only needs to compare the corresponding block hash chain and perform bit-level verification in combination with the binary content of the original data to avoid misjudgment caused by hash conflicts.

[0110] Step 3-3: If the hash values match but the content does not match (conflict scenario), the system starts a forced verification mechanism. It traverses all the entries in the rule library with the same hash value and precisely compares them one by one with the binary stream of the original data. The data that fails to match will trigger a false alarm feedback. According to the Bloom filter level that triggered this match, the corresponding false alarm rate statistics are updated, and the rule features are marked. The Bloom filter level with a high frequency of false alarms will automatically expand the bit array or replace the hash function combination.

[0111] Step 4: For the potentially matching data selected through hash matching, calculate the matching probability based on the time weight and historical matching frequency of the rule. At the same time, according to the set matching probability threshold, determine the probability of rule matching. Only the traffic data with a high probability is handed over to the regular matching module for complete matching.

[0112] The time-series probability model verification method dynamically calculates the threat matching probability by quantifying the time sensitivity and historical credibility of the rule. The specific implementation process is divided into three stages: time weight calculation, historical frequency statistics, and comprehensive probability verification.

[0113] First, the time weight calculation takes the formula as the core, where the difference between the current timestamp T and the last successful matching time T last of the rule directly affects the weight attenuation amplitude, and the time decay coefficient λ can be flexibly adjusted according to the threat scenario. Secondly, the historical matching frequency statistics records the effective trigger ratio of the rule through a long-time window. The calculation formula is where K represents the number of rule matches confirmed by the regular matching module for the rule, M is the total number of regular verifications of all rules in the system. To avoid the problem of too small a denominator for low-frequency rules, a smoothing factor ∈ = 1 is introduced, and the actual frequency calculation is F = (K + ∈) / (M + ∈). Finally, the comprehensive probability calculation linearly combines the time weight W t and the historical frequency F, adjusts the weight through the coefficient μ, and finally compresses it to the probability interval through the Sigmoid function. At the same time, the final probability value is obtained by superimposing the time interval adjustment factor .

[0114] Step 5, the regular matching module only performs rule verification on high-probability input data, verifies whether the data segment fully conforms to the preset rules, and feedbacks and optimizes the parameters of each module according to the matching results.

[0115] Figure 3 The specific process of Step 5 is shown, including the following steps:

[0116] Step 5-1, perform caching processing on the data stream confirmed by the time-frequency weighted probability verification module, and adopt a sliding window method to extract every two consecutive characters as a group. Subsequently, according to the preset regular expression acceptable character table, look up and match the extracted character pairs, filter out the set of regular expressions that can accept the character pairs simultaneously, and determine the address of the regular expression to be activated based on the matching results;

[0117] Step 5-2, query the block random access memory BRAM storing the specific matching information of the regular expression according to the regular expression address obtained in Step 5-1, and allocate the data read from the block random access memory BRAM to the matching functional unit, thereby constructing the matching combination unit corresponding to the regular expression;

[0118] Step 5-3, the constructed regular expression matching combination unit sequentially obtains the data to be matched from the data buffer, and at the same time manages the control signals according to the regular expression state machine read from the block random access memory BRAM, ensuring that the matching functional unit executes in the correct order. The state machine will decide whether to continue matching, whether to roll back to the previous state, or whether to end the current matching process;

[0119] Step 5-4, when the character matching unit and the quantifier processing unit complete the matching in sequence according to the requirements of the regular expression, the hardware system will output the matching result, and at the same time optimize the parameters of each module according to the matching result.

[0120] In step 5-1, the character set that the regular expression can receive refers to the set of all possible characters that a regular expression can match, which defines which characters conform to the regular rule. In this method, two consecutive characters in the buffer are extracted in each clock cycle through a sliding window, and these two consecutive characters are used to query the acceptable character set simultaneously to determine the regular expressions activated in that cycle. The specific operation process is as follows: In the first clock cycle, a batch of regular expressions are determined through the window extraction and table lookup mechanism, and these regular expressions are activated for matching. In each subsequent clock cycle, the subsequent character batches are continuously extracted, and the table lookup mechanism is used to query whether these characters match the activated regular expressions. If the matching results found in the current cycle include the regular expressions activated in the previous cycle, the data is directly transmitted to these activated regular matching modules for continued matching operations. At the same time, if new regular expressions are found in the table lookup results that match the current characters, these new regular expressions are activated, and the matching data is transmitted to the corresponding matching modules. This process will be carried out sequentially. In each clock cycle, new regular expressions are activated according to the table lookup results and the matching data is transmitted to ensure parallel and continuous regular expression matching in hardware.

[0121] The matching functional units described in step 5-2 are independent matching functional units pre-built during the hardware design phase. These functional units are responsible for implementing different regular expression elements (characters, quantifiers, branches, etc.). The units themselves do not store any specific matching data, but instead have preset functions and behaviors, and only execute corresponding operations based on input data and control signals. During the matching process, the hardware units dynamically load the specific content of the regular expressions from the BRAM and construct matching combinations in real time according to the stored data. For example, when it is necessary to match the character 'a', the control unit reads this character from the BRAM and passes it to the character matching unit, which can start executing the character matching operation after receiving the control signal.

[0122] Based on the above method, a specific embodiment of high-speed pattern matching based on FPGA in a network traffic environment is as follows:

[0123] For the detection scenarios of malicious file download (malware.exe) and SQL injection attack ('OR 1=1--), the rule library contains 5,000 rules. Among them, the high-frequency rule "malware.exe" (historical matching frequency: 200 times / hour) is stored in the L1-level SRAM (delay 1 ns), and the low-frequency rule "OR 1=1" (initial frequency: 5 times / day) is stored in the L3-level DDR4 (delay 50 ns);

[0124] When the input data packet "GET / download?file=malware.exe" enters the device, it is first screened by the first-level Bloom filter. The keyword field malware.exe is calculated by the hash function to generate a bit vector index. Using the index to query the bit vector, the results are all 1. It is verified by the Bloom filter and the hash matching module is triggered for verification. The hash matching module constructs a hash tree for the data in chunks. The root hash completely matches the rule library, and the content verification is consistent bit by bit, which is determined as a valid match. Further, the time-frequency weighting module calculates the comprehensive probability P = 0.738 > the threshold 0.5 (the time weight W t = 1, the historical frequency F = 0.0398), and the data enters the regular matching module. The FPGA extracts the character pairs ma / al / lw through the sliding window, dynamically loads the regular state machine in the BRAM, and completes the exact matching of.*malware\.exe.*, and finally outputs a security warning.

[0125] Based on the same technical concept as the above method, an embodiment of the present invention further provides a high-speed pattern matching device based on FPGA, as Figure 4 shown, including the following modules:

[0126] An extraction module, used to perform multi-dimensional feature extraction on the rule set to generate a feature vector including rule length, type, historical matching frequency, and information entropy;

[0127] A clustering module, used to perform real-time clustering analysis on the feature vector to generate multiple rule categories;

[0128] A Bloom filter module, used to quickly screen the input data to determine whether it may match;

[0129] A hash matching module, used to quickly verify whether the data stream screened by the Bloom filter matches the known pattern;

[0130] A probability verification module, used to calculate the correct matching probability of the hash matching module based on the time weight and historical matching frequency of the rule;

[0131] A regular matching module, used to exactly match complex rules;

[0132] A feedback module, used to analyze the regular matching result and dynamically adjust the parameters of the Bloom filter, the distribution of the hash table, and the coefficients of the probability formula;

[0133] A matching result reporting module, used to summarize and analyze the matching information and report the final matching result.

[0134] It should be understood that a high-speed pattern matching device based on FPGA in the embodiments of the present invention can implement all the technical solutions in the above method embodiments. The functions of its respective functional modules can be specifically implemented according to the methods in the above method embodiments, and the specific implementation process can refer to the relevant descriptions in the above embodiments, which will not be elaborated here.

[0135] The final effect of a high-speed pattern matching device and method based on FPGA of the present invention is as follows: collecting data packets in the Ethernet, and through the method or device of the present invention, it is possible to perform matching discrimination on the patterns contained in the data packets, quickly identify the matching rules, and report the matching information.

[0136] The embodiments of the present invention further provide an FPGA device, which includes one or more 10G Ethernet interfaces; GT transceivers; and an FPGA chip. The FPGA device is configured to implement the steps of the above-mentioned FPGA-based high-speed pattern matching method when executed. The Ethernet interface is mainly responsible for the physical layer reception and transmission of network data. In the above method, the functions of the Ethernet interface include continuously receiving network traffic and providing the original data of network data packets to subsequent processing modules. This is the primary step of traffic collection to ensure seamless data acquisition at the physical level. GT transceivers are usually used for high-speed data transmission. In the FPGA chip, the GT transceivers are responsible for processing high-speed serial data communication, including but not limited to Ethernet data. They can very effectively receive data from the Ethernet interface, convert the data into a format that the FPGA can process, while maintaining the integrity and timing accuracy of the data. The FPGA (Field Programmable Gate Array) chip is a highly flexible and configurable hardware device suitable for performing parallel data processing tasks. In the above method, the functions of the FPGA chip include: buffering input data; clustering analysis of rule sets; storing the classified three-level rules; implementing multi-level Bloom filters; performing hash matching and probability verification on data; complete matching verification of regular expressions; and dynamically optimizing according to the matching results.

[0137] Figure 5 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. The electronic device may specifically include: at least one processor, at least one memory, a power supply, a communication interface, an input / output interface, and a communication bus. Among them, the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the relevant steps in a high-speed pattern matching method based on FPGA disclosed in any of the foregoing embodiments. In addition, the electronic device in this embodiment may specifically be an electronic computer.

[0138] In this embodiment, the power supply is used to provide operating voltages for the various hardware devices of the electronic device 2; the communication interface can create a data transmission channel between the electronic device and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and specific limitations thereof are not provided herein; the input / output interface is used to obtain external input data or output data to the outside, and the specific interface type can be selected according to specific application requirements, and specific limitations thereof are not provided herein.

[0139] In addition, the memory, as a carrier for resource storage, can be a read-only memory, a random access memory, a magnetic disk, an optical disk, etc., and the resources stored thereon can include an operating system, a computer program, etc., and the storage method can be transient storage or permanent storage.

[0140] The present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of a high-speed pattern matching method based on FPGA as described above are implemented.

[0141] The present invention provides a high-speed pattern matching device and method based on FPGA. There are many methods and ways to specifically implement this technical solution. The above description is only a preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention. Each component not clearly defined in this embodiment can be implemented using existing technologies.

Claims

1. A high-speed pattern matching device based on FPGA, characterized in that, It includes an extraction module, a clustering module, a Bloom filter module, a hash matching module, a time-frequency weighted probability verification module, a regular matching module, a feedback module, and a matching result reporting module; The extraction module is used to perform multi-dimensional feature extraction on the rule set to generate a feature vector including rule length, type, historical matching frequency, and information entropy; The clustering module is used to perform real-time clustering analysis on the feature vector to generate multiple rule categories; The Bloom filter module is used to quickly screen the input data to determine whether it may match; adjust the Bloom filter parameters, including: Calculate the bit vector length: where m is the bit vector length of the Bloom filter, n is the number of rules in the current storage level, p represents the target false positive rate, τ is the frequency change coefficient, and △f is the matching frequency change rate in the current period; Adjust the number of hash layers: where k represents the number of hash function layers, β is the variance sensitivity coefficient; σ is the rule feature variance; The hash matching module is used to quickly verify whether the data stream screened by the Bloom filter matches the known pattern; The time-frequency weighted probability verification module is used to calculate and verify the probability of passing the hash match based on the time weight and historical matching frequency of the rule; the time-frequency weighted probability verification module performs the following steps: Calculate the time weight: Based on the difference between the current time and the last matching time of the rule, calculate the time weight W of the rule t : where λ is the time decay coefficient, T is the current time, T last is the last matching time of the rule, and e is the natural constant; Statistical historical matching frequency: Record the number of times the rule is matched and the total number of matches, and calculate the matching frequency F of the rule: where K represents the number of times the rule is confirmed to be matched by the regular matching module, M is the total number of regular validations of all rules in the device, and ε is a smoothing factor to prevent the frequency from being zero; Calculate the comprehensive probability: Calculate the matching probability P through the time weight and historical matching frequency; Among them, is the time weight coefficient, μ is the historical frequency coefficient, γ is the time interval adjustment coefficient, and sigmoid is the activation function; The regular matching module is used to accurately match complex rules; The feedback module is used to analyze the regular matching results and dynamically adjust the Bloom filter parameters, hash table distribution, and probability formula coefficients; The matching result reporting module is used to summarize and analyze the matching information and report the final matching result.

2. A high-speed pattern matching method based on FPGA implemented by using the device as described in claim 1, characterized in that, It includes the following steps: Step 1: Perform multi-dimensional feature extraction on the network rule set to generate a feature vector including rule length, type, historical matching frequency, and information entropy. Use the hardware-accelerated clustering module to perform real-time clustering analysis on the feature vector to generate rule categories. Dynamically map the rules to the three-level storage structure of the FPGA according to the clustering results. Configure a hierarchical Bloom filter for each level of structure. Dynamically trigger rule migration and Bloom filter parameter adjustment by real-time monitoring the matching metrics of each storage level; Step 2: The network traffic data is screened through the three-level Bloom filter in turn to determine whether the input data segment may match the rule. If any level of the Bloom filter determines that it may match, trigger the hash matching module; otherwise, directly discard the data; Step 3: Use the hash matching module to match the data determined by the Bloom filter to be possibly matched to verify whether the screened data matches. If a match is determined, the time-frequency weighted probability verification module is used for verification. If no match occurs, it is marked as a false positive and the false positive rate is updated; Step 4: For the potentially matching data selected in Step 3, calculate the matching probability based on the time weight of the rule and the historical matching frequency. At the same time, set the matching probability threshold according to the system resource load and real-time traffic characteristics, and send the traffic data higher than the matching probability threshold to the regular matching module for exact matching; Step 5: The regular matching module performs rule verification on the traffic data higher than the matching probability threshold to verify whether the data segment completely conforms to the preset rules, and feedbacks and optimizes the module parameters according to the matching results.

3. The method according to claim 2, characterized in that, In Step 1, the hardware acceleration clustering module performs the following steps: Step 1-1, feature extraction: Through two or more parallel computing units, use hardware acceleration to preprocess a preset rule set to generate a multi-dimensional feature vector where L i is the length of the i-th rule, T i is the type of the i-th rule, f i is the historical matching frequency of the i-th rule, E i is the information entropy of the i-th rule; Step 1-2: Clustering calculation: Use the Manhattan distance formula to calculate the distance between the feature vector of each rule and the current cluster center: Among which D i represents the Manhattan distance between the feature vector of the i-th rule and the cluster center, V i,d represents the eigenvalue of the d-th dimension of the i-th rule, C k,d represents the cluster center coordinate of the k-th cluster on the d-th dimension; according to the Manhattan distance between the feature vector of each rule and the cluster center, the rules are assigned to the nearest cluster, and the new cluster center is calculated. The updated cluster center will be used for distance calculation in the next cycle; Step 1-3, dynamic cluster adjustment, where the standard deviation σ of each dimension is calculated independently for each cluster d : Among them, V i,d represents the eigenvalue of the i-th rule in the d-th dimension, and C d is the cluster center coordinate of the current cluster in the d-th dimension, and N is the total number of rules included in the current cluster; When the standard deviation between the rules within the cluster is greater than the threshold, it is determined that splitting is required. The splitting operation selects the dimension with the largest variance within the cluster for splitting and forms two sub-clusters; when the centroid distance between the two clusters is less than 0.5 times the average cluster spacing and the similarity of the rule feature distribution is greater than the threshold, merge the two clusters into a new cluster and update the cluster center; Step 1-4: Rule classification and storage mapping: According to the results of the clustering calculation and cluster update, dynamically map each rule to a three-level storage structure to optimize the storage and rule matching efficiency.

4. The method according to claim 3, wherein In Step 1-4, the three-level storage structure includes L1-level storage, L2-level storage, and L3-level storage; among them, L1-level storage is used to store high-frequency and high-entropy rules, L2-level storage is used to store medium-frequency rules, and L3-level storage is used to store low-frequency rules; The rules stored in L1-level storage meet any of the following conditions: rule length ≤ 16 bytes and matching frequency > 80%; average information entropy of the cluster ≥ 1.5 and high consistency of rule types within the cluster; The rules stored in L2-level storage meet any of the following conditions: rule length ranges from 16 to 64 bytes; average matching frequency of the cluster ranges from 30% to 80%; The rules stored in L3-level storage meet all of the following conditions: average length of the rules within the cluster > 64 bytes; matching frequency < 30%; average information entropy of the cluster ≤ 1.0; Dynamically adjust the storage level according to the matching frequency of the rules within the cluster. The specific rule migration conditions are: L1 - level rule degradation: When the historical matching frequency f of a rule stored at the L1 - level i decreases by more than 20% for three consecutive cycles or the average feature distance d between the cluster center and the current data stream avg > 1.5σ, it is degraded to the L2 - level storage; L2-level rule degradation: When the historical matching frequency f of the rule i is lower than the threshold by 30% for 5 consecutive cycles or the information entropy E i < 1.0; then it degrades to L3-level storage; L2-level rule upgrade: When the similarity of the rule feature distribution between the L2 layer and the L1 layer is greater than 0.85, the matching frequency f i increases by more than 30% and the resource occupancy rate is less than 70%; then upgrade to the L1-level storage; Upgrade of L3-level rules: Triggered by burst traffic, upgrade to L2-level storage.

5. The method according to claim 4, wherein In Step 5, the regular matching module disassembles the regular expression matching task into two or more independent hardware units by pre-constructing a functional area. The hardware units are responsible for different regular expression elements. During the matching process, the hardware units dynamically load the state transition relationship of the regular expression from the block random access memory (BRAM) and construct the matching combination in real time according to the state transition relationship. The specific steps are as follows: Step 5-1: Perform cache processing on the data stream confirmed by the time-frequency weighted probability verification module, and use a sliding window method to extract every two consecutive characters as a group. Subsequently, according to the preset regular expression acceptable character table, perform look-up table matching on the extracted character pairs, filter out the set of regular expressions that can accept both character pairs, and determine the address of the regular expression to be activated according to the matching results; Step 5-2: According to the regular expression address obtained in Step 5-1, query the block random access memory (BRAM) that stores the specific matching information of the regular expression, and allocate the data read from the BRAM to the character matching functional unit and the quantifier matching functional unit to construct a regular expression matching combination unit. The character matching functional unit is responsible for processing the direct matching of single characters or character sets in the regular expression, and the quantifier matching functional unit is responsible for processing the logic related to quantifiers in the regular expression, that is, controlling the repetition times of characters or sub-patterns; Step 5-3: The regular expression matching combination unit sequentially obtains the data to be matched from the buffered data stream, and at the same time manages the control signals according to the regular expression state machine read from the block random access memory (BRAM), ensuring that the character matching functional unit and the quantifier matching functional unit execute in the correct order. The state machine will determine whether to continue the match, whether to roll back to the previous state, or whether to end the current matching process; Step 5-4: When the character matching functional unit and the quantifier matching functional unit complete the match in sequence according to the requirements of the regular expression state machine, the regular matching module outputs the matching result and feedbacks the optimized module parameters according to the matching result.

6. The method according to claim 5, characterized in that, Step 5-1 includes: In the first clock cycle, buffer and extract a batch of characters from the input data stream, and then through a look-up table mechanism, query the pre-configured regular expression acceptable character table based on the extracted characters, determine the regular expression that can accept the characters, and activate the regular expression for matching; In each subsequent clock cycle, continue to extract the next batch of characters from the input data stream and use the look-up table mechanism to match the characters. For each pair of characters in each clock cycle, first check whether the look-up table result contains the regular expression activated in the previous cycle. If the look-up table result contains the activated regular expression, directly pass the data of the pair of characters to the activated regular expression matching combination unit and continue to execute the matching operation; for the new regular expression contained in the look-up table result, read the data of the new regular expression from the block random access memory (BRAM) and construct a new regular expression matching combination unit.

7. The method according to claim 6, characterized in that Step 5-4 includes: Step 5-4-1: Real-time collect the output results of the regular matching module, including the rule identifier of successful match, the matching timestamp and traffic feature data, and the false positive sample information of unsuccessful match; Step 5-4-2: Dynamically adjust the rule storage hierarchy according to the output result of Step 5-4-1. When the rule reaches the rule upgrade condition in the low-level storage layer, migrate the rule to the high-level storage layer and update the Bloom filter of the corresponding layer; when the rule reaches the downgrade condition in the high-level storage layer, downgrade the rule to the low-level storage layer and update the Bloom filter of the corresponding layer; Step 5-4-3: Based on the distribution characteristics of false positive samples, reversely correct the hash function parameters and bit array layout of the Bloom filter, calculate the hash collision points of false positive samples, generate a bit correction mask, and perform local reconstruction on the bit vector of the Bloom filter that generates false matches; Step 5-4-4: Iteratively update the cluster center coordinates and cluster boundaries.

8. An FPGA device, characterized in that, It includes: FPGA chip, one or more Ethernet interfaces; The Ethernet interface is used for data transmission with the outside, and one or more programs are stored in the FPGA chip, and when the programs are executed by a processor, the steps of the method described in claim 2 are implemented.

Images