AC pattern matching tree construction method and device and AC pattern matching tree matching method and device

Abstract

The invention provides an AC mode matching tree construction method and device and an AC mode matching tree matching method and device.In one example, the AC mode matching tree construction method comprises the steps that according to a mode set, the mode that all modes in the mode set are inserted into a dictionary tree is adopted, and an AC mode matching tree is constructed; according to a breadth-first traversal mechanism, state identifiers are distributed to all nodes of the AC mode matching tree in sequence; and according to a breadth-first traversal mechanism, for a plurality of different sub-nodes of the same node, according to characters corresponding to edges, connected with the different sub-nodes, of the node, and according to an ASCII code sequence, rearranging state identifiers for the plurality of different sub-nodes to obtain a final AC mode matching tree. By applying the embodiment of the invention, the memory access performance of the AC mode matching tree can be optimized.

Description

Technical Field

[0001] The present invention relates to the technical field of information processing, and particularly to a method and apparatus for constructing an AC pattern matching tree and matching the AC pattern matching tree. Background Art

[0002] In the fields of computer science and information processing, a "pattern" usually refers to a specific string that needs to be searched for in a text dataset, or a keyword searched for in a dataset of other data types. Searching for a pattern in a given dataset involves the recognition and positioning of the pattern, and this process is pattern matching.

[0003] Pattern matching is widely applied in fields such as information retrieval systems, natural language processing, bioinformatics analysis, data compression, image processing, speech recognition, handwriting recognition, and intrusion detection and malware detection in network security services.

[0004] The AC pattern matching algorithm (Aho-Corasick algorithm) is a classic multi-pattern matching algorithm, which is implemented using a Trie tree (also known as a dictionary tree or prefix tree) and a finite state machine, and can complete the matching of all patterns simultaneously in a single scan of the input dataset. Summary of the Invention

[0005] The present invention provides a method and apparatus for constructing an AC pattern matching tree and matching the AC pattern matching tree.

[0006] According to the first aspect of the embodiments of the present invention, there is provided a method for constructing an AC pattern matching tree, including:

[0007] Constructing an AC pattern matching tree by inserting each pattern in the pattern set into a Trie tree respectively according to the pattern set;

[0008] Allocating state identifiers to each node of the AC pattern matching tree in sequence according to the breadth-first traversal mechanism;

[0009] According to the breadth-first traversal mechanism, for multiple different child nodes of the same node, re-arranging the state identifiers of the multiple different child nodes according to the characters corresponding to the edges connecting the node to different child nodes in the ASCII code order to obtain the final AC pattern matching tree.

[0010] According to the second aspect of the embodiments of the present invention, there is provided a method for matching an AC pattern matching tree, including:

[0011] Obtaining a binary file; wherein, the binary file is generated by the method provided in the first aspect;

[0012] Parse the binary file and extract the structure information of the final AC pattern matching tree;

[0013] Send the structure information of the final AC pattern matching tree to the FPGA, so that the FPGA stores the structure information of the final AC pattern matching tree in the storage space and performs pattern matching according to the structure information of the final AC pattern matching tree stored in the storage space.

[0014] According to the third aspect of the embodiments of the present invention, there is provided an apparatus for constructing an AC pattern matching tree, including:

[0015] A construction unit, configured to construct an AC pattern matching tree by inserting each pattern in the pattern set into a trie tree respectively according to the pattern set;

[0016] A reordering unit, configured to assign status identifiers to each node of the AC pattern matching tree in sequence according to the breadth-first traversal mechanism;

[0017] The reordering unit is further configured to, according to the breadth-first traversal mechanism, for multiple different child nodes of the same node, reorder the status identifiers of the multiple different child nodes according to the characters corresponding to the edges connecting the node to different child nodes in the ASCII code order to obtain the final AC pattern matching tree.

[0018] According to the fourth aspect of the embodiments of the present invention, there is provided an apparatus for matching an AC pattern matching tree, including:

[0019] An acquisition unit, configured to acquire a binary file; wherein, the binary file is generated by the method provided in the first aspect;

[0020] A parsing unit, configured to parse the binary file and extract the structure information of the final AC pattern matching tree;

[0021] A sending unit, configured to send the structure information of the final AC pattern matching tree to the FPGA, so that the FPGA stores the structure information of the final AC pattern matching tree in the storage space and performs pattern matching according to the structure information of the final AC pattern matching tree stored in the storage space.

[0022] According to the fifth aspect of the embodiments of the present invention, there is provided a network device, including an upper-layer driver and an FPGA; wherein:

[0023] The upper-layer driver is configured to acquire a binary file; wherein, the binary file is generated by the method provided in the first aspect; parse the binary file and extract the structure information of the final AC pattern matching tree; send the structure information of the final AC pattern matching tree to the FPGA;

[0024] The FPGA is used to store the structure information of the final AC pattern matching tree into a storage space, and perform pattern matching according to the structure information of the final AC pattern matching tree stored in the storage space.

[0025] Applying the technical solution disclosed by the present invention, by adopting the method of inserting each pattern in the pattern set into a trie tree respectively to construct an AC pattern matching tree, and according to the breadth-first traversal mechanism, state identifiers are sequentially assigned to each node of the AC pattern matching tree, which provides technical support for continuous storage of different child nodes of the same node. Thus, memory fragmentation can be reduced, the cache hit rate can be improved, and the memory access performance can be optimized. In addition, according to the breadth-first traversal mechanism, for multiple different child nodes of the same node, according to the characters corresponding to the edges connecting the node to different child nodes, the state identifiers are rearranged in the order of ASCII codes. The State IDs of different child nodes of the same node are arranged in the order of their corresponding characters in the ASCII code, and the State IDs are continuous. Thus, bit-level storage of the child nodes of the node can be performed, the byte length of the AC pattern matching tree structure can be reduced, conditions are created for obtaining relevant data of the AC Node with a single memory access, and the ACNode search efficiency can be improved. Description of the Drawings

[0026] Figure 1 is a schematic flowchart of a method for constructing an AC pattern matching tree provided by an embodiment of the present invention;

[0027] Figure 2A is a schematic diagram of an initially constructed AC pattern matching tree provided by an embodiment of the present invention;

[0028] Figure 2B is a schematic diagram of an AC pattern matching tree after reassigning State IDs according to the breadth-first traversal mechanism provided by an embodiment of the present invention;

[0029] Figure 2C is a schematic diagram of an AC pattern matching tree after rearranging StateIDs of multiple different child nodes of the same node provided by an embodiment of the present invention;

[0030] Figure 3A is a schematic diagram of tree head information provided by an embodiment of the present invention;

[0031] Figure 3B is a schematic diagram of node information provided by an embodiment of the present invention;

[0032] Figure 3C is a schematic diagram of PID header information provided by an embodiment of the present invention;

[0033] Figure 3D It is a schematic diagram of PID information provided by an embodiment of the present invention;

[0034] Figure 3E It is a schematic diagram of bitmap header information provided by an embodiment of the present invention;

[0035] Figure 3F It is a schematic diagram of bitmap information provided by an embodiment of the present invention;

[0036] Figure 4 It is a schematic flow diagram of an AC mode matching tree matching method provided by an embodiment of the present invention;

[0037] Figure 5 It is a schematic diagram of the AC head and partial AC node content of a binary file provided by an embodiment of the present invention;

[0038] Figure 6 It is a schematic flow diagram of an FPGA performing AC mode matching tree matching provided by an embodiment of the present invention;

[0039] Figure 7 It is a schematic structural diagram of an AC mode matching tree construction device provided by an embodiment of the present invention;

[0040] Figure 8 It is a schematic structural diagram of an AC mode matching tree matching device provided by an embodiment of the present invention. Detailed implementation manners

[0041] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the creation and matching implementation of the traditional AC mode matching tree will be briefly described below.

[0042] First, create an empty root (Root) node (Node), which usually does not represent any character.

[0043] For each Pattern, insert it into the Trie tree character by character starting from the Root. The edge between two Nodes is identified by a character, and this edge is called a path (Path), which is the key to checking whether the Pattern prefix exists. Inserting a character into a Node means adding a path to a child Node.

[0044] A Node represents a state in the Trie tree. If the current character exists in the child Nodes of the current Node, that is, the character has identified a Path, then the current Node can reach the next Node, that is, the next State, through this Path. If the current character does not exist in the child Nodes of the current Node, then a new Node is created and the current Node is connected to the new Node through the Path identified by the character.

[0045] When the last character of a Pattern is inserted, the end State is marked as a terminating Node. Usually, this State stores the index of the Pattern (abbreviated as PID) or other flag information, indicating that reaching this State means matching a Pattern in the Pattern dataset.

[0046] After completing the insertion and constructing the mismatch pointers according to the requirements of the Pattern dataset, during the pattern matching process, it can start from the Root node and sequentially perform matching processing according to the characters of the Pattern, and finally find the Pattern matching results that meet the conditions in the Pattern.

[0047] It can be seen that constructing the AC pattern matching tree requires allocating memory to store the tree Nodes. The Node storage includes characters, child Node pointers, mismatch pointers, and the index of the Pattern or other flag information. If the Pattern dataset is large or the Patterns are long, the memory consumption will increase significantly. And during the construction phase, the algorithm needs to insert and construct links for all Patterns, involving a large number of pointer operations and loop traversals, which will generate a certain CPU load on a pure CPU software platform. This load is proportional to the number of Patterns and the length of the Patterns. Moreover, this AC pattern matching tree has the characteristics of discontinuous memory addresses, variable Node lengths, uncertain numbers of child nodes, and extended data bits for the index of the Pattern or other flag information.

[0048] After the AC pattern matching tree is constructed, when performing AC pattern matching, the memory consumption usually does not increase significantly anymore. On a pure CPU software platform, it is usually carried out sequentially based on the characters of the input matching string. This access pattern may lead to discontinuous memory accesses, increasing the number of CPU cache misses. And because the finite state automaton composed of the AC pattern matching tree and the failure pointers usually has complex pointer operations and jumps, this may result in poor locality of Node access, and there may be frequent memory jumps. Especially when the tree structure is large and exceeds the CPU cache capacity, it will also increase the number of CPU cache misses. A high number of cache misses will cause the CPU to frequently access the slower main memory (RAM), which will significantly slow down the overall execution speed of the algorithm.

[0049] In order to make the above objects, features, and advantages of the embodiments of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0050] Please refer to Figure 1 , which is a schematic flowchart of a method for constructing an AC pattern matching tree provided by an embodiment of the present invention. As Figure 1 shown, the method for constructing an AC pattern matching tree may include the following steps:

[0051] Step 101: According to the pattern set, construct an AC pattern matching tree by inserting each pattern in the pattern set into the trie tree respectively.

[0052] In an embodiment of the present invention, in the case where an AC pattern matching tree needs to be constructed, a Pattern set can be obtained, and the Pattern set includes Patterns that need to be inserted into the Trie tree.

[0053] Each Pattern in the Pattern set can be inserted into the Trie tree respectively according to the obtained Pattern set to construct an AC pattern matching tree, and the specific construction method can refer to the relevant description above.

[0054] Step 102: Allocate state identifiers to each node of the AC pattern matching tree in sequence according to the breadth-first traversal mechanism.

[0055] In an embodiment of the present invention, in order to enable different child nodes of the same node in the AC pattern matching tree to be continuously stored in the storage space, after constructing the AC pattern matching tree in the above manner, state identifiers (State ID) can be allocated to each node of the AC pattern matching tree in sequence according to the breadth-first traversal mechanism. Thus, the state identifiers of different child nodes of the same node are consecutive, and when storing the node information of the AC pattern matching tree, the addresses of different child nodes of the same node are consecutive in the storage space.

[0056] For example, assume that the Pattern set is {his, he, she, hers}, then the structure of the AC pattern matching tree created in the manner of step 101 can be referred to Figure 2A .

[0057] Among them, Figure 2A the data meanings and PID annotation information in the AC pattern matching tree shown can be as shown in Table 1.

[0058] Table 1

[0059]

[0060] Among them, "Path" in Table 1 is the character corresponding to the edge connecting a node to its child node; for example, for State0, its child nodes include State1 and State5, and the corresponding characters are h and s respectively.

[0061] "Failed" is the mismatch pointer. For example, the "Failed" value of State6 is 1, indicating that the mismatch pointer of State6 points to State1.

[0062] "Afetr inherit failed PID" is the PID owned by the State after including the PID through inheriting the mismatch pointer. For example, for State7, the PID hanging under itself is 3 (corresponding to "she"), and the PID inherited through the mismatch pointer is 2 (corresponding to "he"), so the "After inherit failed PID" value of State7 is 3 and 2.

[0063] In the case where state identifier allocation is performed according to step 102, the structure of the AC pattern matching tree can be referred to Figure 2B . As Figure 2B shown, the State IDs of different child nodes of the same node are consecutive. For example, for node 1 (the node with State ID 1), the State IDs of its child nodes are 3 and 4 respectively.

[0064] Among them, Figure 2B the data meanings and PID annotation information in the AC pattern matching tree shown can be as shown in Table 2.

[0065] Table 2

[0066]

[0067] Among them, "InitState" is the initial State ID of each node (that is, the State ID before re - allocation according to the breadth - first traversal mechanism, and the value corresponds to Figure 2A ).

[0068] "State" is the State ID of each node after reassigning the State ID to each node in the AC pattern matching tree according to the breadth - first traversal mechanism.

[0069] For example, as Figure 2A and Figure 2B shown, Figure 2A State5 in Figure 2B is State2 in

[0070] after reassigning the State ID according to the breadth - first traversal mechanism.

[0071] Step 103: According to the breadth - first traversal mechanism, for multiple different child nodes of the same node, based on the characters corresponding to the edges connecting the node to different child nodes, in ASCII code order, rearrange the state identifiers for these multiple different child nodes to obtain the final AC pattern matching tree.

[0072] In the embodiments of the present invention, in the case of assigning state identifiers to each node of the AC pattern matching tree in the above - mentioned manner, it is also possible to further, according to the breadth - first traversal mechanism, for multiple different child nodes of the same node, based on the characters corresponding to the edges (paths) connecting the node to different child nodes, in ASCII code order, rearrange the state identifiers for these multiple different child nodes to obtain the final AC pattern matching tree.

[0073] Taking the Figure 2B shown AC pattern matching tree as an example, the characters corresponding to the edges connecting node 1 to child node 3 and child node 4 (which can be simply referred to as the characters corresponding to child node 3 and child node 4) are i and e respectively. For this node (i.e., node 1), based on the characters corresponding to the child nodes, in ASCII code order, rearrange these multiple different child nodes (i.e., child node 3 and child node 4), that is, set the State ID of the child node with the corresponding character e to 3, and set the State ID of the child node with the corresponding character i to 4. The schematic diagram can be seen in Figure 2C .

[0074] Among them, Figure 2C the data meanings and PID annotation information in the shown AC pattern matching tree can be as shown in Table 3.

[0075] Table 3

[0076]

[0077] Among them, "InitState" in Table 3 is the initial State ID of each node (the value corresponds toFigure 2A )。

[0078] "TempState" is the State ID of each node after the State IDs of all nodes are re - allocated according to the breadth - first traversal mechanism, and the value range corresponds to Figure 2B )。

[0079] "State" is the State ID of each node after, according to the breadth - first traversal mechanism, for multiple different child nodes of the same node, the State IDs of these multiple different child nodes are rearranged according to the characters corresponding to the edges connecting the node to different child nodes in ASCII code order.

[0080] For example, as Figure 2B and Figure 2C shown, for State1 in Figure 2B , the State IDs of its child nodes are 3 and 4 respectively, and the corresponding characters are i and e respectively. Since e comes before i in the ASCII code, in the AC - mode matching tree shown in Figure 2C , the child node with State ID 3 corresponds to the character e, and the child node with State ID 4 corresponds to the character i.

[0081] As Figure 2C shown, the State IDs of different child nodes of the same node are arranged in the order of their corresponding characters in the ASCII code, and the State IDs are consecutive. Thus, bit - level storage can be performed on the child nodes of the node, reducing the byte length of the AC - mode matching tree structure, creating conditions for obtaining relevant data of the AC Node with a single memory access, and improving the AC Node search efficiency.

[0082] For example, for the AC - mode matching tree structure shown in Figure 2C , for any node, the existence of a child node corresponding to a certain character of the node can be recorded with a length of 256 bits. For example, the 256 - bit length includes bits from the 0th to the 255th. The value of each bit being 0 indicates that the node does not have a child node corresponding to that character (the character in the same position in the ASCII code sorting); the value being 1 indicates that the node has a child node corresponding to that character.

[0083] It can be seen that in Figure 1In the method flow shown, by adopting the method of inserting each pattern in the pattern set into the trie tree respectively, an AC pattern matching tree is constructed, and according to the breadth-first traversal mechanism, status identifiers are assigned to each node of the AC pattern matching tree in sequence, providing technical support for continuous storage of different child nodes of the same node. Thus, memory fragmentation can be reduced, the cache hit rate can be improved, and the memory access performance can be optimized. In addition, according to the breadth-first traversal mechanism, for multiple different child nodes of the same node, based on the characters corresponding to the edges connecting the node to different child nodes, the status identifiers are rearranged in the order of ASCII codes. The State IDs of different child nodes of the same node are arranged in the order of their corresponding characters in the ASCII code, and the State IDs are continuous. Thus, bit-level storage of the child nodes of the node can be performed, reducing the byte length of the AC pattern matching tree structure, creating conditions for obtaining relevant data of the AC Node with a single memory access, and improving the ACNode search efficiency.

[0084] In some embodiments, when the final AC pattern matching tree is obtained, the AC pattern matching tree construction method provided by the embodiments of the present invention may further include:

[0085] According to the depth-first traversal mechanism, determine the file writing order of the pattern indexes PID hung under each node in the final AC pattern matching tree; where the PID hung under the node includes the PID of the pattern with the node as the tail node, and the PID inherited by the node; for the PID hung under the same node, the writing order of the PID with the node as the tail node is before the PID inherited by the node;

[0086] According to the file writing order of the PID, write the structure information of the final AC pattern matching tree into a binary file in the form of separating the node from the PID for storage.

[0087] Exemplarily, in order to improve the deployment speed of the AC pattern matching tree on the hardware platform, the structure information of the AC pattern matching tree can be stored in a binary file, so that the AC pattern matching tree can be deployed to the hardware platform according to the binary file as needed.

[0088] Among them, before writing the structure information of the final AC pattern matching tree obtained in the above manner into a binary file, the PID hung under each node in the AC pattern matching tree can also be rearranged according to the depth-first traversal mechanism, that is, the PID hung under the node with a greater depth (higher level) is arranged in front of the PID hung under the node with a smaller depth (lower level).

[0089] For example, the PID hung under the node at the third layer is arranged in front of the PID hung under the node at the second layer (if the PID is not inherited).

[0090] Among them, the PIDs hung under a node include the PIDs of the patterns with this node as the tail node, and the PIDs inherited by this node (if any).

[0091] Among them, for a node, if there are PIDs hung under the node pointed to by the mismatch pointer of this node, the PIDs hung under the node pointed to by the mismatch pointer of this node are inherited by the node.

[0092] For example, assume that the mismatch pointer of node 8 points to node 3, and the PID hung under node 3 is 2. Since node 8 inherits the PID of node 3 and PID 2 is included in the PIDs hung under node 8, the PID of node 3 can be represented within the range of the PIDs of node 8.

[0093] Among them, for the PIDs hung under the same node, the writing order of the PIDs with this node as the tail node is before the PIDs inherited by this node.

[0094] By the above method of rearranging the PIDs hung under the node, the relationship between the node and the PID can be mapped by an address offset, so that the node and the PID resources can be stored continuously respectively, and the PIDs hung under the failed jump node (i.e., the node pointed to by the mismatch pointer) inherited by the node are rearranged after the original PIDs of this node, so that the jump node can reuse the PID storage resources of the inherited node, reducing the use of storage resources by the PID and improving the search efficiency of the PID.

[0095] In an example, according to the file writing order of the PID, writing the structure information of the final AC pattern matching tree into a binary file in the form of separating the node and the PID can include:

[0096] Writing the tree head information of the final AC pattern matching tree into a binary file; where the tree head information includes a tree identifier, an AC pattern identifier, and the number of nodes;

[0097] Writing the node information of the final AC pattern matching tree into a binary file in the order of breadth-first traversal; where the node information includes the number of child nodes, the number of PIDs hung under, the starting index of child nodes, the starting index of PIDs, the index of the mismatch pointer, and part or all of the characters corresponding to the child nodes; the characters corresponding to the child nodes are stored at the bit level, with one bit corresponding to one character;

[0098] Writing the PID head information of the final AC pattern matching tree into a binary file; where the PID head information includes a tree identifier and the number of PIDs;

[0099] Write the PID information of the final AC mode matching tree into a binary file according to the file writing order of PID; wherein, the PID information includes PID, matching stop flag, start position of the mode, and end position of the mode.

[0100] Exemplarily, in the process of writing the structure information of the AC mode matching tree into a binary file, the form of information header + information content can be adopted for writing.

[0101] Among them, the structure information of the AC mode matching tree may include node (or tree node) structure information and PID structure information.

[0102] Correspondingly, in the process of writing into the binary file, for the node structure information, the tree header information and the node content information (which can be simply referred to as node information) can be written respectively; for the PID structure information, the PID header information and the PID content information (which can be simply referred to as PID information) can be written respectively.

[0103] Among them, the tree header information may include but is not limited to tree identifier (TrieID), AC mode identifier, and the total number of nodes.

[0104] Among them, the tree identifier is used to uniquely identify a Trie tree; the AC mode identifier is used to identify whether the Tire tree enables the AC mode; the total number of nodes is used to identify the total number of nodes included in the Trie tree.

[0105] For example, the schematic diagram of the tree header information can be seen in Figure 3A . As Figure 3A shown, both the tree identifier and the AC mode identifier (ACMode) are of unsigned char type and each occupies 1 byte; the reserved field includes 2 unsigned char types and occupies a total of 2 bytes; the number of nodes (total_node_numbe) is a 32-bit unsigned integer and occupies 4 bytes.

[0106] The node information may include but is not limited to the number of child nodes, the number of PIDs attached, the start index of child nodes, the start index of PIDs, the mismatch pointer index, and the character corresponding to the child node.

[0107] Among them, the number of child nodes is used to identify the number of child nodes of the node; the number of PIDs attached is used to identify the number of PIDs attached to the node; the start index of child nodes is used to identify the minimum value of the State ID of the child nodes of the node; the start index of PIDs is used to identify the index of the first PID attached to the current node (i.e., the starting position where the PIDs attached to the node are stored); the mismatch pointer index is used to identify the State ID of the node pointed to by the mismatch pointer of this node; the character corresponding to the child node is set in 256 bits and is used to identify the character corresponding to the path of the connection at the node.

[0108] Exemplarily, the starting index of the child node combined with the child node bitmap can calculate the position where the child node is located.

[0109] For example, a schematic diagram of the node information can be seen Figure 3B . As Figure 3B shown, both the number of child nodes (child_count) and the number of attached PIDs (pid_count) adopt the unsigned character type, each occupying 1 byte; the reserved fields include 2 unsigned character types, occupying a total of 2 bytes; the starting index of the child node (child_index_start), the starting index of the PID (pid_index_start), the mismatch pointer index (fail_jump_index), and the character corresponding to the child node (children) all adopt the unsigned character data type (unsigned char); among them, the character corresponding to the child node is represented by an array containing 32 unsigned characters, occupying a total of 32 bytes.

[0110] For example, for the lowercase letters a - z, their sorting in the ASCII code is 96 - 122 in sequence. Thus, for any node, according to the set bit status of bits 96 - 122 in the character corresponding to the child node included in the node information of this node, the existence situation of the character corresponding to the child node can be determined.

[0111] For example, if bit 96 is set, it indicates that the current node has a child node corresponding to the character a; if bit 122 is set, it indicates that the current node has a child node corresponding to the character z. Thus, a single bit can be used to identify the character corresponding to a child node.

[0112] The PID header information can include but is not limited to the tree identifier and the number of PIDs.

[0113] Among them, the total number of PIDs is used to identify the size of the PID array in the case of re - arranging the Patterns of the AC tree according to the depth - first traversal mechanism and mapping them to a consecutive - address PID array.

[0114] Exemplarily, in the case where there is no inheritance of the mismatch pointer PID, the size of the PID array is equal to the number of Patterns in the Pattern set (the Pattern set used to construct the AC tree); in the case where there is inheritance of the mismatch pointer PID, the size of the PID array is greater than or equal to the number of Patterns in the Pattern set.

[0115] For example, when there are multiple mismatch pointers of nodes pointing to the same node, and there is a PID hanging under this node, the PID hanging under this node will appear multiple times in the above PID array through the inheritance method of the mismatch pointer. Furthermore, the size of the PID array will be greater than the number of Patterns in the Pattern set.

[0116] For example, the schematic diagram of the PID header information can be seen in Figure 3C . As Figure 3C shown, the tree identifier uses an unsigned character type and occupies 1 byte; the reserved field includes 3 unsigned character types, occupying a total of 3 bytes; the total number of PIDs (total_pid_sum) uses a 32-bit unsigned integer and occupies 4 bytes.

[0117] The PID information may include, but is not limited to, some or all of the PID, match stop flag, pattern start position, pattern character case storage status, pattern maximum length, pattern end position, and bitmap index.

[0118] Exemplarily, the match stop flag is used to indicate whether to end the pattern matching.

[0119] Among them, the value of the match stop flag may include a valid value or an invalid value; when the value of the match stop flag is a valid value, once a PID carrying the match stop flag is hit, the current matching process can be ended and the PID can be returned to the AC pattern matching caller.

[0120] For example, for the PID corresponding to the virus signature, when the virus signature is found, the current matching process needs to be ended and an interception should be made immediately.

[0121] Exemplarily, the pattern character case storage status may include match lowercase, match uppercase, ignore case, or match case.

[0122] For example, assume the Pattern is aB. Then, for the case of matching uppercase, when "AB" is matched, it is determined that the match is successful; for the case of matching lowercase, when "ab" is matched, it is determined that the match is successful; for the case of ignoring case, when "ab", "aB", "Ab", or "AB" is matched, it is determined that the match is successful; for the case of matching case, when "aB" is matched, it is determined that the match is successful.

[0123] Exemplarily, the pattern maximum length is used to record the length of the longest pattern in the pattern set.

[0124] Among them, in the case where the pattern needs to be case-sensitive during verification, for example, the case storage state of the pattern characters is case-sensitive matching. In this case, to improve processing efficiency, when it is determined that the pattern is matched from the current string, the string matching the maximum length of the pattern can be read forward from the current character position according to the maximum length of the pattern, and the case-sensitive verification of the read string can be performed in the verification channel. Thus, the matching process and the verification process can be processed in parallel.

[0125] Exemplarily, the pattern start position and the pattern end position are used to indicate the start position and the end position of the successfully matched pattern in the matching string.

[0126] Among them, during the pattern matching process, when it is determined that the current node contains the index of the pattern, the start position (which can be called the target start position) and the end position (which can be called the target end position) of the pattern corresponding to the index of the pattern in the matching string can be determined, and it can be determined whether the target start position and the target end position match the pattern start position and the pattern end position; if they match, subsequent processing can be further performed. For example, in the case where case-sensitive verification is required, case-sensitive verification processing is performed; if they do not match, it is determined that the pattern matching is unsuccessful.

[0127] As an example, the PID information may further include a bitmap index, and the bitmap is used to record the case information of each character in the corresponding Pattern;

[0128] Writing the structural information of the final AC pattern matching tree into a binary file according to the file writing order of the PID as described above, in the form of separating nodes from the PID, may further include:

[0129] Writing the bitmap header information of the final AC pattern matching tree into a binary file; wherein, the bitmap header information includes a tree identifier and the number of bitmaps;

[0130] Writing the bitmap information of the final AC pattern matching tree into a binary file; wherein, the bitmap length is determined according to the depth of the AC pattern matching tree.

[0131] Exemplarily, considering that in the actual scenario, the Pattern may also need to be case-sensitive. In this case, the case information of each Pattern can be additionally recorded. Exemplarily, the case information of the Pattern can be recorded by a bitmap.

[0132] Exemplarily, whether the Pattern is case-sensitive can be determined according to the case storage state of the pattern characters, that is, the bitmap information can be used for PID verification guided by the case storage state of the pattern characters.

[0133] For example, when the case storage state of the pattern characters is case-sensitive matching, it is necessary to additionally record the case information of each Pattern. Correspondingly, it is necessary to store the bitmap information correspondingly for case checking of the Pattern corresponding to the PID.

[0134] Correspondingly, the PID information may also include a bitmap index, and the bitmap is used to record the case information of each character in the corresponding Pattern.

[0135] Exemplarily, for any Pattern, the length of its corresponding bitmap can be determined according to the depth of the AC pattern matching tree (which can be denoted as Trie_layer_maxth).

[0136] Exemplarily, Trie_layer_maxth records the length of the longest Pattern in the AC tree Pattern set + 1 (1 is the length occupied by the root node).

[0137] For example, assuming the Pattern set is {"abc", "Bcd", "cDEfgH"}, then Trie_layer_maxth = 6 + 1, and the bitmap length is That is, 8 bit; where, is the ceiling operation.

[0138] Correspondingly, the bitmap information corresponding to each pattern is respectively:

[0139] {00000000} (There is no capital letter in the first Pattern)

[0140] {01000000} (The first character in the second Pattern is a capital letter)

[0141] {00110010} (The second, third, and sixth characters in the third Pattern are capital letters).

[0142] In the process of writing the structure information of the AC pattern matching tree into a binary file, the corresponding bitmap information can also be written into the binary file in the form of a bitmap header + bitmap content.

[0143] It should be noted that in the embodiments of the present invention, for the same tree, the tree header information, node information, PID header information, PID information, bitmap header information (if any), and bitmap information (if any) are continuous in the binary file, and in the subsequent process, the storage space opened by the FPGA for the corresponding structure information is also continuous.

[0144] In some embodiments, the structure information of the final AC pattern matching tree is written into the binary file in the TLLV form;

[0145] The TLLV format includes a value type, the total length of the value, the length of a single piece of data, and the data value; the data value is used to record the structural information of the final AC pattern matching tree.

[0146] Exemplarily, the above-mentioned tree header information, node information, PID header information, PID information, bitmap header information, and bitmap information can all be written into a binary file in the TLLV format.

[0147] Among them, the TLLV format can include a type of value, the total length of the data (which can also be referred to as the total length of the value), the length of a single piece of data, and the data value; the data value can be used to record the structural information of the final AC pattern matching tree.

[0148] Among them, the type of value is used to identify different types of information in the structural information of the AC pattern matching tree. For example, tree header information, node information, PID header information, PID information, bitmap header information, and bitmap information, etc.

[0149] For example, the types of tree header information, node information, PID header information, PID information, bitmap header information, and bitmap information can be 1 to 6 in sequence.

[0150] Among them, the total length of the data is used to identify the total length of the data value; the length of a single piece of data is used to identify the length of a single data value.

[0151] For any type of information (such as the type of tree header information, node information, PID header information, PID information, bitmap header information, or bitmap information), the size of the single data structure is the same.

[0152] For example, for node information, the data size of a single piece of node information (i.e., the length of a single piece of node information) is the same.

[0153] The total length of the data can be equal to the length of the data header + the data length; the data length is equal to the length of a single piece of data * the number of data pieces.

[0154] For example, for node information, the length of a single piece of data is the length of a single piece of node information; the total length of the value is the length of the data header + the total length of all node information.

[0155] In one example, for any type of information, fixed header length information can be stored in the above binary file, which is used to store the data type, the total length of the data, and the size of a single piece of data (corresponding to the "TLL" in the above "TLLV"). The length of this header length information (i.e., the length of the data header) can be preset.

[0156] For example, the length of the header length information can be 12 bytes. Among them, the first 4 bytes are used to store the data type, the middle 4 bytes are used to store the total data length, and the last 4 bytes are used to store the single data size. Its specific implementation can be described in combination with specific examples below.

[0157] During the process of reading a binary file, in the case of the first read, the first 12 bytes can be read and divided into 3 parts of 4 bytes each to respectively determine the stored data type, total data length, and single data size, and based on the total data length and single data size, determine the size of each data and the number of data.

[0158] For example, for node information, assuming the length of the header length information is 12 bytes, the total data length is 492 bytes, and the single data size (i.e., the size of a single node information) is 48 bytes, it indicates that there are 10 nodes in the AC pattern matching tree (the total number of nodes is also recorded in the tree header information).

[0159] Among them, the data value can be stored in the form of an array. It can be seen that in the embodiments of the present invention, for the AC pattern matching tree, in the case where case sensitivity verification is required, the size of the bitmap storage space to be applied can be determined according to the depth (Trie_layer_maxth) of the AC pattern matching tree.

[0160] In addition, mapping the AC pattern matching tree to an array structure, the sizes of the array member structures are the same (the single data structure sizes in any type of information are the same), and it can be used commonly in 32-bit and 64-bit systems, which can effectively reduce the problem of different resource occupation sizes caused by different pointer word lengths in different systems.

[0161] Furthermore, by inheriting the PID of the mismatch pointer for nodes, it can be reused for the mapping of the PID sequence of the mismatch pointer, achieving the purpose of PID resource reuse.

[0162] Please refer to Figure 4 , which is a schematic flowchart of an AC pattern matching tree matching method provided by the embodiments of the present invention. As Figure 4 shown, the AC pattern matching tree matching method may include the following steps:

[0163] Step 401, obtain a binary file.

[0164] In the embodiments of the present invention, in the case where it is necessary to deploy the AC pattern matching tree to a hardware platform (such as an FPGA), a binary file generated in the manner described in the above embodiments (a binary file written with the structure information of the AC pattern matching tree) can be obtained, and the structure information of the AC pattern matching tree can be obtained based on this binary file to implement the matching of the AC pattern matching tree.

[0165] Step 402: Parse the binary file to extract the structural information of the final AC pattern matching tree.

[0166] In the embodiment of the present invention, according to the relevant descriptions in the above embodiments, the structural information of the final AC pattern matching tree is stored in the binary file. Therefore, the structural information of the final AC pattern matching tree can be obtained by parsing the binary file and stored in the storage space (such as memory) of the FPGA. The FPGA can perform the matching of the AC pattern matching tree based on the structural information of the final AC pattern matching tree stored in the memory.

[0167] Correspondingly, in the case of obtaining the above binary file, the obtained binary file can be parsed to extract the structural information of the final AC pattern matching tree (the structural information of the AC pattern matching tree constructed in the manner described in the above embodiments).

[0168] Step 403: Send the structural information of the final AC pattern matching tree to the FPGA, so that the FPGA stores the structural information of the final AC pattern matching tree in the storage space and performs pattern matching based on the structural information of the final AC pattern matching tree stored in the storage space.

[0169] In the embodiment of the present invention, considering that the FPGA hardware platform is not limited by the instruction pipeline in the traditional CPU and is composed of a large number of configurable logic units (CLBs, Configurable Logic Block) internally, each logic unit can be independently configured to execute a specific function and perform different operations. And the FPGA processes multiple input and output streams simultaneously, allowing multiple data streams to pass through different computing paths at the same time. Each data stream can be processed in parallel through pipelines or direct interconnections, and deeper parallelism can also be achieved by configuring the pipeline structure. Multiple tasks can be executed in the form of a pipeline, and each task can be statically or dynamically configured in different regions, thereby realizing deep parallel computing.

[0170] Therefore, the FPGA can be used to implement the storage and pattern matching of the AC pattern matching tree structure.

[0171] Correspondingly, for the structural information of the final AC pattern matching tree extracted in the above manner, it can be sent to the FPGA.

[0172] The FPGA can store the structural information of the final AC pattern matching tree in the storage space (such as memory) and perform pattern matching based on the structural information of the final AC pattern matching tree stored in the storage space.

[0173] It can be seen that in the embodiments of the present invention, by generating the AC pattern matching tree in the above manner and writing its structure information into a binary file, during the application process, the file content in the binary file is read into the memory, and then pattern matching can be performed according to the structure information of the AC pattern matching tree stored in the memory, without the need to construct the AC pattern matching tree again.

[0174] Among them, the binary file generated in the above manner is universal and can be generated offline. During the application process, by sending the above binary file to the corresponding platform (either a software platform or a hardware platform), the AC pattern matching tree can be restored according to the file, which can effectively improve the processing efficiency of pattern matching.

[0175] In some embodiments, the structure information of the final AC pattern matching tree includes tree head information, node information, PID head information, and PID information.

[0176] The above-mentioned process of sending the structure information of the final AC pattern matching tree to the FPGA includes:

[0177] According to the tree head information, send an AC pattern matching tree resource request message to the FPGA, so that the FPGA allocates an AC pattern matching tree storage space for the AC pattern matching tree and stores the tree head information in the AC pattern matching tree storage space;

[0178] Send the node information to the FPGA, so that the FPGA stores the node information in the AC pattern matching tree storage space;

[0179] According to the PID head information, send a PID resource request message to the FPGA, so that the FPGA allocates a PID storage space for the AC pattern matching tree and stores the PID head information in the PID storage space;

[0180] Send the PID information to the PFGA, so that the FPGA stores the PID information in the PID storage space.

[0181] Exemplarily, during the process of storing the structure information of the AC pattern matching tree in the FPGA, the node information and the PID information can be stored separately.

[0182] For the node information, according to the extracted tree head information, the number of nodes can be determined, and according to the head length information corresponding to the tree head information, the data size of the tree head information, the head length information corresponding to the node information, the number of nodes, and the data size of a single node information, the storage space size for storing the tree head information and the node information can be determined.

[0183] For the PID information, the number of PIDs can be determined based on the extracted PID header information, and the storage space size for storing the PID header information and the PID information can be determined based on the header length information corresponding to the PID header information, the data size of the PID header information, the header length information corresponding to the PID information, the number of PIDs, and the data size of a single PID information.

[0184] Exemplarily, based on the tree header information extracted from the binary file, an AC mode matching tree resource request message (which can be referred to as the ACAskResource message) can be sent to the FPGA.

[0185] Exemplarily, the tree header information can be carried in the AC mode matching tree resource request message.

[0186] Among them, the AC mode matching tree resource request message is used to notify the FPGA to allocate a storage space for storing the tree header information and the node information.

[0187] When the FPGA receives the AC mode matching tree resource request message, it can allocate an AC mode matching tree storage space for the AC mode matching tree. This AC mode matching tree storage space is used to store the tree header information and the node information, and the tree header information is stored in the AC mode matching tree storage space.

[0188] After allocating the AC mode matching tree storage space and storing the tree header information in the above manner, the node information can be sent to the FPGA; the FPGA can store the received node information in the AC mode matching tree storage space.

[0189] It should be noted that considering the limitation of the single - transmission data length during the communication with the FPGA, in the case of a large number of nodes, it may not be possible to transmit all the node information in a single transmission. In this case, the node information can be transmitted in batches.

[0190] For example, assume that the maximum single - transmission data length between the upper - layer driver (which can be simply referred to as the driver) and the FPGA is 1500 bytes, the number of AC mode matching tree nodes is 100, and the data structure size of a single node is 48 bytes. Then each time, that is, at most 31 nodes' node information can be transmitted in a single time, and at least times need to be transmitted.

[0191] Exemplarily, in the case where the node information is stored in the form of an array, the number of transmitted nodes can be verified through a cursor.

[0192] Taking the previous example as an example, the sending example according to the binary file content reading order is as follows:

[0193] Drive to send node information for the first time [node0-node30] (i.e., node information of nodes 0 to 30), the number of node information transmitted each time is the node information of 31 nodes, starting from 0 to fill (the initial value of the cursor is 0).

[0194] The FPGA receives the node information sent by the drive for the first time, fills the data content into the allocated memory space according to the position cursor, and sets the position cursor after node30, that is, at the position of node31.

[0195] Drive to send node information for the second time [node31-node61] (i.e., node information of nodes 31 to 61), the number of node information transmitted each time is the node information of 31 nodes, starting from 31 to fill (the cursor is updated to 31).

[0196] The FPGA receives the node information sent by the drive for the second time, fills the data content into the allocated memory space according to the position cursor, and sets the position cursor after node61, that is, at the position of node62.

[0197] And so on.

[0198] Drive to send for the fourth time, [node93-node99], the number of node information transmitted each time is the node information of 7 nodes, starting from 93 to fill.

[0199] The FPGA receives the node information sent by the drive for the fourth time, fills the data content into the allocated memory space according to the position cursor, and sets the position cursor after node99, and the cursor is updated to 100.

[0200] Exemplarily, when the drive has parsed the node information, it can notify the FPGA that the node information distribution of the current AC mode matching tree has ended. When the FPGA has processed the message, it checks whether the current cursor is consistent with the total number of nodes carried in the ACAskResource message, and sends the verification result to the drive to mutually verify whether the node information is complete.

[0201] The distribution of resources in the AC mode matching tree has ended. After the FPGA has processed the message, it checks whether the current cursor is consistent with the total number information carried in the AskResource message, and sends the verification result to the drive to mutually verify whether the resources in the AC mode matching tree are complete.

[0202] Exemplarily, for the specific implementation of the drive sending PID information to the FPGA for storage by the FPGA, reference can be made to the relevant implementation of storing node information described in the above embodiments, and the embodiments of the present application will not be elaborated here.

[0203] In addition, for the integrity check of PID information, reference can also be made to the relevant implementation of the integrity check of node information described in the above embodiments, which will not be elaborated in this embodiment of the present application.

[0204] It should be noted that the maximum data length of a single transmission of the above driver and FPGA is 1500 bytes, which usually refers to the size of the largest data frame that the link layer can transmit. To achieve the maximum utilization rate of the packet space, the AC tree identifier and the cursor start information of data filling can be carried at specific positions in the Ethernet header, and the data field is used to transmit node information.

[0205] Similarly, for the storage of PID information, a PID resource request message (which can be called a PIDAskResource message) can be sent to the FPGA according to the PID header information first.

[0206] When the FPGA receives the PID resource request message, it can allocate PID storage space for the AC mode matching tree, and this PID storage space is used to store the PID header information and PID information.

[0207] After allocating the PID storage space in the above manner and storing the PID header information, node information can be sent to the FPGA; the FPGA can store the received PID information in the PID storage space.

[0208] In one example, the structural information of the final AC mode matching tree further includes a bitmap header and bitmap information;

[0209] The above-mentioned distribution of the structural information of the final AC mode matching tree to the FPGA may further include:

[0210] According to the bitmap header information, send a bitmap resource request message to the FPGA, so that the FPGA allocates bitmap storage space for the AC mode matching tree and stores the bitmap header information in the bitmap storage space;

[0211] Send the bitmap information to the FPGA so that the FPGA stores the bitmap information in the bitmap storage space.

[0212] Exemplarily, when the structural information of the AC mode matching tree further includes bitmap information, the bitmap information also needs to be stored in the FPGA.

[0213] Exemplarily, the number of bitmaps can be determined according to the bitmap header information, and the storage space size for storing the bitmap header information and bitmap information can be determined based on the header length information corresponding to the bitmap header information, the data size of the bitmap header information, the header length information corresponding to the bitmap information, the number of bitmaps, and the data volume size of a single bitmap information.

[0214] According to the bitmap header information extracted from the binary file, a bitmap resource request message (which can be called the BitmapAskResource message) can be sent to the FPGA.

[0215] Exemplarily, the bitmap header information can be carried in the bitmap resource request message.

[0216] Among them, the bitmap resource request message is used to notify the FPGA to allocate a storage space for storing the bitmap header information and the bitmap information.

[0217] When the FPGA receives the bitmap resource request message, it can allocate a bitmap storage space for the AC mode matching tree, and this bitmap storage space is used to store the bitmap header information and the bitmap information.

[0218] After allocating the bitmap storage space in the above manner and storing the bitmap header information, the bitmap information can be sent to the FPGA; the FPGA can store the received bitmap information in the bitmap storage space.

[0219] Exemplarily, for the specific implementation of the driver sending the bitmap information to the FPGA for storage, reference can be made to the relevant implementation of storing node information described in the above embodiments, and the embodiments of the present application will not be elaborated here.

[0220] In addition, for the integrity check of the bitmap information, reference can also be made to the relevant implementation of the integrity check of node information described in the above embodiments, and the embodiments of the present application will not be elaborated here.

[0221] To enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present invention, the technical solutions provided by the embodiments of the present invention will be described below in conjunction with specific examples.

[0222] In this embodiment, based on the AC mode matching tree constructed in the traditional manner, the state machine structure is reconstructed and compressed according to the characteristics of FPGA parallel processing. In order to achieve the goal of adapting to the FPGA hardware platform, reducing resource occupation, and improving the matching efficiency, optimization processing is carried out.

[0223] The following will describe the specific implementation process.

[0224] 1. According to the Pattern set, by inserting each Pattern in the Pattern set into the trie tree respectively, an AC mode matching tree is constructed.

[0225] In this embodiment, taking the Pattern set as {his, he, she, hers} as an example, the structure of the AC mode matching tree can be as Figure 2A shown.

[0226] II. According to the breadth - first traversal mechanism, assign State IDs to each node of the AC pattern matching tree in sequence. The schematic diagram of the AC pattern matching tree after re - assigning State IDs can be as Figure 2B shown.

[0227] III. According to the breadth - first traversal mechanism, for multiple different child nodes of the same node, re - arrange the State IDs of these multiple different child nodes in ascending ASCII code order according to the characters corresponding to the edges connecting the node to different child nodes, and obtain the final AC pattern matching tree. The schematic diagram can be as Figure 2C shown.

[0228] IV. According to the depth - first traversal mechanism, determine the file writing order of the PIDs hanging under each node in the final AC pattern matching tree.

[0229] Among them, for Figure 2C the AC pattern matching tree shown, according to the depth - first traversal mechanism, the sorting of each PID is in turn: 4(“hers”), 3(“she”), 2(“he”) (the PID of “he” belongs to the inherited PID of node 8), 1(“his”).

[0230] V. Write the structure information of the final AC pattern matching tree into a binary file (i.e., a binary file).

[0231] Exemplarily, the structure information of the AC pattern matching tree can be written into the binary file in the form of TLLV (type of value, total value length, single data length, value array) (i.e., the data value is stored in the form of an array).

[0232] Among them, the tree header information, node information, PID header information, PID information, bitmap header information, and bitmap information can be written into the binary file in the form of TLLV in sequence.

[0233] Among them, for any type of information, a fixed header length information can be stored in the binary file, which is used to store the data type, total data length, and single data size (corresponding to “TLL” in the above “TLLV”). The length of this header length information (i.e., the length of the data header) can be preset, such as 12 bytes.

[0234] Among them, the schematic diagrams of the data structures of the tree header information, node information, PID header information, PID information, bitmap header information, and bitmap information can be respectively as Figures 3A - 3F shown.

[0235] For any type of information (such as tree header information, node information, PID header information, PID information, bitmap header information, or the type of bitmap information), the size of the single data structure is the same.

[0236] Taking node 0 as an example, the value of child_count is 2, which means node 0 has two child nodes; the value of pid_count is 0, which means the number of PIDs attached to node 0 is 0; the value of child_index_start is 1, which means the starting StateID (the minimum State ID of the child nodes) of the child nodes of node 0 is 1; pid_index_start is invalid, which means node 0 does not have any attached PIDs; the value of fail_jump_index is 0, which means the mismatch pointer of node 0 points to node 0; the value of children

[32] in binary representation is {00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,10000000,00010000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000}.

[0237] Among them, children

[32] includes 32 unsigned character data types, with a total of 256 bits. Each bit corresponds to a character in the ASCII code. A bit value of 1 indicates that the node has a child node corresponding to that character, and a bit value of 0 indicates that the node does not have a child node corresponding to that character.

[0238] In the previous example, for node 0, the 0th bit (bit positions start from 0) of the 13th unsigned character (unsigned characters start from 0, that is, the 13th unsigned character is the 14th in the 32 unsigned characters of children

[32] ) is 1, and the 3rd bit of the 14th unsigned character is 1. Combining the sorting of each character in the ASCII code, it can be known that node 0 has child nodes corresponding to the characters 'h' and's'.

[0239] Among them, since the starting State ID of the child nodes of node 0 is 1 and the child_count value is 2, the child nodes of node 0 are node 1 and node 2 (or called child node 1 and child node 2). Also, because the State IDs of different child nodes of the same node are sorted according to the corresponding characters in ASCII code order, it can be known that child node 1 corresponds to the character "h" and child node 2 corresponds to the character "s".

[0240] Taking node 8 as an example, the child_count value is 0, that is, node 8 has no child nodes; the pid_count value is 2, that is, the number of PIDs hanging under node 8 is 2; the value of child_index_start is invalid, that is, the starting State ID of the child nodes of node 8 is an invalid value (no child nodes); pid_index_start is 1, that is, the first PID hanging under node 8 is the second one in the re-sorted PIDs (counting from 0, that is, the indexes from front to back are 0, 1, …, and so on. The index is 1, indicating that it is the second one in the re-sorted PIDs); the value of fail_jump_index is 3, that is, the mismatch pointer of node 8 points to node 3; the value of children

[32] in binary representation is {00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000}.

[0241] Among them, since the number of PIDs hanging under node 8 is 2 and the starting index is 1, the PIDs hanging under node 8 are the second and third ones in the re-sorted (re-sorted according to the depth-first traversal mechanism) PIDs, which are "she" and "he" in sequence.

[0242] In this example, the above-mentioned AC mode matching tree structure optimized by the software is the data structure suitable for use on the FPGA hardware platform. The AC head and part of the AC node content of the output binary file can be seen in Figure 5(PID head, PID node, Bitmap head, Bitmap node are understood in the same way).

[0243] As Figure 5 shown, the tree head information totally includes 20 bytes, TTL (the head length information corresponding to the tree head information) occupies 12 bytes, V occupies 8 bytes, and the specific content of the value (V) includes: TrieID, ACMode, reserve[2], and total_node_number.

[0244] Among them, in this example, the value of TrieID is 1, the value of ACMode is 0 (indicating it belongs to the AC tree), and the value of total_node_num is 10, indicating that the AC tree totally includes 10 nodes.

[0245] The node information totally includes 492 bytes, TTL (the head length information corresponding to the node information) occupies 12 bytes, V occupies 480 bytes (the single data length (the node information of one node) is 48 bytes, that is, it includes 10 nodes), and the specific content form of the value can be referred to Figure 3B .

[0246] Among them, taking the 1st node (the root node, that is, node 0) as an example, in this example, the value of child_count is 2, indicating that the number of child nodes of this node (node 0) is 2; the value of pid_count is 0, indicating that no PID is attached to this node; the value of child_index_start is 1, indicating that the State ID of the first child node of this node is 1; the value of fail_jump_index is 0, indicating that the mismatch pointer of this node points to node 0; children

[32] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 128, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0} (decimal) indicates that the 0th bit of the 13th unsigned character of this node is 1 (the decimal value of the 13th unsigned character is 128, and the binary value is 10000000), and the 3rd bit of the 14th unsigned character is 1 (the decimal value of the 14th unsigned character is 64, and the binary value is 00010000).

[0247] In this embodiment, the binary file generated in the above way can be sent to the FPGA hardware platform, and the FPGA replicates the AC mode matching tree and its related resources in the actual hardware. The process is as follows:

[0248] 1) The upper-layer driver of the FPGA reads the binary file, parses the TLLV in the file, matches the tree header information according to the AC mode with datatype 1 in the header length information, extracts the tree header content, organizes an ACAskResource message and sends it to the FPGA, notifying the FPGA to allocate storage resources ACArea for the AC mode matching tree, that is, notifying the FPGA to allocate storage space for storing the tree header information and node information for the AC mode matching tree, and setting the current position cursor of ACArea to 0.

[0249] Among them, when the FPGA allocates ACArea for the AC mode matching tree, the FPGA can write the AC tree header information and node information included in the binary file into this storage space in sequence. During the process of storing node information, since the size of the array data structure is fixed (the data structure size of a single node information is 48 bytes), therefore, the cursor can be offset according to the data structure size (the cursor can be understood as the array subscript).

[0250] 2) The upper-layer driver continues to read the binary file, parses the TLLV in the file, matches the AC mode matching tree Node information with datatype 2 in the header length information, extracts the AC Node content (that is, the node content), summarizes the AC Node information in the order of reading within the maximum communication message length with the FPGA at one time (such as 1500 bytes) (at most 31 nodes' node information can be transmitted at a time (the node information of a single node is 48 bytes)), and sends an AddACNode message to the FPGA together with the already sent AC Node position cursor (initially 0). The FPGA can store the AC mode matching tree node information in ACArea according to the cursor.

[0251] 3) The upper-layer driver continues to read the binary file, parses the TLLV in the file, matches the PID header information with datatype 3 in the header length information, extracts the PID header content, organizes a PIDAskResource message and sends it to the FPGA, notifying the FPGA to allocate storage resources PIDArea for the AC mode matching tree, and setting the current position cursor of PIDArea to 0.

[0252] 4) The upper-layer driver continues to read the binary file, parses the TLLV in the file, matches the PID Node information with datatype 4 in the header length information, extracts the PID Node content, summarizes the PIDNode information in the order of reading within the maximum communication message length with the FPGA at one time, and sends an AddPIDNode message to the FPGA together with the already sent PID Node position cursor (initially 0). The FPGA can store the PID Node information in PIDArea according to the cursor.

[0253] 5) The upper - layer driver continues to read the binary file, parse the TLLV in the file. According to the Bitmap header information with datatype 5 in the header length information, extract the Bitmap header content, organize a BitmapAskResource message and send it to the FPGA, notify the FPGA to allocate storage resources BitmapArea for the AC - mode matching tree, and set the current position cursor of BitmapArea to 0.

[0254] 6) The upper - layer driver continues to read the binary file, parse the TLLV in the file. According to the Bitmap Node information with datatype 6 in the header length information, extract the Bitmap Node content. Within the maximum length of the communication message with the FPGA at one time, summarize the Bitmap Node information in the reading order, and send an AddBitmapNode message to the FPGA together with the position cursor of the already sent Bitmap Node (initially 0). The FPGA can store the Bitmap Node information in BitmapArea according to the cursor.

[0255] 7) After the upper - layer driver finishes parsing the file, it notifies the FPGA that the distribution of the current AC - mode matching tree resources has ended. After the FPGA processes the message, it respectively checks whether the current cursors of ACArea, PIDArea, and BitmapArea are consistent with the total number information (respectively check the total number of nodes, the total number of PIDs, and the total number of Bitmaps) carried in the corresponding AskResource messages (for example, ACArea corresponds to the ACAskResource message), and sends the verification result to the driver to mutually verify whether the AC - mode matching tree resources are complete.

[0256] Among them, since the Pattern dataset in the above example does not carry case - sensitive information, there is only one Node in the Bitmap.

[0257] Among them, when a Pattern is inserted into the AC tree, a corresponding PID will be allocated. The PID will store the subscript of the PIDBitmap array corresponding to the case - sensitive information of the Pattern. In order to achieve the purpose of PIDBitmap resource reuse, specifically specify that all bits in the first element of the PIDBitmap array are not set. Then, for PIDs that do not need to check case - sensitive information or Patterns without uppercase letters, they can all point to the first member of the PIDbitmap array, that is, PIDBitmapIndex is 0.

[0258] The data finally received by the FPGA should be as follows:

[0259] 1. ACHead: {TrieID: 1, total_node_number: 10}.

[0260] In one example, the ACHead information may further include a current_node_index, which is used to indicate the position of the cursor (i.e., from which position the next batch of data starts to be written) after the current batch of data is written into the corresponding storage space.

[0261] Assume that the number of node information transmitted at one time is not less than 10, then current_node_index = 10, that is, the next batch of data starts to be written after node9.

[0262] 2. ACArea:

[0263]

[0264]

[0265]

[0266] Among them, the Pattern set is {his, he, she, hers}, and the corresponding PID is {1, 2, 3, 4}; the PID array rearranged according to the depth-first traversal mechanism is [4, 3, 2, 1]; among them, the array subscript 0 corresponds to the first element.

[0267] Among them, for node9, the PID it hangs under is the first element in the rearranged PID array. Therefore, the value of "pid_index_start" is 0; the number of PIDs hung under node9 is 1, and the value of "pid_count" is 1.

[0268] For node8, the PIDs it hangs under are the second and third elements in the rearranged PID array. Therefore, the value of "pid_index_start" is 1; the number of PIDs hung under node9 is 2, and the value of "pid_count" is 2.

[0269] The same applies to other nodes.

[0270] 3. PIDHead: {TrieID: 1, total_pid_number: 4}.

[0271] In one example, the PIDHead information may further include a current_pid_index, which is used to indicate the position of the cursor after the current batch of data is written into the corresponding storage space.

[0272] Assume that the number of PID information transmitted at one time is not less than 4, then current_pid_index = 4.

[0273] 4. PIDArea:

[0274] PID Index 0 1 2 3 Value 4 3 2 1

[0275] 5. BitmapHead: {TrieID: 1, total_bitmap_number: 1}.

[0276] In one example, the Bitmap information may further include current_bitmap_index, which is used to indicate the position of the cursor (i.e., from which position the next batch of data starts to be written) after the current batch of data is written into the corresponding storage space.

[0277] Exemplarily, since the number of bitmaps is 1 and can be transmitted in one go, therefore, current_bitmap_index = 1.

[0278] 6. BitmapArea:

[0279]

[0280] Among them, for the process of the FPGA using the above resources for pattern matching, reference can be made to Figure 6 .

[0281] As Figure 6 shown, in this embodiment, when the FPGA receives a pattern matching request, it can obtain the matching string in the pattern matching request, and the AC matching tree specified for matching (which can be identified by TireID).

[0282] The caller can traverse the AC pattern matching tree to be matched, or specify an AC pattern matching tree, and input the matching string and the AC pattern matching tree information into the matching channel.

[0283] The FPGA can open the matching channel, read the ACArea according to the specified TrieID, and determine whether there is a corresponding AC pattern matching tree.

[0284] If there is no corresponding AC pattern matching tree, the result is returned to the request initiator (or called the caller), and the AC pattern matching tree matching ends.

[0285] If there is a corresponding AC pattern matching tree, the FPGA can traverse the matching string starting from each character.

[0286] For the currently traversed character, starting from the Root, find the State that the Path identified by the current character can reach.

[0287] In the case where the State is not found (such as when the Root does not have a State that can be reached by the Path identified by the current character), determine whether the current State is the Root.

[0288] In the case where the current State is the Root, continue to traverse the next character of the matching string.

[0289] In the case where the current State is not the Root, continue to search in the mismatch pointer of the current State for the State that can be reached by the Path identified by the character; if not found, continue to search the mismatch pointer of the mismatch pointer until the State that can be reached by the Path identified by the character is found or the mismatch pointer has pointed to the Root.

[0290] In the case where the State is found (i.e., the Root has a State that can be reached by the Path identified by the current character), determine whether the current State contains a PID index.

[0291] In the case where the current State contains a PID index, notify the PID verification channel to perform verification, and determine whether the current State still has a matching stop flag.

[0292] In the case where the current State contains a matching stop flag, notify the PID verification channel that this match ends. In the case where the current State does not contain a PID index, or, the current State contains a PID index but does not contain a matching stop flag, determine whether the last character of the matching string has been traversed; in the case where the last character of the matching string has been traversed, end the matching channel; otherwise, continue to traverse the next character of the matching string, and continue to search from the current State for the State that can be reached by the Path identified by the currently traversed character.

[0293] Exemplarily, in the case of ending the matching channel, it is also possible to notify the PID verification channel that this match ends and return the match result to the caller.

[0294] In the case of starting the PID verification channel, read the PIDArea according to the specified TrieID and PID index, and query the corresponding PID information.

[0295] Traverse each PID.

[0296] Determine whether the currently traversed PID contains a stop matching flag.

[0297] In the case where it is determined that there is no stop matching flag, it is possible to determine whether write verification is required based on the PID verification flag in the queried PID information. For example, based on the "character_case_flag" included in the PID information, the storage state of the case of pattern characters is determined.

[0298] Among them, the value of "character_case_flag" can include matching uppercase, matching lowercase, matching both cases, or ignoring case.

[0299] When the value of "character_case_flag" is matching uppercase, full uppercase verification can be performed, that is, when the string to be verified is all uppercase, it is determined that the verification passes; otherwise, it is determined that the verification fails.

[0300] When the value of "character_case_flag" is matching lowercase, full lowercase verification can be performed, that is, when the string to be verified is all lowercase, it is determined that the verification passes; otherwise, it is determined that the verification fails.

[0301] When the value of "character_case_flag" is matching both cases, the Bitmap Area can be read based on the specified TrieID and the Bitmap index (bitmap_index) information in the PID information, the corresponding Bitmap is queried, and case verification is performed based on the queried Bitmap.

[0302] When the value of "character_case_flag" is ignoring case, it can be determined that case verification is not required.

[0303] It should be noted that when the pattern_max_len is also included in the queried PID information, the FPGA can, based on pattern_max_len, start from the current character and read a string with a length consistent with pattern_max_len (which can be called the target string) from the matching string, and perform case verification on the read string.

[0304] In addition, when the queried PID information also includes pattern_start_position and pattern_end_position, it is also possible to determine whether the start position and end position of the Pattern corresponding to the index of the Pattren included in the current State in the matching string (which can be referred to as the target start position and target end position) match pattern_start_position and pattern_end_position (that is, the start position of the Pattern in the matching string is the same as pattern_start_position, and the end position of the Pattern in the matching string is the same as pattern_end_position); if they match, subsequent processing can be continued, such as performing case sensitivity verification; otherwise, it is determined that the match fails.

[0305] When the verification is successful, record the PID and the corresponding Pattern in the matching result, and continue to verify the next PID until the PID has been traversed, and then end the verification channel; otherwise, verify the next PID until the PID has been traversed, and then end the verification channel.

[0306] It should be noted that for any PID, the above verification success means that: the verification results for this PID (verification of this PID based on the verification flags recorded in the PID information of this PID, such as verification of the pattern start position and pattern end position, case sensitivity verification, etc.) are all successful. In this case, the PID and the corresponding Pattern can be recorded in the matching result.

[0307] Exemplarily, the matching result can also record AC tree information, such as the identifier of the AC pattern matching tree (TrieID), the node information corresponding to the PID in the AC pattern matching tree, etc.

[0308] In addition, when any verification result for the PID is verification failure, it can be determined that the verification result for this PID is verification failure. In this case, it is not necessary to record the PID and the corresponding Pattern in the matching result.

[0309] When it is determined that case sensitivity does not need to be verified, the matched Pattern can be recorded in the matching result, and continue to verify the next PID until the PID has been traversed, and then end the verification channel.

[0310] When the queried PID information includes a stop matching flag, it is possible to determine whether the matching channel has received an end matching message and the last State ID and its PID index information that need to be verified.

[0311] When receiving the message indicating the end of matching for the matching channel and the State ID to be finally verified and its PID index information, determine whether the traversal of PIDs has ended.

[0312] Among them, when receiving the message indicating the end of matching for the matching channel and the State ID to be finally verified and its PID index information, it is determined that the matching channel has ended and there is no need to wait for a new PID verification notification. In this case, the verification can be ended after traversing and verifying the currently received PIDs.

[0313] When the traversal of PIDs has ended, end the verification channel.

[0314] When the traversal of PIDs has not ended, continue to traverse the next PID.

[0315] When not receiving the message indicating the end of matching for the matching channel and the State ID to be finally verified and its PID index information, determine whether the traversal of PIDs has ended.

[0316] Among them, when not receiving the message indicating the end of matching for the matching channel and the State ID to be finally verified and its PID index information, it is determined that the matching channel has not ended and a new PID verification notification needs to be waited for. In this case, even after traversing and verifying the currently received PIDs, it is still necessary to wait for whether a new PID verification notification or the message indicating the end of matching for the matching channel is received.

[0317] When both the matching channel and the verification channel have ended, the matching results including the AC tree information and PIDs can be sorted out and the matching results can be returned to the caller.

[0318] Please refer to Figure 7 , which is a schematic structural diagram of an AC mode matching tree construction device provided by an embodiment of the present invention. As Figure 7 shown, the AC mode matching tree construction device may include:

[0319] A construction unit 710, configured to construct an AC mode matching tree by inserting each mode in the mode set into a trie tree respectively according to the mode set;

[0320] A reordering unit 720, configured to assign state identifiers to each node of the AC mode matching tree in sequence according to the breadth-first traversal mechanism;

[0321] The reordering unit 720 is further configured to, according to the breadth-first traversal mechanism, for multiple different child nodes of the same node, reorder the state identifiers of the multiple different child nodes according to the characters corresponding to the edges connecting the node to different child nodes in the ASCII code order to obtain the final AC mode matching tree.

[0322] In one example, the apparatus further includes:

[0323] A writing unit, configured to, when the final AC pattern matching tree is obtained, determine the file writing order of the pattern indexes PID (Pattern ID) attached to each node in the final AC pattern matching tree according to the depth-first traversal mechanism; wherein, the PIDs attached to a node include the PIDs of the patterns with the node as the tail node, and the PIDs inherited by the node; for the PIDs attached to the same node, the writing order of the PIDs with the node as the tail node is before the PIDs inherited by the node; according to the file writing order of the PIDs, write the structure information of the final AC pattern matching tree into a binary file in a form of separating nodes from PIDs.

[0324] In one example, the writing unit writes the structure information of the final AC pattern matching tree into a binary file in a form of separating nodes from PIDs according to the file writing order of the PIDs, including:

[0325] Write the tree head information of the final AC pattern matching tree into the binary file; wherein, the tree head information includes a tree identifier, an AC pattern identifier, and the number of nodes;

[0326] Write the node information of the final AC pattern matching tree into the binary file according to the breadth-first traversal order; wherein, the node information includes the number of child nodes, the number of attached PIDs, the starting index of child nodes, the starting index of PIDs, the mismatch pointer index, and some or all of the characters corresponding to the child nodes; the characters corresponding to the child nodes are stored at the bit level, and one bit corresponds to one character;

[0327] Write the PID head information of the final AC pattern matching tree into the binary file; wherein, the PID head information includes a tree identifier and the number of PIDs;

[0328] Write the PID information of the final AC pattern matching tree into the binary file according to the file writing order of the PIDs; wherein, the PID information includes some or all of the PID, a matching stop flag, the case storage state of the pattern string characters, the maximum length of the pattern string, the starting position of the pattern, the ending position of the pattern, and the bitmap index.

[0329] In one example, the PID information further includes a bitmap index, and the bitmap is used to record the case information of each character in the corresponding pattern;

[0330] The writing unit writes the structure information of the final AC pattern matching tree into a binary file according to the file writing order of the PID, in a form where nodes are stored separately from the PID, and further includes:

[0331] Write the bitmap header information of the final AC pattern matching tree into the binary file; wherein, the bitmap header information includes a tree identifier and the number of bitmaps;

[0332] Write the bitmap information of the final AC pattern matching tree into the binary file; wherein, the bitmap length is determined according to the depth of the AC pattern matching tree.

[0333] In one example, the structure information of the final AC pattern matching tree is written into the binary file in the TLLV form;

[0334] The TLLV form includes a value type, the total length of the value, the length of a single data, and a data value; the data value is used to record the structure information of the final AC pattern matching tree.

[0335] In one example, the structure information in the TLLV form includes a fixed-length header length information, and at least one single data structure; the header length information includes a value type, the total length of the value, and the length of a single data;

[0336] For the same type of structure information, the sizes of the single data structures are the same.

[0337] Please refer to Figure 8 , which is a schematic structural diagram of an AC pattern matching tree matching device provided by an embodiment of the present invention. As Figure 8 shown, the AC pattern matching tree matching device may include:

[0338] An acquisition unit 810, configured to acquire a binary file; wherein, the binary file is generated by using the method described in the above embodiment;

[0339] An analysis unit 820, configured to analyze the binary file and extract the structure information of the final AC pattern matching tree;

[0340] A sending unit 830, configured to send the structure information of the final AC pattern matching tree to a field programmable gate array (FPGA), so that the FPGA stores the structure information of the final AC pattern matching tree in a storage space and performs pattern matching according to the structure information of the final AC pattern matching tree stored in the storage space.

[0341] In one example, the structure information of the final AC pattern matching tree includes tree header information, node information, PID header information, and PID information;

[0342] The sending unit sends the structure information of the final AC pattern matching tree to the FPGA, including:

[0343] According to the tree header information, send an AC pattern matching tree resource request message to the FPGA, so that the FPGA allocates an AC pattern matching tree storage space for the AC pattern matching tree and stores the tree header information in the AC pattern matching tree storage space;

[0344] Send the node information to the FPGA, so that the FPGA stores the node information in the AC pattern matching tree storage space;

[0345] According to the PID header information, send a PID resource request message to the FPGA, so that the FPGA allocates a PID storage space for the AC pattern matching tree and stores the PID header information in the PID storage space;

[0346] Send the PID information to the FPGA, so that the FPGA stores the PID information in the PID storage space.

[0347] In one example, the structure information of the final AC pattern matching tree further includes a bitmap header and bitmap information;

[0348] The sending unit sends the structure information of the final AC pattern matching tree to the field programmable gate array FPGA, and further includes:

[0349] According to the bitmap header information, send a bitmap resource request message to the FPGA, so that the FPGA allocates a bitmap storage space for the AC pattern matching tree and stores the bitmap header information in the bitmap storage space;

[0350] Send the bitmap information to the FPGA, so that the FPGA stores the bitmap information in the bitmap storage space.

[0351] In one example, the FPGA performs pattern matching according to the structure information of the final AC pattern matching tree stored in the storage space, including:

[0352] When receiving a pattern matching request, obtain the matching string in the pattern matching request and the tree identifier of the specified AC matching tree to be matched;

[0353] When the corresponding AC pattern matching tree is found in the AC pattern matching tree storage space according to the specified tree identifier, traverse the matching string;

[0354] For the currently traversed character, start from the root node of the AC pattern matching tree to find the node that the path Path identified by the character can reach;

[0355] When the node is found and the node contains the index of the pattern, verify the pattern corresponding to the index of the pattern contained in the node, and determine the matching result according to the verification result.

[0356] In one example, the FPGA verifies the pattern corresponding to the index of the pattern contained in the node, and determines the matching result according to the verification result, including:

[0357] Query the corresponding PID information in the PID storage space according to the index of the pattern contained in the node and the specified tree identifier;

[0358] When the corresponding PID information is queried and the PID information includes the pattern start position and the pattern end position, determine the target start position and the target end position of the pattern corresponding to the index of the pattern contained in the node in the matching string;

[0359] When the target start position and the target end position match the pattern start position and the pattern end position included in the PID information, determine that the verification is successful;

[0360] When the target start position and the target end position do not match the pattern start position and the pattern end position included in the PID information, determine that the verification fails;

[0361] And / or,

[0362] When the corresponding PID information is queried and the case storage status of the pattern characters included in the PID information indicates that the case needs to be verified, read the target string with a length consistent with the maximum pattern length from the matching string according to the maximum pattern length included in the PID information;

[0363] If the case storage status of the pattern is to match lowercase, perform a full lowercase verification on the target string;

[0364] If the case storage status of the pattern is to match uppercase, perform a full uppercase verification on the target string;

[0365] If the case storage status of the pattern is to match case, query the corresponding bitmap information in the bitmap storage space according to the bitmap index included in the PID information and the specified tree identifier, and perform a case verification on the target string according to the queried bitmap information;

[0366] For any PID, when the verification results for this PID are all successful, record this PID and the corresponding mode in the matching result.

[0367] An embodiment of the present invention further provides a network device, including an upper-layer driver and an FPGA; wherein:

[0368] The upper-layer driver is used to obtain a binary file; wherein, the binary file is generated by the method described in the above embodiment; parse the binary file, extract the structure information of the final AC pattern matching tree; and send the structure information of the final AC pattern matching tree to the FPGA.

[0369] The FPGA is used to store the structure information of the final AC pattern matching tree in the storage space, and perform pattern matching according to the structure information of the final AC pattern matching tree stored in the storage space.

[0370] Among them, for the specific implementation of the upper-layer driver sending the structure information of the final AC pattern matching tree to the FPGA, reference can be made to the relevant description in the above embodiment.

[0371] For the specific implementation of the FPGA performing pattern matching, reference can be made to the relevant description in the above embodiment.

Claims

1. A method for constructing an AC pattern matching tree, characterized in that, Including: According to the pattern set, an AC pattern matching tree is constructed by inserting each pattern in the pattern set into the trie tree respectively; According to the breadth-first traversal mechanism, status identifiers are assigned to each node of the AC pattern matching tree in sequence; According to the breadth-first traversal mechanism, for multiple different child nodes of the same node, based on the characters corresponding to the edges connecting the different child nodes of the node, in the order of ASCII code, the status identifiers of the multiple different child nodes are rearranged to obtain the final AC pattern matching tree.

2. The method according to claim 1, wherein In the case where the final AC pattern matching tree is obtained, the method further includes: According to the depth-first traversal mechanism, determine the file writing order of the pattern indexes PID hanging under each node in the final AC pattern matching tree; where the PID hanging under the node includes the PID of the pattern with the node as the tail node, and the PID inherited by the node; for the PID hanging under the same node, the writing order of the PID with the node as the tail node is before the PID inherited by the node; According to the file writing order of the PID, write the structure information of the final AC pattern matching tree into a binary file in the form of separating nodes from PIDs for storage.

3. The method according to claim 2, characterized in that, The writing the structure information of the final AC pattern matching tree into a binary file in the form of separating nodes from PIDs for storage according to the file writing order of the PID includes: Write the tree head information of the final AC pattern matching tree into the binary file; where the tree head information includes the tree identifier, the AC pattern identifier, and the number of nodes; Write the node information of the final AC pattern matching tree into the binary file according to the breadth-first traversal order; where the node information includes the number of child nodes, the number of PIDs hanging under, the starting index of the child nodes, the starting index of the PIDs, the index of the mismatch pointer, and some or all of the characters corresponding to the child nodes; the characters corresponding to the child nodes are stored at the bit level, with one bit corresponding to one character; Write the PID head information of the final AC pattern matching tree into the binary file; where the PID head information includes the tree identifier and the number of PIDs; According to the file writing order of the PID, write the PID information of the final AC pattern matching tree into the binary file; where the PID information includes some or all of the PID, the matching stop flag, the case storage status of the pattern characters, the maximum length of the pattern, the starting position of the pattern, the ending position of the pattern, and the bitmap index.

4. The method according to claim 3, wherein The PID information further includes a bitmap index, and the bitmap is used to record the case information of each character in the corresponding pattern; The writing the structure information of the final AC pattern matching tree into a binary file in the form of separating nodes from PIDs for storage according to the file writing order of the PID further includes: Write the bitmap head information of the final AC pattern matching tree into the binary file; where the bitmap head information includes the tree identifier and the number of bitmaps; Write the bitmap information of the final AC pattern matching tree to the binary file; wherein, the bitmap length is determined according to the depth of the AC pattern matching tree.

5. The method according to any one of claims 2-4, characterized in that, The structure information of the final AC pattern matching tree is written to the binary file in the TLLV form; The TLLV form includes value type, total length of the value, length of a single data, and data value; The data value is used to record the structure information of the final AC pattern matching tree.

6. The method according to claim 5, characterized in that, The structure information in the TLLV form includes fixed-length header length information and at least one single data structure; the header length information includes value type, total length of the value, and length of a single data; For the same type of structure information, the sizes of the single data structures are the same.

7. An AC pattern matching tree matching method, characterized in that, Includes: Obtain a binary file; wherein, the binary file is generated by the method according to any one of claims 2-6. Parse the binary file to extract the structure information of the final AC pattern matching tree; Send the structure information of the final AC pattern matching tree to the field programmable gate array FPGA, so that the FPGA stores the structure information of the final AC pattern matching tree in the storage space and performs pattern matching according to the structure information of the final AC pattern matching tree stored in the storage space.

8. The method according to claim 7, wherein The structure information of the final AC pattern matching tree includes tree header information, node information, PID header information, and PID information; The sending the structure information of the final AC pattern matching tree to the FPGA includes: According to the tree header information, send an AC pattern matching tree resource request message to the FPGA, so that the FPGA allocates an AC pattern matching tree storage space for the AC pattern matching tree and stores the tree header information in the AC pattern matching tree storage space; Send the node information to the FPGA, so that the FPGA stores the node information in the AC pattern matching tree storage space; According to the PID header information, send a PID resource request message to the FPGA, so that the FPGA allocates a PID storage space for the AC pattern matching tree and stores the PID header information in the PID storage space; Send the PID information to the FPGA, so that the FPGA stores the PID information in the PID storage space.

9. The method according to claim 8, characterized in that, The structure information of the final AC pattern matching tree further includes a bitmap header and bitmap information; The sending the structure information of the final AC pattern matching tree to the field programmable gate array FPGA further includes: According to the bitmap header information, send a bitmap resource request message to the FPGA, so that the FPGA allocates a bitmap storage space for the AC pattern matching tree and stores the bitmap header information in the bitmap storage space; Send the bitmap information to the FPGA, so that the FPGA stores the bitmap information in the bitmap storage space.

10. The method according to claim 7, wherein The FPGA performing pattern matching according to the structure information of the final AC pattern matching tree stored in the storage space includes: Upon receiving a pattern matching request, obtain the matching string in the pattern matching request and the tree identifier of the AC matching tree specified for matching. When the corresponding AC pattern matching tree is found in the AC pattern matching tree storage space according to the specified tree identifier, traverse the matching string. For the currently traversed character, start from the root node of the AC pattern matching tree to find the node that the path Path identified by the character can reach. When the node is found and the node contains the index of the pattern, verify the pattern corresponding to the index of the pattern contained in the node and determine the matching result according to the verification result.

11. The method according to claim 10, characterized in that, The verifying the pattern corresponding to the index of the pattern contained in the node and determining the matching result according to the verification result includes: Query the corresponding PID information in the PID storage space according to the index of the pattern contained in the node and the specified tree identifier. When the corresponding PID information is found and the PID information includes the pattern start position and the pattern end position, determine the target start position and the target end position of the pattern corresponding to the index of the pattern contained in the node in the matching string. When the target start position and the target end position match the pattern start position and the pattern end position included in the PID information, determine that the verification is successful. When the target start position and the target end position do not match the pattern start position and the pattern end position included in the PID information, determine that the verification fails. And / or When the corresponding PID information is found and the pattern character case storage status included in the PID information indicates that case needs to be verified, read a target string with a length consistent with the maximum pattern length from the matching string according to the maximum pattern length included in the PID information. If the pattern case storage status is to match lowercase, perform a full lowercase verification on the target string. If the pattern case storage status is to match uppercase, perform a full uppercase verification on the target string. If the pattern case storage status is to match both cases, query the corresponding bitmap information in the bitmap storage space according to the bitmap index included in the PID information and the specified tree identifier, and perform a case verification on the target string according to the queried bitmap information. For any PID, when the verification results for this PID are all successful, record this PID and the pattern corresponding to the PID in the matching result.

12. An apparatus for constructing an AC pattern matching tree, characterized in that, Include: A construction unit for constructing an AC pattern matching tree by inserting each pattern in the pattern set into a trie tree respectively according to the pattern set. A reordering unit for sequentially assigning status identifiers to each node of the AC pattern matching tree according to the breadth-first traversal mechanism. The reordering unit is further configured to, according to the breadth-first traversal mechanism, for multiple different child nodes of the same node, reorder the status identifiers of the multiple different child nodes according to the characters corresponding to the edges connecting the node to different child nodes in the ASCII code order to obtain the final AC pattern matching tree.

13. An AC mode matching tree matching device, characterized in that, Include: An acquisition unit, configured to acquire a binary file; wherein, the binary file is generated by using the method according to any one of claims 2-6; A parsing unit, configured to parse the binary file and extract the structural information of the final AC pattern matching tree; A sending unit, configured to send the structural information of the final AC pattern matching tree to a field programmable gate array (FPGA), so that the FPGA stores the structural information of the final AC pattern matching tree in a storage space, and performs pattern matching according to the structural information of the final AC pattern matching tree stored in the storage space.

14. A network device, characterized in that, It includes an upper layer driver and a field programmable gate array (FPGA); wherein: The upper layer driver is configured to acquire a binary file; wherein, the binary file is generated by using the method according to any one of claims 2-6; parse the binary file and extract the structural information of the final AC pattern matching tree; send the structural information of the final AC pattern matching tree to the FPGA; The FPGA is configured to store the structural information of the final AC pattern matching tree in a storage space, and perform pattern matching according to the structural information of the final AC pattern matching tree stored in the storage space.

Images