A multi-pattern matching method for data name matching

Abstract

The present invention relates to a multi-pattern matching method for data name matching. The method includes: for any component of the data name, query the component Code in the preprocessing table with this component as the Key. If found, replace this component with the queried component Code, and query the Data pointer with this component as the Key. The Pattern pointed to by the queried Data pointer is the matching single-component Pattern. If the component Code is not found, replace this component with an abnormal component identifier to obtain a componentized data name. Use the Wu-Manber algorithm with the component Code as a character block to determine the matching non-single-component Pattern. The method of the present invention realizes fast and accurate matching of data names.

Description

Technical Field

[0001] The present invention relates to the fields of named data networking, multi-pattern matching, and packet forwarding, and in particular to a multi-pattern matching method for data name matching. Background Art

[0002] In a new Internet architecture centered on named data, in addition to the data in the packet payload, the data name in the packet header also needs to be matched and recognized. As a representative of such a new Internet architecture, the named data network (NDN) has built-in support for regular matching of data names in its standard protocol stack NDN-CXX. However, with the growth of the network scale and the number of users, especially the arrival of the 5G network era, the number of user rules can reach millions. The single-pattern matching method requires constructing pattern rules one by one for identification, which is impractical and requires the use of a multi-pattern matching algorithm.

[0003] Existing multi-pattern matching algorithms, such as the Aho-Corasick automaton algorithm, the Wu-Manber algorithm, and the multi-pattern matching version of the Shift-OR algorithm built into Intel's open-source tool Hyperscan, are used for byte-level matching of data streams and are applied in many Internet application scenarios, such as intrusion detection systems, access control systems, and deep packet inspection systems. In these scenarios, the detection target is usually the data stream carried in the packet header, such as a URL string.

[0004] A data name is a data type with a hierarchical structure, and its smallest component unit is a component, which is in the form of a string of indefinite length. For example, for the data name A with the specific content / dataset / resource / data / image / png / people, it contains 6 components: dataset, resource, data, image, png, and people. Among them, the ' / ' symbol is only used as a separator for components and is not an essential part of the data name. When performing matching, the Pattern is also in the form of a component. For example, the Patterns that can match the data name A are / dataset / resource, / data, and / image / png, and the Patterns that cannot match are / data / video and / resource / meta.

[0005] The existing multi-pattern matching algorithms have the following problems, making it difficult to directly apply them to the multi-pattern matching scenario of data names: First, the existing multi-pattern matching algorithms are applicable to flattened string data. When applied to the multi-pattern matching of data names, in order to adapt to the algorithm, the data name needs to be first converted into the form of a flattened string. Thus, it cannot be ensured that the matching performed is component-level matching, and further, the unmatched patterns cannot be completely excluded. It is necessary to further verify whether the matching result is valid. For example, the pattern / dataset / res can also match the data name A, but the end position of the matching is only a partial prefix of the component rather than the complete component, and such a matching is invalid. Second, most of the existing multi-pattern matching algorithms are offline algorithms, with low update efficiency or even no support for rule updates. Third, when processing a Pattern set on the scale of millions, the existing algorithms take several minutes or even more than ten minutes to complete the initialization construction, and the storage overhead is at the GB level, with high computing and storage requirements for network devices. Summary of the Invention

[0006] In view of this, the main object of the present invention is to provide a multi-pattern matching method for data name matching, which realizes fast and accurate matching of data names through component-level matching, avoiding additional calculations such as format conversion and result verification.

[0007] To achieve the above object, the present application provides a multi-pattern matching method for data name matching, including:

[0008] For any component of the data name, query the component Code in the preprocessing table with this component as the Key. If found, replace this component with the queried component Code, and query the Data pointer with this component as the Key. The Pattern pointed to by the queried Data pointer is the single-component Pattern that matches; if the component Code is not found, replace this component with an abnormal component identifier to obtain a componentized data name;

[0009] Use the Wu-Manber algorithm with the component Code as the character block to determine the non-single-component Pattern that matches;

[0010] Among them, the preprocessing table is created when initializing the Pattern set. When replacing each component of the non-single-component Pattern with the component Code, the mapping relationship between each component and its component Code is saved; and the mapping relationship between the component of each single-component Pattern and the Data pointer of the single-component Pattern is saved;

[0011] The component Code and the abnormal component identifier have the same length, both being fixed-length strings.

[0012] In a possible implementation, for any component of a non-single-component Pattern, replacing it with component Code and saving the mapping relationship between the component and its component Code includes:

[0013] Query the component Code with this component as the key from the preprocessing table. If the query is successful, replace this component in the non-single-component Pattern with the queried component Code; if the query fails, convert this component to component Code, insert the mapping relationship between this component and the component Code into the preprocessing table, and replace this component in the non-single-component Pattern with this component Code.

[0014] In another possible implementation, maintain an arbitrary initial encoding CodeBase; correspondingly,

[0015] The conversion of the component to component Code includes:

[0016] Increment CodeBase by 1 as CodeBase;

[0017] Use CodeBase as the component Code of this component.

[0018] In another possible implementation, for any component of a non-single-component Pattern, replacing it with component Code and saving the mapping relationship between the component and its component Code includes:

[0019] Query the component Code with this component as the key from the preprocessing table. If the query is successful, replace this component in the non-single-component Pattern with the queried component Code; if the query fails, convert this component to component Code, insert the mapping relationship between this component and the component Code into the preprocessing table, and replace this component in the non-single-component Pattern with this component Code.

[0020] In another possible implementation, use the length of the shortest encoded non-single-component Pattern in the Pattern set as the window length, set the window, and align each encoded non-single-component Pattern to the left of the window; for any encoded non-single-component Pattern, insert the mapping relationship between each component Code and Shift within the window and the mapping relationship between the suffix component Code and the Prefix pointer within the window into the Shift table, and insert the mapping relationship between the prefix component Code and its length into the Prefix table pointed to by this Prefix pointer; insert the mapping relationship between the encoded non-single-component Pattern with length i and its Data pointer into the Rule table: Len = i;

[0021] Among them, for the mapping relationship between the Code and Shift of each component, when a hash conflict occurs, the smaller value of Shift is taken as the valid value;

[0022] The encoded non-single-component Pattern, that is, the non-single-component Pattern in which all components are replaced by component Codes.

[0023] In another possible implementation, the determination of the matching non-single-component Pattern includes:

[0024] Align the window and the encoded data name to the left;

[0025] If there is an abnormal component identifier in the window, shift the window to the right to align with the first component Code after the abnormal component identifier, and continue the match;

[0026] If there is no abnormal component identifier in the window, take the suffix component Code of the encoded data name in the window as the Key, query Shift in the Shift table. If the queried Shift is non-zero, shift the window to the right by that Shift number of components and continue the match; if the queried Shift is 0, take the prefix component Code of the encoded data name in the window as the Key, query the length in the Prefix table pointed to by the Prefix pointer associated with the suffix component Code. If the query is successful, take the length not greater than the number of component Codes from the start position of the window to the end of the encoded data name as k. Starting from the start position of the window, take k adjacent component Codes to the right as the Key, query the Data pointer in the Rule table: Len = k. If the query is successful, the Pattern pointed to by the queried Data pointer is the matching non-single-component Pattern. Shift the window to the right by 1 component and continue the match; if the Shift query fails, the length query fails, there is no length not greater than the number of component Codes from the start position of the window to the end of the encoded data name, or the Data pointer query fails, shift the window to the right by 1 component and continue the match.

[0027] In another possible implementation, before determining the matching non-single-component Pattern, it further includes:

[0028] Create hash tables named Shift table, Prefix table, and Rule table: Len = i respectively, where i ∈ (2, 3,..., n);

[0029] Set the window length to the length of the shortest encoded non - single - component Pattern in the Pattern set. Set the window, and align each encoded non - single - component Pattern with the left side of the window; for any encoded non - single - component Pattern, insert the mapping relationship between each component Code and Shift within the window into the Shift table; and form a Key with its prefix component Code and the suffix component Code within the window, and insert the mapping relationship with its length into the Prefix table; insert the mapping relationship between the encoded non - single - component Pattern with length i and its Data pointer into the Rule table: Len = i; where,

[0030] For the mapping relationship between each component Code and Shift, when a hash collision occurs, take the smaller value of Shift as the valid value.

[0031] In another possible implementation, the determination of the matching non - single - component Pattern includes:

[0032] Align the window with the left side of the encoded data name;

[0033] If there is an abnormal component identifier within the window, shift the window to the left alignment with the first component Code after this abnormal component identifier, and continue the matching;

[0034] If there is no abnormal component identifier within the window, take the suffix component Code of the encoded data name within the window as the Key, query Shift in the Shift table. If the queried Shift is non - zero, shift the window by that Shift number of components and continue the matching; if the queried Shift is 0, form a Key with the prefix component Code and the suffix component Code of the encoded data name within the window, query the length in the Prefix table. If the query is successful, take the length obtained from the query that is not greater than the number of component Codes from the start position of the window to the end of the encoded data name, denoted as k. Starting from the start position of the window, take k adjacent component Codes to the right as the Key, query the Data pointer in the Rule table: Len = k. If the query is successful, the Pattern pointed to by the queried Data pointer is the matching non - single - component Pattern, shift the window by 1 component and continue the matching; if the Shift query fails, the length query fails, there is no length obtained from the query that is not greater than the number of component Codes from the start position of the window to the end of the encoded data name, or the Data pointer query fails, shift the window by 1 component and continue the matching.

[0035] In another possible implementation, add a counter list attribute to the Shift table, and the subscript of the counter list is Shift;

[0036] Add a non - single - component Pattern to the Pattern set. For any component Code of this non - single - component Pattern within the window, perform the following operations on the Shift table entry corresponding to this component Code: increment the counter with the subscript of this component Code Shift by 1; and when a hash collision occurs, take the smaller value of the Shift attribute value.

[0037] Delete a non - single - component Pattern from the Pattern set. For any component Code of this non - single - component Pattern within the window, perform the following operations on the Shift table entry corresponding to this component Code: decrement the counter with the subscript of this component Code Shift by 1. When the counter becomes 0 after the decrement, search for non - zero counters in the counter list. If found, and the subscripts of the found non - zero counters are all different from the Shift attribute value, then use the smaller value among the subscripts of the non - zero counters as the shift attribute value; if not found, delete this Shift table entry. Description of the Drawings

[0038] Figure 1 It is a schematic diagram of the working principle of the Wu - Manber algorithm;

[0039] Figure 2 It is a schematic flowchart of a multi - pattern matching method for data name matching according to an embodiment of the present invention;

[0040] Figure 3 It is a schematic diagram of the working principle of component encoding when the Prefix table adopts a local pattern according to an embodiment of the present invention;

[0041] Figure 4 It is a schematic diagram of the working principle of determining a matching non - single - component Pattern when the Prefix table adopts a local pattern according to an embodiment of the present invention;

[0042] Figure 5 It is a comparison diagram of the Shift table and the Prefix table when the Prefix table adopts local and global patterns according to an embodiment of the present invention;

[0043] Figure 6 It is a schematic diagram of the design principle of the Shift table according to an embodiment of the present invention. Detailed Embodiments

[0044] The Wu - Manber algorithm is a multi - pattern matching algorithm with a hash table as the basic data structure, and its basic working principle is as Figure 1As shown in the figure. The figure contains pattern rules A = "abcdefgf" and B = "fghcdh". After aligning the patterns to the left, a sliding window of a fixed length is selected and aligned with the pattern rules to the left, and the substrings covered within the window are taken. Among them, the length W of the sliding window depends on the shortest rule among the pattern rules. The Wu-Manber algorithm needs to calculate the offsets of each character within the part of the pattern rule covered by the window, and the specific calculation method is to take every B characters as an inseparable character block and calculate the offset of the entire character block from the right side of the window. The value of B is adjustable. Figure 1 In this case, B is taken as 2, that is, every 2 characters form a character block. For pattern rule A, the offset of "ef" is 0 because it is exactly at the rightmost side of the window. For the character blocks "de", "cd", "bc", and "ab" selected from right to left in sequence, their offsets increase. The role of the Shift table is to save the mapping relationship between each character block and its offset (Shift), but there may be conflicts, that is, the same character blocks from different pattern rules, or character blocks of the same pattern but different offsets. Shift adopts the principle of the smallest valid value, that is, when there is a conflict, the record with the smallest Shift is retained. For example, Figure 1 both pattern rules A and B in the figure contain the character block "cd", and the Shifts are 2 and 1 respectively, and only the minimum value is retained in the Shift table, so the final Shift of "cd" is 1. The character blocks with Shift equal to 0 indicate that the character block is the suffix of the part of a certain pattern rule covered by the window, and they will be inserted into another hash table - the associated hash table, which is used to save the mapping relationship between these suffix character blocks and their corresponding rules. The associated hash table also has a conflict problem, that is, different pattern rules have the same suffix character blocks, but different from the principle of the smallest valid value of the Shift table, the linked list method is used to solve the conflict. The linked list that saves the pattern rule conflicts is called the Prefix linked list because each linked list node contains not only the pattern rule information but also the prefix, that is, the leftmost character block within the window coverage range.

[0045] During matching, the target string also requires the assistance of a sliding window, and the window length is the same as when the rules are inserted. The specific steps are as follows:

[0046] During initialization, the window is aligned with the left side of the target string.

[0047] Take the suffix character block within the window and query the Shift table: If the search fails, the window is shifted to the right by the length of the entire window; if the window matches successfully, the window is shifted to the right according to the queried Shift, and repeat step 2; when the Shift of the suffix character block is queried to be 0, enter step 3.

[0048] Query the associated hash table and traverse the corresponding Prefix linked list. If a successfully matched pattern rule is found, record it, and finally shift the window one character block to the right.

[0049] If the remaining part of the target string exceeds the window length, after the window slides, return to step 2; otherwise, end the matching.

[0050] Specifically, the flow of a multi-pattern matching method for data name matching according to an embodiment of the present invention is as Figure 2 shown, including steps 201 to 104.

[0051] 201. For any component of the data name, query the component Code in the preprocessing table with this component as the Key. If found, replace this component with the queried component Code, and query the Data pointer with this component as the Key. The Pattern pointed to by the queried Data pointer is the matching single-component Pattern; if the component Code is not found, replace this component with an abnormal component identifier to obtain a componentized data name.

[0052] 202. Use the Wu-Manber algorithm with the component Code as a character block to determine the matching non-single-component Pattern.

[0053] Among them, the preprocessing table is created when initializing the Pattern set. When replacing each component of the non-single-component Pattern with the component Code, save the mapping relationship between each component and its component Code; and save the mapping relationship between the component of each single-component Pattern and the Data pointer of the single-component Pattern;

[0054] The component Code and the abnormal component identifier have the same length, both are fixed-length strings.

[0055] Here, the component is a variable-length string. In the embodiment of the present invention, it is converted into a fixed-length component code (Code) or an abnormal component identifier to achieve component-level matching. The key of the preprocessing table is the component, and the Data part includes the component Code and the data pointer (PTR), that is, the preprocessing table is used to save the mapping relationship between the component and the corresponding component Code and Data pointer. There are various specific implementation options, such as hash tables, prefix trees (Trie), and red-black trees, etc. In the embodiment of the present invention, the preprocessing table is implemented using a hash table. Specifically, the murmurhash algorithm can be used for implementation, and the linked list method is used to solve hash conflicts.

[0056] In a possible implementation, the preprocessing table includes the attribute Key, Code, and PTR. Correspondingly,

[0057] Replace any component of the non - single - component Pattern with component Code, and insert the mapping relationship between this component and component Code into the pre - processing table, including:

[0058] A1. From the pre - processing table, query component Code with this component as the Key. If the query is successful, execute step B1; if the query fails, execute step C1;

[0059] B1. Replace this component in the non - single - component Pattern with the queried component Code;

[0060] C1. Convert this component to component Code, insert the mapping relationship between this component and component Code into the pre - processing table, and replace this component in the non - single - component Pattern with this component Code.

[0061] In another possible implementation, it further includes maintaining an arbitrary initial coding CodeBase; correspondingly,

[0062] The conversion of the component to component Code includes:

[0063] Increment CodeBase by 1 as CodeBase;

[0064] Use CodeBase as the component Code of this component.

[0065] Here, in order to ensure that there are no Code conflicts, that is, different components are mapped to the same component Code, the embodiments of the present invention adopt a simple increment - coding method as described above.

[0066] In another possible implementation, the pre - processing table includes attribute Key, Code, and PTR. Correspondingly,

[0067] Insert the mapping relationship between the component of any single - component Pattern and its Data pointer into the pre - processing table, including:

[0068] A2. From the pre - processing table, query component Code with this component as the Key. If the query is successful, execute step B2; if the query fails, execute step C2;

[0069] B2. Add a pointer to the Data pointer of this single - component Pattern to the PTR corresponding to this component;

[0070] C2. Insert the mapping relationship between this component and the Data pointer pointing to this single - component Pattern into the pre - processing table.

[0071] In another possible implementation, in step 203, for any component of the data name, replacing the component and recording the Data pointer corresponding to the component Code found includes:

[0072] A3. From the preprocessing table, query the component Code with this component as the Key. If the query is successful, execute step B2; otherwise, execute step C3.

[0073] B3. Replace the component in the data name with the found component Code; and query the Data pointer in the preprocessing table with this component as the Key, and record the found Data pointer.

[0074] C3. Replace the component in the data name with the abnormal component identifier.

[0075] In another possible implementation, the Prefix table adopts a local mode. Before determining the matching non-single-component Pattern, it further includes:

[0076] Create hash tables named Shift table, Prefix table, and Rule table: Len = i respectively, where i ∈ (2, 3,..., n).

[0077] Using the length of the shortest encoded non-single-component Pattern in the Pattern set as the window length, set the window, and align each encoded non-single-component Pattern with the left side of the window; for any encoded non-single-component Pattern, insert the mapping relationship between each component Code and Shift within the window and the mapping relationship between the suffix component Code and the Prefix pointer within the window into the Shift table, and insert the mapping relationship between the prefix component Code and its length into the Prefix table pointed to by this Prefix pointer; insert the mapping relationship between the encoded non-single-component Pattern with length i and its Data pointer into the Rule table: Len = i; where

[0078] For the mapping relationship between each component Code and Shift, when a hash conflict occurs, take the smaller Shift value as the valid value.

[0079] Specifically, as Figure 3 shown in the working principle of component encoding, the Pattern consists of several components of variable-length strings. For example, P1 consists of four components: baidu, pic, org, and path1, and the role of component encoding is to replace the components of variable-length strings with fixed-length component Codes.

[0080] Some components in the preprocessing table have both component Code and Data pointers, while some have only one of them. Generally, they can be divided into the following four categories: The first category: having both component Code and Data pointers; The second category: having only component Code; The third category: having only Data pointers; The fourth category: having neither. Components in the first category exist in both single-component patterns and non-single-component patterns. For example, the component "baidu" exists in the non-single-component patterns P1 and P3, and also exists in the single-component pattern P4. Components in the second and third categories exist only in non-single-component patterns and single-component patterns respectively. Components in the fourth category are those that fail to match. All components that are not inserted into the preprocessing table are components in the fourth category.

[0081] After being processed by the preprocessing table, all components of the patterns are converted into component Codes, and the Data pointers corresponding to the single-component patterns are saved in the preprocessing table. Thus, the subsequent hash table does not need to save these anymore, and the processing of all single-component patterns ends here. The next step is to insert the patterns into the Shift table.

[0082] After completing the component encoding process, align all patterns to the left, select an appropriate window length, and align the window to the left as well. Insert the covered part of the window into the subsequent Shift table and Prefix table. The length of the window depends on the shortest non-single-component pattern in the pattern set, that is, the length of the non-single-component pattern with the fewest components. That is, if there are single-component patterns, such as Figure 3 P4 and P5 in, then remove these single-component patterns and then calculate the window length. As in Figure 3 take the length 3 of P2 as the final window length.

[0083] As Figure 3 shown, the window covers the first 3 components of all non-single-component patterns, and all components of the covered part are inserted into the Shift table. The key is the component Code, and the value is the offset length of the component relative to the right end of the window inside the window. As in Figure 3 when P2 in is inserted into the Shift table, the Shifts of the component Codes cmp3, cmp2, and cmp1 are 2, 1, and 0 respectively. When a hash conflict occurs, that is, when the component Codes are the same but the Shifts are different, take the smaller Shift value as the final valid value. Those components with Shift 0 are the suffix component Codes of the covered part of the non-single-component pattern. These components also associate a hash table pointer in the Shift table, pointing to a hash table called the Prefix table.

[0084] The key of the Prefix table is the prefix component Code of the Pattern window part. For example, Figure 3 After P1 in Figure 3 inserts the Pattern into the Shift table, it inserts the prefix component Code cmp1 into the Prefix table associated with the cmp3 table entry. The value of the Prefix table is a bitmap used to represent the Pattern length. Here, the length is not only the part covered by the window but includes the entire non - single - component Pattern. Therefore, when P1 inserts cmp1, it sets the 4th bit from the right to 1 in the bitmap because the length of P1 is 4. At the same time, the suffix component Code and the prefix component Code of P3 and P1 are the same in the window - covered part. Therefore, they correspond to the same hash - table entries in the Shift table and the Prefix table. However, since their Pattern lengths are different, when processing the bitmap corresponding to the Prefix - table entry, P3 sets the 5th bit to 1 according to its own length.

[0085] Rule table: The key for Len = i is the entire non - single - component Pattern, and the value is a pointer to the Data of this non - single - component Pattern. Non - single - component Patterns of different lengths will be inserted into different Rule tables. For example, Figure 3 P1 and P3 in Figure 3 correspond to the same table entries in the Shift table and the Prefix table. However, due to their different lengths, they will finally be inserted into Rule table: Len = 4 and Rule table: Len = 5 respectively.

[0086] In another possible implementation, using the local pattern based on the Prefix table, in step 202, the determination of the matching non - single - component Pattern includes:

[0087] A′. Align the window and the encoded data name to the left;

[0088] B′. Search for whether there is an abnormal component identifier in the window. If there is, execute step C′; otherwise, execute step E′;

[0089] C′. After searching for the abnormal component identifier, check whether there is a component Code. If there is, execute step D′; otherwise, execute step L′;

[0090] D′. Shift the window to the left to align with the first component Code after this abnormal component identifier;

[0091] E′. Determine whether the number of component Codes in the window is less than the window length. If not less, return to step B′; otherwise, execute step L′;

[0092] F′. Take the suffix component Code of the encoded data name within the window as the Key, query the Shift table. If the query is successful and the Shift found is non-zero, execute step G′; if the Shift found is 0, execute step H′; if the query fails, execute step K′;

[0093] G′. Determine whether the number of remaining component Codes is not less than the Shift found. If so, shift the window to the right by the number of components of that Shift and return to step B′; otherwise, execute step L′;

[0094] H′. Take the prefix component Code of the encoded data name within the window as the Key, query the Prefix table pointed to by the Prefix pointer associated with this suffix component Code. If the query is successful, execute step I′; if the query fails, execute step K′;

[0095] I′. Determine whether there is any length found that is not greater than the number of component Codes from the start position of the window to the end of the encoded data name. If so, for the position not greater than the determined number of component Codes, denote it as k, and execute step J′; otherwise, execute step L′;

[0096] J′. Starting from the start position of the window, take k adjacent component Codes to the right as the Key, query the Rule table: Len = k. If the query is successful, record the Data pointer found and execute step K′; if the query fails, execute step K′;

[0097] K′. Determine whether there are still remaining component Codes. If so, shift the window to the right by 1 component and return to step B; otherwise, execute step L′;

[0098] L′. The matching ends. The non - single - component Pattern corresponding to the encoded non - single - component Pattern pointed to by the recorded Data pointer is the matching non - single - component Pattern.

[0099] Specifically, as Figure 4 shown, it demonstrates the non - single - component Pattern matching process for the data name / http / www / baidu / pic / org / path1. The window length is 3.

[0100] The window is left - aligned with the encoded data name.

[0101] In the T0 stage, there is an abnormal component identifier xxxx within the window, and the window is shifted to the right to be left - aligned with the first component Code cmp1 after this abnormal component identifier xxxx;

[0102] In the T1 stage, there is no abnormal component identifier xxxx within the window. Query the Shift table using the suffix component Code cmp3 of the window. The obtained Shift is 0. Through the associated Prefix pointer, find the corresponding Prefix table. Query the Prefix table using the prefix component Code cmp1 of the window to obtain the bitmap 00011000. There are only 4 components from the start position of the window to the end of the entire encoded data name. Therefore, the Pattern with a length of 5 identified in the bitmap can be ignored and not matched, and directly match the Pattern with a length of 4.

[0103] Using cmp1 / cmp2 / cmp3 / cmp4 combination as the Key, query the Rule table: Len = 4, and the Data pointer pointing to P1 is found. Record the rule P1, and shift the window 1 component to the right.

[0104] In the T2 stage, query the Shift table using the suffix cmp4. The match fails and there are no remaining components, so the matching ends. The matching Patterns of the final encoded data name include the non-single-component Pattern P1 and the single-component Pattern P4.

[0105] In another possible implementation, the Prefix table adopts a global mode. Before determining the matching non-single-component Pattern, it also includes:

[0106] Create hash tables named Shift table, Prefix table, and Rule table: Len = i respectively, where i ∈ (2, 3,..., n);

[0107] Taking the length of the shortest encoded non-single-component Pattern in the Pattern set as the window length, set the window, and align each encoded non-single-component Pattern with the left side of the window; for any encoded non-single-component Pattern, insert the mapping relationship between each component Code and Shift within the window into the Shift table; and use the prefix component Code and the suffix component Code within the window to form the Key, and insert the mapping relationship with its length into the Prefix table; insert the mapping relationship between the encoded non-single-component Pattern with a length of i and its Data pointer into the Rule table: Len = i; where,

[0108] For the mapping relationship between each component Code and Shift, when a hash conflict occurs, take the smaller Shift value as the valid value.

[0109] Specifically, the working principle of component encoding when the Prefix table adopts the global mode is as Figure 5As shown. When the scale of the Pattern set is large, for example, reaching the million level, the number of entries in the Shift table will be very large. Theoretically, if the suffix component Codes in the covered part of each rule window are all different, there will be one million entries with shift 0 in the Shift table, which means the number of Prefix tables is also one million. Some fixed data in the hash table is necessary, such as the redundant space allocated to maintain a certain loading factor, and the metadata of the hash table. When the number of hash tables is too large, the storage overhead brought by these fixed data cannot be ignored. As Figure 5 As shown on the left, the implementation mode in which the entry with shift 0 is associated with an independent Prefix table is called the local mode, which will bring a large storage overhead when dealing with large-scale data sets and is difficult to obtain the benefits in terms of storage locality. Figure 5 As shown on the right, it is the implementation method of the global mode, that is, all Prefix tables are merged. When inserting a Prefix table, not only the prefix component Code in the covered part of the window is inserted, but the suffix component Code is also used as part of the Key. In this way, those Patterns with the same suffix but different prefixes in the covered part of the window are distinguished to avoid a large number of conflicts. That is, when dealing with a Pattern set of the million level, the storage overhead is reduced, as well as the computing and storage requirements for network devices.

[0110] In another possible implementation, based on the global mode of the Prefix table, in step 202, the determination of the matching non-single-component Pattern includes:

[0111] A″. Align the window and the encoded data name to the left;

[0112] B″. Search for whether there is an abnormal component identifier in the window. If there is, execute step C″; otherwise, execute step E″;

[0113] C″. Search for whether there is a component Code after the abnormal component identifier. If there is, execute step D″; otherwise, execute step L″;

[0114] D″. Shift the window to the left to align with the first component Code after the abnormal component identifier;

[0115] E″. Determine whether the number of component Codes in the window is less than the window length. If it is not less, return to step B″; otherwise, execute step L″;

[0116] F″. Take the suffix component Code of the encoded data name in the window as the Key, query the Shift table. If the query is successful and the Shift obtained is non-zero, execute step G″; if the Shift obtained is 0, execute step H″; if the query fails, execute step K″;

[0117] G″. Determine whether the number of remaining component Codes is not less than the retrieved Shift. If so, shift the window to the right by that number of components and return to step B″; otherwise, execute step L″.

[0118] H″. Use the prefix component Code and the suffix component Code of the encoded data name within the window to form a Key, and query the Prefix table. If the query is successful, execute step I″; if the query fails, execute step K″.

[0119] I″. Determine whether there is any length among the retrieved lengths that is not greater than the number of component Codes from the start position of the window to the end of the encoded data name. If so, record the position not greater than the determined number of component Codes as k and execute step J″; otherwise, execute step L″.

[0120] J″. Starting from the start position of the window, take k adjacent component Codes to the right as the Key and query the Rule table: Len = k. If the query is successful, record the retrieved Data pointer and execute step K″; if the query fails, execute step K″.

[0121] K″. Determine whether there are still remaining component Codes. If so, shift the window to the right by 1 component and return to step B″; otherwise, execute step L″.

[0122] L″. The matching ends. The non - single - component Pattern corresponding to the encoded non - single - component Pattern pointed to by the recorded Data pointer is the matched non - single - component Pattern.

[0123] In another possible implementation, in step 204, add a counter list attribute to the Shift table, where the subscript of the counter list is Shift.

[0124] Add a non - single - component Pattern to the Pattern set. For any component Code of this non - single - component Pattern within the window, perform the following operations on the Shift table entry corresponding to this component Code: increment the counter with the subscript of the Shift of this component Code by 1; and when a hash collision occurs, take the smaller value of the Shift attribute.

[0125] Delete a non-single-component Pattern from the Pattern set. For any component Code of this non-single-component Pattern within the window, perform the following operations on the Shift table entry corresponding to this component Code: decrement by 1 the counter whose subscript is the Shift of this component Code. When the counter becomes 0 after the decrement, search for non-zero counters in the counter list. If found, and the subscripts of the found non-zero counters are all different from the Shift attribute value, then use the smaller value among the subscripts of the non-zero counters as the Shift attribute value; if not found, delete this Shift table entry.

[0126] Here, the embodiments of the present invention support Pattern update and have high update efficiency.

[0127] Specifically, the design principle of the Shift table is as Figure 6 shown. To support the incremental update of non-single-component Patterns, the Shift table has been further optimized. When the Shift table encounters a conflict and always takes the minimum Shift value each time, it will cause the previously inserted data to be overwritten, as shown in the part above the 6 horizontal lines. In this way, after inserting P1, P2, and P3, if P2 is subsequently deleted, P3 cannot be matched. In view of this, a counter list is attached to each Shift table entry, as Figure 6 shown in the part below the horizontal line. Each counter in the counter list corresponds to a certain fixed Shift value. When the data names are matched, the counter list does not participate in the matching process. When the Pattern is updated, for example, when deleting P2, the affected components include baidu, pic, and org, and their shifts are 2, 1, and 0 respectively. The Shift table entry corresponding to the component baidu decrements by 1 the counter whose subscript is 2 in the counter list to get 2. Similarly, the counter whose subscript is 0 in the counter list corresponding to the table entry of the component org gets 1. For the table entry corresponding to the component pic, the counter whose subscript is 1 in its counter list gets 0 after the decrement. At this time, it is necessary to find a non-zero counter with a subscript greater than 1 in the counter list and use the subscript of the non-zero counter as the Shift attribute value of this table entry. This step is called Shift fallback. Since the counter list corresponding to the table entry of the component pic is all 0, it means that this table entry is deleted. When P3 is subsequently deleted, the affected table entries are processed according to the same steps, and the Shift attribute value of the table entry corresponding to org is updated to 1 according to the Shift fallback, while the table entry corresponding to the component com will be deleted.

[0128] The above are only the preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention.

Claims

1. A multi-pattern matching method for data name matching, characterized in that, Including: For any component of the data name, query the component Code in the preprocessing table with this component as the Key. If found, replace this component with the queried component Code, and query the Data pointer with this component as the Key. The Pattern pointed to by the queried Data pointer is the matching single-component Pattern; If the component Code is not found, replace this component with the abnormal component identifier to obtain the componentized data name; Take the component Code as a character block and use the Wu-Manber algorithm to determine the matching non-single-component Pattern; Among them, the preprocessing table is created when initializing the Pattern set. When replacing each component of the non-single-component Pattern with the component Code, save the mapping relationship between each component and its component Code; and save the mapping relationship between the component of each single-component Pattern and the Data pointer of the single-component Pattern; The component Code and the abnormal component identifier have the same length, both are fixed-length strings.

2. The method according to claim 1, characterized in that, For any component of the non-single-component Pattern, the replacement with the component Code and saving the mapping relationship between the component and its component Code include: Query the component Code in the preprocessing table with this component as the Key. If the query is successful, replace this component in the non-single-component Pattern with the queried component Code; if the query fails, convert this component to the component Code, insert the mapping relationship between this component and the component Code into the preprocessing table, and replace this component in the non-single-component Pattern with this component Code.

3. The method according to claim 1, wherein It also includes maintaining an arbitrary initial coding CodeBase; correspondingly, The conversion of the component to the component Code includes: Increment CodeBase by 1 as CodeBase; Use CodeBase as the component Code of this component.

4. The method according to claim 1, wherein For any component of the non-single-component Pattern, the replacement with the component Code and saving the mapping relationship between the component and its component Code include: Query the component Code in the preprocessing table with this component as the Key. If the query is successful, replace this component in the non-single-component Pattern with the queried component Code; if the query fails, convert this component to the component Code, insert the mapping relationship between this component and the component Code into the preprocessing table, and replace this component in the non-single-component Pattern with this component Code.

5. The method according to claim 1, wherein Before determining the matching non-single-component Pattern, it also includes: Create hash tables named Shift table, Prefix table, and Rule table respectively: hash tables with Len = i, where i ∈ (2, 3,..., n); Set the window length to the length of the shortest encoded non - single - component Pattern in the Pattern set. Set the window and align each encoded non - single - component Pattern to the left of the window; for any encoded non - single - component Pattern, insert the mapping relationship between each component Code and Shift within the window, the mapping relationship between the suffix component Code and the Prefix pointer within the window into the Shift table, and insert the mapping relationship between the prefix component Code and its length into the Prefix table pointed to by the Prefix pointer; insert the mapping relationship between the encoded non - single - component Pattern with length i and its Data pointer into the Rule table: Len = i; Among them, for the mapping relationship between each component Code and Shift, when a hash conflict occurs, take the smaller Shift value as the valid value; The encoded non - single - component Pattern means a non - single - component Pattern in which all components are replaced by component Codes.

6. The method according to claim 5, wherein The determined matching non - single - component Pattern includes: Align the window and the encoded data name to the left; If there is an abnormal component identifier within the window, shift the window to the left alignment with the first component Code after the abnormal component identifier and continue the matching; If there is no abnormal component identifier within the window, take the suffix component Code of the encoded data name within the window as the Key, query Shift in the Shift table. If the queried Shift is non - zero, shift the window to the right by that Shift number of components and continue the matching; when the queried Shift is 0, take the prefix component Code of the encoded data name within the window as the Key, query the length in the Prefix table pointed to by the Prefix pointer associated with the suffix component Code. If the query is successful, take the length not greater than the number of component Codes from the start position of the window to the end of the encoded data name as k, take k adjacent component Codes to the right from the start position of the window as the Key, query the Data pointer in the Rule table: Len = k. If the query is successful, the Pattern pointed to by the queried Data pointer is the matching non - single - component Pattern, and shift the window to the right by 1 component and continue the matching; if the Shift query fails, the length query fails, there is no length not greater than the number of component Codes from the start position of the window to the end of the encoded data name, or the Data pointer query fails, shift the window to the right by 1 component and continue the matching.

7. The method according to claim 1, characterized in that Before determining the matching non - single - component Pattern, it also includes: Create hash tables named Shift table, Prefix table, and Rule table: Len = i respectively, where i ∈ (2, 3,..., n); Set the window length as the length of the shortest encoded non - single - component Pattern in the Pattern set. Set the window and align each encoded non - single - component Pattern to the left of the window; for any encoded non - single - component Pattern, insert the mapping relationship between each component Code and Shift within the window into the Shift table; and form a Key with its prefix component Code and the suffix component Code within the window, and insert the mapping relationship with its length into the Prefix table; insert the mapping relationship between the encoded non - single - component Pattern with length i and its Data pointer into the Rule table: Len = i; where, For the mapping relationship between each component Code and Shift, when a hash conflict occurs, take the smaller value of Shift as the valid value.

8. The method according to claim 7, wherein The determined matching non - single - component Pattern includes: Align the window and the encoded data name to the left. If there is an abnormal component identifier within the window, shift the window to the left alignment with the first component Code after this abnormal component identifier and continue the matching; If there is no abnormal component identifier within the window, take the suffix component Code of the encoded data name within the window as the Key, query Shift in the Shift table. If the queried Shift is non - zero, shift the window by that Shift number of components and continue the matching; when the queried Shift is 0, form a Key with the prefix component Code and the suffix component Code of the encoded data name within the window, query the length in the Prefix table. If the query is successful, take the length obtained from the query that is not greater than the number of component Codes from the start position of the window to the end of the encoded data name, denoted as k. Take k adjacent component Codes to the right from the start position of the window as the Key, query the Data pointer in the Rule table: Len = k. If the query is successful, the Pattern pointed to by the queried Data pointer is the matching non - single - component Pattern. Shift the window by 1 component and continue the matching; if the Shift query fails, the length query fails, there is no length obtained from the query that is not greater than the number of component Codes from the start position of the window to the end of the encoded data name, or the Data pointer query fails, shift the window by 1 component and continue the matching.

9. The method according to any one of claims 5 to 8, characterized in that Add a counter list attribute to the Shift table, and the subscript of the counter list is Shift; Add a non - single - component Pattern to the Pattern set. For any component Code of this non - single - component Pattern within the window, perform the following operations on the Shift table entry corresponding to this component Code: increment the counter with the subscript of the Shift of this component Code by 1; and when a hash conflict occurs, take the smaller value of Shift as the Shift attribute value; Delete a non-single-component Pattern from the Pattern set. For any component Code of this non-single-component Pattern within the window, perform the following operations on the Shift table entry corresponding to this component Code: decrement the counter with the subscript of this component Code Shift by 1. When the counter becomes 0 after the decrement, search for non-zero counters in the counter list. If found, and the subscripts of the found non-zero counters are all different from the Shift attribute value, then use the smaller value among the subscripts of the non-zero counters as the shift attribute value; if not found, delete this Shift table entry.

Images