A pattern string matching method, device and equipment and computer readable storage medium

Abstract

Embodiments of the present application provide a pattern string matching method, device and equipment and a computer readable storage medium, relating to the technical field of pattern string matching, the method comprising: determining a character to be matched from a current state in a text to be matched as a character to be matched; calculating a hash value of the current state and the character to be matched as an address to be used; obtaining a first position index recorded at the address to be used in a hash mapping table; the hash mapping table is constructed based on a state transition table; obtaining a transition entry recorded at the first position index in the state transition table to obtain a transition entry of a sub-state matched with the current state and the character to be matched as a first transition entry; updating the sub-state matched with the current state and the character to be matched as a new current state, returning to execute the step of determining the character to be matched until there is no character to be matched in the text to be matched, and obtaining a matching result based on the obtained first transition entry. In this way, the storage space is reduced.

Description

Technical Field

[0001] This invention relates to the field of pattern string matching technology, and in particular to a pattern string matching method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] Pattern string matching is a core technology in many data processing scenarios, its core requirement being the rapid identification of multiple predefined pattern strings within a high-speed input data stream. In fields such as network security, log auditing, and data filtering, the performance of this technology directly determines the system's processing throughput and real-time performance.

[0003] In existing technologies, pattern string matching is performed using a pre-built hash map table based on the Aho-Corasick (AC) automaton. The hash map table stores the state transition mappings and the full data of the state (including the matching string of the state, failure chain, parent state information, etc.), resulting in high data redundancy and high collision probability. Consequently, a large amount of storage space needs to be allocated to the hash map table, leading to a large storage space consumption. Summary of the Invention

[0004] The purpose of this invention is to provide a pattern string matching method, apparatus, device, and computer-readable storage medium to reduce storage space usage. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of this application provide a pattern string matching method, the method comprising:

[0006] Identify the characters in the text to be matched that match the current state, and use them as the characters to be matched;

[0007] Calculate the hash value of the current state and the character to be matched, and use it as the address to be used;

[0008] Obtain the location index of the address to be used recorded in the hash map table, and use it as the first location index; wherein, the hash map table is constructed based on the state transition table;

[0009] Obtain the transition entry recorded at the first position index in the state transition table, and obtain the transition entry of the sub-state that matches the current state and the character to be matched, as the first transition entry; wherein, the transition entry of a state includes: the matching string of the state and the failure chain;

[0010] The substate that matches the current state and the character to be matched is updated to the new current state. The process of determining the character to be matched from the text to be matched that matches the current state is then repeated until there are no unmatched characters in the text to be matched. Finally, the matching result is obtained based on the first transition entry.

[0011] Optionally, the method further includes:

[0012] If the address to be used in the hash map table is empty, obtain the second position index; wherein, the second position index represents the position of the current state transition entry in the state transition table;

[0013] Obtain the transition entry recorded at the second position index in the state transition table to obtain the second transition entry;

[0014] Update the current state based on the failure chain in the second transition entry, and return to the step of determining the character to be matched from the text to be matched that matches the current state, until there are no unmatched characters in the text to be matched.

[0015] Optionally, before calculating the hash value of the current state and the character to be matched as the address to be utilized, the method further includes:

[0016] Detect whether the current state and the character to be matched match a pre-built Bloom filter; wherein the Bloom filter is constructed based on the parent state and the characters between the parent state and the child state;

[0017] The calculation of the current state and the hash value of the character to be matched, as the address to be utilized, includes:

[0018] If the current state and the character to be matched match the Bloom filter, calculate the hash value of the current state and the character to be matched, and use it as the address to be utilized;

[0019] The method further includes:

[0020] In the case where the current state and the character to be matched do not match the Bloom filter, a second position index is obtained; wherein the second position index represents the position of the transition entry of the current state in the state transition table;

[0021] Obtain the transition entry recorded at the second position index in the state transition table to obtain the second transition entry;

[0022] Update the current state based on the failure chain in the second transition entry, and return to the step of determining the character to be matched from the text to be matched that matches the current state, until there are no unmatched characters in the text to be matched.

[0023] Optionally, the state identifier for each state is the same as the position index of the transition entry for that state.

[0024] Optionally, in the state transition table, the transition entries for each state are stored consecutively and compactly.

[0025] Optionally, the hash map table also records the parent state and character corresponding to the first position index at the address to be used;

[0026] The step of obtaining the location index of the address to be used in the hash map table as the first location index includes:

[0027] If the parent state and character recorded at the address to be used in the hash map table are consistent with the current state and the character to be matched, the position index recorded at the address to be used in the hash map table is obtained as the first position index.

[0028] Optionally, at least one mapping entry is recorded at the address to be used in the hash mapping table using a linked list structure; a mapping entry includes: a parent state, the position index of the transition entry of the child state of the parent state, and the characters between the parent state and the child state;

[0029] When the parent state and character recorded at the address to be used in the hash map table are consistent with the current state and the character to be matched, the position index recorded at the address to be used in the hash map table is obtained as the first position index, including:

[0030] Obtain the linked list recorded at the address to be used in the hash map table;

[0031] Retrieve from the obtained linked list a mapping entry containing the current state and the character to be matched;

[0032] Retrieve the position index from the queried mapping entry and use it as the first position index.

[0033] Optionally, the state transition table and hash map table are constructed through the following steps:

[0034] Get the pattern string;

[0035] The pattern string is divided into segments according to a first specified number to obtain the transition entries for each state, the parent-child relationships between states, and the characters between the parent and child states.

[0036] Record the transition entries for each state to obtain the state transition table;

[0037] For each sub-state, determine its parent state and the characters between the parent state and the sub-state;

[0038] Calculate the hash value of the determined parent state and character;

[0039] The position index of the transition entry of the sub-state in the state transition table is recorded at the address represented by the calculated hash value to obtain the hash mapping table.

[0040] Optionally, the method is applied to an electronic device, the electronic device comprising a second specified number of virtual machines; each virtual machine is used to execute the method described in any of the above embodiments; the second specified number of virtual machines are in a parallel working mode;

[0041] The characters to be matched processed by each virtual machine for the first time are obtained by sliding the text to be matched through a specified window with a step size of 1; the size of the specified window is a first specified number.

[0042] Each virtual machine is used to slide the text to be matched in the specified window with the second specified number as the step size after the current character to be matched is completed, so as to obtain a new character to be matched.

[0043] Secondly, embodiments of this application provide a pattern string matching device, the device comprising:

[0044] The character to be matched module is used to determine the character in the text to be matched that matches the current state, and use it as the character to be matched.

[0045] The address to be used calculation module is used to calculate the hash value of the current state and the character to be matched, as the address to be used;

[0046] The first position index acquisition module is used to acquire the position index recorded at the address to be used in the hash map table as the first position index; wherein, the hash map table is constructed based on the state transition table;

[0047] The first transition entry acquisition module is used to acquire the transition entry recorded at the first position index in the state transition table, and obtain the transition entry of the sub-state that matches the current state and the character to be matched, as the first transition entry; wherein, the transition entry of a state includes: the matching string of the state and the failure chain;

[0048] The matching result acquisition module is used to update the sub-state that matches the current state and the character to be matched to the new current state, return to the step of determining the character to be matched from the text to be matched that matches the current state as the character to be matched, until there are no unmatched characters in the text to be matched, and obtain the matching result based on the acquired first transition entry.

[0049] Optionally, the device further includes:

[0050] The second position index acquisition module is used to acquire the second position index if the address to be used in the hash mapping table is empty; wherein, the second position index represents the position of the current state transition entry in the state transition table;

[0051] The second transfer entry acquisition module is used to acquire the transfer entry recorded at the second position index in the state transfer table to obtain the second transfer entry;

[0052] The first update module is used to update the current state based on the failure chain in the second transfer entry, triggering the character to be matched determination module until there are no unmatched characters in the text to be matched.

[0053] Optionally, the device further includes:

[0054] The detection module is used to detect whether the current state and the character to be matched match a pre-built Bloom filter before calculating the hash value of the current state and the character to be matched as the address to be used; wherein the Bloom filter is constructed based on the parent state and the characters between the parent state and the child state;

[0055] The address to be used calculation module is specifically used to calculate the hash value of the current state and the character to be matched as the address to be used, given the current state and the case where the character to be matched matches the Bloom filter.

[0056] The device further includes:

[0057] The processing module is configured to obtain a second position index when the current state and the character to be matched do not match the Bloom filter; wherein the second position index represents the position of the transition entry of the current state in the state transition table;

[0058] Obtain the transition entry recorded at the second position index in the state transition table to obtain the second transition entry;

[0059] The current state is updated based on the failure chain in the second transfer entry, triggering the character to be matched determination module until there are no unmatched characters in the text to be matched.

[0060] Optionally, the state identifier for each state is the same as the position index of the transition entry for that state.

[0061] Optionally, in the state transition table, the transition entries for each state are stored consecutively and compactly.

[0062] Optionally, the hash map table also records the parent state and character corresponding to the first position index at the address to be used;

[0063] The first position index acquisition module is specifically used to acquire the position index recorded at the address to be used in the hash map table as the first position index when the parent state and character recorded at the address to be used are consistent with the current state and the character to be matched.

[0064] Optionally, at least one mapping entry is recorded at the address to be used in the hash mapping table using a linked list structure; a mapping entry includes: a parent state, the position index of the transition entry of the child state of the parent state, and the characters between the parent state and the child state;

[0065] The first position index acquisition module is specifically used to acquire the linked list recorded at the address to be used in the hash mapping table;

[0066] Retrieve from the obtained linked list a mapping entry containing the current state and the character to be matched;

[0067] Retrieve the position index from the queried mapping entry and use it as the first position index.

[0068] Optionally, the device further includes:

[0069] The table entry construction module is used to obtain the pattern string;

[0070] The pattern string is divided into segments according to a first specified number to obtain the transition entries for each state, the parent-child relationships between states, and the characters between the parent and child states.

[0071] Record the transition entries for each state to obtain the state transition table;

[0072] For each sub-state, determine its parent state and the characters between the parent state and the sub-state;

[0073] Calculate the hash value of the determined parent state and character;

[0074] The position index of the transition entry of the sub-state in the state transition table is recorded at the address represented by the calculated hash value to obtain the hash mapping table.

[0075] Optionally, the device is applied to an electronic device, which includes a second specified number of virtual machines; each virtual machine includes the above-mentioned modules; the second specified number of virtual machines operate in parallel.

[0076] The characters to be matched processed by each virtual machine for the first time are obtained by sliding the text to be matched through a specified window with a step size of 1; the size of the specified window is a first specified number.

[0077] Each virtual machine is used to slide the text to be matched in the specified window with the second specified number as the step size after the current character to be matched is completed, so as to obtain a new character to be matched.

[0078] Thirdly, embodiments of this application provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0079] Memory, used to store computer programs;

[0080] When a processor executes a program stored in memory, it implements any of the steps described above.

[0081] Fourthly, embodiments of this application provide an electronic device, which includes a second specified number of logic chips, each logic chip being used to execute any of the methods described above; the second specified number of logic chips are in parallel operation mode; the character to be matched processed by each logic chip for the first time is obtained by sliding the text to be matched through a specified window with a step size of 1; the size of the specified window is a first specified number;

[0082] Each logic chip is used to slide the text to be matched according to the specified window with the second specified number as the step size after the current character to be matched is completed, so as to obtain a new character to be matched.

[0083] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the steps described above.

[0084] Sixthly, embodiments of this application provide a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the methods described above.

[0085] Beneficial effects of the embodiments of the present invention:

[0086] The pattern string matching method provided in this invention determines the character to be matched with the current state from the text to be matched, and uses it as the character to be matched; calculates the hash value of the current state and the character to be matched, and uses it as the address to be used; obtains the position index recorded at the address to be used in the hash map table, and uses it as the first position index; wherein, the hash map table is constructed based on the state transition table; obtains the transition entry recorded at the first position index in the state transition table, and obtains the transition entry of the sub-state that matches the current state and the character to be matched, and uses it as the first transition entry; wherein, the transition entry of a state includes: the matching string of the state and the failure chain; updates the sub-state that matches the current state and the character to be matched to the new current state, and returns to execute the step of determining the character to be matched with the current state from the text to be matched, and uses it as the character to be matched, until there are no unmatched characters in the text to be matched, and obtains the matching result based on the obtained first transition entry.

[0087] Based on the pattern string matching method provided in this application, state transition-related data (including matching strings and failure chains of states) are separated into separate records from the hash map table, reducing the data volume of the hash map table and thus reducing the probability of collisions. Compared with the prior art, it eliminates the need to allocate a large amount of storage space to the hash map table, reducing storage space usage.

[0088] Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0089] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0090] Figure 1 This is a schematic diagram of a hash mapping table in the prior art;

[0091] Figure 2 A schematic diagram of the hash mapping table and state transition table provided in the embodiments of this application;

[0092] Figure 3 A flowchart illustrating a pattern string matching method provided in an embodiment of this application;

[0093] Figure 4 A schematic diagram of a state transition diagram in a single-character mode provided in an embodiment of this application;

[0094] Figure 5 A schematic diagram of a state transition diagram in a multi-character mode provided in an embodiment of this application;

[0095] Figure 6 A schematic diagram illustrating the principle of pattern string matching using multiple virtual machines, provided for an embodiment of this application;

[0096] Figure 7 A schematic diagram illustrating the principle of pattern string matching using multiple physical machines, provided for an embodiment of this application;

[0097] Figure 8 This is a schematic diagram of the structure of a pattern string matching device provided in an embodiment of this application;

[0098] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0099] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of the present invention.

[0100] When using the AC automaton for pattern string matching, existing technologies use hash tables to store state transition mappings and the full state data, resulting in high data redundancy and a high probability of collisions. To reduce the probability of collisions, a large amount of storage space needs to be allocated to the hash table, leading to a large storage footprint.

[0101] When performing pattern string matching, the pattern string can be segmented to obtain the transition entries for each state, the parent-child relationships between states, and the characters between parent and child states. This information can be represented in the form of a state transition diagram.

[0102] See Figure 1 , Figure 1 In the hash table, "0", "2", "100", and "1023" represent hash values ​​obtained by hashing the states and corresponding characters in the state transition diagram. `q` represents the current state, `str` represents the character to be matched, and `q0` represents the root state of the state transition diagram. "elep", "tech", "phon", and "nica" represent characters between the root state and its child states in the state transition diagram. The hash value can be used to index the corresponding address in the hash map table, obtaining the state transition mapping recorded at that address, as well as the full data of the state (i.e., ...).<q0, elep> …), to perform matching.

[0103] To address the aforementioned technical problems, embodiments of this application provide a pattern string matching method, apparatus, device, and computer-readable storage medium. The pattern string matching method provided in this application embodiment will be described first below.

[0104] In one embodiment, the pattern string matching method provided in this application includes: determining a character from the text to be matched that matches the current state, as the character to be matched; calculating the hash value of the current state and the character to be matched, as the address to be used; obtaining the position index recorded at the address to be used in the hash map table, as the first position index; wherein, the hash map table is constructed based on a state transition table; obtaining the transition entry recorded at the first position index in the state transition table to obtain the transition entry of the sub-state that matches the current state and the character to be matched, as the first transition entry; wherein, a transition entry of a state includes: the matching string and the failure chain of the state; updating the sub-state that matches the current state and the character to be matched to the new current state, returning to execute the step of determining the character from the text to be matched that matches the current state, as the character to be matched, until there are no unmatched characters in the text to be matched, and obtaining the matching result based on the obtained first transition entry. Based on the above processing, since the data related to state transitions (including the matching string and failure chain of the state) are separated from the hash map table into separate records, the data volume of the hash map table is reduced, thereby reducing the probability of collision. Compared to existing technologies, it eliminates the need to allocate large storage spaces for hash maps, thus reducing storage space usage.

[0105] In addition, existing technologies can use Trie trees (prefix trees) as a basis to implement AC automata for multi-character pattern matching. However, the nodes of a standard Trie tree usually contain a fixed-size array (such as 26 pointers to English characters or more). For applications with large character sets (such as Unicode), this will generate a large number of null pointers, resulting in serious memory waste.

[0106] See Figure 2 , Figure 2 In the diagram, "0", "2", "100", and "1023" represent hash values ​​obtained by hashing the states and corresponding characters in the state transition diagram. `q` represents the current state, and `str` represents the character to be matched. `q0` represents the root state of the state transition diagram. "elep", "tech", "phon", and "nica" represent characters between the root state and its child states in the state transition diagram. The hash value can be used to index the corresponding address in the hash map table, obtaining the position index recorded at that address (e.g., ...). Figure 2 (The positions are "0", "1", "2", and "3"). The position index can be used to access the transition entries recorded in the state transition table, and a match can be made based on these transition entries.

[0107] based on Figure 2 It can also be seen that the method based on the embodiments of this application can eliminate pointers, decouple the hash map table from the state transition table, reduce the size of state transition entries, and achieve dense storage of the state transition table, thereby reducing storage space occupation. Correspondingly, when the pattern string needs to be adjusted, it can be modified in the state transition table without complex adjustments to the hash map table, resulting in low expansion costs, simplified dynamic storage management, and improved efficiency of addition and deletion rules.

[0108] In one embodiment, the pattern string matching method provided in this application can be applied to an electronic device. The electronic device can obtain the text to be matched and perform pattern string matching on the text based on the method provided in this application, that is, determine whether the text to be matched contains a pattern string, and specifically, if so, whether the text contains a pattern string. The hash mapping table and state transition table in this application are stored in the memory of the electronic device.

[0109] See Figure 3 The pattern string matching method provided in this application includes:

[0110] S301: Determine the character in the text to be matched that matches the current state, and use it as the character to be matched.

[0111] In this embodiment of the application, the text to be matched is the text for which it is currently necessary to determine whether it contains a specified string (i.e., a pattern string).

[0112] When performing pattern string matching based on the Aho-Corasick automaton, matching can be performed based on a single-character pattern, in which one character is matched at a time; or, matching can be performed based on a multi-character pattern (also known as the K-character super-character pattern), in which multiple characters are matched at a time.

[0113] For example, the information obtained by segmenting the pattern strings "his", "he", "she", and "hers" according to single-character patterns is represented as follows: Figure 4 The state transition diagram is shown. Figure 4 In this diagram, each circle represents a state, and a solid arrow between two states indicates a transition relationship between them. That is, a parent state points to a child state via a solid arrow, and the character on the solid arrow represents the character between the parent and child states. In other words, for a given parent state, the character on the solid arrow can be matched to the corresponding child state. Dashed arrows represent failure chains.

[0114] The information obtained by segmenting the pattern strings "phone" and "telephone" according to the multi-character pattern (i.e., K=4) is represented as follows: Figure 5 The state transition diagram is shown. Figure 5In this diagram, each circle represents a state (e.g., state q0, state q1, state q2, state q3, state q4, and state q5). A solid arrow between two states indicates a transition relationship between them. A dashed arrow indicates a failure chain. A dotted dashed line indicates that the current state is the tail state; that is, the matching string for state q2 is "phone", and the matching string for state q5 is "telephone".

[0115] Understandably, since the text to be matched contains multiple characters, multiple matches may be required. Based on the order of characters in the text, if matching is based on a single-character pattern, one character is matched each time; that is, the character identified as the matchable character in the current state is one character. If matching is based on a multi-character pattern, multiple characters are matched each time; that is, the character identified as the matchable character in the current state contains multiple characters, depending on the matching pattern used. The current state can also be updated as the matching process progresses. During the first match, the current state is the root state of the state transition graph, for example, Figure 5 The state q0 in the middle.

[0116] In other words, each time a new character to be matched is identified, the electronic device can perform the matching according to the method provided in the embodiments of this application.

[0117] S302: Calculate the hash value of the current state and the character to be matched, and use it as the address to be used.

[0118] S303: Obtain the position index of the record at the address to be used in the hash map table, and use it as the first position index.

[0119] S304: Obtain the transition entry recorded at the first position index in the state transition table, and obtain the transition entry of the sub-state that matches the current state and the character to be matched, and use it as the first transition entry.

[0120] A state transition entry contains: the matching string for that state and the failure chain.

[0121] S305: Update the substate that matches the current state and the character to be matched to the new current state, return to step S301, until there are no unmatched characters in the text to be matched, and obtain the matching result based on the first transition entry obtained.

[0122] The hash map table is built upon the state transition table. The position index indicates the location of the transition entry in the state transition table.

[0123] In one embodiment, the state transition table and hash map are constructed through the following steps: obtaining a pattern string; dividing the pattern string into segments according to a first specified number to obtain transition entries for each state, parent-child relationships between states, and characters between parent and child states; recording the transition entries for each state to obtain a state transition table; for each child state, determining its parent state and characters between the parent and child states; calculating the hash value of the determined parent state and characters; indexing the position of the child state's transition entry in the state transition table and recording it at the address represented by the calculated hash value to obtain the hash map.

[0124] In this embodiment, if the first specified number is 1, it is the single-character mode described above; if the first specified number is greater than 1, it is the multi-character mode described above. The specific value of the first specified number can be set according to business requirements. By adjusting the first specified number to adapt to different scenarios, for example, setting a smaller value (such as 4) is suitable for low-latency scenarios, and setting a larger value (such as 16) is suitable for high-throughput scenarios, the method has wide adaptability.

[0125] After segmenting the pattern string, a state transition table is constructed based on the segmentation results (i.e., the transition entries for each state, the parent-child relationships between states, and the characters between parent and child states). Specifically, for each state determined by the segmentation, the transition entry for that state (containing the matching string and the failure chain) is stored. If a state is the tail state, the matching string is the pattern string; if a state is not the tail state, the matching string is NULL (empty). During storage, the position of the transition entry in the state transition table can be determined based on the state's identifier. For example, the transition entries for each state are stored sequentially without gaps in the state transition table; correspondingly, the identifier of a state indicates the sorting position of its transition entry in the state transition table. Subsequently, the corresponding transition entry can be retrieved from the state transition table based on the state's identifier.

[0126] For example, targeting Figure 5 The segmentation results represented by the state transition diagram shown can be used to construct the state transition table as shown in Table 1.

[0127] Table 1

[0128] Current status Match string Failure chain q0 NULL q0 q1 NULL q0 q2 phone q0 q3 NULL q0 q4 NULL q1 q5 telephone q2

[0129] In Table 1, each row represents a state transition entry. For example, for the row where the current state is q0, the matching string is NULL (empty), and the failure chain is state q0.

[0130] It is understandable that the actual stored transfer entries do not include the first column of Table 1. In other words, when storing the state transfer table, the electronic device only stores the last two columns of Table 1. The first column of Table 1 is used as the location index when storing the last two columns.

[0131] For example, see Figure 2 The transition entries for state q0 are stored according to the position index (e.g., 0) represented by the identifier of state q0. That is, the transition entries for state q0 are stored at position 0 in the state transition table. Similarly, the transition entries for state q1 are stored according to the position index (e.g., 1) represented by the identifier of state q1. That is, the transition entries for state q1 are stored at position 1 in the state transition table. The state transition table stores each transition entry according to a specified size. Specifically, the state transition table is divided into sections of a specified size, and each section records the transition entry for one state, facilitating subsequent queries.

[0132] Correspondingly, a hash map can be constructed based on the state transition table. For example, for each state (i.e., a sub-state) determined by the segmentation, the identifier of the parent state of that state and the characters between that state and its parent state are obtained. Then, the obtained identifier and characters are concatenated, and the hash value of the concatenation result is calculated. Then, the position index representing the transition entry of that state in the state transition table is recorded at the position represented by the hash value in the hash map table.

[0133] Correspondingly, when calculating the hash value of the current state and the character to be matched, the identifier of the current state can be obtained, the obtained identifier and the character to be matched can be concatenated, and the hash value of the concatenation result can be calculated. This hash value is used as the address (i.e. the address to be used) to query the hash mapping table.

[0134] If the address to be used in the hash table contains a position index (i.e., the first position index), it indicates that based on the segmentation result, the current state can be matched with the character to be matched. The first position index in the state transition table records the transition entry for that sub-state. Therefore, the transition entry recorded at the first position index in the state transition table (i.e., the first transition entry) can be obtained, and the matching result can be obtained based on the first transition entry.

[0135] For example, if the first transition entry contains a non-empty matching string, then that matching string is also the pattern string, confirming that the pattern string exists in the text to be matched.

[0136] If a first transition entry exists, it indicates that based on the segmentation result, a sub-state can be matched for the current state according to the character to be matched. Correspondingly, if there are unmatched characters in the text to be matched, i.e., the character to be matched is not the character at the end of the text, then a sub-state matching the current state and the character to be matched can be determined. Since the first transition entry is also the transition entry of this sub-state, and as described in the previous embodiment, the identifier of each state is the same as the position index of the transition entry of that state, therefore, the identifier of the sub-state can also be obtained based on the position index of the first transition entry, i.e., determining which specific state the sub-state is. Then, this sub-state is taken as the new current state, and a new character to be matched is determined, and step S301 is executed. This continues until there are no unmatched characters in the text to be matched.

[0137] When the matching termination condition is met (i.e., there are no unmatched characters in the text to be matched), the matching result of the text to be matched can be obtained based on the matching strings in all the first transition entries obtained during the matching process. For example, if the matching strings in all the first transition entries are NULL, it is determined that the text to be matched does not contain the pattern string; if there is a transition entry (which can be called a matching entry) in all the first transition entries, it is determined that the text to be matched contains the pattern string, and it can be determined that the contained pattern string is the matching string in the matching entry.

[0138] It is understandable that when the matching termination condition is met, if the first transition entry is not obtained, it can be determined that the text to be matched does not contain the pattern string.

[0139] Each time step S301 is returned, new characters to be matched are determined. For example, according to the order of characters in the text to be matched, the first specified number of unmatched characters are determined from the text to be matched and used as new characters to be matched.

[0140] In one embodiment, the pattern string matching method further includes: if the address to be used in the hash map table is empty, obtaining a second position index; wherein the second position index represents the position of the current state's transition entry in the state transition table; obtaining the transition entry recorded at the second position index in the state transition table to obtain a second transition entry; updating the current state based on the failure chain in the second transition entry, and executing S301, until there are no unmatched characters in the text to be matched. In this way, each character in the text to be matched can be matched sequentially, avoiding missed detections.

[0141] In this embodiment, if the address to be used in the hash map table is empty, it indicates that based on the segmentation result, the current state cannot be matched with the character to be matched. In this case, subsequent matching can be performed based on the failure chain of the current state. For example, the state indicated by the failure chain of the current state is taken as the new current state, and the character to be matched with the new current state is determined from the unmatched characters (i.e., the characters after the current character to be matched), resulting in a new character to be matched. This character is then processed according to the method provided in this embodiment until there are no unmatched characters in the text to be matched, that is, until all characters in the text to be matched are matched. Then, the matching result can be obtained based on the first transition entry obtained during the matching process.

[0142] In one embodiment, before step S302, the pattern string matching method further includes: detecting whether the current state and the character to be matched match a pre-constructed Bloom filter; wherein, the Bloom filter is constructed based on the parent state and the characters between the parent state and the child state; step S302 includes: if the current state and the character to be matched match the Bloom filter, calculating the hash value of the current state and the character to be matched as the address to be used; correspondingly, the method further includes: if the current state and the character to be matched do not match the Bloom filter, obtaining a second position index; wherein, the second position index represents the position of the transition entry of the current state in the state transition table; obtaining the transition entry recorded at the second position index in the state transition table to obtain the second transition entry; updating the current state based on the failure chain in the second transition entry, and executing step S301 until there are no unmatched characters in the text to be matched. Thus, by pre-filtering the current state and the character to be matched based on the Bloom filter, the number of queries to the hash mapping table is reduced, further improving the matching efficiency.

[0143] In this embodiment, a Bloom filter can also be pre-constructed based on the segmentation results. A Bloom filter can be understood as a binary array plus multiple hash functions. Before using the Bloom filter for filtering, the state and character combination corresponding to the segmentation results are first determined. Each combination contains a parent state and the characters between the parent state and its child states. For example, for... Figure 5 The state transition diagram shown represents the segmentation results. The constructed combinations (called reference combinations) include: (q0,tele), (q0,phon), (q1,e), (q3,phon), and (q4,e).

[0144] Then, for each reference combination, multiple hash functions are used to process it, resulting in multiple different array indices. The corresponding index in the binary array is then set to 1. Similarly, after obtaining the current state and the character to be matched, multiple hash functions are used to process the combination of the current state and the character to be matched, resulting in multiple different array indices (called the detection indices). If there exists a reference combination whose index completely matches the detection index, then it can be determined that the current state and the character to be matched match the Bloom filter.

[0145] Furthermore, since Bloom filters can produce false positives, a match with a Bloom filter indicates that, based on the segmentation results, the current state might match a sub-state using the character to be matched. Subsequently, the hash values ​​of the current state and the character to be matched are calculated to further determine whether, based on the segmentation results, the current state can match a sub-state using the character to be matched.

[0146] Conversely, if the indices corresponding to all reference combinations are inconsistent with the index to be detected, it can be determined that the current state and the character to be matched do not match the Bloom filter. This situation indicates that, based on the segmentation results, the current state cannot be matched against the character to be matched. Accordingly, the subsequent matching process is carried out based on the failure chain of the current state.

[0147] Since using a Bloom filter for filtering is fast, and in this embodiment, the hash map is not queried when the Bloom filter does not match, the number of times the hash map is queried can be further reduced, the memory access time can be shortened, the matching efficiency can be improved, and the matching throughput can be increased.

[0148] In one embodiment, the state identifier of each state is the same as the position index of the transition entry for that state. This eliminates the need to separately record the correspondence between each state and its own position index, further saving storage space.

[0149] For example, combining Figure 2 and Figure 5 , Figure 2 Record at position 0 in the state transition table Figure 5 The transition entries for state q0, Figure 2 Record at position 1 in the state transition table Figure 5 The transition entries for state q1, Figure 2 Record at position 2 in the state transition table Figure 5 The transition entries for state q2, Figure 2 Record at position 3 in the state transition table Figure 5 The transition entries for state q3. Figure 2 Record at position 4 in the state transition table Figure 5 The transition entry for state q4. Figure 2 Record at position 5 in the state transition table Figure 5 The transition entry for state q5. Correspondingly, if the current state is q0, that is, the state identifier of the current state is 0, if the electronic device needs to query the transition entry for the current state, it can directly determine the transition entry at position 0 in the state transition table, which is the transition entry for state q0.

[0150] In one embodiment, the transition entries for each state in the state transition table are stored contiguously and compactly. This reduces the storage space occupied by the state transition table, further reducing storage space usage. In addition, this storage method does not require dynamic pointer management, making its state access time complexity O(1), which is compatible with software CPU cache and facilitates high-speed parallel reading by Field-Programmable Gate Arrays (FPGAs).

[0151] The transition entries for each state are stored contiguously and compactly; that is, the storage addresses of the transition entries for each state are contiguous. For example... Figure 2 As shown, although the location indices stored in the hash map table may be discrete, the transition entries stored in the state transition table are continuous and compact.

[0152] In one embodiment, the hash map table also records the parent state and character corresponding to the first position index at the address to be used; step S303 includes: if the parent state and character recorded at the address to be used in the hash map table are consistent with the current state and the character to be matched, then obtain the position index recorded at the address to be used in the hash map table as the first position index. This allows for further verification of the current state and the character to be matched, improving the accuracy of the matching result.

[0153] In this embodiment of the application, in the hash map table, for each recorded position index, the corresponding parent state and character are also recorded at the address of that position index. It can be understood that in the hash map table, an address records the position index of a state transition entry, along with the identifier of the parent state of that state, and the character between that state and its parent state.

[0154] Understandably, even if the segmentation result fails to match a sub-state for the current state based on the character to be matched, a hash collision may still indicate that the calculated address to be utilized records a position index. Therefore, to improve matching accuracy, the current state and the character to be matched can be further verified. Specifically, verify whether the current state matches the parent state recorded at the address to be utilized, and verify whether the character to be matched matches the character recorded at the address to be utilized. Only if both verification results indicate consistency can a sub-state be matched for the current state based on the segmentation result and the character to be matched, allowing the acquisition of the first position index recorded at the address to be utilized for subsequent processing. Otherwise, it indicates that a sub-state cannot be matched for the current state based on the segmentation result and the character to be matched, in which case the aforementioned second transition entry is acquired for subsequent processing.

[0155] In one embodiment, at least one mapping entry is recorded in the hash map table at the address to be used using a linked list structure. Each mapping entry includes: a parent state, the position index of the transition entry of the child state of the parent state, and the characters between the parent state and the child state. Step S303 includes: obtaining the linked list recorded at the address to be used in the hash map table; querying the obtained linked list for a mapping entry containing the current state and the character to be matched; and obtaining the position index of the queried mapping entry as a first position index. Thus, based on the linked list structure, the storage space allocated to the hash map table is further reduced, thereby reducing storage space usage.

[0156] In this embodiment of the application, in order to further reduce the storage space required by the hash map table, the hash map table can be constructed based on hash buckets and collision linked lists. That is, when the storage space allocated to the hash map table is not large enough, hash collisions will occur. At this time, multiple location indices need to be recorded at one address (i.e., hash bucket) in the hash map table.

[0157] A mapping entry consists of the position index of a sub-state, its parent state (i.e., the identifier of the parent state), and the characters between the parent state and the sub-state. Since the parent state and characters corresponding to the position index also need to be recorded, meaning that at least one mapping entry needs to be recorded at each hash bucket, the mapping entries can be recorded in the form of a linked list.

[0158] Furthermore, after obtaining the address to be used, the linked list recorded at the address in the hash mapping table can be retrieved. This linked list may contain one mapping entry or multiple mapping entries. Accordingly, the electronic device can query this linked list to see if a mapping entry containing the current state and the character to be matched exists. If it exists, the position index (i.e., the first position index) of the queried mapping entry is obtained.

[0159] Similarly, if the linked list does not contain a mapping entry that includes the current state and the character to be matched, it indicates that based on the segmentation result, the current state cannot be matched with the character to be matched. In this case, the second transition entry mentioned above is obtained for further processing. In this way, by performing a one-step verification on the state and the character, mapping errors are avoided, and the reliability of the matching is improved.

[0160] See Table 2, which is a hash map constructed based on the state transition table shown in Table 1.

[0161] Table 2

[0162] address Parent state K character substate Addr1 q0 "phon" q1 … … … … Addr26 q1 "e” q2 … … … … Addr32 q0 "tele" q3 … … … … Addr58 q3 "phon" q4 … … … … Addr1023 q4 "e” q5

[0163] In this formula, Hash(q0, "phon") = Addr1; Hash(q1, "e") = Addr26; Hash(q0, "tele") = Addr32; Hash(q3, "phon") = Addr58; Hash(q4, "e") = Addr1023. For Hash(q0, "phon") = Addr1, q0 represents the identifier of state q0, and Hash(q0, "phon") represents the hash value (Addr1) of the concatenation result of the identifier of state q0 and the character "phon". Other formulas can be found in the related description.

[0164] In Table 2, each row represents a mapping entry. The "K" character indicates the character between the parent and child states. The child states in Table 2 are the states to which the location index recorded in the mapping entry belongs. In other words, the actual mapping entry records the identifiers (i.e., location indices) of each state in the child state column. It's understandable that the actual stored mapping entries do not include the first column of Table 2; that is, when storing the hash mapping table, the electronic device only stores the last three columns of Table 2. The last column represents the location index. The first column in Table 2 is the address of the corresponding mapping entry in the hash mapping table.

[0165] To improve the efficiency of pattern string matching, the matching process can be executed by multiple execution machines. These multiple execution machines can operate in parallel or serial mode.

[0166] In one embodiment, the method is applied to an electronic device, which includes a second specified number of virtual machines; each virtual machine is used to execute the method described in the above embodiments; the second specified number of virtual machines operate in parallel; the character to be matched processed by each virtual machine initially is obtained by sliding the text to be matched along a specified window with a step size of 1; the size of the specified window is a first specified number; after matching the current character to be matched, each virtual machine slides along the specified window with a step size of the second specified number to obtain a new character to be matched. Thus, by processing in parallel through multiple virtual machines, the matching efficiency is improved.

[0167] In this embodiment of the application, the second specified number can be determined based on the performance of the electronic device itself. That is, the better the performance of the electronic device, the more virtual machines are set, i.e., the larger the second specified number.

[0168] The first character to be matched processed by each virtual machine is obtained by sliding the text to be matched with a step size of 1. That is, the first character to be matched processed by each virtual machine is different, and the first character to be matched processed by each virtual machine corresponds to a different byte offset. The character to be matched processed by each virtual machine in subsequent processing is obtained by sliding the text to be matched with a specified window with a second specified step size, starting from the position of the character to be matched processed in the previous processing.

[0169] For example, starting from the first character of the text to be matched, the first virtual machine captures the first character to be matched (called character 1) within a specified window. Then, it slides the specified window with a step size of 1 to obtain the second character to be matched (called character 2), and so on, until all the characters to be matched by the virtual machines are obtained. Subsequently, the first virtual machine starts from the position of character 1 and slides the specified window for a second specified number of characters to obtain the next character to be matched; the second virtual machine starts from the position of character 2 and slides the specified window for a second specified number of characters to obtain the next character to be matched; the third virtual machine starts from the position of character 3 and slides the specified window for a second specified number of characters to obtain the next character to be matched.

[0170] In this embodiment, multiple execution machines are implemented through software, and multiple virtual machines share CPU computing resources. See also Figure 6 One physical machine can simulate multiple virtual machines. Figure 6(Taking four virtual machines as an example for illustration), the pattern string matching method provided in this application embodiment is executed in parallel by multiple virtual machines, with one virtual machine corresponding to one matching state. Figure 6 (States 0, 1, 2, and 3 in the text). “abphonc” represents the text to be matched.

[0171] Based on the same inventive concept, this application also provides an electronic device, which includes a second specified number of logic chips, each logic chip being used to execute any of the methods in the above embodiments; the second specified number of logic chips are in parallel operation mode; the character to be matched processed by each logic chip for the first time is obtained by sliding the text to be matched according to a specified window with a step size of 1; the size of the specified window is a first specified number; each logic chip is used to slide the text to be matched according to the specified window with a second specified number of steps after the current character to be matched is completed, to obtain a new character to be matched.

[0172] If multiple actuators are implemented in hardware, such as using an FPGA, multiple physical machines (i.e., logic chips) can be built using the FPGA, each of which can be understood as an actuator. See also Figure 7 Electronic devices contain multiple physical machines ( Figure 7 (Taking four physical machines as an example for illustration), the pattern string matching method provided in this application embodiment is executed in parallel by multiple physical machines, with one physical machine corresponding to one matching state. Figure 7 (States 0, 1, 2, and 3 in the text). “abphonc” represents the text to be matched.

[0173] In one embodiment, if multiple actuators are implemented through FPGA hardware, after the initialization of the state transition table, hash map table, and Bloom filter is completed by software, the above data can be loaded into the on-chip or off-chip storage of the hardware acceleration chip, such as Block Random Access Memory (BRAM), Content Addressable Memory (CAM), or Static Random Access Memory (SRAM) space. After loading is completed, the hardware enters the ready state.

[0174] It is understandable that for each logic chip, after the current character to be matched is matched according to the method provided in the above embodiment, the logic chip can slide a second specified number of characters backward in the text to be matched according to the specified window to obtain a new character to be matched, and then the new character to be matched can be matched.

[0175] The following example uses the text to be matched as "telephonx123phone" and combines the state transition table shown in Table 1 and the hash mapping table shown in Table 2. This example contains four execution machines: execution machine 0, execution machine 1, execution machine 2, and execution machine 3. That is, the first specified number is 4, the second specified number is 4, and the specified window size is 4. The reference combinations for constructing the Bloom filter include: (q0,tele), (q0,phon), (q1,e), (q3,phon), and (q4,e).

[0176] Phase 1:

[0177] Determine the character to be matched for the first processing: slide the text to be matched in the specified window with a step size of 1. Accordingly, the capture window of Executor 0 obtains the character to be matched as "tele", the capture window of Executor 1 obtains the character to be matched as "elep", the capture window of Executor 2 obtains the character to be matched as "leph", and the capture window of Executor 3 obtains the character to be matched as "epho".

[0178] Execution machine 0 captures the window and obtains the character to be matched as "tele". The current state is 0. Execution machine 0 determines that it matches the Bloom filter and calculates Hash(q0, "tele") = Addr32. According to Table 2, the position index of the transition entry of state q3 is found at the address Addr32. According to Table 1, the matching string of state q3 is NULL, so the current state is updated to state q3.

[0179] The capture window of Execution Machine 1 obtains the character to be matched as "elep", and the current state is 0. Execution Machine 1 determines that it does not match the Bloom filter. Based on the identifier of the current state q0, it indexes to the transition entry of state q0 in Table 1. The failure chain is state q0, and the current state is still state q0.

[0180] Execution machine 2 captures the character to be matched as "leph" in the capture window, and the current state is 0. Execution machine 2 determines that it does not match the Bloom filter. Based on the identifier of the current state q0, it indexes to the transition entry of state q0 in Table 1. The failure chain is state q0, and the current state is still state q0.

[0181] The capture window of Execution Machine 3 obtains the character to be matched as "epho", and the current state is 0. Execution Machine 3 determines that it does not match the Bloom filter. Based on the identifier of the current state q0, it indexes to the transition entry of state q0 in Table 1. The failure chain is state q0, and the current state is still state q0.

[0182] Phase Two:

[0183] The text to be matched is slid along the specified window with a step size of 4. Execution machine 0 captures the window and obtains the character to be matched as "phon". The current state is state q3. Execution machine 0 determines that it matches the Bloom filter and calculates Hash(q3, "phon") = Addr58. According to Table 2, the position index of the transition entry of state q4 is found at address Addr58. According to Table 1, the matching string of state q4 is NULL, so the current state is updated to state q4.

[0184] The text to be matched is slid along the specified window with a step size of 4. The capture window of the execution machine 1 obtains the character to be matched as "honx". The current state is 0. The execution machine 1 determines that it does not match the Bloom filter. Based on the identifier of the current state q0, it indexes to the transition entry of state q0 in Table 1. The failure chain is state q0, and the current state is still state q0.

[0185] The text to be matched is slid along the specified window with a step size of 4. The execution machine 2 captures the window and obtains the character to be matched as "onx1". The current state is 0. The execution machine 2 determines that it does not match the Bloom filter. Based on the identifier of the current state q0, it indexes to the transition entry of state q0 in Table 1. The failure chain is state q0, and the current state is still state q0.

[0186] The text to be matched is slid along the specified window with a step size of 4. The execution machine 3 captures the window and obtains the character to be matched as "nx12". The current state is 0. The execution machine 3 determines that it does not match the Bloom filter. Based on the identifier of the current state q0, it indexes to the transition entry of state q0 in Table 1. The failure chain is state q0, and the current state is still state q0.

[0187] Phase Three:

[0188] The text to be matched is slid along the specified window with a step size of 4. The execution machine 0 captures the window and obtains the character to be matched as "x123". The current state is state q4. The execution machine 0 determines that it does not match the Bloom filter. Based on the identifier of the current state q4, it indexes the transition entry of state q4 in Table 1. The failure chain is state q0. The current state is updated to state q0.

[0189] The text to be matched is slid along the specified window with a step size of 4. The capture window of the execution machine 1 obtains the character to be matched as "123p". The current state is 0. The execution machine 1 determines that it does not match the Bloom filter. Based on the identifier of the current state q0, it indexes the transition entry of state q0 in Table 1. The failure chain is state q0, and the current state is still state q0.

[0190] The text to be matched is slid along the specified window with a step size of 4. The execution machine 2 captures the window and obtains the character to be matched as "23ph". The current state is 0. The execution machine 2 determines that it does not match the Bloom filter. Based on the identifier of the current state q0, it indexes to the transition entry of state q0 in Table 1. The failure chain is state q0, and the current state is still state q0.

[0191] The text to be matched is slid along the specified window with a step size of 4. The execution machine 3 captures the window and obtains the character to be matched as "3pho". The current state is 0. The execution machine 3 determines that it does not match the Bloom filter. Based on the identifier of the current state q0, it indexes to the transition entry of state q0 in Table 1. The failure chain is state q0, and the current state is still state q0.

[0192] Phase Four:

[0193] The text to be matched is slid along the specified window with a step size of 4. Execution machine 0 captures the window and obtains the character to be matched as "phon". The current state is state q0. Execution machine 0 determines that it matches the Bloom filter and calculates Hash(q0, "phon") = Addr1. According to Table 2, the position index of the transition entry of state q1 is found at the address Addr1. According to Table 1, the matching string of state q1 is NULL, so the current state is updated to state q1.

[0194] The text to be matched is slid along the specified window with a step size of 4. The capture window of the execution machine 1 obtains the character to be matched as "hone". The current state is 0. The execution machine 1 determines that it does not match the Bloom filter. Based on the identifier of the current state q0, it indexes the transition entry of state q0 in Table 1. The failure chain is state q0, and the current state is still state q0.

[0195] The text to be matched is slid along the specified window with a step size of 4. The execution machine 2 captures the window and obtains the character to be matched as "one". The current state is 0. The execution machine 2 determines that it does not match the Bloom filter. Based on the identifier of the current state q0, it indexes to the transition entry of state q0 in Table 1. The failure chain is state q0, and the current state is still state q0.

[0196] The text to be matched is slid along the specified window with a step size of 4. The execution machine 3 captures the window and obtains the character to be matched as "ne". The current state is 0. The execution machine 3 determines that it does not match the Bloom filter. Based on the identifier of the current state q0, it indexes to the transition entry of state q0 in Table 1. The failure chain is state q0, and the current state is still state q0.

[0197] Phase 5:

[0198] The text to be matched is slid along the specified window with a step size of 4. Execution machine 0 captures the window and obtains the character to be matched as "e". The current state is state q1. Execution machine 0 determines that it matches the Bloom filter and calculates Hash(q1, "e") = Addr26. According to Table 2, the position index of the transition entry of state q2 is found at the address Addr26. According to Table 1, the matching string of state q2 is "phone", and it is determined that the text to be matched contains the pattern string "phone".

[0199] Since there are no other unmatched characters in the text to be matched, the matching process of Executor 0, Executor 1, and Executor 2 ends.

[0200] Based on the same inventive concept, embodiments of this application also provide a pattern string matching device, see [link to relevant documentation]. Figure 8 The device includes:

[0201] The character to be matched module 801 is used to determine the character to be matched in the current state from the text to be matched, and use it as the character to be matched;

[0202] The address to be used calculation module 802 is used to calculate the hash value of the current state and the character to be matched, as the address to be used;

[0203] The first position index acquisition module 803 is used to acquire the position index recorded at the address to be used in the hash map table as the first position index; wherein, the hash map table is constructed based on the state transition table;

[0204] The first transition entry acquisition module 804 is used to acquire the transition entry recorded at the first position index in the state transition table, and obtain the transition entry of the sub-state that matches the current state and the character to be matched, as the first transition entry; wherein, the transition entry of a state includes: the matching string of the state and the failure chain;

[0205] The matching result acquisition module 805 is used to update the sub-state that matches the current state and the character to be matched to the new current state, return to the step of determining the character to be matched from the text to be matched that matches the current state as the character to be matched, until there are no unmatched characters in the text to be matched, and obtain the matching result based on the acquired first transition entry.

[0206] Optionally, the device further includes:

[0207] The second position index acquisition module is used to acquire the second position index if the address to be used in the hash mapping table is empty; wherein, the second position index represents the position of the current state transition entry in the state transition table;

[0208] The second transfer entry acquisition module is used to acquire the transfer entry recorded at the second position index in the state transfer table to obtain the second transfer entry;

[0209] The first update module is used to update the current state based on the failure chain in the second transfer entry, triggering the character to be matched determination module 801 until there are no unmatched characters in the text to be matched.

[0210] Optionally, the device further includes:

[0211] The detection module is used to detect whether the current state and the character to be matched match a pre-built Bloom filter before calculating the hash value of the current state and the character to be matched as the address to be used; wherein the Bloom filter is constructed based on the parent state and the characters between the parent state and the child state;

[0212] The address to be used calculation module 802 is specifically used to calculate the hash value of the current state and the character to be matched as the address to be used, given the current state and the case where the character to be matched matches the Bloom filter.

[0213] The device further includes:

[0214] The processing module is used to obtain a second position index if there are unmatched characters in the text to be matched when the current state and the character to be matched do not match the Bloom filter; wherein the second position index represents the position of the transition entry of the current state in the state transition table;

[0215] Obtain the transition entry recorded at the second position index in the state transition table to obtain the second transition entry;

[0216] The current state is updated based on the failure chain in the second transfer entry, triggering the character to be matched determination module 801 until there are no unmatched characters in the text to be matched.

[0217] Optionally, the state identifier for each state is the same as the position index of the transition entry for that state.

[0218] Optionally, in the state transition table, the transition entries for each state are stored consecutively and compactly.

[0219] Optionally, the hash map table also records the parent state and character corresponding to the first position index at the address to be used;

[0220] The first position index acquisition module 803 is specifically used to acquire the position index recorded at the address to be used in the hash map table as the first position index when the parent state and character recorded at the address to be used in the hash map table are consistent with the current state and the character to be matched.

[0221] Optionally, at least one mapping entry is recorded at the address to be used in the hash mapping table using a linked list structure; a mapping entry includes: a parent state, the position index of the transition entry of the child state of the parent state, and the characters between the parent state and the child state;

[0222] The first position index acquisition module 803 is specifically used to acquire the linked list recorded at the address to be used in the hash mapping table;

[0223] Retrieve from the obtained linked list a mapping entry containing the current state and the character to be matched;

[0224] Retrieve the position index from the queried mapping entry and use it as the first position index.

[0225] Optionally, the apparatus further includes: an entry construction module for obtaining a pattern string;

[0226] The pattern string is divided into segments according to a first specified number to obtain the transition entries for each state, the parent-child relationships between states, and the characters between the parent and child states.

[0227] Record the transition entries for each state to obtain the state transition table;

[0228] For each sub-state, determine its parent state and the characters between the parent state and the sub-state;

[0229] Calculate the hash value of the determined parent state and character;

[0230] The position index of the transition entry of the sub-state in the state transition table is recorded at the address represented by the calculated hash value to obtain the hash mapping table.

[0231] Optionally, the device is applied to an electronic device, which includes a second specified number of virtual machines; each virtual machine includes the above-mentioned modules; the second specified number of virtual machines operate in parallel.

[0232] The characters to be matched processed by each virtual machine for the first time are obtained by sliding the text to be matched through a specified window with a step size of 1; the size of the specified window is a first specified number.

[0233] Each virtual machine is used to slide the text to be matched in the specified window with the second specified number as the step size after the current character to be matched is completed, so as to obtain a new character to be matched.

[0234] This invention also provides an electronic device, such as... Figure 9 As shown, it includes a processor 901, a communication interface 902, a memory 903, and a communication bus 904, wherein the processor 901, the communication interface 902, and the memory 903 communicate with each other through the communication bus 904.

[0235] Memory 903 is used to store computer programs;

[0236] When processor 901 executes a program stored in memory 903, it performs the following steps:

[0237] Identify the characters in the text to be matched that match the current state, and use them as the characters to be matched;

[0238] Calculate the hash value of the current state and the character to be matched, and use it as the address to be used;

[0239] Obtain the location index of the address to be used recorded in the hash map table, and use it as the first location index; wherein, the hash map table is constructed based on the state transition table;

[0240] Obtain the transition entry recorded at the first position index in the state transition table, and obtain the transition entry of the sub-state that matches the current state and the character to be matched, as the first transition entry; wherein, the transition entry of a state includes: the matching string of the state and the failure chain;

[0241] The substate that matches the current state and the character to be matched is updated to the new current state. The process of determining the character to be matched from the text to be matched that matches the current state is then repeated until there are no unmatched characters in the text to be matched. Finally, the matching result is obtained based on the first transition entry.

[0242] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0243] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0244] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0245] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0246] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the steps of any of the above-described pattern string matching methods.

[0247] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the pattern string matching methods described above.

[0248] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0249] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0250] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0251] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A pattern string matching method, characterized in that, The method includes: Identify the characters in the text to be matched that match the current state, and use them as the characters to be matched; Calculate the hash value of the current state and the character to be matched, and use it as the address to be used; Obtain the location index of the address to be used recorded in the hash map table, and use it as the first location index; wherein, the hash map table is constructed based on the state transition table; Obtain the transition entry recorded at the first position index in the state transition table, and obtain the transition entry of the sub-state that matches the current state and the character to be matched, as the first transition entry; wherein, the transition entry of a state includes: the matching string of the state and the failure chain; The substate that matches the current state and the character to be matched is updated to the new current state. The process of determining the character to be matched from the text to be matched that matches the current state is then repeated until there are no unmatched characters in the text to be matched. Finally, the matching result is obtained based on the first transition entry.

2. The method according to claim 1, characterized in that, The method further includes: If the address to be used in the hash map table is empty, obtain the second position index; wherein, the second position index represents the position of the current state transition entry in the state transition table; Obtain the transition entry recorded at the second position index in the state transition table to obtain the second transition entry; Update the current state based on the failure chain in the second transition entry, and return to the step of determining the character to be matched from the text to be matched that matches the current state, until there are no unmatched characters in the text to be matched.

3. The method according to claim 1, characterized in that, Before calculating the hash value of the current state and the character to be matched as the address to be utilized, the method further includes: Detect whether the current state and the character to be matched match a pre-built Bloom filter; wherein the Bloom filter is constructed based on the parent state and the characters between the parent state and the child state; The calculation of the current state and the hash value of the character to be matched, as the address to be utilized, includes: If the current state and the character to be matched match the Bloom filter, calculate the hash value of the current state and the character to be matched, and use it as the address to be utilized; The method further includes: In the case where the current state and the character to be matched do not match the Bloom filter, a second position index is obtained; wherein the second position index represents the position of the transition entry of the current state in the state transition table; Obtain the transition entry recorded at the second position index in the state transition table to obtain the second transition entry; Update the current state based on the failure chain in the second transition entry, and return to the step of determining the character to be matched from the text to be matched that matches the current state, until there are no unmatched characters in the text to be matched.

4. The method according to claim 1, characterized in that, The state identifier for each state is the same as the position index of the transition entry for that state.

5. The method according to claim 1, characterized in that, In the state transition table, the transition entries for each state are stored consecutively and compactly.

6. The method according to claim 1, characterized in that, The hash map table also records the parent state and character corresponding to the first position index at the address to be used; The step of obtaining the location index of the address to be used in the hash map table as the first location index includes: If the parent state and character recorded at the address to be used in the hash map table are consistent with the current state and the character to be matched, the position index recorded at the address to be used in the hash map table is obtained as the first position index.

7. The method according to claim 6, characterized in that, At least one mapping entry is recorded at the address to be used in the hash mapping table using a linked list structure; a mapping entry includes: a parent state, the position index of the transition entry of the child state of the parent state, and the characters between the parent state and the child state; When the parent state and character recorded at the address to be used in the hash map table are consistent with the current state and the character to be matched, the position index recorded at the address to be used in the hash map table is obtained as the first position index, including: Obtain the linked list recorded at the address to be used in the hash map table; Retrieve from the obtained linked list a mapping entry containing the current state and the character to be matched; Retrieve the position index from the queried mapping entry and use it as the first position index.

8. The method according to claim 1, characterized in that, The state transition table and hash map table are constructed through the following steps: Get the pattern string; The pattern string is divided into segments according to a first specified number to obtain the transition entries for each state, the parent-child relationships between states, and the characters between the parent and child states. Record the transition entries for each state to obtain the state transition table; For each sub-state, determine its parent state and the characters between the parent state and the sub-state; Calculate the hash value of the determined parent state and character; The position index of the transition entry of the sub-state in the state transition table is recorded at the address represented by the calculated hash value to obtain the hash mapping table.

9. The method according to any one of claims 1 to 8, characterized in that, The method is applied to an electronic device, the electronic device comprising a second specified number of virtual machines; each virtual machine is used to execute the method according to any one of claims 1 to 8; the second specified number of virtual machines are in a parallel working mode; The character to be matched initially processed by each virtual machine is obtained by sliding the text to be matched through a specified window with a step size of 1; the size of the specified window is a first specified number. Each virtual machine is used to slide the text to be matched in the specified window with the second specified number as the step size after the current character to be matched is completed, so as to obtain a new character to be matched.

10. A pattern string matching device, characterized in that, The device includes: The character to be matched module is used to determine the character in the text to be matched that matches the current state, and use it as the character to be matched. The address to be used calculation module is used to calculate the hash value of the current state and the character to be matched, as the address to be used; The first position index acquisition module is used to acquire the position index recorded at the address to be used in the hash map table as the first position index; wherein, the hash map table is constructed based on the state transition table; The first transition entry acquisition module is used to acquire the transition entry recorded at the first position index in the state transition table, and obtain the transition entry of the sub-state that matches the current state and the character to be matched, as the first transition entry; wherein, the transition entry of a state includes: the matching string of the state and the failure chain; The matching result acquisition module is used to update the sub-state that matches the current state and the character to be matched to the new current state, return to the step of determining the character to be matched from the text to be matched that matches the current state as the character to be matched, until there are no unmatched characters in the text to be matched, and obtain the matching result based on the acquired first transition entry.

11. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-9.

12. An electronic device, characterized in that, The electronic device includes a second specified number of logic chips, each logic chip being used to execute the method described in any one of claims 1-8; the second specified number of logic chips are in parallel operation mode; the character to be matched processed by each logic chip for the first time is obtained by sliding the text to be matched along a specified window with a step size of 1; the size of the specified window is a first specified number; Each logic chip is used to slide the text to be matched according to the specified window with the second specified number as the step size after the current character to be matched is completed, so as to obtain a new character to be matched.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-9.

Images