Automaton management method and apparatus
By using fractal automata management methods, the performance and storage problems of AC automata when the pattern string data source changes frequently are solved, achieving efficient dynamic updates and fuzzy matching, and reducing storage space and maintenance costs.
Patent Information
- Application Number
- PCT/CN2025/083870
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-06-27
- Filing Date
- 2025-03-20
- Publication Date
- 2026-01-02
AI Technical Summary
Existing AC automata require complete reconstruction when the pattern string data source changes frequently, resulting in high performance overhead and difficulty in meeting real-time requirements. Furthermore, existing optimization solutions such as DAWG are complex to update and consume more storage space.
The fractal automaton management method is adopted. By constructing a trie and setting failure pointers, it supports dynamic and fast updating and deletion of pattern strings. Wildcard characters are introduced to realize fuzzy search, and data serialization and compression are performed.
It achieves efficient updates and fuzzy matching when the pattern string data source changes frequently, reduces storage space usage and maintenance costs, and meets real-time requirements.
Smart Images

Figure CN2025083870_02012026_PF_FP_ABST
Abstract
Description
Automatic machine management method and device
[0001] Cross-reference to related applications
[0002] The present application claims priority to the Chinese patent application No. 202410855358.6, filed on June 27, 2024, and entitled "Automatic machine management method and device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD
[0003] The present application relates to the technical field of computer, and in particular, to an automatic machine management method and device. BACKGROUND
[0004] The multi-mode string algorithm based on the AC automaton (Aho-Corasick automaton) is a common string matching algorithm (also known as a pattern matching algorithm). In actual application scenarios, the AC automaton not only considers pattern matching, but also considers processing of a pattern string data source. When the pattern string data source frequently changes, the Trie tree of the AC automaton needs to be pushed and reconstructed. This completely reconstructing the AC automaton has a great performance overhead, and it is difficult to meet the real-time requirements. Therefore, it is necessary to design a scheme that supports dynamic and fast construction / update of the AC automaton.
[0005] A current optimization scheme is to realize real-time update of the AC automaton by using a directed acyclic word graph (DAWG) (also known as a suffix automaton). However, the implementation of this scheme needs to introduce a suffix automaton, and the calculation process is complex and cumbersome. In addition, additional storage space is needed to store the related data of the suffix automaton, which leads to more overall storage space occupation, and is not a better optimization scheme.
[0006] Therefore, how to design a scheme that supports dynamic and fast construction / update of the AC automaton is still an important problem to be solved. SUMMARY
[0007] The present application provides an automatic machine management method and device, which are used to provide an automatic machine supporting fuzzy matching and capable of incremental update, and to manage the automatic machine.
[0008] In a first aspect, an embodiment of the present application provides an automatic machine management method, applied to a computing node, which can include: constructing a first dictionary tree according to first information, wherein the first information includes at least one first pattern string, each first pattern string includes at least one character, the first dictionary tree includes a plurality of nodes, if the at least one first pattern string includes at least one wildcard character, the plurality of nodes include a first root node, at least one second root node, and a plurality of non-root nodes, the first root node is empty, one second root node corresponds to one wildcard character, and the non-root node corresponds to one non-wildcard character; constructing a failure pointer of the plurality of nodes to form a first automatic machine, wherein the failure pointer of the first root node or the at least one second root node points to itself, the plurality of non-root nodes include the first non-root node, the first non-root node includes at least one of the following: a parent node, a sibling node, or a child node of any one of the at least one second root node, and the failure pointer of the first non-root node points to the second root node.
[0009] In combination with the first aspect, in a possible implementation, the plurality of non-root nodes further include at least one of the following: a second non-root node, a third non-root node, or a fourth non-root node, wherein the failure pointers of different types of non-root nodes satisfy at least one of the following: the second non-root node includes a child node of the first root node, and the failure pointer of the second non-root node points to the first root node; the third non-root node corresponds to one first pattern string, and the failure pointer of the third non-root node points to the first root node; the failure pointer of the fourth non-root node points to a node with the longest common suffix found upwards along the failure pointer of the parent node, and if the first root node or the second root node is encountered first during the finding process, the finding is stopped, and the failure pointer of the fourth non-root node points to the found first root node or second root node.
[0010] In combination with the first aspect, in a possible implementation, the method further includes: obtaining second information, the second information includes a newly added second pattern string, the second pattern string includes at least one character, and the second pattern string is different from the at least one first pattern string; adding one or more new nodes in the first dictionary tree to obtain a second dictionary tree, wherein if at least one character of the second pattern string has a common prefix with any one of the at least one first pattern string, the number of new nodes is less than the number of characters included in the second pattern string, and the new nodes correspond to non-common characters in the second pattern string; and re-constructing the failure pointers of part of the nodes in the second dictionary tree according to the relationship between the characters corresponding to the one or more new nodes and the characters corresponding to other nodes in the second dictionary tree to form a second automatic machine.
[0011] With reference to the first aspect, in a possible implementation manner, the method further includes: determining that a third mode string in the at least one first mode string is changed to a fourth mode string, the third mode string and the fourth mode string have at least one common character and at least one non-common character; according to the fourth mode string, updating nodes corresponding to the at least one non-common character in the first dictionary tree to obtain a third dictionary tree; and according to a relationship between characters corresponding to other nodes in the third dictionary tree and the at least one non-common character, reconstructing failure pointers of part of nodes in the third dictionary tree to form a third automaton.
[0012] With reference to the first aspect, in a possible implementation manner, the method further includes: determining that a fifth mode string is deleted from the at least one first mode string, the fifth mode string includes at least one character; and starting from a last character of the at least one character in the fifth mode string, sequentially deleting nodes corresponding to the at least one character in the fifth mode string from the first dictionary tree, and performing the following actions after each time a node is deleted: reconstructing a failure pointer for a target node, the target node includes a node to which an original failure pointer points to a deleted node; and stopping the operation until a first non-root node of the first dictionary tree is reached.
[0013] With reference to the first aspect, in a possible implementation manner, the method further includes: starting from the first root node, sequentially serializing and encoding data associated with each node of the first dictionary tree of the first automaton according to a parent-child dependency relationship of each node of the first dictionary tree of the first automaton and a frequency at which each node of the first dictionary tree is pointed to by a failure pointer of another node, to obtain a first data sequence; compressing the first data sequence to obtain a first compressed file; and saving the first compressed file in a persistent storage medium of the computing node.
[0014] With reference to the first aspect, in a possible implementation manner, the method further includes: reading the first compressed file from the persistent storage medium of the computing node; decompressing the first compressed file to obtain the first data sequence; and constructing the first dictionary tree and failure pointers of a plurality of nodes of the first dictionary tree according to the first data sequence to form the first automaton.
[0015] With reference to the first aspect, in a possible implementation manner, any first mode string in the at least one first mode string can be implemented as any one of the following: a single mode string; a multi-mode string; and a mode string including a wildcard character.
[0016] In a second aspect, the embodiments of the present application provide an automatic machine management method, applied to a computing node, the method comprising: obtaining a first automatic machine, wherein the first automatic machine is constructed according to first information, the first information comprising at least one first pattern string, each first pattern string comprising at least one character, a first dictionary tree of the first automatic machine comprising a plurality of nodes, if the at least one first pattern string comprises at least one wildcard character, the plurality of nodes comprising a first root node, at least one second root node, and a plurality of non-root nodes, the first root node being empty, one second root node corresponding to one wildcard character, and the non-root node corresponding to one non-wildcard character; obtaining second information, the second information comprising a newly added second pattern string, the second pattern string comprising at least one character, the second pattern string being different from the at least one first pattern string; adding one or more newly added nodes in the first dictionary tree to obtain a second dictionary tree, wherein if the at least one character of the second pattern string has a common prefix with any first pattern string in the at least one first pattern string, the number of newly added nodes is less than the number of characters contained in the second pattern string, and the newly added nodes correspond to non-common characters in the second pattern string; and reconstructing failure pointers of part of the nodes in the second dictionary tree according to relationships between characters corresponding to the one or more newly added nodes and characters corresponding to other nodes in the second dictionary tree, to form a second automatic machine.
[0017] In a possible implementation manner of the second aspect, the plurality of nodes of the first dictionary tree or the second dictionary tree satisfy at least one of the following conditions: the failure pointer of the first root node or the at least one second root node points to itself; the plurality of non-root nodes further comprise at least one of the following: a first non-root node, a second non-root node, a third non-root node, or a fourth non-root node, wherein the first non-root node comprises at least one of the following of any second root node in the at least one second root node: a parent node, a sibling node, or a child node, and the failure pointer of the first non-root node points to the second root node; the second non-root node comprises a child node of the first root node, and the failure pointer of the second non-root node points to the first root node; the third non-root node corresponds to one first pattern string, and the failure pointer of the third non-root node points to the first root node; and the failure pointer of the fourth non-root node points to a node with the longest common suffix found upwards along the failure pointer of the parent node, and if the first root node or the second root node is encountered first during the finding process, the finding is stopped, and the failure pointer of the fourth non-root node points to the found first root node or second root node.
[0018] With reference to the second aspect, in a possible implementation manner, the method further includes: determining to delete a fifth pattern string from the at least one first pattern string, the fifth pattern string including at least one character; starting from a last character of the at least one character in the fifth pattern string, sequentially deleting, from the first dictionary tree, nodes corresponding to the at least one character in the fifth pattern string, and performing the following actions after deleting each node: reconstructing a failure pointer for a target node, the target node including a node to which the original failure pointer points to the deleted node; and stopping the operation until a first non-root node of the first dictionary tree is reached.
[0019] With reference to the second aspect, in a possible implementation manner, the method further includes: starting from the first root node, sequentially serializing and encoding data associated with each node of the first dictionary tree of the first automaton according to parent-child dependency relationships of the nodes and frequencies at which each node of the first dictionary tree is pointed to by a failure pointer of another node, to obtain a first data sequence; compressing the first data sequence to obtain a first compressed file; and storing the first compressed file in a persistent storage medium of the computing node.
[0020] With reference to the second aspect, in a possible implementation manner, the method further includes: reading the first compressed file from the persistent storage medium of the computing node; decompressing the first compressed file to obtain the first data sequence; and constructing the first dictionary tree and failure pointers of a plurality of nodes of the first dictionary tree according to the first data sequence, to form the first automaton.
[0021] With reference to the second aspect, in a possible implementation manner, any first pattern string in the at least one first pattern string can be implemented as any one of the following: a single pattern string; a multi-pattern string; and a pattern string including a wildcard character.
[0022] In a third aspect, the embodiments of the present application provide an automatic machine management method, applied to a computing node, the method comprising: obtaining a first dictionary tree of a first automatic machine and failure pointers of a plurality of nodes of the first dictionary tree, wherein the first automatic machine is constructed according to first information, the first information comprising at least one first pattern string, each first pattern string comprising at least one character, the first dictionary tree comprising a plurality of nodes, if the at least one first pattern string comprises at least one wildcard character, the plurality of nodes comprising a first root node, at least one second root node and a plurality of non-root nodes, the first root node being empty, one second root node corresponding to one wildcard character, and the non-root node corresponding to one non-wildcard character; starting from the first root node of the first automatic machine, traversing according to the parent-child dependency relationship of each node of the first dictionary tree and the frequency of each node of the first dictionary tree being pointed to by the failure pointer of another node, sequentially serializing and encoding the data associated with each node of the first dictionary tree to obtain a first data sequence; compressing the first data sequence to obtain a first compressed file; and saving the first compressed file in a persistent storage medium of the computing node.
[0023] In combination with the third aspect, in a possible implementation manner, the plurality of nodes of the first dictionary tree satisfy at least one of the following conditions: the failure pointer of the first root node or the at least one second root node points to itself; the plurality of non-root nodes further comprise at least one of the following: a first non-root node, a second non-root node, a third non-root node or a fourth non-root node, wherein the first non-root node comprises at least one of the following of any second root node in the at least one second root node: a parent node, a sibling node or a child node, and the failure pointer of the first non-root node points to the second root node; the second non-root node comprises a child node of the first root node, and the failure pointer of the second non-root node points to the first root node; the third non-root node corresponds to one first pattern string, and the failure pointer of the third non-root node points to the first root node; and the failure pointer of the fourth non-root node points to a node with the longest common suffix found upwards along the failure pointer of the parent node, if the first root node or the second root node is encountered first during the finding process, the finding is stopped, and the failure pointer of the fourth non-root node points to the found first root node or second root node.
[0024] In combination with the third aspect, in a possible implementation manner, the method further comprises: reading the first compressed file from the persistent storage medium of the computing node; decompressing the first compressed file to obtain the first data sequence; constructing the first dictionary tree and the failure pointers of the plurality of nodes of the first dictionary tree according to the first data sequence to form the first automatic machine.
[0025] With reference to the third aspect, in a possible implementation manner, any first pattern string in the at least one first pattern string can be implemented as any one of the following: a single pattern string; a multi-pattern string; and a pattern string containing a wildcard character.
[0026] In a fourth aspect, an embodiment of the present application provides a communication apparatus, comprising at least one processor and an interface circuit, the interface circuit being configured to provide data or code instructions for the at least one processor, and the at least one processor being configured to implement the method according to any one of the first aspect, the second aspect or the third aspect, and possible implementation manners thereof, by means of a logic circuit or executing the code instructions.
[0027] In a fifth aspect, an embodiment of the present application provides a computer readable storage medium, which stores program codes, and when the program codes are run on a computer, the computer is caused to execute the method according to any one of the first aspect, the second aspect or the third aspect, and possible implementation manners thereof.
[0028] In a sixth aspect, an embodiment of the present application provides a computer program product, and when the computer program product is run on a computer, the computer is caused to execute the method according to any one of the first aspect, the second aspect or the third aspect, and possible implementation manners thereof.
[0029] On the basis of the implementation of the above aspects, embodiments of the present application can be further combined to provide more implementations. BRIEF DESCRIPTION OF DRAWINGS
[0030] FIG. 1 is a schematic diagram of a chain structure corresponding to the KMP algorithm;
[0031] FIG. 2 is a schematic diagram of a dictionary (prefix) tree structure corresponding to a traditional AC automaton;
[0032] FIG. 3 shows a schematic diagram of a dictionary (prefix) tree structure of a fractal automaton according to an embodiment of the present application;
[0033] FIG. 4 shows a schematic diagram of a meaning of a state machine element used to describe a fractal automaton according to an embodiment of the present application;
[0034] FIG. 5 shows a state diagram of a dictionary (prefix) tree of a fractal automaton according to an embodiment of the present application;
[0035] FIG. 6 shows a schematic diagram of a flow of an automaton management (construction) method according to an embodiment of the present application;
[0036] FIGS. 7-8 show schematic diagrams of flows of an automaton management (incremental update) method according to an embodiment of the present application;
[0037] Figure 9 shows a flow diagram of the automatic machine management (incremental update) method according to an embodiment of the present application;
[0038] Figures 10-12 show different example diagrams of the automatic machine management (incremental update) according to an embodiment of the present application;
[0039] Figure 13 shows a flow diagram of the automatic machine management (compression) method according to an embodiment of the present application;
[0040] Figure 14 shows a principle example diagram of the automatic machine management (compression) according to an embodiment of the present application;
[0041] Figure 15 shows a principle diagram of the pattern matching method for the vehicle field according to an embodiment of the present application;
[0042] Figures 16-18 show different diagrams of the structure of the communication device according to an embodiment of the present application;
[0043] Figure 19 shows another diagram of the structure of the communication device according to an embodiment of the present application. DETAILED DESCRIPTION
[0044] For the convenience of understanding, the following first explains some terms and technologies involved in the present application.
[0045] 1. Pattern string:
[0046] It is a concept in string matching algorithm, which refers to a specific substring that needs to be found in a main string (such as a string, or a text string, or a data string, etc.), called a pattern string. Such a finding operation is common in computer science and can be used to find a specific sub-pattern or substring in different main strings.
[0047] In the pattern matching process, the pattern string is a string used for comparison in the main string to determine whether it matches. Usually, the length of the pattern string is less than that of the main string. For example, in the string T, the string P is searched, T is the main string, and P is the pattern string. Let the length of T be m and the length of P be n, where m and n are integers, and m is greater than n.
[0048] 2. Single pattern string algorithm:
[0049] Single pattern string algorithm, i.e., one string matches one string. Common single pattern string algorithms include: brute force (Brute Force, BF) algorithm, Rabin-Karp (RK) algorithm, Robert S. Boyer-J Strother Moore (BM) algorithm, Knuth-Morris-Pratt (KMP) algorithm.
[0050] For example, the KMP algorithm, also known as Knuth-Morris-Pratt operation, is an efficient algorithm for string matching, aiming to solve the problem of finding the occurrence position of a pattern string in a main string (e.g., a text string). The core idea of the KMP algorithm is to use known information to avoid unnecessary comparison operations, thereby improving algorithm efficiency.
[0051] The time complexity of the KMP algorithm is O(n+m), where n is the length of the pattern string and m is the length of the main string. Compared with the naive matching algorithm, when the KMP algorithm fails to match, it does not retrace the main string pointer and the pattern string pointer, but keeps the main string pointer unchanged and modifies the pattern string pointer to move to the effective matching position as much as possible. In this way, the KMP algorithm can significantly reduce the number of unnecessary comparisons and improve matching speed. This pointer is called the next pointer.
[0052] For example, for a pattern string aabaaf, the KMP structure is shown in FIG. 1, where a hollow circle represents a node, and a symbol φ represents the root node, which is empty. When constructing the KMP structure, each character in the pattern string aabaaf is mapped to a node in the KMP structure, and a solid arrow represents the connection relationship between different nodes. Filled circles (a kind of representation of solid circles) represent the matching position of each pattern string, that is, the last character of the pattern string. Hollow circles represent non-terminal characters of each pattern string. At the same time, the same prefix length of each substring in the pattern string is recorded. When matching fails in the text string, the mismatch pointer is moved to the corresponding trailing character of the prefix based on the position of the character before the matching failure, that is, the backtracking position. As shown in FIG. 1, the failure pointer of the first character a points to the root node, the failure pointer of the second character a points to the first character a, the failure pointer of the character b points to the root node, the failure pointer of the third character a points to the first character a, the failure pointer of the fourth character a points to the second character a, and the failure pointer of the character f points to the root node. In the case of mismatch of a certain character, moving the failure pointer to the backtracking position based on the mismatch and performing the matching process from this position can reduce unnecessary matching times.
[0053] 3. Multi-pattern string algorithm:
[0054] The multi-pattern string algorithm refers to an algorithm for finding the matching position of multiple pattern strings in a main string. Common multi-pattern string algorithms include multi-pattern string algorithms based on Trie trees and multi-pattern string algorithms based on AC automata (Aho-Corasick automata).
[0055] (1) Trie tree, also known as prefix tree or dictionary tree, is a tree-shaped data structure, which is used to store strings and enable processing of string matching, and is usually used to solve the problem of quickly finding a certain string in a set of character collections.
[0056] The basic idea of the Trie tree is to split a string into single characters, and then build a tree, where the root node does not contain characters, and each child node of a node other than the root node contains (or represents, corresponds to) a character. The characters concatenated along the path from the root node to a leaf node are the string (or key word) corresponding to the leaf node. All child nodes of each node contain different characters. After building the Trie tree, a method similar to depth-first search can be used to find all matching strings in the Trie tree.
[0057] (2) AC automaton is an automatic machine algorithm, which is an improvement based on the Trie tree, and optimizes the matching process of the Trie tree.
[0058] Specifically, in the Trie tree structure of the AC automaton, there is a root node and a plurality of non-root nodes, the non-root nodes are also called leaf nodes, the root node is null, and each non-root node corresponds to a character of a pattern string. The AC automaton borrows the idea of the KMP algorithm, and adds a failure pointer (or called adaptation link point, mismatch pointer) to each node of the Trie tree, which points to another node in the AC automaton, so that after a match failure at the node, it can directly jump to the node pointed to by the failure pointer for judgment, thereby avoiding repeated judgment process. Among them, the failure pointer of the root node points to itself. Assuming that the failure pointer of a node A points to node B, it means that the string upwards of node A matches the string from the root node to node B, and the matching length is the longest, i.e. the string upwards of node A and the string from the root node to node B have the longest common suffix. The KMP algorithm can be regarded as a special case of the AC automaton. The role of the above failure pointer is the same as that of the next pointer in the KMP algorithm
[0059] Figure 2 shows an example of an AC automaton, whose Trie tree structure contains multiple pattern strings: she, shes, his, he, here, hers, each circle represents a tree node, the root node is empty, and each non-root node corresponds to a character of a pattern string. The root node is represented by the symbol φ, and the filled circles (one way of representing solid circles) represent the matching positions of each pattern string, i.e., the last character of the pattern string. The hollow circles represent the non-last characters of each pattern string. For the pattern string she, the failure pointers of the non-root nodes corresponding to the characters s and e point to the root node, and the failure pointer of the node corresponding to the character h points to the h in the pattern string his. For the pattern string shes, the failure pointer of the node corresponding to the non-common character s of the pattern string she points to the root node. For the pattern string his, the failure pointers of the non-root nodes corresponding to the characters h, i, and s point to the root node. For the pattern string he, the failure pointers of the non-root nodes corresponding to the characters h and e point to the root node. For the pattern string hers, the failure pointers of the non-root nodes corresponding to the characters h, e, r, and s point to the root node. For the pattern string here, the failure pointers of the non-root nodes corresponding to the characters h, e, r, and e point to the root node.
[0060] 4. State Machine:
[0061] State Machine is the abbreviation of Finite State Machine (FSM), which is a mathematical model abstracted from the operation rules of real things, and is a conceptual machine that can take certain operations to respond to an external event.
[0062] The components of a state machine can include one or a combination of the following:
[0063] (1) State (or status): describes the state that the system can be in. The state is a certain condition or state in the system, which can be represented by a circle.
[0064] (2) Transition: the transition relationship between states. Transition is a relationship between states, indicating the transition of the system from one state to another, which can be represented by a line with an arrow.
[0065] (3) Event: an event that triggers state transition. Event is a condition that causes the system to transition from one state to another, which can be represented by a rectangle.
[0066] (4) Action: an operation performed during state transition. Action is an operation or behavior performed by the system during the transition from one state to another, which can be represented by a label or tag.
[0067] (5) Condition: Condition of state transition. Condition is a condition or limit of controlling state transition, usually used on the line of transition.
[0068] (6) Initial state: Initial state of state diagram, which can be represented by a hollow circle.
[0069] (7) Final state: Final state of state diagram, which can be represented by a solid circle or a double circle.
[0070] In the embodiments of the present application, the elements of the state machine can be used to describe the relationship between different nodes in the tree structure of the automaton.
[0071] 5. Huffman coding:
[0072] Also known as Huffman coding, it is a coding method, which is a variable length code (VLC). This method completely depends on the character probability to construct the average length of the prefix code word shortest, sometimes called the best coding, generally called Huffman coding, sometimes also called Huffman coding.
[0073] In actual application scenarios, AC automata not only consider pattern matching, but also consider the processing of pattern string data sources (or called keywords / rules, etc.), such as frequent changes (for example, updating or removing data) of pattern string data sources. For such cases, the Trie tree of the AC automaton needs to be pushed and reconstructed constantly. For a real-time system with thousands of pattern string data sources and very frequent updates, the AC automaton needs to be preprocessed for the full amount of pattern string data source every time a new pattern string is added, so as to reconstruct the AC automaton. This way of completely reconstructing the AC automaton has great performance overhead, and it is difficult to meet the real-time requirements. Therefore, in the above application scenarios, a scheme is needed to support dynamic and fast construction / update of the AC automaton.
[0074] A current optimization scheme is to realize real-time update of the AC automaton through a directed acyclic word graph (DAWG) (also known as a suffix automaton). For example, when a new pattern string needs to be added, a DAWG is introduced and the nodes in the DAWG are traversed in turn to perform incremental update on the original AC automaton.
[0075] In this process, on the one hand, the AC automaton cannot support wildcard characters and fuzzy search. On the other hand, a DAWG needs to be introduced, and the updating method of the AC automaton is complex and cumbersome. In addition, the related data of the DAWG needs to be stored additionally, and the total storage space required is twice that before the update, which leads to more storage space occupied and improves the space complexity.
[0076] To solve the above problems, the embodiment of the present application proposes a pattern matching method and device, which introduces a generalized automaton (such as a fractal automaton) to support single pattern matching or multi-pattern matching, fuzzy search, and incremental updating / deletion. In this scheme, the fractal automaton can also be subjected to a special compression / decompression processing method, which reduces the overall debugging and maintenance cost. The method and device are based on the same technical concept, and since the principles of the method and device for solving problems are similar, the implementation of the device and the method can be mutually referred to, and the repeated parts will not be described again. In addition, in various embodiments of the present application, if there is no special description and logical conflict, the terms and / or descriptions between the various embodiments are consistent and can be mutually referred to, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.
[0077] It should be noted that "at least one" in the embodiments of the present application means one or more, and "multiple" means two or more. "And / or" describes the association between the associated objects, which means that there can be three kinds of relationships, for example, A and / or B, which can represent the following cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b, or c, can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.
[0078] In addition, unless otherwise specified, the ordinal numbers mentioned in the embodiments of the present application are used to distinguish multiple objects, and are not used to limit the priority or importance of the multiple objects. For example, the first node and the second node are only used to distinguish different nodes, and do not mean that the priority or importance of the two nodes is different.
[0079] The basic structure of the fractal automaton of the embodiments of the present application will be described in detail below in combination with the drawings.
[0080] In the embodiments of the present application, the fractal automaton is a generalized AC automaton, a special prefix tree (Trie Tree) abstract data type, which can be divided into main root nodes, side root nodes and normal nodes (also referred to as non-root nodes) in structure. In the Trie tree of the fractal automaton, the side root nodes correspond to wildcard characters, and the normal nodes correspond to non-wildcard characters. The "generalization" means that when there is no side root node in the Trie tree structure of the fractal automaton, the fractal automaton is equivalent to a traditional AC automaton, and the structure and failure pointers are as shown in FIG. 2. When there is no side root node and no branch in the Trie tree structure of the fractal automaton, the fractal automaton degenerates into a single pattern string structure (or chain structure) corresponding to the KMP algorithm, and the structure and failure pointers are as shown in FIG. 1. The KMP algorithm and the traditional AC automaton can also be regarded as a special case of the fractal automaton of the embodiments of the present application.
[0081] It should be understood that the "fractal automaton" herein is only an example of the name of the generalized AC automaton and is not any limitation, and in other embodiments, the fractal automaton can also have other names, which are not limited in the embodiments of the present application.
[0082] As an example, the wildcard characters supported by the fractal automaton and the description / explanation of their meanings can be as shown in Table 1:
[0083] Table 1
[0084] If a node in the tree structure of the fractal automaton corresponds to a wildcard character shown in Table 1, when natural language processing, information retrieval or big data processing is performed based on the fractal automaton, the wildcard character can match one or more characters fuzzily, so that the fractal automaton can realize fuzzy processing or fuzzy search function. Therefore, the fractal automaton can support fuzzy matching compared with the traditional AC automaton, and has a larger application range or application scenario.
[0085] In order to facilitate understanding, the basic structure of the fractal automaton of the embodiments of the present application will be introduced first in combination with FIGS. 3-5.
[0086] In an example, as shown in FIG. 3, "*" represents a wildcard character, and the corresponding pattern strings in the Trie tree structure of the fractal automaton include: she*he, his, heehee, he*she, he*his, he*hee, he*here and he*hers.
[0087] In the Trie tree structure, a plurality of nodes are included, and a node is represented by a circle. The plurality of nodes can include a main root node, at least one side root node, and a plurality of non-root nodes. The main root node is the root node of the whole tree, and is represented by a symbol φ. The main root node is empty, i.e., the main root node does not correspond to any character in the pattern strings. Each side root node is non-empty and corresponds to a wildcard character in the pattern strings, for example, *. Each non-root node is non-empty and corresponds to a non-wildcard character in the pattern strings, for example, h, s, i, e, r, etc. The plurality of non-root nodes can further include a node corresponding to the last character of the pattern strings and a node corresponding to a non-last character and a non-wildcard character of the pattern strings. In order to distinguish, the node corresponding to the last character of each pattern string is represented by a filled circle (a representation of a solid circle) in FIG. 3. The hollow circle represents a non-last character of each pattern string. A black solid arrow connects the parent node and the child node in the tree structure, and the arrow points from the parent node to the child node. Different child nodes of the same node are sibling nodes.
[0088] Similar to the AC automaton, in the Trie tree structure of the fractal automaton, each node can have a failure pointer, which is used to move to the node pointed by the failure pointer when the corresponding node is mismatched, so as to reduce unnecessary judgment process. Due to the existence of the node corresponding to the wildcard character, the determination method of the failure pointer of each node of the fractal automaton of the embodiment of the present application is different from that of the AC automaton.
[0089] As an example, in the embodiment of the present application, the failure pointer of each node of the fractal automaton can satisfy at least one of the following conditions:
[0090] ①The failure pointers of the main root node and the side root node both point to themselves.
[0091] ②The failure pointer of the non-root node adjacent to the main root node points to the main root node, and the failure pointer of the non-root node adjacent to the side root node points to the side root node. The non-root node adjacent to the main root node includes the child node of the main root node, and the non-root node adjacent to the side root node includes the parent node, sibling node and child node of the side root node.
[0092] ③The failure pointers of the non-root nodes corresponding to the last characters of the pattern strings all point to the main root node;
[0093] IV. The failure pointer of the non-root node corresponding to the non-terminal character of the pattern string points to the node with the largest common suffix with the current node found by searching upwards (or in other words, backtracking) along the failure pointer of its parent node. If the main root node or the side root node is encountered first during the searching process, the searching process is stopped and the failure pointer is directed to the main root node or the side root node found. Here, the node with the largest common suffix means the node with the same character corresponding to the current node among the child nodes of the node found by searching upwards along the failure pointer of the parent node. Searching upwards along the failure pointer of its parent node means searching along the failure path of the parent node of the current node, that is, if the node with the largest common suffix is not found at the node pointed to by the failure pointer of the parent node and the main root node or the side root node is not encountered, the searching process needs to continue searching along the failure pointer of the parent node of the node pointed to by the failure pointer of the parent node until the node satisfying the condition is found or the main root node or the side root node is encountered first.
[0094] As shown in FIG. 3, the failure pointers of the main root node φ and the two side root nodes corresponding to the wildcard character * point to themselves, and the failure pointers of the nodes corresponding to the characters h and s adjacent to the main root node point to the main root node. The failure pointer of the non-root node adjacent to the side root node (corresponding to the character *) on the left side of FIG. 3 is the node corresponding to the characters e, e, h and s around it, and the failure pointer of the non-root node adjacent to the side root node (corresponding to the character *) on the right side of FIG. 3 is the node corresponding to the characters e and h around it, and the failure pointers of the nodes corresponding to the terminal characters of the pattern strings, for example, the node represented by the filled circle, point to the main root node.
[0095] In determining the failure pointer of the non-root node corresponding to the non-terminal character of the pattern string, starting from the main root node, each node of the Trie tree is traversed layer by layer (from left to right) downwards from the main root node, and for the node corresponding to the non-terminal character in each pattern string, the node with the largest common suffix with the current node is found by searching along the failure path of the parent node of the current node according to the condition IV, that is, the node pointed to by the failure pointer of the current node. If the main root node or the side root node is encountered first during the searching process, the searching process is stopped and the failure pointer is directed to the main root node or the side root node found.
[0096] For example, in Fig. 3, taking the node corresponding to the character i in the third layer as the current node to be confirmed, the parent node of the current node is the node corresponding to the character h in the second layer, the failure pointer of the parent node points to the main root node, and there is no node corresponding to the character i in the current node in the child nodes of the main root node, i.e., the node with the maximum common suffix is not found, and the main root node is encountered first, so the failure pointer of the current node is made to point to the found main root node.
[0097] For example, in Fig. 3, taking the node corresponding to the character i in the third layer as the current node to be confirmed, the parent node of the current node is the node corresponding to the character h in the second layer, the failure pointer of the parent node points to the main root node, and there is no node corresponding to the character i in the current node in the child nodes of the main root node, i.e., the node with the maximum common suffix is not found, and the main root node is encountered first, so the failure pointer of the current node is made to point to the found main root node.
[0098] For example, in Fig. 3, taking the node corresponding to the character i in the third layer as the current node to be confirmed, the parent node of the current node is the node corresponding to the character h in the second layer, the failure pointer of the parent node points to the main root node, and there is no node corresponding to the character i in the current node in the child nodes of the main root node, i.e., the node with the maximum common suffix is not found, and the main root node is encountered first, so the failure pointer of the current node is made to point to the found main root node.
[0099] For example, in Fig. 3, taking the node corresponding to the character i in the third layer as the current node to be confirmed, the parent node of the current node is the node corresponding to the character h in the second layer, the failure pointer of the parent node points to the main root node, and there is no node corresponding to the character i in the current node in the child nodes of the main root node, i.e., the node with the maximum common suffix is not found, and the main root node is encountered first, so the failure pointer of the current node is made to point to the found main root node.
[0100] For example, in Fig. 3, taking the node corresponding to the character i in the third layer as the current node to be confirmed, the parent node of the current node is the node corresponding to the character h in the second layer, the failure pointer of the parent node points to the main root node, and there is no node corresponding to the character i in the current node in the child nodes of the main root node, i.e., the node with the maximum common suffix is not found, and the main root node is encountered first, so the failure pointer of the current node is made to point to the found main root node.
[0101] In another example, the state diagram of the fractal automaton of the embodiments of the present application can be described using relevant elements of a state machine.
[0102] As shown in FIG. 4, the constituent elements of the state diagram of the fractal automaton can include: states; transitions; conditions; start state; and end state. For example, using qi to represent a state, each state corresponds to a node of the Trie tree of the fractal automaton, i can take values of 0, 1, 2, …, m, and m can take values of integers greater than 1. The start state is represented as q0, and the end state is represented by the last character of each pattern string, which is filled in a circle. Black solid arrows represent the transition relationship between different states / nodes. The characters marked beside the black solid arrows represent the conditions for state transition, such as characters a, b, c, etc. Dotted arrows represent the failure pointers added to each node, which also satisfy the conditions ①-④ described above in connection with FIG. 3.
[0103] In FIG. 5, an example is illustrated with m = 30. In this state diagram, the initial state q0 corresponds to the main root node, which is empty. Starting from the main root node, state transition is performed according to the transition conditions that are satisfied, for example, if q0 satisfies the condition (e.g., matches the character b), q0 is transitioned from to q1. If q0 satisfies the condition (e.g., matches the character a), q0 is transitioned from to q2. If q1 satisfies the condition (e.g., matches the character c), q1 is transitioned from to q3. If q1 satisfies the condition (e.g., matches the character d), q1 is transitioned from to q4. If q2 satisfies the condition (e.g., matches the character b), q2 is transitioned from to q5. If q3 satisfies the condition (e.g., matches the character *), q3 is transitioned from to q6, and so on, until the state is transitioned to q30.
[0104] The tree structure corresponds to the following multiple pattern strings: abc*ab, bde, bdebc, bcdbcb, bcdbczb, bc*bc, bc*bbdz, bc*abz, bc*abcc, bc*abcd. The states of different nodes correspond to the prefixes of the multiple pattern strings, as shown below: q1 = b, q2 = a, q3 = bc, q4 = bd, q5 = ab, q6 = bc*, q7 = bcd, q8 = bde, q9 = abc, q10 = bc*a, q11 = bc*b, q12 = bcdb, q13 = bdeb, q14 = abc*, q15 = bc*ab, q16 = bc*bc, q17 = bc*bb, q18 = bcdbc, q19 = bdebc, q20 = abc*a, q21 = bc*abz, q22 = bc*abc, q23 = bc*bbd, q24 = bcdbcb, q25 = bcdbcz, q26 = abc*ab, q27 = bc*abcc, q28 = bc*abcd, q29 = bc*bbdz, q30 = bcdbczb.
[0105] In FIG. 5, the failure pointers of the respective nodes satisfy the conditions 1-4 introduced above in connection with FIG. 3.
[0106] For example, the node corresponding to q0 is the main root node, and its failure pointer points to itself (not shown in the figure). The nodes corresponding to q1 and q2 are ordinary nodes adjacent to the main root node, and their failure pointers point to the main root node. The node corresponding to q6 is a side root node, and its failure pointer points to itself (not shown in the figure). The nodes corresponding to q3, q7 and q10 are nodes adjacent to the side root node corresponding to q6, and their failure pointers point to the side root node. The nodes corresponding to q9 and q20 are nodes adjacent to the side root node corresponding to q14, and their failure pointers point to the side root node. The nodes corresponding to q8, q11, q16, q19, q21, q24, q26, q27, q28, q29 and q30 are the last characters of the corresponding pattern strings, and their failure pointers all point to the main root node. In addition, based on the analysis of the above condition 4, the failure pointer of the node corresponding to q4 points to the main root node, the failure pointer of the node corresponding to q5 points to the node corresponding to q1, the failure pointer of the node corresponding to q12 points to the node corresponding to q11, the failure pointer of the node corresponding to q13 points to the node corresponding to q1, the failure pointer of the node corresponding to q15 points to the node corresponding to q11, the failure pointer of the node corresponding to q17 points to the node corresponding to q1, the failure pointer of the node corresponding to q18 points to the node corresponding to q16, the failure pointer of the node corresponding to q22 points to the node corresponding to q16, the failure pointer of the node corresponding to q23 points to the node corresponding to q4, and the failure pointer of the node corresponding to q25 points to the main root node.
[0107] The fractal automaton shown in FIG. 3 and FIG. 5 is a generalized AC automaton that can support fuzzy matching and has higher real-time performance compared to the BM algorithm, the AC automaton and its variants. Meanwhile, the fractal automaton supports matching of single pattern strings, multiple pattern strings and wildcard pattern strings, and has a wider application scenario.
[0108] The construction process of the fractal automaton shown in FIG. 3 or FIG. 5 will be described below in combination with a method flowchart.
[0109] To facilitate distinction, the main root node in FIG. 3 or FIG. 5 is denoted as a first root node, each of the at least one side root node is denoted as a second root node, and the nodes other than the first root node and the second root node are denoted as non-root nodes. The non-root nodes are divided into first non-root nodes, second non-root nodes, third non-root nodes and fourth non-root nodes. The first non-root node can be a non-root node adjacent to any second root node, including at least one of the following of the second root node: a parent node, a sibling node or a child node, and the failure pointer of the first non-root node points to the corresponding second root node. The second non-root node can be a non-root node adjacent to the first root node, for example, a child node of the first root node, and the failure pointer of the second non-root node points to the first root node. The third non-root node corresponds to a pattern string, specifically, the last character of a pattern string, and the failure pointer of the third non-root node points to the main root node. The failure pointer of the fourth non-root node points to the node with the longest common suffix found along the failure path of the parent node. If the first root node or the second root node is encountered first during the search process, the search is stopped, and the failure pointer of the fourth non-root node points to the found first root node or second root node.
[0110] As shown in FIG. 6, the construction method can include the following steps:
[0111] S610: The computing node constructs a first dictionary tree according to the first information.
[0112] In the embodiment of the application, the first information includes at least one first pattern string, and each first pattern string includes at least one character. When S610 is implemented, the computing node can construct a pattern string dictionary according to the at least one pattern string, and generate the first dictionary tree after encoding processing of the input pattern string.
[0113] For example, the encoding processing of the pattern string can include a merging processing of continuous wildcards, such as merging two continuous * into one *. Or, the encoding processing of the pattern string can also include a merging processing of different pattern strings with a common prefix, such as for the string abc, abd and the string abcd, the string abc and abd both have the same prefix ab, then the string abc and abd can be merged, and the character c and the character d are respectively as the child nodes of the character ab, i.e., the character c and the character d correspond to sibling nodes. The string abc and the string abcd both have the same prefix abc, then the d in the string abcd is as the child node of the c in the string abc.
[0114] The first dictionary tree is a prefix tree, which is a multi-way tree structure composed of "path" and "node". In the process of implementing S610 to generate the first dictionary tree, starting from the first root node, according to each character in the at least one pattern string, the corresponding character path is created, which can record the characters in the pattern string by "path", and the number of passed characters and the number of ending characters by "node". For example, in FIG. 5, the tree nodes are represented by different circles, and the "path" is represented by a black solid arrow.
[0115] The first dictionary tree can include a plurality of nodes, each of which corresponds to a character in the at least one first pattern string. Wherein, if the at least one first pattern string includes at least one wildcard character, the plurality of nodes of the first dictionary tree includes the first root node, at least one second root node and a plurality of non-root nodes, the first root node is empty, one of the second root nodes corresponds to one wildcard character, and one of the non-root nodes corresponds to one non-wildcard character. If the at least one first pattern string does not include a wildcard character, the plurality of nodes of the first dictionary tree includes the first root node and a plurality of non-root nodes, the first root node is empty, and one of the non-root nodes corresponds to one non-wildcard character.
[0116] S620: Calculate the failure pointer of the node to construct the plurality of nodes to form the first fractal automaton.
[0117] In the embodiments of the present application, according to the differences of different nodes, the failure pointers of the plurality of nodes can be constructed according to the character types corresponding to the nodes according to the conditions ①-④ introduced above.
[0118] For example, the plurality of nodes include the first root node and at least one second root node, when implementing S620, the calculation node can first traverse the first root node and at least one second root node layer by layer, and initialize the parent node thereof as itself.
[0119] Then, the computing node can point the failure pointer of the non-root node adjacent to the first root node to the first root node, and point the failure pointer of the non-root node adjacent to each second root node to the corresponding second root node. The non-root node adjacent to the second root node includes the parent node, the sibling node and the child node of the second root node, denoted as a first non-root node, and the failure pointer of the first non-root node points to the corresponding second root node. The non-root node adjacent to the first root node includes the child node of the first root node, denoted as a second non-root node, and the failure pointer of the second non-root node points to the first root node.
[0120] Taking the fractal automaton shown in FIG. 5 as an example, a plurality of tree nodes are distributed in layers, q0 corresponds to the main root node, i.e., the first root node, the non-root node adjacent to the first root node includes each node at the lower layer of the main root node, i.e., the nodes corresponding to q1 and q2, and these nodes are also the child nodes of the main root node, and the failure pointer of the nodes corresponding to q1 and q2 points to the first root node. q6 and q14 correspond to the side root node, i.e., the second root node. The non-root node adjacent to the side root node includes the parent node of the side root node, the sibling node of the side root node, and the child node of the side root node. As shown in FIG. 5, the node corresponding to q3 is the parent node of the node corresponding to q6, the node corresponding to q7 is the sibling node of the node corresponding to q6, and the nodes corresponding to q10 and q11 are the child nodes of the node corresponding to q6, and these nodes, as the non-root node adjacent to the side root node, have their failure pointers pointing to the node corresponding to q6. The node corresponding to q9 is the parent node of the node corresponding to q14, and the node corresponding to q20 is the child node of the node corresponding to q14, and these nodes, as the non-root node adjacent to the side root node, have their failure pointers pointing to the node corresponding to q14.
[0121] Then, starting from the first root node, the main root and the side root are queued in turn, and dequeued in turn, the child nodes of the main root node and the side root node are traversed, the corresponding failure pointers are constructed, and then queued again, and the cycle continues until the queue is empty. A specific child node is denoted as a fourth non-root node, and the construction process of the failure pointer of the fourth non-root node is as follows: when the failure pointer of the non-root node is not empty, along the failure path of the parent node, it is found whether the character corresponding to a certain child node of the corresponding node is the same as the character corresponding to the current non-root node, i.e., whether it has the maximum common suffix, until a node meeting the condition is found, or the main root node or the side root node is encountered, and then the failure pointer is pointed to the node corresponding to the stop of the search. That is, the failure pointer of the fourth non-root node points to the node with the longest common suffix found along the failure path of the parent node, and if the first root node or the second root node is encountered first during the search, the search is stopped, and the failure pointer of the fourth non-root node is pointed to the found first root node or second root node.
[0122] Taking the fractal automaton shown in Fig. 5 as an example, for the node corresponding to q4, its failure pointer is searched along the failure path of its parent node q1, and the main root node is first encountered, so the failure pointer of the node corresponding to q4 points to the main root node. For the node corresponding to q5, its failure pointer is searched along the failure path of its parent node q2, and the child node q1 having the same character as the node corresponding to q5 is first encountered, i.e., the node having the maximum common suffix, so the failure pointer of the node corresponding to q5 points to the node q1. Similarly, for the node corresponding to q13, the failure path of its parent node q8 is searched, and the child node q1 having the same character as the node corresponding to q13 is first encountered, i.e., the node having the maximum common suffix, so the failure pointer of the node corresponding to q13 points to the node q1. For the node corresponding to q15, the failure path of its parent node q10 is searched, and the child node q11 having the same character as the node corresponding to q15 is first encountered, i.e., the node having the maximum common suffix, so the failure pointer of the node corresponding to q15 points to the node q11. For the node corresponding to q17, the failure path of its parent node q11 is searched, and the child node q1 having the same character as the node corresponding to q17 is first encountered, i.e., the node having the maximum common suffix, so the failure pointer of the node corresponding to q17 points to the node q1. For the node corresponding to q18, the failure path of its parent node q12 is searched, and the child node q16 having the same character as the node corresponding to q18 is first encountered, i.e., the node having the maximum common suffix, so the failure pointer of the node corresponding to q18 points to the node q16. For the node corresponding to q22, the failure path of its parent node q15 is searched, and the child node q16 having the same character as the node corresponding to q22 is first encountered, i.e., the node having the maximum common suffix, so the failure pointer of the node corresponding to q22 points to the node q16. For the node corresponding to q23, the failure path of its parent node q17 is searched, and the child node q4 having the same character as the node corresponding to q23 is first encountered, i.e., the node having the maximum common suffix, so the failure pointer of the node corresponding to q23 points to the node q4. For the node corresponding to q25, the failure path of its parent node q18 is searched, and the main root node q0 is first encountered, so the failure pointer of the node corresponding to q25 points to the node q0.
[0123] It should be understood that Fig. 6 is merely an example of the construction method of the fractal automaton of the embodiments of the present application, and is not any limitation. In a specific implementation, if the first information as the pattern string data source does not contain at least one first pattern string containing a wildcard character, the above-mentioned first fractal automaton degenerates into a traditional AC automaton, and the structure is shown in Fig. 2. When at least one first pattern string does not contain a wildcard character and there is only one pattern string, the above-mentioned first fractal automaton degenerates into a KMP algorithm, and the structure is shown in Fig. 1.
[0124] So far, the first fractal automaton is constructed. The computing node can compress the data of the first fractal automaton, and decompress the data of the first fractal automaton when the first fractal automaton is needed, and apply the decompressed first fractal automaton to natural language processing, information retrieval, and big data processing scenarios. The first fractal automaton supports fuzzy matching, and has higher real-time performance compared with the BM algorithm, the AC automaton, and the variant method. At the same time, the fractal automaton supports single pattern string, multi-pattern string, and wildcard pattern string matching, and has a wider application scenario.
[0125] The fractal automaton of the embodiment of the present application supports incremental updating. That is, when the pattern string data source changes, the Trie tree of the first fractal automaton does not need to be rebuilt, but the existing first fractal automaton can be incrementally updated to reduce performance overhead and meet the real-time requirement as much as possible.
[0126] In the embodiment of the present application, the change of the pattern string data source of the fractal automaton can include at least one of the addition, deletion, or update of the pattern string.
[0127] The following describes the incremental updating principle corresponding to different data source change scenarios.
[0128] Example (1): Addition / update of pattern string.
[0129] As shown in FIG. 7, the process of incrementally updating the first fractal automaton constructed in FIG. 6 includes the following steps:
[0130] S701: The computing node initializes the first fractal automaton.
[0131] In the embodiment of the present application, the initialization process can be the initial construction process described in FIG. 6, or can refer to the process of obtaining the data of the first fractal automaton from the storage medium and decompressing the first fractal automaton, which is not limited in the embodiment of the present application.
[0132] S702: The computing node obtains target information.
[0133] For example, in the case of adding a pattern string, the target information can be represented as second information, which includes a second added pattern string, the second pattern string includes at least one character, and the second pattern string is different from the at least one first pattern string.
[0134] Or for example, in the case of updating the mode string, the target information can include an updated fourth mode string, the fourth mode string is used to update the third mode string in the at least one first mode string, and the fourth mode string and the third mode string contain different characters, that is, the fourth mode string and the third mode string have at least one common character and at least one non-common character.
[0135] S703: The computing node determines whether the mode string corresponding to the target information exists in the first fractal automaton. If not, go to S704. If yes, end the update process.
[0136] S704: The computing node updates the first dictionary tree according to the mode string corresponding to the target information.
[0137] For example, in the case of adding a mode string, when S704 is implemented, one or more new nodes can be added in the first dictionary tree to obtain a second dictionary tree. If at least one character of the second mode string has a common prefix with any first mode string in the at least one first mode string, the number of new nodes is less than the number of characters contained in the second mode string, and the new nodes correspond to the non-common characters in the second mode string. If at least one character of the second mode string has no common prefix with the at least one first mode string, the number of new nodes is equal to the number of characters contained in the second mode string.
[0138] Or for example, in the case of updating the mode string, when S704 is implemented, according to the fourth mode string, the nodes corresponding to the at least one non-common character are updated in the first dictionary tree to obtain a third dictionary tree.
[0139] S705: The computing node completes the construction of the second dictionary tree / third dictionary tree.
[0140] S706: The computing node reconstructs the failure pointers of part of the nodes in the second dictionary tree / third dictionary tree to obtain a second fractal automaton or a third fractal automaton.
[0141] For example, in the case of adding a mode string, when S706 is implemented, the failure pointers of part of the nodes in the second dictionary tree can be reconstructed according to the relationship between the characters corresponding to the one or more new nodes and the characters corresponding to the other nodes in the second dictionary tree to form a second fractal automaton. In the case of adding a mode string, the part of the nodes mentioned above includes one or more new nodes and the original nodes associated with the new nodes. Whether it is a new node or an original node, the update of its failure pointer satisfies the conditions ①-④ introduced above, which will not be repeated here.
[0142] Or for example, in the case of the update of the pattern string, when implementing S706, the failure pointers of the nodes in the third dictionary tree can be restructured according to the relationship between the characters corresponding to the other nodes in the third dictionary tree and the at least one non-common character, to form the third fractal automaton. In the case of the update of the pattern string, the nodes mentioned include the nodes corresponding to the at least one updated non-common character, and the original nodes associated with the nodes corresponding to the characters. Whether it is the updated node or the original node, the update of the failure pointer thereof satisfies the conditions ①-④ described above, which will not be repeated here.
[0143] Example (2): deletion of the pattern string.
[0144] As shown in FIG. 8, the process of incrementally updating the first fractal automaton constructed in FIG. 6 includes the following steps:
[0145] S801: The computing node initializes the first fractal automaton.
[0146] In the embodiments of the present application, the initialization process can be the initial construction process described in FIG. 6, or can be a process of obtaining the data of the first fractal automaton from the storage medium and decompressing the first fractal automaton, which is not limited in the embodiments of the present application.
[0147] S802: The computing node determines to delete a fifth pattern string from the at least one first pattern string, and the fifth pattern string includes at least one character.
[0148] S803: The computing node deletes the nodes corresponding to the at least one character in the fifth pattern string from the last character of the at least one character in the fifth pattern string in the first dictionary tree in sequence, and after deleting each node, the following actions are performed: re-constructing the failure pointer for the target node, the target node including the node whose original failure pointer points to the deleted node.
[0149] S804: Determine whether the deletion of the fifth pattern string is completed. If not, return to S803 for traversal until the first non-root node of the first dictionary tree is reached, and stop the operation. If yes, end the incremental update process.
[0150] That is, when deleting the current node, the failure pointer of the node whose failure pointer points to the current node also needs to be re-constructed. At this time, the node with the maximum common suffix can be found along the failure path of the parent node according to the condition ④ described above, and if the first root node or the second root node is first encountered in the finding process, the finding is stopped, and the failure pointer is made to point to the first root node or the second root node found.
[0151] It should be understood that in the process of deleting the mode string shown in Fig. 8, if the fifth mode string to be deleted has a common prefix with other mode strings, the node corresponding to the common prefix does not need to be deleted.
[0152] For the convenience of understanding, the incremental updating process of the fractal automaton and the corresponding state diagram example are described below with respect to the relevant elements of the state machine shown in Fig. 4.
[0153] As shown in Fig. 9, the incremental updating process of the fractal automaton includes the following steps:
[0154] S901: The computing node initializes the first fractal automaton.
[0155] In the embodiments of the present application, the initialization process can be the initial construction process introduced in Fig. 6, or can refer to the process of obtaining the data of the first fractal automaton from the storage medium and decompressing to obtain the first fractal automaton, which is not limited in the embodiments of the present application.
[0156] S902: The computing node obtains the newly added / updated mode string, denoted as p=p1, p2, …, pn, and n is an integer greater than 1.
[0157] S903: The computing node judges whether the newly added / updated mode string already exists in the first fractal automaton. If yes, the updating process is ended. If no, S904 is entered.
[0158] S904: The computing node judges whether the automaton pi state transition exists. i is 1, 2, …, n. If no, S905 is entered. If yes, S907 is entered.
[0159] S905: The computing node adds the transition and state in the first dictionary tree.
[0160] S906: The computing node calculates and updates the failure pointer.
[0161] S907: The computing node judges whether the newly added / updated mode string is traversed to the end. If yes, S908 is entered. If no, S904 is returned.
[0162] S908: The computing node completes the automaton updating.
[0163] In one example, as shown in Fig. 10, if the left diagram represents the state diagram of the first fractal automaton, including the nodes corresponding to the following mode strings: TATAT, ATATAT, ATATATA, ACGATAT, and the specific nodes are as follows:
[0164] q1=A, q2=T, q3=AC, q4=AT, q5=TA, q6=ACG, q7=ATA, q8=TAT, q9=ACGA, q10=ATAT, q11=TATA, q12=ACGAT, q13=ATATA, q14=TATAT, q15=ACGATA, q16=ATATAT, q17=ACGATA, q18=ATATATA.
[0165] q12=ACGAT, q13=ATATA, q14=TATAT, q15=ACGATA, q16=ATATAT, q17=ACGATA, q18=ATATATA.
[0166] The failure pointers of the nodes satisfy at least one of conditions 1-4 introduced above, and are as follows when traversing from the main root node layer by layer: the failure pointers of the nodes corresponding to q1 and q2 point to the main root node; the failure pointers of the nodes corresponding to q14, q16, q17 and q18 point to the main root node; the failure pointer of the node corresponding to q3 points to the main root node; the failure pointer of the node corresponding to q4 points to the node corresponding to q2; the failure pointer of the node corresponding to q5 points to the node corresponding to q1; the failure pointer of the node corresponding to q6 points to the main root node; the failure pointer of the node corresponding to q7 points to the node corresponding to q5; the failure pointer of the node corresponding to q8 points to the node corresponding to q4; the failure pointer of the node corresponding to q9 points to the node corresponding to q1; the failure pointer of the node corresponding to q10 points to the node corresponding to q8; the failure pointer of the node corresponding to q11 points to the node corresponding to q7; the failure pointer of the node corresponding to q12 points to the node corresponding to q4; the failure pointer of the node corresponding to q13 points to the node corresponding to q11; and the failure pointer of the node corresponding to q15 points to the node corresponding to q7.
[0167] When a new pattern string CGATA is added to the first fractal automaton on the left side of FIG. 10, each substring (prefix) of the pattern string CGATA is represented as: p1=C, p2=CG, p3=CGA, p4=CGAT, p5=CGATA. According to the flow shown in FIG. 9, S903 is executed first to analyze whether the pattern string CGATA exists in the first fractal automaton.
[0168] If not, S904 is executed to analyze, starting from the main root node, layer by layer, whether the states and transitions corresponding to each substring of the new pattern string CGATA exist in the first fractal automaton.
[0169] For example, for p1=C, which does not exist in q1-q18 of the first fractal automaton, a transition state corresponding to p1=C is added in the first dictionary tree of the first fractal automaton, as shown on the right side of FIG. 10, the transition is from q0→q19, and the state is represented as q19=C. Then S906 is executed to add a failure pointer to the node corresponding to q19, and according to condition ①, the failure pointer points to the main root node. Meanwhile, since the node corresponding to q19 is newly added and is a child node of the main root node, there is a node corresponding to the same character as the newly added node corresponding to q19 in the nodes originally pointed to by the failure pointer of the parent node, for example, the node corresponding to q3, so the failure pointer of the node corresponding to q3 needs to be updated, and according to condition ④, the updated failure pointer of the node corresponding to q3 points to the node corresponding to q19.
[0170] If the updating process of the newly added pattern string is not completed, the updating process of S904-S907 is continued to continue adding the remaining transitions and states of the pattern string CGATA in the first dictionary tree of the first fractal automaton.
[0171] For example, for p2=CG, which does not exist in q1-q18 of the first fractal automaton, a transition state corresponding to p1=C is added in the first dictionary tree of the first fractal automaton, as shown on the right side of FIG. 10, the transition is from q19→q20, and the state is represented as q20=CG. Then S906 is executed to add a failure pointer to the node corresponding to q20. Since the node corresponding to q20 is a non-root node and the corresponding character is not the last character of the pattern string, condition ④ is searched along the failure path of the parent node to find the main root node, so the failure pointer of the node corresponding to q19 points to the main root node. Meanwhile, since the node corresponding to q20 is newly added and is a child node of the node corresponding to q19, it is necessary to re-calculate whether there is a node to be pointed to q20. After traversal analysis, it is found that there is a node corresponding to the same character as the newly added node corresponding to q20 in the nodes originally pointed to by the failure pointer of the parent node, for example, the node corresponding to q6, so the failure pointer of the node corresponding to q6 needs to be updated, and according to condition ④, the updated failure pointer of the node corresponding to q6 points to the node corresponding to q20.
[0172] Similarly, if the updating process of the newly added pattern string is not completed, the updating process of S904-S907 is continued to continue adding the remaining transitions and states of the pattern string CGATA in the first dictionary tree of the first fractal automaton, for example, q21-q23 in FIG. 10, which are represented as:
[0173] q21=CGA, q22=CGAT, q23=CGATA;
[0174] and updating the failure pointers of the related nodes until the updating of the automaton is completed to form the second fractal automaton.
[0175] It should be understood that FIG. 10 is only an example of adding one pattern string to introduce the incremental updating method of the embodiments of the present application, and does not limit the number of pattern strings to be updated. In a specific implementation, when the fractal automaton is incrementally updated, multiple pattern strings can also be added, and the addition process of the substring (prefix) of each pattern string can only be according to the flowchart shown in FIG. 9 and the example implementation of FIG. 10. Moreover, both the first fractal automaton to be updated and the added pattern string can include nodes or substrings corresponding to wildcard characters, and the embodiments of the present application do not limit this.
[0176] In another example, as shown in FIG. 11, if the left graph represents the state diagram of the first fractal automaton, it includes nodes corresponding to the following pattern strings: TATAT, ATATAT, ATATATA, ACGATAT, CGATA, and the specific implementation is as follows:
[0177] q1 = A, q2 = T, q3 = AC, q4 = AT, q5 = TA, q6 = ACG, q7 = ATA, q8 = TAT, q9 = ACGA, q10 = ATAT, q11 = TATA,
[0178] q12 = ACGAT, q13 = ATATA, q14 = TATAT, q15 = ACGATA, q16 = ATATAT, q17 = ACGATA, q18 = ATATATA, q19 = C, q20 = CG, q21 = CGA, q22 = CGAT, q23 = CGATA.
[0179] The failure pointers of the nodes satisfy at least one of conditions 1-4 introduced above, and are as follows when traversing from the main root node layer by layer: the failure pointers of the nodes corresponding to q1, q2 and q19 point to the main root node; the failure pointers of the nodes corresponding to q14, q16, q17, q18 and q23 point to the main root node; the failure pointer of the node corresponding to q3 points to the node corresponding to q19; the failure pointer of the node corresponding to q4 points to the node corresponding to q2; the failure pointer of the node corresponding to q5 points to the node corresponding to q1; the failure pointer of the node corresponding to q6 points to the node corresponding to q20; the failure pointer of the node corresponding to q7 points to the node corresponding to q5; the failure pointer of the node corresponding to q8 points to the node corresponding to q4; the failure pointer of the node corresponding to q9 points to the node corresponding to q21; the failure pointer of the node corresponding to q10 points to the node corresponding to q8; the failure pointer of the node corresponding to q11 points to the node corresponding to q7; the failure pointer of the node corresponding to q12 points to the node corresponding to q22; the failure pointer of the node corresponding to q13 points to the node corresponding to q11; the failure pointer of the node corresponding to q15 points to the node corresponding to q23; the failure pointer of the node corresponding to q20 points to the main root node; the failure pointer of the node corresponding to q21 points to the node corresponding to q1; and the failure pointer of the node corresponding to q22 points to the node corresponding to q4.
[0180] When it is required to modify the pattern string CGATA in the first fractal automaton on the left side of FIG. 11 into the pattern string CGACA, each substring (prefix) of the pattern string CGACA is represented as: p1=C, p2=CG, p3=CGA, p4=CGAC, and p5=CGACA. According to the flow shown in FIG. 9, S903 is executed first to analyze whether the pattern string CGACA exists in the first fractal automaton.
[0181] If not, S904 is executed to analyze, layer by layer starting from the main root node, whether each substring of the newly added pattern string CGACA corresponds to an existing state transition in the first fractal automaton.
[0182] For example, for p1=C, it exists in q19 of the first fractal automaton, and S907 is entered to analyze whether the traversal is completed. Since there are remaining substrings to be analyzed, S904 is returned to analyze whether p2=CG exists in the first fractal automaton.
[0183] For p2=CG, it exists in q20 of the first fractal automaton, and S907 is entered to analyze whether the traversal is completed. Since there are remaining substrings to be analyzed, S904 is returned to analyze whether p3=CGA exists in the first fractal automaton.
[0184] For p3=CGA, which exists in q21 of the first fractal automaton, it is analyzed in S907 whether the traversal ends. Since there are still remaining substrings to be analyzed, it is returned to S904 to analyze whether p4=CGAC exists in the first fractal automaton.
[0185] If no, it is entered into S905 to add a transition and a state, the transition is from q21→q22, and the state is denoted as q22=CGAC, that is, the substring CGAT corresponding to q22 is modified to CGAC. Meanwhile, since the character corresponding to q22 is updated, it is necessary to perform S906 to calculate and update the failure pointer. At this time, for the new q22, it is still a non-root node, and thus according to condition IV, the failure path of the parent node q21 is searched, and it is found that the node whose child node is the same as the node corresponding to q22 is pointed to by the failure pointer of the parent node, for example, the node corresponding to q3, and thus the failure pointer of q22 is pointed to the node corresponding to q3. Meanwhile, the node corresponding to q12, which is originally pointed to by the failure pointer, needs to re-construct the failure pointer. According to condition IV, after traversal analysis, it is found that q4 meets the requirement by searching the failure path of the parent node of q12, and thus the failure pointer of the node corresponding to q12 is updated to point to the node corresponding to q4.
[0186] Since there are still remaining substrings to be analyzed, it is returned to S904 to analyze whether p5=CGACA exists in the first fractal automaton.
[0187] If no, it is entered into S905 to add a transition and a state, the transition is from q22→q23, and the state is denoted as q23=CGACA. Meanwhile, it is necessary to perform S906 to calculate and update the failure pointer. At this time, for the new q23, it is a non-root node corresponding to the last character of the pattern string, and thus according to condition III, its failure pointer is pointed to the main root node. Meanwhile, the node corresponding to q15, which is originally pointed to by the failure pointer, needs to re-construct the failure pointer. According to condition IV, after traversal analysis, it is found that q7 meets the requirement by searching the failure path of the parent node of q15, and thus the failure pointer of the node corresponding to q15 is updated to point to the node corresponding to q7.
[0188] It should be understood that in FIG. 11, only one pattern string is modified as an example to introduce the incremental updating method of the embodiment of the present application, and the number of pattern strings to be modified is not limited. In a specific implementation, when the fractal automaton is incrementally updated, multiple pattern strings can also be modified, and the modification process of the substrings (prefixes) of each pattern string can only be performed according to the flowchart shown in FIG. 9 and the example implementation of FIG. 11. Moreover, both the first fractal automaton to be updated and the pattern string to be modified can include nodes or substrings corresponding to wildcard characters, which are not limited by the embodiment of the present application.
[0189] In another example, as shown in FIG. 12, if the left graph represents a state diagram of a first fractal automaton, including nodes corresponding to substrings (prefixes) of the following pattern strings: TATAT, ATATAT, ATATATA, ACGATAT, CGACA, and specifically as shown below:
[0190] q1 = A, q2 = T, q3 = AC, q4 = AT, q5 = TA, q6 = ACG, q7 = ATA, q8 = TAT, q9 = ACGA, q10 = ATAT, q11 = TATA,
[0191] q12 = ACGAT, q13 = ATATA, q14 = TATAT, q15 = ACGATA, q16 = ATATAT, q17 = ACGATA, q18 = ATATATA, q19 = C, q20 = CG, q21 = CGA, q22 = CGAC, q23 = CGACA.
[0192] The failure pointers of each node satisfy at least one of the conditions 1-4 introduced above, and are as follows, layer by layer from the main root node: the failure pointers of the nodes corresponding to q1, q2, and q19 point to the main root node; the failure pointers of the nodes corresponding to q14, q16, q17, q18, and q23 point to the main root node; the failure pointer of the node corresponding to q3 points to the node corresponding to q19; the failure pointer of the node corresponding to q4 points to the node corresponding to q2; the failure pointer of the node corresponding to q5 points to the node corresponding to q1; the failure pointer of the node corresponding to q6 points to the node corresponding to q20; the failure pointer of the node corresponding to q7 points to the node corresponding to q5; the failure pointer of the node corresponding to q8 points to the node corresponding to q4; the failure pointer of the node corresponding to q9 points to the node corresponding to q21; the failure pointer of the node corresponding to q10 points to the node corresponding to q8; the failure pointer of the node corresponding to q11 points to the node corresponding to q7; the failure pointer of the node corresponding to q12 points to the node corresponding to q4; the failure pointer of the node corresponding to q13 points to the node corresponding to q11; the failure pointer of the node corresponding to q15 points to the node corresponding to q7; the failure pointer of the node corresponding to q20 points to the main root node; the failure pointer of the node corresponding to q21 points to the node corresponding to q1; the failure pointer of the node corresponding to q22 points to the node corresponding to q3.
[0193] When a new pattern string ACG*GATA is added to the first fractal automaton on the left side of FIG. 12, each substring (prefix) of the pattern string ACG*GATA is represented as: p1 = A, p2 = AC, p3 = AC*, p4 = AC*G, p5 = AC*G A, p6 = AC*G A T, p7 = AC*G A TA. According to the flow shown in FIG. 9, S903 is executed first to analyze whether the first fractal automaton contains the pattern string ACG*GATA.
[0194] If no, then S904 is executed, and the analysis is performed layer by layer starting from the main root node, and it is judged in turn whether the state and transition corresponding to each substring of the new pattern string AC*GATA exist in the first fractal automaton.
[0195] For example, for p1=A, it exists in q1 of the first fractal automaton, then S907 is entered to analyze whether the analysis is completed. Since there is a remaining substring to be analyzed, S904 is returned to analyze whether p2=AC exists in the first fractal automaton.
[0196] For example, for p2=AC, it exists in q3 of the first fractal automaton, then S907 is entered to analyze whether the analysis is completed. Since there is a remaining substring to be analyzed, S904 is returned to analyze whether p3=AC* exists in the first fractal automaton.
[0197] If no, then S905 is entered to add the transition and state, the transition is from q3→q24, q24=A C*, and S906 is executed again to add the failure pointer corresponding to q24. Since the node corresponding to q24 is a side root node, its failure pointer points to itself according to condition ①. At the same time, since the nodes corresponding to q3 and q6 are respectively the parent node and the sibling node of the node corresponding to q24, condition ② is satisfied, and it is necessary to reconstruct the failure pointer of the nodes corresponding to q3 and q6. According to condition ②, the new failure pointer of the nodes corresponding to q3 and q6 points to the node corresponding to q24.
[0198] If the update process of the new pattern string is not completed, the update process of S904-S907 is continued.
[0199] For example, for p4=AC*G, it does not exist in q1-q24 of the first fractal automaton, then p4=AC*G is added in the first dictionary tree of the first fractal automaton, the transition is from q24→q25, and the state is represented as q25=AC*G. S906 is executed again to add the failure pointer corresponding to q25. Since the node corresponding to q25 is a child node of the side root node q24, the failure pointer of the node corresponding to q25 points to the node corresponding to q24 according to condition ②. At the same time, the failure pointer of the node corresponding to q6 is updated to point to the node corresponding to q25 according to condition ④.
[0200] If the update process of the new pattern string is not completed, the update process of S904-S907 is continued.
[0201] For example, for p5=AC*GA, which does not exist in q1-q25 of the first fractal automaton, a transition and a state corresponding to p5=AC*GA are added in the first dictionary tree of the first fractal automaton, the transition is from q25 to q26, and the state is denoted as q26=AC*GA. Then, S906 is executed to add a failure pointer to the node corresponding to q26. Since the node corresponding to q26 is a non-root node, according to condition IV, the first side root node q24 is found along the failure path of the parent node q25, and thus the failure pointer of the node corresponding to q26 points to the node corresponding to q24. Meanwhile, according to condition IV, the failure pointer of the node corresponding to q9 is updated to point to the node corresponding to q26 through traversal analysis.
[0202] If the updating process of the added pattern string is not completed, the updating process of S904-S907 is continued.
[0203] For example, for p6=AC*GAT, which does not exist in q1-q26 of the first fractal automaton, a transition and a state corresponding to p6=AC*GAT are added in the first dictionary tree of the first fractal automaton, the transition is from q26 to q27, and the state is denoted as q27=AC*GAT. Then, S906 is executed to add a failure pointer to the node corresponding to q27. Since the node corresponding to q27 is a non-root node, according to condition IV, the first side root node q24 is found along the failure path of the parent node q26, and thus the failure pointer of the node corresponding to q27 points to the node corresponding to q24. Meanwhile, according to condition IV, the failure pointer of the node corresponding to q12 is updated to point to the node corresponding to q27 through traversal analysis.
[0204] If the updating process of the added pattern string is not completed, the updating process of S904-S907 is continued.
[0205] For example, for p7=AC*GATA, which does not exist in q1-q27 of the first fractal automaton, a transition and a state corresponding to p7=AC*GATA are added in the first dictionary tree of the first fractal automaton, the transition is from q27 to q28, and the state is denoted as q28=AC*GATA. Then, S906 is executed to add a failure pointer to the node corresponding to q28. Since the node corresponding to q28 is the last character of the pattern string, according to condition III, the failure pointer of the node corresponding to q28 points to the main root node. Meanwhile, according to condition IV, the failure pointer of the node corresponding to q15 is updated to point to the node corresponding to q28 through traversal analysis.
[0206] It should be understood that only one mode string containing a wildcard character is added in FIG. 12 as an example to introduce the incremental updating method of the embodiment of the present application, and the number of added mode strings is not limited. In a specific implementation, when the fractal automaton is incrementally updated, a plurality of mode strings containing wildcard characters can be added, and the adding process of the substring (prefix) of each mode string can only be performed according to the flowchart shown in FIG. 9 and the example implementation of FIG. 12. Moreover, both the first fractal automaton to be updated and the mode string added can include nodes or substrings corresponding to non-wildcard characters, which are not limited by the embodiment of the present application.
[0207] In another example, when it is necessary to delete a mode string from the mode string data source of the first fractal automaton, the roles of the two fractal automata in the examples shown in FIG. 10 and FIG. 12 can be interchanged, and the second fractal automaton is updated to the first fractal automaton. In this updating process, first, it is determined to delete a fifth mode string from the at least one first mode string, the fifth mode string including at least one character; from the last character of the at least one character in the fifth mode string, nodes corresponding to the at least one character in the fifth mode string are sequentially deleted from the first dictionary tree, and after each node is deleted, the following actions are performed: re-constructing a failure pointer for a target node, the target node including a node to which the original failure pointer points to the deleted node; until the first non-root node of the first dictionary tree is reached, the operation is stopped.
[0208] For example, as shown in FIG. 10, the mode string CGATA in the second fractal automaton is deleted as the fifth mode string, then in the dictionary tree of the second fractal automaton, the node corresponding to q23 is deleted first, and the failure pointers of the related part of the nodes are updated. For example, according to condition IV, the failure pointer of the node q17 originally pointing to q23 is updated to point to the node corresponding to q7.
[0209] Then, the node corresponding to q22 is deleted, and the failure pointers of the related part of the nodes are updated. For example, according to condition IV, the failure pointer of the node q12 originally pointing to q22 is updated to point to the node corresponding to q4.
[0210] Then, the node corresponding to q21 is deleted, and the failure pointers of the related part of the nodes are updated. For example, according to condition IV, the failure pointer of the node q9 originally pointing to q21 is updated to point to the node corresponding to q1.
[0211] Then, the node corresponding to q20 is deleted, and the failure pointers of the related part of the nodes are updated. For example, according to condition IV, the failure pointer of the node q6 originally pointing to q20 is updated to point to the main root node.
[0212] Then, the node corresponding to q19 is deleted, and the failure pointers of the relevant partial nodes are updated. For example, the failure pointer of the node q3 originally pointing to q19 is updated to point to the main root node according to condition IV.
[0213] The first non-root node q0 is reached, and the operation is stopped.
[0214] Or, for example, as shown in Fig. 12, the pattern string AC*GATA in the second fractal automaton is deleted as the fifth pattern string, then in the dictionary tree of the second fractal automaton, the node corresponding to q28 is deleted, and the failure pointers of the relevant partial nodes are updated. For example, the failure pointer of the node q15 originally pointing to q28 is updated to point to the side root node corresponding to q24 according to condition IV.
[0215] Then, the node corresponding to q27 is deleted, and the failure pointers of the relevant partial nodes are updated. For example, the failure pointer of the node q12 originally pointing to q27 is updated to point to the node corresponding to q24 according to condition IV.
[0216] Then, the node corresponding to q26 is deleted, and the failure pointers of the relevant partial nodes are updated. For example, the failure pointer of the node q9 originally pointing to q26 is updated to point to the node corresponding to q24 according to condition IV.
[0217] Then, the node corresponding to q25 is deleted, and the failure pointers of the relevant partial nodes are updated. For example, the failure pointer of the node q6 originally pointing to q25 is updated to point to the node corresponding to q24 according to condition IV.
[0218] Then, the node corresponding to q24 is deleted, and the failure pointers of the relevant partial nodes are updated. For example, the failure pointers of the nodes q3, q6, q9, q12, and q15 originally pointing to q24 are reconstructed according to condition IV. After traversal analysis, the failure pointer of the node corresponding to q3 is updated to point to the node corresponding to q19, the failure pointer of the node corresponding to q6 is updated to point to the node corresponding to q20, the failure pointer of the node corresponding to q9 is updated to point to the node corresponding to q21, the failure pointer of the node corresponding to q12 is updated to point to the node corresponding to q4, and the failure pointer of the node corresponding to q15 is updated to point to the node corresponding to q5.
[0219] For the nodes corresponding to q3 and q1, since they are common prefixes with other pattern strings, the two nodes are not deleted. The first non-root node q0 is reached, and the operation is stopped.
[0220] Thus, the incremental updating method of the fractal automaton of the embodiments of the present application has been described in the elements of the state machine in combination with FIG. 9-FIG. 12. Through the updating method, even if the pattern string data source of the fractal automaton changes, the Trie tree of the fractal automaton does not need to be pushed to be reconstructed, but the existing fractal automaton can be incrementally updated, so as to reduce the performance overhead and meet the real-time requirement as much as possible. Meanwhile, compared with the updating method through the suffix automaton, the calculation process is simple and no additional storage space is needed.
[0221] In addition, the embodiments of the present application also provide a special compression storage scheme for the fractal automaton, which will be described in detail below in combination with the drawings.
[0222] In the embodiments of the present application, as an example, the Huffman coding method can be used to compress and store the related data of the fractal automaton and to perform decompression processing.
[0223] The first data sequence can be obtained by serializing and encoding the data associated with each node of the first dictionary tree of the first fractal automaton according to the parent-child dependency relationship of each node of the first dictionary tree and the frequency of each node of the first dictionary tree being pointed to by the failure pointer of other nodes, starting from the first root node; the first compressed file can be obtained by compressing the first data sequence; and the first compressed file can be saved in the persistent storage medium of the computing node.
[0224] As shown in FIG. 13, the compression storage method of the fractal automaton can include the following steps:
[0225] S1310: The computing node traverses each node of the dictionary tree of the fractal automaton by level, serializes and sorts the related data of the node.
[0226] For example, taking the serialization and sorting of the related data of each node of the fractal automaton shown in FIG. 14 as an example, the content shown in Table 2 below can be obtained:
[0227] Table 2
[0228] In Table 2, the first column indicates the number assigned to each node of the dictionary tree of the fractal automaton by level, which can correspond to the serial number of the node identifier in the third column. The second column is the “Slast” column, which indicates whether the node of each row is the last node of the level, 0 indicating no and 1 indicating yes. The third column indicates the node identifier of each node. The fourth column indicates the parent node of each node. The fifth column indicates the node pointed to by the failure pointer of each node. The sixth column indicates whether each node itself is a parent node, 1 indicating yes and 0 indicating no.
[0229] S1320: The computing node analyzes the parent-child dependency relationship of each node and the frequency of each node being pointed to by the failure pointer of other nodes according to the serialized data (for example, shown in Table 2).
[0230] For example, the analysis of the content shown in Table 2 obtains the content shown in Table 3:
[0231] Table 3
[0232] In Table 3, the first list represents all the node identifiers pointed to by the failure pointer. The second list represents the frequency of being pointed to by the failure pointer.
[0233] S1330: The computing node sequentially serializes the data associated with each node of the first dictionary tree of the first fractal automaton according to the parent-child dependency relationship of each node of the first dictionary tree and the frequency of each node of the first dictionary tree being pointed to by the failure pointer of other nodes, to obtain a first data sequence.
[0234] In the embodiments of the present application, as an example, the encoding method can be the Huffman method, and the encoding information is in binary. For example, based on the content shown in Table 3, the encoding process shown in the right part of FIG. 14 is performed, starting with the total frequency (the total number of failure pointers), and analyzing from large to small according to the frequency of being pointed to by the failure pointer. Each time a node pointed to by the failure pointer is experienced, the frequency of the node being pointed to is subtracted from the total frequency until the analysis is completed. The combination of the numbers represented by the arrows in the analysis process is obtained. The encoding information of the failure pointer pointing to the corresponding node is obtained. For example, the encoding information of the failure pointer pointing to q0 is 0, the encoding information of the failure pointer pointing to q1 is 100, the encoding information of the failure pointer pointing to q2 is 11, and the encoding information of the failure pointer pointing to q3 is 101.
[0235] Based on Table 2 and the encoding information corresponding to each failure pointer, the first data sequence shown in Table 4 can be obtained:
[0236] Table 4
[0237] In Table 4, the second column represents node identifiers. The first column represents node types, and each row of data in this column includes two digits, which are a combination of the second column and the sixth column in Table 1. The first digit indicates whether the corresponding node is the last node in the layer, and the second digit indicates whether the corresponding node itself is a parent node. Each row of data in the third column represents encoded information of a failure pointer pointing to the node in the row. For example, the failure pointers of nodes q0-q4, q6, and q10 all point to q0, and the corresponding encoded information is 0. The failure pointer of node q5 points to q1, and the corresponding encoded information is 100. The failure pointers of nodes q7, q9, and q11 point to q2, and the corresponding encoded information is 11. The failure pointer of node q8 points to q3, and the corresponding encoded information is 101.
[0238] In S1340, the computing node compresses the first data sequence to obtain a first compressed file. For example, Table 4 is compressed to obtain a corresponding compressed file.
[0239] In S1350, the computing node saves the first compressed file in a persistent storage medium of the computing node. For example, the corresponding compressed file of Table 4 is stored.
[0240] When the fractal automaton needs to be used, a decompression operation is performed, for example, the first compressed file is read from the persistent storage medium of the computing node; the first compressed file is decompressed to obtain the first data sequence; the first dictionary tree is constructed according to the first data sequence, and failure pointers of multiple nodes of the first dictionary tree are constructed to form the first fractal automaton.
[0241] This process is a reverse process of the process shown in FIG. 13, that is, after the content shown in Table 3 is obtained by decompression, the third column of Table 3 is decoded in reverse from back to front according to the analysis process shown on the right side of FIG. 14 to obtain the content shown in Table 2, and then Table 1 is obtained by reverse processing according to the first column and the second column of Table 2 and Table 3. Based on the content shown in Table 1, the dictionary tree shown on the left side of FIG. 14 is constructed, and failure pointers are added (or referred to as filled, constructed, etc.) to each node of the dictionary tree to obtain a corresponding fractal automaton.
[0242] In addition, after the fractal automaton is compressed, the compressed file of the fractal automaton can still be used to implement the pattern matching method of the embodiments of the present application.
[0243] For example, the search process based on the automaton compressed file can include the following steps:
[0244] In S1, after a substring to be searched is input, a main root node (a first non-root node) is traversed, and corresponding child nodes are reversely searched according to a serialization process of layer-by-layer traversal, that is, a dictionary search is implemented.
[0245] S2: When S1 fails, find the encoding information of the corresponding node along the failure pointer, obtain the index of the corresponding node after decoding, calculate the sequence position corresponding to the index, and then repeat S1.
[0246] S3: Knowing the end of input, record the matched pattern string.
[0247] Thus, by the method introduced in FIG. 13-FIG. 14, the special compression storage of the fractal automaton of the embodiments of the present application can be realized. The compression storage method will not distort the data, and at the same time, can greatly reduce the storage space occupied by the fractal automaton, reduce the maintenance difficulty and cost of the fractal automaton. At the same time, in some application scenarios, the decompression processing can also not be performed, but the pattern matching is performed in the compressed file.
[0248] In one example, the fractal automaton and its construction method, incremental updating method, compression storage method and decompression method introduced above can be applied in the vehicle field to perform vehicle log inspection to find problems existing in the vehicle in time.
[0249] As shown in FIG. 15, different users can report the rule file for vehicle log analysis that the user expects to use on the server to the server on the terminal equipment (referred to as front end) used by the user, and dynamically update the rule file. At the same time, the user can also manage the rule file, including but not limited to adding, modifying, querying, deleting or exporting, etc. Optionally, the server can also support regular matching rules, SQL database rules, Python custom rules, or other specific rules in other fields, which are not limited by the embodiments of the present application.
[0250] When a user needs to trigger an inspection task, the user can issue an inspection task to the server on the operation and maintenance interface displayed on the terminal equipment used by the user, and send an operation and maintenance instruction to the server, including but not limited to creating, modifying, querying, deleting, canceling, activating, deactivating, etc. In addition, the user can also manage / view the analysis report and result through the operation and maintenance interface. Among them, the terminal equipment of the user can display a web interface or other types of operation and maintenance interfaces to issue an inspection task, or view an analysis report and result. The inspection task can be at least one of a vehicle inspection task, a daily inspection task, etc. The embodiments of the present application are not limited in this regard.
[0251] The server can provide the user with an inspection task management function, an inspection service management function and an inspection analysis result management function. The inspection task management function can be used to manage the inspection task triggered by the user. The inspection service management function can be used to perform the inspection task in response to the received inspection task and management. The inspection analysis result management function can be used to analyze and manage the inspection result, and present the corresponding analysis report and result to the user.
[0252] For the inspection task, after the server receives the inspection task from the terminal device, the original log file can be obtained, and the corresponding rule file is matched by the analysis engine for retrieval and correlation analysis.
[0253] The original log file can be reported to the server and stored in the server after obtaining the vehicle user authorization in advance, or stored in the vehicle enterprise server and obtained from the vehicle enterprise server after obtaining the authorization, and the source of the log file is not limited in the embodiment of the application.
[0254] The analysis engine can construct (or incrementally update) a fractal automaton according to the rule file, and use the fractal automaton to search the original log file. After correlation analysis, the output result, i.e. the report, is fed back to the terminal device of the user. The terminal device can also perform export and other operations on the result.
[0255] The rule file is the pattern string data source described above. The analysis engine can convert the rule file into a corresponding pattern string according to a preset translator (or other tool). If the server is a computing node as described above, and the fractal automaton constructed historically is stored in the storage medium accessible by the server, then the fractal automaton can be incrementally updated according to the pattern string corresponding to the rule file and the incremental updating method described above. If the fractal automaton is not stored in the storage medium accessible by the server, then the fractal automaton required can be constructed according to the pattern string corresponding to the rule file and the construction method described above. For details, refer to the related description above, which will not be repeated here.
[0256] Thus, through the fractal automaton described above, in the retrieval scene of whole vehicle inspection and the like containing a large number of rules, for a real-time system with thousands of rule sets and frequent updates, the automaton can be dynamically and efficiently updated, rules can be added, deleted or modified at a relatively small cost, which can better meet the needs of real-time systems. At the same time, the special compression storage method for automaton described above can effectively improve the reliability and performance of the system, and the compressed automaton can also effectively reduce the disk space occupation. Directly querying on the compressed automaton sequence can also achieve a better balance in time and space.
[0257] The embodiment of the application also provides a communication device for executing the construction method, or the incremental updating method, or the compression method, or the decompression method, or the pattern matching method executed by the computing node in the above method embodiments. For related features, refer to the above method embodiments, which will not be repeated here.
[0258] When the communication apparatus performs the constructing method performed by the computing node, as shown in FIG. 16, the communication apparatus 1600 can include: a processing unit 1601 configured to construct a first dictionary tree according to first information, wherein the first information includes at least one first pattern string, each first pattern string includes at least one character, the first dictionary tree includes a plurality of nodes, if the at least one first pattern string includes at least one wildcard character, the plurality of nodes include a first root node, at least one second root node, and a plurality of non-root nodes, the first root node is empty, one of the second root nodes corresponds to one wildcard character, and the non-root nodes correspond to non-wildcard characters; and construct failure pointers of the plurality of nodes to form a first fractal automaton, wherein the failure pointer of the first root node or the at least one second root node points to itself, the plurality of non-root nodes include the first non-root node, the first non-root node includes at least one of the following: a parent node, a sibling node, or a child node of any one of the at least one second root node, and the failure pointer of the first non-root node points to the second root node.
[0259] When the communication apparatus performs the updating method performed by the computing node, as shown in FIG. 17, the communication apparatus 1700 can include: an obtaining unit 1701 configured to obtain a first automaton, wherein the first automaton is constructed according to first information, the first information includes at least one first pattern string, each first pattern string includes at least one character, a first dictionary tree of the first automaton includes a plurality of nodes, if the at least one first pattern string includes at least one wildcard character, the plurality of nodes include a first root node, at least one second root node, and a plurality of non-root nodes, the first root node is empty, one of the second root nodes corresponds to one wildcard character, and the non-root nodes correspond to non-wildcard characters; and obtain second information, the second information includes a newly added second pattern string, the second pattern string includes at least one character, and the second pattern string is different from the at least one first pattern string; and a processing unit 1702 configured to add one or more newly added nodes in the first dictionary tree to obtain a second dictionary tree, wherein if at least one character of the second pattern string has a common prefix with any one of the at least one first pattern string, the number of newly added nodes is less than the number of characters included in the second pattern string, and the newly added nodes correspond to non-common characters in the second pattern string; and re-construct failure pointers of part of the nodes in the second dictionary tree according to relationships between characters corresponding to the one or more newly added nodes and characters corresponding to other nodes in the second dictionary tree to form a second automaton.
[0260] When the communication device performs the compression method performed by the computing node, as shown in FIG. 18, the communication device 1800 can include: an acquisition unit 1801 configured to acquire a first dictionary tree of a first automaton and failure pointers of a plurality of nodes of the first dictionary tree, wherein the first automaton is constructed according to first information, the first information includes at least one first pattern string, each first pattern string includes at least one character, the first dictionary tree includes a plurality of nodes, if the at least one first pattern string includes at least one wildcard character, the plurality of nodes include a first root node, at least one second root node and a plurality of non-root nodes, the first root node is empty, one second root node corresponds to one wildcard character, and the non-root node corresponds to one non-wildcard character; a processing unit 1802 configured to traverse from the first root node of the first automaton, sequentially serialize and encode data associated with each node of the first dictionary tree according to the parent-child dependency relationship of each node of the first dictionary tree and the frequency of each node of the first dictionary tree being pointed to by the failure pointer of another node, to obtain a first data sequence; compressing the first data sequence to obtain a first compressed file; and a storage unit 1803 configured to save the first compressed file in a persistent storage medium of the computing node.
[0261] It should be understood that the division of units in the above apparatus is only a logical function division, and all or part of them can be integrated into a physical entity or physically separated when actually implemented. In addition, the units in the apparatus can be implemented in the form of processor calling software; for example, the apparatus includes a processor, the processor is connected with a memory, the memory stores instructions, and the processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of the units of the apparatus, wherein the processor is, for example, a general processor such as a central processing unit (CPU) or a microprocessor, and the memory is a memory in the apparatus or a memory outside the apparatus. Alternatively, the units in the apparatus can be implemented in the form of hardware circuit, and the functions of part or all of the units can be realized by the design of the hardware circuit, which can be understood as one or more processors; for example, in one implementation, the hardware circuit is an application-specific integrated circuit (ASIC), and the functions of part or all of the units are realized by the design of the logical relationship of elements in the circuit; for example, in another implementation, the hardware circuit is realized by a programmable logic device (PLD), and a field programmable gate array (FPGA) is taken as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by a configuration file, so as to realize the functions of part or all of the units. All units of the above apparatus can be realized in the form of processor calling software, or all units can be realized in the form of hardware circuit, or part of the units can be realized in the form of processor calling software, and the remaining part can be realized in the form of hardware circuit.
[0262] In embodiments of the present application, the processor is a circuit with signal processing capability. In one implementation, the processor can be a circuit with instruction reading and running capability, such as a CPU, a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), a digital signal processor (DSP), etc. In another implementation, the processor can implement certain functions through a logical relationship of hardware circuit, which is fixed or reconfigurable, such as an ASIC or a PLD implemented hardware circuit, such as an FPGA. In a reconfigurable hardware circuit, the processor loads a configuration document to implement hardware circuit configuration. It can be understood that the processor loads instructions to implement the functions of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc.
[0263] It can be seen that each unit in the above apparatus can be one or more processors (or processing circuits) configured to implement the above methods, such as a CPU, a GPU, an NPU, a TPU, a DPU, a microprocessor, a DSP, an ASIC, an FPGA, or a combination of at least two of these processor forms.
[0264] In addition, each unit in the above apparatus can be integrated together or can be independently implemented. In one implementation, these units are integrated together to form a system-on-a-chip (SOC). The SOC can include at least one processor for implementing any of the above methods or functions of the units of the apparatus. The at least one processor can be of different types, such as a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.
[0265] In a simple embodiment, those skilled in the art can conceive that the communication apparatus in the above embodiments can all adopt the form shown in FIG. 19.
[0266] As shown in FIG. 19, the apparatus 1900 includes at least one processor 1910 and a communication interface 1930. In an optional design, it can also include a memory 1920.
[0267] The embodiments of the present application do not limit the specific connection medium between the processor 1910 and the memory 1920.
[0268] In the apparatus as shown in FIG. 19, the processor 1910 can perform data transmission through the communication interface 1930 when communicating with other devices.
[0269] When the communication apparatus adopts the form shown in FIG. 19, the processor 1910 in FIG. 19 can invoke the computer-executable instructions stored in the memory 1920, so that the apparatus 1900 can perform any of the above method embodiments.
[0270] The embodiments of the present application also relate to a chip system, which includes a processor for invoking a computer program or computer instructions stored in a memory, so that the processor performs the method of any of the above embodiments.
[0271] In a possible implementation, the processor can be coupled with the memory through an interface.
[0272] In a possible implementation, the chip system can also directly include the memory, which stores the computer program or computer instructions.
[0273] By way of example, the memory can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example and not limitation, many forms of RAM can be used, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).
[0274] The embodiments of the present application also relate to a processor configured to invoke a computer program or computer instructions stored in a memory to cause the processor to perform the method of any of the above embodiments.
[0275] For example, in the embodiments of the present application, the processor is an integrated circuit chip with processing capability of signals. For example, the processor can be an FPGA, a general-purpose processor, a DSP, an ASIC, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, a system on chip (SoC), a CPU, a network processor (NP), a micro controller unit (MCU), a PLD, or other integrated circuits, which can realize or execute the methods, steps, and logical block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the methods disclosed in combination with the embodiments of the present application can be directly embodied as hardware code of the processor, or a combination of hardware and software modules in the processor. The software module can be located in a storage medium such as random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable read-only memory (PROM), an electrically programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a register, or a storage device. The storage medium is located in the storage, and the processor reads information in the storage medium and combines the hardware to complete the steps of the above methods.
[0276] It should be understood that the embodiments of the present application can be provided as a method, a system, or a computer program product.
[0277] In one possible implementation, the embodiments of the present application provide a computer readable storage medium storing program codes, which, when executed on a computer, cause the computer to perform the above method embodiments.
[0278] In one possible implementation, the embodiments of the present application provide a computer program product, which, when executed on a computer, cause the computer to perform the above method embodiments.
[0279] Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer usable program code.
[0280] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart or flowsheets and / or block or blocks of the block diagrams.
[0281] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flowsheets and / or block or blocks of the block diagrams.
[0282] Obviously, persons having ordinary skill in the art can make various modifications and variations to the embodiments of the present application without departing from the scope of the present application. Thus, it is intended that the present application cover modifications and variations of this application provided they come within the scope of the appended claims and their equivalents. In the various embodiments of the present application, the terms and / or descriptions of the various embodiments are consistent and can be referenced mutually if not specifically stated, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.
Claims
1. A method for managing automatic machines, characterized in that, Applied to compute nodes, the method includes: A first trie is constructed based on the first information, wherein the first information includes at least one first pattern string, each first pattern string includes at least one character, the first trie includes multiple nodes, if the at least one first pattern string includes at least one wildcard character, the multiple nodes include a first root node, at least one second node and multiple non-root nodes, the first root node is empty, one second node corresponds to one wildcard character, and the non-root node corresponds to one non-wildcard character. Construct failure pointers for the plurality of nodes to form a first automaton, wherein the failure pointer of the first root node or the at least one second root node points to itself, the plurality of non-root nodes include the first non-root node, the first non-root node includes at least one of the following of the at least one second root node: parent node, sibling node or child node, and the failure pointer of the first non-root node points to the second root node.
2. The method according to claim 1, characterized in that, The plurality of non-root nodes further includes at least one of the following: a second non-root node, a third non-root node, or a fourth non-root node, wherein the failure pointers of different types of non-root nodes satisfy at least one of the following: The second non-root node includes the child nodes of the first root node, and the failure pointer of the second non-root node points to the first root node; The third non-root node corresponds to a first pattern string, and the failure pointer of the third non-root node points to the first root node; The failure pointer of the fourth non-root node points to the node with the longest common suffix found by searching upwards along the failure pointer of the parent node. If the first root node or the second root node is encountered first during the search process, the search stops and the failure pointer of the fourth non-root node points to the first root node or the second root node found.
3. The method according to claim 1 or 2, characterized in that, The method further includes: Obtain second information, the second information including a newly added second pattern string, the second pattern string including at least one character, the second pattern string being different from the at least one first pattern string; Add one or more new nodes to the first trie to obtain a second trie, wherein if at least one character of the second pattern string has a common prefix with any of the at least one first pattern string, the number of new nodes is less than the number of characters contained in the second pattern string, and the new nodes correspond to non-common characters in the second pattern string. Based on the relationship between the characters corresponding to one or more newly added nodes and the characters corresponding to other nodes in the second trie, the failure pointers of some nodes in the second trie are reconstructed to form a second automaton.
4. The method according to claim 1 or 2, characterized in that, The method further includes: Determine to change the third pattern string in at least one first pattern string to a fourth pattern string, wherein the third pattern string and the fourth pattern string have at least one common character and at least one non-common character; Based on the fourth pattern string, in the first trie, update the node corresponding to the at least one non-common character to obtain the third trie; Based on the relationship between the at least one non-public character and the characters corresponding to other nodes in the third trie, the failure pointers of some nodes in the third trie are reconstructed to form a third automaton.
5. The method according to any one of claims 1-4, characterized in that, The method further includes: Determine to delete a fifth pattern string from the at least one first pattern string, the fifth pattern string comprising at least one character; Starting from the last character of at least one character in the fifth pattern string, nodes corresponding to at least one character in the fifth pattern string are sequentially deleted from the first trie. After each node is deleted, the following actions are performed: a failure pointer is reconstructed for the target node, the target node including the node whose original failure pointer pointed to the deleted node; the operation stops when the first non-root node of the first trie is reached.
6. The method according to any one of claims 1-5, characterized in that, The method further includes: Starting from the first root node, the data associated with each node of the first trie of the first automaton is sequentially serialized and encoded according to the parent-child dependency relationship of each node of the first trie and the frequency of each node of the first trie being pointed to by the failure pointers of other nodes, to obtain the first data sequence. The first data sequence is compressed to obtain a first compressed file; The first compressed file is saved in the persistent storage medium of the computing node.
7. The method according to any one of claims 1-6, characterized in that, The method further includes: Read the first compressed file from the persistent storage medium of the computing node; Decompress the first compressed file to obtain the first data sequence; The first trie is constructed based on the first data sequence, and failure pointers for multiple nodes of the first trie are constructed to form the first automaton.
8. The method according to any one of claims 1-7, characterized in that, Any of the at least one first pattern string can be implemented as any of the following: Single-pattern string; multi-pattern string; pattern string containing wildcard characters.
9. A method for managing automatic machines, characterized in that, Applied to compute nodes, the method includes: Obtain a first automaton, wherein the first automaton is constructed based on first information, the first information including at least one first pattern string, each first pattern string including at least one character, the first trie of the first automaton including multiple nodes, if the at least one first pattern string includes at least one wildcard character, the multiple nodes include a first root node, at least one second node and multiple non-root nodes, the first root node is empty, one second node corresponds to one wildcard character, and the non-root node corresponds to one non-wildcard character; Obtain second information, the second information including a newly added second pattern string, the second pattern string including at least one character, the second pattern string being different from the at least one first pattern string; Add one or more new nodes to the first trie to obtain a second trie, wherein if at least one character of the second pattern string has a common prefix with any of the at least one first pattern string, the number of new nodes is less than the number of characters contained in the second pattern string, and the new nodes correspond to non-common characters in the second pattern string. Based on the relationship between the characters corresponding to one or more newly added nodes and the characters corresponding to other nodes in the second trie, the failure pointers of some nodes in the second trie are reconstructed to form a second automaton.
10. The method according to claim 9, characterized in that, Multiple nodes of the first or second trie satisfy at least one of the following conditions: The failure pointer of the first root node or the at least one second root node points to itself; The plurality of non-root nodes further includes at least one of the following: a first non-root node, a second non-root node, a third non-root node, or a fourth non-root node, wherein, The first non-root node includes at least one of the following of any of the at least one second root nodes: a parent node, a sibling node, or a child node, and the failure pointer of the first non-root node points to the second root node; The second non-root node includes the child nodes of the first root node, and the failure pointer of the second non-root node points to the first root node; The third non-root node corresponds to a first pattern string, and the failure pointer of the third non-root node points to the first root node; The failure pointer of the fourth non-root node points to the node with the longest common suffix found by searching upwards along the failure pointer of the parent node. If the first root node or the second root node is encountered first during the search process, the search stops and the failure pointer of the fourth non-root node points to the first root node or the second root node found.
11. The method according to claim 9 or 10, characterized in that, The method further includes: Determine to delete a fifth pattern string from the at least one first pattern string, the fifth pattern string comprising at least one character; Starting from the last character of at least one character in the fifth pattern string, nodes corresponding to at least one character in the fifth pattern string are sequentially deleted from the first trie. After each node is deleted, the following actions are performed: a failure pointer is reconstructed for the target node, the target node including the node whose original failure pointer pointed to the deleted node; the operation stops when the first non-root node of the first trie is reached.
12. The method according to any one of claims 9-11, characterized in that, The method further includes: Starting from the first root node, the data associated with each node of the first trie of the first automaton is sequentially serialized and encoded according to the parent-child dependency relationship of each node of the first trie and the frequency of each node of the first trie being pointed to by the failure pointers of other nodes, to obtain the first data sequence. The first data sequence is compressed to obtain a first compressed file; The first compressed file is saved in the persistent storage medium of the computing node.
13. The method according to any one of claims 9-12, characterized in that, The method further includes: Read the first compressed file from the persistent storage medium of the computing node; Decompress the first compressed file to obtain the first data sequence; The first trie is constructed based on the first data sequence, and failure pointers for multiple nodes of the first trie are constructed to form the first automaton.
14. The method according to any one of claims 9-13, characterized in that, Any of the at least one first pattern string can be implemented as any of the following: Single-pattern string; multi-pattern string; pattern string containing wildcard characters.
15. A method for managing an automatic machine, characterized in that, Applied to compute nodes, the method includes: Obtain the first trie of the first automaton and the failure pointers of multiple nodes of the first trie. The first automaton is constructed based on first information, which includes at least one first pattern string. Each first pattern string includes at least one character. The first trie includes multiple nodes. If the at least one first pattern string includes at least one wildcard character, the multiple nodes include a first root node, at least one second root node, and multiple non-root nodes. The first root node is empty. One second root node corresponds to one wildcard character, and each non-root node corresponds to one non-wildcard character. Starting from the first root node of the first automaton, the data associated with each node of the first trie is sequentially serialized and encoded according to the parent-child dependency relationship of each node of the first trie and the frequency of each node of the first trie being pointed to by the failure pointers of other nodes, to obtain the first data sequence. The first data sequence is compressed to obtain a first compressed file; The first compressed file is saved in the persistent storage medium of the computing node.
16. The method according to claim 15, characterized in that, Multiple nodes of the first trie satisfy at least one of the following conditions: The failure pointer of the first root node or the at least one second root node points to itself; The plurality of non-root nodes further includes at least one of the following: a first non-root node, a second non-root node, a third non-root node, or a fourth non-root node, wherein, The first non-root node includes at least one of the following of any of the at least one second root nodes: a parent node, a sibling node, or a child node, and the failure pointer of the first non-root node points to the second root node; The second non-root node includes the child nodes of the first root node, and the failure pointer of the second non-root node points to the first root node; The third non-root node corresponds to a first pattern string, and the failure pointer of the third non-root node points to the first root node; The failure pointer of the fourth non-root node points to the node with the longest common suffix found by searching upwards along the failure pointer of the parent node. If the first root node or the second root node is encountered first during the search process, the search stops and the failure pointer of the fourth non-root node points to the first root node or the second root node found.
17. The method according to claim 15 or 16, characterized in that, The method further includes: Read the first compressed file from the persistent storage medium of the computing node; Decompress the first compressed file to obtain the first data sequence; The first trie is constructed based on the first data sequence, and failure pointers for multiple nodes of the first trie are constructed to form the first automaton.
18. The method according to any one of claims 15-17, characterized in that, Any of the at least one first pattern string can be implemented as any of the following: Single-pattern string; multi-pattern string; pattern string containing wildcard characters.
19. A communication device, characterized in that, It includes at least one processor and an interface circuit, the interface circuit being used to provide data or code instructions to the at least one processor, the at least one processor being used to implement the method as described in any one of claims 1-8, or the method as described in any one of claims 9-14, or the method as described in any one of claims 15-18 through logic circuits or executing code instructions.
20. A computer-readable storage medium, characterized in that, The computer-readable medium stores program code that, when executed on a computer, causes the computer to perform the method as described in any one of claims 1-8, or the method as described in any one of claims 9-14, or the method as described in any one of claims 15-18.
21. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1-8, or the method as described in any one of claims 9-14, or the method as described in any one of claims 15-18.
Citation Information
Patent Citations
Chinese AC (Aho-Corasick) automaton working method based on keyword dictionary tree structure
CN105260354A
Character string multimode fuzzy matching method based on AC automaton
CN112100361A
Character processing method and device, computer equipment and storage medium
CN113934892A
Computer implemented system and method for high performance visual tracking
US10757369B1