Character string identifier storage method and device, equipment and storage medium

Abstract

The invention provides a character string identifier storage method and device, equipment and a storage medium, and relates to the technical field of data processing. According to the method, character string IDs and storage strategies with the corresponding relation serve as newly-added nodes, the storage positions of the newly-added nodes are obtained in a prefix tree structure body, tree nodes of the prefix tree structure body comprise node storage strategies, child node dynamic arrays and key values, and the key values are used for calculating the positions of all child nodes in the dynamic arrays; judging whether the storage positions of the newly-added nodes have conflicts or not; if yes, determining the conflict node, splitting the child node dynamic array in the conflict node, re-hashing all child nodes of the conflict node into a new child node dynamic array, and determining the storage position of the newly added node in the prefix tree structure body. According to the method, the size of the hash table can be dynamically adjusted when the data volume changes, the problem of hash conflicts is solved, and efficient search performance is kept.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, equipment and storage medium for storing a string identifier. Background Art

[0002] A string identifier (ID) is an identifier represented in the form of a string. It is used to uniquely identify an object or entity and is widely used in scenarios that require unique identifiers, such as database records, file systems, and network protocols. String IDs can contain letters, numbers, and other characters. The specific format and content depend on the application scenario and design requirements. In the current development environment, especially in applications that process large amounts of data or require fast responses, there are many scenarios that require high-frequency matching, searching, and filtering of string IDs. String IDs need to be stored first. The storage method of string IDs greatly affects the efficiency and stability of string ID processing.

[0003] Storing string IDs can be achieved by building a linked list, that is, by building a linear data structure composed of nodes to store string IDs. Each node contains data and a pointer to the next node. However, since each match and search requires traversing the entire string, it is inefficient and not suitable for large-scale data.

[0004] For large-scale data, string IDs can be stored in hash tables, which use hash functions to map string IDs to index positions in an array, thereby achieving fast search. However, due to the possibility of hash conflicts, the search time is unstable when using hash tables to process string IDs, which has obvious defects in scenarios where string IDs need to be processed frequently. Summary of the invention

[0005] The present application provides a string identifier storage method, device, equipment and storage medium to solve the problem that the efficiency and stability of string ID processing are affected by the existence of hash conflicts in the existing string ID storage technology.

[0006] In a first aspect, the present application provides a method for storing a string identifier, comprising:

[0007] Get the string ID and the storage strategy corresponding to the string ID, use the string ID and storage strategy with corresponding relationship as the new node, and get the storage position of the new node in the prefix tree structure. The tree node of the prefix tree structure includes the node storage strategy, the child node dynamic array and the key value. The key value is used to calculate the position of each child node in the dynamic array.

[0008] Determine whether there is a conflict in the storage location of the newly added node;

[0009] If yes, confirm the conflicting node, split the dynamic array of child nodes in the conflicting node, re-hash all child nodes of the conflicting node into the new dynamic array of child nodes, and determine the storage location of the newly added node in the prefix tree structure;

[0010] Associate a storage policy with a storage location.

[0011] Optionally, in the above method, obtaining the storage location of the newly added node in the prefix tree structure includes:

[0012] Starting from the root node of the prefix tree structure, traverse all characters of the string ID;

[0013] The key value of the tree node corresponding to each character is used to calculate the hash value corresponding to the current character, and the storage position of the next character in the child node dynamic array is obtained according to the hash value corresponding to the current character;

[0014] The storage policy is associated with the storage location corresponding to the terminal character of the string ID.

[0015] Optionally, in the above method, when the current character is a non-terminal character, obtaining the storage position of the next character in the child node dynamic array according to the hash value corresponding to the current character includes:

[0016] Based on the hash value corresponding to the current character, determine whether the node in the dynamic array of child nodes in the tree node corresponding to the current character is empty;

[0017] If so, create a new child node in the child node dynamic array and insert the current character;

[0018] If not, traverse to the next character of the string ID.

[0019] Optionally, in the above method, determining whether there is a conflict in the storage location of the newly added node includes:

[0020] Based on the hash value corresponding to the terminal character, determine whether the corresponding node in the dynamic array of child nodes of the tree node corresponding to the terminal character is empty;

[0021] If so, confirm that there is no conflict;

[0022] If not, confirm that there is a conflict.

[0023] Optionally, in the method described above, confirming the conflicting node includes:

[0024] The tree node corresponding to the terminal character is identified as a conflicting node.

[0025] Optionally, as in the above method, the dynamic array of child nodes in the conflicting node is split, and all child nodes of the conflicting node are rehashed into a new dynamic array of child nodes, including:

[0026] The key value of the conflicting node is incremented, a new key value is obtained, and a new child node dynamic array is created, until each child node of the conflicting node and the newly added node can obtain a unique storage position in the new child node dynamic array according to the key value.

[0027] Optionally, in the above method, the tree node includes a root node, and before obtaining the storage location of the newly added node in the prefix tree structure, the method further includes:

[0028] Construct a prefix tree structure;

[0029] Initialize the dynamic array of child nodes of the root node to an empty pointer array;

[0030] Initialize the node storage strategy of the root node to a null pointer;

[0031] Set the root node's key value to the initial value.

[0032] In a second aspect, the present application provides a string identifier storage device, comprising:

[0033] A storage positioning module is used to obtain a string ID and a storage strategy corresponding to the string ID, and use the string ID and storage strategy with a corresponding relationship as a new node to obtain the storage location of the new node in the prefix tree structure. The tree node of the prefix tree structure includes a node storage strategy, a child node dynamic array and a key value. The key value is used to calculate the position of each child node in the dynamic array.

[0034] A conflict identification module is used to determine whether there is a conflict in the storage location of the newly added node;

[0035] The conflict handling module is used to confirm the conflicting node when there is a conflict, split the dynamic array of child nodes in the conflicting node, re-hash all child nodes of the conflicting node into a new dynamic array of child nodes, and determine the storage location of the newly added node in the prefix tree structure;

[0036] A storage module is used to associate storage policies with storage locations.

[0037] In a third aspect, the present application provides an electronic device, including a memory, a processor, and computer-executable instructions stored in the memory and executable on the processor, wherein the processor implements the string identifier storage method described in any one of the first aspects when executing the computer-executable instructions.

[0038] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method for storing a string identifier described in any one of the first aspects above is implemented.

[0039] In a fifth aspect, the present application provides a computer program product. When the computer program product is executed on a terminal device, the terminal device executes the string identifier storage method described in any one of the first aspects above.

[0040] The string identifier storage method, device, equipment and storage medium provided by the present application obtain the string ID and the storage strategy corresponding to the string ID, use the string ID and storage strategy with a corresponding relationship as the newly added node, obtain the storage position of the newly added node in the prefix tree structure, the tree node of the prefix tree structure includes the node storage strategy, the sub-node dynamic array and the key value, the key value is used to calculate the position of each sub-node in the dynamic array; determine whether there is a conflict in the storage position of the newly added node; if so, confirm the conflicting node, split the sub-node dynamic array in the conflicting node, re-hash all the sub-nodes of the conflicting node into the new sub-node dynamic array, and determine the storage position of the newly added node in the prefix tree structure. The method communicated in the present application is based on the prefix tree structure composed of the sub-node dynamic array and key value hashed by a linear hash function, and the prefix tree structure is used to store the string ID and the corresponding storage strategy, supporting fast matching and search operations, ensuring that the size of the hash table can be dynamically adjusted when the amount of data changes, overcoming the problem of hash conflicts, and maintaining efficient search performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0042] Figure 1 A schematic diagram of an application scenario of the string identifier storage method provided in this application;

[0043] Figure 2 The process of the string identifier storage method provided in the embodiment of the present application Figure 1 ;

[0044] Figure 3 The process of the string identifier storage method provided in the embodiment of the present application Figure 2 ;

[0045] Figure 4 A schematic diagram of a string identifier storage device provided in an embodiment of the present application;

[0046] Figure 5A schematic diagram of the structure of an electronic device based on a string identifier storage method provided in an embodiment of the present application.

[0047] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0048] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0049] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0050] First, the terms involved in this application are explained:

[0051] Prefix tree: It is a tree data structure, mainly used to store string sets for efficient string search operations. Each node represents a character, and the character sequence from the root node to a certain node corresponds to a string.

[0052] In the related art, string IDs can be stored in hash tables. The time complexity of these hash functions is O(m), where m is the length of the string. This means that the time required to calculate the hash value is linearly related to the length of the string. In addition, when different strings are hashed to the same hash value, hash conflicts will occur, resulting in the search time not always being stable.

[0053] In response to the above technical problems, the embodiments of the present application aim to propose a string identifier storage method, device, equipment and storage medium. The core concept of the method is: combining the prefix tree with the linear hash, constructing a prefix tree structure through a reasonable hierarchical design, hash conflict resolution strategy and dynamic adjustment mechanism, so as to store the string ID and maintain efficient search performance on large-scale data sets.

[0054] In order to better understand the solution of the embodiment of the present application, an application scenario involved in the embodiment of the present application is first introduced below.

[0055] See also Figure 1 , Figure 1 A schematic diagram of an application scenario of the string identifier storage method provided in an embodiment of the present application, such as Figure 1 As shown, it includes a data provider 11, a server 12 and an application 13. Among them, the data provider 11 sends string ID information to the server 12 to complete the storage of the string ID information in the server 12. The string ID information includes the string ID and its corresponding storage strategy (specific file content). The application 13 sends relevant instructions for string ID processing (such as search, filter, delete, etc.) to the server 12 as needed, and the server 12 executes according to the relevant instructions. Specifically, the server 12 uses a prefix tree structure to store the string ID information. Each string ID information is stored in each tree node of the prefix tree structure. The tree node of the prefix tree structure includes a node storage strategy, and also includes a sub-node dynamic array and key value based on linear hashing. The key value is used to calculate the position of each sub-node in the dynamic array.

[0056] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0057] based on Figure 1 In the application scenario shown, an embodiment of the present application also provides a method for storing a string identifier. Figure 2 The process of the string identifier storage method provided in the embodiment of the present application Figure 1 .like Figure 2 As shown, the method of this embodiment includes:

[0058] S201: Obtain the string identifier ID and the storage strategy corresponding to the string ID, use the string ID and storage strategy with corresponding relationship as a new node, obtain the storage position of the new node in the prefix tree structure, the tree node of the prefix tree structure includes the node storage strategy, the child node dynamic array and the key value, the key value is used to calculate the position of each child node in the dynamic array.

[0059] The execution subject of the embodiment of the present application may be a server, or a string identifier storage system in the server, wherein the string identifier storage system may be implemented by software.

[0060] In this embodiment, the string identifier can be understood as a unique ID of a string, and its storage strategy can be understood as a rule or method for how to store the string, such as storing in memory, on a hard disk, or using a compression algorithm.

[0061] It can be understood that the prefix tree is a data structure for efficiently storing and retrieving strings, and each node represents a prefix of a string. In this step, the string ID and its corresponding storage strategy are added to the prefix tree as a new node.

[0062] For example, for a newly added node, the string ID part is apple, and its storage strategy can be memory storage. In the prefix tree, the storage location of "apple" is searched on the path of "a"->"p"->"p"->"l"->"e", and the key value is used to calculate the position of "apple" in the dynamic array of child nodes of the corresponding node.

[0063] S202: Determine whether there is a conflict in the storage location of the newly added node.

[0064] It is understandable that after the storage position of the newly added node in the prefix tree is determined, it is necessary to check whether the position is already occupied by other nodes. If a node already exists at the target position, a conflict will occur.

[0065] S203: If yes, confirm the conflicting node, split the dynamic array of child nodes in the conflicting node, re-hash all child nodes of the conflicting node into a new dynamic array of child nodes, and determine the storage location of the newly added node in the prefix tree structure.

[0066] In this step, the storage location calculated for the newly added node has been occupied by other nodes. At this time, the dynamic array of the child nodes of the conflicting node needs to be split. The purpose of the split is to reallocate the child nodes of the conflicting node to the new dynamic array to avoid position conflicts. After the split is completed, the positions of all child nodes of the conflicting node in the new dynamic array need to be recalculated. This process is called rehashing. By rehashing, it can be ensured that all child nodes have a unique position in the new dynamic array, thus avoiding conflicts. At this time, the storage position of the newly added node in the prefix tree can be re-determined. Since the conflict has been resolved, the newly added node can be correctly inserted into the prefix tree.

[0067] S204: Associating the storage policy with the storage location.

[0068] The string identifier storage method provided in this embodiment obtains the string ID and the storage strategy corresponding to the string ID, takes the string ID and storage strategy with a corresponding relationship as a new node, obtains the storage position of the new node in the prefix tree structure, and the tree node of the prefix tree structure includes a node storage strategy, a sub-node dynamic array and a key value, and the key value is used to calculate the position of each sub-node in the dynamic array; determines whether there is a conflict in the storage position of the new node; if so, confirms the conflicting node, splits the sub-node dynamic array in the conflicting node, re-hashes all the sub-nodes of the conflicting node into the new sub-node dynamic array, and determines the storage position of the new node in the prefix tree structure. The method communicated in this application is based on the prefix tree structure composed of the sub-node dynamic array and key value hashed by a linear hash function, and uses the prefix tree structure to store the string ID and the corresponding storage strategy, supports fast matching and search operations, ensures that the size of the hash table can be dynamically adjusted when the amount of data changes, overcomes the problem of hash conflicts, and maintains efficient search performance.

[0069] The technical solution of the above-mentioned string identifier storage method is introduced in detail below.

[0070] Figure 3 The process of the string identifier storage method provided in the embodiment of the present application Figure 2 .like Figure 3 As shown, the string identifier storage method includes the following steps:

[0071] S301: Starting from the root node of the prefix tree structure, traverse all characters of the string ID.

[0072] It can be understood that the root node of the prefix tree is the starting point of the entire tree, and all strings are inserted or searched from here.

[0073] S302: Calculate the hash value corresponding to the current character using the key value of the tree node corresponding to each character, and obtain the storage position of the next character in the child node dynamic array according to the hash value corresponding to the current character.

[0074] It should be noted that in the traditional prefix tree, its structure includes a fixed-length pointer array "next" to store pointers to the next layer of nodes. Assuming that we are dealing with alphabetic characters (such as English letters), its size is usually 26 (lowercase letters az) or 52 (uppercase and lowercase letters az and AZ). Each element of the fixed-length pointer array corresponds to a character. In the traditional prefix tree, it is also possible to map characters directly to array indexes, which is actually a simple hashing method. For example, if we are dealing with lowercase letters, then "next[0]" corresponds to the character "a", "next[1]" corresponds to the character "b", and so on, until "next

[25] " corresponds to the character "z". At this time, when we need to find the next layer node corresponding to a certain character, we only need to calculate the index of the character and then directly access the corresponding position of the "next" array. However, in this prefix tree, each node needs to allocate a fixed-size memory block for the "next" array, even if few elements are actually used. This will result in a lot of memory being wasted.

[0075] In the present application, the child node array in the prefix tree structure is no longer fixed-length, but is adjusted according to the key value (hash_key). The position of each character in the child node array is determined by its key value. The key value can be the American Standard Code for Information Interchange (ASCII) code or other unique identifier of the character. The hash value is calculated by the key value to determine the storage position of the character in the child node dynamic array of the current tree node. Specifically, obtaining the storage position of the next character in the child node dynamic array according to the hash value corresponding to the current character can include the following steps:

[0076] a1. Based on the hash value corresponding to the current character, determine whether the node in the dynamic array of child nodes in the tree node corresponding to the current character is empty;

[0077] a2. If yes, create a new child node in the child node dynamic array and insert the current character;

[0078] a3. If not, traverse to the next character of the string ID.

[0079] That is, check whether there is already a node at the position of the current character in the child node dynamic array. If the position is empty, it means that the current character has not been inserted into the prefix tree. If the position is empty, create a new child node at the position and insert the current character into the child node. This means that a new storage location is allocated for the current character; if the position is not empty, it means that the current character already exists in the prefix tree, then continue to traverse the next character of the string ID and repeat the above steps.

[0080] Specifically, in this embodiment, the modulus operator (i.e., value % hash_key) can be used to find the corresponding storage location for the string ID. Its operation rule is to calculate the remainder after dividing two integers. For example, the result of "7 % 3" is 1, because the quotient of 7 divided by 3 is 2, and the remainder is 1.

[0081] S303: Associating the storage strategy with the storage location corresponding to the terminal character of the string ID.

[0082] This step is a loop process, until an empty child node position is found, the entire string ID is inserted into this position. This position is the final storage position of the string ID.

[0083] For example, suppose we have the following strings and their corresponding storage strategies: string 1: abc, storage strategy is memory storage; string 2: abd, storage strategy is hard disk storage. First, add "abc" to the prefix tree, specifically: starting from the root node, traverse character a, calculate the hash value of a, assuming the hash value is 1, position 1 in the dynamic array of child nodes of the root node is empty, insert a; continue to traverse character b, calculate the hash value of b, assuming the hash value is 2, position 2 in the dynamic array of child nodes of node a is empty, insert b; finally traverse character c, calculate the hash value of c, assuming the hash value is 3, position 3 in the dynamic array of child nodes of node b is empty, insert c, and at this time "abc" is successfully inserted without conflict. Then add "abd" to the prefix tree, specifically: starting from the root node, traverse character a; calculate the hash value of a, there is already a node a at position 1, continue traversing; continue traversing character b, calculate the hash value of b, there is already a node b at position 2, continue traversing; finally traverse character d, calculate the hash value of d, assuming the hash value is 4, position 4 in the dynamic array of child nodes of node b is empty, insert d; "abd" is successfully inserted without conflict.

[0084] It is understandable that since linear hashing relies on hash functions to distribute data, and prefix trees rely on the order of characters, there is a possibility of conflict between these two distribution methods, especially in the processing of large-scale data, conflicts are almost inevitable. Therefore, in the prefix tree structure, conflict identification and resolution mechanisms are necessary. In a possible implementation, the string identifier storage method provided by the present application determines whether there is a conflict in the storage location of the newly added node, which may specifically include the following steps:

[0085] b1. Based on the hash value corresponding to the terminal character, determine whether the corresponding node in the dynamic array of child nodes of the tree node corresponding to the terminal character is empty;

[0086] b2. If yes, confirm that there is no conflict;

[0087] b3. If not, confirm that there is a conflict.

[0088] In this embodiment, it is checked whether the position of the last character of the string ID in the child node dynamic array is empty. If the position is empty, it means that no other string ID occupies this position, and it is confirmed that there is no conflict; if the position is not empty, it means that another string ID has occupied this position, and it is confirmed that there is a conflict.

[0089] It is understandable that, based on the above embodiment, the confirmation of the conflicting node may be to confirm the tree node corresponding to the terminal character as the conflicting node. The conflicting node needs to be further processed to ensure the integrity and correctness of the prefix tree.

[0090] For example, suppose we want to insert two characters, the ASCII code values ​​of these two characters are 2 and 3 respectively, and the key value (hash_key) is 1. When character 2 is first inserted, the position is calculated: 2 % 1 = 0, and character 2 is stored at index 0, that is, the child node dynamic array is [2]. On this basis, the second character 3 is inserted, and the position is calculated: 3 % 1 = 0, but at this time index 0 is already occupied, that is, a conflict occurs at this time.

[0091] In one possible implementation, the string identifier storage method provided in the present application eliminates conflicts by rehashing. Specifically, the sub-node dynamic array in the conflicting node is split, and all the sub-nodes of the conflicting node are rehashed into a new sub-node dynamic array, including: numerically incrementing the key value of the conflicting node, obtaining a new key value, and creating a new sub-node dynamic array, until each sub-node of the conflicting node and the newly added node can obtain a unique storage location in the new sub-node dynamic array according to the key value.

[0092] Specifically, after creating a new dynamic array of child nodes, the following steps are included:

[0093] c1. According to the new key value, confirm the storage position of each child node of the conflicting node in the new child node dynamic array;

[0094] c2. According to the new key value, confirm the target hash value of the newly added node, and determine whether the corresponding target node in the new child node dynamic array is empty according to the target hash value;

[0095] c3. If yes, confirm the target node as the storage location of the newly added node.

[0096] For example, suppose we want to insert three characters, the ASCII code values ​​of these three characters are 2, 3 and 5 respectively, and the key value (hash_key) is 1. In the above embodiment, it can be concluded that when the second character 3 is inserted, index 0 is already occupied and a conflict occurs. Therefore, the split is triggered at this time, hash_key increases to 2, the child node dynamic array is reallocated, and the existing elements are rehashed. Recalculate the position: 2 % 2 = 0, 3 % 2 = 1, store character 2 at index 0, and store character 3 at index 1. The stored result, that is, the child node dynamic array is: [2,3].

[0097] When the third character 5 is inserted, the position is calculated: 5 % 2 = 1. It is found that index 1 is already occupied (conflict), which triggers a split, and the hash_key increases to 3. The child node dynamic array is reallocated, and the existing elements are rehashed. The position is recalculated: 2 % 3 = 2, 3 % 3 = 0, 5 % 3 = 2. However, it is found that index 2 is occupied (conflict), and it needs to be split again, so the hash_key increases to 4. The child node dynamic array is reallocated, and the position is recalculated: 2 % 4 = 2, 3 % 4 = 3, 5 % 4 = 1. Character 2 is stored at index 2, character 3 is stored at index 3, and character 5 is stored at index 1. The result of the storage at this time, that is, the child node dynamic array is: [null, 5, 2, 3].

[0098] As mentioned above, after rehashing, the new table corresponds to the new hash_key, the time complexity is still O(1), and there is no conflict between storage and search, and the search consumption is stable.

[0099] In a possible implementation, the tree node includes a root node. The string identifier storage method provided in the present application further includes: constructing a prefix tree structure and initializing the root node before obtaining the storage location of the newly added node in the prefix tree structure. Specifically, initializing the root node may include: initializing the dynamic array of child nodes of the root node to a null pointer array; initializing the node storage policy of the root node to a null pointer; and setting the key value of the root node to an initial value.

[0100] It is understandable that the purpose of setting specific steps during initialization is to ensure the correctness and efficiency of the data structure. For example, each node of the prefix tree usually has a pointer array pointing to its child nodes. Initializing it to an empty pointer array means that each element in this array is set to "null", indicating that the current node does not have any child nodes yet. This saves memory space and makes it easy to check whether a child node exists during subsequent operations (such as inserting a new node). If a position is a null pointer, it means that there is no child node at that position, and a new child node can be directly inserted at that position.

[0101] In this embodiment, the node storage policy of the root node is initialized to a null pointer, which provides flexibility and allows the storage policy to be dynamically allocated and set as needed in subsequent operations. Setting the key value of the root node to the initial value is mainly to avoid misuse of the root node, and in subsequent operations, the root node can be clearly distinguished from other nodes storing actual data.

[0102] Based on the above introduction to the string identifier storage method provided by the present application, the present application further explains the advantages of the string identifier storage method of the present application over the existing storage methods in conjunction with specific application cases.

[0103] For example, in a large log processing system, when processing logs generated by different log units, it is necessary to store these processes in different paths or store them with different strategies according to different log group IDs. Therefore, it is necessary to save the corresponding storage strategy in the memory. When each log enters the processing flow, the corresponding storage strategy needs to be found. The system stores the correspondence between these IDs and storage strategies in the form of a linked list. As the number of log groups increases and the log traffic increases, the search consumption will increase linearly, greatly affecting the performance of the entire system. By replacing the entire search structure with the solution proposed by this patent, using the log group ID as the key to construct a prefix tree structure, and saving the storage strategy at each node, the search time consumption can be reduced to the O(1) level. After a long period of verification, after replacing it with this solution, the performance of the entire log system has changed from being affected by the number of log groups and log traffic to being affected only by log traffic, and the maximum log traffic that can be tolerated has increased to about 5 times the original.

[0104] In a possible implementation, the string identifier storage method provided in the present application may also be applied to license plate queries.

[0105] It is understandable that in license plate-related application scenarios, high-frequency matching, searching, and filtering operations of string IDs are very important, especially in systems with high requirements for real-time performance and high accuracy. In traffic management systems, efficient license plate matching and filtering are the key to achieving real-time traffic flow analysis and optimization control. The number of license plate additions and deletions in the entire system is extremely low compared to search operations, but the storage and search of license plates require a considerable amount of data processing. In this scenario, a prefix tree structure can be constructed, and the license plate is used as a string ID. The vehicle management information is stored in the corresponding tree node, which can achieve extremely high search efficiency. By combining the prefix tree with linear hashing, space utilization can be optimized, and problems such as memory management and performance optimization can be solved on the basis of meeting the requirements of efficient search and dynamic expansion. The string identifier storage method provided by this method provides efficient search performance and dynamic expansion capabilities, which meet the needs of related application scenarios.

[0106] It should be noted that the aforementioned application embodiments are only examples and not limitations. The string identifier storage provided by the present application can also be applied in a variety of large-scale data processing scenarios. For example, for non-real-time online game servers, the game ID can also be used to construct a search tree to improve the game processing capabilities of a single server.

[0107] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0108] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0109] Figure 4 A schematic diagram of a string identifier storage device provided in an embodiment of the present application. Figure 4 As shown, the string identifier storage device 40 includes:

[0110] The storage positioning module 41 is used to obtain the string ID and the storage strategy corresponding to the string ID, and use the string ID and the storage strategy with the corresponding relationship as the newly added node, and obtain the storage position of the newly added node in the prefix tree structure. The tree node of the prefix tree structure includes the node storage strategy, the child node dynamic array and the key value, and the key value is used to calculate the position of each child node in the dynamic array;

[0111] A conflict identification module 42 is used to determine whether there is a conflict in the storage location of the newly added node;

[0112] The conflict handling module 43 is used to confirm the conflict node when there is a conflict, split the dynamic array of child nodes in the conflict node, re-hash all child nodes of the conflict node into a new dynamic array of child nodes, and determine the storage position of the newly added node in the prefix tree structure;

[0113] The storage module 44 is used to associate the storage policy with the storage location.

[0114] In a possible design, the storage positioning module 41 is specifically used for:

[0115] Starting from the root node of the prefix tree structure, traverse all characters of the string ID;

[0116] The key value of the tree node corresponding to each character is used to calculate the hash value corresponding to the current character, and the storage position of the next character in the child node dynamic array is obtained according to the hash value corresponding to the current character;

[0117] The storage policy is associated with the storage location corresponding to the terminal character of the string ID.

[0118] In a possible design, when the current character is a non-terminal character, the storage positioning module 41 is specifically used to:

[0119] Based on the hash value corresponding to the current character, determine whether the node in the dynamic array of child nodes in the tree node corresponding to the current character is empty;

[0120] If so, create a new child node in the child node dynamic array and insert the current character;

[0121] If not, traverse to the next character of the string ID.

[0122] In a possible design, the conflict identification module 42 is specifically used for:

[0123] Based on the hash value corresponding to the terminal character, determine whether the corresponding node in the dynamic array of child nodes of the tree node corresponding to the terminal character is empty;

[0124] If so, confirm that there is no conflict;

[0125] If not, confirm that there is a conflict.

[0126] In a possible design, the conflict handling module 43 is specifically used for:

[0127] The tree node corresponding to the terminal character is identified as a conflicting node.

[0128] In a possible design, the conflict handling module 43 is specifically used for:

[0129] The key value of the conflicting node is incremented, a new key value is obtained, and a new child node dynamic array is created, until each child node of the conflicting node and the newly added node can obtain a unique storage position in the new child node dynamic array according to the key value.

[0130] In a possible design, the string identifier storage device provided in the embodiment of the present application further includes an initialization module (not shown), and the initialization module is specifically used to:

[0131] Construct a prefix tree structure;

[0132] Initialize the dynamic array of child nodes of the root node to an empty pointer array;

[0133] Initialize the node storage strategy of the root node to a null pointer;

[0134] Set the root node's key value to the initial value.

[0135] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0136] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0137] Figure 5 A schematic diagram of the structure of an electronic device based on a string identifier storage method provided in an embodiment of the present application. Figure 5 As shown, the electronic device of this embodiment includes: at least one processor 50 ( Figure 5 Only one is shown in the figure), a memory 51, and a computer program stored in the memory 51 and executable on at least one processor 50, wherein the processor 50 implements the steps in any of the above-mentioned method embodiments when executing the computer program.

[0138] The electronic device may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will appreciate that Figure 5 These are merely examples of electronic devices and do not constitute a limitation on the electronic devices. The electronic devices may include more or fewer components than those shown in the figure, or a combination of certain components, or different components. For example, the electronic devices may also include input and output devices, network access devices, etc.

[0139] The processor 50 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0140] The specific implementation process of the processor 50 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0141] In some embodiments, the memory 51 may be an internal storage unit of the electronic device, such as the memory of the electronic device. In other embodiments, the memory 51 may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory 51 may also include both an internal storage unit of the electronic device and an external storage device. The memory 51 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of a computer program. The memory 51 may also be used to temporarily store data that has been output or is to be output.

[0142] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0143] The computer-readable storage medium mentioned above may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium may be any available medium that can be accessed by a general or special-purpose computer.

[0144] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in the above-mentioned electronic device.

[0145] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0146] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0147] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0148] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for storing a string identifier, characterized in that: Obtain a string identifier ID and a storage strategy corresponding to the string identifier ID, use the string identifier ID and the storage strategy with a corresponding relationship as a newly added node, obtain the storage position of the newly added node in a prefix tree structure, wherein the tree node of the prefix tree structure includes a node storage strategy, a child node dynamic array and a key value, wherein the key value is used to calculate the position of each child node in the dynamic array; Determine whether there is a conflict in the storage location of the newly added node; If yes, confirm the conflicting node, split the dynamic array of child nodes in the conflicting node, re-hash all child nodes of the conflicting node into a new dynamic array of child nodes, and determine the storage position of the newly added node in the prefix tree structure; The storage policy is associated with the storage location.

2. The method according to claim 1, characterized in that The step of obtaining the storage location of the newly added node in the prefix tree structure includes: Starting from the root node of the prefix tree structure, traverse all characters of the string identifier ID; Calculate the hash value corresponding to the current character using the key value of the tree node corresponding to each character, and obtain the storage position of the next character in the child node dynamic array according to the hash value corresponding to the current character; The storage strategy is associated with a storage location corresponding to the terminal character of the string identifier ID.

3. The method according to claim 2, characterized in that When the current character is a non-terminal character, obtaining the storage position of the next character in the child node dynamic array according to the hash value corresponding to the current character includes: Based on the hash value corresponding to the current character, determine whether the node in the dynamic array of child nodes in the tree node corresponding to the current character is empty; If yes, create a new child node in the child node dynamic array and insert the current character; If not, traverse to the next character of the string identifier ID.

4. The method according to claim 2, characterized in that: The determining whether there is a conflict in the storage location of the newly added node includes: Based on the hash value corresponding to the terminal character, determine whether the corresponding node in the child node dynamic array of the tree node corresponding to the terminal character is empty; If so, confirm that there is no conflict; If not, confirm that there is a conflict.

5. The method according to any one of claims 2 to 4, characterized in that: The confirming the conflicting nodes comprises: The tree node corresponding to the terminal character is confirmed as a conflicting node.

6. The method according to any one of claims 1 to 4, characterized in that: The step of splitting the dynamic array of child nodes in the conflicting node and rehashing all child nodes of the conflicting node into a new dynamic array of child nodes includes: The key value of the conflicting node is incremented to obtain a new key value, and a new child node dynamic array is created until each child node and newly added node of the conflicting node can obtain a unique storage position in the new child node dynamic array according to the key value.

7. The method according to any one of claims 1 to 4, characterized in that: The tree node includes a root node, and before obtaining the storage location of the newly added node in the prefix tree structure, further includes: Construct a prefix tree structure; Initialize the dynamic array of child nodes of the root node to a null pointer array; Initialize the node storage strategy of the root node to a null pointer; The key value of the root node is set to the initial value.

8. A device for storing string identifiers, characterized in that: include: A storage positioning module, used to obtain a string identifier ID and a storage strategy corresponding to the string identifier ID, use the string identifier ID and the storage strategy with a corresponding relationship as a newly added node, and obtain the storage position of the newly added node in a prefix tree structure, wherein the tree node of the prefix tree structure includes a node storage strategy, a child node dynamic array and a key value, and the key value is used to calculate the position of each child node in the dynamic array; A conflict identification module, used to determine whether there is a conflict in the storage location of the newly added node; A conflict handling module is used to confirm the conflict node when there is a conflict, split the dynamic array of child nodes in the conflict node, re-hash all child nodes of the conflict node into a new dynamic array of child nodes, and determine the storage position of the newly added node in the prefix tree structure; A storage module is used to associate the storage policy with the storage location.

9. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images