A method for creating a string matching retrieval structure and a retrieval method

Abstract

An embodiment of the present invention provides a method for creating a string matching retrieval structure and a retrieval method, which relate to the technical field of databases. The method for creating a string matching retrieval structure includes: generating a plurality of node strings and a check string corresponding to a target string; calculating the hash values of the plurality of node strings, and storing the hash values of all the node strings corresponding to the target strings with the same first-order node string among the plurality of target strings as a tree structure, so as to obtain a forest structure corresponding to the plurality of target strings; respectively associating child nodes with each leaf node of each tree structure as the hash check nodes of the target string corresponding to the leaf node, and calculating the hash value of the check string corresponding to the target string, and storing the hash value of the check string corresponding to the target string in the hash check node of the target string; associating the hash check node of the target string with the target retrieval value corresponding to the target string.

Description

Technical Field

[0001] The present invention relates to the technical field of databases, and particularly to a method for creating a string matching retrieval structure and a retrieval method. Background Art

[0002] In Java, a custom class loader can be used to achieve dynamic loading and unloading of jar packages in a running Java program without restarting. However, for garbage collection to occur, it is necessary to ensure that there are no references to the object instances, class objects, and class loaders of the class. That is, if an update to a jar package needs to be loaded, all the class loaders involved need to be destroyed together. Generally, there is a tree-like relationship between class loaders. When a class needs to load its dependent classes, the class loader first searches upward through the parent delegation mechanism, and at this time, only a class name can be obtained. For most projects, a jar package may contain tens of thousands of classes. If a many-to-one mapping relationship between class names and jar packages needs to be established, the required storage space is extremely large.

[0003] In the process of implementing the present invention, the applicant found that at least the following problems exist in the prior art:

[0004] How to quickly locate the target in a many-to-one retrieval in Java and minimize memory occupancy and query time. Summary of the Invention

[0005] Embodiments of the present invention provide a method for creating a string matching retrieval structure and a retrieval method to solve the problem of how to quickly locate the target in a many-to-one retrieval in Java and minimize memory occupancy and query time.

[0006] To achieve the above object, on the one hand, embodiments of the present invention provide a method for creating a string matching retrieval structure, including:

[0007] For each target string among multiple target strings, generate multiple node strings and a check string corresponding to the target string; wherein, there is a sequential order among the multiple node strings corresponding to the same target string;

[0008] Calculate the hash values of the multiple node strings corresponding to each target string, and store the hash values of all the node strings corresponding to the target strings with the same first-order node string among the multiple target strings in a tree structure according to the predecessor-successor relationship between all the corresponding node strings, so as to obtain a forest structure corresponding to the multiple target strings;

[0009] Associate child nodes with each leaf node of each tree structure respectively as the hash verification nodes of the target string corresponding to the leaf node, calculate the hash value of the verification string corresponding to the target string, and store the hash value of the verification string corresponding to the target string in the hash verification node of the target string;

[0010] Associate the hash verification node of the target string with the target retrieval value corresponding to the target string; wherein, each target retrieval value has a preset association relationship with at least one target string;

[0011] Among the multiple node strings corresponding to the same target string, the node string in the subsequent order contains all the information of all the node strings in the previous order before it; the verification string corresponding to the target string contains all the information of at least one node string among the multiple node strings corresponding to the target string and partial information of each of the remaining node strings;

[0012] The root nodes of different tree structures store the hash values of different first-order node strings; each leaf node of a tree structure uniquely corresponds to a target string; each leaf node of a tree structure stores the hash value of the last-order node string of the target string corresponding to the leaf node;

[0013] The root nodes of the tree structures in the forest structure are sorted according to the stored hash values, and the child nodes of the same layer under each parent node in each tree structure are sorted according to the stored hash values.

[0014] Further, generating the multiple node strings and verification strings corresponding to the target string includes:

[0015] Divide the target string into N sequentially arranged substrings;

[0016] Concatenate the first n substrings among the N sequentially arranged substrings in order to form the n-th order node string of the target string, and obtain the multiple node strings corresponding to the target string.

[0017] Further, generating the multiple node strings and verification strings corresponding to the target string includes:

[0018] Concatenate the first letters of the first N - 1 substrings and the N-th substring in order to obtain the verification string corresponding to the target string.

[0019] Further, calculate the hash values of the multiple node strings corresponding to each target string, and store the hash values of all the node strings corresponding to the target strings with the same first-order node string among the multiple target strings in a tree structure according to the predecessor-successor relationship between the corresponding all node strings, so as to obtain the forest structure corresponding to the multiple target strings, including:

[0020] Aggregate and sort the multiple target strings in the order in which the same node strings appear;

[0021] Obtain each target string in the order after the multiple target strings are aggregated and sorted, and sequentially traverse the multiple node strings corresponding to the target string, and calculate the hash value of each node string;

[0022] Sequentially traverse the multiple node strings corresponding to the target string, and in the one-dimensional array floors[i][] of the i-th row in the two-dimensional array floors[][], search for the hash value of the i-th order node string traversed currently; where i increases from 0 to the number of all node strings corresponding to the target string - 1;

[0023] If the hash value of the i-th order node string is not found in the one-dimensional array floors[i][] of the i-th row, then write the hash value of the i-th order node string into the lower 32 bits of the one-dimensional array floors[i] ,

[0024] And when the i-th order node string is not the last-order node string of the target string, write the number of used storage positions corresponding to the one-dimensional array floors[i + 1][] of the (i + 1)-th row of the two-dimensional array floors[][] into the one-dimensional array floors[i] , and when the i-th order node string is the last-order node string of the target string, set the highest bit in the one-dimensional array floors[i] to 1; Increment the number of used storage positions corresponding to the one-dimensional array floors[i][];

[0025] ;

[0026] Among them, the two-dimensional array floors[][] is pre-created in Java to store the forest structure, and the initial number of used storage positions of each row's one-dimensional array in the two-dimensional array floors[][] is initialized to 0; the sequence numbers of the multiple node strings corresponding to the target string are sequentially numbered starting from 0

[0027] Further, for each leaf node of each tree structure, associate child nodes as hash verification nodes for the target string corresponding to the leaf node, and calculate the hash value of the verification string corresponding to the target string, and store the hash value of the verification string corresponding to the target string in the hash verification node of the target string, including:

[0028] When the node string in the i-th position is the last-position node string of the target string, it further includes:

[0029] Write the number of used storage positions in the one-dimensional array floors[k][] of the k-th row of the two-dimensional array floors[][] into the 32nd to 55th bits of the one-dimensional array floors[i] , and write the hash value of the verification string corresponding to the target string into the lower 32 bits of the one-dimensional array floors[k] ;

[0030] Increment the number of used storage positions in the one-dimensional array floors[k][]; ;

[0031] where k is the maximum value among the number of node strings of all target strings

[0032] Further, associate the hash verification node of the target string with the target retrieval value corresponding to the target string, including:

[0033] When the node string in the i-th position is the last-position node string of the target string, it further includes:

[0034] When it is determined that the target retrieval value corresponding to the target string does not exist in the target retrieval value array leaves[], add the target retrieval value corresponding to the target string to the target retrieval value array leaves[], and increment the number of used storage positions v of the target retrieval value array leaves[];

[0035] When it is determined that the target retrieval value corresponding to the target string exists in the target retrieval value array leaves[], write the index value of the storage position of the target retrieval value corresponding to the target string in the target retrieval value array leaves[] into the 32nd to 55th bits of the one-dimensional array floors[k] ;

[0036] Among them, a target retrieval value array leaves[] is pre-created in Java, and the used storage location number v of the target retrieval value array leaves[] is set to 0.

[0037] Further, the method further includes:

[0038] For each element in the one-dimensional array floors[0][] at the 0th row of the two-dimensional array floors[][], sort the elements in the one-dimensional array floors[0][] in ascending order according to the hash values stored in the lower 32 bits of the elements;

[0039] Take each one-dimensional array floors[d][] from the 1st row to the (k - 1)th row of the two-dimensional array floors[][] as a child node layer, each element in the one-dimensional array floors[d][] as a child node element, take the one-dimensional array floors[d - 1][] as the parent node layer of the one-dimensional array floors[d][], each element in the one-dimensional array floors[d - 1][] as a parent node element, and according to the value corresponding to the 32nd to 55th bits of each parent node element in the one-dimensional array floors[d - 1][], determine the child node elements in the one-dimensional array floors[d][] that have the parent node element in the one-dimensional array floors[d - 1][] as the parent node, group all the child node elements in the one-dimensional array floors[d][] in the order of the corresponding parent node elements in the one-dimensional array floors[d - 1][], and sort the child node elements in each group in the one-dimensional array floors[d][] in ascending order according to the hash values of the lower 32 bits of the child node elements;

[0040] According to the highest bit of each element, count the number of consecutive leaf nodes in each one-dimensional array from the 1st row to the (k - 1)th row of the two-dimensional array floors[][], and take 100 as a period, write the 56th to 62nd bits of the element where each leaf node is located into the number of consecutive leaf nodes starting from the leaf node; among them, the elements with the highest bit marked as 1 in the one-dimensional arrays from the 1st row to the (k - 1)th row of the two-dimensional array floors[][] correspond to the leaf nodes of the tree structure;

[0041] Shrink and release the unused storage space at the end of each one-dimensional array in the two-dimensional array floors[][] according to the used storage location number of each one-dimensional array in the two-dimensional array floors[][].

[0042] Further, the method further includes merging two forest structures to be merged according to the following steps:

[0043] Define a target forest structure array of two-dimensional array type in Java in advance, and initialize the number of elements in each one-dimensional array of the target forest structure array to the sum of the number of elements in the corresponding one-dimensional arrays of the two forests to be merged;

[0044] Merge all the elements in the 0th row of the two forests to be merged by performing merge sort according to the values of the lower 32 bits of the elements, and write all the sorted elements into the 0th row of the target forest structure array;

[0045] Process the child nodes of each element corresponding node in the 0th row of the target forest structure array that has been written in the order of left subtree first by using depth-first search. Use merge sort within the interval under the same parent node in each layer, record and increment the maximum subscript value of each layer; wherein, the maximum subscript value records the space continuously occupied by the one-dimensional array of each layer starting from the index 0 position, and is used to delete the idle space at the rear of each layer according to the maximum subscript value or determine the position where a new node is written when adding a new node;

[0046] When merging to a leaf node, start recording the counter of consecutive leaf nodes at the current layer until the counter is not 0 and the node is a non-leaf node, or the counter reaches 100. Starting from the first recorded leaf node, assign values to bits 56 to 62, and decrement the counter by 1 for each leaf node assigned;

[0047] After the two forests to be merged are merged, resize each one-dimensional array of each layer of the target forest structure array according to the maximum subscript value of each layer so that there are no subsequent blank elements.

[0048] On the other hand, an embodiment of the present invention provides a string matching and retrieval method, including:

[0049] Generate multiple node strings and check strings corresponding to the string to be retrieved, and calculate the hash values of the multiple node strings and check strings;

[0050] According to the hash values of the multiple node strings corresponding to the string to be retrieved, use the binary search method to traverse the forest structure created according to any of the previous string matching and retrieval structure creation methods according to the strategy of first breadth-first and then left-depth-first to determine the leaf node corresponding to the string to be retrieved in the forest structure;

[0051] Obtain the check hash value from the hash check nodes associated with the leaf node corresponding to the string to be retrieved, and compare it with the hash value of the check string corresponding to the string to be retrieved;

[0052] When the hash value of the check string corresponding to the to-be-retrieved string is equal to the check hash value, obtain the target retrieval value associated with the hash check node associated with the leaf node corresponding to the to-be-retrieved string as the retrieval result of the to-be-retrieved string.

[0053] The above technical solution has the following beneficial effects: By generating multiple node characters and hash check strings from the target string and calculating the hash values to store them in a tree structure, the hash mapping of a single target string is transformed into the hash mapping of multiple node strings, thereby reducing the probability of hash collisions. In a many-to-one retrieval application, multiple target strings are converted into multiple hash values and stored in a forest structure, and the leaf nodes are associated with the target retrieval values. For the case where multiple different target strings correspond to the same target retrieval value, only one copy of the target retrieval value needs to be saved. Thus, it can not only improve the retrieval efficiency by using the strategy of forest structure and hash value comparison, but also reduce the storage space required for storing multiple different target strings corresponding to the same target retrieval value, achieving the technical effects of improving the retrieval efficiency and saving storage space. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0055] Figure 1 is a flowchart of a method for creating a string matching retrieval structure according to one of the embodiments of the present invention;

[0056] Figure 2 is a flowchart of a string matching retrieval method according to one of the embodiments of the present invention;

[0057] Figure 3 is a schematic diagram of a forest structure according to one of the embodiments of the present invention;

[0058] Figure 4 is a schematic diagram of the bit field definition of the node storage data according to one of the embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0059] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0060] On the one hand, as Figure 1 shown, an embodiment of the present invention provides a method for creating a string matching retrieval structure, including:

[0061] Step S10, for each target string among multiple target strings, generate multiple node strings and a check string corresponding to the target string; wherein, there is a sequential order among the multiple node strings corresponding to the same target string;

[0062] Step S11, calculate the hash values of the multiple node strings corresponding to each target string, and store the hash values of all the node strings corresponding to the target strings with the same first-order node string among the multiple target strings in a tree structure according to the precursor-successor relationship between the corresponding all node strings, so as to obtain a forest structure corresponding to the multiple target strings;

[0063] Step S12, respectively associate child nodes for each leaf node of each tree structure as the hash check node of the target string corresponding to the leaf node, calculate the hash value of the check string corresponding to the target string, and store the hash value of the check string corresponding to the target string in the hash check node of the target string;

[0064] Step S13, associate the hash check node of the target string with the target retrieval value corresponding to the target string; wherein, each target retrieval value has a preset association relationship with at least one target string;

[0065] Wherein, among the multiple node strings corresponding to the same target string, the node string in the subsequent order contains all the information of all the node strings in the previous order of the node string in the subsequent order; the check string corresponding to the target string contains all the information of at least one node string among the multiple node strings corresponding to the target string and partial information of each of the remaining node strings;

[0066] The root nodes of different tree structures store the hash values of different first-order node strings; each leaf node of a tree structure uniquely corresponds to a target string; each leaf node of a tree structure stores the hash value of the last-order node string of the target string corresponding to the leaf node;

[0067] The root nodes of the tree structures in the forest structure are sorted according to the stored hash values, and the child nodes of the same layer under each parent node in each tree structure are sorted according to the stored hash values.

[0068] In some embodiments, the multiple target strings are different from each other; the multiple node strings are in order; taking the order into account, the queues formed by the multiple node strings corresponding to different target strings are different from each other; between different target strings, all of the multiple node strings can be different, some of the node strings can be the same, or all of the node strings can be the same but the order of the node strings is different. For example, the target string "a.b.c.d" is different from the target string "a.b.c.e", and the target string "a.b.c.d" is different from the target string "b.a.c.d". For example, the node strings corresponding to the target string "a.b.c.d" are sequentially "a", "a.b", "a.b.c", "a.b.c.d"; the node strings of the target string "a.b.c.e" are sequentially "a", "a.b", "a.b.c", "a.b.c.e"; the node strings of the target string "b.a.c.d" are sequentially "b", "b.a", "b.a.c", "b.a.c.d"; since the first-order node strings of the target string "a.b.c.d" and the target string "a.b.c.e" are both "a", the target string "a.b.c.d" and the target string "a.b.c.e" are constructed into one tree; the first-order node string of the target string "b.a.c.d" is "b", which is constructed into another tree. In some specific embodiments, the target retrieval value can be the jar package name, and each jar package can contain many classes, and the target string can be the corresponding class name. The embodiments of the present invention can implement the mapping from the fully qualified class name to the package name. When constructing the tree, only one branch of the tree will be generated for the paths in the tree formed by the same node string sequence, which can significantly save memory compared with the traditional key-value pair storage method, and moreover, each node stores a hash value instead of a string, which can further save memory. For example, the node strings corresponding to the target string "a.b.c.d" are sequentially "a", "a.b", "a.b.c", "a.b.c.d"; the node strings of the target string "a.b.c.e" are sequentially "a", "a.b", "a.b.c", "a.b.c.e", and the two target strings are in one tree, and the node paths in the tree corresponding to "a", "a.b", "a.b.c" are the same, and only two branches need to be separated after the node corresponding to "a.b.c" to connect "a.b.c.d" and "a.b.c.e" respectively. Thus, the two target strings can share the branches corresponding to "a", "a.b", "a.b.c". For the case where there are many repeated parent domain nodes in the fully qualified class names in the jar package, the embodiments of the present invention can greatly save memory. When retrieving based on the forest structure of the embodiments of the present invention, integer numerical comparisons are performed instead of string comparisons, which can also significantly improve the retrieval speed.Preferably, the string matching retrieval structure is created in a Java environment. Java is different from C / C++. C / C++ can directly operate on memory, while Java uses objects that also have object headers. The more objects there are, the more memory is wasted, which is not conducive to storing massive amounts of data or overly complex structures. The embodiments of the present invention at least solve the problems of large memory occupation and low retrieval efficiency in the many-to-one string retrieval application in the Java environment.

[0069] The embodiments of the present invention have the following technical effects: By generating multiple node characters and hash verification strings for the target string and calculating the hash values to be stored as a tree structure, the hash mapping of a single target string is transformed into the hash mapping of multiple node strings, thereby reducing the probability of hash collisions. In the many-to-one retrieval application, multiple target strings are converted into multiple hash values and stored as a forest structure, and the leaf nodes are associated with the target retrieval value. For the case where multiple different target strings correspond to the same target retrieval value, only one copy of the target retrieval value needs to be saved. Thus, it can both improve the retrieval efficiency by using the strategy of comparing the forest structure and hash values, and reduce the storage space required for storing multiple different target strings corresponding to the same target retrieval value, achieving the technical effects of improving the retrieval efficiency and saving storage space.

[0070] Further, generating the multiple node strings and verification strings corresponding to the target string includes:

[0071] Dividing the target string into N sequentially arranged substrings;

[0072] Concatenating the first n substrings in the N sequentially arranged substrings in order to form the n-th ranked node string of the target string, and obtaining the multiple node strings corresponding to the target string.

[0073] In some embodiments, the target string "a.b.c.d" is divided into multiple substrings "a", ".b", ".c", ".d" with "." as the interval, a total of N = 4 substrings. Multiple node strings are generated in order. The first-ranked node string is the first 1 substring, that is, "a", the second-ranked node string is the first 2 substrings, that is, "a.b", and so on to obtain multiple node strings.

[0074] Further, generating the multiple node strings and verification strings corresponding to the target string includes:

[0075] Concatenating the first letters of the first N - 1 substrings and the N-th character in order to obtain the verification string corresponding to the target string.

[0076] In some embodiments, the first letters of the first N - 1 substrings and the Nth sub - character can be concatenated in sequence to obtain the check string corresponding to the target string; for example, the check string corresponding to the target string "a1.b2.c3.d4" can be "abc.d4". When there are separator symbols used as separators in the target string, the substrings can be split by the separator symbols, the first letters of the first N - 1 substrings and the separator symbols are concatenated, and then concatenated with the Nth sub - character in sequence to obtain the check string corresponding to the target string; for example, the check string corresponding to the target string "a1.b2.c3.d4" can be "a.b.c.d4".

[0077] Further, calculate the hash values of the multiple node strings corresponding to each target string, and store the hash values of all the node strings corresponding to the target strings with the same first - order node string among the multiple target strings as a tree structure according to the predecessor - successor relationship between the corresponding node strings, so as to obtain the forest structure corresponding to the multiple target strings, including:

[0078] Aggregate and sort the multiple target strings in the order in which the same node strings appear;

[0079] Obtain each target string in the order after the multiple target strings are aggregated and sorted, traverse the multiple node strings corresponding to the target string in sequence, and calculate the hash value of each node string;

[0080] Traverse the multiple node strings corresponding to the target string in sequence, and in the one - dimensional array floors[i][] of the i - th row in the two - dimensional array floors[][], search for the hash value of the i - th order node string traversed currently; where i increments from 0 to the number of all node strings corresponding to the target string minus 1;

[0081] If the hash value of the i - th order node string is not found in the one - dimensional array floors[i][] of the i - th row, write the hash value of the i - th order node string into the lower 32 bits of the one - dimensional array floors[i] ,

[0082] And when the i - th order node string is not the last - order node string of the target string, write the number of used storage positions corresponding to the one - dimensional array floors[i + 1][] of the (i + 1) - th row of the two - dimensional array floors[][] into the one - dimensional array floors[i] From the 32nd to the 55th bit of [], when the node string at the i-th position is the node string of the last position of the target string, set the highest bit in the one-dimensional array floors[i] to 1;

[0083] Increment the number of used storage positions corresponding to the one-dimensional array floors[i][[]] ;

[0084] Among them, the two-dimensional array floors[][] is pre-created in Java for storing the forest structure, and the initial number of used storage positions of each row's one-dimensional array in the two-dimensional array floors[][] is initialized to 0; the sequence numbers of the node strings corresponding to the target string are sequentially numbered starting from 0

[0085] In some embodiments, for example, multiple target strings include "a.b.c", "a.b.c.d", "g.h.k", "a.b.c.e", "g.f"; first, cluster and sort the multiple target strings in the order of the appearance of the same node string. Then, the order of the appearance of the node strings corresponding to "a.b.c", "a.b.c.d", "a.b.c.e" is "a", "a.b", "a.b.c", so these three target strings are clustered together, and "g.h.k" and "g.f" are clustered together. After clustering and sorting, it can be "a.b.c", "a.b.c.d", "a.b.c.e", "g.h.k", "g.f"; such clustering and sorting is to cooperate with the subsequent steps to store the children nodes of the same parent node continuously in the next layer in the two-dimensional array floors[][], so that the parent node only needs to record the start position index value of the continuous storage position of the children nodes in the next layer, thereby reducing the amount of data stored in the nodes and saving memory space. By retrieving the start index positions of the respective children nodes recorded by the same-layer parent nodes, the end position index value of the continuous storage position of the children nodes in the next layer of any parent node can be counted.

[0086] Furthermore, associate children nodes with each leaf node of each tree structure as the hash check nodes of the target string corresponding to the leaf node, and calculate the hash value of the check string corresponding to the target string, and store the hash value of the check string corresponding to the target string in the hash check node of the target string, including:

[0087] When the node string at the i-th position is the node string of the last position of the target string, it further includes:

[0088] Write the number of used storage positions in the one-dimensional array floors[k][] of the k-th row of the two-dimensional array floors[][] into the 32nd to 55th bits of the one-dimensional array floors[i] , and write the hash value of the check string corresponding to the target string into the lower 32 bits of the one-dimensional array floors[k] ;

[0089] Increment the number of used storage positions in the one-dimensional array floors[k][] ;

[0090] where k is the maximum value among the number of node strings of all target strings

[0091] Furthermore, associating the hash check node of the target string with the target retrieval value corresponding to the target string includes:

[0092] When the i-th node string is the last node string of the target string, it further includes:

[0093] When it is determined that the target retrieval value corresponding to the target string does not exist in the target retrieval value array leaves[], add the target retrieval value corresponding to the target string to the target retrieval value array leaves[], and increment the number of used storage positions v of the target retrieval value array leaves[];

[0094] When it is determined that the target retrieval value corresponding to the target string exists in the target retrieval value array leaves[], write the index value of the storage position of the target retrieval value corresponding to the target string in the target retrieval value array leaves[] into the 32nd to 55th bits of the one-dimensional array floors[k] ;

[0095] where a target retrieval value array leaves[] is created in Java in advance, and the number of used storage positions v of the target retrieval value array leaves[] is set to 0.

[0096] In some embodiments, in the target retrieval value array leaves[], one or more target retrieval values corresponding to multiple target strings are stored. Taking the case where a jar package contains multiple fully qualified class names as an example, the multiple fully qualified class names are multiple target strings, and the jar package name is the target retrieval value. There can be multiple jar package names, and each jar package name is associated with multiple fully qualified class names in the jar package.

[0097] Furthermore, the method further includes:

[0098] For each element in the one-dimensional array floors[0][] at the 0th row of the two-dimensional array floors[][], sort the elements in the one-dimensional array floors[0][] in ascending order according to the hash values stored in the lower 32 bits of said elements;

[0099] Take each one-dimensional array floors[d][] from the 1st row to the (k - 1)th row of the two-dimensional array floors[][] as a child node layer, each element in the one-dimensional array floors[d][] as a child node element, take the one-dimensional array floors[d - 1][] as the parent node layer of the one-dimensional array floors[d][], and each element in the one-dimensional array floors[d - 1][] as a parent node element. According to the value corresponding to the 32nd to 55th bits of each parent node element in the one-dimensional array floors[d - 1][], determine, in the one-dimensional array floors[d][], the child node elements with the parent node element in the one-dimensional array floors[d - 1][] as the parent node. Group all the child node elements in the one-dimensional array floors[d][] in the order of the corresponding parent node elements in the one-dimensional array floors[d - 1][], and sort the child node elements in each group in ascending order according to the hash values of the lower 32 bits of said child node elements;

[0100] According to the highest bit of each element, count the number of consecutive leaf nodes in each one-dimensional array from the 1st row to the (k - 1)th row of the two-dimensional array floors[][]. Taking 100 as a period, write the 56th to 62nd bits of the element where each leaf node is located into the number of consecutive leaf nodes starting from said leaf node; wherein, the elements with the highest bit marked as 1 in each one-dimensional array from the 1st row to the (k - 1)th row of the two-dimensional array floors[][] correspondingly serve as the leaf nodes of the tree structure;

[0101] Shrink and release the unused storage space at the end of each one-dimensional array in the two-dimensional array floors[][] according to the number of used storage positions of each one-dimensional array.

[0102] In some embodiments, sorting can facilitate the use of binary search during retrieval to quickly retrieve the corresponding hash value. In Java, the unused space at the end of the array can be released, which can further save space.

[0103] Furthermore, the method further includes merging two forest structures to be merged according to the following steps:

[0104] Define a two-dimensional array type target forest structure array in Java in advance, and initialize the number of elements in each row of the one-dimensional array of the target forest structure array to the sum of the number of elements in the corresponding rows of the two forest structures to be merged;

[0105] Merge all elements in row 0 of the two forest structures to be merged by performing merge sort on the lower 32-bit values of the elements, and write all the sorted elements into row 0 of the target forest structure array;

[0106] The child nodes of the node corresponding to each element in row 0 of the target forest structure array that has been written are processed in a depth-first order with left subtree first, a merge sort is used within the interval under the same parent node in each layer, and the maximum subscript value of each layer is recorded and incremented; wherein the maximum subscript value records the continuous occupied space of the one-dimensional array of each layer starting from index 0, and is used to delete the free space at the end of each layer according to the maximum subscript value or to determine the writing position of a new node when adding a new node;

[0107] When merging into a leaf node, the layer starts recording the counters of consecutive leaf nodes until the counter is not 0 and the node is a non-leaf node, or the counter reaches 100. Starting from the first recorded leaf node, the 56th to 62nd bits are assigned values. Each time a leaf node is assigned, the counter is reduced by 1.

[0108] After the two forest structures to be merged are merged, the one-dimensional array of each layer is reduced to no subsequent blank elements according to the maximum subscript value of each layer of the target forest structure array.

[0109] On the other hand, Figure 2 As shown, an embodiment of the present invention provides a string matching retrieval method, including:

[0110] Step S20: Generate multiple node strings and verification strings corresponding to the string to be searched, and calculate hash values of the multiple node strings and verification strings;

[0111] In some embodiments, a plurality of node strings and a verification string corresponding to the string to be searched are generated, and a hash value is calculated in the same manner as the aforementioned string matching search structure creation method.

[0112] Step S21, based on the hash values of the multiple node strings corresponding to the string to be searched, a binary search method is used to traverse the forest structure created according to any of the above string matching search structure creation methods in a breadth-first and then left-depth-first strategy to determine the leaf node corresponding to the string to be searched in the forest structure;

[0113] Step S22: Obtain the verification hash value from the hash verification nodes associated with the leaf node corresponding to the to-be-retrieved string, and compare it with the hash value of the verification string corresponding to the to-be-retrieved string;

[0114] Step S23: When the hash value of the verification string corresponding to the to-be-retrieved string is equal to the verification hash value, obtain the target retrieval value associated with the hash verification node associated with the leaf node corresponding to the to-be-retrieved string as the retrieval result of the to-be-retrieved string.

[0115] The embodiment of the present invention is an embodiment of a string matching retrieval method for a forest structure created based on the foregoing string matching retrieval structure creation method. The embodiment of the present invention can be understood according to the embodiment of the foregoing string matching retrieval structure creation method, and will not be elaborated here.

[0116] The above technical solutions of the embodiments of the present invention will be described in detail below in conjunction with specific application examples. For technical details not introduced during the implementation process, reference can be made to the relevant descriptions above.

[0117] The inventor found through the analysis and comparison of various technologies that the trie or radix tree is a design for compressing plain text strings to save memory. It is generally used when there is a large amount of similar / repeated information, and can achieve a compression rate of less than 20%. However, the memory savings of the trie are based on programming languages such as C / C++ that support memory pointers. Each node on the trie generally occupies no more than 5 bytes of space, and the radix tree can also save a part of the number of nodes through branch compression; C / C++ can directly operate on pointers to handle continuous memory, and can also achieve continuous memory or pointer operations of objects through structures. There is no memory pointer type similar to C / C++ in Java, and there is no such means to directly operate on memory. Java represents everything through objects. Due to the existence of object headers and references for each object and its set of sub-items, the actual statistics show that a single node can be as high as 200 bytes. In addition, the implementation of some data types has its specific specifications, which will increase more memory usage. If we consider the garbage collection mechanism of the JVM again, all associated reference sub-nodes need to be recycled together, which may cause the garbage collection to take too long or block. For example, the jar package corresponding to the class name io.netty.buffer.AbstractByteBuf is netty-all.jar, and netty-all.jar may contain tens of thousands of classes. Any class name in it needs to point to netty-all.jar. In Java, this type of directional index uses Map (mapping). Map is a data structure that stores key-value pairs. Each of its elements contains a key and a value (i.e., a key-value pair); if we retrieve from netty-all.jar to these more than ten thousand classes, the storage space can be greatly optimized, but the purpose of the embodiments of the present invention is the opposite. The embodiments of the present invention are to find netty-all.jar from io.netty.buffer.AbstractByteBuf. Then, according to the data structure definition of Map in Java, the total number of bytes required for storing each element (key-value pair) is at least: 80 + key length + value length. As the number of hash collisions increases, the storage space complexity and operation time complexity of the Map storage structure will increase again after it changes to a red-black tree (the expected number of stored classes may exceed one million). For example:

[0118] The first case: If the key is each class and the value is the package name, more than ten thousand key-value pairs are required to describe a package. In this case, the memory occupancy is huge, but the retrieval is fast. The second case: Only one key-value pair is established for a package. For example, the key is netty-all.jar and the value is an array of ten thousand classes. At this time, the storage space is much smaller than the first case, but the retrieval takes time; directly relying on the standard mechanism of Java cannot solve the space and time problems simultaneously in Java.

[0119] The physical design of the database index can synchronize and update by creating a materialized view to aggregate multiple tables / records that meet specific conditions. When the query statement in the database is executed, an execution plan will be constructed and an attempt will be made to use the index to optimize the execution path; in the database mechanism, when it is found during query statement parsing that the query content can be included in a created materialized view, the construction of the query plan will be aborted and the materialized view will be directly queried. The judgment of whether the query statement belongs to the materialized view can be accelerated by constructing a materialized view index graph of multiple tables for the acceleration of multi-table association queries in distributed data analysis. For example, the steps for querying a distributed data warehouse include: establishing a parse tree based on the query statement (SQL), then calculating the step-by-step grouping query plan, and then calculating the time complexity and space complexity of creating an intermediate table to calculate the plan with the minimum (or locally minimum) cost, and finally executing the query plan according to the plan and aggregating the query results. Among these, in the second step of calculating the step-by-step grouping query plan, it can be optimized through an index. Specifically, if there is already a materialized view for the data from multiple tables, that is, the materialized view already contains the data to be queried, the generation process of the entire query plan will be aborted and the query result of the materialized view will be directly returned, and the mapping relationship between the original table and the materialized view in the parse tree is also of this structure. For example, if the original tables to be queried include three tables A, B, and C, then an index on a string like A.B.C can be established to provide index filtering for the materialized view.

[0120] The embodiments of the present invention are to solve the problem of quickly locating the target in the many-to-one retrieval in Java and minimizing the memory occupancy and query time as much as possible (that is, making the time complexity and space complexity as small as possible under the trade-off), and can be applied to aspects such as dynamic class loading, data sharding strategy, index optimization of distributed database SQL queries, etc. to improve system performance. For example: The data sharding strategy is equivalent to a routing strategy, which can be through, such as primary key hash, by date, etc., or an index relationship can be established between a string (such as "database name.table name.region name") and the corresponding routing partition (0, 1, 2, 3, 4...) through this index.

[0121] 1. Data structure design of the embodiments of the present invention:

[0122] 1) The storage structure of the index forest (forest structure) is as follows Figure 3 shown. The hollow circles are general index nodes including the root node and ordinary nodes located between the root node and the leaf nodes. The circles with a cross in the center are leaf nodes, the circles with a triangle in the center are hash check nodes, and the shaded circles are the corresponding values of the leaf nodes (the target retrieval value is the preset corresponding value of the string corresponding to the path from the root node to the leaf node. For example, in the application of retrieving the package name from the class name string in the previous example, when the string corresponding to the path from the root node to the leaf node is io.netty.buffer.AbstractByteBuf, the preset corresponding value of io.netty.buffer.AbstractByteBuf, which is netty-all.jar, is pre-stored in the corresponding shaded leaf node value).

[0123] 1.1) Since objects in Java occupy a relatively large amount of space, a two-dimensional long integer array (the variable name is defined as floors in the embodiments of the present invention) is used to store the structure of the entire tree. Each node is a 64-bit long integer, occupying 8 bytes of space. Figure 4 shows the bit field definition of the 64-bit long integer of the node. It is stratified according to the depth of the tree, and each layer occupies a row of the two-dimensional array (each layer is equivalent to a one-dimensional array). Since the two-dimensional array in Java is essentially composed of different one-dimensional arrays for each row, the entire two-dimensional array does not have to be a matrix (compared with programming languages such as C, whose two-dimensional array is a matrix with the number of elements in two dimensions multiplied in memory), and the number of nodes in each row can be different. Based on this, the embodiments of the present invention can further optimize the storage space.

[0124] 1.2) To save storage space, for each node in the floors[0] to floors[k - 1] layers, the hash value of the dictionary string is stored, that is, a 32-bit integer calculated according to the standard function hashcode() method of strings in Java. The calculation method is that the total number starts from 0, from left to right, and is calculated through a loop. The loop content is the total number multiplied by 31 plus the numerical value (ASCII code) of the current character. Since the maximum value of a 32-bit integer is -1. When it exceeds, only the last 32 bits will be retained. For example, for the class name: io.netty.buffer.AbstractByteBuf, the hash value of the substring "io" is stored in the node in the floors[0] layer, the hash value of the substring "io.netty" is stored in the node in the floors[1] layer, the hash value of the substring "io.netty.buffer" is stored in the node in the floors[2] layer, and the hash value of the substring "io.netty.buffer.AbstractByteBuf" is stored in the node in the floors[3] layer.

[0125] 1.3) Since the hash values generated by the standard function hashcode() method of strings in Java have a probability of collision exceeding 50% after reaching about 50,000 strings, and usually there may be 10,000 classes in 1 jar package in a project, and there may be multiple jar packages in the project, the class name strings are very likely to reach 50,000 and may even reach millions. Therefore, in order to avoid collisions, for the nodes in floors[k], another hash algorithm needs to be used for double verification. The method is to only take the string formed by the first letters of the nodes when stratifying and execute the hashcode() method to generate the hash value. For example, for the class name: io.netty.buffer.AbstractByteBuf, the hash value of the substring "io" is stored in the node in the floors[0] layer, the hash value of the substring "io.netty" is stored in the node in the floors[1] layer, the hash value of the substring "io.netty.buffer" is stored in the node in the floors[2] layer, and the hash value of the substring "io.netty.buffer.AbstractByteBuf" is stored in the node in the floors[3] layer. Assuming that the floors[] array has 11 layers, that is, k = 10, the hash value of the string "i.n.b.AbstractByteBuf" used to generate the verification hash value corresponding to the class name io.netty.buffer.AbstractByteBuf is stored in the floors[k] layer.

[0126] Preferably, the calculation is performed following the retrieval path of the tree (i.e., the calculations in steps 1.2 and 1.3 are synchronously calculated in the same loop) rather than generating a complete string and then executing hashcode(), thereby reducing the number of loops and improving the calculation efficiency. This verification hash value is placed at the bottom layer of the tree, and the leaf nodes store the indices corresponding to their verification hashes. For example, the hash value corresponding to the tail node is in the form of i.n.b.AbstractByteBuf, and its hash algorithm is the hash value of "i.n.b." multiplied by 31 plus the hash value of AbstractByteBuf. The java hashcode() method calculates character by character. The hash value calculation for the entire string "io.netty.buffer.AbstractByteBuf" is performed during the character-by-character scan. Therefore, it can be compared that if the java algorithm is directly called, for these 31 characters, the string is traversed 3 times, and the total calculation time complexity is 31 + 31 + 6 + 15 = 83. However, if the calculation is performed during the scan, the string is traversed only once, thus saving efficiency.

[0127] 1.4) Let the maximum depth of the tree be k, then the first dimension of the array is defined as 0 to k. That is, in java, space is allocated as follows: long [][] floors = new long[k + 1][]; where the last layer floors[k] stores the verification hash value described in 1.3.

[0128] 2. The data structure of a single node is as Figure 4 shown. Let the long integer value of the node be E. The 64-bit long integer is divided into four main regions from left to right:

[0129] 2.1) Bit63: Leaf node flag, occupying 1 bit. Since the highest bit in the long integer is the sign bit, it can be determined whether it is a leaf node by the result of E < 0.

[0130] 2.2) Bit62~Bit56: The number of consecutive leaf nodes after this node (including this node), occupying 7 bits. For non-leaf nodes, these 7 bits are 0. Since the maximum value that 7-bit binary can represent is only 127, it is set that the maximum number of consecutive leaf nodes recorded is 100. When the number of consecutive leaf nodes exceeds 100, the excess part starts counting from 1 again. This is mainly for the design of quickly skipping leaf nodes in depth-first search. The value of the number of consecutive nodes can be obtained through the bit operation of (E >>> 24) & 0x7F. For example, if the value of Bit62~Bit56 of the leaf node at index 0 of a one-dimensional array corresponding to a certain layer is 100, it means that leaf nodes exist at indices 0 to 99. If there are only these 100 leaf nodes in this layer, then the Bit62~Bit56 fields of the leaf nodes at indices 0 to 99 decrease from 100 to 1. If there are actually 10 consecutive leaf nodes behind, then the Bit62~Bit56 fields of the leaf nodes at indices 0 to 99 decrease from 100 to 1, and the Bit62~Bit56 fields of the leaf nodes at indices 100 to 109 decrease from 11 to 1.

[0131] 2.3) Bit55~Bit32: The starting subscript of the child nodes of the next layer of this node, occupying 24 bits. Numerically, it can record approximately 16 million node subscripts, so it is necessary to ensure that the last layer does not exceed 16 million (2^24). The starting subscript of the child nodes can be obtained through the bit operation of (E >>> 32) & 0x0FFFFFFF. The Bit55~Bit32 fields of any node in each layer record the starting subscript value of the child nodes of the next layer of that arbitrary node. By querying the Bit55~Bit32 fields of that arbitrary node and the next node of that arbitrary node in this layer, the starting subscript value and the ending subscript value of the child nodes of the next layer of that arbitrary node can be determined. Then, the ending subscript value of the child nodes of the next layer of the last node in a layer can be determined by the maximum subscript value of the layer where it is located. For each layer, the current maximum subscript value of that layer is recorded.

[0132] 2.4) Bit31~Bit0: The remaining lower 32 bits correspond to a 32-bit integer, which is used to store the hash value of the node string. The hash value can be obtained through the bit operation of E & 0xFFFFFFFF.

[0133] The designs of Bit62~Bit56 and Bit55~Bit32 are to reserve as many bits as possible to represent child nodes. For example, the 24 bits of Bit55~Bit32 can represent 16 million nodes. If the occupation of the number of consecutive leaf nodes after this node and the occupation of the starting subscript of the child nodes of the next layer of this node are evenly divided, then only [31 / 2] = 15 bits can be used to represent the child node subscript, so there can only be That is, there are only 32,768 child nodes.

[0134] 3. Arrangement order of each layer:

[0135] For the convenience of description and to avoid confusion, the index number in floors[] is used as the layer number in the text description. For example, floors[0] corresponds to the 0th layer, floors[k - 1] corresponds to the (k - 1)th layer, and floors[k] corresponds to the kth layer.

[0136] The topmost layer is the root node, that is, the layer represented by floors[0], which is the 0th layer. The nodes in this layer are sorted in ascending order according to the low 32 - bit hash value.

[0137] As Figure 3 shown, Figure 3 the hollow circle numbered 30 in Figure 3 represents the root node of a tree. There are three root nodes in the same layer as the hollow circle numbered 30 in Figure 3 which correspond to three trees respectively.

[0138] The last layer (floors[k], that is, floors[k][], which is the layer where the hash verification nodes are located) and the retrieval value (leaves[v]) do not need to be sorted. For each leaf node (floors[d][i]) retrieved, the p value of its subscript is incremented by 1 to store the verification hash value (hashcode0); if the retrieval value corresponding to the string (such as the class name) corresponding to the currently traversed path already exists in the leaves[] array, directly use the index value corresponding to its position in the leaves[] array as the v value. If the retrieval value corresponding to the string (such as the class name) corresponding to the currently traversed path does not exist in the leaves[] array, add the retrieval value corresponding to the string (such as the class name) corresponding to the currently traversed path to the free space in the leaves[] array, and use the index value corresponding to the position of this retrieval value in the leaves[] array as the v value. In Java, the corresponding source code is:

[0139] floors[d][i] |= (++p)<<32;

[0140] floors[k][p]=(v<<32)|(hashcode0 & 0xFFFFFFFFL);

[0141] Among them, floors[d][i] represents the node at the i-th position of the d-th layer of the currently retrieved leaf node; p is the current maximum subscript value of floors[k], that is, the k-th layer; hashcode0 is the verification hash value of the currently retrieved leaf node; the values in leaves[] are the retrieval values to be retrieved.

[0142] When the above tree changes, the nodes in the verification hash layer do not need to be adjusted in position, and the subscript values within the leaf nodes do not need to be moved.

[0143] Continuous setting of leaf nodes:

[0144] The first bit of the leaf node is set to 1 through E | (1L <<63), where E is the current value of the leaf node;

[0145] For the tail node (hash verification node) of the floors[k] layer verification hash, there is no need to mark the number of continuous leaf nodes.

[0146] For the leaves that appear from the first layer to the k - 1 layer (floors[1]~floors[k - 1]), the high bits Bit62~Bit56 are the number of continuous leaf nodes, so that when retrieving a single layer, the leaf nodes can be quickly skipped and the retrieval can continue to the next layer.

[0147] Retrieval of the tree includes single-layer retrieval and leaf node retrieval;

[0148] Single - layer retrieval: In single - layer retrieval, breadth - first search and binary search are used. However, when the number of nodes in the remaining search range (for example, in binary search, there are upper and lower bounds. Suppose we need to search for n elements. Initially, it is 0~n. After one search, if it is in the first half, then it becomes 0~n / 2) is less than 16 (judged by the bit operation (i2>>4)<=i1, where i1 is the starting index and i2 is the ending index), directly traverse the remaining nodes.

[0149] Leaf - node retrieval: When the retrieved and matched node is a leaf node, the index x of floors[k] is obtained through (E >>> 32) & 0x0FFFFFFF for hash verification. If the verification passes, the index v is obtained through (floors[k][x] >>> 32) &0x0FFFFFFF, and the value of leaves[v] is returned. Here, >>> represents unsigned right - shift in the Java language.

[0150] Merging of two forests:

[0151] For each layer floors[d], first initialize a one - dimensional array with a size equal to the sum of the array sizes of the two forests at that layer;

[0152] The nodes in the first - layer floors[0] are merged using merge sort;

[0153] When the sorting of the parent nodes is completed and written into the array, their child nodes are processed in the order of left - subtree first using depth - first search. Within the range of the same parent node for each layer, merge sort is used, and the maximum index value of each layer is recorded and incremented. This maximum index value is equivalent to recording the continuously occupied space of the one - dimensional array of each layer starting from index 0. Based on this maximum index value, the idle space at the rear of each layer can be deleted to save memory, or the position where a new node can be written can be determined when adding a new node.

[0154] When merging to the leaf nodes, start recording the counter of consecutive leaf nodes at the current layer. Until the counter is non - zero and the node is a non - leaf node, or the counter reaches 100, starting from the first recorded leaf node, assign values to bits 62~56 of the high - order Bit. For each leaf node assigned, the counter is decremented by 1.

[0155] After the entire tree merging is completed, according to the maximum index value of each layer, resize the one - dimensional array of each layer floors[d] to have no subsequent blank elements.

[0156] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of the present disclosure. The appended method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy recited.

[0157] In the above detailed description, various features are combined in a single embodiment to simplify the present disclosure. This disclosure method should not be construed as reflecting an intention that the embodiments of the claimed subject matter require more features than those clearly recited in each claim. On the contrary, as reflected in the appended claims, the present invention lies in a state with fewer features than all the features of the disclosed single embodiment. Therefore, the appended claims are hereby expressly incorporated into the detailed description, where each claim stands alone as a separate preferred embodiment of the present invention.

[0158] The above-described disclosed embodiments are described to enable any person skilled in the art to implement or use the present invention. For those skilled in the art, various modification methods of these embodiments are obvious, and the general principles defined herein can also be applied to other embodiments without departing from the spirit and scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0159] The above description includes examples of one or more embodiments. Of course, it is impossible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but those of ordinary skill in the art should recognize that the various embodiments can be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications, and variations that fall within the scope of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, the coverage of this term is similar to the term "including". In addition, any term "or" used in the claims or the specification is to mean "non-exclusive or".

[0160] Those skilled in the art can also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention can be implemented by electronic hardware, computer software, or a combination of both. To clearly show the interchangeability of hardware and software, the above-mentioned various illustrative components, units, and steps have been generally described in terms of their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art can use various methods to implement the described functions for each specific application, but such implementation should not be construed as exceeding the scope protected by the embodiments of the present invention.

[0161] In the embodiments of the present invention, the various illustrative logical blocks or units can be implemented or operate the described functions through a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of the above designs. The general-purpose processor can be a microprocessor. Optionally, the general-purpose processor can also be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, multiple microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0162] The steps of the methods or algorithms described in the embodiments of the present invention can be directly embedded in hardware, software modules executed by a processor, or a combination of both. The software modules can be stored in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and the storage medium can be disposed in an ASIC, and the ASIC can be disposed in a user terminal. Optionally, the processor and the storage medium can also be disposed in different components of the user terminal.

[0163] In one or more exemplary designs, the functions described in embodiments of the present invention may be implemented in hardware, software, firmware, or any combination of the three. If implemented in software, these functions may be stored on a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. A computer-readable medium includes both computer storage media and communication media that facilitate transfer of a computer program from one place to another. The storage media may be any available media that can be accessed by a general-purpose or special-purpose computer. For example, such computer-readable media may include, but are not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store program code in the form of instructions or data structures and that can be read by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. In addition, any connection can be properly defined as a computer-readable medium, for example, if software is transmitted from a website, server, or other remote source via a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wirelessly, such as infrared, wireless, and microwave, it is also included in the defined computer-readable medium. The disks (disk) and discs (disc) include compact disks, laser disks, optical disks, DVDs, floppy disks, and Blu-ray disks. Disks typically reproduce data magnetically, while discs typically reproduce data optically using lasers. The above combinations may also be included in the computer-readable medium.

[0164] The specific embodiments described above have further elaborated on the objectives, technical solutions, and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for creating a string matching retrieval structure, characterized in that: include: For each target string among the multiple target strings, generating multiple node strings and verification strings corresponding to the target string; wherein the multiple node strings corresponding to the same target string have a sequence; Calculating hash values of multiple node strings corresponding to each target string, and storing hash values of all node strings corresponding to target strings with the same first-order node string in the multiple target strings into a tree structure according to a predecessor-successor relationship between all corresponding node strings, to obtain a forest structure corresponding to the multiple target strings; Associating a child node with each leaf node of each tree structure as a hash check node of a target string corresponding to the leaf node, calculating a hash value of a check string corresponding to the target string, and storing the hash value of the check string corresponding to the target string in the hash check node of the target string; Associating the hash check node of the target string with a target search value corresponding to the target string; wherein each target search value has a preset association relationship with at least one target string; Among the multiple node strings corresponding to the same target string, the node string with the suffix position includes all the information of all the node strings with the suffix position preceding the node string; the verification string corresponding to the target string includes all the information of at least one node string among the multiple node strings corresponding to the target string and partial information of each of the remaining node strings; The root nodes of different tree structures store hash values of different first-order node strings; each leaf node of the tree structure uniquely corresponds to a target string; each leaf node of the tree structure stores the hash value of the last-order node string of the target string corresponding to the leaf node; The root nodes of the tree structures in the forest structure are sorted according to the stored hash values, and the child nodes of the same layer under each parent node in each tree structure are sorted according to the stored hash values.

2. The method for creating a string matching retrieval structure according to claim 1, wherein: Generate multiple node strings and verification strings corresponding to the target string, including: Divide the target string into N sequentially arranged substrings; The first n substrings of the N sequentially arranged substrings are sequentially concatenated to form an n-th node string of the target string, thereby obtaining a plurality of node strings corresponding to the target string.

3. The method for creating a string matching retrieval structure according to claim 2, wherein: Generate multiple node strings and verification strings corresponding to the target string, including: The first letter of the first N-1 substrings is concatenated with the Nth subcharacter in sequence to obtain a verification string corresponding to the target string.

4. The method for creating a string matching retrieval structure according to claim 1, wherein: Calculating hash values of multiple node strings corresponding to each target string, and storing hash values of all node strings corresponding to target strings with the same first-order node string in the multiple target strings into a tree structure according to a predecessor-successor relationship between all corresponding node strings, to obtain a forest structure corresponding to the multiple target strings, including: Aggregate and sort the multiple target strings in the order in which the same node strings appear; Obtain each target string in the order in which the multiple target strings are aggregated and sorted, traverse the multiple node strings corresponding to the target string in order, and calculate the hash value of each node string; Traverse the multiple node strings corresponding to the target string in order, and search for the hash value of the currently traversed i-th node string in the i-th row of the one-dimensional array floors[i][] of the two-dimensional array floors[][]; where i increments from 0 to the number of all node strings corresponding to the target string - 1; If the hash value of the node string at the i-th position is not found in the one-dimensional array floors[i][] of the i-th row, the hash value of the node string at the i-th position is written into the one-dimensional array floors[i][ ], And when the node string at the i-th order is not the last node string of the target string, the number of used storage locations corresponding to the one-dimensional array floors[i+1][] at the i+1th row of the two-dimensional array floors[][] is replaced by Write to the one-dimensional array floors[i][ ] from the 32nd to the 55th position, in the case where the node string at the i-th order is the last node string of the target string, the one-dimensional array floors[i][ ] is set to 1; Increment the number of used storage locations corresponding to the one-dimensional array floors[i][] ; Among them, the two-dimensional array floors[][] is created in advance in Java to store the forest structure, and the initial number of used storage locations of each row of the one-dimensional array of the two-dimensional array floors[][] is initialized to 0; the sequence numbers of the multiple node strings corresponding to the target string are numbered sequentially starting from 0.

5. The method for creating a string matching retrieval structure according to claim 4, wherein: Associating a child node with each leaf node of each tree structure as a hash check node of a target string corresponding to the leaf node, calculating a hash value of a check string corresponding to the target string, and storing the hash value of the check string corresponding to the target string in the hash check node of the target string, including: In the case where the node string at the i-th order is the last node string of the target string, the method further includes: Return the number of used storage locations in the one-dimensional array floors[k][] of the kth row of the two-dimensional array floors[][] Write to the one-dimensional array floors[i][ ] from the 32nd to the 55th bit, write the hash value of the verification string corresponding to the target string into the one-dimensional array floors[k][ ]; Increment the number of used storage locations in the one-dimensional array floors[k][] ; Where k is the maximum number of node strings of all target strings.

6. The method for creating a string matching retrieval structure according to claim 5, wherein: Associating the hash check node of the target character string with a target search value corresponding to the target character string, including: In the case where the node string at the i-th order is the last node string of the target string, the method further includes: If it is determined that the target search value corresponding to the target string does not exist in the target search value array leaves[], the target search value corresponding to the target string is added to the target search value array leaves[], and the number of used storage locations v of the target search value array leaves is incremented; When it is determined that the target search value array leaves[] contains the target search value corresponding to the target string, the index value of the storage position of the target search value corresponding to the target string in the target search value array leaves[] is written into the one-dimensional array floors[k][ ] from 32nd to 55th; Here, a target retrieval value array leaves[] is created in advance in Java, and the number of used storage locations v of the target retrieval value array leaves[] is set to 0.

7. The method for creating a string matching retrieval structure according to claim 6, wherein: The method further comprises: For each element in the one-dimensional array floors[0][] at row 0 of the two-dimensional array floors[][], sort the elements in the one-dimensional array floors[0][] in ascending order according to the hash value stored in the lower 32 bits of the element; Take each one-dimensional array floors[d][] from the 1st row to the k-1th row of the two-dimensional array floors[][] as the child node layer, each element of the one-dimensional array floors[d][] as the child node element, take the one-dimensional array floors[d-1][] as the parent node layer of the one-dimensional array floors[d][], each element in the one-dimensional array floors[d-1][] as the parent node element, and determine the child node elements in the one-dimensional array floors[d][] with the parent node element in the one-dimensional array floors[d-1][] as the parent node according to the values corresponding to the 32nd to 55th bits of each parent node element in the one-dimensional array floors[d-1][], gather and group all the child node elements in the one-dimensional array floors[d][] according to the order of the corresponding parent node elements in the one-dimensional array floors[d-1][], and sort the child node elements in the one-dimensional array floors[d][] in each group in ascending order according to the hash value of the lower 32 bits of the child node elements; According to the highest bit of each element, count the number of consecutive leaf nodes in each row of the one-dimensional array from the 1st row to the k-1th row in the two-dimensional array floors[][], and write the 56th to 62nd bits of the element where each leaf node is located to the number of consecutive leaf nodes starting with the leaf node, with a period of 100; among them, the elements with the highest bit marked as 1 in the one-dimensional array from the 1st row to the k-1th row in the two-dimensional array floors[][] correspond to the leaf nodes of the tree structure; According to the number of used storage locations of each row of the one-dimensional array in the two-dimensional array floors[][], the unused storage space at the end of each row of the one-dimensional array is reduced and released.

8. The method for creating a string matching retrieval structure according to claim 1, wherein: The method further comprises merging the two to-be-merged forest structures according to the following steps: Define a two-dimensional array type target forest structure array in Java in advance, and initialize the number of elements in each row of the one-dimensional array of the target forest structure array to the sum of the number of elements in the corresponding rows of the two forest structures to be merged; Merge all elements in row 0 of the two forest structures to be merged by performing merge sort on the lower 32-bit values of the elements, and write all the sorted elements into row 0 of the target forest structure array; The child nodes of the node corresponding to each element in row 0 of the target forest structure array that has been written are processed in a depth-first order with left subtree first, a merge sort is used within the interval under the same parent node in each layer, and the maximum subscript value of each layer is recorded and incremented; wherein the maximum subscript value records the continuous occupied space of the one-dimensional array of each layer starting from index 0, and is used to delete the free space at the end of each layer according to the maximum subscript value or to determine the writing position of a new node when adding a new node; When merging into a leaf node, the layer starts recording the counters of consecutive leaf nodes until the counter is not 0 and the node is a non-leaf node, or the counter reaches 100. Starting from the first recorded leaf node, the 56th to 62nd bits are assigned values. Each time a leaf node is assigned, the counter is reduced by 1. After the two forest structures to be merged are merged, the one-dimensional array of each layer is reduced to no subsequent blank elements according to the maximum subscript value of each layer of the target forest structure array.

9. A string matching retrieval method, characterized in that: include: Generate multiple node strings and verification strings corresponding to the string to be retrieved, and calculate hash values of the multiple node strings and verification strings; traversing the forest structure created by the string matching retrieval structure creation method according to any one of claims 1 to 8 using a binary search method according to a breadth-first and then left-side depth-first strategy based on hash values of multiple node strings corresponding to the string to be searched, and determining a leaf node corresponding to the string to be searched in the forest structure; Obtaining a verification hash value from the hash verification node associated with the leaf node corresponding to the to-be-searched string, and comparing it with the hash value of the verification string corresponding to the to-be-searched string; When the hash value of the verification string corresponding to the to-be-searched string is equal to the verification hash value, a target search value associated with the hash verification node associated with the leaf node corresponding to the to-be-searched string is obtained as a search result of the to-be-searched string.

Images