A method for constructing a sparse lookup table

Abstract

The present invention provides a method for constructing a sparse lookup table, which belongs to the technical fields of computer technology, data structure and algorithm. The sparse table reuses parent nodes and child nodes at the same time, and i1 ],[0,1,2,...,n i2 ],...,[0,1,2,...,n im ]]→V i When the characteristic sequence of " is converted into a sparse lookup table, a directed acyclic graph is constructed, which occupies as little storage space as possible to store data and ensures that the computer can access the corresponding value in a constant time by searching on the directed acyclic graph. The present invention can achieve the optimization of storage space resource utilization.

Description

Technical Field

[0001] The present invention relates to a new data structure of a lookup table, in particular to a method for constructing a data structure of a lookup table with sparse storage space, and belongs to the technical field related to computer technology, data structure and algorithm. Background Art

[0002] In computer science, a lookup table is a basic data structure that traditionally uses a high-dimensional array to store data and uses an index to access the data. For example, a two-dimensional array is formally equivalent to a table with a certain number of rows and columns, and any element in the table can be uniquely accessed based on the row and column indexes. Since extracting values ​​from memory is often much faster than complex calculations, the speed improvement obtained in this way is very significant. However, in some specific scenarios, the storage space consumed by the lookup table is too large and unacceptable. Nevertheless, according to the actual problems faced and the characteristics of the data, by improving the data structure of the lookup table, limited resources can be fully utilized to obtain the same advantages as the lookup table, that is, less computing resources are consumed and the access time is constant.

[0003] The following will use a specific example to illustrate the characteristics of the problem solved by the method of the present invention, and compare the applicability of related background technologies on this problem.

[0004] In a real scenario, we face the following problem: how to index a value based on a feature sequence. For example, if the feature sequence is "it is a fruit, it is large, green, has stripes, is sweet, and has a lot of water", the value "watermelon" should be indexed based on these features; if the feature sequence is "it is a fruit, it is not large, red, has no stripes, is sweet or sour, and has a moderate amount of water", the value "apple" should be indexed based on these features; if the feature sequence is "it is a fruit, it is not large, orange, has no stripes, is sweet or sour, and has a moderate amount of water", the feature "orange" should be indexed based on these features. If we represent "is / is not a fruit" with "0 / 1", "large / small size" with "0 / 1", "green / red / orange" with "0 / 1 / 2", "with stripes / no stripes" with "0 / 1", "sweet / sour" with "0 / 1", and "more / moderate / less water" with "0 / 1 / 2", the features connected by the logical conjunction "or" are represented by a "list" (for example, "sweet or sour" is represented by [0,1], and "more or moderate or less water" is represented by [0,1,2]), the feature sequence is represented by a "list", and the feature sequence and its corresponding value are connected by the symbol "→", then the above three features should be represented as "[0,0,0,0,0,0,0]→watermelon", "[0,1,1,1,[0,1],1]→apple", and "[0,1,2,1,[0,1],1]→orange". Then, when the feature sequence "[0, 1, 2, 1, 0, 1]" is given, since "[0, 1, 2, 1, [0, 1], 1] → orange", we know that the value corresponding to this feature sequence is "orange". Therefore, the problem we face can be abstracted as, given a series of feature sequences and their corresponding values, such as "[0, 0, 0, 0, 0] → watermelon, [0, 1, 1, 1, [0, 1], 1] → apple, [0, 1, 2, 1, [0, 1], 1] → orange", given another feature, such as "[0, 1, 2, 1, 0, 1]", how to index to the value corresponding to the latter, such as "orange".

[0005] The most common technical solution is to use a high-dimensional array to store a "lookup table", store the values ​​corresponding to all possible features in the array, and index the values ​​by array subscript access. For example, for the above example, a 2×2×3×2×2×3 array with a shape of 2×2×3×2×3 must be stored in memory, and the value "watermelon" is written at position [0, 0, 0, 0, 0] of the array, the value "apple" is written at positions [0, 1, 1, 1, 0, 1] and [0, 1, 1, 1, 1, 1], the value "orange" is written at positions [0, 1, 2, 1, 0, 1] and [0, 1, 2, 1, 1, 1], and the value "null" is written at the remaining positions. In this way, given another feature [0, 1, 2, 1, 0, 1], the correct value "orange" can be indexed by accessing the array using the feature as the subscript of the array.

[0006] An improved storage method is to use sparse tables for storage. Sparse tables use the sparsity of data in a certain dimension to compress the storage space of the array. For example, for Figure 1 In the densely stored table shown in a, since the data is dense in columns 0, 3, 6, and 8, and all other columns are 0, we can discard the columns that are all 0 and only keep the columns that contain 1, and store the column labels, and get the following: Figure 1 b shows a sparsely stored table.

[0007] Another more common sparse storage method is to use a prefix tree (Trie) to store. A prefix tree is a tree-like data structure that was originally used for string matching. In the above example scenario, given multiple feature sequences, the prefix tree merges those parts with the same "prefix" into one node, thereby compressing the storage space and achieving constant time access to the value. The prefix tree corresponding to the above scenario is as follows Figure 2 shown.

[0008] However, if the actual scenario is much more complicated than the above example scenario, the traditional method is not applicable. For example, suppose you need to use a list of length 20 as a feature sequence to index a specific value, and the value range of each number in the feature sequence is an integer between 0 and 9, and the type of each value to be accessed is char (occupying 1 byte of memory), then this lookup table will require 10 20 Bytes of storage space are needed, and such astronomical resource usage is unacceptable in engineering. Therefore, the "lookup table" method is not applicable at this time. In this example, the traditional sparse table is not applicable because the data is not empty in some positions and has values ​​in a few positions, although all positions have values ​​and have the same value; in this example, the prefix tree is not applicable because if the prefix tree is used, the feature sequence must first be split into 10 20 feature sequences, and then construct a prefix tree. Even if the split is successful and enters the prefix tree construction phase, the prefix tree still needs about 10+10 2 +10 3 +...+10 20 The storage space of the unit is used to store all nodes. This is almost impossible in engineering practice. And similar complex scenarios are actually faced in engineering. Summary of the invention

[0009] The purpose of the present invention is to provide a method for constructing a sparse lookup table, which optimizes the utilization of storage space resources by simultaneously reusing parent nodes and reusing child nodes.

[0010] The sparse lookup table construction method involved in the present invention is to give a series of feature sequences and their corresponding values

[0011] [[0,1,2,...,n1,],[0,1,2,...,n 12 ],...,[0,1,2,...,n 1m ]]→V1

[0012] [[0, 1, 2, ..., n 21 ],[0,1,2,...,n 22 ],...,[0,1,2,...,n 2m ]]→V2

[0013] …

[0014] [[0, 1, 2, ..., n i1 ],[0,1,2,...,n i2 ],...,[0,1,2,...,n im ]]→V i

[0015] When constructing a directed acyclic graph, the graph takes up as little storage space as possible to store data, and ensures that the computer can access the corresponding value in constant time by searching on the directed acyclic graph. All nodes without parent nodes are called "starting nodes", all nodes without child nodes are called "ending nodes", and the number of edges from the starting node to any node is called the "depth" of the node. Then, the characteristic of the directed acyclic graph is that the depth of all the ending nodes is equal.

[0016] Figure 4 It is a simplified representation of node connections. Figure 4 a is a graph between two groups of nodes, where corresponding nodes are connected in pairs. This is a generally known representation method. To simplify the representation, all nodes in the first group are connected in pairs with all nodes in the second group. For convenience of representation, we use Figure 4 A simplified representation of b is to represent the first set of nodes in one node and the second set of nodes in another node, with arrows pointing from the former to the latter. For the convenience of the following description, when a set of nodes is represented as one node, the node is called a "node set".

[0017] Figure 5 are two examples of converting a single feature list into a sparse lookup table. A feature can be represented as [[0, 1, 2, ..., n i1 ],[0,1,2,...,n i2 ],...,[0,1,2,...,n im]], in this representation, [n] can be simplified to n. A feature sequence and its corresponding value can be directly converted into a sparse lookup table by converting each feature in the feature sequence into a node set, and connecting each node set in the order of the feature sequence until the last feature, and pointing the last feature to the node corresponding to the value.

[0018] The present invention provides a method for constructing a sparse lookup table, the specific steps of which include:

[0019] A. Convert each feature into a sparse lookup table by traversing each feature in the feature sequence, generating a node set, and using the feature as the content of the node set; according to the order of the features in the feature sequence, establish edges connecting the previous node set and the next node set in turn, thereby obtaining multiple sparse lookup tables;

[0020] B. The first sparse lookup table is used as the target sparse lookup table, and the second sparse lookup table is used as the sparse lookup table to be fused;

[0021] C. Merge the sparse lookup table to be fused into the target sparse lookup table to obtain a new sparse lookup table;

[0022] D. Use the new sparse lookup table obtained in step C as the target sparse lookup table, use the third to the last sparse lookup table as the sparse lookup table to be fused, and repeat step C;

[0023] E. After the last sparse lookup table is merged into the target lookup table, the resulting target sparse lookup table is the final constructed sparse lookup table.

[0024] Among them, in step B and step C, the specific steps of integrating the sparse lookup table to be integrated into the target sparse lookup table include:

[0025] 1) The first node set in the sparse lookup table to be fused is used as the node set to be fused, and the set consisting of all node sets with a depth of 1 in the target sparse lookup table is used as the set of target node sets;

[0026] 2) For each node set in the set of target node sets, calculate the number of parent node sets of the node set, calculate the intersection of the node set and the node set to be merged, calculate the difference between the node set and the node set to be merged, and calculate the difference between the node set to be merged and all node sets in the set of all target node sets. "The number of parent node sets of the target node set", "the difference between the target node set and the node set to be merged", "the difference between the node set to be merged and the target node set", and "whether the parent node set of the node set to be merged is in the target lookup table" are the four judgment bases;

[0027] 3) According to the four judgment criteria obtained in step 2), selectively split the target node set, split the node set to be fused, add or delete the incoming and outgoing edges of the target node set, and add or delete the incoming and outgoing edges of the node set to be fused;

[0028] 4) The second node set in the sparse lookup table to be fused is used as the node set to be fused. Correspondingly, the set consisting of all node sets with a depth of 2 in the target sparse lookup table is used as the set of target node sets. Repeat steps 2) and 3) until the depth is n, where n is the depth of the termination node.

[0029] Technical effects of the present invention:

[0030] The sparse lookup table construction method involved in the present invention constructs a directed acyclic graph by reusing parent nodes and child nodes at the same time. The directed acyclic graph is named "sparse lookup table". The graph takes up as little storage space as possible to store data, and ensures that the computer can access the corresponding value in constant time by searching on the directed acyclic graph. The graph takes up as little storage space as possible to store data, and ensures that the computer can access the corresponding value in constant time by searching on the directed acyclic graph. The sparse lookup table construction method of the present invention optimizes the utilization of storage space resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 An example of a traditional sparse table. (a) in the figure shows a densely stored table. A position with a value of 0 indicates that there is no value there, and a position with a value of 1 indicates that there is a value there. Positions with values ​​are sparse. (b) in the figure shows a sparsely stored table. The columns with values ​​(columns 0, 3, 6, and 8) are indexed by a linked list. Only columns with values ​​are stored in the dense table, so that the storage resources occupied by columns without values ​​are saved.

[0032] Figure 2 This is an example of a prefix tree. In a prefix tree, for sequences with a common "prefix", those parts of the common prefix are merged into one node for storage. In other words, the parent nodes of multiple nodes with a common "prefix" are "reused", thus avoiding storing the parent nodes with the same content multiple times and saving storage space.

[0033] Figure 3This is an example of the present invention converting a feature list into a sparse lookup table. Given the feature list "[[0, 1, 2, .., 9], [0, 1, 2, .., 9], ..., [0, 1, 2, ..9],]→V", the constructed sparse lookup table is shown in the figure. The reason why the sparse lookup table is superior to the prefix tree is that the prefix tree saves storage space by reusing the parent node, but its child nodes are not reused, that is, the constructed graph is a strict tree structure, and still occupies a considerable amount of redundant storage space resources when stored; in the sparse lookup table, for a group of nodes, the parent nodes with the same content are merged into one node, and the child nodes with the same content are also merged into one node, that is, the parent node and the child node are "reused", which saves storage space to a greater extent. For example, in the figure, for each node in the second layer, its parent node contains a node with content "0", so all the content of the parent node with content "0" is merged into one node; its child nodes contain a node with content "0", so all the content of the child nodes with content "0" is merged into one node.

[0034] Figure 4 Simplified representation of node connections; in figure a), the nodes of the first layer are connected to the nodes of the second layer in pairs. Then, in terms of representation, all the nodes of the first layer can be represented as one node set, and the nodes of the second layer can be represented as another node set. The two node sets are connected by a directed edge, as shown in figure b).

[0035] Figure 5 An example of converting a feature list into a sparse lookup table. The method is to traverse each feature in the feature sequence, generate a node set, and use the feature as the content of the node set. According to the order of the features in the feature sequence, the edges connecting the previous node set and the next node set are established in sequence.

[0036] Figure 6 This is an example of converting multiple feature lists into a sparse lookup table in the present invention. Given four feature sequences and their corresponding values ​​"[[0, 1, 2], 0, 1, [0, 1, 2]] → a, [[0, 1, 2], 0, [0, 1, 2], [0, 1, 2]] → b, [[0, 1, 2], 1, [0, 1, 2], [0, 1, 2]] → c, [0, 2, [0, 1, 2], [0, 1, 2]] → d", the constructed sparse lookup table is shown in the figure.

[0037] Figure 7 Schematic diagram of step 5.1 of a specific embodiment; in the left figure, the first row is a part of the target sparse lookup table splut1, and the second row is a part of the sparse lookup table splut2 to be fused. In the dotted box, the node set at the top is the node set of the target The node set at the bottom is the node set to be merged, Node2 d Node Set The number of parent nodes of is equal to 1, and With Node2 d The number of elements in the intersection of is greater than 0, and With Node2 d The number of elements in the difference of is equal to 0, and Node2 d The parent node set Node2 d-1 Not added to the target sparse lookup table splut1. The figure on the right is the result of fusion, the node set to be fused Node2 d is not added to the target's sparse lookup table splut1, so Node2 d There are no edges established with any node set in splut1.

[0038] Figure 8 Schematic diagram of step 5.2 of a specific embodiment; for the left figure, the first row is a part of the target sparse lookup table splut1, and the second row is a part of the sparse lookup table splut2 to be fused. In the dotted box, the node set at the top is the node set of the target The node set at the bottom is the node set to be merged, Node2 d Node Set The number of parent nodes of is equal to 1, and With Node2 d The number of elements in the intersection of is greater than 0, and With Node2 d The number of elements in the difference of is equal to 0, and Node2 d The parent node set Node2 d-1 It has been added to the target's sparse lookup table splut1. The figure on the right is the result of the fusion, the node set to be fused Node2 d is added to the target's sparse lookup table splut1 and is associated with its parent node set Node2 d-1 Create edges between corresponding sets of nodes in splut1.

[0039] Fig. 9 This is a schematic diagram of step 5.3 of a specific embodiment. For the left figure, the first row is a part of the target sparse lookup table splut1, and the second row is a part of the sparse lookup table splut2 to be fused. In the dotted box, the node set in the upper part is the node set of the target. The node set at the bottom is the node set to be merged, Node2 d Node Set The number of parent nodes of is equal to 1, and With Node2 d The number of elements in the intersection of is greater than 0, and With Node2 d The number of elements in the difference of Greater than 0, and Node2 d The parent node set Node2 d-1 Not added to the target sparse lookup table splut1. The figure on the right is the result of fusion, the node set to be fused Node2 d is added to the target's sparse lookup table splut1 and is associated with The parent node set of Create edges between them.

[0040] Fig.10 This is a schematic diagram of step 5.4 of a specific embodiment. For the left figure, the first row is a part of the target sparse lookup table splut1, and the second row is a part of the sparse lookup table splut2 to be fused. In the dotted box, the node set at the top is the node set of the target. The node set at the bottom is the node set to be merged, Node2 d Node Set The number of parent nodes of is equal to 1, and With Node2 d The number of elements in the intersection of is greater than 0, and With Node2 d The number of elements in the difference of Greater than 0, and Node2 d The parent node set Node2 d-1 has been added to the target's sparse lookup table splut1. The figure on the right is the result of the fusion, the node set Replaced by With Node2 d The difference of And the node set to be merged Node2 d is added to the target's sparse lookup table splut1 twice and is The parent node set of Create an edge between them and Node2 d The parent node set Node2 d-1 Create edges between corresponding sets of nodes in splut1.

[0041] Fig.11This is a schematic diagram of step 5.5 of a specific embodiment. For the left figure, the first row is a part of the target sparse lookup table splut1, and the second row is a part of the sparse lookup table splut2 to be fused. In the dotted box, the node set in the upper part is the node set of the target. The node set at the bottom is the node set to be merged, Node2 d Node Set The number of parent nodes of is greater than 1, and With Node2 d The number of elements in the intersection of is greater than 0, and With Node2 d The number of elements in the difference of is equal to 0, and Node2 d The parent node set Node2 d-1 Not added to the target sparse lookup table splut1. The figure on the right is the result of fusion, the node set to be fused Node2 d is not added to the target's sparse lookup table splut1, so Node2 d There are no edges established with any node set in splut1.

[0042] Fig.12 This is a schematic diagram of step 5.6 of a specific embodiment. For the left figure, the first row is a part of the target sparse lookup table splut1, and the second row is a part of the sparse lookup table splut2 to be fused. In the dotted box, the node set in the upper part is the node set of the target. The node set at the bottom is the node set to be merged, Node2 d Node Set The number of parent nodes of is greater than 1, and With Node2 d The number of elements in the intersection of is greater than 0, and With Node2 d The number of elements in the difference of is equal to 0, and Node2 d The parent node set Node2 d-1 It has been added to the target's sparse lookup table splut1. The figure on the right is the result of the fusion, the node set to be fused Node2 d is added to the target's sparse lookup table splut1 and is associated with its parent node set Node2 d-1 Create edges between corresponding sets of nodes in splut1.

[0043] Fig.13This is a schematic diagram of step 5.7 of a specific embodiment. For the left figure, the first row is a part of the target sparse lookup table splut1, and the second row is a part of the sparse lookup table splut2 to be fused. In the dotted box, the node set at the top is the node set of the target. The node set at the bottom is the node set to be merged, Node2 d Node Set The number of parent nodes of is greater than 1, and With Node2 d The number of elements in the intersection of is greater than 0, and With Node2 d The number of elements in the difference of Greater than 0, and Node2 d The parent node set Node2 d-1 Not added to the target's sparse lookup table splut1. The figure on the right is the result of the fusion, the node set Replaced by With Node2 d The difference of And the node set Node2d to be fused is added to the target sparse lookup table splut1 twice to establish the corresponding edges.

[0044] Fig.14 This is a schematic diagram of step 5.8 of a specific embodiment. For the left figure, the first row is a part of the target sparse lookup table splut1, and the second row is a part of the sparse lookup table splut2 to be fused. In the dotted box, the node set in the upper part is the node set of the target. The node set at the bottom is the node set to be merged, Node2 d Node Set The number of parent nodes of is greater than 1, and With Node2 d The number of elements in the intersection of is greater than 0, and With Node2 d The number of elements in the difference of Greater than 0, and Node2 d The parent node set Node2 d-1 has been added to the target's sparse lookup table splut1. The figure on the right is the result of the fusion, the node set Replaced by With Node2 d The difference of And the node set to be merged Node2 d It is added to the target's sparse lookup table splut1 twice and the corresponding edges are established.

[0045] Fig.15 This is a schematic diagram of step 5.9 of a specific embodiment. For the left figure, the first row is a part of the target sparse lookup table splut1, and the second row is a part of the sparse lookup table splut2 to be fused. In the dotted box, the node set in the upper part is the node set of the target. The node set at the bottom is the node set to be merged, Node2 d Node Set The number of parent nodes of is equal to 1, and With Node2 d The number of elements in the intersection of is equal to 0, and With Node2 d The number of elements in the difference of Greater than 0, and Node2 d The parent node set Node2 d-1 Not added to the target sparse lookup table splut1. The figure on the right is the result of fusion, the node set to be fused Node2 d is added to the target's sparse lookup table splut1 and is associated with The parent node set of Create edges between them.

[0046] Fig.16 This is a schematic diagram of step 5.10 of a specific embodiment. For the left figure, the first row is a part of the target sparse lookup table splut1, and the second row is a part of the sparse lookup table splut2 to be fused. In the dotted box, the node set in the upper part is the node set of the target. The node set at the bottom is the node set to be merged, Node2 d Node Set The number of parent nodes of is equal to 1, and With Node2 d The number of elements in the intersection of is equal to 0, and With Node2 d The number of elements in the difference of Greater than 0, and Node2 d The parent node set Noe2 d-1 It has been added to the target's sparse lookup table splut1. The figure on the right is the result of the fusion, the node set to be fused Node2 d Added to the target's sparse lookup table splut1, and its parent node set Node2 d-1 The corresponding node in splut1 and its Create edges between them.

[0047] Fig.17This is a schematic diagram of step 5.11 of a specific embodiment. For the left figure, the first row is a part of the target sparse lookup table splut1, and the second row is a part of the sparse lookup table splut2 to be fused. In the dotted box, the node set at the top is the node set of the target. The node set at the bottom is the node set to be merged, Node2 d Node Set The number of parent nodes of is greater than 1, and With Node2 d The number of elements in the intersection of is equal to 0, and With Node2 d The number of elements in the difference of Greater than 0, and Node2 d The parent node set Node2 d-1 Not added to the target sparse lookup table splut1. The figure on the right is the result of fusion, the node set to be fused Nd e2 d It is added to the target’s sparse lookup table splut1 and a corresponding edge is created.

[0048] Fig.18 This is a schematic diagram of step 5.12 of a specific embodiment. For the left figure, the first row is a part of the target sparse lookup table splut1, and the second row is a part of the sparse lookup table splut2 to be fused. In the dotted box, the node set in the upper part is the node set of the target. The node set at the bottom is the node set to be merged, Node2 d Node Set The number of parent nodes of is greater than 1, and With Node2 d The number of elements in the intersection of is equal to 0, and With Node2 d The number of elements in the difference of Greater than 0, and Node2 d The parent node set Node2 d-1 It has been added to the target's sparse lookup table splut1. The figure on the right is the result of the fusion, the node set to be fused Node2 d It is added to the target’s sparse lookup table splut1 and a corresponding edge is created. DETAILED DESCRIPTION

[0049] Embodiment 1:

[0050] Given a feature list “[[0, 1, 2, .., 9], [0, 1, 2, .., 9], ..., [0, 1, 2, .., 9],] → V” (where there are 20 features “[0, 1, 2, .., 9]” in the list), using the traditional lookup table to store it requires 10 20 Units of storage space are required, but the sparse lookup table is used for storage. This example only requires about 200 units of storage space to store all nodes and about 1900 units of storage space to store all edges. Figure 3 As shown. According to step A, the first feature is converted into a sparse lookup table. Since there is only one feature in the feature list, the first feature processed is the last feature, and step E is completed, that is, the construction of the coefficient lookup table is completed. It should be noted that Figure 3 The sparse lookup table in does not use Figure 4 b, but uses the simplified representation shown in Figure 4 The general representation shown in a.

[0051] Embodiment 2:

[0052] The process of converting multiple feature lists into a sparse lookup table is: given a series of feature sequences and their corresponding values

[0053] [[0, 1, 2, ..., n 11 ],[0,1,2,...,n 12 ],...,[0,1,2,...,n 1m ]]→V1

[0054] [[0, 1, 2, ..., n 21 ],[0,1,2,...,n 22 ],...,[0,1,2,...,n 2m ]]→V2

[0055] …

[0056] [[0, 1, 2, ..., n i1 ],[0,1,2,...,n i2 ],...,[0,1,2,...,n im ]]→V i

[0057] First, each feature sequence and its corresponding value are converted into a Figure 5The sparse lookup table shown in FIG. Next, take the first sparse lookup table, denoted as splut, merge the second sparse lookup table into the splut, and denoted the merged new sparse lookup table as splut. Sequentially take out the third to the i-th sparse lookup table and repeat the fusion process until the splut fused with the i-th sparse lookup table is obtained, which is the final sparse lookup table.

[0058] The above method of merging a sparse lookup table into another sparse lookup table is the core content of the present invention. The steps of merging are described below.

[0059] Given a sparse lookup table splut1 (the sparse lookup table as the target) and another sparse lookup table splut2 (the sparse lookup table to be fused) directly converted from the feature sequence:

[0060] Step 1, set d = 0, and let the length of the feature sequence corresponding to splut2 be D.

[0061] Step 2: If d is greater than or equal to the length of the feature sequence corresponding to splut2, jump to step 6; otherwise, jump to step 3.

[0062] Step 3, take out the dth node set in splut2 and record it as Node2 d Take all the node sets with depth d in splut1 and record the set they form as Remember S d The i-th node set in

[0063] Step 4: Traverse S one by one d Each node set in Compute node set With Node2 d The intersection of (The number of its elements is recorded as ), node set With Node2 d The difference of (The number of its elements is recorded as ),Node set Node2 d and The difference of (The number of its elements is recorded as ).

[0064] Step 5: Calculate the node set According to the above calculation results, the number of parent nodes of Node2 d Add splut1 and add and delete edges.

[0065] Step 5.1, such as Figure 7 If The number of parent nodes of is equal to 1, and is greater than 0, and is equal to 0, and Node2 d-1 If it is not added to splut1, then: Node2 is not added to d Add splut1; now, let d=d+1 and recursively execute process 1.

[0066] Step 5.2, such as Figure 8 If The number of parent nodes of is equal to 1, and is greater than 0, and is equal to 0, and Node2 d-1 It has been added to splut1, and its corresponding node in splut1 is recorded as Then: Node2 d Add it to splut1, and its corresponding node in splut1 is recorded as Establish point to edge; at this time, let d = d + 1 and recursively execute process 1.

[0067] Step 5.3, such as Fig. 9 If The number of parent nodes of is equal to 1, and is greater than 0, and Greater than 0, and Node2 d-1 Not added to splut1, then: Replace with At the same time, Node2 d Add it to splut1, and its corresponding node in splut1 is recorded as Establish point to edge; at this time, let d = d + 1 and recursively execute process 1.

[0068] Step 5.4, such as Fig.10 If The number of parent nodes of is equal to 1, and is greater than 0, and Greater than 0, and Node2 d-1 It has been added to splut1, and its corresponding node in splut1 is recorded as Then: Replace with At the same time, Node2 d Add it to splut1 twice, and the corresponding nodes in splut1 are recorded as and Establish point to The edge, build point to The edge, build Point to S d+1 All edges of all node sets in ; at this time, let d = d + 1 and recursively execute process 1.

[0069] Step 5.5, such as Fig.11 If The number of parent nodes of is greater than 1, and is greater than 0, and is equal to 0, and Node2 d-1 If it is not added to splut1, then: Node2 is not added to d Add splut1; now, let d=d+1 and recursively execute process 1.

[0070] Step 5.6, such as Fig.12 If The number of parent nodes of is greater than 1, and is greater than 0, and is equal to 0, and Node2 d-1 It has been added to splut1, and its corresponding node in splut1 is recorded as Then: Node2 d Add it to splut1, and its corresponding node in splut1 is recorded as Establish point to edge; at this time, let d = d + 1 and recursively execute process 1.

[0071] Step 5.7, such as Fig.13 If The number of parent nodes of is greater than 1, and is greater than 0, and Greater than 0, and Node2 d-1 Not added to splut1, then: All parent nodes are concentrated on Node2 d-1 The set of equal nodes is Will Replace with At the same time, Node2 d Add it to splut1 twice, and the corresponding nodes in splut1 are recorded as and Establish All parent nodes except All nodes other than All the edges of point to The edge, build Point to S d+1 All edges of all node sets in ; at this time, let d = d + 1 and recursively execute process 1.

[0072] Step 5.8, such as Fig.14 If The number of parent nodes of is greater than 1, and is greater than 0, and Greater than 0, and Node2 d-1 It has been added to splut1, and its corresponding node in splut1 is recorded as Then: Remember All parent nodes are concentrated on Node2 d-1 The set of equal nodes is Will Replace with At the same time, Node2 d Add it to splut1 twice, and the corresponding nodes in splut1 are recorded as and Establish All parent nodes except All nodes other than All the edges of point to The edge, build point to The edge, build Point to S d+1 All edges of all node sets in ; at this time, let d = d + 1 and recursively execute process 1.

[0073] Step 5.9, such as Fig.15 If The number of parent nodes of is equal to 1, and is equal to 0, and Greater than 0, and Node2 d-1 Not added to splut1, then: All parent nodes are concentrated on Node2 d-1 The set of equal nodes is Node2 d Add it to splut1, and its corresponding node in splut1 is recorded as Establish point to ; then, Add sput1 one by one and create edges in order. Jump to step 6.

[0074] Step 5.10, such as Fig.16 If The number of parent nodes of is equal to 1, and is equal to 0, and Greater than 0, and Node2 d-1 It has been added to splut1, and its corresponding node in splut1 is recorded as Then: Node2 d Add it to splut1, and its corresponding node in splut1 is recorded as Establish point to The edge, build Point to S d All edges of all node sets in Add sput1 one by one and create edges in order. Jump to step 6.

[0075] Step 5.11, such as Fig.17 If The number of parent nodes of is greater than 1, and is equal to 0, and Greater than 0, and Node2 d-1 Not added to splut1, then: All parent nodes are concentrated on Node2 d-1 The set of equal nodes is Node2 d Add it to splut1, and its corresponding node in splut1 is recorded as Establish point to ; then, Add sput1 one by one and create edges in order. Jump to step 6.

[0076] Step 5.12, such as Fig.18 If The number of parent nodes of is greater than 1, and is equal to 0, and Greater than 0, and Node2 d-1 It has been added to splut1, and its corresponding node in splut1 is recorded as Then: Node2 d Add it to splut1, and its corresponding node in splut1 is recorded as Establish point to The edge, build point to ; then, Add sput1 one by one and create edges in order. Jump to step 6.

[0077] Step 6, obtain the established coefficient lookup table splut1.

[0078] The "Process 1" described in the above steps includes Step 2 to Step 6.

[0079] It should be noted that the purpose of publishing the embodiments is to help further understand the present invention, but those skilled in the art can understand that various substitutions and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the contents disclosed in the embodiments, and the scope of protection claimed by the present invention shall be subject to the scope defined in the claims.

Claims

1. A method for constructing a sparse lookup table, by reusing parent nodes and child nodes at the same time, to construct a directed acyclic graph, the directed acyclic graph is named as a sparse lookup table, the graph occupies as little storage space as possible to store data, and ensures that a computer can access the corresponding value in a constant time by searching on the directed acyclic graph, characterized in that: The specific steps include: A. Convert each feature into a sparse lookup table by traversing each feature in the feature sequence, generating a node set, and using the feature as the content of the node set; according to the order of the features in the feature sequence, establish edges connecting the previous node set and the next node set in turn, thereby obtaining multiple sparse lookup tables; B. A sparse lookup table with the first sparse lookup table as the target, and the second lookup table as the sparse lookup table to be fused; C. Merge the sparse lookup table to be fused into the target sparse lookup table to obtain a new sparse lookup table; D. Use the new sparse lookup table obtained in step C as the target sparse lookup table, use the third to the last sparse lookup table as the sparse lookup table to be fused, and repeat step C; E. After the last sparse lookup table is merged into the target lookup table, the target sparse lookup table obtained is the final constructed sparse lookup table; In the above step C, the specific steps of integrating the sparse lookup table to be integrated into the target sparse lookup table include: 1) The first node set in the sparse lookup table to be fused is used as the node set to be fused, and the set consisting of all node sets with a depth of 1 in the target sparse lookup table is used as the set of target node sets; 2) For each node set in the set of target node sets, calculate the number of parent node sets of the node set, calculate the intersection of the node set and the node set to be fused, calculate the difference between the node set and the node set to be fused, and calculate the difference between the node set to be fused and all node sets in the set of all target node sets, and obtain four judgment bases; 3) According to the four judgment criteria obtained in step 2), selectively split the target node set, split the node set to be fused, add or delete the incoming and outgoing edges of the target node set, and add or delete the incoming and outgoing edges of the node set to be fused; 4) The second node set in the sparse lookup table to be fused is used as the node set to be fused. Correspondingly, the set consisting of all node sets with a depth of 2 in the target sparse lookup table is used as the set of target node sets. Repeat steps 2) and 3) until the depth is n, where n is the depth of the termination node.

2. The method for constructing a sparse lookup table according to claim 1, wherein: The target sparse lookup table is denoted as splut1, and the sparse lookup table to be fused is denoted as splut2; for the dth node set in splut2, the node set is denoted as Node2 d , all nodes with depth d in splut1, the set they constitute is Remember S d The i-th node set in Calculate the intersection of the node set and the node set to be fused, and the number of elements is recorded as Calculate the difference between the node set and the node set to be fused, and the number of elements is recorded as Calculate the difference between the node set to be fused and all the node sets in the set of all target node sets, and the number of elements is recorded as Combined with the number of parent node sets of the node set, 12 fusion methods of merging the sparse lookup table to be fused into the target sparse lookup table are obtained.

3. The method for constructing a sparse lookup table according to claim 2, wherein: like The number of parent nodes of is equal to 1, and is greater than 0, and is equal to 0, and Node2 d-1 If it is not added to splut1, then: Node2 is not added to d Add splut1.

4. The method for constructing a sparse lookup table according to claim 2, wherein: like The number of parent nodes of is equal to 1, and is greater than 0, and Greater than 0, and Node2 d-1 Not added to splut1, then: Replace with At the same time, Node2 d Add it to splut1, and its corresponding node in splut1 is recorded as Establish point to edge.

5. The method for constructing a sparse lookup table according to claim 2, wherein: like The number of parent nodes of is equal to 1, and is greater than 0, and Greater than 0, and Node2 d-1 It has been added to splut1, and its corresponding node in splut1 is recorded as Then: Replace with At the same time, Node2 d Add it to splut1 twice, and the corresponding nodes in splut1 are recorded as and Establish point to The edge, build point to The edge, build Point to S d+1 All edges of all node sets in .

6. The method for constructing a sparse lookup table according to claim 2, wherein: like The number of parent nodes of is greater than 1, and is greater than 0, and is equal to 0, and Node2 d-1 If it is not added to splut1, then: Node2 is not added to d Add splut1.

7. The method for constructing a sparse lookup table according to claim 2, wherein: like The number of parent nodes of is greater than 1, and is greater than 0, and is equal to 0, and Node2 d-1 It has been added to splut1, and its corresponding node in splut1 is recorded as Then: Node2 d Add it to splut1, and its corresponding node in splut1 is recorded as Establish point to edge.

8. The method for constructing a sparse lookup table according to claim 2, wherein: like The number of parent nodes of is greater than 1, and is greater than 0, and Greater than 0, and Node2 d-1 Not added to splut1, then: All parent nodes are concentrated on Node2 d-1 The set of equal nodes is Will Replace with At the same time, Node2 d Add it to splut1 twice, and the corresponding nodes in splut1 are recorded as and Establish All parent nodes except All nodes other than All the edges of point to The edge, build Point to S d+1 All edges of all node sets in .

9. The method for constructing a sparse lookup table according to claim 2, wherein: like The number of parent nodes of is greater than 1, and is greater than 0, and Greater than 0, and Node2 d-1 It has been added to splut1, and its corresponding node in splut1 is recorded as Then: Remember All parent nodes are concentrated on Node2 d-1 The set of equal nodes is Will Replace with At the same time, Node2 d Add it to splut1 twice, and the corresponding nodes in splut1 are recorded as and Establish All parent nodes except All nodes other than All the edges of point to The edge, build point to The edge, build Point to S d+1 All edges of all node sets in .

10. The method for constructing a sparse lookup table according to claim 2, wherein: like The number of parent nodes of is equal to 1, and is equal to 0, and Greater than 0, and Node2 d-1 Not added to splut1, then: All parent nodes are concentrated on Node2 d-1 The set of equal nodes is Node2 d Add it to splut1, and its corresponding node in splut1 is recorded as Establish point to ; then, Add sput1 one by one and create edges in order.

11. The method for constructing a sparse lookup table according to claim 2, wherein: like The number of parent nodes of is equal to 1, and is equal to 0, and Greater than 0, and Node2 d-1 It has been added to splut1, and its corresponding node in splut1 is recorded as Then: Node2 d Add it to splut1, and its corresponding node in splut1 is recorded as Establish point to The edge, build Point to S d All edges of all node sets in Add sput1 one by one and create edges in order.

12. The method for constructing a sparse lookup table according to claim 2, wherein: like The number of parent nodes of is greater than 1, and is equal to 0, and Greater than 0, and Node2 d-1 Not added to splut1, then: All parent nodes are concentrated on Node2 d-1 The set of equal nodes is Node2 d Add it to splut1, and its corresponding node in splut1 is recorded as Establish point to ; then, Add sput1 one by one and create edges in order.

13. The method for constructing a sparse lookup table according to claim 2, wherein: like The number of parent nodes of is greater than 1, and is equal to 0, and Greater than 0, and Node2 d-1 It has been added to splut1, and its corresponding node in splut1 is recorded as Then: Node2 d Add it to splut1, and its corresponding node in splut1 is recorded as Establish point to The edge, build point to ; then, Add sput1 one by one and create edges in order.

Images