Data processing methods, data processing devices, electronic devices, and readable storage media

Abstract

This application discloses a data processing method, a data processing apparatus, an electronic device, and a readable storage medium. The data processing method includes: compressing the storage space of a trie structure according to the traversal order of left node, root node, and right node to obtain a target array. Each element of the target array corresponds to a node of the trie structure. The nodes of the trie structure are only used to store the branching points of data strings, and the node pointers of the nodes are used to indicate the branching direction of the nodes; receiving a query request for the target data string, the query request carrying the string identifier of the target data string; and indexing the target data string according to the branching points and node pointers corresponding to the elements of the target array.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, specifically relating to a data processing method, a data processing device, an electronic device, and a readable storage medium. Background Technology

[0002] Currently, computer systems widely use indexed data structures such as B-trees, red-black trees, and Trie trees for data indexing, for example, to retrieve passwords by username or verify login identity. However, when using indexed data structures for data indexing, the memory consumption increases significantly as the amount of data to be indexed grows, leading to uncertain space requirements and problems with low memory utilization and low data indexing efficiency. Summary of the Invention

[0003] The purpose of this application is to provide a data processing method, a data processing apparatus, an electronic device, and a readable storage medium, aiming to at least solve the problems of uncertain space requirements, low memory utilization efficiency, and low data indexing efficiency in the prior art.

[0004] In a first aspect, embodiments of this application provide a data processing method, which includes:

[0005] The target array is obtained by compressing the storage space of the trie structure according to the traversal order of left node, root node, and right node. Each element of the target array corresponds to a node of the trie structure. The nodes of the trie structure are only used to store the branching points of the data string, and the node pointer is used to indicate the branching direction of the node.

[0006] Receive a query request for the target data string, which carries the string identifier of the target data string;

[0007] The target data string is indexed based on the branch points and node pointers corresponding to the elements of the target array.

[0008] Secondly, embodiments of this application provide a data processing apparatus, which includes:

[0009] The conversion module is used to compress the storage space of the trie structure according to the traversal order of left node, root node, and right node to obtain the target array. Each element of the target array corresponds to a node of the trie structure. The nodes of the trie structure are only used to store the branching points of the data string, and the node pointer is used to indicate the branching direction of the node.

[0010] The receiving module is used to receive query requests for target data strings, which carry string identifiers of the target data strings.

[0011] The indexing module is used to index the target data string based on the branch points and node pointers corresponding to the elements of the target array.

[0012] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions implementing the steps of the method as described in the first aspect when executed by the processor.

[0013] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, and when the program or instructions are executed by a processor, they implement the steps of the method as described in the first aspect.

[0014] Fifthly, embodiments of this application provide a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the method as described in the first aspect.

[0015] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method as described in the first aspect.

[0016] In this embodiment, the nodes of the trie structure (i.e., the trie tree structure) store the branching points of the data strings. Specifically, the nodes of the trie structure store the branching points between the (i+1)th and the ith data string, where the branching point is the bit where the (i+1)th and ith data strings differ in value, and i is a positive integer. This branching point is then stored on the nodes of the trie tree structure. Further, according to the inorder traversal method (i.e., first visiting the left subtree, then the root node, and finally the right subtree), the storage space of the trie structure is compressed into a target array. Further, upon receiving an index request for the target data string, the target array is partitioned according to the branching points and node pointers corresponding to the elements of this compressed target array to enable the query of the target data string to be queried. The index request carries the string identifier of the target data string, which is one of the (i+1)th data strings. In this embodiment, on the one hand, by storing only the branching points between data strings in the nodes of the Trie tree structure, instead of storing the complete strings, a fixed-length node is added for each inserted record, thus determining the space required. On the other hand, the Trie tree nodes are compressed using inorder traversal, reducing the storage space occupied and thereby improving memory utilization. Furthermore, the target array is divided according to the branching points and branch directions to realize the query of the target data string, which can significantly reduce the space consumption of the index and improve the data access efficiency. Attached Figure Description

[0017] Figure 1 This is a schematic flowchart of the data processing method according to an embodiment of this application;

[0018] Figure 2 This is one of the schematic diagrams of the Trie tree structure in the embodiments of this application;

[0019] Figure 3 This is a second schematic diagram of the Trie tree structure in an embodiment of this application;

[0020] Figure 4 This is the third schematic diagram of the Trie tree structure in the embodiments of this application;

[0021] Figure 5 This is one of the schematic diagrams of the Trie tree structure in related technologies;

[0022] Figure 6 This is the fourth schematic diagram of the Trie tree structure in the embodiments of this application;

[0023] Figure 7 This is the fifth schematic diagram of the Trie tree structure in the embodiments of this application;

[0024] Figure 8 This is the sixth schematic diagram of the Trie tree structure in the embodiments of this application;

[0025] Figure 9 This is the seventh schematic diagram of the Trie tree structure in the embodiments of this application;

[0026] Figure 10 This is one of the schematic diagrams of the clipping of the target array in the embodiments of this application;

[0027] Figure 11 This is the second schematic diagram of the clipping of the target array in an embodiment of this application;

[0028] Figure 12 This is the third schematic diagram of the clipping of the target array in the embodiments of this application;

[0029] Figure 13 This is the eighth schematic diagram of the Trie tree structure in the embodiments of this application;

[0030] Figure 14 This is the ninth schematic diagram of the Trie tree structure in the embodiments of this application;

[0031] Figure 15 This is the tenth schematic diagram of the Trie tree structure in the embodiments of this application;

[0032] Figure 16 This is eleventh of the schematic diagrams of the Trie tree structure in the embodiments of this application;

[0033] Figure 17 This is the twelfth schematic diagram of the Trie tree structure in the embodiments of this application;

[0034] Figure 18 This is the second schematic diagram of the Trie tree structure in related technologies;

[0035] Figure 19 This is a schematic block diagram of a data processing apparatus according to an embodiment of this application;

[0036] Figure 20 This is a schematic block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0037] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0038] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0039] The data processing method, data processing device, electronic device, and readable storage medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0040] This application provides a data processing method, such as... Figure 1 As shown, the method includes:

[0041] Step 102: Compress the storage space of the trie structure according to the traversal order of left node, root node, and right node to obtain the target array. Each element of the target array corresponds to a node of the trie structure. The nodes of the trie structure are only used to store the branching points of the data string, and the node pointers are used to indicate the branching direction of the node.

[0042] Step 104: Receive a query request for the target data string, which carries the string identifier of the target data string;

[0043] Step 106: Index the target data string according to the branch points and node pointers corresponding to the elements of the target array.

[0044] In this embodiment, the nodes of the trie structure (i.e., the trie tree structure) store the divergence points of the data strings. Specifically, the nodes of the trie structure store the divergence points between the (i+1)th and the ith data string. The divergence point is the bit in the (i+1)th data string that differs from the ith data string in value, where i is a positive integer. This divergence point is then stored on the nodes of the trie tree structure.

[0045] Furthermore, based on the inorder traversal method, that is, first visiting the left subtree, then visiting the root node, and finally visiting the right subtree, the storage space of the trie structure is compressed into the target array.

[0046] For example, for such Figure 2 The 11-node Trie tree structure shown, if represented using pointers according to relevant techniques, would be as follows: Figure 3 As shown, each node in the Trie tree structure is represented by one uint64_t (occupying 8 bytes) and two pointers (one pointer occupies 8 bytes on a 64-bit CPU). Leaf nodes are represented by uint64_t. The Trie tree structure consumes a total of 168 bytes, which is a significant waste of storage space. In this embodiment, through... Figure 4 The inorder traversal order shown will Figure 2 The Trie tree structure shown is compressed into a target array 2, 1, 2, 0, 4, 3, 4, 2, 3, 1, 2. The target array consumes a total of 11 uint16 bytes, totaling 22 bytes.

[0047] Furthermore, upon receiving an index request for the target data string, the target array is partitioned based on the branching points and node pointers corresponding to the elements of the compressed target array to enable the query of the target data string to be queried. The index request carries a string identifier for the target data string, which is one of the (i+1) data strings.

[0048] In this embodiment, on the one hand, by storing only the branching points between data strings in the nodes of the Trie tree structure, instead of storing the complete strings, a fixed-length node is added for each inserted record, thus determining the space required. On the other hand, the Trie tree nodes are compressed using inorder traversal, reducing the storage space occupied and thereby improving memory utilization. Furthermore, the target array is divided according to the branching points and branch directions to realize the query of the target data string, which can significantly reduce the space consumption of the index and improve the data access efficiency.

[0049] Furthermore, in one embodiment of this application, before compressing the storage space of the trie structure according to the traversal order of left node, root node, and right node to obtain the target array, the method further includes: obtaining the (i+1)th data string; storing the branch point between the (i+1)th data string and the i-th data string at the i-th node of the trie structure; recording the pointer address of the first node pointer of the i-th node as a first value and pointing the first node pointer to the first branch direction of the i-th node; and recording the pointer address of the second node pointer of the i-th node as a second value and pointing the second node pointer to the second branch direction of the i-th node; wherein, the first value is one of the value of the (i+1)th data string at the branch point and the value of the i-th data string at the branch point, and the second value is the other.

[0050] In this embodiment, when storing data strings using a trie structure, the current data string (i.e., the (i+1)th data string) is compared with its previous data string (i.e., the i-th data string) to obtain the divergence point between the two. The divergence point is the bit where the two values ​​are different, and then the divergence point is stored on the node of the Trie tree structure.

[0051] In related technologies, such as Figure 5 The diagram shows the Trie tree structure with indices AB, AC, and ADEFGHIJKLMN. The space occupied by node DEFGHIJKLMN is greater than that of nodes B and C because the string DEFGHIJKLMN is longer than the strings B and C. Clearly, memory usage is positively correlated with the length of the added record's key, and since the key length is not fixed, there is a possibility of excessively long keys.

[0052] For example, in this embodiment of the application, for records with a key length not exceeding 8192 bytes, the index space overhead is a constant of approximately 16 bytes for each new record added.

[0053] After storing the branch point at the i-th node, set the first node pointer and the second node pointer of the i-th node. Specifically, record the pointer address of the first node pointer as a first value and point the first node pointer to the first branch direction of the i-th node; and record the pointer address of the second node pointer as a second value and point the second node pointer to the second branch direction of the i-th node.

[0054] For example, Figures 6 to 9This describes the process of inserting the numbers 9, 1, 5, and 7. The solid lines represent the indexes; LP (LeftPointer) is the left pointer (first pointer) of the current node, pointing to the left node; RP (RightPointer) is the right pointer (second pointer) of the current node, pointing to the right node. It should be noted that the dashed lines represent indexed records, but are not part of the index itself.

[0055] Taking binary strings as an example, such as Figure 6 As shown, the binary representation of 9 is 1001. Since the index is empty, the root pointer can directly point to 9.

[0056] like Figure 7 As shown, the binary representation of 1 is 0001, and the divergence point between 1 and 9 (1001) is on the 0th bit. Therefore, only one new node needs to be added to the index and the divergence point is recorded as the 0th bit. The 0th bit of 1 (0001) is 0, so the LP of the newly added node points to 1 (0001). The 0th bit of 9 (1001) is 1, so the RP of the newly added node points to 9 (1001).

[0057] like Figure 8 As shown, the binary representation of 5 is 0101, with the 0th bit being 0. Therefore, it is compared with 1 (0001), and the divergence point is found to be on the 1st bit. Similarly, a new node is added and the divergence point is recorded as 1. The 1st bit of 1 (0001) is 0, so the LP of the new node points to 1. The 1st bit of 5 (0101) is 1, so the RP of the new node points to 5.

[0058] like Figure 9 As shown, the binary representation of 7 is 0111, with the 0th bit being 0, so it will move along LP to the next node. This node records the branch point as 1, with the 1st bit of 7 (0101) being 1, moving along RP to 5 (0101). The branch point between 7 (0111) and 5 (0101) is on the 2nd bit, so a new node is created, with LP pointing to 5 and RP pointing to 7.

[0059] Compared to related technologies, the embodiments of this application only store the branching points between data strings in the nodes of the Trie tree structure, instead of storing the complete strings. Furthermore, by setting node pointers to indicate the next data string, a fixed-length node is added every time a record is inserted, and the space required is determined. This allows us to promise users the memory required for indexing data and avoid the problem of resource waste caused by users reserving too much memory.

[0060] Further, in one embodiment of this application, indexing the target data string according to the branching points and node pointers corresponding to the elements of the target array includes: dividing the target array into subarrays according to the value of the target data string at the first branching point of the target array and the direction of the pointer of the first target pointer with the value, wherein the first branching point is the smallest element in the target array, and the first target pointer is the first node pointer or the second node pointer corresponding to the first branching point; dividing the subarray into subarrays according to the value of the target data string at the second branching point of the subarray and the direction of the pointer of the second target pointer with the value, until the number of elements in the subarray is 1, wherein the second branching point is the smallest element in the subarray, and the second target pointer is the first node pointer or the second node pointer corresponding to the second branching point.

[0061] In this embodiment, since the lower level of the divergence point is always larger than the upper level, the RangeMinQuery method can be used to index the target data string on this compressed target array. Specifically, upon receiving an index request for the target data string (which carries the string identifier of the target data string), the target array is divided into subarrays based on the smallest element of the target array as the dividing point and the direction of the pointer corresponding to the smallest element. This step is repeated until the number of elements in the resulting subarrays is 1, thus completing the query for the target data string. The direction of the pointer is determined by the value of the bit corresponding to the smallest element in the target data string.

[0062] Alternatively, the first target pointer and the second target pointer can be used as index information for the target data string.

[0063] Taking query 12 (01100) as an example, such as Figure 10 As shown, first find the smallest element 0 (i.e., the first branch point) in the target array [2, 1, 2, 0, 4, 3, 4, 2, 3, 1, 2]. This indicates that the branch point of the root node is on the 0th bit. The 0th bit of 12 (01100) is 0, so the first target pointer of the first branch point points to the left. That is, as... Figure 11 As shown, the target array is pruned into a subarray [2, 1, 2]. The minimum element is then found, resulting in 1 (i.e., the second branching point). This indicates that the lower-level branching point is on the first bit. The first bit of 12 (01100) is 1, therefore the second target pointer of the second branching point points to the right. That is, as... Figure 12As shown, the array is pruned to 2, with only one branch point, so no further pruning is needed. The second bit of 12 (01100) is 1, and the second target pointer of the second branch point points to the right. In summary, the pointer to the final record has been found, and the index lookup operation is complete.

[0064] It should be noted that the interval minimum value query uses the IntelSSE vectorized instruction set (i.e., node pointers) to speed up the process and improve query efficiency.

[0065] In this embodiment, based on compressed Trie tree nodes, the minimum value query method is used for indexing operations, which significantly reduces the index space and improves data access efficiency without sacrificing range query capability and performance.

[0066] Furthermore, in one embodiment of this application, before compressing the storage space of the trie structure according to the traversal order of left node, root node, and right node to obtain the target array, the method further includes: dividing the trie structure into N subtree structures; storing the N subtree structures; wherein each subtree structure includes at least one node of the trie structure, and N is a positive integer.

[0067] In this embodiment, the Trie tree structure is divided into blocks, that is, it is divided into N subtree structures, each of which includes at least one node.

[0068] For example, such as Figure 13 As shown, the Trie tree structure includes 11 nodes. To improve caching efficiency, it is divided into subtree structures 702 and 704. After block partitioning, the height of the Trie tree structure is reduced from 5 to 2. The block partitioning algorithm uses the Kundu algorithm. Specifically, it finds the bottom-level node that has not yet been partitioned, generates a subtree containing only this node, and attempts to merge the parent node and its subtrees upwards. If the total number of nodes in the subtree does not exceed the subtree size limit, it continues to attempt to merge. If the total number of nodes in the subtree exceeds the limit, the current subtree is the best partition, and merging is no longer attempted. This process continues to find the bottom-level node that has not yet been partitioned and partition it until there are no more unpartitioned nodes, at which point the process ends. For example, as shown... Figure 14 As shown, find node 4, merge its parent node and its subtree upwards, as follows: Figure 15 As shown, when the total number of nodes in the subtree exceeds the upper limit threshold of 4, the current subtree is considered an optimal partition. The dashed line indicates that this subtree is logically "disconnected" from the other nodes and will not participate in subsequent partitions. Continue partitioning, as shown... Figure 16 As shown, find node 2 among the remaining nodes, and merge its parent node and its subtree upwards, as follows. Figure 17As shown, when the total number of nodes in the subtree exceeds the upper limit threshold of 5, the current subtree becomes another optimal partition. At this point, there are no unpartitioned nodes, and the partitioning process ends.

[0069] In related technologies, if the fan-out of a Trie tree structure is very small, then a single query path will be very long, which will lead to serious cache misses. For example... Figure 18 As shown, each node has a fan-out of 2. It only takes 2 nodes to reach node D, but it takes 4 nodes to reach node I. Too many nodes will cause serious cache invalidation problems and reduce the running speed.

[0070] Compared to related technologies, the Trie tree structure in this application does not store a continuous string, but uses a block-based technique, which can reduce the cache invalidation problem caused by too many nodes, thereby improving the system running speed.

[0071] The data processing method provided in this application can be executed by a data processing device. This application uses an example of a data processing device executing the data processing method to illustrate the data processing device provided in this application.

[0072] This application provides a data processing apparatus, such as... Figure 19 As shown, the data processing device 1900 includes:

[0073] The conversion module 1902 is used to compress the storage space of the trie structure according to the traversal order of left node, root node, and right node to obtain the target array. Each element of the target array corresponds to a node of the trie structure. The nodes of the trie structure are only used to store the branching points of the data string, and the node pointers of the nodes are used to indicate the branching direction of the nodes.

[0074] The receiving module 1904 is used to receive a query request for a target data string, the query request carrying a string identifier of the target data string;

[0075] Index module 1906 is used to index the target data string based on the branch points and node pointers corresponding to the elements of the target array.

[0076] In this embodiment, the nodes of the trie structure (i.e., the trie tree structure) store the divergence points of the data strings. Specifically, the nodes of the trie structure store the divergence points between the (i+1)th and the ith data string. The divergence point is the bit in the (i+1)th data string that differs from the ith data string in value, where i is a positive integer. This divergence point is then stored on the nodes of the trie tree structure.

[0077] Furthermore, based on the inorder traversal method—that is, first visiting the left subtree, then the root node, and finally the right subtree—the storage space of the trie structure is compressed into a target array. Further, upon receiving an index request for the target data string, the target array is partitioned according to the branching points and node pointers corresponding to the elements of this compressed target array to enable the query of the target data string. The index request carries the string identifier of the target data string, which is one of the (i+1)th data strings.

[0078] In this embodiment, on the one hand, by storing only the branching points between data strings in the nodes of the Trie tree structure, instead of storing the complete strings, a fixed-length node is added for each inserted record, thus determining the space required. On the other hand, the Trie tree nodes are compressed using inorder traversal, reducing the storage space occupied and thereby improving memory utilization. Furthermore, the target array is divided according to the branching points and branch directions to realize the query of the target data string, which can significantly reduce the space consumption of the index and improve the data access efficiency.

[0079] Furthermore, in one embodiment of this application, the data processing device 1900 further includes: an acquisition module for acquiring the (i+1)th data string; and a first storage module for storing the divergence point between the (i+1)th data string and the i-th data string at the i-th node of the trie structure.

[0080] The recording module is used to record the pointer address of the first node pointer of the i-th node as a first value and point the first node pointer to the first branch direction of the i-th node, and to record the pointer address of the second node pointer of the i-th node as a second value and point the second node pointer to the second branch direction of the i-th node; wherein, the first value is one of the value of the (i+1)-th data string at the branch point and the value of the i-th data string at the branch point, and the second value is the other.

[0081] Further, in one embodiment of this application, the indexing module is specifically used to: divide the target array into subarrays based on the value of the target data string at the first branch point of the target array and the direction of the pointer of the first target pointer with the value, wherein the first branch point is the smallest element in the target array and the first target pointer is the first node pointer or the second node pointer corresponding to the first branch point; divide the subarray into subarrays based on the value of the target data string at the second branch point of the subarray and the direction of the pointer of the second target pointer with the value, until the number of elements in the subarray is 1, wherein the second branch point is the smallest element in the subarray and the second target pointer is the first node pointer or the second node pointer corresponding to the second branch point.

[0082] Furthermore, in one embodiment of this application, the data processing device 1900 further includes: a partitioning module for partitioning the dictionary structure into N subtree structures; and a second storage module for storing the N subtree structures; wherein each subtree structure includes at least one node of the dictionary structure, and N is a positive integer.

[0083] The data processing device 1900 in this embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This embodiment does not specifically limit the specific type of device.

[0084] The data processing device 1900 in this embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this embodiment does not specifically limit its use.

[0085] The data processing device 1900 provided in this application embodiment can achieve... Figure 1 The various processes implemented in the data processing method embodiment will not be described again here to avoid repetition.

[0086] Optionally, such as Figure 20 As shown, this application embodiment also provides an electronic device 2000, including a processor 2002 and a memory 2004. The memory 2004 stores a program or instructions that can run on the processor 2002. When the program or instructions are executed by the processor 2002, they implement the various steps of the above-described data processing method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0087] It should be noted that the electronic devices in the embodiments of this application include the aforementioned mobile electronic devices and non-mobile electronic devices.

[0088] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described data processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0089] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0090] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above data processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0091] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0092] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0093] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0094] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A data processing method, characterized in that, include: The storage space of the trie structure is compressed according to the traversal order of left node, root node, and right node to obtain the target array. Each element of the target array corresponds to a node of the trie structure. The nodes of the trie structure are only used to store the branching points of data strings, and the node pointers of the nodes are used to indicate the branching direction of the nodes. Receive a query request for a target data string, wherein the query request carries a string identifier of the target data string; Based on the branching points and node pointers corresponding to the elements of the target array, a minimum value lookup method is used. The minimum element of the target array is taken as the branching point, and the target array is divided according to the direction of the pointer corresponding to the minimum element. The division is repeated until the number of elements in the resulting subarray is 1, so as to complete the indexing process of the target data string. The direction of the pointer is determined according to the value of the bit corresponding to the minimum element in the target data string.

2. The data processing method according to claim 1, characterized in that, Before compressing the storage space of the trie structure according to the traversal order of left node, root node, and right node to obtain the target array, the following steps are also included: Get the (i+1)th data string; The i-th node of the trie structure stores the divergence point between the (i+1)-th data string and the i-th data string; Record the pointer address of the first node pointer of the i-th node as a first value, and point the first node pointer to the first branch direction of the i-th node; and record the pointer address of the second node pointer of the i-th node as a second value, and point the second node pointer to the second branch direction of the i-th node. Wherein, the first value is one of the value of the (i+1)th data string at the branch point and the value of the ith data string at the branch point, and the second value is the other.

3. The data processing method according to claim 2, characterized in that, The step involves using a minimum value lookup method based on the branching points and node pointers corresponding to the elements of the target array. This method uses the minimum element of the target array as the branching point and divides the target array according to the direction indicated by the pointer corresponding to the minimum element. This division is repeated until the subarrays contain only one element, thus completing the indexing process for the target data string. Based on the value of the target data string at the first branch point of the target array and the direction of the pointer address of the first target pointer that is the value, the target array is divided into subarrays, wherein the first branch point is the smallest element in the target array and the first target pointer is the first node pointer or the second node pointer corresponding to the first branch point. Based on the value of the target data string at the second branch point of the subarray and the direction of the second target pointer whose pointer address is the value, the subarray is divided into subarrays until the number of elements in the subarray is 1. The second branch point is the smallest element in the subarray, and the second target pointer is the first node pointer or the second node pointer corresponding to the second branch point.

4. The data processing method according to any one of claims 1 to 3, characterized in that, Before compressing the storage space of the trie structure according to the traversal order of left node, root node, and right node to obtain the target array, the following steps are also included: Divide the aforementioned trie structure into N subtree structures; Store N of the aforementioned subtree structures; Each of the subtree structures includes at least one node of the trie structure, where N is a positive integer.

5. A data processing apparatus, characterized in that, include: The conversion module is used to compress the storage space of the trie structure according to the traversal order of left node, root node, and right node to obtain a target array. Each element of the target array corresponds to a node of the trie structure. The nodes of the trie structure are only used to store the branching points of data strings, and the node pointers of the nodes are used to indicate the branching direction of the nodes. A receiving module is used to receive a query request for a target data string, wherein the query request carries a string identifier of the target data string; The indexing module is used to perform range minimum value lookup based on the branching points and node pointers corresponding to the elements of the target array. The minimum element of the target array is used as the branching point, and the target array is divided into subarrays according to the direction of the pointer corresponding to the minimum element. The subarrays are divided repeatedly until the number of elements in the subarrays is 1, so as to complete the indexing process of the target data string. The direction of the pointer is determined according to the value of the bit corresponding to the minimum element in the target data string.

6. The data processing apparatus according to claim 5, characterized in that, Also includes: The acquisition module is used to retrieve the (i+1)th data string; The first storage module is used to store the divergence point between the (i+1)th data string and the ith data string at the ith node of the trie structure; The recording module is used to record the pointer address of the first node pointer of the i-th node as a first value and point the first node pointer to the first branch direction of the i-th node, and to record the pointer address of the second node pointer of the i-th node as a second value and point the second node pointer to the second branch direction of the i-th node; Wherein, the first value is one of the value of the (i+1)th data string at the branch point and the value of the ith data string at the branch point, and the second value is the other.

7. The data processing apparatus according to claim 6, characterized in that, The index module is specifically used for: Based on the value of the target data string at the first branch point of the target array and the direction of the pointer address of the first target pointer that is the value, the target array is divided into subarrays, wherein the first branch point is the smallest element in the target array and the first target pointer is the first node pointer or the second node pointer corresponding to the first branch point. Based on the value of the target data string at the second branch point of the subarray and the direction of the second target pointer whose pointer address is the value, the subarray is divided into subarrays until the number of elements in the subarray is 1. The second branch point is the smallest element in the subarray, and the second target pointer is the first node pointer or the second node pointer corresponding to the second branch point.

8. The data processing apparatus according to any one of claims 5 to 7, characterized in that, Also includes: The partitioning module is used to divide the trie structure into N subtree structures; The second storage module is used to store N of the subtree structures; Each of the subtree structures includes at least one node of the trie structure, where N is a positive integer.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the data processing method as described in any one of claims 1 to 4.

10. A readable storage medium having a program or instructions stored thereon, characterized in that, When the program or instructions are executed by the processor, they implement the steps of the data processing method as described in any one of claims 1 to 4.

Images