An index construction method, a data reading method, and an index construction device

Abstract

The application provides an index construction method, a data reading method and an index construction device. The index construction method determines a target prefix length according to the initial data and a preset spatial magnification threshold of the first target node; divides the initial data into a plurality of sub-data sets and to-be-stored data according to the target prefix length; determines an index model according to the node prefix and the to-be-stored data, and calculates the storage positions of the node prefix and the to-be-stored data through the index model; stores each node prefix, a pointer pointing to a sub-node and the to-be-stored data in the corresponding storage positions, constructs the first target node, and repeats the above process to construct the sub-nodes until the entire index is constructed. The constructed index structure is flatter, has better performance in different data sets and write-intensive workloads, maintains stable performance when the data distribution changes, and has better tail delay performance indicators.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data storage, in particular to an index construction method, a data reading method and an index construction device. BACKGROUND

[0002] Prefix tree: also known as Trie Tree or dictionary tree, is a multi-way tree structure for fast retrieval. A prefix tree can be used to index numerical values, strings and other data types (when indexing numerical values, it is also called Radix Tree). Each node of the prefix tree has multiple child nodes, and all child nodes of each node contain different characters. In the prefix tree, each node represents a string (prefix), and the characters on the path from the root node to a certain node are concatenated to form the string corresponding to the node. The core idea of the prefix tree is to exchange space for time. The common prefix of the string is used to reduce the query time overhead to improve efficiency.

[0003] However, the time and space efficiency of the prefix tree model is directly related to the amount of data, and the big data era puts forward higher capacity and performance requirements for the storage system. SUMMARY

[0004] Therefore, the purpose of the present application is to provide an index construction method, a data reading method and an index construction device. The constructed index structure is more flat, and has better performance in different data sets and write-intensive workloads. At the same time, the performance remains stable when the data distribution changes, and the tail delay performance index is also better.

[0005] The index construction method provided by the embodiment of the present application comprises:

[0006] The initial data is input into a first target node of a prefix tree, and the target prefix length of the first target node is determined according to the initial data and a preset space amplification rate threshold of the first target node;

[0007] According to the target prefix length of the first target node, the initial data is divided into a plurality of data subsets, and / or the to-be-stored data stored in the first target node is screened out from the initial data; wherein the initial data in the data subset has the same node prefix;

[0008] The index model of the first target node is determined according to the node prefix and / or the to-be-stored data, and the pointer of the pointing child node corresponding to the node prefix of each data subset and the storage location are calculated respectively by the index model, and / or the storage location of each to-be-stored data is calculated respectively;

[0009] storing each node prefix and a pointer pointing to a child node, and / or each data to be stored in a corresponding storage location, to build an index structure of the first target node;

[0010] Taking the child node of the built first target node as a new first target node and taking the child data set corresponding to the child node as new initial data, an index structure of the new first target node is built until a building completion condition is met, and the entire index is built.

[0011] In some embodiments, in the index building method, a target prefix length of the first target node is determined according to the initial data and a preset spatial amplification threshold of the first target node, including:

[0012] At least one to-be-verified prefix length is determined according to a data distribution feature of the initial data;

[0013] A spatial amplification corresponding to each to-be-verified prefix length is calculated respectively;

[0014] A to-be-verified prefix length corresponding to a spatial amplification not greater than the preset spatial amplification threshold of the first target node is screened out;

[0015] The largest to-be-verified prefix length screened out is determined as the target prefix length.

[0016] In some embodiments, in the index building method, an index model of the first target node is determined according to a node prefix and / or data to be stored, including:

[0017] A target global slope satisfying a preset spatial amplification threshold is calculated according to the node prefix and / or the data to be stored;

[0018] A linear model is determined as the index model according to the calculated target global slope.

[0019] In some embodiments, in the index building method, after the entire index is built, the method further includes:

[0020] When new data is inserted in the built index, it is judged whether a target slot of a second target node to which the new data is mapped satisfies a preset adjustment condition;

[0021] If yes, an index structure of the second target node is adjusted according to data of the second target node including the new data by a preconfigured adjustment strategy.

[0022] In some embodiments, in the index building method, the index structure of the second target node is adjusted according to the data of the second target node including the new data, including:

[0023] adjusting a spatial magnification of the second target node to expand a storage space of the second target node;

[0024] or, re-determining an index model of the second target node according to data of the second target node including the new data;

[0025] or, taking a key-value pair of a data slot of the second target node, a node prefix of a pointer slot, and a key-value pair of a pointer group as initial data of the second target node;

[0026] re-determining a target prefix length and an index model of the second target node according to the initial data of the second target node and a preset spatial magnification threshold, to reconstruct an index structure of the second target node.

[0027] In some embodiments, the preset adjustment condition in the index construction method is at least one of the following: a full load rate of the second target node reaches a preset full load rate threshold; the target slot of the second target node to which the new data is mapped is a data slot; the target slot of the second target node to which the new data is mapped is a pointer slot, and the new data does not match a longest common prefix of a child node pointed to by the pointer slot.

[0028] In some embodiments, in the index construction method, when a difference between a child node of the first target node and a node prefix of the first target node is less than a preset length threshold, the child node of the first target node adopts an ART node.

[0029] In some embodiments, a data reading method is also provided, which is applied to an index constructed by the index construction method; the reading method comprises:

[0030] starting from a root node of the index, judging whether a node prefix of a current node matches data to be read;

[0031] if the node prefix matches the data to be read, calculating a target slot of the data to be read through an index model in the current node;

[0032] if the target slot is a data slot, judging whether data in the data slot matches the data to be read, and returning the data in the data slot if the data matches the data to be read;

[0033] if the target slot is a pointer slot, querying the data to be read in a child node pointed to by the pointer slot until a reading result is returned.

[0034] In some embodiments, the data reading method further comprises:

[0035] determining boundary data of a data range to be read, wherein the boundary data is a largest data and / or a smallest data in the data range.

[0036] determining a storage position of the boundary data in the index;

[0037] and returning a read result matching the data range to be read according to the storage order of the data in the index and the storage position of the boundary data.

[0038] In some embodiments, an index construction apparatus is also provided, comprising:

[0039] A determining module is configured to input initial data into a first target node of a prefix tree, and determine a target prefix length of the first target node according to the initial data and a preset spatial magnification threshold of the first target node.

[0040] A dividing module is configured to divide the initial data into a plurality of data subsets according to the target prefix length of the first target node, and / or to filter out data to be stored in the first target node from the initial data; wherein the initial data in the data subsets have the same node prefix.

[0041] A calculating module is configured to determine an index model of the first target node according to the node prefix and / or the data to be stored, and to calculate a pointer to a child node corresponding to each node prefix of each data subset and a storage position of each data to be stored respectively through the index model.

[0042] A first construction module is configured to store each node prefix and the pointer to the child node, and / or each data to be stored in a corresponding storage position, and to construct an index structure of the first target node.

[0043] A second construction module is configured to take a child node of the constructed first target node as a new first target node, to take a data subset corresponding to the child node as new initial data, to construct an index structure of the new first target node, and to construct an entire index until a construction completion condition is met.

[0044] The index construction method, data reading method and index construction device are provided in the embodiments of the present application. In the index construction method, initial data is input into a first target node of a prefix tree, and a target prefix length of the first target node and an index model are determined according to the initial data and a preset space amplification threshold of the first target node. The initial data is divided into a plurality of data sets according to the target prefix length of the first target node, and / or the data to be stored in the first target node is screened from the initial data. The index model is used to calculate a pointer to a child node and a storage location corresponding to each node prefix of each data set, and / or the storage location of each data to be stored is calculated. Each node prefix and the pointer to the child node, and / or each data to be stored is stored in the corresponding storage location, and the index structure of the first target node is constructed. The child node of the first target node is constructed as a new first target node, and the child data set corresponding to the child node is constructed as new initial data, and the index structure of the new first target node is constructed until a construction completion condition is met, and the entire index is constructed. That is, compared with a traditional index structure, the index constructed by the embodiments of the present application has a learning index model matched with the data of each node in the index, and has the ability to fit the data distribution of the learning index, thereby having comprehensive advantages over the traditional index in basic query performance. At the same time, the index has corresponding competitiveness compared with other learning indexes. For other learning indexes, the constructed index has the overall robustness of the traditional prefix tree index (for example, ART), can have better performance in different data sets and write-intensive workloads, and can maintain stable performance when the data distribution changes and has better tail delay performance indicators. The index construction algorithm in the embodiments of the present application not only has low time complexity, but also can guarantee the flat structure of the entire tree, and can map different prefixes to different positions required by the insertion adjustment strategy. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0046] Figure 1 A method flowchart of the index construction method according to the embodiments of the present application is shown;

[0047] Figure 2 A prefix tree index structure diagram constructed by the present application is shown;

[0048] Figure 3A method flow chart for determining the target prefix length of the first target node is shown in the embodiments of the present application.

[0049] Figure 4 A method flow chart for determining the index model of the first target node according to the node prefix and / or the data to be stored is shown in the embodiments of the present application.

[0050] Figure 5 A method flow chart for adjusting the index structure of the second target node is shown in the embodiments of the present application.

[0051] Figure 6 A method flow chart for the data reading method is shown in the embodiments of the present application.

[0052] Figure 7 A method flow chart for the data reading method is shown in the embodiments of the present application.

[0053] Figure 8 A structural schematic diagram of the index construction device is shown in the embodiments of the present application.

[0054] Figure 9 A structural schematic diagram of the electronic device is shown in the embodiments of the present application. DETAILED DESCRIPTION

[0055] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of description and illustration, and are not used to limit the protection scope of the present application. In addition, it should be understood that the schematic drawings are not drawn according to the actual proportions. The flow charts used in the present application show the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow charts can not be implemented in sequence, and the steps without logical context relationship can be reversed in sequence or implemented simultaneously. In addition, one or more other operations can be added to the flow chart or removed from the flow chart under the guidance of the content of the present application by those skilled in the art.

[0056] In addition, the described embodiments are only some of the embodiments of the present application, not all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0057] It should be noted that the term "comprising" will be used in the embodiments of the present application to specify the presence of stated features, but not to exclude the addition of further features.

[0058] Database Index: Database Index is a data structure used to accelerate data access operations. By storing data identifiers (or data keys, i.e. Key) and data actual storage locations (or data values, i.e. Value) in the index structure when storing data. Although the creation and maintenance of the index structure requires additional space cost, the index can accelerate the subsequent access to data elements, avoiding the high cost of direct scanning data element queries. The operation of accessing data can be input into the index through the data identifier (Key) to query and then get the actual storage location of the data (Value). In addition, index data structures are also widely used in other fields, such as key-value storage, file system, search engine, etc.

[0059] Sorted Index: Sorted Index is a type of index structure, and the most prominent feature is that it supports efficient range queries, prefix queries, etc. in addition to point queries. This is mainly achieved by explicitly organizing the index structure according to the order of data elements. Point query is to query the existence of data elements according to data identifiers (Key), and if the data element exists, return the actual storage location of the data (Value). Range query is to give a range of data identifiers (Key), and return the actual storage locations (Value) of all data elements in the range. Prefix query is to give a data identifier (Key), and return the maximum value of all data elements smaller than the data identifier (Key), and its corresponding data storage location (Value). Other types of index structures such as hash index only support point queries and cannot support efficient range queries or prefix queries.

[0060] Trie Tree: Trie Tree, also known as Dictionary Tree, is a multi-way tree structure used for fast retrieval. A prefix tree can be used to index numerical values, strings and other data types (when indexing numerical values, it is also called Radix Tree). Each node of the prefix tree has multiple child nodes, and all child nodes of each node contain different characters. In the prefix tree, each node represents a string (prefix), and the characters on the path from the root node to a certain node are concatenated to form the string corresponding to the node. The core idea of the prefix tree is to trade space for time. The common prefix of the string is used to reduce the query time overhead to improve efficiency.

[0061] However, the time and space efficiency of the prefix tree model is directly related to the amount of data, and the large data era puts forward higher capacity and performance requirements for storage systems.

[0062] In the prior art, various improvements have also been made to the traditional prefix tree model to improve the capacity and performance of the prefix tree model. For example, an adaptive radix tree (ART) is proposed, that is, in order to solve the problem of excessive space consumption of the radix tree in the worst case, ART adaptively selects a compact and efficient data structure for internal nodes, while using delay expansion and path compression to reduce the height of the tree, so ART has higher space efficiency. ART index records and distinguishes part of the prefix of the key value through 8Bit at each layer, and determines the node type according to the data amount of each node, which has Node4, Node16, Node48, Node256 four types, Each type ensures efficient access through methods such as SIMD parallel instructions, indirect indexing, etc. Its search performance exceeds that of a highly optimized read-only search tree, and it also supports very efficient insertion and deletion. While the performance of ART is comparable to that of a hash table, ART also maintains data in sorted order, so it can support operations such as range scanning and prefix lookup that hash tables cannot support. Moreover, since the insertion operation of ART only needs to modify at most two nodes, the parent node and the current node, it is more friendly to multi-threading parallelism, and can block the execution of other parallel operations less during the insertion operation.

[0063] And in the patent with publication number CN112732725A, an adaptive prefix tree construction method and system based on NVM hybrid memory, and a medium are described. In order to reduce the space consumption of the prefix tree index structure to DRAM, while maintaining high performance, the scheme provides an adaptive prefix tree construction method based on NVM hybrid memory, which constructs a global index data structure to build and maintain all data and nodes in the NVM address space. In the DRAM address space, maintain a quick index data structure for new data and nodes, reduce the impact of NVM address space on index performance. When the DRAM address space occupancy reaches a certain proportion, trigger the migration thread to migrate the new data and nodes to the NVM address space, which can effectively reduce the space occupancy of the index structure to DRAM, reduce the storage cost, and efficiently implement various database operation requests.

[0064] Learned Index: Learned Index is a new type of index structure constructed using machine learning methods and ideas. Learned Index regards the index structure itself as a model that takes data identifiers (Keys) as input and outputs the corresponding location information stored in the index structure. In this perspective, various machine learning models can be used to replace or accelerate the retrieval of the index model. By fitting the input-output function relationship directly, the data can be searched through model calculation combined with some corrections. Through faster and simpler model calculation operations, the traditional index preliminary sequential search, binary search and other search operations are replaced, thereby speeding up the overall query efficiency. In actual query execution, Learned Index predicts the location in the current index node through the input of the model. If the model has errors, additional lookup corrections are needed. Through the lookup of multiple models, the output is finally obtained.

[0065] Although Learned Index can achieve better performance than traditional index in most data sets and workloads, its performance is not as good as that of traditional index, such as ART, when facing difficult-to-learn data sets and write-intensive workloads. Moreover, when the data distribution changes greatly, the performance fluctuates, and the tail latency of different workloads is not as stable as that of traditional index. Overall, the robustness of Learned Index is not as good as that of traditional index. In addition, the current Learned Index is mainly optimized for fixed-length numerical data and cannot handle variable-length character data, and its functionality is not as complete as that of traditional index.

[0066] The present application proposes a new type of index structure to solve the robustness problem of Learned Index and the performance problem of traditional index. By introducing a learning model based on the traditional prefix tree, it combines the robustness of traditional index and the high-efficiency query performance of Learned Index, has the ability to handle variable-length character data, and is superior to traditional index in all aspects.

[0067] Based on this, in order to improve the performance and capacity of the index, the index construction method, data reading method and index construction device are provided. In the index construction method, the initial data is input into a first target node of a prefix tree, and according to the initial data and a preset space amplification threshold of the first target node, a target prefix length of the first target node and an index model are determined. According to the target prefix length of the first target node, the initial data is divided into a plurality of data subsets, and / or the data to be stored in the first target node is screened out from the initial data. According to the index model, the pointer and storage location corresponding to the child node of each node prefix of each data subset are calculated, and / or the storage location of each data to be stored is calculated. Each node prefix and the pointer to the child node, and / or each data to be stored is stored in the corresponding storage location, and the index structure of the first target node is constructed. The child node of the first target node is constructed as a new first target node, and the child data set corresponding to the child node is constructed as new initial data, and the index structure of the new first target node is constructed until the construction completion condition is met, and the entire index is constructed. That is, compared with the traditional index structure, in the index constructed by the present application, each node has a learning index model matched with the data of the node, which has the ability to fit the data distribution of the learning index, thereby having comprehensive advantages over the traditional index in basic query performance. At the same time, compared with other learning indexes, the constructed index has the overall robustness of the traditional prefix tree index (such as ART), and can have better performance in different data sets and write-intensive workloads. At the same time, the performance remains stable when the data distribution changes, and the tail delay performance index is also better. The index construction algorithm described in the present application not only has low time complexity, but also can guarantee the flat structure of the entire tree, and can map different prefixes to different positions required by the insertion adjustment strategy.

[0068] Please refer to Figure 1 , Figure 1 The method flowchart of the index construction method described in the present application is shown. Specifically, the index construction method includes the following steps S101-S105:

[0069] S101, input the initial data into the first target node of the prefix tree, and determine the target prefix length of the first target node according to the initial data and the preset space amplification threshold of the first target node;

[0070] S102, according to the target prefix length of the first target node, the initial data is divided into a plurality of data subsets, and / or the data to be stored in the first target node is screened out from the initial data; wherein the initial data in the data subset has the same node prefix;

[0071] S103, determining an index model of the first target node according to the node prefix and / or the data to be stored, and calculating a pointer to a child node corresponding to the node prefix of each sub-data set and a storage location of the pointer to the child node, and / or calculating a storage location of each data to be stored, through the index model respectively;

[0072] S104, storing each node prefix and the pointer to the child node, and / or each data to be stored in the corresponding storage location, and constructing an index structure of the first target node;

[0073] S105, taking the child node of the constructed first target node as a new first target node, taking the sub-data set corresponding to the child node as new initial data, constructing an index structure of the new first target node, until a construction completion condition is met, and constructing the entire index.

[0074] Compared with the traditional index structure, the index constructed by the index construction method of the embodiments of the present application has a learning index model matching the data of each node in the index, has the ability to fit the data distribution of the learning index, and thus has comprehensive advantages over the traditional index in basic query performance. Meanwhile, the index also has corresponding competitiveness compared with other learning indexes. For other learning indexes, the constructed index has the overall robustness of the traditional prefix tree index (such as ART), can have better performance in different data sets and write-intensive workloads, and can maintain stable performance when the data distribution changes and has better tail latency performance indicators. The index construction algorithm of the embodiments of the present application not only has low time complexity, but also can guarantee the flat structure of the entire tree, and can provide the property that different prefixes are mapped to different positions required by the insertion adjustment strategy.

[0075] In the step S101, the initial data is input into the first target node of the prefix tree, and the target prefix length and the index model of the first target node are determined according to the initial data and a preset space amplification rate threshold of the first target node.

[0076] Please refer to Figure 2 , Figure 2 A prefix tree index structure constructed by the present application is shown; the prefix tree index includes a root node, the root node as a parent node has a plurality of child nodes, and the same applies to the following.

[0077] The first target node can be a root node or other nodes; that is, the first target node is a node being built at present, and does not particularly refer to a root node. When the first target node is a root node, the initial data is all data used for building an index; when the first target node is other nodes, the initial data input to the first target node is a child data set corresponding to a pointer of a parent node to the first target node.

[0078] Each node records a longest common prefix of data inserted into a subtree with the current node, and each node has an index model and a node prefix of a key extracted for pre-processing before input to the index model. Each node has an array for storing pointers to child nodes and data, and two bits are used to distinguish empty slots, pointer slots storing pointers to child nodes, and data slots storing data.

[0079] Please refer to Figure 3 According to the initial data and a preset space amplification rate threshold of the first target node, a target prefix length of the first target node is determined, including the following steps S301-S304.

[0080] S301, at least one to-be-verified prefix length is determined according to a data distribution characteristic of the initial data.

[0081] S302, a space amplification rate corresponding to each to-be-verified prefix length is calculated.

[0082] S303, a to-be-verified prefix length corresponding to a space amplification rate not greater than the preset space amplification rate threshold of the first target node is screened out.

[0083] S304, the maximum to-be-verified prefix length screened out is determined as the target prefix length.

[0084] After the maximum prefix length of the first target node is determined, an index model of the first target node is determined according to the determined maximum prefix length.

[0085] Please refer to Figure 4 In the index building method, an index model of the first target node is determined according to a node prefix and / or to-be-stored data, including the following steps S401-S402.

[0086] S401, a target global slope satisfying a preset space amplification rate threshold is calculated according to a node prefix and / or to-be-stored data.

[0087] S402, a linear model is determined as the index model according to the calculated target global slope.

[0088] For the learning index, the construction algorithm of the index model is the core. At present, there are various choices for different optimization objectives, such as a linear regression model based on least squares method and an FMCD method based on minimizing the maximum conflict rate.

[0089] The embodiment of the present application proposes a new index model construction algorithm, which aims to have the same sub-node segmented data with the same prefix under the premise of taking the same prefix length (denoted as span), and the data stored in different positions of the sub-node with different prefixes. The optimization design objective of the index model construction algorithm described in the embodiment of the present application is to use the largest possible prefix length span to reduce the height of the prefix tree, so as to make the constructed index more flat, and at the same time, make different prefixes not conflict with each other and fall into different positions of the node.

[0090] The method for determining the largest possible prefix length is as described in steps S301-S304.

[0091] In some embodiments, in the method for determining the target prefix length of the first target node described in steps S301-S304, the at least one to-be-verified prefix length can be determined according to the character length of the initial data, and the to-be-verified prefix lengths are verified in descending order. For example, if the character length of the shortest data in the initial data is 5, the largest to-be-verified prefix length is determined to be 5, and then the space amplification rate corresponding to the determined to-be-verified prefix length 5 is calculated. If the space amplification rate corresponding to the to-be-verified prefix length 5 is less than or equal to the preset space amplification rate threshold, the to-be-verified prefix length 5 is determined as the target prefix length. If the space amplification rate corresponding to the to-be-verified prefix length 5 is greater than the preset space amplification rate threshold, the space amplification rate corresponding to the to-be-verified prefix length 4 is verified whether it is less than or equal to the preset space amplification rate threshold, until the largest possible to-be-verified prefix length is screened out as the target prefix length.

[0092] Here, the space amplification rate determines the storage space of the first target node, or the number of slots of the first node. If the space amplification rate corresponding to the prefix length is greater than the preset space amplification rate threshold, it means that the number of slots required for storing the data or node prefix in the initial data is not enough, so the prefix length needs to be reduced to reduce the number of sub-data sets or data segments divided from the initial data, so as to ensure that different node prefixes do not conflict with each other and fall into different positions of the node.

[0093] After the target prefix length of the first target node is determined, the target global slope satisfying the preset space amplification rate threshold is calculated according to the target prefix length and the initial data.

[0094] Specifically, the target global slope satisfying the preset space amplification rate threshold is calculated according to the target prefix length and the initial data, including:

[0095] According to the target prefix length and the initial data, the to-be-stored data directly stored in the first target node in the initial data and the node prefix of the first target node are determined.

[0096] According to the to-be-stored data and the node prefix, the target global slope satisfying the preset space amplification rate threshold is calculated.

[0097] Here, the node prefix is also taken as data, so that the target global slope satisfying the preset space amplification rate threshold is calculated according to the to-be-stored data and the node prefix, for example, by determining the target global slope through the least square method.

[0098] In the embodiment of the application, another method for determining the index model of the first target node is further provided.

[0099] Here, the index model of the first target node needs to traverse the initial data twice, and the time complexity is O(N+log2 K), where N is the data amount of the initial data, and K represents any data element in the initial data.

[0100] The first traversal determines the size of the target prefix length span, and the second traversal determines the index model. The specific process is as follows: in the first traversal, span is initialized as 0, and then the minimum conflict-free slope A is calculated according to two adjacent data X k , X k+1 in the initial data k , that is, A k needs to satisfy A k (X k+1 -X k )=1, at this time, whether the space amplification rate is less than or equal to the preset space amplification rate threshold α is calculated under the condition that the slope is A k , if yes, the remaining data is continuously traversed; if no, it is indicated that the two data X k , X k+1 are too close, and the common prefix can be taken to make them belong to the same sub-data set, and span is increased until X k >>span=X k+1 >>span, and then the remaining data is continuously traversed, and after all the data is traversed, the value of the target prefix length span can be determined; in the second traversal, A k is calculated again according to span, and the global slope A satisfying the preset space amplification rate threshold α is obtained, that is, A=max(A,A k ), here, the global slope A needs to take the maximum value to ensure that all data will not conflict; the linear model is further determined according to the global slope A.

[0101] In the step S102, the initial data is divided into a plurality of sub-data sets according to the target prefix length of the first target node, and / or the data to be stored in the first target node is screened from the initial data; wherein the initial data in the sub-data set has the same node prefix.

[0102] Here, the initial data is divided into a plurality of sub-data sets, and the data in the sub-data set is similar, which needs to be stored in the sub-node of the first target node, and the first target node only stores the common node prefix of the data in the sub-data set.

[0103] At the same time, some data in the initial data can be directly stored in the first target node.

[0104] Therefore, according to the target prefix length of the first target node, the initial data is divided into a plurality of sub-data sets, and / or the data to be stored in the first target node is screened from the initial data, which has three cases: the first case is that the initial data is divided into a plurality of sub-data sets according to the target prefix length of the first target node, for example, the root node usually does not directly store data, but only stores the common prefix of the sub-data set; the second case is that the initial data is divided into a plurality of sub-data sets according to the target prefix length of the first target node, and the data outside the sub-data set is regarded as the data to be stored, and some relatively late sub-nodes usually store data and the common prefix of the sub-data set at the same time; the third case is that in the lowest sub-node, only data is stored, and there is no longer sub-node, and there is no longer common prefix.

[0105] The initial data is divided into a plurality of sub-data sets, and the initial data in the sub-data set has the same node prefix, that is, has the same longest common prefix.

[0106] For example, the initial data includes absent, abnormal, append, apposition, bicycle, bigamy, which can be divided into three sub-data sets, namely absent, abnormal; append, apposition; bicycle, bigamy.

[0107] The node prefix of absent and abnormal is ab; the node prefix of append and apposition is ap; and the node prefix of bicycle and bigamy is bi.

[0108] If the initial data includes absent, abnormal, append, apposition, bicycle, bigamy, and name, in addition to the three sub-data sets of absent, abnormal, append, apposition, bicycle, and bigamy, the data name to be stored in the first target node is also included.

[0109] It should be noted that, according to the target prefix length of the first target node, the initial data is divided into a plurality of sub-data sets, and / or the to-be-stored data stored in the first target node is screened from the initial data, including:

[0110] According to the target prefix length of the first target node, the node prefix is determined;

[0111] According to the determined node prefix, the initial data is divided into a plurality of sub-data sets, and / or the to-be-stored data stored in the first target node is screened from the initial data.

[0112] For example, if it is determined that the target prefix length of the first target node is 2, the node prefixes of the initial data are respectively ab, ap, and bi, and then three sub-data sets of absent, abnormal, append, apposition, bicycle, and bigam are divided.

[0113] In the step S103, the index model of the first target node is determined according to the node prefix and / or the to-be-stored data, and the pointer of the sub-node and the storage location corresponding to the node prefix of each sub-data set are calculated through the index model, and / or the storage location of each to-be-stored data is calculated.

[0114] Here, the index model of the first target node is determined according to the node prefix and / or the to-be-stored data, and the slope of the linear model can be calculated according to the node prefix and / or the to-be-stored data, that is, the index model can be obtained.

[0115] The pointer of the sub-node and the storage location corresponding to the node prefix of each sub-data set are calculated through the index model, and / or the storage location of each to-be-stored data is calculated; that is, for the sub-data set, the slot position of the node prefix in the first target node is determined; for the to-be-stored data, the slot position of the to-be-stored data in the first target node is determined.

[0116] In the step S104, each node prefix and the pointer to the child node, and / or each data to be stored is stored in the corresponding storage location, and the index structure of the first target node is constructed. That is, each node prefix and the pointer to the child node is stored in the slot in the first target node, and each data to be stored is stored in the slot in the first target node, and the first target node is constructed.

[0117] In the step S105, the child node of the constructed first target node is taken as a new first target node, and the corresponding child data set of the child node is taken as new initial data, and the index structure of the new first target node is constructed, until the construction completion condition is met, and the entire index is constructed.

[0118] That is, after the first target node is constructed, the data in the child data set in the first target node needs to be stored in the child node of the first target node. One child data set corresponds to one word node.

[0119] The child node is constructed through the steps S101-S104, and after the child node is constructed, the child node is taken as a parent node, until the construction completion condition is met, and the entire index is constructed.

[0120] The construction completion condition is that all data for constructing the index is stored in the node. Alternatively, the initial data of the first target node does not exist the child data set.

[0121] In some embodiments, when the child node of the first target node and the node prefix of the first target node are different by less than a preset length threshold when the child node is constructed, the child node of the first target node adopts an ART node.

[0122] Specifically, if the difference between the child node and the parent node in SPAN (that is, the actual number of bits used by the node to distinguish the key) is less than or equal to 8, the node can be replaced by an ART node. After the actual replacement, the query strategy for querying the node is different, and the rest of the operations are the same, that is, the index structure is also compatible with the index structure adjustment design of other prefix trees.

[0123] To ensure that the model can accurately direct the input value to the corresponding position without error (accurate mapping), a series of adjustment operations need to be performed when inserting new data. The insertion adjustment strategy not only ensures the property of accurate mapping, but also keeps the structure of the entire tree as flat as possible, reduces the tree height, and optimizes the structure of the tree to improve the overall query performance. However, the adjustment operation to optimize the tree structure usually needs to modify a large number of nodes, which has high latency; on the other hand, the adjustment operation is performed at the right time, which requires recording the corresponding meta information during the insertion process to control the frequency of insertion adjustment and reduce the performance loss caused by frequent adjustment. However, the update overhead caused by modifying multiple meta information (such as the number of elements inserted into the entire subtree) during one insertion will cause contention in multi-threaded concurrency, thereby affecting scalability. In summary, a low-latency, especially low-tail-latency, robust, concurrent-friendly, and efficient insertion adjustment strategy is crucial for an updatable learning index.

[0124] Based on this, after the entire index is constructed, the index construction method further comprises:

[0125] When inserting new data into the constructed index, it is determined whether the target slot of the second target node to which the new data is mapped satisfies a preset adjustment condition;

[0126] If yes, the index structure of the second target node is adjusted according to the data of the second target node including the new data through a pre-configured adjustment strategy.

[0127] Here, the preset adjustment condition is at least one of the following: the full load rate of the second target node reaches a preset full load rate threshold; the target slot of the second target node to which the new data is mapped is a data slot; the target slot of the second target node to which the new data is mapped is a pointer slot, and the longest common prefix of the new data and the child node pointed to by the pointer slot does not match.

[0128] According to the data of the second target node including the new data, the index structure of the second target node is adjusted, comprising:

[0129] Adjusting the space amplification rate of the second target node to expand the storage space of the second target node;

[0130] Alternatively, according to the data of the second target node including the new data, the index model of the second target node is re-determined;

[0131] Alternatively, the key-value pair of the data slot, the node prefix and the pointer key-value pair of the pointer slot of the second target node are taken as the initial data of the second target node;

[0132] According to the initial data of the second target node and a preset space amplification threshold, a target prefix length and an index model of the second target node are re-determined to re-construct an index structure of the second target node.

[0133] Here, the first target node is any node in the index construction, not specifically the root node. The second target node is a node in the index update, which can also be any node, generally a child node. The first target node and the second target node are only distinguished in naming, and do not represent a parent-child relationship between them.

[0134] Please refer to Figure 5 , Figure 5 A method flowchart for adjusting the index structure of the second target node is shown.

[0135] In the embodiment of the application, the index model can also be referred to as a prefix tree, so the position of the data in the index model can also be referred to as positioning in the tree.

[0136] When inserting new data, the target slot of the new data needs to be determined by the index model in each level of node, and three situations will be encountered:

[0137] The first situation is that the index model is mapped to an empty slot, in which case the insertion can be completed by directly occupying the empty slot. At this time, if a preset adjustment condition is met, a vertical expansion adjustment strategy will be triggered.

[0138] The second situation is that the index model is mapped to a data slot, in which case the preset adjustment condition is met, and a new child node needs to be built to accommodate the data in the slot and the newly inserted data, and then the new child node is placed in the original slot. If the length of the longest common prefix of the new child node is smaller than the SPAN of its parent node and smaller than a threshold T, the adjustment strategy of the parent node will be triggered.

[0139] The third situation is that the index model is mapped to a pointer slot pointing to a child node, but the new inserted value does not match the longest common prefix of the child node, and the preset adjustment condition is met. In this case, the new longest common prefix between the new inserted value and the longest common prefix recorded by the child node needs to be calculated first, and then the length of the new longest common prefix is compared with the SPAN of its parent node. If it is smaller than the SPAN of its parent node and smaller than a threshold T, the adjustment strategy of the parent node will be triggered. Otherwise, the insertion value can still be inserted into the child node (the insertion process is executed recursively), and the longest common prefix recorded by the child node is updated.

[0140] Here, the core idea of the adjustment strategy is that the longest common prefix of the child node can be regarded as a new data key, and the pointer to the child node can be regarded as a new data value, so that the current node can be adjusted with as few modifications to the child node as possible. When the newly inserted data conflicts with the already stored data, the conflict adjustment strategy is triggered, which consists of three steps to gradually try to adjust the model as little as possible:

[0141] The first step is to adjust the space magnification of the second target node to expand the storage space of the second target node, that is, to try to accommodate the newly inserted conflict value by adjusting the space size. This situation only occurs when data is inserted outside the two ends. By adjusting the space size to try to accommodate the conflict value, if the newly inserted conflict value can be accommodated within the given space magnification, the adjustment is complete; if not, the second step is tried.

[0142] The second step is to try to accommodate the newly inserted conflict value by adjusting the linear model. The specific calculation process and construction algorithm are similar, and the new inserted conflict value and the longest common prefix of the original slot or the key of the original slot data are needed to calculate a new slope, and then to see if all nodes after mapping can be accommodated within the given magnification based on the linear model given by the new slope. If so, the adjustment is complete, otherwise the third step is tried.

[0143] The third step is to perform local reconstruction of the SPAN of the second target node that needs to be adjusted; this step scans all data slot key-value pairs of the original node; and all pointer slots are regarded as key-value pairs with the longest common prefix as the key and the pointer to the child node as the value. At the same time, if the conflict slot of the newly inserted conflict data is a child node, the slot of the child node is also scanned in the above form as a key-value pair. The key-value pairs after scanning in order are already ordered by key and do not need further sorting. Then a new sub-tree is constructed using the index construction algorithm according to these new key-value pairs, and the new sub-tree is inserted into the original node position.

[0144] In addition, in addition to the conflict adjustment strategy due to the conflict of the newly inserted value described above, a vertical adjustment strategy of vertical expansion is also performed according to the fullness rate of the node; that is, when the empty slot of a node is mostly occupied and the total slot number of the node is close to the maximum size allowed by the SPAN of the node (that is, 2SPAN), the node is vertically expanded to increase the SPAN, so as to achieve vertical merging, reduce the tree height, and ensure the flattening effect of the tree structure. The process of vertical expansion is similar to the third step of the conflict adjustment, which needs to scan all the slots of the node and all the slots of the child nodes to form key-value pairs, and then reconstruct according to the index construction algorithm. This vertical expansion adjustment strategy is generally triggered when the insertion condition is checked in the empty slot to reduce the tail delay.

[0145] Compared with the traditional index structure, the index constructed in the embodiment of the application has the ability to learn the data distribution of the index, thereby having comprehensive advantages over the traditional index in the basic query performance, and also having corresponding competitiveness compared with other learning indexes; compared with other learning indexes, the embodiment of the application has the overall robustness of the traditional prefix tree index (such as ART), can have better performance in different data sets and write-intensive workloads, and can keep stable performance when the data distribution changes, and also has better tail delay performance.

[0146] The index construction algorithm proposed in the embodiment of the application not only has low time complexity, but also can guarantee the flattening structure of the whole tree, and has the property of meeting the insertion adjustment strategy; in order to solve the problem that the continuously inserted learning index can guarantee high performance, the embodiment of the application also proposes a new insertion adjustment strategy, which has the properties of low tail delay, high robustness, concurrency friendliness, and maintaining high-efficiency index structure.

[0147] In addition, the present application can also efficiently support variable-length data types and character data.

[0148] Please refer to Figure 6 In some embodiments, a data reading method is also provided, which is applied to the index constructed by the index construction method described in the embodiment of the application; the reading method includes the following steps S601-S604:

[0149] S601, starting from the root node of the index, judging whether the node prefix of the current node matches the data to be read;

[0150] S602, if matched, calculating the target slot of the data to be read through the index model in the current node;

[0151] S603, if the target slot is a data slot, judging whether the data in the data slot matches the data to be read, and returning the data in the data slot if the data matches the data to be read;

[0152] S604, if the target slot is a pointer slot, querying the data to be read in a child node pointed to by the pointer slot until a read result is returned.

[0153] Specifically, please refer to Figure 7 After starting to read the data, the search function is called to query the value corresponding to the data to be read key. Starting from the root node, it is compared whether the prefix corresponding to the current node is equal to the prefix of the key. If not, NULL is directly returned. If yes, the position of the data to be read in the node (i.e. the target slot) is predicted according to the key and the model in the node, and the position is checked. There are three possibilities in total: first, there is no data in the position, and NULL is returned; second, there is data in the position, and the key of the data is compared with the key to be queried. If the keys are equal, the corresponding value is returned. If the keys are not equal, NULL is returned; third, the position stores a pointer to a child node, and the child node is recursively queried, and the above process is repeated until the read result is returned.

[0154] Here, the returned read result includes NULL or the value corresponding to the data to be read.

[0155] Since the index constructed by the index construction method described in the embodiments of the present application maintains the storage order of the data, range query can be performed.

[0156] The data reading method provided in the embodiments of the present application further includes:

[0157] For the data range to be read, the boundary data of the data range is determined. The boundary data is the largest data and / or the smallest data in the data range.

[0158] The storage position of the boundary data in the index is determined.

[0159] According to the storage order of the data in the index and the storage position of the boundary data, a read result matching the data range to be read is returned.

[0160] Specifically, taking all KVs with a query key not less than lower_key as an example, since the index constructed by the index construction method described in the embodiments of the present application maintains the order of stored data, the position of the object corresponding to the lower_key can be determined first when querying, and this process is similar to the reading process. After predicting the position of the value in the node according to the lower_key, the position is checked. There are three possibilities in total: first, there is no data in the position, and the process is repeated by continuing to find in the rear of the position until the first key not less than the lower_key is found; second, there is data in the position, and the key of the data is compared with the key of the object to be updated. If the key is less than or equal to the key of the object to be updated, the process is repeated by continuing to find in the rear of the position until the first key not less than the lower_key is found; third, the position stores a pointer to a child node, and the child node is recursively queried, and the above process is repeated until the first key not less than the lower_key is found. Then, starting from the position of the key, all subsequent positions are traversed and accessed. If the position has no data, it is skipped. If the position has data, the data is added to the result set. If the position stores a pointer to a child node, the same process is performed on the child node, and the process is repeated until all positions are traversed, and the result of the range query is returned.

[0161] Here, it should be noted that when inserting new data, the insertion process is similar to the data reading process.

[0162] When the index constructed by the index construction method described in the embodiments of the present application deletes data, the deletion process is as follows: similar to the reading process, the position of the corresponding object (i.e. the target slot) must be found according to the key before the deletion operation can be performed; after predicting the position of the value in the node according to the key, the position is checked. There are three possibilities in total: first, there is no data in the position, and false is returned; second, there is data in the position, and the key of the data is compared with the key of the object to be deleted. If they are equal, the object is deleted, and it is determined whether the current Node (i.e. the current node) is empty. If it is empty, the current Node and related metadata are deleted, and true is returned; third, the position stores a pointer to a child node, and the child node is recursively queried, and the above process is repeated until the result is returned.

[0163] Here, although the deletion operation affects the actual longest common prefix of the node, the longest common prefix is not recalculated for performance considerations, as this has no effect on correctness. The original longest common prefix is only slightly shorter than the actual new longest common prefix, but it is still a common prefix.

[0164] When the index built by the index building method described in the embodiments of the present application is deleted, the update process is as follows: similar to the process of reading data, first, the position (i.e. the target slot) of the corresponding object is found according to the key, and then the update operation is performed; after the position of the value in the node is predicted according to the key, the position is checked, and there are three possibilities in total: first, there is no data in the position, and false is returned; second, there is data in the position, and the key of the data is compared with the key of the object to be updated, and if they are equal, the value is modified, and true is returned; third, the position stores a pointer to a child node, and the child node is recursively queried, and the above process is repeated until the result is returned.

[0165] Based on the same inventive concept, the embodiments of the present application also provide an index building device corresponding to the index building method. Since the principle of solving problems by the device in the embodiments of the present application is similar to the index building method described above, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described again.

[0166] Please refer to Figure 8 , Figure 8 The structure of the index building device described in the embodiments of the present application is shown, and specifically, the index building device comprises:

[0167] The determining module 801 is configured to input initial data into a first target node of a prefix tree, determine a target prefix length of the first target node according to the initial data and a preset space amplification threshold of the first target node, and determine a target prefix length of the first target node according to the initial data and the preset space amplification threshold of the first target node.

[0168] The dividing module 802 is configured to divide the initial data into a plurality of data sets according to the target prefix length of the first target node, and / or filter out to-be-stored data stored in the first target node from the initial data; wherein the initial data in the data sets have the same node prefix.

[0169] The calculating module 803 is configured to determine an index model of the first target node according to the node prefix and / or the to-be-stored data, and calculate the storage position of each to-be-stored data and the pointer to the child node corresponding to each node prefix of each data set according to the index model.

[0170] The first building module 804 is configured to store each node prefix and the pointer to the child node, and / or each to-be-stored data in the corresponding storage position, and build the index structure of the first target node.

[0171] The second construction module 805 is configured to take the constructed sub-node of the first target node as a new first target node, take the corresponding sub-data set of the sub-node as new initial data, construct an index structure of the new first target node, and construct the entire index until a construction completion condition is met.

[0172] Based on the same inventive concept, the electronic device corresponding to the index construction method is also provided in the embodiments of the present application. Since the principle of the electronic device in the embodiments of the present application for solving the problem is similar to the index construction method described above, the implementation of the electronic device can be referred to the implementation of the method, and the repeated parts will not be described herein.

[0173] Please refer to Figure 9 , Figure 9 The structure schematic diagram of the electronic device is shown, the electronic device 900 includes a processor 901, a memory 902 and a bus, the memory 902 stores machine readable instructions executable by the processor 901, when the electronic device 900 runs, the processor 901 and the memory 902 communicate through the bus, and the machine readable instructions are executed by the processor 901 to execute the steps of the index construction method.

[0174] Based on the same inventive concept, the computer readable storage medium corresponding to the index construction method is also provided in the embodiments of the present application. Since the principle of the computer readable storage medium in the embodiments of the present application for solving the problem is similar to the index construction method described above, the implementation of the computer readable storage medium can be referred to the implementation of the method, and the repeated parts will not be described herein.

[0175] The embodiments of the present application also provide a computer readable storage medium, and the computer readable storage medium stores a computer program, and the computer program is executed by a processor to execute the steps of the index construction method.

[0176] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiments, and will not be described herein. In the several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other ways. The device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and in actual implementation, there can be another division way, and for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed mutual ones can be indirect coupling or communication connection through some communication interfaces, devices or modules, and can be electrical, mechanical or other forms.

[0177] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, i.e., may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0178] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0179] The functions, if realized in the form of software functional units and sold or used as independent products, can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present application or the part of the present application that essentially contributes to the prior art or the part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a platform server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various storage media that can store program codes, such as U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk.

[0180] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An index construction method, characterized by, The method comprises the following steps: inputting initial data into a first target node of a prefix tree, and determining a target prefix length of the first target node according to the initial data and a preset space amplification rate threshold of the first target node; dividing the initial data into a plurality of data sets according to the target prefix length of the first target node, and / or screening out to-be-stored data stored in the first target node from the initial data; wherein the initial data in the data sets have the same node prefix; determining an index model of the first target node according to the node prefix and / or the to-be-stored data, and calculating a pointer to a child node and a storage location corresponding to each node prefix of each data set by using the index model, and / or calculating a storage location of each to-be-stored data; storing each node prefix and the pointer to the child node, and / or each to-be-stored data in a corresponding storage location, and constructing an index structure of the first target node; taking a child node of the constructed first target node as a new first target node, and taking a child data set corresponding to the child node as new initial data, and constructing an index structure of the new first target node until a construction completion condition is met, and constructing an entire index; after the entire index is constructed, when new data is inserted into the constructed index, determining whether a target slot of a second target node to which the new data is mapped satisfies a preset adjustment condition; if yes, adjusting an index structure of the second target node according to data of the second target node including the new data by using a preconfigured adjustment strategy; adjusting the index structure of the second target node according to the data of the second target node including the new data, comprising: adjusting a space amplification rate of the second target node to expand a storage space of the second target node; or, re-determining an index model of the second target node according to the data of the second target node including the new data; or, taking a key-value pair of a data slot of the second target node, a key-value pair composed of a node prefix and a pointer of a pointer slot as initial data of the second target node; re-determining a target prefix length and an index model of the second target node according to the initial data of the second target node and a preset space amplification rate threshold, and re-constructing an index structure of the second target node; the preset adjustment condition is at least one of the following: a full load rate of the second target node reaches a preset full load rate threshold; the target slot of the second target node to which the new data is mapped is a data slot; the target slot of the second target node to which the new data is mapped is a pointer slot, and a longest common prefix of the new data and a child node pointed to by the pointer slot does not match.

2. The index construction method of claim 1, wherein, determining the target prefix length of the first target node according to the initial data and the preset space amplification rate threshold of the first target node, comprising: determining at least one to-be-verified prefix length according to a data distribution feature of the initial data; calculating a space amplification rate corresponding to each to-be-verified prefix length; screening out the to-be-verified prefix length corresponding to the space amplification rate not greater than the preset space amplification rate threshold of the first target node. Determine the largest to-be-verified prefix length screened out as a target prefix length.

3. The index building method of claim 1, wherein, Determine an index model of the first target node according to the node prefix and / or the to-be-stored data, including: According to the node prefix and / or the to-be-stored data, calculate a target global slope that satisfies a preset spatial amplification rate threshold; According to the calculated target global slope, determine a linear model as the index model.

4. The index building method of claim 1, wherein, When the difference between the child node of the first target node and the node prefix of the first target node is less than a preset length threshold, the child node of the first target node adopts an ART node.

5. A data reading method characterized by, The reading method is applied to the index constructed by the index construction method in any one of claims 1-4, and the reading method includes: Starting from the root node of the index, judge whether the node prefix of the current node matches the to-be-read data; If matched, calculate the target slot of the to-be-read data through the index model in the current node; If the target slot is a data slot, judge whether the data in the data slot matches the to-be-read data, and if matched, return the data in the data slot; If the target slot is a pointer slot, query the to-be-read data in the child node pointed to by the pointer slot until the reading result is returned.

6. The data reading method according to claim 5, wherein The reading method further includes: For a data range to be read, determine the boundary data of the data range; wherein the boundary data is the largest data and / or the smallest data in the data range; Determine the storage location of the boundary data in the index; And according to the storage order of the data in the index and the storage location of the boundary data, return the reading result matching the data range to be read.

7. An index construction apparatus characterized by comprising: The construction device includes: A determination module is configured to input initial data to a first target node of a prefix tree, and determine a target prefix length of the first target node according to the initial data and a preset spatial amplification rate threshold of the first target node; A division module is configured to divide the initial data into a plurality of data sets according to the target prefix length of the first target node, and / or screen out to-be-stored data stored in the first target node from the initial data; wherein the initial data in the data sets has the same node prefix; A calculation module is configured to determine an index model of the first target node according to the node prefix and / or the to-be-stored data, and calculate a pointer of a child node pointed to by each node prefix and a storage location through the index model, and / or calculate a storage location of each to-be-stored data; A first construction module is configured to store each node prefix and the pointer of the child node, and / or each to-be-stored data in the corresponding storage location, and construct an index structure of the first target node; A second construction module is configured to take a child node of the constructed first target node as a new first target node, take a child data set corresponding to the child node as new initial data, construct an index structure of the new first target node, and construct an entire index until a construction completion condition is met; after the entire index is constructed, when new data is inserted in the constructed index, judge whether a target slot of a second target node to which the new data is mapped satisfies a preset adjustment condition. If yes, adjusting the index structure of the second target node according to the data of the second target node including the new data by a pre-configured adjustment strategy; Adjusting the index structure of the second target node according to the data of the second target node including the new data, comprising: Adjusting the spatial magnification of the second target node to expand the storage space of the second target node; Or, re-determining the index model of the second target node according to the data of the second target node including the new data; Or, taking the key-value pair of the data slot of the second target node, the node prefix of the pointer slot, and the key-value pair of the pointer as the initial data of the second target node; According to the initial data of the second target node and the preset spatial magnification threshold, re-determining the target prefix length and the index model of the second target node to reconstruct the index structure of the second target node; The preset adjustment condition is at least one of the following: the full load rate of the second target node reaches a preset full load threshold; the target slot of the second target node mapped by the new data is a data slot; the target slot of the second target node mapped by the new data is a pointer slot, and the new data does not match the longest common prefix of the child node pointed to by the pointer slot.

Images